Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for golang web crawler
golang
x
web-crawler
x
12 search results found
Crawlab
⭐
10,521
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
Katana
⭐
7,995
A next-generation crawling and spidering framework.
Rod
⭐
4,505
A Devtools driver for web automation and scraping
Hakrawler
⭐
4,120
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
Gospider
⭐
2,190
Gospider - Fast web spider written in Go
Soup
⭐
2,074
Web Scraper in Go, similar to BeautifulSoup
Till
⭐
770
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.
Fetchbot
⭐
758
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
Dcrawl
⭐
411
Simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names.
Google Maps Scraper
⭐
330
scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
Gopa
⭐
281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Ant
⭐
271
A web crawler for Go
Crawley
⭐
208
The unix-way web crawler
Awesomewebscraping
⭐
189
List of libraries, tools and APIs for web scraping and data processing.
Digger
⭐
180
Digger is a powerful and flexible web crawler implemented by pure golang
Antch
⭐
177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
Goscrape
⭐
172
Web scraper that can create an offline readable version of a website
Gotor
⭐
150
This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.
Clock
⭐
147
可视化任务调度系统,精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
Sqrape
⭐
145
Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)
Anirip
⭐
144
🎬 A Crunchyroll show/season ripper
Amazon Scraper
⭐
140
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
Cascadia
⭐
128
Go cascadia package command line CSS selector
Evine
⭐
117
Interactive CLI Web Crawler
Get Sauce
⭐
97
A command line program to download Hentai videos and images from multiple websites
Instago
⭐
87
Download/access photos, videos, stories, story highlights, postlives, following and followers of Instagram
Jsongenius
⭐
85
Get structured JSON data from any page.
Bathyscaphe
⭐
83
Fast, highly configurable, cloud native dark web crawler.
Gocrawler
⭐
60
A distributed web crawler implemented using Go, Postgres, RabbitMQ and Docker
Feedbridge
⭐
57
Plugin based RSS feed generator for sites that don't offer any. Serves RSS, Atom and JSON Feeds.
Owlcrawler
⭐
54
Crawl the web using nats.io and Go
Slurp
⭐
54
One repo to rule them all !!?!?!! 🤓 😎
Ganda
⭐
52
fast cmd-line app that quickly request millions of urls and save/echo the results
Cultured Downloader
⭐
44
A project to automate the process of downloading images and other attachment files from Pixiv, Pixiv Fanbox, and Fantia
Letterboxd Watchlist Picker
⭐
43
A simple website that gives you a random film off your Letterboxd watchlist (or any list).
Scrapemate
⭐
39
Golang Crawling and scraping framework
Goscrapy
⭐
34
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
Jsonscraper
⭐
33
JSON configurable concurrent scraper
Goanime
⭐
30
A cli tool to browse and play anime in pt-br
Goskyr
⭐
27
A configurable command-line web scraper written in go with auto configuration capability
Imdb Api
⭐
23
[🚧 WIP] Cross-platform microservice to scrape the IMDb website.
Burgoking
⭐
23
Burger King - Free Burger Code Generator
Yast
⭐
23
Yet Another Streaming Tool
Spinarago
⭐
22
A basic web crawler written in Go
Kraken
⭐
19
A parallelised web crawler in Go
Scrape
⭐
18
Depth controllable Web scraper and Sitemap Generator in Go
Proxi
⭐
14
Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.
Sentry
⭐
14
Parallelized web crawler written in Golang
Webcrawlertokopedia
⭐
13
It is a web crawler and scrapper for https://www.Tokopedia.com. The project scrape the product-ID, product URL and product videos present under the product images present at right bottom of the page.
Musicscraper
⭐
13
CLI tool for scraping information from musical websites (Rateyourmusic, Metal Archives), with nice album ASCII art
Grabber
⭐
12
Grabber is a concurrent declarative web scraper and downloader
Cnu Cse Discord Bot
⭐
11
Discord bot inform CNU-CSE Notices
Webhog
⭐
11
Downloads and stores a given URL (including js, css, and images) for offline use.
Td Spider
⭐
10
Via Text Density Simple Web Crawler With Go
Gsoup
⭐
10
A tiny web scraper written in Go with similar features to jsoup
Doubak
⭐
9
豆备 (Doubak) - 豆瓣 (Douban.com) 命令行版备份工具,备份完整页面镜像,Golang实现。
Tube Virality
⭐
8
Develop an API to retrieve statistics and information around Youtube trending videos. Perform descriptive statistics analysis, and build models able to project the likelihood of a trending video to become viral.
Golang Web Scraping
⭐
8
Learn how to scrape web content from HTML and see how web scraping differs to web crawling
Go Facebook Post Parser
⭐
7
web scrape facebook post and extract data
Peepingjim
⭐
7
A take on peepingTom in GO
Suck
⭐
6
I eat c̶h̶i̶l̶d̶r̶e̶n̶ websites.
Devread
⭐
6
DevRead - ứng dụng tổng hợp kiến thức cho developer
Jsonextract
⭐
6
Go package for finding and extracting any JavaScript object (not just JSON) from an io.Reader
Hackergo
⭐
6
HackerGo is a CLI tool that aids developers stay productive and up-to-date with the technology world
Goigstorylink
⭐
6
Get Links (URL) of Instagram Stories in Go
Blog Aggregator
⭐
6
A RSS feed aggregator in Go
Go Manga Downloader
⭐
5
Golang Web Scraper
⭐
5
A tutorial for building a web scraper in Golang
Related Searches
Golang Docker (7,769)
Golang Command Line (7,504)
Golang Http (4,290)
Golang Server (4,285)
Javascript Golang (3,372)
Golang Database (2,927)
Golang Json (2,652)
Golang Proxy (2,577)
Golang Grpc (2,432)
Python Golang (2,332)
1-12 of 12 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.