Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Golang Web Crawler Open Source Projects

Open source projects categorized as Golang Web Crawler
Crawlab ⭐ 10,521
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
total releases 1most recent commit 3 months ago
Katana ⭐ 7,995
A next-generation crawling and spidering framework.
dependent packages 1total releases 8most recent commit 3 months ago
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406most recent commit 3 months ago
Hakrawler ⭐ 4,120
Simple, fast web crawler designed for easy, quick discovery of endpoints and assets within a web application
total releases 11most recent commit 3 months ago
Gospider ⭐ 2,190
Gospider - Fast web spider written in Go
dependent packages 1total releases 17most recent commit 5 months ago
Soup ⭐ 2,074
Web Scraper in Go, similar to BeautifulSoup
dependent packages 57total releases 10most recent commit 6 months ago
Till ⭐ 770
DataHen Till is a companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable, with minimal code changes on your scraper. Integrates with any scraper in 5 minutes.
total releases 37most recent commit 2 years ago
Fetchbot ⭐ 758
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
dependent packages 3total releases 7most recent commit 3 years ago
Dcrawl ⭐ 411
Simple, but smart, multi-threaded web crawler for randomly gathering huge lists of unique domain names.
most recent commit 5 years ago
Google Maps Scraper ⭐ 330
scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
total releases 12most recent commit 4 months ago
Gopa ⭐ 281
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
total releases 6most recent commit 3 years ago
Ant ⭐ 271
A web crawler for Go
total releases 8most recent commit a year ago
Crawley ⭐ 208
The unix-way web crawler
total releases 60most recent commit 4 months ago
Awesomewebscraping ⭐ 189
List of libraries, tools and APIs for web scraping and data processing.
most recent commit 7 months ago
Digger ⭐ 180
Digger is a powerful and flexible web crawler implemented by pure golang
most recent commit 2 years ago
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1most recent commit 4 years ago
Goscrape ⭐ 172
Web scraper that can create an offline readable version of a website
total releases 2most recent commit 3 months ago
Gotor ⭐ 150
This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.
total releases 3most recent commit 5 months ago
Clock ⭐ 147
可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
most recent commit a year ago
Sqrape ⭐ 145
Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)
total releases 1most recent commit 7 years ago
Anirip ⭐ 144
🎬 A Crunchyroll show/season ripper
most recent commit 2 years ago
Amazon Scraper ⭐ 140
Free Trial Amazon Scraper API for extracting search, product, offer listing, reviews, question and answers, best sellers and sellers data.
total releases 1most recent commit 5 months ago
Cascadia ⭐ 128
Go cascadia package command line CSS selector
total releases 9most recent commit 10 months ago
The 7 Most Depended On Golang Web Crawler Open Source Projects
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406latest release November 06, 2023most recent commit 3 months ago
Soup ⭐ 2,074
Web Scraper in Go, similar to BeautifulSoup
dependent packages 57total releases 10latest release January 16, 2022most recent commit 6 months ago
Fetchbot ⭐ 758
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
dependent packages 3total releases 7latest release May 20, 2021most recent commit 3 years ago
Katana ⭐ 7,995
A next-generation crawling and spidering framework.
dependent packages 1total releases 8latest release September 14, 2023most recent commit 3 months ago
Jsonextract ⭐ 6
Go package for finding and extracting any JavaScript object (not just JSON) from an io.Reader
dependent packages 1total releases 33latest release June 21, 2021most recent commit 2 years ago
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1latest release December 25, 2017most recent commit 4 years ago
Gospider ⭐ 2,190
Gospider - Fast web spider written in Go
dependent packages 1total releases 17latest release December 25, 2021most recent commit 5 months ago
The 10 Latest Releases In Golang Web Crawler Open Source Projects
Crawley ⭐ 208
The unix-way web crawler
total releases 60latest release December 07, 2023most recent commit 4 months ago
Goskyr ⭐ 27
A configurable command-line web scraper written in go with auto configuration capability
total releases 66latest release November 28, 2023most recent commit 3 months ago
Rod ⭐ 4,505
A Devtools driver for web automation and scraping
dependent packages 140total releases 406latest release November 06, 2023most recent commit 3 months ago
Get Sauce ⭐ 97
A command line program to download Hentai videos and images from multiple websites
total releases 81latest release October 29, 2023most recent commit 4 months ago
Goscrapy ⭐ 34
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
total releases 26latest release September 20, 2023most recent commit 3 months ago
Katana ⭐ 7,995
A next-generation crawling and spidering framework.
dependent packages 1total releases 8latest release September 14, 2023most recent commit 3 months ago
Cascadia ⭐ 128
Go cascadia package command line CSS selector
total releases 9latest release June 30, 2023most recent commit 10 months ago
Goscrape ⭐ 172
Web scraper that can create an offline readable version of a website
total releases 2latest release February 11, 2023most recent commit 3 months ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.