Go Crawler

A web crawling framework implemented in Golang, it is simple to write and delivers powerful performance. It comes with a wide range of practical middleware and supports various parsing and storage methods. Additionally, it supports distributed deployment. 基于golang实现的爬虫框架,编写简单,性能强劲。内置了丰富的实用中间件,支持多种解析、保存方式,
Alternatives To Go Crawler
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Scrapy49,9184,1854456 months ago96September 18, 2023692bsd-3-clausePython
Scrapy, a fast high-level web crawling & scraping framework for Python.
6 days ago6otherJavaScript
A visual no-code/code-free web crawler/spider易采集:一个可视化浏览器自动化测试/数据采集/爬虫软件,可以无代码图形化的设计和执行爬虫任务。别名:ServiceWrapper面向Web应用的智能化服务封装系统。
Lux25,976811 days ago40November 06, 2023477mitGo
👾 Fast and simple video download library and CLI tool written in Go
Colly22,51681328a month ago22March 08, 2022181apache-2.0Go
Elegant Scraper and Crawler Framework for Golang
Newspaper13,147222979 months ago18September 28, 2018498mitPython
News, full-text, and article metadata extraction in Python 3. Advanced docs:
Crawlee12,8714221 days ago747December 10, 202396apache-2.0TypeScript
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Webmagic11,080734226 months ago25September 10, 2023353apache-2.0Java
A scalable web crawler framework for Java.
a year ago85PHP
AV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Awesome Web Scraping6,060
8 months ago1otherMakefile
List of libraries, tools and APIs for web scraping and data processing.
Awesome Crawler5,859
7 months ago27mit
A collection of awesome web crawler,spider in different languages
Alternatives To Go Crawler
Select To Compare

Alternative Project Comparisons
Popular Crawler Projects
Popular Scraper Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.