Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Spider Flow | 8,075 | a year ago | 20 | mit | Java | |||||
新一代爬虫平台,以图形化方式定义爬虫流程,不写代码即可完成爬虫。 | ||||||||||
Ecommercecrawlers | 3,724 | a year ago | 43 | mit | Python | |||||
实战🐍多种网站、电商数据爬虫🕷。包含🕸:淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼、阿里任务、博客园、微博、百度贴吧、豆瓣电影、包图网、全景网、豆瓣音乐、某省药监局、搜狐新闻、机器学习文本采集、fofa资产采集、汽车之家、国家统计局、百度关键词收录数、蜘蛛泛目录、今日头条、豆瓣影评、携程、小米应用商店、安居客、途家民宿❤️❤️❤️。微信爬虫展示项目: | ||||||||||
Appcrawler | 1,023 | 3 years ago | 3 | apache-2.0 | Scala | |||||
基于appium的app自动遍历工具 | ||||||||||
Vessel | 586 | 1 | 6 months ago | 3 | March 09, 2021 | 2 | mit | Ruby | ||
Fast high-level web crawling Ruby framework | ||||||||||
Fbcrawl | 415 | 4 years ago | 25 | apache-2.0 | Python | |||||
A Facebook crawler | ||||||||||
Dude | 397 | 5 months ago | 43 | August 12, 2023 | 24 | agpl-3.0 | Python | |||
dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators | ||||||||||
Crawlerforreader | 293 | 4 years ago | apache-2.0 | Java | ||||||
Android 本地网络小说爬虫,基于jsoup及xpath | ||||||||||
Scrape | 135 | 2 | 2 years ago | 112 | February 20, 2022 | 4 | mit | Python | ||
a command-line web scraping tool | ||||||||||
Graphquery | 104 | 3 | 4 | 5 years ago | 1 | March 17, 2019 | apache-2.0 | Go | ||
GraphQuery is a query language and execution engine tied to any backend service. | ||||||||||
Docs | 102 | 5 years ago | 3 | |||||||
《数据采集从入门到放弃》源码。内容简介:爬虫介绍、就业情况、爬虫工程师面试题 ;HTTP协议介绍; Requests使用 ;解析器Xpath介绍; MongoDB与MySQL; 多线程爬虫; Scrapy介绍 ;Scrapy-redis介绍; 使用docker部署; 使用nomad管理docker集群; 使用EFK查询docker日志 |