Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Kimuraframework | 874 | 4 | 2 | 2 years ago | 10 | January 30, 2019 | 34 | mit | Ruby | |
Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites | ||||||||||
Xxl Crawler | 650 | 2 | 1 | a year ago | 6 | October 15, 2022 | 20 | apache-2.0 | Java | |
A distributed web crawler framework.(分布式爬虫框架XXL-CRAWLER) | ||||||||||
Lagoujob | 250 | 5 years ago | apache-2.0 | Python | ||||||
Job data mining repo for lagou.com | ||||||||||
Awesome Web Scraper | 214 | 9 months ago | 41 | mit | ||||||
A collection of awesome web scaper, crawler. | ||||||||||
Pkulaw_spider | 109 | 6 years ago | 7 | Python | ||||||
爬取北大法宝网http://www.pkulaw.cn/Case/ | ||||||||||
Tspider | 71 | 7 years ago | 1 | Python | ||||||
Yet Another Web Spider | ||||||||||
Taobao_spider | 52 | 7 years ago | PHP | |||||||
这是一个淘宝爬虫,填写任意一个淘宝链接可抓取此淘宝店铺的所有信息(店铺名字,店铺信用,店铺ID,所有的商品 价格 优惠 销量 图片 等等) | ||||||||||
Comicspider | 42 | 7 years ago | 3 | gpl-3.0 | Python | |||||
动漫之家漫画站电脑版原图爬虫 | ||||||||||
Salmonjs | 33 | 2 | 4 years ago | 5 | May 26, 2014 | 21 | JavaScript | |||
[WIP] Web Crawler in Node.js to spider dynamically whole websites. | ||||||||||
Node Tarantula | 23 | 3 | 5 | 9 years ago | 9 | April 18, 2014 | 7 | mit | JavaScript | |
web crawler/spider for nodejs |