Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Galer | 189 | a year ago | 2 | November 05, 2021 | 5 | mit | Go | |||
A fast tool to fetch URLs from HTML attributes by crawl-in. | ||||||||||
Ignareo Isml Auto Voter | 186 | 7 months ago | 20 | mit | Python | |||||
Ignareo the Carillon, a web crawler/spider template of ultimate high concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns! (ISML=international saimoe; 2022 ISML is last ISML) | ||||||||||
Second Spider | 56 | 10 years ago | Python | |||||||
one more spider based on gevent requests pyquery | ||||||||||
Spider2 | 42 | 1 | 8 years ago | 6 | December 19, 2015 | 2 | JavaScript | |||
A 2nd generation spider to crawl any article site, automatic read title and article. | ||||||||||
Benchmark Http | 16 | a year ago | 24 | February 21, 2023 | mit | Ruby | ||||
Input Field Finder | 11 | 7 years ago | 4 | July 11, 2016 | Go | |||||
Spiders given URLs for input fields. | ||||||||||
Dazongdianping | 10 | 4 years ago | 1 | Python | ||||||
爬取大众点评中11205条厦门美食商铺信息,其中包含店名、人均消费、所属菜系、所属商圈、详细地址、口味评分、环境评分、服务评分信息。 | ||||||||||
Multi Selenium In Scrapy | 8 | 6 years ago | Python | |||||||
通过headless chrome实现selenium+scrapy的伪并发,提高动态网站爬取效率。 | ||||||||||
Python3 Concurrency Aqi | 6 | 6 years ago | 1 | Python | ||||||
并发爬取全国城市空气质量日报数据,数据来源: http://datacenter.mep.gov.cn |