Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Scrapy | 49,918 | 4,185 | 445 | 4 months ago | 96 | September 18, 2023 | 692 | bsd-3-clause | Python | |
Scrapy, a fast high-level web crawling & scraping framework for Python. | ||||||||||
Crawlab | 10,521 | 4 months ago | 1 | March 03, 2019 | 58 | bsd-3-clause | Go | |||
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架 | ||||||||||
Awesome Crawler | 5,859 | 5 months ago | 27 | mit | ||||||
A collection of awesome web crawler,spider in different languages | ||||||||||
Webscraping From 0 To Hero | 1,305 | 10 months ago | ||||||||
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python | ||||||||||
Django Dynamic Scraper | 1,069 | 26 | 2 years ago | 65 | June 25, 2021 | 38 | bsd-3-clause | Python | ||
Creating Scrapy scrapers via the Django admin interface | ||||||||||
Faster Than Requests | 1,061 | 4 | 4 months ago | 18 | February 24, 2021 | mit | Nim | |||
Faster requests on Python 3 | ||||||||||
Scrapy Fake Useragent | 654 | 50 | 5 | 8 months ago | 12 | August 15, 2020 | 5 | mit | Python | |
Random User-Agent middleware based on fake-useragent | ||||||||||
Phpscraper | 486 | 6 months ago | 34 | June 18, 2023 | 20 | gpl-3.0 | PHP | |||
A universal web-util for PHP. | ||||||||||
Scrapple | 452 | 1 | 5 years ago | 10 | September 24, 2016 | 4 | mit | Python | ||
A framework for creating semi-automatic web content extractors | ||||||||||
City Scrapers | 315 | 4 months ago | 54 | mit | Python | |||||
Scrape, standardize and share public meetings from local government websites |