Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Scrapy | 49,918 | 4,185 | 445 | 3 months ago | 96 | September 18, 2023 | 692 | bsd-3-clause | Python | |
Scrapy, a fast high-level web crawling & scraping framework for Python. | ||||||||||
Portia | 8,982 | 9 | 2 | 8 months ago | 26 | May 25, 2015 | 127 | bsd-3-clause | Python | |
Visual scraping for Scrapy | ||||||||||
Awesome Crawler | 5,859 | 5 months ago | 27 | mit | ||||||
A collection of awesome web crawler,spider in different languages | ||||||||||
Scrapely | 1,668 | 101 | 2 | 4 years ago | 13 | November 28, 2019 | 29 | HTML | ||
A pure-python HTML screen-scraping library | ||||||||||
Scrapy Cluster | 1,137 | 18 | 2 | 6 months ago | 15 | December 23, 2020 | 17 | mit | Python | |
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster. | ||||||||||
Django Dynamic Scraper | 1,069 | 26 | 2 years ago | 65 | June 25, 2021 | 38 | bsd-3-clause | Python | ||
Creating Scrapy scrapers via the Django admin interface | ||||||||||
Querido Diario | 944 | 3 months ago | 182 | mit | Python | |||||
📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian government gazettes, accessible to everyone. | ||||||||||
Kimuraframework | 874 | 4 | 2 | 2 years ago | 10 | January 30, 2019 | 34 | mit | Ruby | |
Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites | ||||||||||
Scrapyrt | 793 | 5 | 6 months ago | 7 | September 20, 2023 | 31 | bsd-3-clause | Python | ||
HTTP API for Scrapy spiders | ||||||||||
Easy Scraping Tutorial | 618 | 3 years ago | 7 | mit | Jupyter Notebook | |||||
Simple but useful Python web scraping tutorial code. |