Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scraper Scrapy Open Source Projects

Open source projects categorized as Scraper Scrapy
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 3 months agopypi Scrapy} Downloads
Portia ⭐ 8,982
Visual scraping for Scrapy
dependent packages 2total releases 26most recent commit 8 months agopypi slybot} Downloads
Awesome Crawler ⭐ 5,859
A collection of awesome web crawler,spider in different languages
most recent commit 5 months ago
Scrapely ⭐ 1,668
A pure-python HTML screen-scraping library
dependent packages 2total releases 13most recent commit 4 years agopypi scrapely} Downloads
Scrapy Cluster ⭐ 1,137
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
dependent packages 2total releases 15most recent commit 6 months agopypi scutils} Downloads
Django Dynamic Scraper ⭐ 1,069
Creating Scrapy scrapers via the Django admin interface
total releases 65most recent commit 2 years ago
Querido Diario ⭐ 944
📰 Diários oficiais brasileiros acessíveis a todos | 📰 Brazilian government gazettes, accessible to everyone.
most recent commit 3 months ago
Kimuraframework ⭐ 874
Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites
dependent packages 2total releases 10most recent commit 2 years agorubygems kimurai} Downloads
Scrapyrt ⭐ 793
HTTP API for Scrapy spiders
total releases 7most recent commit 6 months ago
Easy Scraping Tutorial ⭐ 618
Simple but useful Python web scraping tutorial code.
most recent commit 3 years ago
Linkedin ⭐ 602
Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
most recent commit 3 months ago
Alltheplaces ⭐ 502
A set of spiders and scrapers to extract location information from places that post their location on the internet.
most recent commit 3 months ago
Phpscraper ⭐ 486
A universal web-util for PHP.
total releases 34most recent commit 5 months ago
Spidermon ⭐ 486
Scrapy Extension for monitoring spiders execution.
total releases 18most recent commit 5 months ago
Scrapple ⭐ 452
A framework for creating semi-automatic web content extractors
total releases 10most recent commit 5 years ago
Awesome Scrapy ⭐ 450
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
most recent commit a year ago
Fbcrawl ⭐ 415
A Facebook crawler
most recent commit 4 years ago
Advanced Web Scraping Tutorial ⭐ 370
The Zipru scraper developed in the Advanced Web Scraping Tutorial.
most recent commit 7 years ago
Files ⭐ 369
Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects
most recent commit 5 years ago
Scrapy Zyte Smartproxy ⭐ 348
Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy
total releases 16most recent commit 2 months ago
City Scrapers ⭐ 315
Scrape, standardize and share public meetings from local government websites
most recent commit 3 months ago
Web Scraping ⭐ 281
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
most recent commit 3 months ago
Post Tuto Deployment ⭐ 269
Build and deploy a machine learning app from scratch 🚀
most recent commit 4 years ago
The 10 Most Depended On Scraper Scrapy Open Source Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Node Scrapy ⭐ 114
Simple, lightweight and expressive web scraping with Node.js
dependent packages 11total releases 11latest release August 24, 2020most recent commit 4 years agonpm node-scrapy} Downloads
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Scrapy Wayback Machine ⭐ 70
A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
dependent packages 4total releases 3latest release February 15, 2021most recent commit 3 years agopypi scrapy-wayback-machine} Downloads
Scrapely ⭐ 1,668
A pure-python HTML screen-scraping library
dependent packages 2total releases 13latest release November 28, 2019most recent commit 4 years agopypi scrapely} Downloads
Scrapy Cluster ⭐ 1,137
This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
dependent packages 2total releases 15latest release December 23, 2020most recent commit 6 months agopypi scutils} Downloads
Portia ⭐ 8,982
Visual scraping for Scrapy
dependent packages 2total releases 26latest release May 25, 2015most recent commit 8 months agopypi slybot} Downloads
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1latest release December 25, 2017most recent commit 4 years ago
Scrapy Zyte Api ⭐ 30
Zyte API integration for Scrapy
dependent packages 1total releases 23latest release October 19, 2023most recent commit 3 months agopypi scrapy-zyte-api} Downloads
The 10 Latest Releases In Scraper Scrapy Open Source Projects
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Scrapy Zyte Api ⭐ 30
Zyte API integration for Scrapy
dependent packages 1total releases 23latest release October 19, 2023most recent commit 3 months agopypi scrapy-zyte-api} Downloads
Scrapyrt ⭐ 793
HTTP API for Scrapy spiders
total releases 7latest release September 20, 2023most recent commit 6 months ago
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Spidermon ⭐ 486
Scrapy Extension for monitoring spiders execution.
total releases 18latest release September 01, 2023most recent commit 5 months ago
Scrapingant Client Python ⭐ 26
ScrapingAnt API client for Python.
total releases 16latest release August 09, 2023most recent commit 9 months ago
Phpscraper ⭐ 486
A universal web-util for PHP.
total releases 34latest release June 18, 2023most recent commit 5 months ago
Scrapyteer ⭐ 9
Web crawling & scraping framework for Node.js on top of headless Chrome browser
total releases 8latest release June 07, 2023most recent commit 10 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.