Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Web Crawler Scrapy Open Source Projects

Open source projects categorized as Web Crawler Scrapy
Categories > Data Processing > Web Crawler
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 3 months agopypi Scrapy} Downloads
Crawlab ⭐ 10,521
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
total releases 1most recent commit 4 months ago
Awesome Crawler ⭐ 5,859
A collection of awesome web crawler,spider in different languages
most recent commit 5 months ago
Webscraping From 0 To Hero ⭐ 1,305
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
most recent commit 10 months ago
Django Dynamic Scraper ⭐ 1,069
Creating Scrapy scrapers via the Django admin interface
total releases 65most recent commit 2 years ago
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18most recent commit 3 months agopypi faster-than-requests} Downloads
Scrapy Fake Useragent ⭐ 654
Random User-Agent middleware based on fake-useragent
dependent packages 5total releases 12most recent commit 8 months agopypi scrapy-fake-useragent} Downloads
Phpscraper ⭐ 486
A universal web-util for PHP.
total releases 34most recent commit 5 months ago
Scrapple ⭐ 452
A framework for creating semi-automatic web content extractors
total releases 10most recent commit 5 years ago
City Scrapers ⭐ 315
Scrape, standardize and share public meetings from local government websites
most recent commit 3 months ago
Web Scraping ⭐ 281
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
most recent commit 3 months ago
Football Data Collection ⭐ 246
Web Scraper used to create Kaggle European Soccer database
most recent commit 5 years ago
Netflix Clone ⭐ 245
Netflix like full-stack application with SPA client and backend implemented in service oriented architecture
most recent commit a year ago
Wayback Machine Scraper ⭐ 219
A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
total releases 8most recent commit 3 years ago
Awesome Web Scraper ⭐ 214
A collection of awesome web scaper, crawler.
most recent commit 8 months ago
Crawlab Lite ⭐ 184
Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
most recent commit a year ago
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1most recent commit 4 years ago
Juno_crawler ⭐ 147
Scrapy crawler to collect data on the back catalog of songs listed for sale.
most recent commit 5 years ago
Estela ⭐ 142
estela, an elastic web scraping cluster 🕸
most recent commit 3 months ago
Scrapy Training ⭐ 141
Scrapy Training companion code
most recent commit 5 years ago
Raspagem De Dados Para Iniciantes ⭐ 115
Raspagem de dados para iniciante usando Scrapy e outras libs básicas
most recent commit a year ago
Seleniumcrawler ⭐ 105
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
most recent commit 5 years ago
Wswp ⭐ 95
Code for the second edition Web Scraping with Python book by Packt Publications
most recent commit 4 years ago
The 6 Most Depended On Web Crawler Scrapy Open Source Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Scrapy Fake Useragent ⭐ 654
Random User-Agent middleware based on fake-useragent
dependent packages 5total releases 12latest release August 15, 2020most recent commit 8 months agopypi scrapy-fake-useragent} Downloads
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18latest release February 24, 2021most recent commit 3 months agopypi faster-than-requests} Downloads
Scrapy Wayback Machine ⭐ 70
A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
dependent packages 4total releases 3latest release February 15, 2021most recent commit 3 years agopypi scrapy-wayback-machine} Downloads
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Antch ⭐ 177
Antch, a fast, powerful and extensible web crawling & scraping framework for Go
dependent packages 1total releases 1latest release December 25, 2017most recent commit 4 years ago
The 10 Latest Releases In Web Crawler Scrapy Open Source Projects
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Goscrapy ⭐ 34
GoScrapy: Harnessing Go's power for efficient web scraping, inspired by Python's Scrapy framework.
total releases 26latest release September 20, 2023most recent commit 3 months ago
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Scrapingant Client Python ⭐ 26
ScrapingAnt API client for Python.
total releases 16latest release August 09, 2023most recent commit 9 months ago
Phpscraper ⭐ 486
A universal web-util for PHP.
total releases 34latest release June 18, 2023most recent commit 5 months ago
Scrapyteer ⭐ 9
Web crawling & scraping framework for Node.js on top of headless Chrome browser
total releases 8latest release June 07, 2023most recent commit 10 months ago
Scalpel ⭐ 38
A fast and powerful web scraping library
total releases 4latest release June 02, 2022most recent commit 3 months ago
Django Dynamic Scraper ⭐ 1,069
Creating Scrapy scrapers via the Django admin interface
total releases 65latest release June 25, 2021most recent commit 2 years ago
City Scrapers Core ⭐ 6
Core functionality for City Scrapers projects
total releases 37latest release March 25, 2021most recent commit 2 years ago
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18latest release February 24, 2021most recent commit 3 months agopypi faster-than-requests} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.