Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Crawler Scrapy Open Source Projects

Open source projects categorized as Python Crawler Scrapy
Scalpel ⭐ 38
A fast and powerful web scraping library
total releases 4most recent commit 3 months ago
Facebookspider ⭐ 38
crawl facebook public post's information who likes, comments, shares
most recent commit 5 years ago
Searchenginescrapy ⭐ 38
Scrape data from Google.com, Bing.com, Baidu.com, Ask.com, Yahoo.com, Yandex.com
most recent commit 5 years ago
Ershoufang ⭐ 37
use scrapy crawl the second-hand house price
most recent commit 6 years ago
Proyecto Colibri ⭐ 37
Las votaciones del Congreso de los Diputados de España (via API)
most recent commit 5 years ago
Ip_proxy_pool ⭐ 37
Generating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.
most recent commit 6 years ago
Taobao_crawler ⭐ 37
A demo project based on Scrapy (with Selenium) crawling air conditioner sales data from Taobao.
most recent commit 8 years ago
Uefi Spider ⭐ 37
Crawl/download UEFI/BIOS updates from ISV/OEM distributors.
most recent commit 10 years ago
Scrapy Kafka Redis ⭐ 35
Distributed crawling/scraping, Kafka And Redis based components for Scrapy
total releases 5most recent commit 3 years ago
Python_crawler ⭐ 34
Python爬虫和Python数据分析小项目(Some Python crawlers and data analysis projects)
most recent commit 3 years ago
Scry ⭐ 34
Web scraping engines with Python and Scrapy
most recent commit 4 years ago
Learn Python Crawler ⭐ 34
Python爬虫学习路径图
most recent commit 7 years ago
Scrapy Flask ⭐ 34
Execute Scrapy spiders in a Flask web application
most recent commit 6 years ago
Scrapy Useragents ⭐ 33
A middleware to use random user agent in Scrapy crawler.
most recent commit 11 years ago
Scrapy Zyte Api ⭐ 30
Zyte API integration for Scrapy
dependent packages 1total releases 23most recent commit 3 months agopypi scrapy-zyte-api} Downloads
Crawlerflow ⭐ 30
Web Crawlers orchestration framework that lets you create datasets from multiple web sources using yaml configurations.
most recent commit 6 months ago
Tieba Crawler ⭐ 30
简单的一逼的，到处硬编码的，有可能泄露个人信息的百度贴吧爬虫，基于scrapy.
most recent commit 7 years ago
Isbullshit Crawler ⭐ 30
A crawling spider gathering all blogposts from http://isbullsh.it, using Scrapy
most recent commit 11 years ago
Awesome Scrapy ⭐ 29
一些基于Scrapy的数据采集爬虫，qq music, netease music, douyin, instagram, weibo, apple store, huawei store, xiaohongshu等。
most recent commit 2 years ago
Python Scrapilicious ⭐ 28
Unmaintained: A horridly implemented scrapy app that will scrape all (?) of Delicious' bookmarks.
most recent commit 13 years ago
Amazon Pricetracker ⭐ 27
Track the prices of different products on Amazon.in
most recent commit 5 years ago
Dianping ⭐ 27
scrapy抓取数据存储至本地mysql数据库-大众点评爬虫
most recent commit 3 years ago
Structure_spider ⭐ 26
组合多请求，抓取结构化数据，基于scrapy组件
total releases 49most recent commit 2 years ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.