Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Website Scrapy Open Source Projects

Open source projects categorized as Website Scrapy
Findtrip ⭐ 324
机票爬虫（去哪儿和携程网）。flight tickets multiple webspider.(scrapy + selenium + phantomjs + mongodb)
most recent commit 8 years ago
City Scrapers ⭐ 315
Scrape, standardize and share public meetings from local government websites
most recent commit 3 months ago
Major Scrapy Spiders ⭐ 196
Scrapy spiders of major websites. Google Play Store, Facebook, Instagram, Ebay, YTS Movies, Amazon
most recent commit 7 years ago
Soul Manga ⭐ 117
react + flask + scrapy 构建的单页应用漫画网站
most recent commit 6 years ago
Scraply ⭐ 114
Scraply a simple dom scraper to fetch information from any html based website
total releases 1most recent commit 2 years ago
Capturer ⭐ 94
capture pictures from website like sina, lofter, huaban and so on
most recent commit 2 years ago
Proxy_server_crawler ⭐ 85
an awesome public proxy server crawler based on scrapy framework
most recent commit 7 years ago
Openscraper ⭐ 80
An open source webapp for scraping: towards a public service for webscraping
most recent commit 2 years ago
Itbooks ⭐ 75
Get itbooks from ebooks's website for free,such as allitebooks,digilibraries,etc
most recent commit 7 years ago
Pyfeeds ⭐ 73
DIY Atom feeds in times of social media and paywalls
total releases 3most recent commit 4 months ago
Argus ⭐ 67
ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-0
most recent commit 2 years ago
Scrape_news ⭐ 60
using python Scrapy framework, do multiprocess scrape news
most recent commit 6 years ago
Scrapy.org ⭐ 57
The scrapy.org website
most recent commit 4 months ago
Recruitment ⭐ 57
A project crawling online recruitment websites getting offer information
most recent commit 7 years ago
Fishfishjump ⭐ 57
Fish Fish Jump is a solution in the python that simply and basic for search engines. 🐟 🐟 🐟
total releases 10most recent commit 6 years ago
Torrents Crawler ⭐ 50
This is a scrapy project in which I have implemented several crawlers for different torrent and direct link websites.
most recent commit 4 years ago
Pronhubspider ⭐ 50
pornhubをクロールしているWebHubBotプロジェクトの模倣、効率が遅すぎる、方法を探してい
most recent commit 6 months ago
Bus_catchers ⭐ 35
Python scripts for scraping bus ticket data from the websites of BoltBus, Greyhound, Megabus, GoBus, Amtrak, Peterpan, and EasternTravel.
most recent commit 11 years ago
Amazon Pricetracker ⭐ 27
Track the prices of different products on Amazon.in
most recent commit 5 years ago
Google_search_module ⭐ 24
Retrieve google results using python
most recent commit 10 years ago
Wanfangdata ⭐ 19
spider and a web application for WanFang scholar website 万方数据爬虫+Web展示+TF-IDF相似度分析
most recent commit 3 years ago
Netease_spider ⭐ 17
网易严选爬虫
most recent commit 7 years ago
Agespider ⭐ 15
A simple spider for a anime website
most recent commit 2 years ago
The 4 Latest Releases In Website Scrapy Open Source Projects
Scraply ⭐ 114
Scraply a simple dom scraper to fetch information from any html based website
total releases 1latest release July 07, 2022most recent commit 2 years ago
Pyfeeds ⭐ 73
DIY Atom feeds in times of social media and paywalls
total releases 3latest release June 18, 2022most recent commit 4 months ago
Bookscrape ⭐ 10
Bookscrape - Scrape and export books from various websites
total releases 25latest release February 13, 2019most recent commit 2 years ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.