Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scrapyd Open Source Projects

Open source projects categorized as Scrapyd
Categories > Scrapyd
Gerapy ⭐ 2,924
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
total releases 47most recent commit 2 days ago
Spiderkeeper ⭐ 2,631
admin ui for scrapy/open source scrapinghub
total releases 6most recent commit 14 days ago
Scrapyd ⭐ 2,602
A service daemon to run Scrapy spiders
dependent packages 13total releases 7most recent commit 19 days agopypi scrapyd} Downloads
Scrapydweb ⭐ 2,408
Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉
total releases 17most recent commit a year ago
House Renting ⭐ 768
Possibly the best practice of Scrapy 🕷 and renting a house 🏡
most recent commit a year ago
Scrapyd Client ⭐ 706
Command line client for Scrapyd server
dependent packages 6total releases 7most recent commit 21 days agopypi scrapyd-client} Downloads
Timliu Python ⭐ 492
python资源集合与开源硬件
most recent commit 3 years ago
Scrapydouban ⭐ 490
豆瓣电影/豆瓣读书 Scarpy 爬虫
most recent commit a year ago
Awesome Scrapy ⭐ 450
A curated list of awesome packages, articles, and other cool resources from the Scrapy community.
most recent commit 3 months ago
Files ⭐ 369
Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects
most recent commit 4 years ago
Spider ⭐ 356
爬虫实例：微博、b站、csdn、淘宝、今日头条、知乎、豆瓣、知乎APP、大众点评
most recent commit 4 years ago
Python Scrapyd Api ⭐ 253
A Python wrapper for working with Scrapyd's API.
dependent packages 3total releases 6most recent commit 2 months agopypi python-scrapyd-api} Downloads
Newscrawl ⭐ 247
狠心开源企业级舆情新闻爬虫项目：支持任意数量爬虫一键运行、爬虫定时任务、爬虫批量删除；爬虫一键部署； 配置集群爬虫分配策略；👉 现成的docker一键部署文档已为大家踩坑
most recent commit 3 months ago
Crawlab Lite ⭐ 184
Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
most recent commit 5 months ago
Pinduoduo ⭐ 128
拼多多商品信息爬虫
most recent commit 4 years ago
Scrapyd Django Template ⭐ 103
Basic setup to run ScrapyD + Django and save it in Django Models. You can be up and running in just a few minutes
most recent commit 5 months ago
Scrapyd Cluster On Heroku ⭐ 90
Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉
most recent commit 3 years ago
Distributed Multi User Scrapy System With A Web Ui ⭐ 71
Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner
most recent commit 5 years ago
Spideradmin ⭐ 71
SpiderAdmin 一个集爬虫Scrapy+Scrapyd爬虫项目查看 和 爬虫任务定时调度的可视化管理工具
total releases 24most recent commit 2 years ago
Argus ⭐ 67
ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-0
most recent commit a year ago
Scrapy Heroku ⭐ 61
total releases 8most recent commit 5 years ago
City58 ⭐ 56
58同城 (全国) 房屋信息爬虫
most recent commit 4 years ago
Steam Scraper ⭐ 55
A pair of spiders for scraping product data and reviews from Steam.
most recent commit 5 years ago
The 4 Most Depended On Scrapyd Open Source Projects
Scrapyd ⭐ 2,602
A service daemon to run Scrapy spiders
dependent packages 13total releases 7latest release January 12, 2022most recent commit 19 days agopypi scrapyd} Downloads
Scrapyd Client ⭐ 706
Command line client for Scrapyd server
dependent packages 6total releases 7latest release May 03, 2022most recent commit 21 days agopypi scrapyd-client} Downloads
Python Scrapyd Api ⭐ 253
A Python wrapper for working with Scrapyd's API.
dependent packages 3total releases 6latest release April 01, 2018most recent commit 2 months agopypi python-scrapyd-api} Downloads
Logparser ⭐ 43
A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.
dependent packages 1total releases 3latest release August 04, 2019most recent commit 3 years agopypi logparser} Downloads
The 10 Latest Releases In Scrapyd Open Source Projects
Scrapyd Client ⭐ 706
Command line client for Scrapyd server
dependent packages 6total releases 7latest release May 03, 2022most recent commit 21 days agopypi scrapyd-client} Downloads
Scrapyd ⭐ 2,602
A service daemon to run Scrapy spiders
dependent packages 13total releases 7latest release January 12, 2022most recent commit 19 days agopypi scrapyd} Downloads
Gerapy ⭐ 2,924
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js
total releases 47latest release December 30, 2021most recent commit 2 days ago
Scrapyd Heroku ⭐ 14
Wrapper for running Scrapyd in Heroku or locally as a service
total releases 5latest release November 28, 2020most recent commit 2 years ago
Logparser ⭐ 43
A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.
dependent packages 1total releases 3latest release August 04, 2019most recent commit 3 years agopypi logparser} Downloads
Python Scrapyd Api ⭐ 253
A Python wrapper for working with Scrapyd's API.
dependent packages 3total releases 6latest release April 01, 2018most recent commit 2 months agopypi python-scrapyd-api} Downloads
Spiderkeeper ⭐ 2,631
admin ui for scrapy/open source scrapinghub
total releases 6latest release September 24, 2017most recent commit 14 days ago
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.