Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Crawlab | 10,521 | 3 months ago | 1 | March 03, 2019 | 58 | bsd-3-clause | Go | |||
| Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架 | ||||||||||
| Spider Flow | 8,075 | 10 months ago | 20 | mit | Java | |||||
| 新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。 | ||||||||||
| Awesome Web Scraping | 6,060 | 5 months ago | 1 | other | Makefile | |||||
| List of libraries, tools and APIs for web scraping and data processing. | ||||||||||
| Awesome Crawler | 5,859 | 4 months ago | 27 | mit | ||||||
| A collection of awesome web crawler,spider in different languages | ||||||||||
| Douyin_tiktok_download_api | 4,844 | 5 months ago | 21 | September 23, 2023 | 60 | mit | Python | |||
| 🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具，支持API调用，在线批量解析及下载。 | ||||||||||
| Browser Fingerprinting | 3,353 | a year ago | 7 | JavaScript | ||||||
| Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web? | ||||||||||
| Grab | 2,292 |  | 107 | 3 | 10 months ago | 120 | June 24, 2018 | 1 | mit | Python |
| Web Scraping Framework | ||||||||||
| Gospider | 2,190 | 1 | 5 months ago | 17 | December 25, 2021 | 50 | mit | Go | ||
| Gospider - Fast web spider written in Go | ||||||||||
| Abot | 1,991 |  | 54 | 8 | 2 years ago | 153 | August 09, 2021 | 2 | apache-2.0 | C# |
| Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1. | ||||||||||
| Pspider | 1,675 | 2 years ago | 1 | bsd-2-clause | Python | |||||
| 简单易用的Python爬虫框架，QQ交流群：597510560 | ||||||||||
Alternatives To SpidrSelect To Compare
Alternative Project Comparisons
Popular Spider Projects
Popular Web Crawler Projects
Popular Data Processing Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Ruby
Scraper
Crawler
Spider
Web Crawler