Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Photon | 10,244 | 5 | 4 months ago | 18 | January 25, 2019 | 52 | gpl-3.0 | Python | ||
Incredibly fast crawler designed for OSINT. | ||||||||||
Swiftlinkpreview | 1,347 | 12 | 8 months ago | 28 | February 01, 2021 | 25 | mit | Swift | ||
It makes a preview from an URL, grabbing all the information such as title, relevant texts and images. | ||||||||||
Easy Scraping Tutorial | 618 | 3 years ago | 7 | mit | Jupyter Notebook | |||||
Simple but useful Python web scraping tutorial code. | ||||||||||
Music Recover | 397 | 2 years ago | 2 | mit | Python | |||||
:musical_note: 缓存文件转换为 MP3 文件 | ||||||||||
Sinew | 254 | 3 | 1 | 8 months ago | 14 | July 09, 2021 | mit | Ruby | ||
A Ruby DSL for structured web crawling, with a robust caching system. | ||||||||||
Filesensor | 207 | 4 years ago | 1 | Python | ||||||
Dynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具 | ||||||||||
Evine | 117 | 3 years ago | 8 | August 01, 2020 | gpl-3.0 | Go | ||||
Interactive CLI Web Crawler | ||||||||||
Hyperspider | 115 | 2 | 3 | 12 years ago | 2 | August 09, 2012 | JavaScript | |||
A declarative HATEOAS API crawler for node.js | ||||||||||
Graphquery | 104 | 3 | 4 | 5 years ago | 1 | March 17, 2019 | apache-2.0 | Go | ||
GraphQuery is a query language and execution engine tied to any backend service. | ||||||||||
Scrapingspider | 73 | 11 years ago | 1 | C# | ||||||
业余时间开发的,支持多线程,支持关键字过滤,支持正文内容智能识别的爬虫。 |