Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Gau | 3,273 | 1 | 4 months ago | 31 | November 02, 2023 | 22 | mit | Go | ||
Fetch known URLs from AlienVault's Open Threat Exchange, the Wayback Machine, and Common Crawl. | ||||||||||
Pystock Crawler | 299 | 1 | 2 years ago | 14 | October 19, 2014 | 4 | mit | Python | ||
Crawl and parse financial reports (XBRL) from SEC EDGAR, and daily stock prices from Yahoo Finance | ||||||||||
Python Automation Scripts | 264 | 3 years ago | gpl-3.0 | Python | ||||||
Simple yet powerful automation stuffs. | ||||||||||
Galer | 189 | a year ago | 2 | November 05, 2021 | 5 | mit | Go | |||
A fast tool to fetch URLs from HTML attributes by crawl-in. | ||||||||||
Scraply | 114 | 2 years ago | 1 | July 07, 2022 | apache-2.0 | Go | ||||
Scraply a simple dom scraper to fetch information from any html based website | ||||||||||
Driveit | 94 | 7 years ago | 1 | wtfpl | Python | |||||
A New Multithreading Crawler Supports Multiple Websites | ||||||||||
Ferrit | 60 | 10 years ago | 2 | mit | Scala | |||||
Ferrit is a web crawler service written in Scala using Akka, Spray and Cassandra. | ||||||||||
Spiderman | 49 | 1 | 10 years ago | 10 | July 10, 2014 | JavaScript | ||||
a crawler with visualized config board | ||||||||||
Anilist Crawler | 43 | 3 months ago | mit | JavaScript | ||||||
Crawl data from anilist API and store in MariaDB. | ||||||||||
Relspider | 28 | 10 years ago | 6 | JavaScript | ||||||
A web crawler that indexes the identity/social graph of profiles on the web |