Crawlers
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Crawling Infrastructure | 321 | 2 years ago | 22 | agpl-3.0 | TypeScript | |||||
| Distributed crawling infrastructure running on top of severless computation, cloud storage (such as S3) and sophisticated queues. | ||||||||||
| Bathyscaphe | 83 | a year ago | 16 | March 05, 2021 | 4 | gpl-3.0 | Go | |||
| Fast, highly configurable, cloud native dark web crawler. | ||||||||||
| Youtube_livechat_replay_crawler | 60 | a year ago | mit | Python | ||||||
| This script's purpose is to get live chat comments from Youtube. | ||||||||||
| Findopendata | 49 | a year ago | 6 | December 26, 2019 | 3 | apache-2.0 | Python | |||
| A search engine for Open Data | ||||||||||
| Scrapeulous | 30 | 3 years ago | 7 | JavaScript | ||||||
| Cloud crawler functions for scrapeulous | ||||||||||
| Serverless Crawler Demo | 27 | 4 years ago | 4 | mit | JavaScript | |||||
| Serverless Architecture Crawler demo | ||||||||||
| Animeapi | 25 | 8 years ago | 1 | mit | JavaScript | |||||
| :cloud: A RESTful API for Anime Data. | ||||||||||
| Akka React Cloudant | 21 | 2 years ago | 3 | CSS | ||||||
| A Soccer Dashboard created by scraping EPL website using Akka backend and ReactJS frontend and IBM Cloudant for object storage. IBM Cloud Foundry is used to host both frontend and backend app. | ||||||||||
| Twds Crawler | 18 | 2 years ago | Python | |||||||
| Highly scalable webcrawler for towardsdatascience.com by using Python, Selenium, Docker, Kubernetes and the infrastructure of the Google Cloud Platform | ||||||||||
| Web Scraper Gcp | 15 | 4 years ago | 1 | mit | JavaScript | |||||
| Scrape all the pages and links of a given domain and write the results to Google Cloud BigQuery. | ||||||||||
Alternatives To CrawlersSelect To Compare
Alternative Project Comparisons
Popular Crawler Projects
Popular Cloud Computing Projects
Popular Data Processing Categories
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Html
Cloud Computing
Crawler
Downloader