Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Gerapy | 3,144 | 8 | 5 months ago | 49 | July 19, 2023 | 60 | mit | Python | ||
Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js | ||||||||||
Brozzler | 613 | 2 | 5 months ago | 23 | January 02, 2020 | 40 | apache-2.0 | Python | ||
brozzler - distributed browser-based web crawler | ||||||||||
Thea11ymachine | 553 | 4 | 2 | 5 years ago | 26 | February 08, 2017 | 35 | JavaScript | ||
The A11y Machine is an automated accessibility testing tool which crawls and tests pages of any web application to produce detailed reports. | ||||||||||
Sparkler | 401 | a year ago | 55 | apache-2.0 | Java | |||||
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark. | ||||||||||
Ghcrawler | 301 | 3 | 2 | 4 years ago | 43 | March 09, 2017 | 28 | mit | JavaScript | |
Crawl GitHub APIs and store the discovered orgs, repos, commits, ... | ||||||||||
Seo Audits Toolkit | 284 | a year ago | 31 | Python | ||||||
SEO & Security Audit for Websites. Lighthouse & Security Headers crawler, Sitemap/Keywords/Images Extractor, Summarizer, etc ... | ||||||||||
Portia Dashboard | 190 | 6 years ago | 6 | other | Python | |||||
portia-dashboard is a visual web crawler based on scrapinghub/portia | ||||||||||
Memex Explorer | 106 | 8 years ago | 67 | bsd-2-clause | Python | |||||
Viewers for statistics and dashboarding of Domain Search Engine data | ||||||||||
Fishfishjump | 57 | 1 | 6 years ago | 10 | March 08, 2018 | mit | JavaScript | |||
Fish Fish Jump is a solution in the python that simply and basic for search engines. :fish: :fish: :fish: | ||||||||||
Pomp Craigslist Example | 33 | 7 years ago | 2 | HTML | ||||||
Extract data from Craigslist.org by python3 and pomp framework |