Crawlergo
A powerful browser crawler for web vulnerability scanners
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Crawlee | 12,059 |  | 42 | 2 days ago | 747 | December 10, 2023 | 96 | apache-2.0 | TypeScript | |
| Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation. | ||||||||||
| Katana | 7,995 | 1 | 3 months ago | 8 | September 14, 2023 | 67 | mit | Go | ||
| A next-generation crawling and spidering framework. | ||||||||||
| Headless Chrome Crawler | 5,051 |  | 10 | 12 | 3 years ago | 21 | June 11, 2018 | 28 | mit | JavaScript |
| Distributed crawler powered by Headless Chrome | ||||||||||
| Rod | 4,505 | 140 | 3 months ago | 406 | November 06, 2023 | 106 | mit | Go | ||
| A Devtools driver for web automation and scraping | ||||||||||
| Crawlergo | 2,642 | 5 months ago | 2 | December 06, 2022 | 32 | gpl-3.0 | Go | |||
| A powerful browser crawler for web vulnerability scanners | ||||||||||
| Awesome Puppeteer | 2,245 | 4 months ago | 19 | |||||||
| A curated list of awesome puppeteer resources. | ||||||||||
| Rendora | 1,950 | a year ago | 1 | January 04, 2019 | 28 | apache-2.0 | Go | |||
| dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites | ||||||||||
| Kimuraframework | 874 |  | 4 | 2 | 2 years ago | 10 | January 30, 2019 | 34 | mit | Ruby |
| Kimurai is a modern web scraping framework written in Ruby which works out of box with Headless Chromium/Firefox, PhantomJS, or simple HTTP requests and allows to scrape and interact with JavaScript rendered websites | ||||||||||
| Jvppeteer | 549 | 10 months ago | 15 | October 30, 2021 | 67 | apache-2.0 | Java | |||
| Headless Chrome For Java （Java 爬虫） | ||||||||||
| Nodejs Stuff | 484 | 4 years ago | other | |||||||
| Node.js libs I want to keep in mind. | ||||||||||
Alternatives To CrawlergoSelect To Compare
Alternative Project Comparisons
Popular Crawler Projects
Popular Headless Browsers Projects
Popular Data Processing Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Golang
Crawler
Headless Browsers
Vulnerability Scanner
Headless Chrome