Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Crawlee | 12,106 | 42 | 3 days ago | 747 | December 10, 2023 | 96 | apache-2.0 | TypeScript | ||
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation. | ||||||||||
Headless Chrome Crawler | 5,051 | 10 | 12 | 3 years ago | 21 | June 11, 2018 | 28 | mit | JavaScript | |
Distributed crawler powered by Headless Chrome | ||||||||||
Browser Fingerprinting | 3,353 | a year ago | 7 | JavaScript | ||||||
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web? | ||||||||||
Thal | 2,268 | 3 years ago | mit | JavaScript | ||||||
Getting started with Puppeteer and Chrome Headless for Web Scraping | ||||||||||
Awesome Puppeteer | 2,245 | 4 months ago | 19 | |||||||
A curated list of awesome puppeteer resources. | ||||||||||
Fingerprint Suite | 587 | 13 | 3 months ago | 87 | December 01, 2023 | 26 | apache-2.0 | TypeScript | ||
Browser fingerprinting tools for anonymizing your scrapers. Developed by Apify. | ||||||||||
Secret Agent | 575 | 32 | a year ago | 33 | May 02, 2021 | 43 | mit | TypeScript | ||
The web scraper that's nearly impossible to block - now called @ulixee/hero | ||||||||||
Linvo Scraper | 553 | 2 | 8 months ago | 6 | October 09, 2022 | 14 | mit | TypeScript | ||
Linkedin Automation Bot with every possible scraping! Valid for 2022 used by Linvo.io | ||||||||||
Jvppeteer | 549 | 10 months ago | 15 | October 30, 2021 | 67 | apache-2.0 | Java | |||
Headless Chrome For Java (Java 爬虫) | ||||||||||
Phpscraper | 486 | 5 months ago | 34 | June 18, 2023 | 20 | gpl-3.0 | PHP | |||
A universal web-util for PHP. |