Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Scrapy | 49,918 | 4,185 | 445 | a year ago | 96 | September 18, 2023 | 692 | bsd-3-clause | Python | |
Scrapy, a fast high-level web crawling & scraping framework for Python. | ||||||||||
Crawlee | 12,871 | 42 | 9 months ago | 747 | December 10, 2023 | 96 | apache-2.0 | TypeScript | ||
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation. | ||||||||||
Autoscraper | 5,159 | 1 | 2 years ago | 16 | July 17, 2022 | 9 | mit | Python | ||
A Smart, Automatic, Fast and Lightweight Web Scraper for Python | ||||||||||
Douyin_tiktok_download_api | 4,844 | a year ago | 21 | September 23, 2023 | 60 | mit | Python | |||
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具,支持API调用,在线批量解析及下载。 | ||||||||||
Rod | 4,505 | 140 | a year ago | 406 | November 06, 2023 | 106 | mit | Go | ||
A Devtools driver for web automation and scraping | ||||||||||
Node Osmosis | 4,083 | 218 | 58 | 2 years ago | 27 | March 01, 2019 | 117 | JavaScript | ||
Web scraper for NodeJS | ||||||||||
Automatic Udemy Course Enroller Get Paid Udemy Courses For Free | 3,010 | a year ago | 11 | June 03, 2022 | 28 | gpl-3.0 | Python | |||
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE! | ||||||||||
Snoop | 2,530 | a year ago | 1 | other | Python | |||||
Snoop — инструмент разведки на основе открытых данных (OSINT world) | ||||||||||
Trafilatura | 2,447 | 66 | a year ago | 39 | November 29, 2023 | 66 | gpl-3.0 | Python | ||
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments | ||||||||||
Grab | 2,292 | 107 | 3 | 2 years ago | 120 | June 24, 2018 | 1 | mit | Python | |
Web Scraping Framework |