Web Scraping

Code samples of web scraping using Java.
Alternatives To Web Scraping
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Scrapy49,9184,1854452 months ago96September 18, 2023692bsd-3-clausePython
Scrapy, a fast high-level web crawling & scraping framework for Python.
Huginn40,32869522 months ago8September 22, 2017698mitRuby
Create agents that monitor and act on your behalf. Your agents are standing by!
Crawlee11,736429 days ago747December 10, 202396apache-2.0TypeScript
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
Awesome Web Scraping6,060
4 months ago1otherMakefile
List of libraries, tools and APIs for web scraping and data processing.
Awesome Crawler5,859
4 months ago27mit
A collection of awesome web crawler,spider in different languages
Autoscraper5,1591a year ago16July 17, 20229mitPython
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Douyin_tiktok_download_api4,844
4 months ago21September 23, 202360mitPython
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具,支持API调用,在线批量解析及下载。
Rod4,5051402 months ago406November 06, 2023106mitGo
A Devtools driver for web automation and scraping
Node Osmosis4,083218588 months ago27March 01, 2019117JavaScript
Web scraper for NodeJS
Browser Fingerprinting3,353
a year ago7JavaScript
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️‍♂️ when scraping the web?
Alternatives To Web Scraping
Select To Compare


Alternative Project Comparisons
Popular Scraper Projects
Popular Web Crawler Projects
Popular Data Processing Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Java
Scraper
Web Crawler
Jsoup