Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Scraper Open Source Projects

Open source projects categorized as Scraper
Categories > Data Processing > Scraper
Scrapy ⭐ 46,586
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 373total releases 87most recent commit 9 hours agopypi Scrapy} Downloads
Huginn ⭐ 37,907
Create agents that monitor and act on your behalf. Your agents are standing by!
dependent packages 35total releases 8most recent commit 2 days agorubygems huginn_agent} Downloads
Devdocs ⭐ 31,110
API Documentation Browser
most recent commit a day ago
Cheerio ⭐ 26,114
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
dependent packages 16,650total releases 70most recent commit 2 days agonpm cheerio} Downloads
Lux ⭐ 20,441
👾 Fast and simple video download library and CLI tool written in Go
dependent packages 5total releases 31most recent commit 3 days ago
Colly ⭐ 19,134
Elegant Scraper and Crawler Framework for Golang
dependent packages 201total releases 22most recent commit 13 days ago
Requests Html ⭐ 13,007
Pythonic HTML Parsing for Humans™
dependent packages 166total releases 36most recent commit 11 days agopypi requests-html} Downloads
Newspaper ⭐ 12,516
News, full-text, and article metadata extraction in Python 3. Advanced docs:
dependent packages 48total releases 18most recent commit 18 days agopypi newspaper3k} Downloads
Webmagic ⭐ 10,761
A scalable web crawler framework for Java.
dependent packages 15total releases 21most recent commit 4 days ago
Jsoup ⭐ 10,043
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
dependent packages 1,305total releases 40most recent commit 18 hours ago
Chinese Xinhua ⭐ 9,789
📙 中华新华字典数据库。包括歇后语，成语，词语，汉字。
most recent commit 4 months ago
Portia ⭐ 8,781
Visual scraping for Scrapy
dependent packages 2total releases 26most recent commit 10 days agopypi slybot} Downloads
Avbook ⭐ 8,777
AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
most recent commit 23 days ago
Changedetection.io ⭐ 8,480
The best and simplest self-hosted free open source website change detection, monitor and notification service. Restock Monitor, change detection. Designed for simplicity - the main goal is to simply monitor which websites had a text change for free. Free Open source web page change detection, Restock Monitoring, Visualping and Apify alternative
most recent commit 3 days ago
Crawlee ⭐ 7,949
Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.
dependent packages 16total releases 246most recent commit 2 days agonpm @crawlee/types} Downloads
Tabula ⭐ 5,629
Tabula is a tool for liberating data tables trapped inside PDF files
most recent commit a year ago
Awesome Web Scraping ⭐ 5,576
List of libraries, tools and APIs for web scraping and data processing.
most recent commit 3 months ago
Awesome Crawler ⭐ 5,308
A collection of awesome web crawler,spider in different languages
most recent commit 3 months ago
Ferret ⭐ 5,251
Declarative web scraping
dependent packages 3total releases 34most recent commit 5 days ago
Headless Chrome Crawler ⭐ 5,051
Distributed crawler powered by Headless Chrome
dependent packages 9total releases 21most recent commit 2 years agonpm headless-chrome-crawler} Downloads
Autoscraper ⭐ 4,803
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit 3 months agopypi autoscraper} Downloads
Undetected Chromedriver ⭐ 4,446
Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
dependent packages 31total releases 35most recent commit 3 days agopypi undetected-chromedriver} Downloads
Mechanize ⭐ 4,277
Mechanize is a ruby library that makes automated web interaction easy.
dependent packages 966total releases 84most recent commit 12 days agorubygems mechanize} Downloads
The 10 Most Depended On Scraper Open Source Projects
Cheerio ⭐ 26,114
The fast, flexible, and elegant library for parsing and manipulating HTML and XML.
dependent packages 16,650total releases 70latest release June 26, 2022most recent commit 2 days agonpm cheerio} Downloads
Jsoup ⭐ 10,043
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
dependent packages 1,305total releases 40latest release August 24, 2022most recent commit 18 hours ago
Mechanize ⭐ 4,277
Mechanize is a ruby library that makes automated web interaction easy.
dependent packages 966total releases 84latest release June 09, 2022most recent commit 12 days agorubygems mechanize} Downloads
Node Ytdl Core ⭐ 3,653
YouTube video downloader in javascript.
dependent packages 520total releases 184latest release September 03, 2022most recent commit 17 days agonpm ytdl-core} Downloads
Scrapy ⭐ 46,586
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 373total releases 87latest release March 01, 2022most recent commit 9 hours agopypi Scrapy} Downloads
Colly ⭐ 19,134
Elegant Scraper and Crawler Framework for Golang
dependent packages 201total releases 22latest release March 08, 2022most recent commit 13 days ago
Scraper ⭐ 1,291
HTML parsing and querying with CSS selectors
dependent packages 177total releases 20latest release April 24, 2022most recent commit 21 days agocargo scraper} Downloads
Requests Html ⭐ 13,007
Pythonic HTML Parsing for Humans™
dependent packages 166total releases 36latest release February 17, 2019most recent commit 11 days agopypi requests-html} Downloads
Parsel ⭐ 884
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors
dependent packages 98total releases 20latest release May 07, 2020most recent commit a month agopypi parsel} Downloads
Scrape It ⭐ 3,894
🔮 A Node.js scraper for humans.
dependent packages 72total releases 42latest release September 07, 2021most recent commit 7 days agonpm scrape-it} Downloads
The 10 Latest Releases In Scraper Open Source Projects
Crawlee ⭐ 7,949
Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.
dependent packages 16total releases 246latest release September 23, 2022most recent commit 2 days agonpm @crawlee/types} Downloads
Rod ⭐ 3,556
A Devtools driver for web automation and scraping
dependent packages 58total releases 385latest release September 23, 2022most recent commit 5 days ago
Instagram Php Scraper ⭐ 2,783
Get account information, photos, videos, stories and comments.
dependent packages 17total releases 112latest release September 07, 2022most recent commit 2 months agopackagist raiym/instagram-php-scraper} Downloads
Embed ⭐ 1,944
Get info from any web service or page
dependent packages 62total releases 160latest release September 06, 2022most recent commit 3 months agopackagist embed/embed} Downloads
Node Ytdl Core ⭐ 3,653
YouTube video downloader in javascript.
dependent packages 520total releases 184latest release September 03, 2022most recent commit 17 days agonpm ytdl-core} Downloads
Jsoup ⭐ 10,043
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
dependent packages 1,305total releases 40latest release August 24, 2022most recent commit 18 hours ago
Google Play Scraper ⭐ 1,920
Node.js scraper to get data from Google Play
dependent packages 22total releases 84latest release August 23, 2022most recent commit 3 days agonpm google-play-scraper} Downloads
Ferret ⭐ 5,251
Declarative web scraping
dependent packages 3total releases 34latest release July 30, 2022most recent commit 5 days ago
Recipe Scrapers ⭐ 1,111
Python package for scraping recipes data
dependent packages 3total releases 141latest release July 02, 2022most recent commit 2 days agopypi recipe-scrapers} Downloads
Facebook Scraper ⭐ 1,647
Scrape Facebook public pages without an API key
dependent packages 6total releases 74latest release June 28, 2022most recent commit 4 days agopypi facebook-scraper} Downloads
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Classification
Scraper
Transaction
Data Science
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.