Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Crawler Scrape Open Source Projects

Open source projects categorized as Crawler Scrape
Categories > Data Processing > Crawler
Categories > Scrape
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit a year agopypi autoscraper} Downloads
Scrapely ⭐ 1,668
A pure-python HTML screen-scraping library
dependent packages 2total releases 13most recent commit 4 years agopypi scrapely} Downloads
Mlscraper ⭐ 935
🤖 Scrape data from HTML websites automatically by just providing examples
total releases 4most recent commit a year ago
Skrape.it ⭐ 714
A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
dependent packages 3total releases 14most recent commit 4 months ago
Voyager ⭐ 618
crawl and scrape web pages in rust
total releases 4most recent commit 10 months ago
Stweet ⭐ 308
Advanced python library to scrap Twitter (tweets, users) from unofficial API
total releases 20most recent commit a year ago
Node Readability ⭐ 302
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
dependent packages 4total releases 67most recent commit 6 years agonpm read-art} Downloads
Python Automation Scripts ⭐ 264
Simple yet powerful automation stuffs.
most recent commit 3 years ago
Rcrawler ⭐ 240
An R web crawler and scraper
total releases 9most recent commit 4 years ago
Nudecrawler ⭐ 231
Crawl telegra.ph searching for nudes!
total releases 38most recent commit 7 months ago
Wayback Machine Scraper ⭐ 219
A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
total releases 8most recent commit 3 years ago
Portia Dashboard ⭐ 190
portia-dashboard is a visual web crawler based on scrapinghub/portia
most recent commit 6 years ago
Scrapy Samples ⭐ 183
Scrapy examples crawling Craigslist
most recent commit 8 years ago
Hncrawl ⭐ 150
A scrapy-based Hacker News crawler.
most recent commit 11 years ago
Media Scraper ⭐ 150
Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
most recent commit 4 years ago
Scrape ⭐ 135
a command-line web scraping tool
total releases 112most recent commit 2 years ago
Grawler ⭐ 128
Grawler is a tool written in PHP which comes with a web interface that automates the task of using google dorks, scrapes the results, and stores them in a file.
most recent commit 3 years ago
Ca Property Tax ⭐ 78
CA property tax visualization
most recent commit 2 years ago
Quora Crawler ⭐ 64
Python crawler for quora.com
most recent commit 10 years ago
Goscraper ⭐ 58
Golang pkg to quickly return a preview of a webpage (title/description/images)
dependent packages 6total releases 1most recent commit 4 years ago
Diffbot Php Client ⭐ 53
[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library
total releases 27most recent commit 6 years ago
Li ⭐ 53
Next-generation serverless crawler for COVID-19 data
most recent commit 3 years ago
Yelpcrawl ⭐ 45
Crawl and scrape Yelp's restaurant data for every zip code in the United States (or a specified zipcode). Yelp Crawler.
most recent commit 7 years ago
The 10 Most Depended On Crawler Scrape Open Source Projects
Extension ⭐ 32
web scraping extension
dependent packages 13total releases 7latest release September 26, 2020most recent commit a year agonpm get-set-fetch-extension-commons} Downloads
Goscraper ⭐ 58
Golang pkg to quickly return a preview of a webpage (title/description/images)
dependent packages 6total releases 1latest release August 27, 2019most recent commit 4 years ago
Node Readability ⭐ 302
Scrape/Crawl article from any site automatically. Make any web page readable, no matter Chinese or English.
dependent packages 4total releases 67latest release August 01, 2018most recent commit 6 years agonpm read-art} Downloads
Logo Scrape ⭐ 17
🕷🚀 Scrapes/Crawls the logo from a provided url(s)/website for your Node.js applications.
dependent packages 3total releases 8latest release March 11, 2019most recent commit 3 years agonpm logo-scrape} Downloads
Noscrape ⭐ 32
obfuscate text via node to make scraping your content really difficult
dependent packages 2total releases 20latest release September 28, 2023most recent commit 4 months agonpm @noscrape/noscrape} Downloads
Scrapely ⭐ 1,668
A pure-python HTML screen-scraping library
dependent packages 2total releases 13latest release November 28, 2019most recent commit 4 years agopypi scrapely} Downloads
Nspider ⭐ 5
a lightweight nodejs spider framework
dependent packages 2total releases 12latest release December 21, 2018most recent commit 5 years agonpm nspider22} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16latest release July 17, 2022most recent commit a year agopypi autoscraper} Downloads
Soccer Scrape ⭐ 16
:page_with_curl: Scrape football data from Bet365
dependent packages 1total releases 2latest release March 16, 2018most recent commit 6 years agonpm soccer-scrape} Downloads
The 10 Latest Releases In Crawler Scrape Open Source Projects
Noscrape ⭐ 32
obfuscate text via node to make scraping your content really difficult
dependent packages 2total releases 20latest release September 28, 2023most recent commit 4 months agonpm @noscrape/noscrape} Downloads
Scrapyteer ⭐ 9
Web crawling & scraping framework for Node.js on top of headless Chrome browser
total releases 8latest release June 07, 2023most recent commit 10 months ago
Nudecrawler ⭐ 231
Crawl telegra.ph searching for nudes!
total releases 38latest release April 30, 2023most recent commit 7 months ago
Stweet ⭐ 308
Advanced python library to scrap Twitter (tweets, users) from unofficial API
total releases 20latest release February 06, 2023most recent commit a year ago
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16latest release July 17, 2022most recent commit a year agopypi autoscraper} Downloads
Mlscraper ⭐ 935
🤖 Scrape data from HTML websites automatically by just providing examples
total releases 4latest release June 24, 2022most recent commit a year ago
Scrape ⭐ 135
a command-line web scraping tool
total releases 112latest release February 20, 2022most recent commit 2 years ago
Ptt Crawler ⭐ 18
ptt-crawler is a web crawler module designed to scarpe data from Ptt.
total releases 12latest release January 29, 2022most recent commit a year ago
Voyager ⭐ 618
crawl and scrape web pages in rust
total releases 4latest release January 12, 2022most recent commit 10 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.