Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Webscraper Open Source Projects

Open source projects categorized as Python Webscraper
Categories > Webscraper
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit a year agopypi autoscraper} Downloads
Pspider ⭐ 1,675
简单易用的Python爬虫框架，QQ交流群：597510560
most recent commit 2 years ago
100projectsofcode ⭐ 1,293
A list of practical knowledge-building projects.
most recent commit 3 years ago
Lightnovel Crawler ⭐ 1,185
Generate and download e-books from online sources.
dependent packages 1total releases 30most recent commit 3 months agopypi ebook-crawler} Downloads
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18most recent commit 3 months agopypi faster-than-requests} Downloads
Scrapple ⭐ 452
A framework for creating semi-automatic web content extractors
total releases 10most recent commit 5 years ago
Wereadscan ⭐ 447
扫描“微信读书”已购图书并下载本地PDF的爬虫
total releases 11most recent commit 7 months ago
Kochat ⭐ 383
Opensource Korean chatbot framework
total releases 27most recent commit a year ago
Basketball_reference_web_scraper ⭐ 382
NBA Stats API via Basketball Reference
total releases 39most recent commit 3 months ago
Proxy_requests ⭐ 381
a class that uses scraped proxies to make http GET/POST requests (Python requests)
dependent packages 2total releases 49most recent commit 3 years agopypi proxy-requests} Downloads
Scrape Linkedin Selenium ⭐ 353
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
most recent commit 2 years ago
Archivebot ⭐ 328
ArchiveBot, an IRC bot for archiving websites
most recent commit 5 months ago
Social Media Profile Scrapers ⭐ 322
Fetch user's data across social media
most recent commit 7 months ago
Spidy ⭐ 287
The simple, easy to use command line web crawler.
total releases 12most recent commit 2 years ago
Web Scraping ⭐ 276
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
most recent commit 2 years ago
Lagoujob ⭐ 250
Job data mining repo for lagou.com
most recent commit 5 years ago
Summarizer ⭐ 236
A Reddit bot that summarizes news articles written in Spanish or English. It uses a custom built algorithm to rank words and sentences.
most recent commit 2 years ago
Amazon Scraper ⭐ 219
A simple web scraper to extract Product Data and Pricing from Amazon
most recent commit 10 months ago
Rightmove_webscraper.py ⭐ 219
Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object
total releases 8most recent commit 4 months ago
Portia Dashboard ⭐ 190
portia-dashboard is a visual web crawler based on scrapinghub/portia
most recent commit 6 years ago
Ignareo Isml Auto Voter ⭐ 186
Ignareo the Carillon, a web crawler/spider template of ultimate high concurrency built for leprechauns. Carillons as the best web spiders; Long live the golden years of leprechauns! (ISML=international saimoe; 2022 ISML is last ISML)
most recent commit 7 months ago
Daath Ai Parser ⭐ 184
Daath AI Parser is an open-source application that uses OpenAI to parse visible text of HTML elements.
most recent commit 9 months ago
Zhihu Crawler People ⭐ 179
A simple distributed crawler for zhihu && data analysis
most recent commit 4 years ago
The 6 Most Depended On Python Webscraper Open Source Projects
Faster Than Requests ⭐ 1,061
Faster requests on Python 3
dependent packages 4total releases 18latest release February 24, 2021most recent commit 3 months agopypi faster-than-requests} Downloads
Proxy_requests ⭐ 381
a class that uses scraped proxies to make http GET/POST requests (Python requests)
dependent packages 2total releases 49latest release July 04, 2020most recent commit 3 years agopypi proxy-requests} Downloads
Cazy_webscraper ⭐ 10
Web scraper to retrieve protein data catalogued by the CAZy, UniProt, NCBI, GTDB and PDB websites/databases.
dependent packages 1total releases 29latest release January 16, 2023most recent commit 3 months agopypi cazy-webscraper} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16latest release July 17, 2022most recent commit a year agopypi autoscraper} Downloads
Lightnovel Crawler ⭐ 1,185
Generate and download e-books from online sources.
dependent packages 1total releases 30latest release December 03, 2018most recent commit 3 months agopypi ebook-crawler} Downloads
The 10 Latest Releases In Python Webscraper Open Source Projects
Dmr.py ⭐ 11
Pull data from the danish vehicle registry with dmr.py
total releases 13latest release November 30, 2023most recent commit 5 months ago
Letterboxdpy ⭐ 20
A letterboxd webscraper
total releases 22latest release November 24, 2023most recent commit 3 months ago
Kenpompy ⭐ 61
A simple yet comprehensive web scraper for kenpom.com.
total releases 9latest release November 22, 2023most recent commit 5 months ago
Basketball_reference_web_scraper ⭐ 382
NBA Stats API via Basketball Reference
total releases 39latest release November 14, 2023most recent commit 3 months ago
Python Scrapfly ⭐ 14
Scrapfly Python SDK for headless browsers and proxy rotation
total releases 33latest release October 27, 2023most recent commit 4 months ago
Scraperx ⭐ 53
Library for scraping websites or apis at any scale
total releases 75latest release September 25, 2023most recent commit 7 months ago
Cazy_webscraper ⭐ 10
Web scraper to retrieve protein data catalogued by the CAZy, UniProt, NCBI, GTDB and PDB websites/databases.
dependent packages 1total releases 29latest release January 16, 2023most recent commit 3 months agopypi cazy-webscraper} Downloads
Get Sentral ⭐ 7
A simple Python function to summon your timetable from Sentral.
total releases 6latest release December 20, 2022most recent commit 4 months ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.