Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Scraper Web Crawler Open Source Projects

Open source projects categorized as Python Scraper Web Crawler
Categories > Data Processing > Scraper
Categories > Data Processing > Web Crawler
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 3 months agopypi Scrapy} Downloads
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit a year agopypi autoscraper} Downloads
Douyin_tiktok_download_api ⭐ 4,844
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、T
total releases 21most recent commit 5 months ago
Automatic Udemy Course Enroller Get Paid Udemy Courses For Free ⭐ 3,010
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
total releases 11most recent commit 3 months ago
Snoop ⭐ 2,530
Snoop — инструмент разведки на основе открытых данных (OSINT world)
most recent commit 3 months ago
Trafilatura ⭐ 2,447
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39most recent commit 3 months agopypi trafilatura} Downloads
Grab ⭐ 2,292
Web Scraping Framework
dependent packages 3total releases 120most recent commit 10 months agopypi grab} Downloads
Django Dynamic Scraper ⭐ 1,069
Creating Scrapy scrapers via the Django admin interface
total releases 65most recent commit 2 years ago
Gazpacho ⭐ 716
🥫 The simple, fast, and modern web scraping library
dependent packages 8total releases 19most recent commit 5 months agopypi gazpacho} Downloads
Instascrape ⭐ 554
Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically
total releases 56most recent commit a year ago
Complete Life Cycle Of A Data Science Project ⭐ 499
Complete-Life-Cycle-of-a-Data-Science-Project
most recent commit 4 months ago
Jekyll ⭐ 498
Jekyll-based static site for The Programming Historian
most recent commit 3 months ago
Scrapple ⭐ 452
A framework for creating semi-automatic web content extractors
total releases 10most recent commit 5 years ago
Google Search Results Python ⭐ 432
Google Search Results via SERP API pip Python Package
dependent packages 99total releases 41most recent commit 7 months agopypi google-search-results} Downloads
Dude ⭐ 397
dude uncomplicated data extraction: A simple framework for writing web scrapers using Python decorators
total releases 43most recent commit 3 months ago
Basketball_reference_web_scraper ⭐ 382
NBA Stats API via Basketball Reference
total releases 39most recent commit 3 months ago
Scrape Linkedin Selenium ⭐ 353
`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
most recent commit 2 years ago
City Scrapers ⭐ 315
Scrape, standardize and share public meetings from local government websites
most recent commit 3 months ago
Mov Cli ⭐ 314
A cli tool to browse and watch Movies/Shows/TV/Sports.
dependent packages 1total releases 103most recent commit 3 months agopypi mov-cli} Downloads
Web Scraping ⭐ 281
Más de 50 ejemplos de web scraping utilizando: Requests | Scrapy | Selenium | LXML | BeautifulSoup
most recent commit 3 months ago
Web Scraping ⭐ 276
Detailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, MacroTrends, SHFE and alternative data crawlers on Tomtom, BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
most recent commit 2 years ago
Youtube Projects ⭐ 272
This repository contains all the code I use in my YouTube tutorials.
most recent commit 3 years ago
Dark Fantasy Hack Tool ⭐ 248
DDOS Tool: To take down small websites with HTTP FLOOD. Port scanner: To know the open ports of a site. FTP Password Cracker: To hack file system of websites.. Banner Grabber: To get the service or software running on a port. (After knowing the software running google for its vulnerabilities.) Web Spider: For gathering web application hacking information. Email scraper: To get all emails related to a webpage IMDB Rating: Easy way to access the movie database. Both .exe(compressed as zip) and .py
most recent commit 2 years ago
The 10 Most Depended On Python Scraper Web Crawler Open Source Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Google Search Results Python ⭐ 432
Google Search Results via SERP API pip Python Package
dependent packages 99total releases 41latest release March 10, 2023most recent commit 7 months agopypi google-search-results} Downloads
Trafilatura ⭐ 2,447
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39latest release November 29, 2023most recent commit 3 months agopypi trafilatura} Downloads
Gazpacho ⭐ 716
🥫 The simple, fast, and modern web scraping library
dependent packages 8total releases 19latest release October 09, 2020most recent commit 5 months agopypi gazpacho} Downloads
Requests Random User Agent ⭐ 73
Configures the requests library to randomly select a desktop User-Agent
dependent packages 5total releases 48latest release January 14, 2023most recent commit 6 months agopypi requests-random-user-agent} Downloads
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Scrapy Wayback Machine ⭐ 70
A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
dependent packages 4total releases 3latest release February 15, 2021most recent commit 3 years agopypi scrapy-wayback-machine} Downloads
Grab ⭐ 2,292
Web Scraping Framework
dependent packages 3total releases 120latest release June 24, 2018most recent commit 10 months agopypi grab} Downloads
Selectorlib ⭐ 55
A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them
dependent packages 3total releases 14latest release January 08, 2020most recent commit a year agopypi selectorlib} Downloads
The 10 Latest Releases In Python Scraper Web Crawler Open Source Projects
Trafilatura ⭐ 2,447
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39latest release November 29, 2023most recent commit 3 months agopypi trafilatura} Downloads
Mov Cli ⭐ 314
A cli tool to browse and watch Movies/Shows/TV/Sports.
dependent packages 1total releases 103latest release November 25, 2023most recent commit 3 months agopypi mov-cli} Downloads
Kenpompy ⭐ 61
A simple yet comprehensive web scraper for kenpom.com.
total releases 9latest release November 22, 2023most recent commit 5 months ago
Dputils ⭐ 6
open source library from digipodium
total releases 29latest release November 22, 2023most recent commit 5 months ago
Web Poet ⭐ 85
Web scraping Page Objects core library
dependent packages 4total releases 23latest release November 21, 2023most recent commit 3 months agopypi web-poet} Downloads
Basketball_reference_web_scraper ⭐ 382
NBA Stats API via Basketball Reference
total releases 39latest release November 14, 2023most recent commit 3 months ago
Hockey Scraper ⭐ 134
Python Package for scraping NHL Play-by-Play and Shift data
total releases 55latest release October 30, 2023most recent commit 4 months ago
Popular Programming Languages Categories
Python
Javascript
Java
Typescript
C Plus Plus
Php
Golang
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.