Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Python Web Crawler Open Source Projects

Open source projects categorized as Python Web Crawler
Scrapy ⭐ 49,480
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96most recent commit 12 hours agopypi Scrapy} Downloads
Changedetection.io ⭐ 13,361
The best and simplest free open source website change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
most recent commit a day ago
Autoscraper ⭐ 5,159
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
dependent packages 1total releases 16most recent commit 7 months agopypi autoscraper} Downloads
Douyin_tiktok_download_api ⭐ 4,844
🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、T
total releases 21most recent commit 6 days ago
Helium ⭐ 3,707
Lighter web automation for Python
most recent commit 10 days ago
Automatic Udemy Course Enroller Get Paid Udemy Courses For Free ⭐ 2,940
Do you want to LEARN NEW STUFF for FREE? Don't worry, with the power of web-scraping and automation, this script will find the necessary Udemy coupons & enroll you for PAID UDEMY COURSES, ABSOLUTELY FREE!
total releases 11most recent commit 4 months ago
Snoop ⭐ 2,397
Snoop — инструмент разведки на основе открытых данных (OSINT world)
most recent commit 15 hours ago
Grab ⭐ 2,292
Web Scraping Framework
dependent packages 3total releases 120most recent commit 5 months agopypi grab} Downloads
Trafilatura ⭐ 2,240
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39most recent commit 6 days agopypi trafilatura} Downloads
30 Days Of Python ⭐ 1,926
Learn Python for the next 30 (or so) Days.
most recent commit a year ago
Pythoncode Tutorials ⭐ 1,893
The Python Code Tutorials
most recent commit 16 hours ago
Pspider ⭐ 1,675
简单易用的Python爬虫框架，QQ交流群：597510560
most recent commit a year ago
Dat8 ⭐ 1,549
General Assembly's 2015 Data Science course in Washington, DC
most recent commit a year ago
Webscraping From 0 To Hero ⭐ 1,305
The web scraping open project repository aims to share knowledge and experiences about web scraping with Python
most recent commit 5 months ago
100projectsofcode ⭐ 1,293
A list of practical knowledge-building projects.
most recent commit 3 years ago
Scrapeghost ⭐ 1,283
👻 Experimental library for scraping websites using OpenAI's GPT API.
most recent commit 13 days ago
Requests Cache ⭐ 1,187
Persistent HTTP cache for python requests
dependent packages 466total releases 90most recent commit 4 days agopypi requests-cache} Downloads
Lightnovel Crawler ⭐ 1,134
Generate and download e-books from online sources.
dependent packages 1total releases 30most recent commit 8 days agopypi ebook-crawler} Downloads
Django Dynamic Scraper ⭐ 1,069
Creating Scrapy scrapers via the Django admin interface
total releases 65most recent commit 2 years ago
Faster Than Requests ⭐ 1,045
Faster requests on Python 3
dependent packages 4total releases 18most recent commit 4 days agopypi faster-than-requests} Downloads
Crosslinked ⭐ 964
LinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping
total releases 3most recent commit 19 days ago
Youtube_tutorials ⭐ 889
Collection of scripts corresponding to LucidProgramming YouTube tutorials
most recent commit a year ago
Selectolax ⭐ 887
Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors).
dependent packages 42total releases 44most recent commit a month agopypi selectolax} Downloads
The 10 Most Depended On Python Web Crawler Open Source Projects
Requests Cache ⭐ 1,187
Persistent HTTP cache for python requests
dependent packages 466total releases 90latest release November 19, 2023most recent commit 4 days agopypi requests-cache} Downloads
Scrapy ⭐ 49,480
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 12 hours agopypi Scrapy} Downloads
Google Search Results Python ⭐ 432
Google Search Results via SERP API pip Python Package
dependent packages 99total releases 41latest release March 10, 2023most recent commit 2 months agopypi google-search-results} Downloads
Trafilatura ⭐ 2,240
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39latest release November 29, 2023most recent commit 6 days agopypi trafilatura} Downloads
Selectolax ⭐ 887
Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors).
dependent packages 42total releases 44latest release October 17, 2023most recent commit a month agopypi selectolax} Downloads
Curl_cffi ⭐ 821
Python binding for curl-impersonate via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
dependent packages 27total releases 34latest release November 25, 2023most recent commit 3 days agopypi curl-cffi} Downloads
Gazpacho ⭐ 717
🥫 The simple, fast, and modern web scraping library
dependent packages 8total releases 19latest release October 09, 2020most recent commit a day agopypi gazpacho} Downloads
Scrapy Fake Useragent ⭐ 654
Random User-Agent middleware based on fake-useragent
dependent packages 5total releases 12latest release August 15, 2020most recent commit 3 months agopypi scrapy-fake-useragent} Downloads
Faster Than Requests ⭐ 1,045
Faster requests on Python 3
dependent packages 4total releases 18latest release February 24, 2021most recent commit 4 days agopypi faster-than-requests} Downloads
Grab ⭐ 2,292
Web Scraping Framework
dependent packages 3total releases 120latest release June 24, 2018most recent commit 5 months agopypi grab} Downloads
The 10 Latest Releases In Python Web Crawler Open Source Projects
Trafilatura ⭐ 2,240
Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments
dependent packages 66total releases 39latest release November 29, 2023most recent commit 6 days agopypi trafilatura} Downloads
Mov Cli ⭐ 293
A cli tool to browse and watch Movies/Shows/TV/Sports.
dependent packages 1total releases 103latest release November 25, 2023most recent commit 8 days agopypi mov-cli} Downloads
Curl_cffi ⭐ 821
Python binding for curl-impersonate via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
dependent packages 27total releases 34latest release November 25, 2023most recent commit 3 days agopypi curl-cffi} Downloads
Requests Cache ⭐ 1,187
Persistent HTTP cache for python requests
dependent packages 466total releases 90latest release November 19, 2023most recent commit 4 days agopypi requests-cache} Downloads
Basketball_reference_web_scraper ⭐ 373
NBA Stats API via Basketball Reference
total releases 39latest release November 14, 2023most recent commit a day ago
Selectolax ⭐ 887
Python binding to Modest and Lexbor engines (fast HTML5 parser with CSS selectors).
dependent packages 42total releases 44latest release October 17, 2023most recent commit a month agopypi selectolax} Downloads
Twitter Scraper Selenium ⭐ 207
Python's package to scrap Twitter's front-end easily
total releases 27latest release October 02, 2023most recent commit 2 months ago
Popular Programming Languages Categories
Javascript
Python
Java
Typescript
Php
C Plus Plus
Ruby
C
C Sharp
Shell
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2023 Awesome Open Source.  All rights reserved.