Awesome Open Source

Programming Languages

Search results for python scrape

636 search results found

You Get ⭐ 48,778

⏬ Dumb downloader that scrapes the web

Twint ⭐ 15,469

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

Portia ⭐ 8,982

Visual scraping for Scrapy

Autoscraper ⭐ 5,159

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Facebook Page Post Scraper ⭐ 2,014

Data scraper for Facebook Pages, and also code accompanying the blog post How to Scrape Data From Facebook Page Posts for Statistical Analysis

Facebook Scraper ⭐ 1,936

Scrape Facebook public pages without an API key

Twitterscraper ⭐ 1,852

Scrape Twitter for Tweets

Email2phonenumber ⭐ 1,601

A OSINT tool to obtain a target's phone number just by having his email address

Linkedin_scraper ⭐ 1,534

A library that scrapes Linkedin for user data

Jobfunnel ⭐ 1,533

Scrape job websites into a single spreadsheet with no duplicates.

Twitter Api Client ⭐ 1,149

Implementation of X/Twitter v1, v2, and GraphQL APIs

Redditdownloader ⭐ 1,045

Scrapes Reddit to download media of your choice.

Onionsearch ⭐ 989

OnionSearch is a script that scrapes urls on different .onion search engines.

Mlscraper ⭐ 935

🤖 Scrape data from HTML websites automatically by just providing examples

A simple and unlimited twitter scraper : scrape tweets, likes, retweets, following, followers, user info, images...

Scrapedin ⭐ 712

A tool to scrape LinkedIn without API restrictions for data reconnaissance

Python_and_the_web ⭐ 662

Build Bots, Scrape a website or use an API to solve a problem.

Universal Reddit Scraper - A comprehensive Reddit scraping command-line tool.

Imagescraper ⭐ 572

✂️ High performance, multi-threaded image scraper

Automated_youtube_channel ⭐ 564

Automated youtube that can scrape content, edit a compilation, and upload to youtube daily.

Audiobooks.bundle ⭐ 554

Plex metadata scraper for Audiobooks

Instascrape ⭐ 554

Powerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically

Scrapli ⭐ 537

Fast, flexible, sync/async, Python 3.7+ screen scraping client specifically for network devices

Openwebtext ⭐ 463

Open clone of OpenAI's unreleased WebText dataset scraper. This version uses pushshift.io files instead of the API for speed.

Scrape Linkedin Selenium ⭐ 353

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

City Scrapers ⭐ 315

Scrape, standardize and share public meetings from local government websites

Advanced python library to scrap Twitter (tweets, users) from unofficial API

Python web scraping framework

Recreation Gov Campsite Checker ⭐ 294

Scrapes the recreation.gov website to check for campsite availabilities 🏕🏕

Python Automation Scripts ⭐ 264

Simple yet powerful automation stuffs.

Proxy Scraper ⭐ 260

scrape proxies from more than 5 different sources and check which ones are still alive

Yahoo_fin ⭐ 251

Scrape stock price history from new (Spring 2017) Yahoo Finance layout

Openparliament ⭐ 246

Keeping tabs on Canada's Parliament

Spatula ⭐ 233

A modern Python library for writing maintainable web scrapers.

Nudecrawler ⭐ 231

Crawl telegra.ph searching for nudes!

Proxyscrape ⭐ 224

Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).

Sota Extractor ⭐ 221

The SOTA extractor pipeline

Rightmove_webscraper.py ⭐ 219

Python class to scrape data from rightmove.co.uk and return listings in a pandas DataFrame object

Wayback Machine Scraper ⭐ 219

A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

Amazon Scraper ⭐ 219

A simple web scraper to extract Product Data and Pricing from Amazon

Transistor ⭐ 209

Transistor, a Python web scraping framework for intelligent use cases.

Ha Multiscrape ⭐ 198

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

Letterboxd_recommendations ⭐ 190

Scraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username

Portia Dashboard ⭐ 190

portia-dashboard is a visual web crawler based on scrapinghub/portia

Scrapyz ⭐ 188

"Scrape Easy" - an extension of the Scrapy framework.

Scrapy Samples ⭐ 183

Scrapy examples crawling Craigslist

Instagram Follower Scraper ⭐ 181

Scrapes all the data of followers of any instagram account

Atp World Tour Tennis Data ⭐ 162

Using Python to scrape ATP World Tour tennis data

Hncrawl ⭐ 150

A scrapy-based Hacker News crawler.

Media Scraper ⭐ 150

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

Scrapy Webdriver ⭐ 147

News Fetch ⭐ 140

A Python Package which helps to scrape all news details from any news websites

a command-line web scraping tool

Hockey Scraper ⭐ 134

Python Package for scraping NHL Play-by-Play and Shift data

Scrape Linkedin ⭐ 132

Scrape a public LinkedIn profile.

Pentest_scripts ⭐ 129

penetration testing scripts

Paperscraper ⭐ 128

Tools to scrape publication metadata from pubmed, arxiv, medrxiv and chemrxiv.

Dorkscanner ⭐ 125

A typical search engine dork scanner scrapes search engines with dorks that you provide in order to find vulnerable URLs.

Ao3scraper ⭐ 123

A Python scraper for getting fan fiction content and metadata from Archive of Our Own.

Gsoc Organisation Scraper ⭐ 120

Scrape GSoC organisations using a single script.

Instagram Python Scraper ⭐ 116

A instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!

Geeksforgeeksscrapper ⭐ 116

Scrapes g4g and creates PDF

Twitter Scrape ⭐ 113

Download streaming tweets that match specific keywords, and dump the results to a file.

Jimutmap ⭐ 112

API to get enormous amount of high resolution satellite images from satellites.pro quickly through multi-threading! create map your own map dataset. Bringing data to Humans.

Linkedinscraper ⭐ 112

Scrapes public information off of LinkedIn

Giggity ⭐ 110

Wraps github api for openly available information about an organization, user, or repo

Pastepwn ⭐ 109

Python framework to scrape Pastebin pastes and analyze them

Fb_scrape_public ⭐ 107

Scrapes posts and comments from public Facebook pages.

Scraper ⭐ 104

Web scraper for scraping, tracking and visualizing prices of products on various websites.

Gesetze Tools ⭐ 103

Scripts to maintain German law git repository

Big Cases ⭐ 99

The basic code behind the @big_cases Twitter bot

Vsco Scraper ⭐ 96

Easily allows for scraping a VSCO

Scrapemark ⭐ 94

Super-convenient web scraping in Python

Snapchat Map Scraper ⭐ 92

Facebook Post Scraper ⭐ 89

Facebook Post Scraper 🕵️🖱️

Funda Scraper ⭐ 88

Scraper of the Dutch real estate website www.funda.nl, implemented in Python with Scrapy

Glassdoor Review Scraper ⭐ 88

Scrape reviews from Glassdoor

Twitterscraper ⭐ 86

Scrape a User's Twitter data! Bypass the 3,200 tweet API limit for a User!

Instagramy ⭐ 86

Python Package for Instagram Users, Posts, and Hashtag data.

Twitter Word Cloud ⭐ 84

Twitter word cloud generator

News Aggregator ⭐ 84

Django project to scrape a news website using Beautiful soup and display in our template.

Highanonproxypool ⭐ 84

A proxy pool that scrapes free anonymous proxies and maintains its proxies' availability.

Search Google ⭐ 84

Scrape google search results

Trending Youtube Scraper ⭐ 83

Python script that scrapes the currently trending YouTube videos in a variety of countries

Linkedin Jobs Scraper ⭐ 82

Scrape LinkedIn job postings using Selenium WebDriver with python bindings

Tableau Scraping ⭐ 79

Tableau scraper python library. R and Python scripts to scrape data from Tableau viz

Ca Property Tax ⭐ 78

CA property tax visualization

Goodwe Sems Home Assistant ⭐ 76

Sensor for Home Assistant pulling data from the GoodWe SEMS API for solar panel production metrics.

Web Scraping ⭐ 76

Learn how to leverage Python's amazing tools to scrape data from other websites. The end goal of this course is to scrape blogs to analyze trending keywords and phrases. We'll be using Python 3.6, Requests, BeautifulSoup, Asyncio, Pandas, Numpy, and more!

Cs109 Dl Videos ⭐ 74

Shell script to scrape Harvard CS109 (Intro to Data Science) lecture videos

Yahoo Earnings Calendar ⭐ 73

Scrapes Yahoo! Finance earnings calendar to get data for a specific date or a date range.

Emojitaco ⭐ 72

Alfred Emoji Script with Taco and other unicode 9 emoji

Pbpstats ⭐ 72

A package to scrape and parse NBA, WNBA and G-League play-by-play data.

Tool to scrape LinkedIn

Blinkist Scraper ⭐ 71

📚 Python tool to download book summaries and audio from Blinkist.com, and generate some pretty output

Scrapy Wayback Machine ⭐ 70

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

Bhadoo Index Scraper ⭐ 69

script to scrape direct download links (ddls) from google drive index.

Linkedin Scraper ⭐ 69

Tool to scrape linkedin

Tripadvisor Scraper ⭐ 68

TripAdvisor scraper

Related Searches

Python Django (28,897)

Python Machine Learning (20,195)

Python Flask (17,643)

Python Dataset (14,792)

Python Docker (13,757)

Python Tensorflow (13,736)

Python Deep Learning (13,092)

Python Jupyter Notebook (12,976)

Python Network (11,495)

Python Algorithms (10,033)

1-100 of 636 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.