Awesome Open Source

Programming Languages

Search results for database scraper

57 search results found

Avbook ⭐ 8,777

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Cinemagoer ⭐ 1,156

Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies

Fansly Downloader ⭐ 1,103

Easy to use fansly.com content downloading tool. Written in python, but ships as a standalone Executable App for Windows too. Enjoy your Fansly content offline anytime, anywhere in the highest possible content resolution! Fully customizable to download in bulk or single: photos, videos & audio from timeline, messages, collection & specific posts 👍

Unofficial API for finviz.com

Mstables ⭐ 158

MorningStar.com scraper that consolidates tens of thousands of financial records into a SQLite relational database. Class 'dataframes' easily converts the SQLite data into pandas DataFrames (see Jupyter notebook for examples)

The Scripting Engine that Combines Speed, Safety, and Simplicity

Web Database Analytics ⭐ 144

Web scrapping and related analytics using Python tools

Od Database ⭐ 113

Distributed crawler, database and web frontend for public directories indexing

FP Scala Back-end of the Smart Backpacker App

Scrapes sites. Gets news. Eventually events.

Vinted Scraper ⭐ 63

This is a tool to scrape/download images and data from Vinted & Depop using the API and stores the data in a SQLite database.

Xa Scraper ⭐ 62

Nbascrape ⭐ 55

Scraper Boilerplate ⭐ 54

Awesome Georgian Datasets ⭐ 46

Useful datasets, specific to Georgia

Fec Scraper ⭐ 46

Scripts to scrape the FEC website and parse campaign filings

Funko Pop Data ⭐ 42

Open-source database of all Funko Pop data.

Fifa Fut Data ⭐ 39

Web-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB

Facebookcrawler ⭐ 39

Facebook Crawler - Crawl information from facebook

Django Kickstarter Scraper ⭐ 36

Scrapes kickstarter and stores to a mySQL database using django.

Myanimelist Data Set Creator ⭐ 35

Collection of some simple python scripts to create https://myanimelist.net/ anime and user data set.

Scrapers ⭐ 34

scrapers for building your own image databases

Foundation ⭐ 31

🧱 A uniform template to use as a foundation for Puppeteer bot construction.

Social_scrapers ⭐ 28

Various Python scripts to scrape sites that store data about you.

Proxy Db ⭐ 23

Manage free and private proxies on local db for Python Projects

Regenesis ⭐ 22

Scraper for the GENESIS statistical database

Scraping Manual Assistant ⭐ 22

Scraping Manual Assistant - A scraper for game manuals

Exoskeleton ⭐ 22

A Python framework to build polite, but tenacious crawlers / scrapers with a MariaDB backend

Flight Scrappper ⭐ 21

Web scraper made with nodejs and selenium-webdriver that gathers flight data and stores it in a mongodb database.

Php Movie Database ⭐ 21

A PHP movie database that uses a imdB scraper

Crossfitrankings ⭐ 21

Scraping and analysis of the crossfit open scores and ranking system

Savantscraper ⭐ 21

Using Python to add Baseball Savant data to a SQLite database

Scraping data from the web and loading it into a graph database

🐍 A tool to export, save, and analyze your Garmin Connect data

Tagalog Dictionary Scraper ⭐ 19

Builds a Tagalog dictionary by collecting Tagalog words from tagalog.pinoydictionary.com

Jedeschule Scraper ⭐ 19

Falocalrepo ⭐ 18

Pure Python program to download submissions, journals, and user folders from the FurAffinity forum in an easily handled database.

Mangascrap ⭐ 18

MangaScrap is a Ruby based scrapper that generates a local website with the downloaded mangas

An intelligent generator of well-balanced meals.

Reddit Scraper ⭐ 15

A scraper for www.reddit.com written in Python

Fbscraper ⭐ 15

Google Seo Allintitle Scraper ⭐ 15

Google SEO scraper for "allintitle:keyword" queries.

Python scripts for scraping various data from Newegg.com and storing it in an SQLite database.

Open Carnatic Music Database

cpcbccr data scraping

Calpheonjs ⭐ 13

Calpheon.js is an interface to retrieve data about the BDO game by scraping BDOCodex.

Gojek Scraper ⭐ 13

[OLD] scrapin' gojek database 🏍 🤔

Football Data Scraper ⭐ 13

Scrape publicly available football data and store in a MySQL database

Mtqinfrascraper ⭐ 13

Transports Quebec Infrastructure Database Scraper

Vancouver Parking Tickets ⭐ 12

A scraper for the Vancouver Sun's parking ticket database

Myschool ⭐ 12

Scraper, parser, backed database and data visualisation of the www.myschool.edu.au data.

Prntscr Scraper ⭐ 12

A web scraper designed to link prntscr.com URLs with their associated Imgur images, and archive them.

Game Database Info ⭐ 11

Game database info to be used in creation of xml files or kodi scraper files

Tvdb Scraper ⭐ 10

Scrapes TV metadata from thetvdb

Cazy_webscraper ⭐ 10

Web scraper to retrieve protein data catalogued by the CAZy, UniProt, NCBI, GTDB and PDB websites/databases.

Investable ⭐ 10

A web app used to research potential rental real estate investments.

Math Genealogy Scraper ⭐ 10

Code for scraping (and a mirror of) the Math Genealogy Database

oclean is a bio inspired solution to help clean up the ocean

Python Scraping To Database Sample ⭐ 9

Some sample code examples in Python to scrape data from the web and saving it to a database.

Fantasy Football Database ⭐ 9

Fantasy football data scraping and projections.

A Collection of Pokedex data and tools

Pywebcash ⭐ 8

Cloud-hosted personal finance tracking

Dbs Scraper ⭐ 8

DBS Card Game Scraper & Deck Builder

Simple Web Scraper ⭐ 8

Simple web scraper to get player data using beatiful-soup4 and PostgreSQL as a database. SQLAlchemy as an ORM

Offerings Huntsman ⭐ 8

Offerings Huntsman - API for Bilkent Offerings

Twitter Scraper ⭐ 7

Python script to scrape Twitter for data based on hashtags or mentions.

Nba Topshot Scraper ⭐ 7

Node script that will use Selenium to scrape card information from NBA Topshot including card names, rarity, and lowest cost at the moment. Data is scraped once per day.

A little whois scraper service

March Madness prediction

Kimurai Dashboard ⭐ 7

Simple dashboard for Kimurai web scraping framework

Simplified Search Engine ⭐ 7

Multithreaded Web Crawler, Scraper, Indexer

Hltv Csgolounge Scraper ⭐ 7

A scraper for hltv.org and csgolounge.com.

Nepali News Portal Kbd ⭐ 7

Online webapp that scrapes news from different new portals of Nepal and worldwide. Hosted at heroku.

Permaculture ⭐ 7

Permaculture design app built on scraped plant databases. Drag-n-drop GUI with detailed design plan generator.

C++ project for scraping from Reddit

CredShed - a scalable database for credential leaks. Written in Python, it can scrape Pastebin and easily ingest poorly-formatted files or entire directories into a searchable database. MongoDB is used in the backend.

Otomoto Scraper ⭐ 6

🚗 Simple scraper for otomoto with interface and database.

Waseda Syllabus Scraper ⭐ 6

A web scraper for scraping the syllabus search database at Waseda University.

Myfitnesspal ⭐ 6

A python script for scraping the diary data from MyFitnessPal and putting it into a SQLite database

Api de datos de contribuyentes del Paraguay

A python tool for scraping okcupid data

Binbaz_database ⭐ 6

مكشطة وقاعدة بيانات لـ موقع سماحة الشيخ الإمام ابن باز رحمه الله

BidWire notifies you when it finds new activity on the sites you care about

Imdb_pyscraper ⭐ 5

python program that scraps imdb.com and stores data into mysql db

Coronavirusdatabase ⭐ 5

I have used web scraping to collect the data and stored it into a .json file and further using the .json file to add to SQLite Database and also trying to make an API using Flask.

Opensource Tracker ⭐ 5

Opensource scraper for analyse of social networks. Create nodes with egdes for you to visualize on editors like gephi.

Sgshowtimesapi ⭐ 5

Scrapes Singapore movie showtimes

Dns Scraper ⭐ 5

Tools for scanning DNS RRs

Webscraping_workshop ⭐ 5

Materials for my workshop "Web Scraping with Scrapy and MongoDB running on Docker" at PyLadies, Berlin, March 31, 2019.

Google Places Scraper ⭐ 5

Scraper for Google Places website

Toronto City Hall Api ⭐ 5

Powering municipal apps in Toronto.

I don't like house hunting. I'd rather write a tool instead.

Nightmare Firebase Web Scraper ⭐ 5

A browser automation script with nightmare.js that navigates to a page and saves some information to a firebase database.

Govtrack Scrapers ⭐ 5

This is a mirror of the GovTrack.us svn legislative data scrapers

Related Searches

Command Line Database (33,932)

Javascript Database (9,210)

Python Database (7,277)

Php Database (5,990)

Java Database (5,934)

Database Postgresql (5,359)

Mysql Database (4,382)

Database Sql (3,816)

Python Scraper (3,513)

Docker Database (3,147)

1-57 of 57 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.