Awesome Open Source

Programming Languages

Search results for command line scraper

21 search results found

Ferret ⭐ 5,540

Declarative web scraping

Awesome Puppeteer ⭐ 2,245

A curated list of awesome puppeteer resources.

Scanless ⭐ 1,061

online port scan scraper

📖 The most advanced (yet simple) cli manga downloader in the entire universe! Lua scrapers, export formats, anilist integration, fancy TUI and more!

Episode Rename ⭐ 699

电视剧/番剧自动化重命名工具, 一键批量改名. 可配合QBittorrent下载后自动重命名, 方便Emby自动刮削. 支持Windows, Linux, MacOS, Docker 和群晖套件环境运行

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.

Universal Reddit Scraper - A comprehensive Reddit scraping command-line tool.

Imagescraper ⭐ 572

✂️ High performance, multi-threaded image scraper

Pdf.tocgen ⭐ 444

A CLI toolset to generate table of contents for PDF files automatically.

Search Engine Parser ⭐ 416

Lightweight package to query popular search engines and scrape for result titles, links and descriptions

Mov Cli ⭐ 314

A cli tool to browse and watch Movies/Shows/TV/Sports.

Lightnovel_epub ⭐ 233

🍭 epub generator for (light)novels (轻)小说 epub 生成器，支持站点：轻之国度、轻小说文库

Tiktok Hashtag Analysis ⭐ 229

Provides tools to analyze hashtags within posts scraped from TikTok.

Torrodle ⭐ 198

⚡️ A magnet link scraper for streaming videos (movies, tv shows, anime, porn) along with subtitles.

Anime Dl ⭐ 196

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

Anipy Cli ⭐ 191

Little tool in python to watch and download anime from the terminal (the better way to watch anime). Also applicable as an API

GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)

Scrape Twitter ⭐ 155

🐦 Access Twitter data without an API key. [DEPRECATED]

Site Audit Seo ⭐ 151

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

Instascrape ⭐ 145

🚀 A fast and lightweight utility and Python library for downloading posts, stories, and highlights from Instagram.

Scrape the web in the eink era. Convert websites into ebooks and markdown.

CLI utility to scrape emails from websites

Spyscrap ⭐ 124

CLI and GUI for OSINT. Are you very exhibited on the Internet? Check it! Twitter, Tinder, Facebook, Google, Yandex, BOE. It uses facial recognition to provide more accurate results.

Xword Dl ⭐ 121

⬛⬜⬛ Command line tool to scrape crosswords from online solvers and save them as .puz files ⬛⬜⬛

Interactive CLI Web Crawler

Headlesschrome ⭐ 112

A Go package for working with headless Chrome. Run interactive JavaScript commands on web pages with Go and Chrome.

comic updater, mangafied

Reddit Post Scraping Tool ⭐ 72

Given a subreddit name and a keyword, this program returns all top (by default) posts that contain the specified keyword.

Configurable Scraper & Downloader, Powered by RegExp and Go

Bibtex Autocomplete ⭐ 61

Python package to autocomplete bibtex bibliographies

Wayback Google Analytics ⭐ 60

A lightweight tool for scraping current and historic Google Analytics data

Spam Bot 3000 ⭐ 59

Social media research and promotion, semi-autonomous CLI bot

W3crapcli ⭐ 59

A collection of CLI tools to deal with the web

Perform lexical analysis on words, one word at a time.

Kaa.si Cli ⭐ 53

Stream anime from kaa.si and sync with anilist

Pitchfork Npm ⭐ 50

An Unofficial Pitchfork Music API client for Node.js

Web Scraping Framework

Wishlist ⭐ 46

Read an Amazon wishlist programmatically with Python

Jw Wpcli Shortcode Scraper ⭐ 41

A WP-CLI script to scrape all blog posts for shortcodes ( registered or not ) and provides you a report after complete with CSV capability.

Youtube Unofficial ⭐ 40

Access parts of your account unavailable through normal YouTube API access.

Ronin Web ⭐ 40

ronin-web is a collection of useful web helper methods and commands.

The most inefficient Manga downloader for PC

Vk Url Scraper ⭐ 35

Scrape VK URLs to fetch info and media - python API or command line tool.

Readability Cli ⭐ 35

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

Trainline Python ⭐ 33

Non-official Python wrapper and CLI tool for Trainline

Fulldom Server ⭐ 31

Proxy-like server that will show you the DOM of a page after JS runs

A pluggable command line based scraper with REPL support.

📉 Uptime and error monitoring CLI

Linkedin_scraper ⭐ 29

🙋 A Selenium based automated program that scrapes profiles data,stores in CSV,follows them and saves their profile in PDF.

A cli tool to streamline financial markets data analysis 🔧

Search your favorite PS4 games from Playstation Store using the Command Line

Go Wenku Downloader ⭐ 24

Go轻小说文库下载工具

Emec Api ⭐ 21

API Python para consulta na base de dados oficial do e-MEC

Jonyfive ⭐ 21

CLI tool for doing useful things after scraping WWDC session metadata

Spyscraper ⭐ 19

Osint tool to scraper websites

Chan Downloader ⭐ 18

CLI to download all images/webms in a 4chan thread

Pydarkstar ⭐ 18

A python module for interacting with a darkstar server.

Ethz Document Fetcher ⭐ 18

Fetches files from ethz websites

Go API for Finviz

Mangareaderscraper ⭐ 17

Search and download mangas from the command line

Zippydamn ⭐ 15

Unofficial Zippyshare CLI tools (download and search)

Goscholar ⭐ 15

Google scholar scraper written in Go

Reddit Post Exporter ⭐ 15

Export desired amount of posts from specified subreddit and category/sort without any API wrappers

Libgenbot ⭐ 13

A Python Scraper Designed to Scrape PDFs from Libgen and Scihub

Markify is an open source command line application written in python which scrapes data from your social media accounts and utilises markov chains to generate new sentences based on the scraped data

Airscraper ⭐ 12

Airtable backup script package

Command-line utility to convert HTML tables into CSV files

9anime Scraper ⭐ 11

A node cli tool for scraping and viewing episodes hosted on 9anime

CPTM and Metro lines status CLI

Scrape It Cli ⭐ 11

🔮 CLI for scrape-it. A Node.js scraper for humans. 🚀

Vocabulary Scrapper ⭐ 10

This is a django app for getting the meaning of an English word from command line.

Meta Spy ⭐ 10

👾 CLI MetaSpy (Facebook, Instagram) scraper and crawler - instagram account, facebook accounts, pages and search

Auto Registrar ⭐ 9

A CLI script to check/register courses from KFUPM Banner 9.

Cli Scrape ⭐ 9

Do simple scraping from the command line using XPath or CSS Selectors!

Github Trending ⭐ 9

Command line tool for fetching GitHub trending repositories

Declarative Web Scraping

Linkscraper ⭐ 8

A powerful tool to scrape and manage links from web pages.

Letterboxd Downloader ⭐ 8

Exports letterboxd lists as csv file.

Let your CLI find available solutions for errors / exceptions online on commands you hit, for you, no need open a Browser. and find something yourself

CLI for scraping a web page to create a Spotify playlist

Sinonimos ⭐ 8

🇧🇷 Find synonyms without leaving your terminal

🇯🇵📰🗻 NHK News Web (Easy) word frequency (core list) scraper for Japanese language learners.

Scraper of Nicorepo on nicovideo.jp

Fetch new releases from http://www.juno.co.uk/.

Webnovelyoinker ⭐ 7

Downloads converts webnovels to Epub or PDF. Supported websites: wuxiaworld.com, crimsonmagic.me

Opensourcewatch ⭐ 7

Monolithic repository containing the OSW app, scrapers, and control CLI.

Instascraper ⭐ 7

📊 A web scraping CLI tool for Instagram profiles.

Comics Scraper ⭐ 7

Command line tool to download comics in pdf from a website

Cli Data Gem Assessment ⭐ 7

Kotlin Headlessbrowser Selenium Jsoup Parser ⭐ 7

kotlin製のスクレイパーです

Social media web scraper CLI for post file downloads.

Urasunday ⭐ 7

Your basic urasunday(https://urasunday.com) scraper

Nelson Pdf Downloader ⭐ 6

Downloads PDFs from the myNelson interface.

🌊 CLI to fetch and parse buoy data

Indeed Job Scraper ⭐ 6

A command-line job scraper which scrapes Indeed for particular jobs and exports them to a spreadsheet

Npm License Scraper ⭐ 6

📜 Dead simple license scraper for npm packages (with zero real dependencies)

Freshproxies ⭐ 6

Fresh Proxies | Proxied Browser | HTTP/HTTPS/SOCKS4/SOCKS5

▶️ A Go library (and command-line utility) for fetching YouTube videos.

Get Torrents Cli ⭐ 6

Node CLI Web scraper that extracts magnetic links from Google or from the informed URL

Related Searches

Ruby Command Line (35,913)

Command Line Database (33,932)

Javascript Command Line (22,285)

Typescript Command Line (22,062)

Command Line Testing (20,826)

Command Line Angular (20,741)

Golang Command Line (8,406)

Python Scraper (3,513)

Java Command Line (3,348)

Command Line Terminal (3,339)

1-21 of 21 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.