Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
The Top 10 Command Line Web Crawler Open Source Projects
Open source projects categorized as Command Line Web Crawler
Categories
>
Command Line Interface
>
Command Line
Edit Category
projectdiscovery/katana
⭐
7,995
A next-generation crawling and spidering framework.
dependent packages
0
total releases
0
most recent commit
over 2 years ago
pystardust/ani-cli
⭐
5,724
A cli tool to browse and play anime
dependent packages
0
total releases
0
most recent commit
over 2 years ago
gsh199449/spider
⭐
907
A configurable web spider with a easy-to-use web console
dependent packages
0
total releases
0
most recent commit
almost 8 years ago
PuerkitoBio/fetchbot
⭐
758
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
dependent packages
0
total releases
0
most recent commit
about 5 years ago
benibela/xidel
⭐
611
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
dependent packages
0
total releases
0
most recent commit
over 2 years ago
mov-cli/mov-cli
⭐
314
A cli tool to browse and watch Movies/Shows/TV/Sports.
dependent packages
0
total releases
0
most recent commit
over 2 years ago
s0rg/crawley
⭐
208
The unix-way web crawler
dependent packages
0
total releases
0
most recent commit
over 2 years ago
DedSecInside/gotor
⭐
146
This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.
dependent packages
0
total releases
0
most recent commit
over 2 years ago
s32x/anirip
⭐
144
:clapper: A Crunchyroll show/season ripper
dependent packages
0
total releases
0
most recent commit
over 4 years ago
suntong/cascadia
⭐
128
Go cascadia package command line CSS selector
dependent packages
0
total releases
0
most recent commit
almost 3 years ago
Get A Weekly Email With Trending Command Line Projects
No Spam. Unsubscribe easily at any time.
Command Line
Web Crawler
Subscribe
Javascript must be enabled to subscribe.
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2026 Awesome Open Source. All rights reserved.
