Awesome Open Source

Programming Languages

Search results for crawler wget

10 search results found

Awesome Datahoarding ⭐ 892

List of data-hoarding related tools

Archivebot ⭐ 328

ArchiveBot, an IRC bot for archiving websites

Bitextor ⭐ 260

Bitextor generates translation memories from multilingual websites

Google Group Crawler ⭐ 213

[Deprecated] Get (almost) original messages from google group archives. Your data is yours.

Authority Data ⭐ 83

官方权威数据：统计年签，统计公报，互联网行业报告，工信部数据，ICT报告等 Official authoritative data (Chinese)

Fetchurls ⭐ 79

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

Wget Lua ⭐ 72

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

wmirror allows you to download any website from the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from the server to your computer.

Metagoofeel ⭐ 10

Web crawler and downloader based on GNU Wget.

Pywebquery ⭐ 10

a jquery liked pythonic web crawler library ,it's based on BeautifulSoup and wget

Crawling Emails ⭐ 8

Very simple bash script to crawl email addresses from a specific website.

Spidertaobao ⭐ 5

spider taobao suggest

Related Searches

Python Crawler (4,545)

Javascript Crawler (1,142)

Crawler Scrapy (988)

Scraper Crawler (863)

Java Crawler (807)

Crawler Spider (709)

Shell Wget (536)

Python Wget (395)

1-10 of 10 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.