Awesome Open Source

Programming Languages

Search results for rust crawler

24 search results found

Voyager ⭐ 618

crawl and scrape web pages in rust

The fastest web crawler written in Rust. Maintained by @a11ywatch.

Woothee ⭐ 304

User-Agent parser/classifier for multi languages

Crypto Crawler Rs ⭐ 137

A rock-solid cryptocurrency crawler library.

Ungoliant ⭐ 132

🕷️ The pipeline for the OSCAR corpus

Dyer is designed for reliable, flexible and fast web crawling, providing some high-level, comprehensive features without compromising speed.

Starfish Ql ⭐ 111

✴️ An experimental graph database

Warc Parquet ⭐ 96

🗄️ A simple CLI for converting WARC to Parquet.

Web Crawler for Crabs

Carbonbot ⭐ 55

A command line tool based on the crypto-crawler library.

Rust Web Crawler saving pages on Redis

gRPC web crawler turbo charged for performance

Dht Crawler ⭐ 27

Tools to crawl and index the BitTorrent DHT.

Shader Prepper ⭐ 27

Shader include parser and crawler for Rust

Reason Rust Scraper ⭐ 21

🦀 Scraping & crawling websites using Rust, and ReasonML

Chan Downloader ⭐ 18

CLI to download all images/webms in a 4chan thread

corroding misbehaving web bots since 2012

Installer ⭐ 17

Installation script for Codename SCNR.

Rust_web_crawler ⭐ 14

Crawling and scraping the Web for fun and profit

Python-based Scraping and parsing toolkit

🤖 Automatic uzimaru bot projects🤖

Quick_crawler ⭐ 8

A configurable async Rust crate that provides a simple way to declaratively navigate to multiple webpages, scrape contents, and follow links to scrape more..

Stock_crawler ⭐ 6

Taiwan stock crawler

Spider Nodejs ⭐ 6

Spider ported to Node.js

Mc Crawler ⭐ 5

A MobileCoin network crawler. Corresponding preprint available on arXiv (https://arxiv.org/pdf/2111.12364.pdf).

Related Searches

Python Crawler (4,545)

Command Line Rust (3,187)

Javascript Rust (1,421)

Video Game Rust (1,177)

Rust Bindings (1,155)

Javascript Crawler (1,142)

Typescript Rust (1,111)

Rust Language (1,068)

Crawler Scrapy (988)

1-24 of 24 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.