Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Cloud Computing Search Engine

A cloud-based web search engine computing Hadoop MapReduce on Amazon EC2 consisting of crawler, indexer, PageRank.
Stars
7
License
No license specified
Open Issues
0
Most Recent Commit
11 years ago
Programming Language
Java
Categories
Repo
Alternatives To Cloud Computing Search Engine
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Commoncrawl Crawler208
a year agogpl-3.0Java
The Common Crawl Crawler Engine and Related MapReduce code (2008-2012)
Elasticrawl50
17 years ago10February 15, 20171mitRuby
Launch AWS Elastic MapReduce jobs that process Common Crawl data.
Common_crawl_types28
12 years agoRuby
A simple Ruby example of how to process Common Crawl files using Elastic MapReduce
Real_time_social_media_mining24
5 months ago21mitHTML
DevOps pipeline for Real Time Social/Web Mining
Cs205_ga16
10 years agoPython
How deep does Google Analytics go? Efficiently tackling Common Crawl using AWS & MapReduce
Googleplay Web Crawler15
7 years agoJava
Mapreduce project by Hadoop, Nutch, AWS EMR, Pig, Tez, Hive
Warc Mapreduce11
9 years agoJava
warc and wet support for Hadoop's mapreduce api
Cc Mrjob9
8 years ago8mitPython
Demonstration of using Python to process the Common Crawl dataset with the mrjob framework
Common_crawl8
9 years agomitPython
Simple Python MapReduce jobs for processing the Common Crawl plus command-line utilities
Inforetrieval8
5 years agomitHTML
Inverted Indexer, web crawler, sort, search and poster steamer written using Python for information retrieval.
Alternatives To Cloud Computing Search Engine
Select To Compare


Suggest An Alternative To cloud-computing-search-engine
Alternative Project Comparisons
Cloud Computing Search Engine vs Commoncrawl Crawler
Cloud Computing Search Engine vs Elasticrawl
Cloud Computing Search Engine vs Common_crawl_types
Cloud Computing Search Engine vs Real_time_social_media_mining
Cloud Computing Search Engine vs Cs205_ga
Cloud Computing Search Engine vs Googleplay Web Crawler
Cloud Computing Search Engine vs Warc Mapreduce
Cloud Computing Search Engine vs Cc Mrjob
Cloud Computing Search Engine vs Common_crawl
Cloud Computing Search Engine vs Inforetrieval
Popular Crawler Projects
Scrapy ⭐ 49,918
Scrapy, a fast high-level web crawling & scraping framework for Python.
dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months agopypi Scrapy} Downloads
Lux ⭐ 24,752
👾 Fast and simple video download library and CLI tool written in Go
dependent packages 8total releases 40latest release November 06, 2023most recent commit 17 days ago
Colly ⭐ 21,902
Elegant Scraper and Crawler Framework for Golang
dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago
Popular Mapreduce Projects
Powerjob ⭐ 6,249
Enterprise job scheduling middleware with distributed computing ability.
dependent packages 5total releases 13latest release September 03, 2023most recent commit 3 months ago
Popular Data Processing Categories
Jupyter Notebook
Dataset
Sql
Validation
Pipeline
Translation
Data Science
Classification
Transaction
Scraper

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Java
Crawler
Hadoop
Search Engine
Mapreduce
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.