Project Name	Stars	Repos Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Sparkler	401		a year ago			55	apache-2.0	Java
Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
See	27		2 years ago				gpl-3.0	Erlang
Search Engine in Erlang
Information Retrieval	15		a year ago					Python
Elasticsearch, MongoDB, Tornado Server, RESTful API, Python, Information Retrieval, Machine Learning, Web Crawler
Serritor	13	2	4 years ago	12	June 11, 2020		apache-2.0	Java
Serritor is an open source web crawler framework built upon Selenium and written in Java. It can be used to crawl dynamic web pages that require JavaScript to render data.
Covid19_stats	9		3 years ago				gpl-3.0	Python
코로나-19 에 대한 확진/완치/사망 에 대한 국내, 해외 정보를 수집합니다. Data scrapes Covid-19 Confirmed/Cured/Deceases Cases.
Inforetrieval	8		5 years ago				mit	HTML
Inverted Indexer, web crawler, sort, search and poster steamer written using Python for information retrieval.
Web Search Engine Uic	6		5 years ago					Python
CS 582 Information Retrieval at University of Illinois at Chicago. Multithreaded crawling of UIC domain, inverted index, page rank, SEO with Context Pseudo-Relevance Feedback
Sce	5		6 years ago			35	apache-2.0	Shell
Sparkler Crawl Environment - a packaged, dockerized version of http://github.com/USCDataScience/sparkler.git
Machine_learning_focused_crawler	5		5 years ago					Python
A focused web crawler that uses Machine Learning to fetch better relevant results.

Alternatives To Inforetrieval

Select To Compare

Sparkler ⭐ 401

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

most recent commit a year ago

See ⭐ 27

Search Engine in Erlang

most recent commit 2 years ago

Information Retrieval ⭐ 15

Elasticsearch, MongoDB, Tornado Server, RESTful API, Python, Information Retrieval, Machine Learning, Web Crawler

most recent commit a year ago

Serritor ⭐ 13

Serritor is an open source web crawler framework built upon Selenium and written in Java. It can be used to crawl dynamic web pages that require JavaScript to render data.

total releases 12most recent commit 4 years ago

Covid19_stats ⭐ 9

코로나-19 에 대한 확진/완치/사망 에 대한 국내, 해외 정보를 수집합니다. Data scrapes Covid-19 Confirmed/Cured/Deceases Cases.

most recent commit 3 years ago

Inforetrieval ⭐ 8

Inverted Indexer, web crawler, sort, search and poster steamer written using Python for information retrieval.

most recent commit 5 years ago

Web Search Engine Uic ⭐ 6

CS 582 Information Retrieval at University of Illinois at Chicago. Multithreaded crawling of UIC domain, inverted index, page rank, SEO with Context Pseudo-Relevance Feedback

most recent commit 5 years ago

Sce ⭐ 5

Sparkler Crawl Environment - a packaged, dockerized version of http://github.com/USCDataScience/sparkler.git

most recent commit 6 years ago

Machine_learning_focused_crawler ⭐ 5

A focused web crawler that uses Machine Learning to fetch better relevant results.

most recent commit 5 years ago

Suggest An Alternative To infoRetrieval

Alternative Project Comparisons

Inforetrieval vs Sparkler

Inforetrieval vs See

Inforetrieval vs Information Retrieval

Inforetrieval vs Serritor

Inforetrieval vs Covid19_stats

Inforetrieval vs Web Search Engine Uic

Inforetrieval vs Sce

Inforetrieval vs Machine_learning_focused_crawler

Popular Crawler Projects

Scrapy ⭐ 49,918

Scrapy, a fast high-level web crawling & scraping framework for Python.

dependent packages 445total releases 96latest release September 18, 2023most recent commit 3 months ago

Lux ⭐ 24,752

👾 Fast and simple video download library and CLI tool written in Go

dependent packages 8total releases 40latest release November 06, 2023most recent commit 25 days ago

Colly ⭐ 21,902

Elegant Scraper and Crawler Framework for Golang

dependent packages 328total releases 22latest release March 08, 2022most recent commit a month ago

Easyspider ⭐ 20,149

A visual no-code/code-free web crawler/spider易采集：一个可视化浏览器自动化测试/数据采集/爬虫软件，可以无代码图形化

most recent commit 23 days ago

Proxy_pool ⭐ 19,442

Python ProxyPool for web spider

most recent commit 4 months ago

Popular Information Retrieval Projects

Easyocr ⭐ 20,438

Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

dependent packages 69total releases 32latest release September 04, 2023most recent commit 4 months ago

Gensim ⭐ 15,180

Topic Modelling for Humans

dependent packages 766total releases 93latest release August 24, 2023most recent commit 19 days ago

Haystack ⭐ 12,474

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

dependent packages 30total releases 100latest release November 09, 2023most recent commit 3 months ago

Weaviate ⭐ 9,527

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database.

dependent packages 14total releases 131latest release December 06, 2023most recent commit 12 hours ago

Danswer ⭐ 6,435

Ask Questions in natural language and get Answers backed by private sources. Connects to tools like Slack, GitHub, Confluence, etc.

most recent commit 3 months ago

Popular Data Processing Categories