The Top 23 Information Retrieval Open Source Projects

Open source projects categorized as Information Retrieval

Categories > Data Processing > Information Retrieval

Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

dependent packages 69total releases 32most recent commit 4 months ago

Gensim ⭐ 15,180

Topic Modelling for Humans

dependent packages 766total releases 93most recent commit 20 days ago

Haystack ⭐ 12,474

🔍 LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

dependent packages 30total releases 100most recent commit 3 months ago

Weaviate ⭐ 9,527

Weaviate is an open-source vector database that stores both objects and vectors, allowing for the combination of vector search with structured filtering with the fault tolerance and scalability of a cloud-native database.

dependent packages 14total releases 131most recent commit 2 days ago

Danswer ⭐ 6,435

Ask Questions in natural language and get Answers backed by private sources. Connects to tools like Slack, GitHub, Confluence, etc.

most recent commit 3 months ago

Txtai ⭐ 6,143

💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

dependent packages 9total releases 35most recent commit 3 months ago

Unstructured ⭐ 4,404

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

dependent packages 65total releases 114most recent commit 3 months ago

Lucene Solr ⭐ 4,363

Apache Lucene and Solr open-source search software

most recent commit 3 months ago

Marqo ⭐ 3,893

Unified embedding generation and search engine. Also available on cloud - cloud.marqo.ai

total releases 11most recent commit 3 months ago

Screenfetch ⭐ 3,690

Fetches system/theme information in terminal for Linux desktop screenshots.

total releases 1most recent commit 4 months ago

Nlpir ⭐ 3,369

most recent commit 4 months ago

Catalyst ⭐ 3,151

Accelerated deep learning R&D

dependent packages 19total releases 108most recent commit 10 months ago

Flagembedding ⭐ 2,797

Dense Retrieval and Retrieval-augmented LLMs

dependent packages 2total releases 13most recent commit 3 months ago

Ranking ⭐ 2,666

Learning to Rank in TensorFlow

dependent packages 12total releases 22most recent commit 7 months ago

Invoicenet ⭐ 2,297

Deep neural network to extract intelligent information from invoice documents.

most recent commit 5 months ago

Lucene ⭐ 2,200

Apache Lucene open-source search software

most recent commit 3 months ago

Llmware ⭐ 1,859

Providing enterprise-grade LLM-based development framework, tools, and fine-tuned models.

most recent commit 3 months ago

Instructor Embedding ⭐ 1,630

[ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings

most recent commit a month ago

Knowledge Graphs ⭐ 1,610

A collection of research on knowledge graphs

most recent commit 2 years ago

Pke ⭐ 1,431

Python Keyphrase Extraction module

dependent packages 1total releases 1most recent commit 10 months ago

Beir ⭐ 1,370

A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.

dependent packages 8total releases 29most recent commit a month ago

Telegram Scraper ⭐ 1,356

telegram group scraper tool. fetch all information about group members

most recent commit 3 years ago

Pyserini ⭐ 1,291

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

dependent packages 14total releases 33most recent commit 3 months ago

The 10 Most Depended On Information Retrieval Open Source Projects

Gensim ⭐ 15,180

Topic Modelling for Humans

dependent packages 766total releases 93latest release August 24, 2023most recent commit 20 days ago

Easyocr ⭐ 20,438

Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

dependent packages 69total releases 32latest release September 04, 2023most recent commit 4 months ago

Unstructured ⭐ 4,404

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

dependent packages 65total releases 114latest release November 30, 2023most recent commit 3 months ago

Rank_bm25 ⭐ 613

A Collection of BM25 Algorithms in Python

dependent packages 52total releases 4latest release February 16, 2022most recent commit 10 months ago

Talisman ⭐ 666

Straightforward fuzzy matching, information retrieval and NLP building blocks for JavaScript.

dependent packages 48total releases 30latest release January 21, 2021most recent commit a year ago

Haystack ⭐ 12,474

:mag: LLM orchestration framework to build customizable, production-ready LLM applications. Connect components (models, vector DBs, file converters) to pipelines or agents that can interact with your data. With advanced retrieval methods, it's best suited for building RAG, question answering, semantic search or conversational agent chatbots.

dependent packages 30total releases 100latest release November 09, 2023most recent commit 3 months ago

Rust Stemmers ⭐ 61

A rust implementation of some popular snowball stemming algorithms

dependent packages 23total releases 6latest release November 17, 2019most recent commit 3 years ago

Catalyst ⭐ 3,151

Accelerated deep learning R&D

dependent packages 19total releases 108latest release April 29, 2022most recent commit 10 months ago

Weaviate ⭐ 9,527

dependent packages 14total releases 131latest release December 06, 2023most recent commit 2 days ago

Pyserini ⭐ 1,291

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

dependent packages 14total releases 33latest release November 17, 2023most recent commit 3 months ago

The 10 Latest Releases In Information Retrieval Open Source Projects

Weaviate ⭐ 9,527

dependent packages 14total releases 131latest release December 06, 2023most recent commit 2 days ago

Unstructured ⭐ 4,404

Open source libraries and APIs to build custom preprocessing pipelines for labeling, training, or production machine learning pipelines.

dependent packages 65total releases 114latest release November 30, 2023most recent commit 3 months ago

Ranx ⭐ 228

⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍

dependent packages 3total releases 45latest release November 28, 2023most recent commit 5 months ago

Archive Query Log ⭐ 18

📜 The Archive Query Log.

total releases 28latest release November 27, 2023most recent commit 3 months ago

Flagembedding ⭐ 2,797

Dense Retrieval and Retrieval-augmented LLMs

dependent packages 2total releases 13latest release November 25, 2023most recent commit 3 months ago

Octopus ⭐ 34

On-device Speech-to-Index engine powered by deep learning

dependent packages 1total releases 14latest release November 23, 2023most recent commit 4 months ago

Stringzilla ⭐ 999

Up to 10x faster string search, split, sort, and shuffle for long strings and multi-gigabyte files in Python and C, leveraging SIMD with just a few lines of Arm Neon and x86 AVX2 & AVX-512 intrinsics 🦖

total releases 5latest release November 19, 2023most recent commit 3 months ago

Pyserini ⭐ 1,291

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

dependent packages 14total releases 33latest release November 17, 2023most recent commit 3 months ago

Anserini ⭐ 944

Anserini is a Lucene toolkit for reproducible information retrieval research

total releases 39latest release November 17, 2023most recent commit 3 months ago

Simsimd ⭐ 514

total releases 23latest release November 11, 2023most recent commit 3 months ago

Popular Data Processing Categories