Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Natural Language Processing Tokenization Open Source Projects

Open source projects categorized as Natural Language Processing Tokenization
Categories > Machine Learning > Natural Language Processing
Categories > Tokenization
Spacy ⭐ 28,628
💫 Industrial-strength Natural Language Processing (NLP) in Python
dependent packages 1,367total releases 226most recent commit 10 days agopypi spacy} Downloads
Youtokentome ⭐ 943
Unsupervised text tokenizer focused on computational efficiency
dependent packages 17total releases 14most recent commit 21 days agopypi youtokentome} Downloads
Trankit ⭐ 693
Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing
dependent packages 2total releases 20most recent commit 5 months agopypi trankit} Downloads
Spacy Streamlit ⭐ 688
👑 spaCy building blocks and visualizers for Streamlit apps
total releases 17most recent commit 9 months ago
Datacamp Python Data Science Track ⭐ 655
All the slides, accompanying code and exercises all stored in this repo. 🎈
most recent commit 9 months ago
Ekphrasis ⭐ 583
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
total releases 54most recent commit 2 years ago
Php Text Analysis ⭐ 484
PHP Text Analysis is a library for performing Information Retrieval (IR) and Natural Language Processing (NLP) tasks using the PHP language
dependent packages 6total releases 42most recent commit 7 months agopackagist yooper/php-text-analysis} Downloads
Vibrato ⭐ 275
🎤 vibrato: Viterbi-based accelerated tokenizer
dependent packages 1total releases 11most recent commit 4 months agocargo vibrato} Downloads
Razdel ⭐ 226
Rule-based token, sentence segmentation for Russian language
dependent packages 7total releases 5most recent commit 9 months agopypi razdel} Downloads
Tokenizer ⭐ 224
Fast and customizable text tokenization library with BPE and SentencePiece support
dependent packages 5total releases 68most recent commit 9 months agopypi pyonmttok} Downloads
Vaporetto ⭐ 206
🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
dependent packages 3total releases 16most recent commit 5 months agocargo vaporetto} Downloads
Python_natural_language_processing ⭐ 164
This repository consists of a complete guide on natural language processing (NLP) in Python where we'll learn various techniques for implementing NLP including parsing & text processing and understand how to use NLP for text feature engineering.
most recent commit 2 years ago
Vtext ⭐ 110
Simple NLP in Rust with Python bindings
dependent packages 1total releases 4most recent commit 3 years agopypi vtext} Downloads
Simplemma ⭐ 100
Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
dependent packages 9total releases 14most recent commit 7 months agopypi simplemma} Downloads
Nlp Cheat Sheet Python ⭐ 98
NLP Cheat Sheet, Python, spacy, LexNPL, NLTK, tokenization, stemming, sentence detection, named entity recognition
most recent commit a year ago
Tweebanknlp ⭐ 94
[LREC 2022] An off-the-shelf pre-trained Tweet NLP Toolkit (NER, tokenization, lemmatization, POS tagging, dependency parsing) + Tweebank-NER dataset
most recent commit 3 months ago
Lima ⭐ 92
The Libre Multilingual Analyzer, a Natural Language Processing (NLP) C++ toolkit.
total releases 22most recent commit 5 months ago
Wordtokenizers.jl ⭐ 63
High performance tokenizers for natural language processing and other related tasks
most recent commit 3 years ago
Nlpcloud Python ⭐ 63
NLP Cloud serves high performance pre-trained or custom models for NER, sentiment-analysis, classification, summarization, paraphrasing, intent classification, product description and ad generation, chatbot, grammar and spelling correction, keywords and keyphrases extraction, text generation, image generation, code generation, and more...
dependent packages 29total releases 41most recent commit 5 months agopypi nlpcloud} Downloads
Spacy Server ⭐ 57
🦜 Containerized HTTP API for industrial-strength NLP via spaCy and sense2vec
most recent commit 3 years ago
Mbti Personality Classifier ⭐ 51
A model which uses your social media posting predict your MBTI personality type.
most recent commit 4 months ago
Tkseem ⭐ 49
Arabic Tokenization Library. It provides many tokenization algorithms.
dependent packages 1total releases 3most recent commit a year agopypi tkseem} Downloads
Attacut ⭐ 47
A Fast and Accurate Neural Thai Word Segmenter
dependent packages 3total releases 17most recent commit 3 years agopypi attacut} Downloads
The 10 Most Depended On Natural Language Processing Tokenization Open Source Projects
Spacy ⭐ 28,628
💫 Industrial-strength Natural Language Processing (NLP) in Python
dependent packages 1,367total releases 226latest release October 16, 2023most recent commit 10 days agopypi spacy} Downloads
Youtokentome ⭐ 943
Unsupervised text tokenizer focused on computational efficiency
dependent packages 17total releases 14latest release February 12, 2020most recent commit 21 days agopypi youtokentome} Downloads
Simplemma ⭐ 100
Simple multilingual lemmatizer for Python, especially useful for speed and efficiency
dependent packages 9total releases 14latest release January 20, 2023most recent commit 7 months agopypi simplemma} Downloads
Razdel ⭐ 226
Rule-based token, sentence segmentation for Russian language
dependent packages 7total releases 5latest release March 26, 2020most recent commit 9 months agopypi razdel} Downloads
Uax29 ⭐ 35
A tokenizer based on Unicode text segmentation (UAX #29), for Go. Split words, sentences and graphemes.
dependent packages 6total releases 40latest release May 26, 2023most recent commit 6 months ago
Java ⭐ 10
Rosette API Client Library for Java
dependent packages 5total releases 59latest release July 11, 2023most recent commit 3 months ago
The 10 Latest Releases In Natural Language Processing Tokenization Open Source Projects
Spacy ⭐ 28,628
💫 Industrial-strength Natural Language Processing (NLP) in Python
dependent packages 1,367total releases 226latest release October 16, 2023most recent commit 10 days agopypi spacy} Downloads
Nlpashto ⭐ 10
Pashto Natural Language Processing Toolkit
total releases 23latest release October 09, 2023most recent commit 6 months ago
Php ⭐ 5
Rosette API Client Library for PHP
total releases 22latest release July 12, 2023most recent commit 3 months ago
Popular Machine Learning Categories
Deep Learning
Machine Learning
Pytorch
Tensorflow
Natural Language Processing
Neural Network
Neural
Computer Vision
Convolutional Neural Networks
Opencv
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.