Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

The Top 23 Natural Language Processing Word Segmentation Open Source Projects

Open source projects categorized as Natural Language Processing Word Segmentation
Categories > Machine Learning > Natural Language Processing
Categories > Machine Learning > Word Segmentation
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34most recent commit 3 months agopypi sentencepiece} Downloads
Youtokentome ⭐ 943
Unsupervised text tokenizer focused on computational efficiency
dependent packages 17total releases 14most recent commit a month agopypi youtokentome} Downloads
Pythainlp ⭐ 902
Thai Natural Language Processing in Python.
dependent packages 51total releases 101most recent commit 3 months agopypi pythainlp} Downloads
Jieba Rs ⭐ 585
The Jieba Chinese Word Segmentation Implemented in Rust
dependent packages 15total releases 40most recent commit 9 months agocargo jieba-rs} Downloads
Ekphrasis ⭐ 583
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
total releases 54most recent commit 2 years ago
M3tl ⭐ 544
BERT for Multitask Learning
most recent commit a year ago
Vncorenlp ⭐ 472
A Vietnamese natural language processing toolkit (NAACL 2018)
most recent commit a year ago
Ckip Transformers ⭐ 439
CKIP Transformers
most recent commit a year ago
Nagisa ⭐ 365
A Japanese tokenizer based on recurrent neural networks
dependent packages 7total releases 22most recent commit 3 months agopypi nagisa} Downloads
Jumanpp ⭐ 334
Juman++ (a Morphological Analyzer Toolkit)
most recent commit a year ago
Kiwi ⭐ 330
Kiwi(지능형 한국어 형태소 분석기)
most recent commit 4 months ago
Adaseq ⭐ 295
AdaSeq: An All-in-One Library for Developing State-of-the-Art Sequence Understanding Models
total releases 11most recent commit 5 months ago
Pycantonese ⭐ 290
Cantonese Linguistics and NLP
total releases 24most recent commit a year ago
Multi Criteria Cws ⭐ 260
Simple Solution for Multi-Criteria Chinese Word Segmentation
most recent commit 5 years ago
Monpa ⭐ 222
MONPA 罔拍是一個提供正體中文斷詞、詞性標註以及命名實體辨識的多任務模型
total releases 10most recent commit 2 years ago
Kiwipiepy ⭐ 182
Python API for Kiwi
dependent packages 4total releases 42most recent commit 5 months agopypi kiwipiepy} Downloads
Bi Lstm Crf ⭐ 180
A PyTorch implementation of the BI-LSTM-CRF model.
total releases 5most recent commit a year ago
Deeplearning_nlp ⭐ 149
基于深度学习的自然语言处理库
most recent commit 5 years ago
Id Cnn Cws ⭐ 130
Source codes and corpora of paper "Iterated Dilated Convolutions for Chinese Word Segmentation"
most recent commit 3 years ago
Nlpcc Wordseg Weibo ⭐ 121
NLPCC 2016 微博分词评测项目
most recent commit 3 years ago
Toiro ⭐ 110
A comparison tool of Japanese tokenizers
total releases 8most recent commit 9 months ago
Ckipnlp ⭐ 100
CKIP CoreNLP Toolkits
total releases 26most recent commit a year ago
Rdrsegmenter ⭐ 67
A Fast and Accurate Vietnamese Word Segmenter (LREC 2018)
most recent commit a year ago
The 9 Most Depended On Natural Language Processing Word Segmentation Open Source Projects
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34latest release May 02, 2023most recent commit 3 months agopypi sentencepiece} Downloads
Pythainlp ⭐ 902
Thai Natural Language Processing in Python.
dependent packages 51total releases 101latest release November 26, 2023most recent commit 3 months agopypi pythainlp} Downloads
Youtokentome ⭐ 943
Unsupervised text tokenizer focused on computational efficiency
dependent packages 17total releases 14latest release February 12, 2020most recent commit a month agopypi youtokentome} Downloads
Jieba Rs ⭐ 585
The Jieba Chinese Word Segmentation Implemented in Rust
dependent packages 15total releases 40latest release July 16, 2023most recent commit 9 months agocargo jieba-rs} Downloads
Nagisa ⭐ 365
A Japanese tokenizer based on recurrent neural networks
dependent packages 7total releases 22latest release July 30, 2023most recent commit 3 months agopypi nagisa} Downloads
Kiwipiepy ⭐ 182
Python API for Kiwi
dependent packages 4total releases 42latest release November 20, 2023most recent commit 5 months agopypi kiwipiepy} Downloads
Python Vncorenlp ⭐ 26
A Python wrapper for VnCoreNLP using a bidirectional communication channel.
dependent packages 4total releases 3latest release July 25, 2018most recent commit 6 years agopypi vncorenlp} Downloads
Sentencepiece ⭐ 21
R package for Byte Pair Encoding / Unigram modelling based on Sentencepiece
dependent packages 1total releases 6latest release November 13, 2022most recent commit a year ago
Rakutenma Python ⭐ 14
Rakuten MA (Python version)
dependent packages 1total releases 10latest release May 22, 2017most recent commit 7 years agopypi rakutenma} Downloads
The 10 Latest Releases In Natural Language Processing Word Segmentation Open Source Projects
Pythainlp ⭐ 902
Thai Natural Language Processing in Python.
dependent packages 51total releases 101latest release November 26, 2023most recent commit 3 months agopypi pythainlp} Downloads
Kiwipiepy ⭐ 182
Python API for Kiwi
dependent packages 4total releases 42latest release November 20, 2023most recent commit 5 months agopypi kiwipiepy} Downloads
Adaseq ⭐ 295
AdaSeq: An All-in-One Library for Developing State-of-the-Art Sequence Understanding Models
total releases 11latest release November 15, 2023most recent commit 5 months ago
Toiro ⭐ 110
A comparison tool of Japanese tokenizers
total releases 8latest release July 31, 2023most recent commit 9 months ago
Nagisa ⭐ 365
A Japanese tokenizer based on recurrent neural networks
dependent packages 7total releases 22latest release July 30, 2023most recent commit 3 months agopypi nagisa} Downloads
Jieba Rs ⭐ 585
The Jieba Chinese Word Segmentation Implemented in Rust
dependent packages 15total releases 40latest release July 16, 2023most recent commit 9 months agocargo jieba-rs} Downloads
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34latest release May 02, 2023most recent commit 3 months agopypi sentencepiece} Downloads
Sentencepiece ⭐ 21
R package for Byte Pair Encoding / Unigram modelling based on Sentencepiece
dependent packages 1total releases 6latest release November 13, 2022most recent commit a year ago
Popular Machine Learning Categories
Deep Learning
Machine Learning
Pytorch
Tensorflow
Natural Language Processing
Neural Network
Neural
Computer Vision
Convolutional Neural Networks
Opencv
Categories
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.