Tok Tok
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Wordless | 649 | 3 months ago | gpl-3.0 | Python | ||||||
| An Integrated Corpus Tool With Multilingual Support for the Study of Language, Literature, and Translation | ||||||||||
| Sentences | 391 | 31 | 127 | 5 months ago | 7 | May 26, 2021 | 5 | mit | Go | |
| A multilingual command line sentence tokenizer in Golang | ||||||||||
| Text2text | 268 | 3 months ago | 134 | October 21, 2023 | 27 | other | Python | |||
| Text2Text: Crosslingual NLP/G toolkit | ||||||||||
| Bitextor | 260 | 7 months ago | 4 | gpl-3.0 | Python | |||||
| Bitextor generates translation memories from multilingual websites | ||||||||||
| Wink Tokenizer | 47 |  | 29 | 15 | 2 years ago | 19 | January 27, 2022 | mit | JavaScript | |
| Multilingual tokenizer that automatically tags each token with its type | ||||||||||
| Hottosns Bert | 41 | 3 years ago | 2 | other | Python | |||||
| hottoSNS-BERT: 大規模SNSコーパスによる文分散表現モデル | ||||||||||
| Bert Korean Model | 34 | 4 years ago | 1 | apache-2.0 | ||||||
| BERT with SentencePiece for Korean text | ||||||||||
| Tok Tok | 26 | 7 years ago | 1 | apache-2.0 | Python | |||||
| A fast, simple, multilingual tokenizer | ||||||||||
| Ilmulti | 12 | 4 years ago | 2 | August 30, 2020 | 4 | mit | Python | |||
| Tooling to play around with multilingual machine translation for Indian Languages. | ||||||||||
| Tokenizer | 11 | 5 years ago | 1 | November 28, 2018 | apache-2.0 | Go | ||||
| Natural Language Tokenizer | ||||||||||
Alternatives To Tok TokSelect To Compare
Alternative Project Comparisons
Popular Tokenizer Projects
Popular Multilingual Projects
Popular Compilers Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Language
Perl
Natural Language Processing
Multilingual
Tokenizer