Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Vaporetto

🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
Stars
206
License
apache-2.0
Open Issues
0
Most Recent Commit
5 months ago
Programming Language
Rust
Total Downloads
cargo vaporetto Downloads
Dependent Packages
3
Total Releases
16
Latest Release
April 01, 2023
Categories
Site
Repo
Alternatives To Vaporetto
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Sentencepiece8,851pypi sentencepiece} Downloads1207873 months ago34May 02, 202332apache-2.0C++
Unsupervised text tokenizer for Neural Network-based text generation.
Tokenizers8,056pypi tokenizers} Downloads3623 months ago85November 14, 2023233apache-2.0Rust
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
Gpt2 Chinese7,249
3 months ago105mitPython
Chinese version of GPT2 training code, using BERT tokenizer.
Hazm1,096pypi hazm} Downloads17132 days ago20October 01, 202312mitPython
Persian NLP Toolkit
Natasha1,085pypi natasha} Downloads397 months ago19July 24, 202324mitPython
Solves basic Russian NLP tasks, API for lower level Natasha projects
Kobert1,035
a year ago5apache-2.0Jupyter Notebook
Korean BERT pre-trained cased (KoBERT)
Nlp With Ruby1,002
10 months ago5cc0-1.0Ruby
Curated List: Practical Natural Language Processing done in Ruby
Soynlp801pypi soynlp} Downloads492 years ago33August 25, 201954otherPython
한국어 자연어처리를 위한 파이썬 라이브러리입니다. 단어 추출/ 토크나이저 / 품사판별/ 전처리의 기능을 제공합니다.
Ekphrasis583
72 years ago54May 17, 202218mitPython
Ekphrasis is a text processing tool, geared towards text from social networks, such as Twitter or Facebook. Ekphrasis performs tokenization, word normalization, word segmentation (for splitting hashtags) and spell correction, using word statistics from 2 big corpora (english Wikipedia, twitter - 330mil english tweets).
Open Korean Text55266a year ago14August 07, 201813apache-2.0Scala
Open Korean Text Processor - An Open-source Korean Text Processor
Alternatives To Vaporetto
Select To Compare
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34most recent commit 3 months agopypi sentencepiece} Downloads
Tokenizers ⭐ 8,056
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
dependent packages 362total releases 85most recent commit 3 months agopypi tokenizers} Downloads
Hazm ⭐ 1,096
Persian NLP Toolkit
dependent packages 13total releases 20most recent commit 2 days agopypi hazm} Downloads
Natasha ⭐ 1,085
Solves basic Russian NLP tasks, API for lower level Natasha projects
dependent packages 9total releases 19most recent commit 7 months agopypi natasha} Downloads
Open Korean Text ⭐ 552
Open Korean Text Processor - An Open-source Korean Text Processor
dependent packages 6total releases 14most recent commit a year ago


Suggest An Alternative To vaporetto
Alternative Project Comparisons
Vaporetto vs Sentencepiece
Vaporetto vs Tokenizers
Vaporetto vs Gpt2 Chinese
Vaporetto vs Hazm
Vaporetto vs Natasha
Vaporetto vs Kobert
Vaporetto vs Nlp With Ruby
Vaporetto vs Soynlp
Vaporetto vs Ekphrasis
Vaporetto vs Open Korean Text
Popular Tokenizer Projects
Php Token Stream ⭐ 6,457
Wrapper around PHP's tokenizer extension.
dependent packages 188total releases 36latest release November 30, 2020most recent commit 3 years agopackagist phpunit/php-token-stream} Downloads
Tokenizer ⭐ 5,084
A small library for converting tokenized PHP source code into XML (and potentially other formats)
dependent packages 11total releases 8latest release November 20, 2023most recent commit 4 months agopackagist theseer/tokenizer} Downloads
File Type ⭐ 3,366
Detect the file type of a Buffer/Uint8Array/ArrayBuffer
dependent packages 1,894total releases 141latest release November 11, 2023most recent commit 3 months agonpm file-type} Downloads
Tntsearch ⭐ 3,004
A fully featured full text search engine written in PHP
dependent packages 27total releases 63latest release July 19, 2023most recent commit 3 months agopackagist teamtnt/tntsearch} Downloads
Chevrotain ⭐ 2,381
Parser Building Toolkit for JavaScript
dependent packages 272total releases 170latest release August 14, 2023most recent commit 8 days agonpm chevrotain} Downloads
Popular Natural Language Processing Projects
Transformers ⭐ 124,049
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
dependent packages 2,484total releases 125latest release November 15, 2023most recent commit 6 days agopypi transformers} Downloads
Ailearning ⭐ 37,934
AiLearning：数据分析+机器学习实战+线性代数+PyTorch+NLTK+TF2
dependent packages 2total releases 8latest release March 20, 2022most recent commit a month agonpm ailearning} Downloads
Bert ⭐ 36,099
TensorFlow code and pre-trained models for BERT
dependent packages 10total releases 5latest release August 11, 2020most recent commit 6 months agopypi bert-tensorflow} Downloads
Made With Ml ⭐ 35,496
Learn how to design, develop, deploy and iterate on production-grade ML applications.
total releases 5latest release May 15, 2019most recent commit 4 months ago
Popular Compilers Categories
Llvm
Abstract Syntax Tree
Gcc
Clang
Emscripten
Wsl
Lexer
Jit
Ghc
Ida
Related Searches
Natural Language Processing Tokenizer
Natural Language Processing Segmentation
Natural Language Processing Japanese
Natural Language Processing Tokenization
Rust Natural Language Processing
Natural Language Processing Morphological Analysis
Japanese Tokenizer
Rust Tokenizer
Rust Japanese
Tokenizer Tokenization

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Rust
Natural Language Processing
Segmentation
Japanese
Tokenizer
Morphological Analysis
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.