Project Name	Stars	Repos Using This	Packages Using This	Most Recent Commit	Total Releases	Latest Release	Open Issues	License	Language
Sentences	391	31	127	6 months ago	7	May 26, 2021	5	mit	Go
A multilingual command line sentence tokenizer in Golang
New Hope	6		2	3 years ago	9	August 01, 2020		mit	TypeScript
Esperanto tokenizer

Alternatives To New Hope

Select To Compare

Sentences ⭐ 391

A multilingual command line sentence tokenizer in Golang

dependent packages 127total releases 7most recent commit 6 months ago

New Hope ⭐ 6

Esperanto tokenizer

dependent packages 2total releases 9most recent commit 3 years ago

Suggest An Alternative To new-hope

Alternative Project Comparisons

New Hope vs Sentences

Popular Tokenizer Projects

Sentencepiece ⭐ 8,851

Unsupervised text tokenizer for Neural Network-based text generation.

dependent packages 787total releases 34latest release May 02, 2023most recent commit 4 months ago

Tokenizers ⭐ 8,056

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

dependent packages 362total releases 85latest release November 14, 2023most recent commit 4 months ago

Gpt2 Chinese ⭐ 7,249

Chinese version of GPT2 training code, using BERT tokenizer.

most recent commit 5 months ago

Php Token Stream ⭐ 6,457

Wrapper around PHP's tokenizer extension.

dependent packages 188total releases 36latest release November 30, 2020most recent commit 3 years ago

packagist phpunit/php-token-stream} Downloads

Tokenizer ⭐ 5,084

A small library for converting tokenized PHP source code into XML (and potentially other formats)

dependent packages 11total releases 8latest release November 20, 2023most recent commit 6 months ago

Popular Amd Projects

Webpack ⭐ 63,881

A bundler for javascript and friends. Packs many modules into a few bundled assets. Code Splitting allows for loading parts of the application on demand. Through "loaders", modules can be CommonJs, AMD, ES6 modules, CSS, Images, JSON, Coffeescript, LESS, ... and your custom stuff.

dependent packages 226,953total releases 831latest release October 13, 2023most recent commit 4 months ago

Awesome Javascript ⭐ 32,289

🐢 A collection of awesome browser-side JavaScript libraries, resources and shiny things.

most recent commit 4 months ago

Fastclick ⭐ 18,606

Polyfill to remove click delays on browsers with touch UIs

dependent packages 1,135total releases 28latest release January 26, 2015most recent commit 3 years ago

Highcharts ⭐ 11,705

Highcharts JS, the JavaScript charting framework

dependent packages 11total releases 22latest release November 24, 2017most recent commit 4 months ago

Mono ⭐ 10,734

Mono open source ECMA CLI, C# and .NET implementation.

dependent packages 126total releases 7latest release June 10, 2020most recent commit 4 months ago

Popular Compilers Categories