Programming Languages

Languages

All Categories

Parser Toolkit

A toolkit that makes it easier to write recursive-descent parsers in Zig.

Categories > Compilers > Tokenizer

Suggest Alternative

Stars

License

mit

Open Issues

Most Recent Commit

4 months ago

Programming Language

Zig

Categories

Compilers > Tokenizer

Repo

Alternatives To Parser Toolkit

Select To Compare

Sentencepiece ⭐ 8,851

Unsupervised text tokenizer for Neural Network-based text generation.

dependent packages 787total releases 34most recent commit 4 months ago

Tokenizers ⭐ 8,056

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

dependent packages 362total releases 85most recent commit 4 months ago

Gpt2 Chinese ⭐ 7,249

Chinese version of GPT2 training code, using BERT tokenizer.

most recent commit 4 months ago

Php Token Stream ⭐ 6,457

Wrapper around PHP's tokenizer extension.

dependent packages 188total releases 36most recent commit 3 years ago

packagist phpunit/php-token-stream} Downloads

Tokenizer ⭐ 5,084

A small library for converting tokenized PHP source code into XML (and potentially other formats)

dependent packages 11total releases 8most recent commit 5 months ago

File Type ⭐ 3,366

Detect the file type of a Buffer/Uint8Array/ArrayBuffer

dependent packages 1,894total releases 141most recent commit 4 months ago

Tntsearch ⭐ 3,004

A fully featured full text search engine written in PHP

dependent packages 27total releases 63most recent commit 4 months ago

Chevrotain ⭐ 2,381

Parser Building Toolkit for JavaScript

dependent packages 272total releases 170most recent commit a month ago

Text ⭐ 1,172

Making text a first-class citizen in TensorFlow.

dependent packages 134total releases 67most recent commit 4 months ago

Hazm ⭐ 1,120

Persian NLP Toolkit

dependent packages 13total releases 20most recent commit 10 days ago

Suggest An Alternative To parser-toolkit

Popular Compilers Categories