Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Toktok Alternatives

Generic tokenizer written in Nim language 👑 Powered by Nim's Macros
Stars
23
License
mit
Open Issues
0
Most Recent Commit
a year ago
Programming Language
Nim
Latest Release
September 28, 2023
Categories
Site
Repo
Alternatives To Toktok
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Php Token Stream6,457packagist phpunit/php-token-stream} Downloads104,2881883 years ago36November 30, 2020otherPHP
Wrapper around PHP's tokenizer extension.
Superpower848nuget Superpower} Downloads64442 years ago99June 23, 202210apache-2.0C#
A C# parser construction toolkit with high-quality error reporting
Php Parser500npm php-parser} Downloads327100a year ago74June 07, 202334bsd-3-clauseJavaScript
:herb: NodeJS PHP Parser - extract AST or tokens
Js Tokens410npm js-tokens} Downloads586,3601,870a year ago21September 09, 2023mitJavaScript
Tiny JavaScript tokenizer.
Lexmachine3701452 years ago6July 15, 20225otherGo
Lex machinary for go.
Coccoc Tokenizer295
4 years ago3lgpl-3.0C++
high performance tokenizer for Vietnamese language
Tsql Parser278nuget TSQL.Parser} Downloads3152 years ago27July 15, 20226apache-2.0C#
Library Written in C# For Parsing SQL Server T-SQL Scripts in .Net
Segmentit208npm segmentit} Downloads162 years ago17December 22, 20196mitJavaScript
任何 JS 环境可用的中文分词包，fork from leizongmin/node-segment
Lex142
4 years agobsl-1.0C++
Replaced by foonathan/lexy
Tokenizer138packagist nette/tokenizer} Downloads333972 years ago9February 09, 2022otherPHP
[DISCONTINUED] Source code tokenizer
Alternatives To Toktok
Select To Compare
Php Token Stream ⭐ 6,457
Wrapper around PHP's tokenizer extension.
dependent packages 188total releases 36most recent commit 3 years agopackagist phpunit/php-token-stream} Downloads
Superpower ⭐ 848
A C# parser construction toolkit with high-quality error reporting
dependent packages 44total releases 99most recent commit 2 years agonuget Superpower} Downloads
Php Parser ⭐ 500
:herb: NodeJS PHP Parser - extract AST or tokens
dependent packages 100total releases 74most recent commit a year agonpm php-parser} Downloads
Js Tokens ⭐ 410
Tiny JavaScript tokenizer.
dependent packages 1,870total releases 21most recent commit a year agonpm js-tokens} Downloads
Lexmachine ⭐ 370
Lex machinary for go.
dependent packages 45total releases 6most recent commit 2 years ago
Tsql Parser ⭐ 278
Library Written in C# For Parsing SQL Server T-SQL Scripts in .Net
dependent packages 15total releases 27most recent commit 2 years agonuget TSQL.Parser} Downloads
Segmentit ⭐ 208
任何 JS 环境可用的中文分词包，fork from leizongmin/node-segment
dependent packages 6total releases 17most recent commit 2 years agonpm segmentit} Downloads
Lex ⭐ 142
Replaced by foonathan/lexy
most recent commit 4 years ago
Tokenizer ⭐ 138
[DISCONTINUED] Source code tokenizer
dependent packages 97total releases 9most recent commit 2 years agopackagist nette/tokenizer} Downloads


Suggest An Alternative To toktok
Alternative Project Comparisons
Toktok vs Php Token Stream
Toktok vs Superpower
Toktok vs Php Parser
Toktok vs Js Tokens
Toktok vs Lexmachine
Toktok vs Coccoc Tokenizer
Toktok vs Tsql Parser
Toktok vs Segmentit
Toktok vs Lex
Toktok vs Tokenizer
Popular Tokenizer Projects
Tokenizers ⭐ 8,056
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
dependent packages 362total releases 85latest release November 14, 2023most recent commit 10 months agopypi tokenizers} Downloads
File Type ⭐ 3,366
Detect the file type of a Buffer/Uint8Array/ArrayBuffer
dependent packages 1,894total releases 141latest release November 11, 2023most recent commit a year agonpm file-type} Downloads
Tntsearch ⭐ 3,004
A fully featured full text search engine written in PHP
dependent packages 27total releases 63latest release July 19, 2023most recent commit a year agopackagist teamtnt/tntsearch} Downloads
Hazm ⭐ 1,216
Persian NLP Toolkit
dependent packages 13total releases 20latest release October 01, 2023most recent commit 5 months agopypi hazm} Downloads
Popular Token Projects
Apprise ⭐ 9,254
Apprise - Push Notifications that work with just about every platform!
dependent packages 35total releases 54latest release October 15, 2023most recent commit a year agopypi apprise} Downloads
Satellizer ⭐ 8,017
Token-based AngularJS Authentication
dependent packages 7total releases 56latest release August 30, 2016most recent commit 5 years agonpm satellizer} Downloads
Popular Compilers Categories
Llvm
Abstract Syntax Tree
Gcc
Clang
Emscripten
Wsl
Lexer
Tokenizer
Dex
Jit
Related Searches
Token Tokenizer
Token Lexer
Lexer Lex
Programming Language Lexer
Lexer Tokenizer
Token Lex
Programming Language Nim
Programming Language Lex
Token Programming Language
Programming Language Tokenizer

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Token
Programming Language
Nim
Lexer
Tokenizer
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.