Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Llmsharp Tokenizers

BPE Tokenizer implementations in C# for Anthropic, OpenAI LLM offerings
Stars
9
License
mit
Open Issues
0
Most Recent Commit
9 months ago
Programming Language
C#
Total Downloads
nuget LLMSharp.Anthropic.Tokenizer Downloads
Dependent Packages
1
Total Releases
6
Latest Release
October 04, 2023
Categories
Repo
Alternatives To Llmsharp Tokenizers
Select To Compare
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34most recent commit 5 months agopypi sentencepiece} Downloads
Tokenizers ⭐ 8,056
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
dependent packages 362total releases 85most recent commit 5 months agopypi tokenizers} Downloads
Php Token Stream ⭐ 6,457
Wrapper around PHP's tokenizer extension.
dependent packages 188total releases 36most recent commit 3 years agopackagist phpunit/php-token-stream} Downloads
File Type ⭐ 3,366
Detect the file type of a Buffer/Uint8Array/ArrayBuffer
dependent packages 1,894total releases 141most recent commit 5 months agonpm file-type} Downloads
Tntsearch ⭐ 3,004
A fully featured full text search engine written in PHP
dependent packages 27total releases 63most recent commit 5 months agopackagist teamtnt/tntsearch} Downloads
Chevrotain ⭐ 2,408
Parser Building Toolkit for JavaScript
dependent packages 272total releases 170most recent commit a month agonpm chevrotain} Downloads
Text ⭐ 1,172
Making text a first-class citizen in TensorFlow.
dependent packages 134total releases 67most recent commit 5 months agopypi tensorflow-text} Downloads
Hazm ⭐ 1,145
Persian NLP Toolkit
dependent packages 13total releases 20most recent commit 9 days agopypi hazm} Downloads


Suggest An Alternative To llmsharp-tokenizers
Popular Compilers Categories
Llvm
Abstract Syntax Tree
Gcc
Clang
Emscripten
Wsl
Lexer
Jit
Ghc
Ida
Related Searches
Openai Gpt 4
Openai Gpt 35 Turbo
C Sharp Openai
C Sharp Tokenizer
C Sharp Gpt 4
Openai Claude
Openai Anthropic
Gpt 4 Claude
Claude Anthropic
Openai Tokenizer

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
C Sharp
Tokenizer
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.