Awesome Open Source

Programming Languages

Search results for c plus plus tokenizer

32 search results found

Sentencepiece ⭐ 8,851

Unsupervised text tokenizer for Neural Network-based text generation.

Making text a first-class citizen in TensorFlow.

Autophrase ⭐ 978

AutoPhrase: Automated Phrase Mining from Massive Text Corpora

支持中文和拼音的 SQLite fts5 全文搜索扩展｜ A SQLite3 fts5 tokenizer which supports Chinese and PinYin

Fugashi ⭐ 339

A Cython MeCab wrapper for fast, pythonic Japanese tokenization and morphological analysis.

Jumanpp ⭐ 334

Juman++ (a Morphological Analyzer Toolkit)

Coccoc Tokenizer ⭐ 295

high performance tokenizer for Vietnamese language

Tokenizer ⭐ 224

Fast and customizable text tokenization library with BPE and SentencePiece support

R package for Tokenization, Parts of Speech Tagging, Lemmatization and Dependency Parsing Based on the UDPipe Natural Language Processing Toolkit

Replaced by foonathan/lexy

C++ String Toolkit Library

Hunspell ⭐ 105

High-Performance Stemmer, Tokenizer, and Spell Checker for R

Simhash Cpp ⭐ 94

Simhashing in C++

Tokenizer ⭐ 53

Convert source code into numerical tokens

Smart Language Model

Android Sqlite Fts5 Tokenizer ⭐ 33

集成了FTS5中文分词器的Sqlite3源码

Python Mecab ⭐ 27

A repository to bind mecab for Python 3.5+. Not using swig nor pybind. (Not Maintained Now)

Sentencepiece Jni ⭐ 27

Java JNI wrapper for SentencePiece: unsupervised text tokenizer for Neural Network-based text generation.

Cppassist ⭐ 25

C++ sanctuary for small but powerful and frequently required, stand alone features.

Trainable Tokenizer ⭐ 22

Fast and trainable tokenizer for natural languages relying on maximum entropy methods.

Mecab Ios ⭐ 20

MeCab Framework for iOS 10.3 - 12.x (Japanese Parser & Tokenizer)

Mini Json Parser ⭐ 19

A Tiny Json Parser

Sphinx Jieba ⭐ 18

sphinx search engine with jieba tokenizer

Tokenizer ⭐ 18

Boost.org tokenizer module

Tivars_lib_cpp ⭐ 14

A C++ library to interact with TI-z80 (82/83/84 series) calculators files (programs, lists, matrices, etc.)

Fast Mosestokenizer ⭐ 12

c++ mosestokenizer

C++ library for generating LALR(1) parsers

Sctokenizer ⭐ 9

A Source Code Tokenizer

An experimental lexer and parser generator

Boosting Tree Tokenizer ⭐ 9

Gradient Boosting Dicision Tree(LightGBM)を用い、教師ありで自然言語の分かちと形態素の推定を学習＆予想します。名称

Rtfreader ⭐ 7

Text segmenter and tokeniser for Danish, English and other languages. Reads an RTF or flat text file and outputs the text, one line per sentence & optionally tokenized.

jsmn++ is a tiny json parser embedded in your C++ project for configuration.

Arduino Stringtokenizer Library ⭐ 6

A very simple arduino library to use java like string-tokenizer functions to split a string with delimiters.

Cjk Tokenizer ⭐ 5

Related Searches

C Plus Plus Qt (8,557)

C Plus Plus Video Game (8,255)

C Plus Plus Cmake (8,010)

C Plus Plus Algorithms (6,194)

Python C Plus Plus (4,508)

C Plus Plus Opengl (4,396)

C Plus Plus 3d Graphics (3,196)

C Plus Plus Testing (2,735)

Java C Plus Plus (2,629)

C Plus Plus Command Line (2,304)

1-32 of 32 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.