Awesome Open Source

Programming Languages

Search results for character tokenizer

13 search results found

Mustard ⭐ 686

🌭 Mustard is a Swift library for tokenizing strings when splitting by whitespace doesn't cut it.

Spyglass ⭐ 378

A library for mentions on Android

Tokenizers ⭐ 170

Fast, Consistent Tokenization of Natural Language Text

KoRean based BERT pre-trained models (KR-BERT) for Tensorflow and PyTorch

Sqlitesubstringsearch ⭐ 76

An open source tokenizer which supports fast substring search with sqlite FTS (full text search)

Lex is an implementation of lex tool in Ruby.

Koreancharacterbert ⭐ 17

Korean BERT model using character tokenizer

Rftokenizer ⭐ 17

A character-wise tokenizer for morphologically rich languages

HTML parser built in Zig

Parsinghelper ⭐ 9

.NET text parsing helper class.

Lulalala_address_tokenizer ⭐ 9

Postal addresses tokenizer using Wapiti model

Elasticsearch Analysis Hashsplitter ⭐ 8

A N-Gram tokenizer generating non-overlapping positioned token chunks for partial token search

Tokenizer-based character diff tool

Related Searches

Javascript Character (3,077)

Python Character (2,543)

Video Game Character (892)

Java Character (888)

Character Unicode (849)

C Character (836)

C Sharp Character (787)

C Plus Plus Character (737)

Php Character (585)

Ruby Character (565)

1-13 of 13 search results

Privacy | About | Terms | Follow Us On Twitter

Copyright 2018-2024 Awesome Open Source. All rights reserved.