Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About
Search results for golang tokenizer
golang
x
tokenizer
x
11 search results found
Kagome
⭐
769
Self-contained Japanese Morphological Analyzer written in pure Go
Goro
⭐
673
PHP in Go
Tokenmonster
⭐
399
Ungreedy subword tokenizer and vocabulary trainer for Python, Go & Javascript
Sentences
⭐
391
A multilingual command line sentence tokenizer in Golang
Lexmachine
⭐
370
Lex machinary for go.
Shield
⭐
131
Bayesian text classifier with flexible tokenizers and storage backends for Go
Lingo
⭐
102
package lingo provides the data structures and algorithms required for natural language processing
Jargon
⭐
98
Tokenizers and lemmatizers for Go
Tokenizer
⭐
75
NLP tokenizers written in Go language
Go Gpt 3 Encoder
⭐
74
Go BPE tokenizer (Encoder+Decoder) for GPT2 and GPT3
Howtowriteacompiler
⭐
63
How to write a compiler from scratch in 30 minutes
Agency
⭐
59
A fast user agent string parser for Go.
Tokenizer
⭐
57
Tokenizer (lexer) for golang
Refmt
⭐
45
Object mapping for golang.
Uax29
⭐
35
A tokenizer based on Unicode text segmentation (UAX #29), for Go. Split words, sentences and graphemes.
Nlpt
⭐
35
Natural Language Processing Toolkit written in Go (DEPRECATED see individual packages prefixed nlpt-)
Cwsharp Go
⭐
30
cwsharp-go, Golang中文分词库,支持多种分词模式,支持自定义字典和扩展。
Goselect
⭐
28
SQL like 'select' interface for files
Bleve Sego Tokenizer
⭐
24
a Chinese tokenizer for bleve, using sego as the segmenter
Tokenizer
⭐
19
Package tokenizer provides encoding for tokens that can carry user data.
Go Khaiii
⭐
19
Shamoji
⭐
13
The shamoji (杓文字) is a word filtering package
Nihongo
⭐
12
Japanese utilities for Go
Tokenizer
⭐
11
Natural Language Tokenizer
Go Tokenizer
⭐
11
A Text Tokenizer library for Golang
Assocentity
⭐
11
Package assocentity returns the mean distance from tokens to an entity and its synonyms
Masquerade
⭐
10
High-performance, real-time, multi-location data obfuscation tool
Jsonlex
⭐
10
[MODULE] Fast JSON lexer (tokenizer) with no memory footprint and no garbage collector pressure (zero-alloc). 5x faster compared to Go's default encoding/json tokenizer.
Zearch
⭐
9
ragecoded code search with json endpoint
Gotokenizer
⭐
8
A tokenizer based on the dictionary and Bigram language models for Go. (Now only support chinese segmentation)
Tfidf
⭐
6
a golang library to calculate tf-idf weight for giving document, also prepares Chinese tokenizer packaging and cosine similarity compulation.
Punkt
⭐
5
A port of the Punkt sentence tokenizer to Go
Medium Rss Api
⭐
5
A REST API wrapper for Medium RSS Feed with built in cache mechanism and HTML Tokenizer that parses Medium's plain HTML string into DOM objects. Just set your medium's user profile name or publication and you're good to go!
Related Searches
Golang Command Line (8,308)
Golang Docker (7,769)
Golang Http (4,290)
Golang Server (4,285)
Javascript Golang (3,372)
Golang Database (2,927)
Golang Json (2,652)
Golang Proxy (2,577)
Golang Grpc (2,432)
Python Golang (2,332)
1-11 of 11 search results
Privacy
|
About
|
Terms
|
Follow Us On Twitter
Copyright 2018-2024 Awesome Open Source. All rights reserved.