Awesome Open Source
Search
Programming Languages
Languages
All Categories
Categories
About

Segtok

Segtok v2 is here: https://github.com/fnl/syntok -- A rule-based sentence segmenter (splitter) and a word tokenizer using orthographic features.
Stars
151
License
mit
Open Issues
3
Most Recent Commit
2 years ago
Programming Language
Python
Monthly Downloads
pypi segtok Downloads
Dependent Repos
34
Dependent Packages
16
Total Releases
23
Latest Release
December 15, 2021
Categories
Site
Repo
Alternatives To Segtok
Select To Compare
Sentencepiece ⭐ 8,851
Unsupervised text tokenizer for Neural Network-based text generation.
dependent packages 787total releases 34most recent commit 4 months agopypi sentencepiece} Downloads
Tokenizers ⭐ 8,056
💥 Fast State-of-the-Art Tokenizers optimized for Research and Production
dependent packages 362total releases 85most recent commit 3 months agopypi tokenizers} Downloads
Php Token Stream ⭐ 6,457
Wrapper around PHP's tokenizer extension.
dependent packages 188total releases 36most recent commit 3 years agopackagist phpunit/php-token-stream} Downloads
File Type ⭐ 3,366
Detect the file type of a Buffer/Uint8Array/ArrayBuffer
dependent packages 1,894total releases 141most recent commit 4 months agonpm file-type} Downloads
Tntsearch ⭐ 3,004
A fully featured full text search engine written in PHP
dependent packages 27total releases 63most recent commit 4 months agopackagist teamtnt/tntsearch} Downloads
Chevrotain ⭐ 2,381
Parser Building Toolkit for JavaScript
dependent packages 272total releases 170most recent commit a month agonpm chevrotain} Downloads
Text ⭐ 1,172
Making text a first-class citizen in TensorFlow.
dependent packages 134total releases 67most recent commit 4 months agopypi tensorflow-text} Downloads
Hazm ⭐ 1,112
Persian NLP Toolkit
dependent packages 13total releases 20most recent commit 6 days agopypi hazm} Downloads


Suggest An Alternative To segtok
Popular Compilers Categories
Llvm
Abstract Syntax Tree
Gcc
Clang
Emscripten
Wsl
Lexer
Jit
Ghc
Ida

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Tokenizer
Privacy | About | Terms | Follow Us On Twitter

Downloads, Dependent Repos, Dependent Packages, Total Releases, Latest Releases data powered by Libraries.io.

Copyright 2018-2024 Awesome Open Source.  All rights reserved.