Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
---|---|---|---|---|---|---|---|---|---|---|
Kermit | 52 | a year ago | mit | JavaScript | ||||||
🐸 KERMIT - A lightweight library to encode and interpret Universal Syntactic Embeddings | ||||||||||
Tpann | 40 | 7 years ago | 3 | Python | ||||||
Tokenization And Word Embedding Compatibility | 5 | 5 years ago | Jupyter Notebook | |||||||
The Quora Insincere Question Classification competition allows us to use the four embeddings: glove.840B.300d (GloVe), paragram_300_sl999 (paragram), wiki-news-300d-1M (wiki) and GoogleNews-vectors-negative300 (GoogleNews). In a kernel titled: "How to: Preprocessing when Using Embeddings", the author raises the issue of tokenization and its effect on how much of the training vocabulary is covered by words in an embedding. The author uses Google news embeddings to illustrate this point. In this kernel I expand on this point by exploring the effect of tokenization assumptions on the other three embeddings: GloVe, Paragram, and Wiki News. |