Ernie Pytorch
| Project Name | Stars | Downloads | Repos Using This | Packages Using This | Most Recent Commit | Total Releases | Latest Release | Open Issues | License | Language |
|---|---|---|---|---|---|---|---|---|---|---|
| Transformers | 88,463 |  | 64 | 911 | a day ago | 91 | June 21, 2022 | 618 | apache-2.0 | Python |
| 🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX. | ||||||||||
| Real Time Voice Cloning | 40,272 | 8 days ago | 104 | other | Python | |||||
| Clone a voice in 5 seconds to generate arbitrary speech in real-time | ||||||||||
| Ray | 24,807 |  | 80 | 199 | a day ago | 76 | June 09, 2022 | 2,872 | apache-2.0 | Python |
| Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads. | ||||||||||
| Netron | 21,780 |  | 4 | 63 | a day ago | 489 | July 04, 2022 | 22 | mit | JavaScript |
| Visualizer for neural network, deep learning, and machine learning models | ||||||||||
| D2l En | 16,954 | 12 days ago | 83 | other | Python | |||||
| Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 400 universities from 60 countries including Stanford, MIT, Harvard, and Cambridge. | ||||||||||
| Ncnn | 16,760 | 2 days ago | 19 | July 01, 2022 | 957 | other | C++ | |||
| ncnn is a high-performance neural network inference framework optimized for the mobile platform | ||||||||||
| Datasets | 15,633 |  | 9 | 208 | a day ago | 52 | June 15, 2022 | 532 | apache-2.0 | Python |
| 🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools | ||||||||||
| Deeplearning Models | 15,594 | a month ago | 5 | mit | Jupyter Notebook | |||||
| A collection of various deep learning architectures, models, and tips | ||||||||||
| Onnx | 14,351 |  | 148 | 245 | a day ago | 26 | June 18, 2022 | 373 | apache-2.0 | Python |
| Open standard for machine learning interoperability | ||||||||||
| Horovod | 13,142 |  | 20 | 10 | 6 days ago | 72 | June 21, 2022 | 344 | other | Python |
| Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet. | ||||||||||
Alternatives To Ernie PytorchSelect To Compare
Alternative Project Comparisons
Readme
ERNIE-Pytorch
This project is to convert ERNIE from paddlepaddle to huggingface's format (in Pytorch).
News: ERNIE has been merged into huggingface/[email protected] !!
Get Started
pip install --upgrade transformers
Take ernie-1.0-base-zh as an example:
from transformers import BertTokenizer, ErnieModel
tokenizer = BertTokenizer.from_pretrained("nghuyong/ernie-1.0-base-zh")
model = ErnieModel.from_pretrained("nghuyong/ernie-1.0-base-zh")
Supported Models
| Model Name | Language | Description |
|---|---|---|
| ernie-1.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-base-en | English | Layer:12, Heads:12, Hidden:768 |
| ernie-2.0-large-en | English | Layer:24, Heads:16, Hidden:1024 |
| ernie-3.0-xbase-zh | Chinese | Layer:20, Heads:16, Hidden:1024 |
| ernie-3.0-base-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-3.0-medium-zh | Chinese | Layer:6, Heads:12, Hidden:768 |
| ernie-3.0-mini-zh | Chinese | Layer:6, Heads:12, Hidden:384 |
| ernie-3.0-micro-zh | Chinese | Layer:4, Heads:12, Hidden:384 |
| ernie-3.0-nano-zh | Chinese | Layer:4, Heads:12, Hidden:312 |
| ernie-health-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
| ernie-gram-zh | Chinese | Layer:12, Heads:12, Hidden:768 |
You can find all the supported models from huggingface's model hub: huggingface.co/nghuyong, and model details from paddle's official repo: PaddleNLP and ERNIE.
Details
I want to convert the model from paddle version by myself
The following will take ernie-1.0-base-zh as an example to show how to convert.
- Download the paddle-paddle version ERNIE model from here , move to this project path and unzip the file.
pip install -r requirements.txtpython convert.py- Now, a folder named
convertwill be in the project path, and there will be three files in this folder:config.json,pytorch_model.binandvocab.txt.
I want to check the calculation results before and after model conversion
python test.py --task logit_check
You will get the output:
huggingface result
pool output: [-1. -1. 0.9981035 -0.9996652 -0.78173476 -1. -0.9994901 0.97012603 0.85954666 0.9854131 ]
paddle result
pool output: [-0.99999976 -0.99999976 0.9981028 -0.9996651 -0.7815545 -0.99999976 -0.9994898 0.97014064 0.8594844 0.985419 ]
It can be seen that the result of our convert version is the same with the official paddlepaddle's version.
I want to reproduce the cloze test in ERNIE1.0's paper
python test.py --task cloze_check
You will get the output:
huggingface result
prediction shape: torch.Size([47, 18000])
predict result: ['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
[CLS] logit: [-15.693626 -19.522263 -10.429456 ... -11.800728 -12.253127 -14.375117]
paddle result
prediction shape: [47, 18000]
predict result: ['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '']
[CLS] logit: [-15.693538 -19.521954 -10.429307 ... -11.800765 -12.253114 -14.375412]
Citation
If you use this work in a scientific publication, I would appreciate that you can also cite the following BibTex entry:
@misc{[email protected],
title={ERNIEPytorch},
author={Yong Hu},
howpublished={\url{https://github.com/nghuyong/ERNIE-Pytorch}},
year={2019}
}
Popular Pytorch Projects
Popular Tensorflow Projects
Popular Machine Learning Categories
Related Searches
Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Pytorch
Tensorflow
Chinese