This repository contains the example code from our O'Reilly book Natural Language Processing with Transformers:
You can run these notebooks on cloud platforms like Google Colab or your local machine. Note that most chapters require a GPU to run in a reasonable amount of time, so we recommend one of the cloud platforms as they come pre-installed with CUDA.
To run these notebooks on a cloud platform, just click on one of the badges in the table below:
| Chapter | Colab | Kaggle | Gradient | Studio Lab |
|---|---|---|---|---|
| Introduction |  |
 |
 |
 |
| Text Classification | |
|
|
|
| Transformer Anatomy | |
|
|
|
| Multilingual Named Entity Recognition | |
|
|
|
| Text Generation | |
|
|
|
| Summarization | |
|
|
|
| Question Answering | |
|
|
|
| Making Transformers Efficient in Production | |
|
|
|
| Dealing with Few to No Labels | |
|
|
|
| Training Transformers from Scratch | |
|
|
|
| Future Directions | |
|
|
|
Nowadays, the GPUs on Colab tend to be K80s (which have limited memory), so we recommend using Kaggle, Gradient, or SageMaker Studio Lab. These platforms tend to provide more performant GPUs like P100s, all for free!
To run the notebooks on your own machine, first clone the repository:
git clone https://github.com/nlp-with-transformers/notebooks
cd notebooks-test
Next, you'll need to install a few packages that depend on your operating system and hardware:
- PyTorch
- TensorFlow (optional, since only used in a few chapters)
- PyTorch Scatter (only used in Chapter 11)
- librosa (only used in Chapter 11)
- libsndfile (only used in Chapter 11)
Once you have install the above requirements, create a virtual environment and install the remaining Python dependencies:
conda create -n book python=3.8 -y && conda activate book
from install import *
install_requirements()
# Use the following to run Chapter 7
# install_requirements(is_chapter7)