GluonNLP is a toolkit that helps you solve NLP problems. It provides easy-to-use tools that helps you load the text data, process the text data, and train models.
See our documents at https://nlp.gluon.ai/master/index.html.
First of all, install the MXNet 2 release such as MXNet 2 Alpha. You may use the following commands:
# Install the version with CUDA 10.2 python3 -m pip install -U --pre "mxnet-cu102>=2.0.0a" # Install the version with CUDA 11 python3 -m pip install -U --pre "mxnet-cu110>=2.0.0a" # Install the cpu-only version python3 -m pip install -U --pre "mxnet>=2.0.0a"
To install GluonNLP, use
python3 -m pip install -U -e . # Also, you may install all the extra requirements via python3 -m pip install -U -e ."[extras]"
If you find that you do not have the permission, you can also install to the user folder:
python3 -m pip install -U -e . --user
For Windows users, we recommend to use the Windows Subsystem for Linux.
To facilitate both the engineers and researchers, we provide command-line-toolkits for downloading and processing the NLP datasets. For more details, you may refer to GluonNLP Datasets and GluonNLP Data Processing Tools.
# CLI for downloading / preparing the dataset nlp_data help # CLI for accessing some common data processing scripts nlp_process help # Also, you can use `python -m` to access the toolkits python3 -m gluonnlp.cli.data help python3 -m gluonnlp.cli.process help
You may go to tests to see how to run the unittests.
You can use Docker to launch a JupyterLab development environment with GluonNLP installed.
# GPU Instance docker pull gluonai/gluon-nlp:gpu-latest docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:gpu-latest # CPU Instance docker pull gluonai/gluon-nlp:cpu-latest docker run --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:cpu-latest
For more details, you can refer to the guidance in tools/docker.