Transformers Tutorials

Github repo with tutorials to fine tune transformers for diff NLP tasks
Alternatives To Transformers Tutorials
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Flaml2,4964a day ago68June 17, 2022165mitJupyter Notebook
A fast library for AutoML and tuning. Join our Discord:
4 years ago9Python
Pre-training of Deep Bidirectional Transformers for Language Understanding: pre-train TextCNN
Bert Multi Label Text Classification761
2 months ago40mitPython
This repo contains a PyTorch implementation of a pretrained BERT model for multi-label text classification.
Transformers Tutorials678
4 months ago17mitJupyter Notebook
Github repo with tutorials to fine tune transformers for diff NLP tasks
Azureml Bert374
a year ago22mitJupyter Notebook
End-to-End recipes for pre-training and fine-tuning BERT using Azure Machine Learning Service
2 years ago23November 10, 20217apache-2.0Jupyter Notebook
An easy to use Natural Language Processing library and framework for predicting, training, fine-tuning, and serving up state-of-the-art NLP models.
Bert4doc Classification353
2 years ago12apache-2.0Python
Code and source for paper ``How to Fine-Tune BERT for Text Classification?``
3 years ago13mitJupyter Notebook
The code to reproduce results from paper "MultiFiT: Efficient Multi-lingual Language Model Fine-tuning"
a year ago2mitPython
Sentiment analysis neural network trained by fine-tuning BERT, ALBERT, or DistilBERT on the Stanford Sentiment Treebank.
Kogpt2 Finetuning210
20 days ago3apache-2.0Python
🔥 Korean GPT-2, KoGPT2 FineTuning cased. 한국어 가사 데이터 학습 🔥
Alternatives To Transformers Tutorials
Select To Compare

Alternative Project Comparisons

PyTorch Transformers Tutorials

Transformer Tutorials

GitHub issues GitHub forks Github Stars GitHub license


The field of NLP was revolutionized in the year 2018 by introduction of BERT and his Transformer friends(RoBerta, XLM etc.).

These novel transformer based neural network architectures and new ways to training a neural network on natural language data introduced transfer learning to NLP problems. Transfer learning had been giving out state of the art results in the Computer Vision domain for a few years now and introduction of transformer models for NLP brought about the same paradigm change in NLP.

Companies like Google and Facebook trained their neural networks on large swathes of Natural Language Data to grasp the intricacies of language thereby generating a Language model. Finally these models were fine tuned to specific domain dataset to achieve state of the art results for a specific problem statement. They also published these trained models to open source community. The community members were now able to fine tune these models to their specific use cases.

Hugging Face made it easier for community to access and fine tune these models using their Python Package: Transformers.


Despite these amazing technological advancements applying these solutions to business problems is still a challenge given the niche knowledge required to understand and apply these method on specific problem statements. Hence, In the following tutorials i will be demonstrating how a user can leverage technologies along with some other python tools to fine tune these Language models to specific type of tasks.

Before i proceed i will like to mention the following groups for the fantastic work they are doing and sharing which have made these notebooks and tutorials possible:

Please review these amazing sources of information and subscribe to their channels/sources.

The problem statements that i will be working with are:

Notebook Github Link Colab Link Kaggle Kernel
Text Classification: Multi-Class Github Open In Colab Kaggle
Text Classification: Multi-Label Github Open In Colab Kaggle
Sentiment Classification with Experiment Tracking in WandB! Github Open In Colab
Named Entity Recognition: with TPU processing! Github Open In Colab Kaggle
Question Answering
Summary Writing: with Experiment Tracking in WandB! Github Open In Colab Kaggle

Directory Structure

  1. data: This folder contains all the toy data used for fine tuning.
  2. utils: This folder will contain any miscellaneous script used to prepare for the fine tuning.
  3. models: Folder to save all the artifacts post fine tuning.

Further Watching/Reading

I will try to cover the practical and implementation aspects of fine tuning of these language models on various NLP tasks. You can improve your knowledge on this topic by reading/watching the following resources.

Popular Natural Language Processing Projects
Popular Tuning Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Jupyter Notebook
Deep Learning
Natural Language Processing
Named Entity Recognition
Pytorch Tutorial