Autonomous Learning Library

A PyTorch library for building deep reinforcement learning agents.
Alternatives To Autonomous Learning Library
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Ray25,9038019918 hours ago76June 09, 20222,894apache-2.0Python
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.
Annotated_deep_learning_paper_implementations22,4641a month ago76June 27, 202217mitJupyter Notebook
🧑‍🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
D2l En18,039
21 hours ago101otherPython
Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 400 universities from 60 countries including Stanford, MIT, Harvard, and Cambridge.
Pytorch Tutorial7,372
2 months ago28mitJupyter Notebook
Build your neural network easy and fast, 莫烦Python中文教学
Wandb6,1473923218 hours ago237July 05, 2022844mitPython
🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.
Stable Baselines35,83734a day ago49June 14, 202281mitPython
PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
12 days ago40unlicenseJupyter Notebook
A course in reinforcement learning in the wild
Deep Reinforcement Learning4,419
3 months ago2mitJupyter Notebook
Repo for the Deep Reinforcement Learning Nanodegree program
19 hours ago76mitPython
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
Polyaxon3,323411a day ago334June 05, 2022122apache-2.0
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
Alternatives To Autonomous Learning Library
Select To Compare

Alternative Project Comparisons

The Autonomous Learning Library: A PyTorch Library for Building Reinforcement Learning Agents

The autonomous-learning-library is an object-oriented deep reinforcement learning (DRL) library for PyTorch. The goal of the library is to provide the necessary components for quickly building and evaluating novel reinforcement learning agents, as well as providing high-quality reference implementations of modern DRL algorithms. The full documentation can be found at the following URL:

Tools for Building New Agents

The primary goal of the autonomous-learning-library is to facilitate the rapid development of new reinforcement learning agents by providing common tools for building and evaluation agents, such as:

  • A flexible function Approximation API that integrates features such as target networks, gradient clipping, learning rate schedules, model checkpointing, multi-headed networks, loss scaling, logging, and more.
  • Various memory buffers, including prioritized experience replay (PER), generalized advantage estimation (GAE), and more.
  • A torch-based Environment interface that simplies agent implementations by cutting out the numpy middleman.
  • Common wrappers and agent enhancements for replicating standard benchmarks.
  • Slurm integration for running large-scale experiments.
  • Plotting and logging utilities including tensorboard integration and utilities for generating common plots.

See the documentation guide for a full description of the functionality provided by the autonomous-learning-library. Additionally, we provide an example project which demonstrates the best practices for building new agents.

High-Quality Reference Implementations

The autonomous-learning-library separates reinforcement learning agents into two modules: all.agents, which provides flexible, high-level implementations of many common algorithms which can be adapted to new problems and environments, and all.presets which provides specific instansiations of these agents tuned for particular sets of environments, including Atari games, classic control tasks, and PyBullet robotics simulations. Some benchmark results showing results on-par with published results can be found below:

atari40 pybullet

As of today, all contains implementations of the following deep RL algorithms:

  • [x] Advantage Actor-Critic (A2C)
  • [x] Categorical DQN (C51)
  • [x] Deep Deterministic Policy Gradient (DDPG)
  • [x] Deep Q-Learning (DQN) + extensions
  • [x] Proximal Policy Optimization (PPO)
  • [x] Rainbow (Rainbow)
  • [x] Soft Actor-Critic (SAC)

It also contains implementations of the following "vanilla" agents, which provide useful baselines and perform better than you may expect:

  • [x] Vanilla Actor-Critic
  • [x] Vanilla Policy Gradient
  • [x] Vanilla Q-Learning
  • [x] Vanilla Sarsa


First, you will need a new version of PyTorch (>1.3), as well as Tensorboard. Then, you can install the core autonomous-learning-library through PyPi:

pip install autonomous-learning-library

You can also install all of the extras (such as Gym environments) using:

pip install autonomous-learning-library[all]

Finally, you can install directly from this repository including the dev dependencies using:

git clone
cd autonomous-learning-library
pip install -e .[dev]

Running the Presets

If you just want to test out some cool agents, the library includes several scripts for doing so:

all-atari Breakout a2c

You can watch the training progress using:

tensorboard --logdir runs

and opening your browser to http://localhost:6006. Once the model is fully trained, you can watch the trained model play using:

all-watch-atari Breakout "runs/a2c_[id]/"

where id is the ID of your particular run. You should should be able to find it using tab completion or by looking in the runs directory. The autonomous-learning-library also contains presets and scripts for classic control and PyBullet environments.

If you want to test out your own agents, you will need to define your own scripts. Some examples can be found in the examples folder). See the docs for information on building your own agents!


This library was built in the Autonomous Learning Laboratory (ALL) at the University of Massachusetts, Amherst. It was written and is currently maintained by Chris Nota (@cpnota). The views expressed or implied in this repository do not necessarily reflect the views of the ALL.

Citing the Autonomous Learning Library

We recommend the following citation:

  author = {Nota, Chris},
  title = {The Autonomous Learning Library},
  year = {2020},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{}},
Popular Pytorch Projects
Popular Reinforcement Learning Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Deep Learning
Reinforcement Learning
Deep Reinforcement Learning
Proximal Policy Optimization