Cloud-native Deep Reinforcement Learning. 🔥
Alternatives To Elegantrl
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Ray25,9038019919 hours ago76June 09, 20222,894apache-2.0Python
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.
Annotated_deep_learning_paper_implementations22,4641a month ago76June 27, 202217mitJupyter Notebook
🧑‍🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
D2l En18,039
a day ago101otherPython
Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 400 universities from 60 countries including Stanford, MIT, Harvard, and Cambridge.
Pytorch Tutorial7,372
2 months ago28mitJupyter Notebook
Build your neural network easy and fast, 莫烦Python中文教学
Wandb6,1473923220 hours ago237July 05, 2022844mitPython
🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.
Stable Baselines35,844345 hours ago49June 14, 202280mitPython
PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
12 days ago40unlicenseJupyter Notebook
A course in reinforcement learning in the wild
Deep Reinforcement Learning4,419
3 months ago2mitJupyter Notebook
Repo for the Deep Reinforcement Learning Nanodegree program
an hour ago77mitPython
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
Polyaxon3,324411a day ago334June 05, 2022122apache-2.0
MLOps Tools For Managing & Orchestrating The Machine Learning LifeCycle
Alternatives To Elegantrl
Select To Compare

Alternative Project Comparisons

ElegantRL “小雅”: Massively Parallel Library for Cloud-native Deep Reinforcement Learning

Downloads Downloads Python 3.6 PyPI

ElegantRL (website) is developed for users/developers with the following advantages:

  • Cloud-native: follows a cloud-native paradigm through micro-service architecture and containerization, and supports ElegantRL-Podracer and FinRL-Podracer.

  • Scalable: fully exploits the parallelism of DRL algorithms, making it easily scale out to hundreds or thousands of computing nodes on a cloud platform, say, a DGX SuperPOD platform with thousands of GPUs.

  • Elastic: allows to elastically and automatically allocate computing resources on the cloud.

  • Lightweight: the core codes have <1,000 lines (check Elegantrl_Helloworld).

  • Efficient: in many testing cases (e.g., single-GPU/multi-GPU/GPU-cloud), we find it more efficient than Ray RLlib.

  • Stable: much much much more stable than Stable Baselines 3 by utilizing various ensemble methods.

  • Practical: used in multipe projects (ElegantRL_Solver, FinRL, and FinRL-Meta etc.)

ElegantRL implements the following model-free deep reinforcement learning (DRL) algorithms:

  • DDPG, TD3, SAC, PPO, REDQ for continuous actions in single-agent environment,
  • DQN, Double DQN, D3QN for discrete actions in single-agent environment,
  • QMIX, VDN, MADDPG, MAPPO, MATD3 in multi-agent environment.

For the details of DRL algorithms, please refer to the educational webpage OpenAI Spinning Up.

ElegantRL supports the following simulators:

  • Isaac Gym for massively parallel simulation,
  • OpenAI Gym, MuJoCo, PyBullet, FinRL for benchmarking.





For beginners, we maintain ElegantRL-Helloworld as a tutorial. Its goal is to get hands-on experience with ELegantRL.

One sentence summary: an agent (agent.py) with Actor-Critic networks (net.py) is trained (run.py) by interacting with an environment (env.py).

File Structure

  • elegantrl # main folder

    • agents # a collection of DRL algorithms
      • AgentXXX.py # a collection of one kind of DRL algorithms
      • net.py # a collection of network architectures
    • envs # a collection of environments
      • XxxEnv.py # a training environment for RL
    • train # a collection of training programs - demo.py # a collection of demos
      • config.py # configurations (hyper-parameter)
      • run.py # training loop
      • worker.py # the worker class (explores the env, saving the data to replay buffer)
      • learner.py # the learner class (update the networks, using the data in replay buffer)
      • evaluator.py # the evaluator class (evaluate the cumulative rewards of policy network)
      • replay_buffer.py # the buffer class (save sequences of transitions for training)
  • elegantrl_helloworld # tutorial version

    • config.py # configurations (hyper-parameter)
    • agent.py # DRL algorithms
    • net.py # network architectures
    • run.py # training loop
    • env.py # environments for RL training
  • examples # a collection of example codes

  • ready-to-run Google-Colab notebooks

    • quickstart_Pendulum_v1.ipynb
    • tutorial_BipedalWalker_v3.ipynb
    • tutorial_Creating_ChasingVecEnv.ipynb
    • tutorial_LunarLanderContinuous_v2.ipynb
  • unit_tests # a collection of tests

Experimental Demos

More efficient than Ray RLlib

Experiments on Ant (MuJoCo), Humainoid (MuJoCo), Ant (Isaac Gym), Humanoid (Isaac Gym) # from left to right

ElegantRL fully supports Isaac Gym that runs massively parallel simulation (e.g., 4096 sub-envs) on one GPU.

More stable than Stable-baseline 3

Experiment on Hopper-v2 # ElegantRL achieves much smaller variance (average over 8 runs).

Also, PPO+H in ElegantRL completed the training process of 5M samples about 6x faster than Stable-Baseline3.

Testing and Contributing

Our tests are written with the built-in unittest Python module for easy access. In order to run a specific test file (for example, test_training_agents.py), use the following command from the root directory:

python -m unittest unit_tests/test_training_agents.py

In order to run all the tests sequentially, you can use the following command:

python -m unittest discover

Please note that some of the tests require Isaac Gym to be installed on your system. If it is not, any tests related to Isaac Gym will fail.

We welcome any contributions to the codebase, but we ask that you please do not submit/push code that breaks the tests. Also, please shy away from modifying the tests just to get your proposed changes to pass them. As it stands, the tests on their own are quite minimal (instantiating environments, training agents for one step, etc.), so if they're breaking, it's almost certainly a problem with your code and not with the tests.

We're actively working on refactoring and trying to make the codebase cleaner and more performant as a whole. If you'd like to help us clean up some code, we'd strongly encourage you to also watch Uncle Bob's clean coding lessons if you haven't already.


| Python 3.6+     |
| PyTorch 1.6+    |

Not necessary:
| Numpy 1.18+     | For ReplayBuffer. Numpy will be installed along with PyTorch.
| gym 0.17.0      | For env. Gym provides tutorial env for DRL training. (env.render() bug in gym==0.18 pyglet==1.6. Change to gym==0.17.0, pyglet==1.5)
| pybullet 2.7+   | For env. We use PyBullet (free) as an alternative of MuJoCo (not free).
| box2d-py 2.3.8  | For gym. Use pip install Box2D (instead of box2d-py)
| matplotlib 3.2  | For plots.

pip3 install gym==0.17.0 pybullet Box2D matplotlib # or pip install -r requirements.txt

To install StarCraftII env,
bash ./elegantrl/envs/installsc2.sh
pip install -r sc2_requirements.txt


To cite this repository:

  author = {Liu, Xiao-Yang and Li, Zechu and Zhu, Ming and Wang, Zhaoran and Zheng, Jiahao},
  title = {{ElegantRL}: Massively Parallel Framework for Cloud-native Deep Reinforcement Learning},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/AI4Finance-Foundation/ElegantRL}},
  title={ElegantRL-Podracer: Scalable and elastic library for cloud-native deep reinforcement learning},
  author={Liu, Xiao-Yang and Li, Zechu and Yang, Zhuoran and Zheng, Jiahao and Wang, Zhaoran and Walid, Anwar and Guo, Jian and Jordan, Michael I},
  journal={NeurIPS, Workshop on Deep Reinforcement Learning},
Popular Reinforcement Learning Projects
Popular Pytorch Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Reinforcement Learning