Mushroom Rl

Python library for Reinforcement Learning.
Alternatives To Mushroom Rl
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Cs Video Courses56,273
5 days ago17
List of Computer Science courses with video lectures.
Ray25,916801992 hours ago76June 09, 20222,905apache-2.0Python
Ray is a unified framework for scaling AI and Python applications. Ray consists of a core distributed runtime and a toolkit of libraries (Ray AIR) for accelerating ML workloads.
Applied Ml24,242
12 days ago3mit
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Annotated_deep_learning_paper_implementations22,4641a month ago76June 27, 202217mitJupyter Notebook
🧑‍🏫 59 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠
D2l En18,049
a day ago101otherPython
Interactive deep learning book with multi-framework code, math, and discussions. Adopted at 400 universities from 60 countries including Stanford, MIT, Harvard, and Cambridge.
Ml Agents14,81512144 hours ago44April 01, 2022166otherC#
The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
Tensor2tensor13,70182114 days ago79June 17, 2020589apache-2.0Python
Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
Deep Learning Drizzle10,767
6 months ago6HTML
Drench yourself in Deep Learning, Reinforcement Learning, Machine Learning, Computer Vision, and NLP by learning from these exciting lectures!!
Tensorflow Tutorials8,644
2 years ago2mitJupyter Notebook
TensorFlow Tutorials with YouTube Videos
Amazon Sagemaker Examples8,437
6 hours ago801apache-2.0Jupyter Notebook
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
Alternatives To Mushroom Rl
Select To Compare


Alternative Project Comparisons
Readme

MushroomRL

Continuous Integration Documentation Status Maintainability Test Coverage

MushroomRL: Reinforcement Learning Python library.

What is MushroomRL

MushroomRL is a Python Reinforcement Learning (RL) library whose modularity allows to easily use well-known Python libraries for tensor computation (e.g. PyTorch, Tensorflow) and RL benchmarks (e.g. OpenAI Gym, PyBullet, Deepmind Control Suite). It allows to perform RL experiments in a simple way providing classical RL algorithms (e.g. Q-Learning, SARSA, FQI), and deep RL algorithms (e.g. DQN, DDPG, SAC, TD3, TRPO, PPO).

Full documentation and tutorials available here.

Installation

You can do a minimal installation of MushroomRL with:

pip3 install mushroom_rl

Installing everything

MushroomRL contains also some optional components e.g., support for OpenAI Gym environments, Atari 2600 games from the Arcade Learning Environment, and the support for physics simulators such as Pybullet and MuJoCo. Support for these classes is not enabled by default.

To install the whole set of features, you will need additional packages installed. You can install everything by running:

pip3 install mushroom_rl[all]

This will install every dependency of MushroomRL, except the Plots dependency. For ubuntu>20.04, you may need to install pygame and gym dependencies:

sudo apt -y install libsdl-image1.2-dev libsdl-mixer1.2-dev libsdl-ttf2.0-dev \
                 libsdl1.2-dev libsmpeg-dev libportmidi-dev ffmpeg libswscale-dev \
                 libavformat-dev libavcodec-dev swig

Notice that you still need to install some of these dependencies for different operating systems, e.g. swig for macOS

Below is the code that you need to run to install the Plots dependencies:

sudo apt -y install python3-pyqt5
pip3 install mushroom_rl[plots]

You might need to install external dependencies first. For more information about mujoco-py installation follow the instructions on the project page

WARNING! when using conda, there may be issues with QT. You can fix them by adding the following lines to the code, replacing <conda_base_path> with the path to your conda distribution and <env_name> with the name of the conda environment you are using:
import os
os.environ['QT_QPA_PLATFORM_PLUGIN_PATH'] = '<conda_base_path>/envs/<env_name>/bin/platforms'

To use dm_control MushroomRL interface, install dm_control following the instruction that can be found here

Using Habitat and iGibson with MushroomRL

Habitat and iGibson are simulation platforms providing realistic and sensory-rich learning environments. In MushroomRL, the agent's default observations are RGB images, but RGBD, agent sensory data, and other information can also be used.

If you have previous versions of iGibson or Habitat already installed, we recommend to remove them and do clean installs.

iGibson Installation

Follow the official guide and install its assets and datasets.

For <MUSHROOM_RL PATH>/mushroom-rl/examples/igibson_dqn.py you need to run

python -m igibson.utils.assets_utils --download_assets
python -m igibson.utils.assets_utils --download_demo_data
python -m igibson.utils.assets_utils --download_ig_dataset

You can also use third party datasets.

The scene details are defined in a YAML file, that needs to be passed to the agent. See <IGIBSON PATH>/igibson/test/test_house.YAML for an example.

Habitat Installation

Follow the official guide and do a full install with habitat_baselines. Then you can download interactive datasets following this and this. If you need to download other datasets, you can use this utility.

Basic Usage of Habitat

When you create a Habitat environment, you need to pass a wrapper name and two YAML files: Habitat(wrapper, config_file, base_config_file).

  • The wrapper has to be among the ones defined in <MUSHROOM_RL PATH>/mushroom-rl/environments/habitat_env.py, and takes care of converting actions and observations in a gym-like format. If your task / robot requires it, you may need to define new wrappers.

  • The YAML files define every detail: the Habitat environment, the scene, the sensors available to the robot, the rewards, the action discretization, and any additional information you may need. The second YAML file is optional, and overwrites whatever was already defined in the first YAML.

    If you use YAMLs from habitat-lab, check if they define a YAML for BASE_TASK_CONFIG_PATH. If they do, you need to pass it as base_config_file to Habitat(). habitat-lab YAMLs, in fact, use relative paths, and calling them from outside its root folder will cause errors.

  • If you use a dataset, be sure that the path defined in the YAML file is correct, especially if you use relative paths. habitat-lab YAMLs use relative paths, so be careful with that. By default, the path defined in the YAML file will be relative to where you launched the python code. If your data folder is somewhere else, you may also create a symbolic link.

Rearrange Task Example

  • Download the ReplicaCAD datasets (--data-path data downloads them in the folder from where you are launching your code)
python -m habitat_sim.utils.datasets_download --uids replica_cad_dataset --data-path data
  • For this task we use <HABITAT_LAB PATH>/habitat_baselines/config/rearrange/rl_pick.yaml. This YAML defines BASE_TASK_CONFIG_PATH: configs/tasks/rearrange/pick.yaml, and since this is a relative path we need to overwrite it by passing its absolute path as base_config_file argument to Habitat().
  • Then, pick.yaml defines the dataset to be used with respect to <HABITAT_LAB PATH>. If you have not used --data-path argument with the previous download command, the ReplicaCAD datasets is now in <HABITAT_LAB PATH>/data and you need to make a link to it
ln -s <HABITAT_LAB PATH>/data/ <MUSHROOM_RL PATH>/mushroom-rl/examples/habitat
  • Finally, you can launch python habitat_rearrange_sac.py.

Navigation Task Example

  • Download and extract Replica scenes

    WARNING! The dataset is very large!

sudo apt-get install pigz
git clone https://github.com/facebookresearch/Replica-Dataset.git
cd Replica-Dataset
./download.sh replica-path
  • For this task we only use the custom YAML file pointnav_apartment-0.yaml.

  • DATA_PATH: "replica_{split}_apartment-0.json.gz" defines the JSON file with some scene details, such as the agent's initial position and orientation. The {split} value is defined in the SPLIT key.

    If you want to try new positions, you can sample some from the set of the scene's navigable points. After initializing a habitat environment, for example mdp = Habitat(...), run mdp.env._env._sim.sample_navigable_point().

  • SCENES_DIR: "Replica-Dataset/replica-path/apartment_0" defines the scene. As said before, this path is relative to where you launch the script, thus we need to link the Replica folder. If you launch habitat_nav_dqn.py from its example folder, run

ln -s <PATH TO>/Replica-Dataset/ <MUSHROOM_RL PATH>/mushroom-rl/examples/habitat
  • Finally, you can launch python habitat_nav_dqn.py.

Editable Installation

You can also perform a local editable installation by using:

pip install --no-use-pep517 -e .

To install also optional dependencies:

pip install --no-use-pep517 -e .[all]

How to set and run and experiment

To run experiments, MushroomRL requires a script file that provides the necessary information for the experiment. Follow the scripts in the "examples" folder to have an idea of how an experiment can be run.

For instance, to run a quick experiment with one of the provided example scripts, run:

python3 examples/car_on_hill_fqi.py

Cite MushroomRL

If you are using MushroomRL for your scientific publications, please cite:

@article{JMLR:v22:18-056,
    author  = {Carlo D'Eramo and Davide Tateo and Andrea Bonarini and Marcello Restelli and Jan Peters},
    title   = {MushroomRL: Simplifying Reinforcement Learning Research},
    journal = {Journal of Machine Learning Research},
    year    = {2021},
    volume  = {22},
    number  = {131},
    pages   = {1-5},
    url     = {http://jmlr.org/papers/v22/18-056.html}
}

How to contact us

For any question, drop an e-mail at [email protected].

Follow us on Twitter @Mushroom_RL!

Popular Reinforcement Learning Projects
Popular Deep Learning Projects
Popular Machine Learning Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.
Python
Shell
Deep Learning
Pytorch
Reinforcement Learning
Gym
Rl
Dqn
Deep Reinforcement Learning
Openai Gym
Ddpg
Mujoco
Q Learning
Trpo