Project1 Boptest Gym

Alternatives To Project1 Boptest Gym
Project NameStarsDownloadsRepos Using ThisPackages Using ThisMost Recent CommitTotal ReleasesLatest ReleaseOpen IssuesLicenseLanguage
Baselines14,94937216 days ago6February 26, 2018497mitPython
OpenAI Baselines: high-quality implementations of reinforcement learning algorithms
Stable Baselines36,99579a day ago80November 17, 202381mitPython
PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.
Captum4,299592 days ago10December 05, 2023188bsd-3-clausePython
Model interpretability and understanding for PyTorch
a year ago30mitPython
A Platform for Many-agent Reinforcement Learning
Imitation1,015110 hours ago9October 31, 202380mitPython
Clean PyTorch implementations of imitation and reward learning algorithms
4 years ago5mitJupyter Notebook
lagom: A PyTorch infrastructure for rapid prototyping of reinforcement learning algorithms.
9 months ago1Java
Checklist of videos and exercises that you can follow if you want to get a good baseline for Data Structures and Algorithms.
Stable Baselines221
a year ago1mitPython
Mirror of Stable-Baselines: a fork of OpenAI Baselines, implementations of reinforcement learning algorithms
8 years ago3Python
Automatic keyword extraction - no alchemy required!
H Baselines154
2 years ago8mitPython
A repository of high-performing hierarchical reinforcement learning models and algorithms.
Alternatives To Project1 Boptest Gym
Select To Compare

Alternative Project Comparisons


BOPTESTS-Gym is the OpenAI-Gym environment for the BOPTEST framework. This repository accommodates the BOPTEST API to the OpenAI-Gym convention in order to facilitate the implementation, assessment and benchmarking of reinforcement learning (RL) algorithms for their application in building energy management. RL algorithms from the Stable-Baselines 3 repository are used to exemplify and test this framework.

The environment is described in this paper.


  • contains the core functionality of this Gym environment.
  • environment.yml contains the dependencies required to run this software.
  • /examples contains prototype code for the interaction of RL algorithms with an emulator building model from BOPTEST.
  • /testing contains code for unit testing of this software.

Quick-Start (using BOPTEST-Service)

BOPTEST-Service allows to directly access BOPTEST test cases in the cloud, without the need to run it locally. Interacting with BOPTEST-Service requires less configuration effort but is considerably slower because of the communication overhead between the agent and the test case running in the cloud. Use this approach when you want to quickly check out the functionality of this repository.

  1. Create a conda environment from the environment.yml file provided (instructions here).
  2. Check out the boptest-gym-service branch and run the example below that uses the Bestest hydronic case with a heat-pump and the DQN algorithm from Stable-Baselines:
from boptestGymEnv import BoptestGymEnv, NormalizedObservationWrapper, DiscretizedActionWrapper
from stable_baselines3 import DQN

# url for the BOPTEST service. 
url = '' 

# Decide the state-action space of your test case
env = BoptestGymEnv(
        url                  = url,
        testcase             = 'bestest_hydronic_heat_pump',
        actions              = ['oveHeaPumY_u'],
        observations         = {'time':(0,604800),
        predictive_period    = 24*3600, 
        regressive_period    = 6*3600, 
        random_start_time    = True,
        max_episode_length   = 24*3600,
        warmup_period        = 24*3600,
        step_period          = 3600)

# Normalize observations and discretize action space
env = NormalizedObservationWrapper(env)
env = DiscretizedActionWrapper(env,n_bins_act=10)

# Instantiate an RL agent
model = DQN('MlpPolicy', env, verbose=1, gamma=0.99,
            learning_rate=5e-4, batch_size=24, 
            buffer_size=365*24, learning_starts=24, train_freq=1)

# Main training loop

# Loop for one episode of experience (one day)
done = False
obs, _ = env.reset()
while not done:
  action, _ = model.predict(obs, deterministic=True) 
  obs,reward,terminated,truncated,info = env.step(action)
  done = (terminated or truncated)

# Obtain KPIs for evaluation

Quick-Start (running BOPTEST locally)

Running BOPTEST locally is substantially faster

  1. Create a conda environment from the environment.yml file provided (instructions here).
  2. Run a BOPTEST case with the building emulator model to be controlled (instructions here).
  3. Check out the master branch of this repository and run the example above replacing the url to be url = '' and avoiding the testcase argument to the BoptestGymEnv class.

Quick-Start (running BOPTEST locally in a vectorized environment)

To facilitate the training and testing process, we provide scripts that automate the deployment of multiple BOPTEST instances using Docker Compose and then train an RL agent with a vectorized BOPTEST-gym environment. The deployment dynamically checks for available ports, generates a Docker Compose YAML file, and takes care of naming conflicts to ensure smooth deployment. Running a vectorized environment allows you to deploy as many BoptestGymEnv instances as cores you have available for the agent to learn from all of them in parallel (see here for more information, we specifically use SubprocVecEnv). This substantially speeds up the training process.


  1. Specify the BOPTEST root directory either by passing it as a command-line argument or by defining the boptest_root variable at the beginning of the script The script prioritizes the command-line argument if provided. Users are allowed to change the Start Port number and Total Services as needed.

Example using command-line argument:

python absolute_boptest_root_dir
  1. Train an RL agent with parallel learning with the vectorized BOPTEST-gym environment. See /examples/ for an example on how to do so.

Versioning and main dependencies

Current BOPTEST-Gym version is v0.5.0 which is compatible with BOPTEST v0.5.0 (BOPTEST-Gym version should always be even with the BOPTEST version used). The framework has been tested with gymnasium==0.28.1 and stable-baselines3==2.0.0. You can see testing/Dockerfile for a full description of the testing environment.

Citing the project

Please use the following reference if you used this repository for your research.

	author = {Javier Arroyo and Carlo Manna and Fred Spiessens and Lieve Helsen},
	title = {{An OpenAI-Gym environment for the Building Optimization Testing (BOPTEST) framework}},
	year = {2021},
	month = {September},
	booktitle = {Proceedings of the 17th IBPSA Conference},
	address = {Bruges, Belgium},

Popular Algorithms Projects
Popular Baseline Projects
Popular Computer Science Categories
Related Searches

Get A Weekly Email With Trending Projects For These Categories
No Spam. Unsubscribe easily at any time.