|Project Name||Stars||Downloads||Repos Using This||Packages Using This||Most Recent Commit||Total Releases||Latest Release||Open Issues||License||Language|
|🏰 An exciting game of programming and Artificial Intelligence|
|Universe||7,191||51||5 years ago||12||March 05, 2017||mit||Python|
|Universe: a software platform for measuring and training an AI's general intelligence across the world's supply of games, websites and other applications.|
|Recastnavigation||5,539||12 days ago||122||zlib||C++|
|Navigation-mesh Toolset for Games|
|Openttd||4,886||3 hours ago||322||other||C++|
|OpenTTD is an open source simulation game based upon Transport Tycoon Deluxe|
|🦄 An Artificial Inteligence to teach Google's Dinosaur to jump cactus|
|Behaviac||2,594||2 months ago||93||other||C#|
|behaviac is a framework of the game AI development, and it also can be used as a rapid game prototype design tool. behaviac supports the behavior tree, finite state machine and hierarchical task network(BT, FSM, HTN)|
|Rlcard||2,181||1||13 days ago||38||March 23, 2022||53||mit||Python|
|Reinforcement Learning / AI Bots in Card (Poker) Games - Blackjack, Leduc, Texas, DouDizhu, Mahjong, UNO.|
|Behaviortree.cpp||2,041||7 days ago||43||mit||C++|
|Behavior Trees Library in C++. Batteries included.|
|Awesome Unity3d||1,687||17 days ago||2||unlicense|
|A categorized collection of awesome opensource unity3d repos|
|A curated list of project tutorials for project-based learning.|
This repository has been deprecated in favor of the Retro (https://github.com/openai/retro) library. See our Retro Contest (https://blog.openai.com/retro-contest) blog post for detalis.
Universe is a software
platform for measuring and training an AI's general intelligence
across the world's supply of games, websites and other
applications. This is the
universe open-source library, which
provides a simple Gym
interface to each Universe environment.
Universe allows anyone to train and evaluate AI agents on an extremely wide range of real-time, complex environments.
Universe makes it possible for any existing program to become an OpenAI Gym environment, without needing special access to the program's internals, source code, or APIs. It does this by packaging the program into a Docker container, and presenting the AI with the same interface a human uses: sending keyboard and mouse events, and receiving screen pixels. Our initial release contains over 1,000 environments in which an AI agent can take actions and gather observations.
Additionally, some environments include a reward signal sent to the agent, to guide reinforcement learning. We've included a few hundred environments with reward signals. These environments also include automated start menu clickthroughs, allowing your agent to skip to the interesting part of the environment.
We'd like the community's help to grow the number of available environments, including integrating increasingly large and complex games.
The following classes of tasks are packaged inside of publicly-available Docker containers, and can be run today with no work on your part:
We've scoped out integrations for many other games, including completing a high-quality GTA V integration (thanks to Craig Quiter and NVIDIA), but these aren't included in today's release.
Contents of this document
We currently support Linux and OSX running Python 2.7 or 3.5.
We recommend setting up a conda environment before getting started, to keep all your Universe-related packages in the same place.
To get started, first install
git clone https://github.com/openai/universe.git cd universe pip install -e .
If this errors out, you may be missing some required packages. Here's the list of required packages we know about so far (please let us know if you had to install any others).
On Ubuntu 16.04:
pip install numpy sudo apt-get install golang libjpeg-turbo8-dev make
On Ubuntu 14.04:
sudo add-apt-repository ppa:ubuntu-lxc/lxd-stable # for newer golang sudo apt-get update sudo apt-get install golang libjpeg-turbo8-dev make
You might need to install Command Line Tools by running:
pip install numpy incremental brew install golang libjpeg-turbo
The majority of the environments in Universe run inside Docker
containers, so you will need to install Docker (on OSX, we
recommend Docker for Mac). You should be able to
docker ps and get something like this:
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
The above instructions result in an agent that runs as a regular python process in your OS, and launches docker containers as needed for the remotes. Alternatively, you can build a docker image for the agent and run it as a container as well. You can do this in any operating system that has a recent version of docker installed, and the git client.
To get started, clone the
git clone https://github.com/openai/universe.git cd universe
Build a docker image, tag it as 'universe':
docker build -t universe .
This may take a while the first time, as the docker image layers are pulled from docker hub.
Once the image is built, you can do a quick run of the test cases to make sure everything is working:
docker run --privileged --rm -e DOCKER_NET_HOST=172.17.0.1 -v /var/run/docker.sock:/var/run/docker.sock universe pytest
Here's a breakdown of that command:
docker run- launch a docker container
--rm- delete the container once the launched process finishes
-e DOCKER_NET_HOST=172.17.0.1- tells the universe remote (when launched) to make its VNC connection back to this docker-allocated IP
-v /var/run/docker.sock:/var/run/docker.sock- makes the docker unix socket from the host available to the container. This is a common technique used to allow containers to launch other containers alongside itself.
universe- use the imaged named 'universe' built above
pytest- run 'pytest' in the container, which runs all the tests
At this point, you'll see a bunch of tests run and hopefully all pass.
To do some actual development work, you probably want to do another volume map from the universe repo on your host into the container, then shell in interactively:
docker run --privileged --rm -it -e DOCKER_NET_HOST=172.17.0.1 -v /var/run/docker.sock:/var/run/docker.sock -v (full path to cloned repo above):/usr/local/universe universe python
As you edit the files in your cloned git repo, they will be changed in your docker container and you'll be able to run them in python.
Note if you are using docker for Windows, you'll need to enable the relevant shared drive for this to work.
universe, you may see
warningmessages. These lines occur when installing numpy and are normal.
go versionof at least 1.5. Ubuntu 14.04 has an older Go, so you'll need to upgrade your Go installation.
A Universe environment is similar to any other Gym environment:
the agent submits actions and receives observations using the
Internally, a Universe environment consists of two pieces: a client and a remote:
The code in this repository corresponds to the client side of the Universe environments. Additionally, you can freely access the Docker images for the remotes. We'll release the source repositories for the remotes in the future, along with tools to enable users to integrate new environments. Please sign up for our beta if you'd like early access.
Now that you've installed the
universe library, you should make
sure it actually works. You can paste the example below into your
python REPL. (You may need to press enter an extra time to make
while loop is executing.)
import gym import universe # register the universe environments env = gym.make('flashgames.DuskDrive-v0') env.configure(remotes=1) # automatically creates a local docker container observation_n = env.reset() while True: action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n] # your agent here observation_n, reward_n, done_n, info = env.step(action_n) env.render()
The example will instantiate a client in your Python process,
automatically pull the
and will start that image as the remote. (In our remotes documentation page, we explain other ways you can run
It will take a few minutes for the image to pull the first time. After that, if all goes well, a window like the one below will soon pop up. Your agent, which is just pressing the up arrow repeatedly, is now playing a Flash racing game called Dusk Drive. Your agent is programmatically controlling a VNC client, connected to a VNC server running inside of a Docker container in the cloud, rendering a headless Chrome with Flash enabled:
You can even connect your own VNC client to the environment, either
just to observe or to interfere with your agent. Our
gym-core images conveniently bundle a browser-based VNC
client, which can be accessed at
http://localhost:15900/viewer/?password=openai. If you're on Mac,
connecting to a VNC server is as easy as running:
(If using docker-machine, you'll need to replace "localhost" with the
IP address of your Docker daemon, and use
openai as the password.)
So we managed to run an agent, what did all the code actually mean? We'll go line-by-line through the example.
universe, which registers all the Universe environments.
import gym import universe # register the universe environments
gymlooks up the registration for
flashgames.DuskDrive-v0, and instantiates a VNCEnv object which has been wrapped to add a few useful diagnostics and utilities. The
VNCEnvobject is the client part of the environment, and it is not yet connected to a remote.
env = gym.make('flashgames.DuskDrive-v0')
configure()connects the client to a remote environment server. When called with
configure(remotes=1), Universe will automatically create a Docker image running locally on your computer. The local client connects to the remote using VNC. (More information on client-remote communication can be found in the page on universe internal communication protocols. More on configuring remotes is at remotes.)
When starting a new environment, you call
environments run in real-time, rather than stepping synchronously
with the agent's actions, so
reset is asynchronous and returns
immediately. Since the environment will not have waited to finish
connecting to the VNC server before returning, the initial observations
reset will be
None to indicate that there is
not yet a valid observation.
Similarly, the environment keeps running in the background even
if the agent does not call
env.step(). This means that an agent
that successfully learns from a Universe environment cannot take
"thinking breaks": it must keep sending actions to the environment at all times.
Additionally, Universe introduces the vectorized Gym
API. Rather than controlling a single environment at a time, the agent
can control a fixed-size vector of
n environments, each with its
own remote. The return value from
reset is therefore a vector
of observations. For more information, see the separate page on
observation_n = env.reset()
step() call, the agent submits a vector of actions; one for
each environment instance it is controlling. Each VNC action is a
list of events; above, each action is the single event "press the
ArrowUp key". The agent could press and release the key in one
action by instead submitting
[('KeyEvent', 'ArrowUp', True),
('KeyEvent', 'ArrowUp', False)] for each observation.
In fact, the agent could largely have the same effect by just
('KeyEvent', 'ArrowUp', True) once and then calling
env.step([ for ob in observation_n]) thereafter, without ever
releasing the key using
('KeyEvent', 'ArrowUp', False). The
browser running inside the remote would continue to statefully
represent the arrow key as being pressed. Sending other unrelated
keypresses would not disrupt the up arrow keypress; only explicitly
releasing the key would cancel it. There's one slight subtlety:
when the episode resets, the browser will reset, and will forget
about the keypress; you'd need to submit a new
ArrowUp at the
start of each episode.
action_n = [[('KeyEvent', 'ArrowUp', True)] for ob in observation_n]
After we submit the action to the environment and render one frame,
step() returns a list of observations, a list of rewards, a
list of "done" booleans indicating whether the episode has ended,
and then finally an info dictionary of the form
...]}, in which you can access the info for environment
info message contains useful diagnostic
information, including latency data, client and remote timings,
VNC update counts, and reward message counts.
observation_n, reward_n, done_n, info = env.step(action_n) env.render()
stepin what looks like a busy loop. In reality, there is a Throttle wrapper on the client which defaults to a target frame rate of 60fps, or one frame every 16.7ms. If you call it more frequently than that,
stepwill sleep with any leftover time.
We are using pytest for tests. You can run them via:
pytest --help for useful options, such as
pytest -s (disables output capture) or
pytest -k <expression> (runs only specific tests).
More documentation not covered in this README can be found in the doc folder of this repository.
If you encounter a problem that is not addressed in this README page or in the extra docs, then try our wiki page of solutions to common problems - and add to it if your solution isn't there!
You can also search through the issues on this repository and our discussion board to see if another user has posted about the same problem or to ask for help from the community.
If you still can't solve your problem after trying all of the above steps, please post an issue on this repository.