Awesome Open Source
Awesome Open Source

# In this repo

1. Python replication of all the plots from Reinforcement Learning: An Introduction
2. Solution for all of the exercises
3. Anki flashcards summary of the book

## 1. Replicate all the figures

To reproduce a figure, say figure 2.2, do:

```cd chapter2
python figures.py 2.2
```

### Chapter 4

1. Figure 4.2: Jack’s car rental problem (value function, policy)
2. Figure 4.3: The solution to the gambler’s problem (value function, policy)

## 2. Solution to all of the exercises (text answers)

To reproduce the results of an exercise, say exercise 2.5 do:

```cd chapter2
python figures.py ex2.5
```

### Chapter 4

1. Exercise 4.7: Modified Jack's car rental problem (value function, policy)

2. Exercise 4.9: Gambler’s problem with ph = 0.25 (value function, policy) and ph = 0.55 (value function, policy)

### Chapter 5

1. Exercise 5.14: Modified MC Control on the racetrack (1, 2)

## Appendix

```numpy
matplotlib
seaborn
```

### Credits

All of the code and answers are mine, except for mountain car's tile coding (url in the book).

This README is inspired from ShangtongZhang's repo.

### Design choices

1. All of the chapters are self-contained.
2. The environments use a gym-like API with methods:
```s = env.reset()
s_p, r, d, dict = env.step(a)
```

### How long did it take

The entire thing (plots, exercises, anki cards (including reviewing)) took about 400h of focused work.

Get A Weekly Email With Trending Projects For These Topics
No Spam. Unsubscribe easily at any time.
python (55,580
reinforcement-learning (591
matplotlib (127
anki (49