Description
The goal of the project is to explore and get an experience of building reinforcement learning environments, following the OpenAI Gym standards. The project consists of building deterministic and stochastic environments that are based on Markov decision process, and applying a tabular method to solve them.
Part 1 – Build a deterministic environment
Define a deterministic environment, where P(s0,r|s,a) = {0,1}. It has to have more than one state and more than one action.
Environment requirements:
- Min number of states: 4
- Min number of actions: 2
- Min number of rewards: 3
Environment definition should follow OpenAI Gym structure, that includes the following basic methods:
def __init__:
# Initializes the class
# Define action and observation space
def step:
# Executes one timestep within the environment
# Input to the function is an action
def reset:
# Resets the state of the environment to an initial state
def render:
# Visualizes the environment
# Any form like vector representation or visualizing using matplotlib will be sufficient
Part 2 – Build a stochastic environment
Define a stochastic environment, where Ps0,r P(s0,r|s,a) = 1. A modified version of the environment defined in Part 1 should be used.
Part 3 – Implement tabular method
Apply a tabular method to solve environments, that were built in Part 1 and Part 2.
Tabular methods options:
- Dynamic programming
- Q-learning
- SARSA
- TD(0)
- Monte Carlo
1 Deliverables
There are two parts in your submission:
1.1 Report
Report should be delivered as a pdf file, NIPS template is a suggested report structure to follow.
In your report:
- Describe the deterministic/stochastic environments, that were defined (set of actions/states/rewards, main objective, etc)
- What is the differences between the deterministic/stochastic environments?
- Show your transition-probability matrix for stochastic environment.
- Discuss the main components of the RL environment.
- Show your results after applying an algorithm to solve deterministic and stochastic types of problems, that might include plots and your interpretation of the results.
- Explain tabular method that was used to solve the problems.
