Reinforcement Learning & other bedtime stories in PyTorch. This repository is pytorch implementation of the Intro RL course & Advanced RL course. The name cherry
is derived from Yann Lecun's statement regarding RL being the cherry on the ML cake. For training/playing the agent within conda env, please install the external dependencies. Not to be confused with a package of similar name.
- Installation
- Sample usage
- Dependencies (Conda only)
- Architectures & Agents
- Environments
- Design principle
- Reproducibility
Tested with ubuntu:18.04 only
# After installed external dependencies
conda create -n cherry python=3.6
conda activate cherry
pip install -r requirements.txt
pip install -e .
# CPU
docker pull moabitcoin/cherry-pytorch-cpu
docker run moabitcoin/cherry-pytorch-cpu cherry --help
# GPU
docker pull moabitcoin/cherry-pytorch-gpu
docker run moabitcoin/cherry-pytorch-gpu cherry --help
# CPU
make install dockerfile=Dockerfile.cpu dockerimage=moabitcoin/cherry-pytorch-cpu
make run dockerimage=moabitcoin/cherry-pytorch-cpu
# GPU
make install dockerfile=Dockerfile.gpu dockerimage=moabitcoin/cherry-pytorch-gpu
make gpu dockerimage=moabitcoin/cherry-pytorch-gpu
# Inside the virtual environment / Docker
cherry --help
cherry train --help
usage: cherry train [-h] -c CONFIG_FILE [-d {gpu,cpu}] [-l {info,debug}]
optional arguments:
-h, --help show this help message and exit
-c CONFIG_FILE, --config_file CONFIG_FILE
Path to Config file (default: None)
-d {gpu,cpu} Device to run the train/test (default: gpu)
-l {info,debug}, --log {info,debug}
Set verbosity for the logger (default: info)
cherry train -c configs/control.yaml -d cpu
# <model_dest> in configs/control.yaml
cherry play -c <model_dest>/control-<commit-gitsha>.yaml -d cpu -m <model_dest>/agent-final-<commit-gitsha>.pth
# <state_dest> in configs/control.yaml
vlc <state_dest>
We support 4 feedforward architectures within cherry
. We plan to expand the list of architectures to include Recurrent/Transformer/Memory architectures. If your personal model flavour is missing, please open an issue with links to architecture details.
We support 4 Agents within cherry
. We plan to expand the list of agents to include TRPO/PPO. Please feel free to make an agent request by opening an issue with useful links to publication(s)/existing implementation.
We support 4 environments within cherry
. This list is planned to be expanded to include Robotics and other 3rd party environments.
Install dependencies as outlined here
sudo apt-get install libcanberra-gtk-module
pip install vizdoom
# If on a headless machine
export DISPLAY=:0
https://github.com/openai/gym/blob/master/docs/environments.md#atari
Needed for Deepmind styled pre-processing wrappers. Needs TF>=1.14 🤦♀️
pip install tensorflow_gpu==1.14
https://github.com/openai/baselines/tree/master#installation
https://github.com/openai/gym/tree/0.15.4#rendering-on-a-server
xvfb-run -s "-screen 0 1400x900x24" bash