This is a participant repository for the Robot Air Hockey Challenge 2023.
drl_air_hockey_self_play.mp4
The implemented approach focuses on applying a model-based deep reinforcement learning algorithm DreamerV3 to acquire a policy capable of playing air hockey with continuous observations and actions.
- Additional details about the approach are presented in the report here.
- Models of pre-trained agents can be downloaded from here.
As a quick test, you can try evaluating a pre-trained agent in a self-play mode by running .docker/run.bash
directly via curl
as shown below.
curl -sSfL "https://raw.githubusercontent.com/AndrejOrsula/drl_air_hockey/main/.docker/run.bash" | DOCKER_RUN_OPTS="--rm" bash -s -- drl_air_hockey/scripts/eval_dreamerv3.py -r
Install air_hockey_challenge
and drl_air_hockey
Python modules with pip
.
pip3 install git+https://github.com/AndrejOrsula/air_hockey_challenge.git
pip3 install git+https://github.com/AndrejOrsula/drl_air_hockey.git
Warning
Not all dependencies have their versions pinned, e.g. transitive dependencies of dreamerv3
. Therefore, the functionality of this repository may be affected in future builds. Consider using the pre-built Docker image for a stable environment.
To train a new agent, you can run the included train_dreamerv3.py
script.
scripts/train_dreamerv3.py
To configure the training process, consider modifying any of these files directly:
config.py
for algorithm-specific parametersrewards.py
andtournament_agent_strategies.py
for the desired behavior of the agentagents
for additional changes to the observation and action spaces of the agent
Once you are satisfied with the training progress, you can evaluate the agent by adjusting the included eval_dreamerv3.py
script.
scripts/eval_dreamerv3.py
To install Docker on your system, you can run
.docker/host/install_docker.bash
to configure Docker with NVIDIA GPU support..docker/host/install_docker.bash
To build a new Docker image from Dockerfile
, you can run .docker/build.bash
as shown below. Note that only the pre-built Docker image already contains models of pre-trained agents.
.docker/build.bash ${TAG:-latest} ${BUILD_ARGS}
To run the Docker container, you can use .docker/run.bash
as shown below.
.docker/run.bash ${TAG:-latest} ${CMD:-bash}
To run the Docker container in a development mode (source code mounted as a volume), you can use .docker/dev.bash
as shown below.
.docker/dev.bash ${TAG:-latest} ${CMD:-bash}
As an alternative, VS Code users familiar with Dev Containers can modify the included .devcontainer/devcontainer.json
to their needs. For convenience, .devcontainer/open.bash
script is available to open this repository as a Dev Container in VS Code.
.devcontainer/open.bash
To join a running Docker container from another terminal, you can use .docker/join.bash
as shown below.
.docker/join.bash ${CMD:-bash}
@article{orsula2023learning,
title = {{Learning to Play Air Hockey with Model-Based Deep Reinforcement Learning}},
author = {Andrej Orsula},
journal = {NeurIPS 2023 --- The Robot Air Hockey Challenge},
year = {2023},
}