Our project explores the algorithms of reinforcement learning (RL) used in game playing. We study several RL algorithms and implement them in a car racing game. The RL methods include, but not limited to, DQN, Double DQN, and Dueling DQN. After the RL methods been implemented, we make comparison between these results and analyze the characteristics of each method.
Playing Atari with Deep Reinforcement Learning
Human-level control through deep reinforcement learning
Deep Reinforcement Learning with Double Q-learning
Dueling Network Architectures for Deep Reinforcement Learning
OpenAI Car Racing-v0
We try DQN, Double DQN(DDQN) and dueling DQN. Please refer to the presentation for detailed algorithm explanation. Also, you should have some basic knowledge on Reinforcement Learning and Q-learning.
-
pip install -r requirements.txtnecessary module:
tensorflow,pygame,gym,Box2D,VC++ 14.0... -
In DQN/DDQN/dueling DQN folder, run
python car_runner_main.py -
If you'd like to utilize the trained model, switch
load_checkpoint = Truein python car_runner_main.py -
On CPU, it takes about 8 hours to get a well-trained model.
The DQN,DDQN and dueling DQN have similar structures. Take DQN for example:
DQN/car_runner_main.py - main entrance, the executable file
DQN/dqn/agent.py - DQN model
DQN/dqn/experience_replay.py - experience replay
data/plot.py - plot figures
| DQN | DDQN | Dueling DQN | Human |
|---|---|---|---|
| 755 | 784.95 | 737.35 | 216.35 |
The dueling DQN doesn't perform as well as we expected. Some speculated reasons are in the presentation.
Oct.15,2018 Project Proposal
Nov.19,2018 Project Progress
Dec.12,2018 Presentation
Dec.14,2018 [Final Report] (Coming Soon...)