This repository contains a set of environments and algorithms for transferring reinforcement learning policies across agents.
The code is based on the original repository for the paper Hierarchically Decoupled Imitation for Morphological Transfer. The code has been cleaned up in comparison to the original repo and new environments and algorithm variants have been added.
Warning: This repository has both environment and algorithmic differences from that of the original publication, and hyper-parameter sweeps have not yet been run. As I have not run every experiment configuration, there may be bugs. For exact comparisons with the aforementioned publication, please use its repository until this one has been benchmarked.
- Support for training low level policies with Hindsight Experience Replay (HER)
- New Sawyer Robot environments courtesy of Hardware Conditioned Policies by Tao Chen, Adithyavairavan Murali, and Abhinav Gupta.
- Cleaner code and algorithm implementations
- Install the required python dependencies using the requirements.txt file. Note that if you want to use GPU, you must swap tensorflow for tensorflow-gpu
- Install the bot_transfer package by running
pip install -e .from the root of the repository. This will use thesetup.pyfile.
Example experiments are included in the configs directory. Run them from the root directory of the repo. Code will output to the data directory. To train low or high level policies using existing models, save those models to a folder called low_levels and high_levels respectively. The code will search for models in these directories to finetune from.
To see a list of all environments, look at the bot_transfer/envs/__init__.py file.
To understand how to addd your own environment, look at bot_transfer/envs/base.py.