Skip to content

Code for the paper Learning Stable Normalizing-Flow Control for Robotic Manipulation, IEEE ICRA, 2021

License

Notifications You must be signed in to change notification settings

shbz80/normflow_policy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

65 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Learning Stable Normalizing-Flow Control for Robotic Manipulation

This repo is the code base for the paper Learning Stable Normalizing-Flow Control for Robotic Manipulation, Khader, S. A., Yin, H., Falco, P., & Kragic, D. (2021), IEEE International Conference on Robotics and Automation (ICRA). [IEEE] [arXiv]

https://www.youtube.com/watch?v=Vl8HFq-lk94&t=1s

Paper abstract

Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does not exist in general, especially one that scales to complex manipulation tasks. We contribute towards closing this gap by introducing normalizing-flow control structure, that can be deployed in any latest deep RL algorithms. While stable exploration is not guaranteed, our method is designed to ultimately produce deterministic controllers with provable stability. In addition to demonstrating our method on challenging contact-rich manipulation tasks, we also show that it is possible to achieve considerable exploration efficiency–reduced state space coverage and actuation efforts– without losing learning efficiency.

Prerequisites

  • garage Deep reinforcement learning toolkit
  • PyTorch Deep learning framework
  • MuJoCo Physics simulator

Run experiments

2D Block-Insertion

  • Run block2D_nfppo_garage.py for the proposed method
  • Run block2D_ppo_torch_garage.py for the baseline method

YuMi Peg-In-Hole

  • Run yumipeg_nfppo_torch_garage.py for the proposed method
  • Run yumipeg_ppo_torch_garage.py for the baseline method

About

Code for the paper Learning Stable Normalizing-Flow Control for Robotic Manipulation, IEEE ICRA, 2021

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages