A Framework to Modify Simulations Online

A simple framework to change visual and physical parameters of various simulations without reloading the simulation and through a standardized interface.

Changing parameters online is especially useful for domain randomization and curriculum learning in robotics research. Thus several algorithms for domain randomization are directly included (Uniform Domain Randomization, Automatic Domain Randomization).

Parameters can be changed directly with the simulation attributes or if applicable over corresponding OpenAI Gym wrapper.

Currently only Mujoco through the mujoco-py package is supported. PyBullet, Gazebo and other simulations will follow soon.

Furthermore, online modification of built-in Python variables of the simulation class is supported.

Installation

Clone the repo and cd into it:

git clone https://github.com/MoritzTaylor/simmod.git
cd simulation-modification-framework

Install simmod package:

pip install -e .

Structure

Each simulation can be changed through a handful of modifiers. Every modifier changes a specific category of parameters of the simulation (i.e. material of the object, body properties, etc.). Modifiers can be configured with configurations which specify the parameters/objects to change. The elements of the configurations are stored in parameterization objects.

Usage

Usage is simple and can be performed in 3 steps which is generally structured as follows:

sim = Simulation(...)  # Initialize the simulation
mod = ModifierClass(sim)  # Initialize the modifier
mod.set_property(object_name, new_value)  # Change the property to the new value
sim.step()  # Usually the property gets changed in the next simulation step

Functions like set_property are individual for each modifier and can be retrieved by the modifier property standard_setters. The simulation objects which can be changed by the modifier can be retrieved similarly using the modifier property names .

Using a domain randomization algorithm in addition is just as simple:

alg = Algorithm(mod1, mod2, mod3, ...)  # An algorithm gets one or more modifiers as input
alg.step()  # Change the parameters defined in the modifiers in one step

An example looks like this:

from mujoco_py import load_model_from_xml, MjSim, MjViewer

from simmod import load_yaml
from simmod.algorithms import UniformDomainRandomization
from simmod.modification.mujoco import MujocoBodyModifier

# Create and load the simulation
model = load_model_from_xml(MODEL_STRING)
sim = MjSim(model)
sim.step()

# Define modifier and algorithm for randomization
config = load_yaml('./examples/assets/algorithm_example.yaml')
mod_body = MujocoBodyModifier(sim=sim, config=config)
alg = UniformDomainRandomization(mod_body)

# Run algorithm and simulation
alg.step()
sim.step()

More examples can be found in the examples directory.

Configuration

As you can see in the example above, all modifiers can be configured with configurations so that specific objects with individual upper and lower value bounds can be defined. That is especially useful when using domain randomization algorithms to define their sampling range.

Configurations are stored as dictionaries and can be externally loaded from yaml files.

Modifiers

Each modifier is built on a configuration. If no configuration is given it is taken from a standard configuration file in simmod/modification/data/ which usually defines the object parameter bounds at infinity or zero. This might make the simulation unstable. Therefore, it is recommended to define a configuration for each modifier individual to your specific application. Each property of the modifier (i.e. mass) holds the specific objects that should be changed.

Each configuration part can include multiple options:

• For each of these objects the corresponding randomization distribution can be specified under distribution. Available distributions are "uniform", "normal" and "loguniform". The default one is "uniform" if no other distribution is specified.

• The input ranges for each distribution must be described by a tuple (one tuple for each dimension) at the attribute values (upper and lower bound for the uniforms and mean and standard deviation for the normal).

Example:

---
mass:
  pole:
    distribution: "uniform"
    values:
      - [0.018, 0.03]  # [lower bound, upper bound]
  arm:
    distribution: "normal"
    values:
      - [0.05175, 0.043125]  # [mean, standard deviation]

This configuration only changes the mass of the objects named pole and arm in the simulation when using a modifier which changes the property mass (usually modifier which change bodies).

If a configuration is given, only those objects which are defined in this configuration are used by the domain randomization algorithms. If all other objects should be changed in a specific range this can be specified via an additional object called default:

---
mass:
  arm:
    distribution: "uniform"
    values:
      - [0.05175, 0.08625]
  default:
    distribution: "uniform"
    values:
      - [0, .inf] 
    # lower and upper bound of each object except for object with name 'arm'

Therefore, it is important that no object in the simulation is named default to avoid unexpected behavior.

Experiments

To run many experiments in one single run, configuration files for modifiers can be combined into a single larger configuration file:

---
experiment-1:
    MujocoBodyModifier:
        mass:
            pole:
              values:
                - [0.018, 0.03]

experiment-2:
    MujocoBodyModifier:
        mass:
            pole:
                values:
                  - [0.018, 0.03]
            arm:
              values:
                - [0.05175, 0.08625]
    MujocoJointModifier:
        damping:
            base_motor:
              values:
                - [ 0.000373125, 0.000621875 ]
            arm_pole:
              values:
                - [ 0.0000748875, 0.0001248125 ]

Those experiment configurations can be loaded with special utility functions:

from simmod.utils.experiment_utils import GymExperimentScheduler

exp_scheduler = GymExperimentScheduler()
exp_scheduler.load_experiments("./examples/assets/experiment_example.yaml")

for experiment in iter(experiments):
    modifiers = exp_scheduler.create_modifiers(experiment.configurations, env)
    ...

OpenAI Gym Wrapper

All modifiers and algorithms are combined in OpenAI Gym wrapper. Those can be used in place of the before used algorithm:

import gym
from simmod import load_yaml
from simmod.wrappers import UDRMujocoWrapper
from simmod.modification.mujoco import MujocoBodyModifier

# Create the environment as you would normally do
env = gym.make('InvertedPendulum-v2')

# Define modifier and algorithm for randomization
config = load_yaml('./examples/assets/algorithm_example.yaml')
# env.sim is the Mujoco simulation in the environment class
mod_body = MujocoBodyModifier(sim=env.sim, config=config)
env = UDRMujocoWrapper(env, mod_body)

# Run algorithm and simulation
env.step()

Software used

MuJoCo

Some of the modifiers use MuJoCo (multi-joint dynamics in contact) physics simulator, which is proprietary and requires binaries and a license (temporary 30-day license can be obtained from www.mujoco.org). To run MuJoCo with Python the mujoco-py package is needed. Instructions on setting up mujoco-py can be found here.