Evolutionary Reinforcement Learning for OpenAI Gym

Implementation of Augmented-Random-Search for OpenAI Gym environments in Python. Performance is defined as the sample efficiency of the algorithm i.e how good is the average reward after using x episodes of interaction in the environment for traning. The paper can be found here: Simple random search provides a competitive approach to reinforcement learning

Augmented-Random-Search (ARS)

ARS is an Evolutionary Strategy where the policy is linear with weights w_p

Given an observation s_t an action a_t is chosen by:

Continuous case:
a_t = w_p * s_t

Discrete case:
a_t = softmax(w_p * s_t)

The weights are mutated by adding i.i.d. normal distributed noise to every weight.

w_new = w_p + α * N(0, 1)

Then the policy weights w_p are updated in the direction of the best performing mutated weights.