Skip to content
/ NES Public

OpenAI gym agents learning using Natural Evolution Strategies (NES)

Notifications You must be signed in to change notification settings

A-make/NES

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Neural agents for OpenAI gym

This repo consists of multiple implementations of neural agents that work with some of the OpenAI Gym classical control environments. The agents have primarily been developed for use with the NES training algorithm (see below).

Each agent has the following methods:

set_weights()
get_action()

Gym environments

  • The CartPole problem is a classical underactuated control example. The task is to balance a pole connected to a cart such that it stays upright. The system is underactuated since we cannot manipulate the pole angle directly, but instead we must move the cart in order to effect the pendulum.
  • The MountainCar problem is a task where we have an underactuated car on a hill and would like the car to get over the hill. For this we need to build up momentum.
  • Pendulum
  • Acrobot

Resources

Install

Install: pip install -r requirements.txt

Tested with Python 3.8.2

Natural Evolution Strategies (NES)

Natural Evolution Strategies (NES) is an evolutionary inspired optimization algorithm that can be used for adjusting the parameters of an ANN. It works by creating a fixed standard deviation Gaussian distribution around some current parameter set, and testing the fitness of a set of parameters within this distribution. The current parameters are updated each iteration. The optimization process is similar to hill climbing, but uses instead a population of agents where each agent represents a single parameter vector in the set.

Background Reading

Inspiration

Different implementations in the NES Jupyter notebook


Uber Research EvoGrad package for NES

The EvoGrad package uses PyTorch to create networks and performs NES to optimize the networks parameters/weights.

See

Standalone install:

pip install evograd

About

OpenAI gym agents learning using Natural Evolution Strategies (NES)

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published