Neural Turing Machines

An attempt at replicating "Neural Turing Machines" (by Alex Graves, Greg Wayne, and Ivo Danihelka) in Keras.

Prerequisites

• Python 2.7
• Numpy 1.12.0
• Scipy 0.18.1
• Theano 0.8.2
• Keras 1.2.0
• CUDA 8.0
• cuDNN 5.0
• Matplotlib 1.5.3

Results

Algorithms Learning

Repeat Copy NTM Memory Use During the Copy Task

Associative Recall (in progress)

Priority Sort (in progress)

Usage

To train a repeat copy task with LSTM:

    $ python learning_repeat_copy_lstm.py

To train a associative recall task with LSTM:

    $ python learning_associative_recall_lstm.py

To train a priority sort task with LSTM:

    $ python learning_priority_sort_lstm.py

To train three different tasks one by one with LSTM:

    $ python learning_algorithm_lstm.py

Other NTM Implementations

• carpedm20/NTM-tensorflow. Check out: https://github.com/carpedm20/NTM-tensorflow
• shawntan/neural-turing-machines. Check out: https://github.com/shawntan/neural-turing-machines
• snipsco/ntm-lasagne. Check out: https://github.com/snipsco/ntm-lasagne

Future works

• Training NTM to learning repeat copy.
• Training NTM to learning associative recall.
• Training NTM to learning dynamical n-grams.
• Training NTM to learning priority sort.
• Using NTM for other natural language processing tasks such as neural language model.

Author

Zhibin Quan / @SigmaQuan