Skip to content

Tensorflow implementation of "Language Modeling with Gated Convolutional Networks"

Notifications You must be signed in to change notification settings

anantzoid/Language-Modeling-GatedCNN

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Language Modeling with Gated Convolutional Networks

This is a Tensorflow implementation of Facebook AI Research Lab's paper: Language Modeling with Gated Convolutional Networks. This paper applies a convolutional approach to language modelling with a novel Gated-CNN model.

Architecture

Architecture

Requirements

Usage

To train the model using the default hyperparameters:

$ python main.py
$ tensorboard --logdir=logs --host=0.0.0.0

Check main.py for tunable hyperparameter flags.

TODO

  • Replace NCE loss with Adaptive Softmax.
  • Remove restricted training on fixed sized sentences (20, for now) and extend to account for all varied sentence lenghts.
  • Implement Weight Normalisation for faster convergence.
  • Train extensively on deeper models to match the results with the paper.

About

Tensorflow implementation of "Language Modeling with Gated Convolutional Networks"

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages