One of the fastest matlab's RNN libs.
model:A LSTM model has [1024,1024,1024] hidensizes and 10
timestep with a 256 dims input.
Device: i7-4710hq,GTX940m
LSTMtoolbox: 60sec/epoch Keras(1.2.2,Tensorflow backend,cudnn5.1): 29sec/epoch
High parallel Implementation.
- Concatance the weights of 4 gates to W and the values of x and h of every timesteps in a batch to a 3D tensor xh.Compute x*W for every timesteps of every samples in a batch at one time.
- Compute the activated values of input,forget ,ouput gates at one time.
OOP style
- Use
struct
type to define a layer class and a model class.Define ff, bp, optimize methods by using aFunctionHandle
.
- A
model
is a set oflayers
,data
andoptimizer
. model=model_init(input_shape,configs,optimizer)
input_shape
: avector
,[input_dim,batchsize]
or[input_dim,timestep,batchsize]
configs
:cell
,configures of each layersoptimizer
:struct
,keywords:opt
(type of optimizer) ,learningrate
- example:
input_shape=[100,10,64];
hiddensize=[512,512,512];
for l=1:length(hiddensize)
configs{l}.type='lstm';
configs{l}.hiddensize=hiddensize(l);
configs{l}.return_sequence=1;
end
configs{l+1}.type='activation';
configs{l+1}.act_fun='softmax';
configs{l+1}.loss='categorical_cross_entropy';
optimizer.learningrate=0.1;
optimizer.momentum=0.2;
optimizer.opt='sgd'; model=model_init(input_shape,configs,optimizer);
- attributes:
type
:string
,type of the layer,available types:input
,dense
,lstm
,activation
prelayer_type
:string
,type of the previous layer,available types:input
,dense
,lstm
,activation
trainable
:bool
,is the layer trainableinput_shape
: avector
,[input_dim,batchsize]
or[input_dim,timestep,batchsize]
output_shape
: avector
,[hiddensize,batchsize]
or[hiddensize,timestep,batchsize]
batch
:int
,how many batches have been passedepoch
: same tobatch
- methods:
layer=layer_init(prelayer,loss,kwgrs)
- Built and init a layer.If the layer is a
input
layer,prelayer
argument should beinput_shape
- Built and init a layer.If the layer is a
layer=layer.ff(layer,prelayer)
layer=layer.bp(layer,nextlayer)