Long Short Term Memory (LSTM) introduction

 

Long Short Term Memory

 

Diagrom of Long Short Term Memory. Cite from https://en.wikipedia.org/wiki/File:Peephole_Long_Short-Term_Memory.svg Originally created by Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton

 

Long short term memory is advanced version of RNN, which have “Cell” c to keep long term information.

 

LSTM network Implementation with Chainer

LSTM function and link is provided by Chainer, so we can just use it to construct a neural network with LSTM.

Sample implementation is following, (referred from official example code)

 

Update: [Note]

self.params() will return all the “learnable” parameter in this Chain class (for example W and b in Linear link to calculate x * W + b

Thus, below code will replace all the initial parameter by uniformly distributed value between -0.1 and 0.1.

 

 

Appendix: chainer v1 code

It was written as follows until chainer v1. From Chainer v2, the train flag in function (ex. dropout function) has been removed ans chainer global config is used instead.

 

 

 

Sponsored Links

2 responses

    • Thank you for your question,

      self.params() will return all the “learnable” parameter in this Chain class (for example W and b in Linear link to calculate x * W + b
      Thus, below code will replace all the initial parameter by uniformly distributed value between -0.1 and 0.1.

Leave a Reply

Your email address will not be published.