So how to implement custom extensions for trainer in Chainer? There are mainly 3 approaches. Define function Use decorator, @chainer.training.extension.make_extension Define class Most of the case, 1. Define function is the easiest way to quickly implement your extension. 1. Define function Just a function can be a trainer extension. Simply, define a function […]
Continue reading →Predict code is pretty much the same with Predict code for simple sequence dataset, so I won’t explain in detail. Code The code is on the github, predict_ptb.py.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
"""Inference/predict code for simple_sequence dataset model must be trained before inference, train_simple_sequence.py must be executed beforehand. """ from __future__ import print_function import argparse import os import sys import matplotlib import numpy as np matplotlib.use('Agg') import matplotlib.pyplot as plt import chainer import chainer.functions as F import chainer.links as L from chainer import training, iterators, serializers, optimizers, Variable, cuda from chainer.training import extensions sys.path.append(os.pardir) from RNN import RNN from RNN2 import RNN2 from RNN3 import RNN3 from RNNForLM import RNNForLM def main(): archs = { 'rnn': RNN, 'rnn2': RNN2, 'rnn3': RNN3, 'lstm': RNNForLM } parser = argparse.ArgumentParser(description='simple_sequence RNN predict code') parser.add_argument('--arch', '-a', choices=archs.keys(), default='rnn', help='Net architecture') #parser.add_argument('--batchsize', '-b', type=int, default=64, # help='Number of images in each mini-batch') parser.add_argument('--unit', '-u', type=int, default=100, help='Number of LSTM units in each layer') parser.add_argument('--gpu', '-g', type=int, default=-1, help='GPU ID (negative value indicates CPU)') parser.add_argument('--primeindex', '-p', type=int, default=1, help='base index data, used for sequence generation') parser.add_argument('--length', '-l', type=int, default=100, help='length of the generated sequence') parser.add_argument('--modelpath', '-m', default='', help='Model path to be loaded') args = parser.parse_args() print('GPU: {}'.format(args.gpu)) #print('# Minibatch-size: {}'.format(args.batchsize)) print('') train, val, test = chainer.datasets.get_ptb_words() n_vocab = max(train) + 1 # train is just an array of integers print('#vocab =', n_vocab) print('') # load vocabulary ptb_word_id_dict = chainer.datasets.get_ptb_words_vocabulary() ptb_id_word_dict = dict((v, k) for k, v in ptb_word_id_dict.items()) # Model Setup model = archs[args.arch](n_vocab=n_vocab, n_units=args.unit) classifier_model = L.Classifier(model) if args.gpu >= 0: chainer.cuda.get_device(args.gpu).use() # Make a specified GPU current classifier_model.to_gpu() # Copy the model to the GPU xp = np if args.gpu < 0 else cuda.cupy if args.modelpath: serializers.load_npz(args.modelpath, model) else: serializers.load_npz('result/{}_ptb.model'.format(args.arch), model) # Dataset preparation prev_index = args.primeindex # Predict predicted_sequence = [prev_index] for i in range(args.length): prev = chainer.Variable(xp.array([prev_index], dtype=xp.int32)) current = model(prev) current_index = np.argmax(cuda.to_cpu(current.data)) predicted_sequence.append(current_index) prev_index = current_index predicted_text_list = [ptb_id_word_dict[i] for i in predicted_sequence] print('Predicted sequence: ', predicted_sequence) print('Predicted text: ', ' '.join(predicted_text_list)) if __name__ == '__main__': main() |
Given the first text by the index, args.primeindex, model will predict the following sequence as word id. The last three line converts the […]
Continue reading →This post mainly explains train_ptb.py, uploaded on github. We have already learned RNN and LSTM network architecture, let’s apply it to PTB dataset. It is quite similar to train_simple_sequence.py explained in Training RNN with simple sequence dataset, so no much explanation is necessary. Train code I will just paste whole the training code […]
Continue reading →This post is based on the jupyter notebook ptb_dataset_introduction.ipynb uploaded on github. Penn Treebank dataset, known as PTB dataset, is widely used in machine learning of NLP (Natural Language Processing) research. Dataset if provided by the official page: Treebank-3 In Chainer, PTB dataset can be obtained with build-in function. Let’s see the dataset structure.
|
1 2 3 4 5 6 7 8 9 |
from __future__ import print_function import os import matplotlib.pyplot as plt %matplotlib inline import numpy as np import chainer |
[…]
Continue reading →
Long Short Term Memory Long short term memory is advanced version of RNN, which have “Cell” c to keep long term information. LSTM network Implementation with Chainer LSTM function and link is provided by Chainer, so we can just use it to construct a neural network with LSTM. Sample implementation is […]
Continue reading →
If this is the first time to use python and you have not built any python development environment, setting up IDE (Integrated Development Environment) might be a one good choice to start coding quite easily. I will introduce how to setup PyCharm, one of the major python development tool, which I am also using […]
Continue reading →If you are familiar with machine learning before deep learning becomes popular, you might have been using sklearn (scikit-learn), which is very popular machine learning library in python. Its interface is used for a long time, and I thought it is better to support this interface with python to allow users to try deep learning more […]
Continue reading →
Once you have setup python environment, now we can install chainer. Nowadays it is important for deep learning library to use GPU to enhance its calculation speed, and Chainer provides several levels of GPU support. The packages which you need to install depends on your PC envitonment, please follow this figure to understand what category you […]
Continue reading →
※ This post is mainly just a summary/translation of the Japanese blog, データサイエンティストを目指す人のpython環境構築 2016 TL;DR; I recommend to install “anaconda” instead of using “official python package”. If you just want to proceed environment setup, jump to “Environment setup for each OS”. At first, I will explain little bit about the background […]
Continue reading →
This tutorial corresponds to 03_custom_dataset_mlp folder in the source code. We have trained the model with own dataset, MyDataset, in previous post, let’s write predict code. Source code: predict_custom_dataset1.py predict_custom_dataset2.py Prepare test data It is not difficult for the model to fit to the train data, so we will check how the model is […]
Continue reading →