Chainer

English summary of “Chainer Advent Calendar 2016”

Posted on / 0 Comment

I will summarize Chainer Advent Calendar 2016, to introduce what kind of deep learning projects are going on in Japan.

Chainer

Deep learning framework, chainer, is developed by Japanese company Preferred Networks and the community is quite active in Japan. Since interesting projects are presented and discussed, I would like to introduce them by summarizing it in English.

Advent Calendar

In Japan, there is a popular programming blog website called Qiita, and the “Advent Calendar” in Qiita is an event that interested people can join and write a blog for specific theme during Dec 1 to Dec 25.

If you want to read the details of each blog, you may use Google Translate Web to translate Japanese to English.

TL;DR

Based on personal interest, Day 25 and Day 14 were very interesting project (both of them are image processing related). Day 22 is must-read article for those who major image processing. Day 12 are quite interesting project in Reinforcement learning field.

For those who is interested in internal behavior of deep learning framework, Day 8 article is “rare” material that explains it in source code level in quite detail. 

Contents

Day 1. Implementation of “joking” neural network using chainer

The project goal is given some image, it tries to say a joke about this image. By joining Convolutional Neural Network (CNN) to Recurrent Neural Network (RNN)

“Boke” is a Japanese word for saying joking, and there is a website “bokete“. Author collected teacher data from this website.

Day 2. Research projects using chainer

There is a English version of post, “Research projects using Chainer“.

Day 3. Implementation of serial learning using variable length minibatch with Chainer

At first, it introduces how to use NStepLSTM in chainer. Then he implements Bi-directional LSTM in chainer and apply it to Chinese word segmentation task.

Day 4. Report of Chainer Meetup event etc

Report of on-site event of chainer in Japan.

Day 5. Ranking system by CNN+RankNet

Model is trained using image set with ranking. The objective is to make a ranking of unranked image set.

Day 6. Try English-Japanese translation using chainer

He tries Encoder-Decoder translation model with Attention to try English-Japanese translation task.

Day 7. Implement ListNet, Learning to Rank, using Chainer

Rank learning task problem setup is “given teacher data with relative value, learn order of the data”.

Advantage of ListNet compared to RankNet is the computational speed.

Day 8. Make 1-file Chainer

He introduces how back-end of deep learning framework works in chainer, by making 1 file size of “very light version of chainer”. I think it is quite good to understand how back propagation is implemented in deep learning framework.

Day 9. Make people smile using deep learning

He is testing Deep Feature Interpolation for Image Content Changes (DFI) algorithm to insert specific feature (smile, beard etc.) into image.

元画像 笑顔 笑顔+口を開ける
dfi01_org.jpg out9_04.jpg out20_04.jpg
dfi02_org.jpg out10_04.jpg out22_04.jpg
dfi03_org.jpg out7_04.jpg out21_04.jpg
Cite from http://qiita.com/dsanno/items/d3bb8375285c0ea1e6d9

Day 10. sentence learning & generation with DNC (Differentiable Neural Computers)

He made some modification to the DNC implementation in Chainer done by another person below. And he tries some sentence generation.

Day 11. Chord progression recognition with DNN-CRF 

Chord progression recognition task is music dictation task, the goal is to make a chord sequence of given music. It is categorized as sequence labeling problem, so this may applicable to another field too. He tried the task using the model DNN followed by CRF.

Day 12. Tried Value Iteration Networks

VIN (Value Iteration Networks) took the best award in NIPS 2016, which is a famous machine learning conference. This post summarizes/introduces VIN and implemented in Chainer and tested its performance. 

In reinforcement learning, DQN is quite popular but it still needs to learn the model independently if the environment changes. VIN has more general adaptive power that it does not require so much environment knowledge to estimate the future value function.

Day 13. Detect ☆ using Chainer

The task is to detect the 5 edges of star ☆ in the picture. And he achieve the task using Convolution and Deconvolution Network. He made the data by himself.

Day 14. Change the style of picture into “Shinkai Makoto” style

Makoto Shinkai is a director of “Kiminonaha” (Your name), which is now big hit anime movie in Japan. He took his own picture and changed the style of that picture based on the style of “Kiminonaha”.

He used the library already developed by other people.

  1. mattya/chainer-gogh
  2. yusuketomoto/chainer-fast-neuralstyle

Day 15. How to train Neural Network in Chainer and creating Web API Server to publish the result  

This post consists of 2 chapters.

1. Explain how to train your own model in image classification task, starting from how to prepare image set data.

2. Creating a Web server in python to access model from browser. If you are thinking to make web service with some deep learning algorithm working in background, this is a good introduction tutorial. 

Day 16. Try implementing abstraction layer of Chainer

He made a library to use chainer in scikit-learn way. Given data, the model can be parameterized (trained) using fit and predict, predict_proba can be used to use this trained model in scikit-learn. 

With his library, now chainer can be used as follows,

    neural_network = ChainerNeuralNetwork(model=model, optimizer=optimizer, extensions=extensions_list, gpu=args.gpu,
                                          batch_size=args.batchsize, epochs=args.epoch, result=args.out)
    neural_network.fit(X=X, y=y, valid_X=test)
    print(neural_network.predict_proba(test_X).shape)
    print(neural_network.predict(test_X).shape)

    neural_network.save_weights("model.npz")
    neural_network.load_weights("model.npz")

Day 17. PokemonGAN

Tried to create new pokemon, based on current existing pokemon image set, using DCGAN and EBGAN.

It seems the trial did not succeed, below is what DCGAN generated

http://www.monthly-hack.com/entry/201612170000

The new pokemon is fazzy…

Day 18. Re-implementing super resolution network made by Twitter in Chainer

The research of super resolution by neural network gets popular recently. He introduces a recent result by Twitter “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network“.

This paper’s strength is to adapt the latest technics for image processing like below,

  • Use ResNet
  • Use “Content Loss” for loss calculation to make “subjective” loss function
  • Use “Adversal loss” to make more “subjective” loss function
  • Use Pixel Shuffler instead of Deconvolution

Result is as follows,

条件結果
入力低解像度f:id:Hi_king:20161218081357p:plain
二乗誤差最小化で超解像度(工夫1と2のみ)f:id:Hi_king:20161218081417p:plain
工夫１〜４を全部入れた超解像度f:id:Hi_king:20161218082232p:plain
正解高解像度f:id:Hi_king:20161218081438p:plain
cite from http://hi-king.hatenablog.com/entry/2016/12/18/094146

Day 19. Source code explanation of Chat Bot resented in PyCon 2016

The source code explanation of chat bot app, Chainer-Slack-Twitter-Dialogue, in detail. He explains each function meaning.

If you are interested in language processing, chatting app, it is nice article to read.

Day 20. Classify pen pineapple and apple pen

Based on the trending word PPAP, he tried to classify the pen pineapple and apple pen using YOLO(You Only Look Once).

Previous work of object bounding box detection need a several trial of CNN computation, but YOLO needs on time of CNN computation. 

out01.gif

cite from http://qiita.com/ashitani/items/566cf9234682cb5f2d60

Day 21. Human activity recognition with RNN

Task is multi-class classification of human activity recognition (HAR) given the 3-axis acceleration sensor from smartphone.

qiitaperson102_HASC1004.png

cite from http://qiita.com/gomi-kuzu/items/9ee1fe6c20f6175f3a15

The model succeed to recognize human activity (walk, jog, shit down etc) based on the 3-axis acceleration sensor data.

Day 22. deep learning image recognition model in 2016

He explains recent update of image processing, introducing 24 papers, implementing 22 models in chainer (source code is in github).

If your major is in image processing, you should read this post. This is a well summarized material for the recent update of this field.

Day 23. Machine learning in application

Practical Machine Learning Implementation In the Application

Day 24. Keras.js: use chainer model in javascript

For now, Chainer works only with python, there is no straight way to use Chainer in browser client environment. On the otherhand, another deep learning framework, keras, has Keras.js which enables keras model to work in browser environment using javascript. 

He tried the following procedure

  1. Train model with Chainer
  2. Make the same model with Keras
  3. Copy trained parameter from chainer to keras (he made library to copy parameter)
  4. Convert Keras model into keras.js model
  5. Run the model in browser client environment

There is after story,,, after this blog post, Keras developer Francois Chollet commented in Twitter in Japanese,

意味分からない。最初からKeras使った方が良くない？流石日本人。Chainer好きすぎでしょ。 https://t.co/zDSTDBT8TX

— François Chollet (@fchollet) 2016年12月24日

Day 25. Beginner tried colorization of line drawing

★ This post got the most like! action among Chainer advent calendar 2016.

He trained U-net for colorization task of line drawing of anime image using 600,000 training dataset images. 

The result is impressive that neural network returns a well colored output image.

スクリーンショット 2016-12-25 15.56.10.jpg

cite from http://qiita.com/taizan/items/cf77fd37ec3a0bef5d9d

He further improved the model that user is allowed to put a color to indicate what color you want to use around this area. Now neural network put a color based on the request color given in the input image.

スクリーンショット 2016-12-25 14.53.10.jpg

cite from http://qiita.com/taizan/items/cf77fd37ec3a0bef5d9d

Leave a Comment

Your email address will not be published. Required fields are marked *