I will summarize Chainer Advent Calendar 2016, to introduce what kind of deep learning projects are going on in Japan.
- 1 Chainer
- 2 Advent Calendar
- 3 TL;DR
- 4 Contents
- 4.1 Day 1. Implementation of “joking” neural network using chainer
- 4.2 Day 2. Research projects using chainer
- 4.3 Day 3. Implementation of serial learning using variable length minibatch with Chainer
- 4.4 Day 4. Report of Chainer Meetup event etc
- 4.5 Day 5. Ranking system by CNN+RankNet
- 4.6 Day 6. Try English-Japanese translation using chainer
- 4.7 Day 7. Implement ListNet, Learning to Rank, using Chainer
- 4.8 Day 8. Make 1-file Chainer
- 4.9 Day 9. Make people smile using deep learning
- 4.10 Day 10. sentence learning & generation with DNC (Differentiable Neural Computers)
- 4.11 Day 11. Chord progression recognition with DNN-CRF
- 4.12 Day 12. Tried Value Iteration Networks
- 4.13 Day 13. Detect ☆ using Chainer
- 4.14 Day 14. Change the style of picture into “Shinkai Makoto” style
- 4.15 Day 15. How to train Neural Network in Chainer and creating Web API Server to publish the result
- 4.16 Day 16. Try implementing abstraction layer of Chainer
- 4.17 Day 17. PokemonGAN
- 4.18 Day 18. Re-implementing super resolution network made by Twitter in Chainer
- 4.19 Day 19. Source code explanation of Chat Bot resented in PyCon 2016
- 4.20 Day 20. Classify pen pineapple and apple pen
- 4.21 Day 21. Human activity recognition with RNN
- 4.22 Day 22. deep learning image recognition model in 2016
- 4.23 Day 23. Machine learning in application
- 4.25 Day 25. Beginner tried colorization of line drawing
Deep learning framework, chainer, is developed by Japanese company Preferred Networks and the community is quite active in Japan. Since interesting projects are presented and discussed, I would like to introduce them by summarizing it in English.
In Japan, there is a popular programming blog website called Qiita, and the “Advent Calendar” in Qiita is an event that interested people can join and write a blog for specific theme during Dec 1 to Dec 25.
If you want to read the details of each blog, you may use Google Translate Web to translate Japanese to English.
Based on personal interest, Day 25 and Day 14 were very interesting project (both of them are image processing related). Day 22 is must-read article for those who major image processing. Day 12 are quite interesting project in Reinforcement learning field.
- Day 25. Beginner tried colorization of line drawing
- Day 14. Change the style of picture into “Shinkai Makoto” style
- Day 22. deep learning image recognition model in 2016
- Day 12. Tried Value Iteration Networks
For those who is interested in internal behavior of deep learning framework, Day 8 article is “rare” material that explains it in source code level in quite detail.
Day 1. Implementation of “joking” neural network using chainer
The project goal is given some image, it tries to say a joke about this image. By joining Convolutional Neural Network (CNN) to Recurrent Neural Network (RNN)
“Boke” is a Japanese word for saying joking, and there is a website “bokete“. Author collected teacher data from this website.
Day 2. Research projects using chainer
There is a English version of post, “Research projects using Chainer“.
- Research projects using Chainer on github
Day 3. Implementation of serial learning using variable length minibatch with Chainer
At first, it introduces how to use
NStepLSTM in chainer. Then he implements Bi-directional LSTM in chainer and apply it to Chinese word segmentation task.
Day 4. Report of Chainer Meetup event etc
Report of on-site event of chainer in Japan.
Day 5. Ranking system by CNN+RankNet
Model is trained using image set with ranking. The objective is to make a ranking of unranked image set.
Day 6. Try English-Japanese translation using chainer
He tries Encoder-Decoder translation model with Attention to try English-Japanese translation task.
Day 7. Implement ListNet, Learning to Rank, using Chainer
Rank learning task problem setup is “given teacher data with relative value, learn order of the data”.
Advantage of ListNet compared to RankNet is the computational speed.
Day 8. Make 1-file Chainer
He introduces how back-end of deep learning framework works in chainer, by making 1 file size of “very light version of chainer”. I think it is quite good to understand how back propagation is implemented in deep learning framework.
Day 9. Make people smile using deep learning
He is testing Deep Feature Interpolation for Image Content Changes (DFI) algorithm to insert specific feature (smile, beard etc.) into image.
Cite from http://qiita.com/dsanno/items/d3bb8375285c0ea1e6d9
元画像 笑顔 笑顔+口を開ける
Day 10. sentence learning & generation with DNC (Differentiable Neural Computers)
- original title: (Chainer) DNC(Differentiable Neural Computers)で文字列の学習&生成
- author: hatoo@github
- category: DNC
He made some modification to the DNC implementation in Chainer done by another person below. And he tries some sentence generation.
Day 11. Chord progression recognition with DNN-CRF
Chord progression recognition task is music dictation task, the goal is to make a chord sequence of given music. It is categorized as sequence labeling problem, so this may applicable to another field too. He tried the task using the model DNN followed by CRF.
Day 12. Tried Value Iteration Networks
VIN (Value Iteration Networks) took the best award in NIPS 2016, which is a famous machine learning conference. This post summarizes/introduces VIN and implemented in Chainer and tested its performance.
In reinforcement learning, DQN is quite popular but it still needs to learn the model independently if the environment changes. VIN has more general adaptive power that it does not require so much environment knowledge to estimate the future value function.
Day 13. Detect ☆ using Chainer
The task is to detect the 5 edges of star ☆ in the picture. And he achieve the task using Convolution and Deconvolution Network. He made the data by himself.
Day 14. Change the style of picture into “Shinkai Makoto” style
Makoto Shinkai is a director of “Kiminonaha” (Your name), which is now big hit anime movie in Japan. He took his own picture and changed the style of that picture based on the style of “Kiminonaha”.
He used the library already developed by other people.
Day 15. How to train Neural Network in Chainer and creating Web API Server to publish the result
This post consists of 2 chapters.
1. Explain how to train your own model in image classification task, starting from how to prepare image set data.
2. Creating a Web server in python to access model from browser. If you are thinking to make web service with some deep learning algorithm working in background, this is a good introduction tutorial.
Day 16. Try implementing abstraction layer of Chainer
He made a library to use chainer in scikit-learn way. Given data, the model can be parameterized (trained) using
predict_proba can be used to use this trained model in scikit-learn.
With his library, now chainer can be used as follows,
neural_network = ChainerNeuralNetwork(model=model, optimizer=optimizer, extensions=extensions_list, gpu=args.gpu, batch_size=args.batchsize, epochs=args.epoch, result=args.out) neural_network.fit(X=X, y=y, valid_X=test) print(neural_network.predict_proba(test_X).shape) print(neural_network.predict(test_X).shape) neural_network.save_weights("model.npz") neural_network.load_weights("model.npz")
Day 17. PokemonGAN
Tried to create new pokemon, based on current existing pokemon image set, using DCGAN and EBGAN.
It seems the trial did not succeed, below is what DCGAN generated
The new pokemon is fazzy…
Day 18. Re-implementing super resolution network made by Twitter in Chainer
The research of super resolution by neural network gets popular recently. He introduces a recent result by Twitter “Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network“.
This paper’s strength is to adapt the latest technics for image processing like below,
- Use ResNet
- Use “Content Loss” for loss calculation to make “subjective” loss function
- Use “Adversal loss” to make more “subjective” loss function
- Use Pixel Shuffler instead of Deconvolution
Result is as follows,
Day 19. Source code explanation of Chat Bot resented in PyCon 2016
- original title: PyCon 2016で発表したChat Botをコードベースから解説(Chainerを利用した雑談応答編）
- author: GushiSnow
- category: Attention, Conversation
The source code explanation of chat bot app, Chainer-Slack-Twitter-Dialogue, in detail. He explains each function meaning.
If you are interested in language processing, chatting app, it is nice article to read.
Day 20. Classify pen pineapple and apple pen
- original title: ペンパイナッポーとアッポーペンを識別する(ChainerでYOLO ver2)
- author: ashitani
- category: YOLO (You Only Look Once)
Based on the trending word PPAP, he tried to classify the pen pineapple and apple pen using YOLO(You Only Look Once).
Previous work of object bounding box detection need a several trial of CNN computation, but YOLO needs on time of CNN computation.
Day 21. Human activity recognition with RNN
Task is multi-class classification of human activity recognition (HAR) given the 3-axis acceleration sensor from smartphone.
The model succeed to recognize human activity (walk, jog, shit down etc) based on the 3-axis acceleration sensor data.
Day 22. deep learning image recognition model in 2016
He explains recent update of image processing, introducing 24 papers, implementing 22 models in chainer (source code is in github).
If your major is in image processing, you should read this post. This is a well summarized material for the recent update of this field.
Day 23. Machine learning in application
- original title: とりあえず代打。Chainerでアプリケーションに機械学習を組み込む
- author: icoxfog417
- category: chainer in application
Practical Machine Learning Implementation In the Application
- author: Hiroshiba
- category: chainer library, client browser usage of Chainer
He tried the following procedure
- Train model with Chainer
- Make the same model with Keras
- Copy trained parameter from chainer to keras (he made library to copy parameter)
- Convert Keras model into keras.js model
- Run the model in browser client environment
There is after story,,, after this blog post, Keras developer Francois Chollet commented in Twitter in Japanese,
— François Chollet (@fchollet) 2016年12月24日
Day 25. Beginner tried colorization of line drawing
★ This post got the most like! action among Chainer advent calendar 2016.
He trained U-net for colorization task of line drawing of anime image using 600,000 training dataset images.
The result is impressive that neural network returns a well colored output image.
He further improved the model that user is allowed to put a color to indicate what color you want to use around this area. Now neural network put a color based on the request color given in the input image.