※ This is unofficial tutorial, the content of this blog is written based on personal opinion/understanding.
※ [Update: 2017.06.11] This tutorial is compatible with Chainer v2, which has been released recently. I summarized the difference between chainer v2 and v1.
Deep learning hands on tutorial using Chainer. The source code of this tutorial is available on github.
You can learn how to write deep learning program in python (which is one of the most popular programming language used in data-science community including) and how to use deep learning framework Chainer.
- Chainer official page
- pfnet/chainer on github
Chainer is “flexible deep learning framework”. The difference from other famous deep learning framework like tensorflow or caffe is that Chainer constructs neural network dynamically, which enables to write your neural network in more flexible way as you are going to learn.
Especially, I want to recommend Chainer for
- Deep learning framework beginners
– Easy environment setup: One command,
pip install chainer, for installing chainer.
– Easy to debug: You can see python stack trace log when error happened.
– Flexisible: Chainer is very flexisible due to its base concept, “define by run” scheme. You can define complicated network easily.
– Extensible/Customizable: It is also easy to develop your own function, your own neural network layer only using python with Chainer. Suitable for research to try new idea quickly.
– Educative: Chainer is open source and it is also good for studying deep learning framework. Since chainer is written in python, you can jump to the function definition and read python docstring if you want to dig in the internal behavior.
See Why Chainer for more details.
Setup development environment
Getting familiar with Chainer
Chainer basic module introduction
Chainer basic module introduction 2
Hello MNIST! : Multi Layer Perceptron
MNIST dataset classification task using MLP (Multi Layer Perceptron)
MNIST training with Multi Layer Perceptron
Design patterns for defining model
Writing organized, reusable, clean training code using Trainer module
Training with your own dataset : Customizing trainer module
Try regression task with custom dataset using MLP (Multi Layer Perceptron)
Create dataset class from your own data with DatasetMixin
Write predict code using concat_examples
Image Classification: Convolutional Neural Network
CIFAR 10, CIFAR 100 image classification with CNN (Convolutional Neural Network)
Basic image processing tutorial
Understanding convolutional layer
CIFAR-10, CIFAR-100 dataset introduction
CIFAR-10, CIFAR-100 training with Convolutional Neural Network
CIFAR-10, CIFAR-100 inference code
Natual Language Processing: Recurrent Neural Network, Long Short Term Memory
NLP task with RNN, LSTM.
Recurrent Neural Network (RNN) introduction
Training RNN with simple sequence dataset
Predict code for simple sequence dataset
Long Short Term Memory (LSTM) introduction
Penn Tree Bank (PTB) dataset introduction
Training LSTM model with Penn Bank Tree (ptb) dataset
Predict code for Penn Bank Tree (ptb) dataset
Training tips: How to train deep neural network to achieve more high accuracy
Learning rate annealing
Choose Iterator – SerialIterator / MultiProcessIterator
Choose trainer extensions
Writing own trainer extension
These are currently not categorized into section, but it might be moved future.
Define your own trainer extensions in Chainer
This is on-going project, I am going to write more topics when I can manage time…
Other topics I want to write in the future.
- PyCharm (development environment setup, how to debug, how to profile, connecting remote server via ssh etc.)
- How to use pretrained model: use for content loss/fine tuning etc.
- VAE (Generative model)
- GAN (Adversarial network)
- Approach to Semi-supervised learning
- How to contribute to chainer development (PyCharm setup etc)
- ChainerRL – Deep Reinforcement learning
- ChainerCV – Advanced deep learning library for computer vision
- ChainerMN – Multi node, distributed deep learning training