Deep learning tutorial with Chainer

※ This is unofficial tutorial, the content of this blog is written based on personal opinion/understanding.

※ [Update: 2017.06.11] This tutorial is compatible with Chainer v2, which has been released recently. I summarized the difference between chainer v2 and v1.

Deep learning hands on tutorial using Chainer. The source code of this tutorial is available on github.

You can learn how to write deep learning program in python (which is one of the most popular programming language used in data-science community including) and how to use deep learning framework Chainer.

Chainer is  “flexible deep learning framework”. The difference from other famous deep learning framework like tensorflow or caffe is that Chainer constructs neural network dynamically, which enables to write your neural network in more flexible way as you are going to learn.

Especially, I want to recommend Chainer for

  • Deep learning framework beginners
    – Easy environment setup:     One command, pip install chainer, for installing chainer.
    – Easy to debug: You can see python stack trace log when error happened.
  • Researchers
    – Flexisible: Chainer is very flexisible due to its base concept, “define by run” scheme. You can define complicated network easily.
    – Extensible/Customizable:     It is also easy to develop your own function, your own neural network layer only using python with Chainer. Suitable for research to try new idea quickly.
  • Students
    – Educative: Chainer is open source and it is also good for studying deep learning framework. Since chainer is written in python, you can jump to the function definition and read python docstring if you want to dig in the internal behavior.

See Why Chainer for more details.

Setup development environment

Setup Python environment

Install Chainer

Setup IDE

Getting familiar with Chainer

Chainer modules

Chainer basic module introduction

Chainer basic module introduction 2

Hello MNIST! : Multi Layer Perceptron

MNIST dataset classification task using MLP (Multi Layer Perceptron)

MNIST dataset introduction

MNIST training with Multi Layer Perceptron

Refactoring MNIST training

Design patterns for defining model

Writing organized, reusable, clean training code using Trainer module

MNIST inference code

Training with your own dataset : Customizing trainer module

Try regression task with custom dataset using MLP (Multi Layer Perceptron)

Create dataset class from your own data with DatasetMixin

Training code for MyDataset

Write predict code using concat_examples

Image Classification: Convolutional Neural Network

CIFAR 10, CIFAR 100 image classification with CNN (Convolutional Neural Network)

Basic image processing tutorial

Understanding convolutional layer

CIFAR-10, CIFAR-100 dataset introduction

CIFAR-10, CIFAR-100 training with Convolutional Neural Network

CIFAR-10, CIFAR-100 inference code

Natual Language Processing: Recurrent Neural Network, Long Short Term Memory

NLP task with RNN, LSTM.

Recurrent Neural Network (RNN) introduction

Training RNN with simple sequence dataset

Predict code for simple sequence dataset

Long Short Term Memory (LSTM) introduction

Penn Tree Bank (PTB) dataset introduction

Training LSTM model with Penn Bank Tree (ptb) dataset

Predict code for Penn Bank Tree (ptb) dataset

Training tips: How to train deep neural network to achieve more high accuracy

[TBD]

Data augmentation

Learning rate annealing

Choose Iterator – SerialIterator / MultiProcessIterator

Choose trainer extensions

Writing own trainer extension

Miscs

These are currently not categorized into section, but it might be moved future.

Chainer sklearn wrapper

Define your own trainer extensions in Chainer

This is on-going project, I am going to write more topics when I can manage time…

Other topics I want to write in the future.

  • PyCharm (development environment setup, how to debug, how to profile, connecting remote server via ssh etc.)
  • How to use pretrained model: use for content loss/fine tuning etc.
  • VAE (Generative model)
  • GAN (Adversarial network)
  • Approach to Semi-supervised learning
  • How to contribute to chainer development (PyCharm setup etc)
  • ChainerRL – Deep Reinforcement learning
  • ChainerCV – Advanced deep learning library for computer vision
  • ChainerMN – Multi node, distributed deep learning training