SeRanet: Quick start guide

This post explains SeRanet project, super resolution software through deep learning.

Preparation

Dependencies – third party library

Install python, pip

The software is written in python, and I’m using python version 2.7.6.

If you are using OS Ubuntu 14.04, python2 is pre-installed by default. So you don’t need to install explicitly. The version of python can be checked by typing 

$ python -V

 in the terminal.

If you can’t find python, then try below.

$ sudo apt-get install python python-dev

To install third party python library, pip command is often used. To install pip, type below in command line

$ sudo apt-get install python-pip

Install popular libraries, numpy, scipy, matplotlib

numpy, scipy, matplotlib are widely used for data processing in python.

$ sudo apt-get install python-numpy python-scipy python-matplotlib

Install OpenCV

OpenCV is used for image processing. See Installation page for installation.

There are several ways to install OpenCV, try below methods if you could not install.

1. Install by pip

$ sudo pip install cv2

2. Install python-opencv

$ sudo apt-get install python-opencv

Ref: Installing OpenCV for Python on Ubuntu, getting ImportError: No module named cv2.cv

3. Install using script

Create a file named “opencv.sh” and write following,

version="$(wget -q -O - http://sourceforge.net/projects/opencvlibrary/files/opencv-unix | egrep -m1 -o '\"[0-9](\.[0-9]+)+' | cut -c2-)"
echo "Installing OpenCV" $version
mkdir OpenCV
cd OpenCV
echo "Removing any pre-installed ffmpeg and x264"
sudo apt-get -qq remove ffmpeg x264 libx264-dev
echo "Installing Dependenices"
sudo apt-get -qq install libopencv-dev build-essential checkinstall cmake pkg-config yasm libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils ffmpeg cmake qt5-default checkinstall
echo "Downloading OpenCV" $version
wget -O OpenCV-$version.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/$version/opencv-"$version".zip/download
echo "Installing OpenCV" $version
unzip OpenCV-$version.zip
cd opencv-$version
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON ..
make -j2
sudo checkinstall
sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig
echo "OpenCV" $version "ready to be used"

Then execute this shell script.

Ref: OpenCV on Ubuntu community

4. PYTHONPATH setting

Add following for your ~/.bashrc 

export PYTHONPATH=/usr/local/lib/python2.7/site-packages:$PYTHONPATH

Install Chainer

Chainer is open source library for implementing neural networks. SeRanet uses chainer library to implement convolutional neural network (CNN) to achieve super resolution deep learning.

Follow github page for installation. Minimum installation (= only use CPU) is easy, just type

$ pip install -U setuptools
$ pip install chainer

suffices.

* In my environment I needed to install google apputils beforehand

$ sudo pip install google-apputils

Use GPU (Skip this process if you don’t have NVIDIA GPU)

For machine learning, a lot of calculation will be executed during the training and setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is highly recommended to prepare NVIDIA GPU and setup it if you want to do deep learning seriously.

(I’m running the code with GTX 980 Ti in my develop environment)

• CUDA setup

Follow the section “Installation with CUDA” on github page for GPU setup.

• cuDNN setup

cuDNN, NVIDIA CUDA Deep Neural Network library, is GPU-accelerated library. GPU can be used by only installing the CUDA, however enabling cuDNN enhances the efficiency GPU usage much more. Especially, GPU memory efficiency is significantly improved by enabling cuDNN and it is necessary for running big size Convolutional Neural Network of SeRanet.

 To install cuDNN, you need to register developer account on NVIDIA website to download cuDNN library.

• NVIDIA cuDNN

Clone SeRanet

Clone project into your local PC.

$ git clone --depth 1 https://github.com/corochann/SeRanet.git

Download training_images 

Skip this process if you are not running training.

I’m using PEXELS photos for the training of SeRanet.

• PEXELSThe website provides high quality photos under Creative Commons Zero (CC0) license.

Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,

• Download PEXELS training_images (jump to google drive, file size around 269MB)

After extract, copy this to data/training_images to start your own training.

Running the inference code

The explanation will be done later, let’s try running the sample code for Upscaling the image anyway.

Go to SeRanet directory and type

$ python src/inference.py path-to-input-image.jpg -a basic_cnn_small

Then it will upscale a picture and save the output image into same directory with input image. -a option specifies the covolutional neural network architecture, and “basic_cnn_small” is used in above example. This architecture is for testing purpose (calculation is very small compared to other architecture) so that even you use CPU, it works without problem.

But of course, if you want to get a good picture upscaling/super resolution result, you must choose the better architecture

$ python src/inference.py path-to-input-image.jpg -a seranet_v1

1	$ python src/inference.py path-to-input-image.jpg -a seranet_v1

For CPU user, it might take a long time to convert image using seranet_v1 architecture. In this case, please try to use smaller image as input.

For GPU user (after setup CUDA and cuDNN), you can specify -g option to use GPU. 

$ python src/inference.py path-to-input-image.jpg -a seranet_v1 -g 0

Running the training code

Go to SeRanet directory and type

$ python src/train.py -a basic_cnn_small -c yonly

-a option specifies the model architecture to train.

-c option specifies the color scheme for the model. ‘yonly’ and ‘rgb’ are available.

For more information please look the SlideShare material.