This post explains SeRanet project, super resolution software through deep learning.
Dependencies – third party library
Install python, pip
The software is written in python, and I’m using python version 2.7.6.
If you are using OS Ubuntu 14.04, python2 is pre-installed by default. So you don’t need to install explicitly. The version of python can be checked by typing
$ python -V
in the terminal.
If you can’t find python, then try below.
$ sudo apt-get install python python-dev
To install third party python library,
pip command is often used. To install pip, type below in command line
$ sudo apt-get install python-pip
Install popular libraries, numpy, scipy, matplotlib
numpy, scipy, matplotlib are widely used for data processing in python.
$ sudo apt-get install python-numpy python-scipy python-matplotlib
OpenCV is used for image processing. See Installation page for installation.
There are several ways to install OpenCV, try below methods if you could not install.
1. Install by pip
$ sudo pip install cv2
2. Install python-opencv
$ sudo apt-get install python-opencv
3. Install using script
Create a file named “opencv.sh” and write following,
version="$(wget -q -O - http://sourceforge.net/projects/opencvlibrary/files/opencv-unix | egrep -m1 -o '\"[0-9](\.[0-9]+)+' | cut -c2-)" echo "Installing OpenCV" $version mkdir OpenCV cd OpenCV echo "Removing any pre-installed ffmpeg and x264" sudo apt-get -qq remove ffmpeg x264 libx264-dev echo "Installing Dependenices" sudo apt-get -qq install libopencv-dev build-essential checkinstall cmake pkg-config yasm libjpeg-dev libjasper-dev libavcodec-dev libavformat-dev libswscale-dev libdc1394-22-dev libxine-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libv4l-dev python-dev python-numpy libtbb-dev libqt4-dev libgtk2.0-dev libfaac-dev libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libtheora-dev libvorbis-dev libxvidcore-dev x264 v4l-utils ffmpeg cmake qt5-default checkinstall echo "Downloading OpenCV" $version wget -O OpenCV-$version.zip http://sourceforge.net/projects/opencvlibrary/files/opencv-unix/$version/opencv-"$version".zip/download echo "Installing OpenCV" $version unzip OpenCV-$version.zip cd opencv-$version mkdir build cd build cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON .. make -j2 sudo checkinstall sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf' sudo ldconfig echo "OpenCV" $version "ready to be used"
Then execute this shell script.
4. PYTHONPATH setting
Add following for your
Chainer is open source library for implementing neural networks. SeRanet uses chainer library to implement convolutional neural network (CNN) to achieve super resolution deep learning.
Follow github page for installation. Minimum installation (= only use CPU) is easy, just type
$ pip install -U setuptools $ pip install chainer
* In my environment I needed to install google apputils beforehand
$ sudo pip install google-apputils
Use GPU (Skip this process if you don’t have NVIDIA GPU)
For machine learning, a lot of calculation will be executed during the training and setting up GPU (enable CUDA, cuDNN) accelerates the calculation. It is highly recommended to prepare NVIDIA GPU and setup it if you want to do deep learning seriously.
(I’m running the code with GTX 980 Ti in my develop environment)
- CUDA setup
Follow the section “Installation with CUDA” on github page for GPU setup.
- cuDNN setup
cuDNN, NVIDIA CUDA Deep Neural Network library, is GPU-accelerated library. GPU can be used by only installing the CUDA, however enabling cuDNN enhances the efficiency GPU usage much more. Especially, GPU memory efficiency is significantly improved by enabling cuDNN and it is necessary for running big size Convolutional Neural Network of SeRanet.
To install cuDNN, you need to register developer account on NVIDIA website to download cuDNN library.
Clone project into your local PC.
$ git clone --depth 1 https://github.com/corochann/SeRanet.git
Skip this process if you are not running training.
I’m using PEXELS photos for the training of SeRanet.
- PEXELSThe website provides high quality photos under Creative Commons Zero (CC0) license.
Thanks to the pexels team and photographers, I can re-distribute training images dataset which I used. It consists of 5000 medium size PEXELS photos. You can download from below,
- Download PEXELS training_images (jump to google drive, file size around 269MB)
After extract, copy this to data/training_images to start your own training.
Running the inference code
The explanation will be done later, let’s try running the sample code for Upscaling the image anyway.
Go to SeRanet directory and type
$ python src/inference.py path-to-input-image.jpg -a basic_cnn_small
Then it will upscale a picture and save the output image into same directory with input image.
-a option specifies the covolutional neural network architecture, and “basic_cnn_small” is used in above example. This architecture is for testing purpose (calculation is very small compared to other architecture) so that even you use CPU, it works without problem.
But of course, if you want to get a good picture upscaling/super resolution result, you must choose the better architecture
$ python src/inference.py path-to-input-image.jpg -a seranet_v1
|$ python src/inference.py path-to-input-image.jpg -a seranet_v1
For CPU user, it might take a long time to convert image using seranet_v1 architecture. In this case, please try to use smaller image as input.
For GPU user (after setup CUDA and cuDNN), you can specify
-g option to use GPU.
$ python src/inference.py path-to-input-image.jpg -a seranet_v1 -g 0
Running the training code
Go to SeRanet directory and type
$ python src/train.py -a basic_cnn_small -c yonly
-a option specifies the model architecture to train.
-c option specifies the color scheme for the model. ‘yonly’ and ‘rgb’ are available.
For more information please look the SlideShare material.