Data loading from web
– Android TV application hands on tutorial 16

video-data-from-web

* You can see the JSON video data used in this post at here: https://raw.githubusercontent.com/corochann/AndroidTVappTutorial/master/app/src/main/assets/video_lists.json 

Manage data online, keep updated.

In the previous chapter, Background data loading – Android TV application hands on tutorial 15, I introduced LoaderManager and Loader class which helps to load/prepare (maybe time consuming) data in background. One of the example of “time consuming” data preparation is loading data from network. If you can provide the data from the web, app can always show updated, latest information.

In this chapter I will implement web data loading, to show our video contents information. We will prepare data in json format, and upload it to the web (https://raw.githubusercontent.com/corochann/AndroidTVappTutorial/master/app/src/main/assets/video_lists.json for now). It means that the video contents can be modified by just changing this json file, and without modifying any java source code.

(NOTE: the sample app works correctly only when your Android TV is connected to the internet from this chapter.)

Video data preparation

By proceeding to show video contents data dynamically from web, I changed the video source. I was using the video contents from PEXELS VIDEOS, this is public domain videos so that we can use it freely.

I summarized Finding videos, photos, musics which you can use freely for introduction of web pages which distributes CC licensed media contents.

Video data list in JSON format

Until this chapter, I was preparing video data using MovieProvider class. It prepares Movie items in hard-coded way. Instead, we want to prepare data in more organized way and JSON format is used to prepare video data. This is the real video list data in JSON format.

I don’t cover JSON format itself in detail in this post. For those who are not familiar with JSON yet,  I will put some of the links how to parse JSON below.

It is not so difficult, and you may also get feeling by looking this post’s sample code for how to parse JSON data. JSON data is consisting of either JSONObject or JSONArray (this relation is similar to the variable and array in usual program language).

There are some useful JSON analyze tool on the web like below

You can try copy and paste this JSON to visually understand what kind of data structure it has, which helps you to understand how you can parse JSON data more easily.

Data loading trigger – VideoItemLoader

Let’s proceed to implementation. As explained in previous chapter, data preparation is now done in loadInBackground method in VideoItemLoader.

Now we want to modify this method to prepare data from the web.
* It may take some time for loading data, which is suitable to do it in background! This is the purpose that we introduced Loader in previous chapter.

Before that modify loadInBackground  method as follows. New class VideoProvider is introduced here for the web data loading.

    @Override
    public LinkedHashMap<String, List<Movie>> loadInBackground() {
        Log.d(TAG, "loadInBackground");

        /*
         * Executed in background thread.
         * Prepare data here, it may take long time (Database access, URL connection, etc).
         * return value is used in onLoadFinished() method in Activity/Fragment's LoaderCallbacks.
         */
        //LinkedHashMap<String, List<Movie>> videoLists = prepareData();
        LinkedHashMap<String, List<Movie>> videoLists = null;
        try {
            videoLists = VideoProvider.buildMedia(getContext());
        } catch (JSONException e) {
            Log.e(TAG, "buildMedia failed", e);
            //cancelLoad();
        }
        return videoLists;
    }

As you can see VideoLoader class only triggered to do the video data loading. Real loading procedure is done in VideoItemProvider class.

Data loading process – VideoItemProvider

Create a java new class called VideoItemProvider in com.corochann.androidtvapptutorial.data package. Below is the whole source code of this class.

package com.corochann.androidtvapptutorial.data;

import android.content.Context;
import android.content.res.Resources;
import android.net.Uri;
import android.util.Log;

import com.corochann.androidtvapptutorial.model.Movie;

import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.URLConnection;
import java.net.URLDecoder;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;

/**
 *
 */
public class VideoProvider {

    private static final String TAG = VideoProvider.class.getSimpleName();
    public  static final String VIDEO_LIST_URL = "https://raw.githubusercontent.com/corochann/AndroidTVappTutorial/master/app/src/main/assets/video_lists.json";
    public  static final String PREFIX_URL = "http://corochann.com/wp-content/uploads/2015/11/";

    private static String TAG_ID = "id";
    private static String TAG_MEDIA = "videos";
    private static String TAG_VIDEO_LISTS = "videolists";
    private static String TAG_CATEGORY = "category";
    private static String TAG_STUDIO = "studio";
    private static String TAG_SOURCES = "sources";
    private static String TAG_DESCRIPTION = "description";
    private static String TAG_CARD_THUMB = "card";
    private static String TAG_BACKGROUND = "background";
    private static String TAG_TITLE = "title";

    private static LinkedHashMap<String, List<Movie>> sMovieList;

    private static Resources sResources;
    private static Uri sPrefixUrl;

    public static void setContext(Context context) {
        if (null == sResources) {
            sResources = context.getResources();
        }
    }

    /**
     * It may return null when data is not prepared yet by {@link #buildMedia}.
     * Ensure that data is already prepared before call this function.
     * @return
     */
    public static LinkedHashMap<String, List<Movie>> getMedia() {
        return sMovieList;
    }

    /**
     *  ArrayList of movies within specified "category".
     *  If argument is null, then returns all movie list.
     * @param category
     * @return
     */
    public static ArrayList<Movie> getMovieItems(String category) {
        if(sMovieList == null) {
            Log.e(TAG, "sMovieList is not prepared yet!");
            return null;
        } else {
            ArrayList<Movie> movieItems = new ArrayList<>();
            for (Map.Entry<String, List<Movie>> entry : sMovieList.entrySet()) {
                String categoryName = entry.getKey();
                if(category !=null && !category.equals(categoryName)) {
                    continue;
                }
                List<Movie> list = entry.getValue();
                for (int j = 0; j < list.size(); j++) {
                    movieItems.add(list.get(j));
                }
            }
            if(movieItems == null) {
                Log.w(TAG, "No data foud with category: " + category);
            }
            return movieItems;
        }
    }

    public static LinkedHashMap<String, List<Movie>> buildMedia(Context ctx) throws JSONException{
        return buildMedia(ctx, VIDEO_LIST_URL);
    }

    public static LinkedHashMap<String, List<Movie>> buildMedia(Context ctx, String url)
            throws JSONException {
        if (null != sMovieList) {
            return sMovieList;
        }
        sMovieList = new LinkedHashMap<>();
        //sMovieListById = new HashMap<>();

        JSONObject jsonObj = parseUrl(url);

        if (null == jsonObj) {
            Log.e(TAG, "An error occurred fetching videos.");
            return sMovieList;
        }

        JSONArray categories = jsonObj.getJSONArray(TAG_VIDEO_LISTS);

        if (null != categories) {
            final int categoryLength = categories.length();
            Log.d(TAG, "category #: " + categoryLength);
            long id;
            String title;
            String videoUrl;
            String bgImageUrl;
            String cardImageUrl;
            String studio;
            for (int catIdx = 0; catIdx < categoryLength; catIdx++) {
                JSONObject category = categories.getJSONObject(catIdx);
                String categoryName = category.getString(TAG_CATEGORY);
                JSONArray videos = category.getJSONArray(TAG_MEDIA);
                Log.d(TAG,
                        "category: " + catIdx + " Name:" + categoryName + " video length: "
                                + (null != videos ? videos.length() : 0));
                List<Movie> categoryList = new ArrayList<Movie>();
                Movie movie;
                if (null != videos) {
                    for (int vidIdx = 0, vidSize = videos.length(); vidIdx < vidSize; vidIdx++) {
                        JSONObject video = videos.getJSONObject(vidIdx);
                        String description = video.getString(TAG_DESCRIPTION);
                        JSONArray videoUrls = video.getJSONArray(TAG_SOURCES);
                        if (null == videoUrls || videoUrls.length() == 0) {
                            continue;
                        }
                        id = video.getLong(TAG_ID);
                        title = video.getString(TAG_TITLE);
                        videoUrl = PREFIX_URL + getVideoSourceUrl(videoUrls);
                        bgImageUrl = PREFIX_URL + video.getString(TAG_BACKGROUND);
                        cardImageUrl = PREFIX_URL + video.getString(TAG_CARD_THUMB);
                        studio = video.getString(TAG_STUDIO);

                        movie = buildMovieInfo(id, categoryName, title, description, studio,
                                videoUrl, cardImageUrl, bgImageUrl);
                        categoryList.add(movie);
                    }
                    sMovieList.put(categoryName, categoryList);
                }
            }
        }
        return sMovieList;
    }

    private static Movie buildMovieInfo(long id,
                                        String category,
                                        String title,
                                        String description,
                                        String studio,
                                        String videoUrl,
                                        String cardImageUrl,
                                        String bgImageUrl) {
        Movie movie = new Movie();
        movie.setId(id);
        //movie.setId(Movie.getCount());
        //Movie.incrementCount();
        movie.setTitle(title);
        movie.setDescription(description);
        movie.setStudio(studio);
        movie.setCategory(category);
        movie.setCardImageUrl(cardImageUrl);
        movie.setBackgroundImageUrl(bgImageUrl);
        movie.setVideoUrl(videoUrl);

        return movie;
    }


    // workaround for partially pre-encoded sample data
    private static String getVideoSourceUrl(final JSONArray videos) throws JSONException {
        try {
            final String url = videos.getString(0);
            return (-1) == url.indexOf('%') ? url : URLDecoder.decode(url, "UTF-8");
        } catch (UnsupportedEncodingException e) {
            throw new JSONException("Broken VM: no UTF-8");
        }
    }

    protected static JSONObject parseUrl(String urlString) {
        Log.d(TAG, "Parse URL: " + urlString);
        BufferedReader reader = null;

        //sPrefixUrl = Uri.parse(sResources.getString(R.string.prefix_url));
        sPrefixUrl = Uri.parse(PREFIX_URL);

        try {
            java.net.URL url = new java.net.URL(urlString);
            URLConnection urlConnection = url.openConnection();
            reader = new BufferedReader(new InputStreamReader(
                    urlConnection.getInputStream()));
                    //urlConnection.getInputStream(), "iso-8859-1"));
            StringBuilder sb = new StringBuilder();
            String line;
            while ((line = reader.readLine()) != null) {
                sb.append(line);
            }
            String json = sb.toString();
            return new JSONObject(json);
        } catch (Exception e) {
            Log.d(TAG, "Failed to parse the json for media list", e);
            return null;
        } finally {
            if (null != reader) {
                try {
                    reader.close();
                } catch (IOException e) {
                    Log.d(TAG, "JSON feed closed", e);
                }
            }
        }
    }
}

This class owns a static member LinkedHashMap<String, List<Movie>> sMovieList. It is the data that we want to download from web, and the loading is done in buildMedia method. This method will get the data in following procedure. 

  1. Get JSON data
    JSON data is obtained from the web, URL is defined as
    VIDEO_LIST_URL = "https://raw.githubusercontent.com/corochann/AndroidTVappTutorial/master/app/src/main/assets/video_lists.json"
    and
    parseUrl(String url) method, which is called beginning of buildMedia method, is accessing this URL and returns JSONObject.
  2. Parse JSON data
    Parsing JSON data is done inside buildMedia method, by using getJSONObjectgetJSONArray, getString, getInt methods etc which are methods of JSONObject & JSONArray class.You can compare with above code and this JSON checking by Online JSON Viewer to understand how the data is parsed. Note that JSONObject is enclosed by { }, and JSONArray is enclosed by [ ].
  3. Construct Movie item and put it to sMovieList
    At the end of buildMedia method, movie instance is created from parsed data and is added to sMovieList.
movie = buildMovieInfo(id, categoryName, title, description, studio, videoUrl, cardImageUrl, bgImageUrl);
categoryList.add(movie);

sMovieList.put(categoryName, categoryList);

AndroidManifest

Make sure again that app has a permission to access Internet, otherwise app fails to download video list data from web.

<uses-permission android:name="android.permission.INTERNET" />

Build and run!

video-data-from-web

Once you have launched the app, you can notice the video contents has changed from previous chapter’s implementation. This is of course because the data source is completely changed from hardcoded source to JSON formatted source on the web.

Now I can change/update video data anytime without any modification of JAVA source code.

Best data management architecture across application?

In previous chapter, I commented that Loader class is disappointing in the sense that Loader instance and its member cannot be shared among other Activities. We want to use video list data ontained from web. But accessing the internet to prepare data is time consuming and costly operation, we want to access web as less time as possible. So how we can manage data efficiently by reusing data among Activities? The implementation is originally done in Google’s sample source code and I just followed this implementation (and I try to explain its meaning here). 

Since data should be independent from Activity, we can just have a class which handles/manages data. VideoProvider class, introduced in this chapter, is  exactly doing this. There is a static member sMovieList declared as,   

private static LinkedHashMap<String, List<Movie>> sMovieList;

it is the data we have downloaded from the web. And once the data is downloaded, from next time of call of buildMedia method don’t access web but simply returns already created data

    public static LinkedHashMap<String, List<Movie>> buildMedia(Context ctx, String url)
            throws JSONException {
        if (null != sMovieList) {
            return sMovieList;
        }
        ...
    }

Loader class only triggers the timing of data loading, and the real data is managed by VideoProvider class. Its static instance can be referenced from all the activities in this application and we can access same data among activities.

Leave a Comment

Your email address will not be published. Required fields are marked *