Building an analytics portal for streaming media utilization

Dashboard in different colors displayed on screen

 

This guest post is about how Bouquet built an analytics dashboard for tracking usage of streaming video and audio from an asset library with over a million titles.

This is a guest post illustrates the use of Highcharts in a high-performance analytics application. While the application in focus is quite robust, no endorsement or recommendation is implied.

 

About the Customer-Facing Analytics Portal

Our client is Alexander Street – a ProQuest Company (ASP). They are the world’s leading provider of academic streaming media (primarily video & audio) to libraries with over 1.2 million titles. ASP has thousands of customers who manage their subscriptions through an online portal.

ASP wanted to help libraries better understand how students and researchers engage with content by providing usage analytics through their online portal. We build their analytical dashboards using Highcharts. Click here for a quick demo of the result:

[arve url=”https://openbouquet.io/v/bouquet-usecase-v2.mp4″]

Project requirements:

  • Data volumes: the usage analytics portal needs to process millions of  page views per month on thousands of video titles and over 5 years of historical data
  • Security: users log in via single sign-on with the online portal
  • Data restrictions: each customer must only see their own data or the data of their children accounts
  • User experience: custom UX was required to match the visual design created by ASP
  • Metrics: charts must display custom metrics like Engagement
  • Interactive: users must be able to filter, drill down, pivot and search
  • Delivery: the usage analytics app needed to go live within 12 weeks

 

Choice of Tools

These are the tools that we chose to build the analytics portal:

 

DeliverablesExecuted byExecuted with
UX designDesigner from ASPDetailed mockups
Data visualizationsFront-end dev from ATCHighcharts
Front end frameworkFront-end dev from ATCReact
Analytics frameworkBusiness analyst from
Squid Solutions
Bouquet
Data capture and processingData plumber from
Squid Solutions
AWS Redshift
TestingASP Quality Assurance team

 

Here you can see a detailed mock-up of a page of the analytics app:

 

 

Output

The metrics concern:

  • playbacks displayed,
  • playback types,
  • items embedded, cited, shared etc.
  • browsers, devices, operating systems usedother details about how users have accessed the content online.

We implemented these metrics in Bouquet by creating a metadata dictionary and saving the configuration in the Bookmarks (see below). That process went smoothly and the overall creation of Bookmarks took one day of our business analyst.

Finally, we mapped each Bookmarks to its corresponding chart. You can see a code example on how to do that below.

Example of a doughnut chart to display browser types:

Figure 1 Result visualization for browsers used

The data came from the Bouquet API:

https://inqwell.squidsolutions.com/release/v4.2/analytics/@'usage'
[email protected]'56d04e8a15abcc21bf059b5d'/query?
groupBy=%27Browsers%27+as+%27browsers%27&
metrics=%27%23+Visits%27+as+%27_visits%27&
period=%27Event+Date%27&
timeframe=2016-12-04&
timeframe=2017-01-31&
orderBy=DESC%28%27%23+Visits%27%29&
limit=10&style=HUMAN&
access_token=f9befb5b-f4ee-4f89-8316-227e1ent4026

 


Figure  Data from Bouquet API (first 20) as they were received to feed the User Interface for ‘Browsers Used’

User of the analytics app can change parameters in the data like date ranges and types of data. The charts are interactive when users change the filters.

The images below show how title engagement changes between with default dates and custom dates that user has selected.

Figure 3 Default range of engagement titles

After changing the date range:

Figure 4 User modified date for engagement titles

 

When users click on playbacks table titles, charts are dynamically placed into popups with data coming from table:

Figure 6 Dynamic popup chart for playback title

 

We used different charts depending on what we needed to show. Below you will find the ‘Playlist Viewed’ column chart and stacked bar chart for ‘Playbacks Subject Areas’:

Figure 7 Playlist Viewed bar chart
Figure 8 Stacked bar chart for Playbacks Subject Areas

Finally, tabs that look like menu items in the front end interface are actually data directly coming from the API.

 

Here for example, the “number of stars” menu comes directly from data. We could imagine a dynamic list that would have shown “6 stars” if this entry had existed into the data on the selected period.

Evaluation Process

We selected the tools based on the benefits provided for the project.

 

ToolBenefits
Highcharts
  • Wide variety of chart types: In the project we used Line charts, Bar charts, Doughnut chart, Stacked Column chart, Multiple Series chart
  • Highly customizable: the built-in chart configuration makes it easier to implement custom styling without using CSS
  • Detailed documentation: documentation and plenty of examples are displayed on Highcharts demo pages and other sites from 3rd party developers
  • Rich features: Highcharts natively supports drill-down and export to pdf/image files in the configuration instead of having to implement the feature ourselves
Bouquet
  • REST API: easy for front-end developer to use. The JSON output required small or no formatting to use with HighCharts.
  • SSO: Bouquet API supports authentication, data restrictions per user. Single-sign on (SSO) was set-up with ASP online portal
  • Interactivity: the Bouquet API supports drill-down, pivot, filter, drill‑down
  • Pagination: the Bouquet API supports server-side pagination to control data sent to the browser.
  • Metadata dictionary: the dimensions, metrics, relations defined by the business analyst are available in the API
  • SQL query generation: SQL queries are dynamically generated without any hand-written code
  • Automatic data caching to make charts load faster
React
  • Popular framework that front-end dev are accustomed to
  • Useful to create clean architectures and build complex applications
  • Highcharts works with React through an open source plug-in available in GitHub supported by a 3rd party
  • This adds some complexity compared to using pure javascript
AWS Redshift
  • Performance: the MPP architecture makes queries run faster even on big data volumes
  • Cost: all inclusive, low cost solution for an analytical database
  • Easy to set-up in the AWS Cloud

 

Implementation


The project had a tight 12-week time-frame to be rolled-out including testing and validation. It was therefore critical for all of the tasks and technologies to connect seamlessly in an agile process.

Figure 2 shows the iterative process to create the analytics app. Let’s assume that data are loaded into Redshift.

 

Figure 2: The iterative process

The challenge was to make the charts interactive by tying the data in the database to the front-end charts. We solved this challenge using Bouquet Bookmarks.

We create a Bookmark for each chart, which provided the API endpoints to retrieve the data from the big data Redshift repository. Each API call generates a SQL query to retrieve data from the database or from the cache.

The table below shows the architecture of the app and the corresponding Bookmarks:

Figure 3: Mapping the various charts with the bookmarks

 

To illustrate on how the mapping works, let’s focus on one of these items: Playbacks / Titles:

 

 

The Bookmarks appear as API URLs. Bookmarks include the scope of data available to the user, the metrics to display, and the interactions that users would have the option to do on each chart.

Here’s how we defined and tested the available API parameters (view the full list here):

 

Figure Partial documentation of the Bouquet Bookmarks API

 

Single Sign-On (SSO)

Having done a model, analyses & bookmarks, we wanted to automate the provisioning of users. They must only see the data from their own accounts.

We used the Bouquet Single Sign On (SSO). It gave us the ability to use SSO authentication with the ASP portal. We configured data restrictions in order to give specific accesses to each user. No coding was required, everything was done through configuration.

Conclusion

The analytics app is now live and used by hundreds of accounts. Each one has multiple end users and can access only the data they are entitled to see.

ASP customers now browse through 5 tabs, 22 pages of interactive analytics to get a full understanding of how content is consumed. By combining the usability of HighCharts with the power of Bouquet, we were able to build a sophisticated customer-facing portal to drive engagement and deliver value to each end customer.