Open Data API for Chart Creation (Part 2)

In the first part of this post, we looked at how to programmatically request and parse a simple data set via the World Bank’s data API. In this second and final post, we’ll look at how to effectively retrieve multiple data sets, and then use Highcharts for data visualization.

As you remember from the first post, we figured out how to request data on the GDP of Norway for a specific period of time. Let’s bring some additional data into the mix in order to make it a more insightful and interesting presentation.

Your choice of data comes down to what story we want to tell: For example, do we want to visualize the same indicator for countries in a common region, or with similar GDP levels, or by comparing it to the world average?

For illustration purposes, let’s go ahead and craft a request that will display Afghanistan (AFG), South Asia region (SAS), Low income countries (LIC) and the World (WLD), for 20 years (MRV=20).

As I request data for 4 countries, I will need 4*20=80 data to be displayed on the page (per_page=80) to have them all in a single page.

Click here to see the corresponding request.

A way to parse the data could be to simply iterate over the keys as previously explained

for (var i = 0; i & lt; arrayString.length; i++) {
  if (i & lt; lenght) {
    if (arrayString[i] != null) {
      array_1.push(parseFloat(arrayString[i]))
    } else {
      array_1.push(null)
    }
  }
  if (i & gt; = lenght & amp; & amp; i & lt; lenght * 2) {
    if (arrayString[i] != null) {
      array_2.push(parseFloat(arrayString[i]))
    } else {
      array_2.push(null)
    }
  }
  if (i & gt; = lenght * 2 & amp; & amp; i & lt; lenght * 3) {
    if (arrayString[i] != null) {
      array_3.push(parseFloat(arrayString[i]))
    } else {
      array_3.push(null)
    }
  }
  if (i & gt; = lenght * 3 & amp; & amp; i & lt; lenght * 4) {
    if (arrayString[i] != null) {
      array_4.push(parseFloat(arrayString[i]))
    } else {
      array_4.push(null)
    }
  }
}

Although this is a quick way to make it work, it could be problematic; as explained previously the API response provides data sorted alphabetically, not in the order of our request. (I.e. “Afghanistan; South Asia ; Low income ; World”.)  Instead, we ended up with 80 data entries displayed as: entries [0] to [19] corresponding to Afghanistan; [20] to [39] corresponding to Low income, [40] to [59] to South Asia and [60] to [79] to World.

Data structure:

There is probably different ways to work around this issue. A simple solution is to call distinct requests. Each request is treated separately and explicitly stored in a specific vector. As you can see, South Asia is now displayed before Low income and we can keep this coherence whatever the request.

The limitation of this solution is that it increases the API response time, and thus causes an undesirable lag in the rendering of the chart. Moreover; increasing the number of requests is usually not considered an optimal solution.

A way to circumvent this issue is to craft a smarter query that parses the data into a more efficient structure. In the code below, I create lists which contain the necessary information to identify the series and affect them in the desired manner.

First step, I create an exhaustive list for regions and income categories:

var region_list = ["Africa", "Arab World", "Central Europe and the Baltics", 
"East Asia & Pacific (all income levels)", "East Asia & Pacific (developing only)", 
"Europe & Central Asia (all income levels)", "Europe & Central Asia (developing only)", 
"European Union", "Latin America & Caribbean (all income levels)", 
"Latin America & Caribbean (developing only)", "Least developed countries: UN classification", 
"Middle East & North Africa (all income levels)", "Middle East & North Africa (developing only)", 
"North America", "South Asia", "Sub-Saharan Africa (all income levels)", 
"Sub-Saharan Africa (developing only)", "Sub-Saharan Africa (excluding high income)"];
var income_list = ["High income", "High income: OECD", "High income: nonOECD", 
"Low income", "Middle income", "Lower middle income", "Upper middle income", 
"Low & middle income", "OECD members"];

Then, I test the parsed data to identify to which list it belongs:

$.getJSON(url, function(json) {
      $.each(json[1], function(i, data) {
            // test if country name is in region_list
            if ($.inArray(data.country.value, region_list) != -1) {
              //Store Region Name
              Region_name = data.country.value;
              // Append values to arrayRegion
              if (data.value != null) {
                arrayRegion.push(parseFloat(data.value));
              } else {
                arrayRegion.push(null)
              }
              // test if country name is in income_list
            } else if ($.inArray(data.country.value, income_list) != -1) {
              // Store Income name
              Income_name = data.country.value;
              if (data.value != null) {
                arrayIncome.push(parseFloat(data.value));
              } else {
                arrayIncome.push(null)
              }...

Once the data distributed in the appropriate array, I can easily draw the chart.

A SMALL APP TO VISUALIZE ALL WORLD BANK DATA

Other calls are possible and explained on World Bank data page.

While we’re at it, why not also visualize all indicators provided by the World Bank. Here is an ‘app’ that was easy and quick to set up in Highcharts. You can check out the result and the underlying code here.