Illustrating asylum seekers route with a flow map and open data

Illustration route patterns with a flow map and open data

In late March, at the French Agency for Development, we organized a conference on the use of data for crisis response in humanitarian context. We had intense and exciting discussions on the use of Big Data for development assistance. We celebrated the end of the day by rolling up our sleeves during a mapathon organized with CartONG; a hackathon with the support of Techfugees and a dataviz expedition to see how we could map asylum seekers’ flows using the API developed by the United Nations High Commission for Refugee – UNHCR.
In this tutorial, we will explain how to adapt the Highmaps flight routes example from Highcharts to map asylum seekers’ flows.

The challenge

There is an organization challenge when it comes to visualizing flows, as the origin/destination database is needed to track each flow. Here are the steps to overcome this challenge:

  1. Get the number of people who applied as asylum seekers
  2. Create the origin and destination dataset of countries
  3. Add the routes

1. The number of asylum seekers

We use UNHCR’s API to get the data, and to fetch the information, for example about citizens from Syria seeking asylum in the UK, we send the following query http://data.unhcr.org/api/stats/asylumseekers.json?year=2013&countryof_asylum=GBR&countryoforigin=SYR
And here the answer:

   [{
     ...
     },
        {
            "year": 2013,
            "country_of_asylum": "GBR",
            "country_of_asylum_en": "United Kingdom",
            "country_of_origin": "SYR",
            "country_of_origin_en": "Syrian Arab Republic",
            "rsd_procedure": "G / FI",
            "rsd_procedure_en": "Government / First instance",
            "pending_start_of_year_total_persons": "313",
            "pending_start_of_year_of_which_unhcr_assisted": null,
            "applied_during_year": "2036",
                 //...
        }
    ]

The number of people who applied as asylum seekers is stored under the key ["applied_during_year"], in this case, 2036 people.

2. Create the origin and destination countries

The main idea is to trace asylum seekers’ journey throughout the world, from their country of origin to their destination. This implies creating an origin-destination dataset for the 172 countries referenced in the UNHCR database, and that means sending 172²-172=29412 queries to the API.
First, we create a list of all countries using iso3 codes (also used by UNHCR):

   //Dictionary containing all countries
    var country_iso3_dict = { 
        "Afghanistan": "AFG",
        "Albania": "ALB",
        "Algeria": "DZA",
            //....
        "Zambia": "ZMB",
        "Zimbabwe": "ZWE"
    }

Second, we set up a loop using two variables, iso3_origin and iso3_destination, from the list above to use on the API query:
var url ="http://data.unhcr.org/api/stats/asylum_seekers.json?year=2013&country_of_asylum="+iso3_origin+"&country_of_origin="+iso3_destination
Of course, the combination process is designed to avoid the 172 cases where the origin and the destination is identical.
Remark: The UNHCR database is currently not delivering data fresher than 2013. It seems it will be fixed soon.
Here is the loop structure to call the API:

 
    var origin;
    var iso3_origin;
    var destination;
    var iso3_destination;
    var matrix='{"empty":0}';

    for (i in country_iso3_dict) {
      origin = i            
      iso3_origin = country_iso3_dict[i]

      for (j in country_iso3_dict) {
        if (i != j) {
          destination = j
          iso3_destination = country_iso3_dict[j]

            // query        
            var url ="http://data.unhcr.org/api/stats/asylum_seekers.json?year=2013&country_of_asylum="+iso3_origin+"&country_of_origin="+iso3_destination

         $.getJSON(url, function(json) {
            if (json.length!=0 ){
                var value=json[2]["applied_during_year"]
                var asylum=json[2]["country_of_asylum_en"]
                var api_origin=json[2]["country_of_origin_en"]

            if (value!=null) {
                matrix=matrix+',{"origin":"'+api_origin+'","destination":"'+asylum+'","value":'+value+'}'
                    }
                }
             });
            } 
            } 
        }

It takes time to send and process 29412 queries; this would delay the display of the chart, which is probably already long due to the amount of information to draw. However, since the data is updated on a yearly basis, there is no need to plug the chart directly to the API, so let’s set up a script to fetch the data and save it on a dedicated JSON file. Here is a result of the script execution:

[{"origin":"Pakistan","destination":"Afghanistan","value":18},
    //....
{"origin":"Democratic Republic of the Congo","destination":"Zambia","value":2490}]

Here is the complete script to create our JSON file:


And we saved the result in this JSON file:
http://stats4dev.com/data/dataHCR.json

3. Add the routes

Our data now includes the number of asylum seekers and the origin/destination countries. In this section we add the flight routes. As there are many nodes and flows it is time-consuming to write down each case within the code, and to overcome this issue we use this loop structure:

for (n in country_iso3_dict) {
    country = n
    var data_array = [];
    for (var i = 0; i < json.length; i++) {
      var line = json[i];
      if (line["destination"] == country) {
        origin = n;
        value = line["value"]
        destination = line["origin"]
        if (destination == "Serbia (and Kosovo: S/RES/1244 (1999))") {
          destination = "Serbia"
        }
    console.log(typeof getlinecolor(value))
    data_array.push('{"id": "From '+destination+' to '+origin+'. 
 Number of refugees: '+value+'", "path": ' + '"' + pointsToPath(chart.get(origin), chart.get(destination)) + '", "lineWidth":"' + getlinewidth(value) + '","color":"' + getlinecolor(value) + '"}')
      }
    }
    var dataMap = JSON.parse('[' + data_array + ']')
    chart.addSeries({
      name: 'Destination: ' + n,
      type: 'mapline',
      data: dataMap,
      visible: false
    });
  }

We use the number of asylum seekers, using the color intensity and the thickness of the flow’s arc, to show the volume of the flow on each route:

function getlinewidth(nb) {
    var nb = parseInt(nb)
    return 1 + Math.log(nb) / 2
  }

  function getlinecolor(nb) {
    var nb = parseInt(nb)
    var a = (1 - Math.exp(-nb / 300)) + 0.2
    if (a > 1) {
      a = 1
    }
    return 'rgba(225,109,74,' + a + ')'
  }

Now, all the challenges are resolved, and the chart is ready.

We really enjoyed this project, and we hope it will help you.


Remark
This article was created by Thomas Roca and Etienne David.
Thomas Roca, PhD, Data Officer, Agence Française de Développement.
Etienne David, Data-scientist, Agence Française de Développement.