Working With Data & API

In this article, we will compare two ways to visualize data from a CSV file:

1. The featured Highcharts data-module API solution using the data.cvsURL.
2. A custom parser written in pure JavaScript, for more flexibility in advanced cases.

Let’s get started 🙂

Whichever way you choose, you have to go through the basics when dealing with data, such as find the data source, load the data, and process the data.

Remark
In our documentation and API, you can find many examples and solutions that allow you to start working/processing data.

We will use the data set from NASA (National Aeronautics and Space Administration), particularly from the NASA Goddard Institute for Space Studies. The idea is to visualize the CSV file that includes the global average temperature from 1880 to 2019.

1. Data source

Let’s grab the Zonal annual means CSV file, store it on GitHub to be used for the dedicated Highcharts solution, and locally be used for the custom way. The CSV file has several columns: Year, Glob, Nhem, northern hemisphere, etc. The columns are separated by commas. The file isn’t really meant to be human-readable; there are ways of viewing a CSV that’s more human-readable. For example, in the custom demo, you can see how the data looks in a spreadsheet format using the Highcharts export-data module, which we highly recommend using for that purpose.

2. Data loading

To start working with the data, we need to load it into the script. For the built-in Highcharts solution, this is as easy as setting the data.cvsURL option.

For the custom parser, we will use the Fetch API standard. We will also use the await and async syntax for the custom solution supported in modern browsers. Remember to check the browser’s capacity when working with the newest features of JavaScript. When you want to get support for older browsers it’s best to compile your code to the older ES5 syntax.

3. Data parsing

If you go with the built-in Highcharts solution, the data module will spare you the pain to go through the data parsing and you can skip directly to the data visualization section (see section 4. Data visualization).

However, for the custom solution, there are various JavaScript libraries to parse a CSV. By parsing, we mean to figure out where all the commas are located, break up the data, and put it into objects to make it usable. For this data set, it’s simple enough to do the parsing manually with the split function. Btw, a piece of text in a JavaScript variable is a string object and has a function called split. That function allows you to take any arbitrary text and split it up into different elements of an array. And that’s basically what we want to do; we want to split up all the rows, and then in each row, we split up all the columns. The split function requires a single argument – a separator or otherwise known as a delimiter. And in this case, we have two kinds of delimiters. For each row, the delimiter, which differentiates one row from another, is a line break. So first, let’s call split with a line break. Also, we don’t need this first row – the first row is really useful information for us as human beings to think about what the data is, but for that use, we want to take years as a category.

Now it’s important to know this data is clean: no empty date, no mistakes, no empty pieces. But, if the data has commas, this parsing system is going to break down. Even though there are conventions for CSV files to use quotes around the information that shouldn’t be split, where there is a comma in there, you always have to check the data. You also might find that your data isn’t already in CSV format, so be sure to get rid of the unnecessary spaces and commas in the data either manually or through CSV converter/cleaner tools.. This might be a lot of work that can go into parsing and cleaning data for your projects.

4. Data visualization

Now, we’re ready to visualize the data using a simple line chart.
For the dedicated Highcharts solution, the data is hosted in GitHub, and it takes only one line of code to fetch the data (thanks to the data-module):

data: {
  csvURL: 'https://raw.githubusercontent.com/mekhatria/demo_highcharts/master/globalTemperatureChange.csv',
}

The Highcharts data module does all the heavy lifting for you to fetch, process, and prepare the data to be visualized. Yes, it is that simple, and the result is in the demo below:

Below, you can see the same data and chart type using a customised code (see chart below):

Even though the charts look similar, the code behind the scene is different. Instead of one line to fetch the data using the data module, now, there is a whole function for the same job:

async function getData() {
  const date = [],
    globalTemp = [],
    northernTemp = [],
    southernTemp = [],
    response = await fetch('../ZonAnn.Ts+dSST.csv'),
    data = await response.text(),
    rows = data.split(/\n/).slice(1);

  rows.forEach(element => {
    const row = element.split(','),
      year = row[0],
      gt = +row[1],
      nt = +row[2],
      st = +row[3];

    date.push(year);
    globalTemp.push(gt);
    northernTemp.push(nt);
    southernTemp.push(st);
  })

  return {
    date,
    globalTemp,
    northernTemp,
    southernTemp
  }
}

The function above fetches the data from local storage, cleans it, processes it, and returns the result in four different arrays.

For both examples, the idea is to load tabular data in the form of a CSV and visualize it. We had to go through the data processing, then display the data on the charts. In the first demo, the Highcharts data module offers an out-of-the-box solution to fetch and process the data. In most cases, we recommend using this option. Nevertheless, if you need to make specific processing, you can always go with a custom code.

We encourage you to use what you learned in this article to create interactive charts with different data and techniques. You are always welcome to leave your questions and comments under this article.