{"id":16647,"date":"2018-06-18T16:24:09","date_gmt":"2018-06-18T14:24:09","guid":{"rendered":"http:\/\/www.highcharts.com\/blog\/?p=16647"},"modified":"2020-06-19T12:08:02","modified_gmt":"2020-06-19T11:08:02","slug":"visualize-wikipedia-data-with-nodejs-and-highcharts","status":"publish","type":"post","link":"https:\/\/www.highcharts.com\/blog\/tutorials\/visualize-wikipedia-data-with-nodejs-and-highcharts\/","title":{"rendered":"Visualize Wikipedia Data with NodeJS and Highcharts"},"content":{"rendered":"

 <\/p>\n

Wikipedia is a great source of information and data, with a rate of over 10 edits per second. The English Wikipedia<\/a> alone gets 600 new articles per day. But Wikipedia also offers many tools for exploring pages\u2019 statistics, such as Pageviews Analysis<\/a>, Wikipedia Ranking<\/a>, Wikipedia API<\/a>, etc. And if you are a DataViz enthusiastic like me, this is a treasure trove of data!<\/p>\n

In this tutorial, I will show you how to extract and visualize the Pageviews Analysis data using Wikipedia API, NodeJS, and Highcharts.<\/p>\n

The good news is that MediaWiki provides\u00a0an easy and straightforward Wikipedia API, with no need for an API key.<\/p>\n

Let\u2019s get started!<\/p>\n

I will extract the dates<\/b> and the users\u2019 views<\/b> of the Wikipedia webpage International Space Station<\/a> from 7\/1\/2017 – 6\/3\/2018, then plot the trends in an interactive chart (see GIF below):<\/p>\n

\"\"<\/p>\n

Remark<\/b><\/i><\/p>\n

You can download the code used in this article from the following Github link<\/a>.<\/p>\n

 <\/p>\n

I use the following Wikipedia API structure: https:\/\/wikimedia.org\/api\/rest_v1\/metrics\/pageviews\/per-article\/en.wikipedia\/all-access\/user\/International_Space_Station\/daily\/2017070100\/2018060300. Notice the use of the name of the page in the 10th field and the dates in the 12th field. For more details about Wikipedia API, click here<\/a>.<\/p>\n

To handle the API call, I use the request-promise package<\/a>.<\/p>\n

First, let\u2019s create a folder to save the code. Browse to the folder you created and install the request-promise package:<\/p>\n

npm install --save request<\/code>
\nnpm install --save request-promise<\/code><\/p>\n

\"\"<\/p>\n

As I am using the highcharts library, I need to install it as well with this command line:
\nnpm install highcharts<\/code>
\n\"\"<\/p>\n

The last package to install is browserify.
\nnpm install browserify<\/code>
\n\"\"<\/p>\n

Browserify allows me to compile the whole code (including Highcharts library) into a single js file, that I can include it as a script in the HTML webpage.<\/p>\n

I will first display the code (you may copy and paste), run it; then, I will review the code for you.<\/p>\n

The code<\/h2>\n

Create a new js file (ex: code.js), and copy\/paste the code below:<\/p>\n

var rp = require('request-promise');\r\nvar Highcharts = require('highcharts');\r\n\r\nvar options = {\r\n  method: 'GET',\r\n  uri: 'https:\/\/wikimedia.org\/api\/rest_v1\/metrics\/pageviews\/per-article\/en.wikipedia\/all-access\/user\/International_Space_Station\/daily\/2017070100\/2018060300',\r\n  json: true,\r\n};\r\n\r\nrp(options)\r\n  .then((parseBody) => {\r\n    var arrData = [];\r\n    var year, month, day;\r\n\r\n    for (i = 0; i < parseBody.items.length; i++) {\r\n      year = parseBody.items[i].timestamp.slice(0, 4);\r\n      month = parseBody.items[i].timestamp.slice(4, 6);\r\n      day = parseBody.items[i].timestamp.slice(6, 8);\r\n      arrData.push([new Date(year + '-' + month + '-' + day).toDateString(), parseBody.items[i].views]);\r\n    }\r\n\r\n    year = parseBody.items[0].timestamp.slice(0, 4);\r\n    month = parseBody.items[0].timestamp.slice(4, 6);\r\n    day = parseBody.items[0].timestamp.slice(6, 8);\r\n\r\n    \/\/ Create the chart    \r\n    Highcharts.chart('container', {\r\n      title: {\r\n        text: 'Views of the International Space Station Wikipedia webpage'\r\n      },\r\n      subtitle: {\r\n        useHTML: true,\r\n        text: 'Source: Wikipedia<\/a>'\r\n      },\r\n      xAxis: {\r\n        type: 'datetime',\r\n        dateTimeLabelFormats: {\r\n          day: '%y\/%b\/%e'\r\n        }\r\n      },\r\n      yAxis: {\r\n        title: {\r\n          text: 'Number of views'\r\n        }\r\n      },\r\n      series: [{\r\n        name: 'views',\r\n        data: arrData,\r\n        pointStart: Date.UTC(year, month, day),\r\n        pointInterval: 24 * 3600 * 1000 \/\/ one day\r\n      }]\r\n    });\r\n  });<\/pre>\n
Don\u2019t forget to also create an HTML file (ex: chart.html) then copy\/paste the code below:<\/p>\n
<html>\t\r\n    <head>>\t\r\n        <script src=\"bundle.js\"><\/script>\t \r\n    <\/head>\t\r\n    <body>\t\r\n        <div id=\"container\"><\/div>       \r\n    <\/body>\t\r\n<\/html><\/pre>\n
Run the code<\/h2>\n
To run the code just execute this command line on the terminal browserify code.js > bundle.js<\/code>, then click on the HTML file to see the result.<\/p>\n
Explanations<\/h2>\n
I create the Options object that holds all the necessary information to make a request. This route does not require any authentication, so it should be pretty simple.<\/p>\n
var options = {\r\n  method: 'GET',\r\n  uri: 'https:\/\/wikimedia.org\/api\/rest_v1\/metrics\/pageviews\/per-article\/en.wikipedia\/all-access\/user\/International_Space_Station\/daily\/2017070100\/2018060300',\r\n  json: true,\r\n};<\/pre>\n
The object includes:<\/p>\n