djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Data order with time stamp descending order issue

Hi

From what I have understood, is that if you want to display data with a timestamp, it needs to be in ascending order.

I have large data being outputted from any API in descending order, where the most recent data is sent first. I then updated my array and updated the plot. The plot is being updated multiple times.

How can I get my timestamp on the plot if the order is descending? As if I wait till all data is download it would take to long.

How can i sove this issue?
kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

Hi there,

Thank you for contacting us.

You are right, the data in Highcharts should be sorted in ascending order. So the first solution that comes to my mind would be the JS Array.reverse() method.

You fetch the data, store into some variable, reverse the array and then assign it to series.data.
Once you fetch the data again, you can also reverse it and append it to the existing data array by Series.setData or Series.addPoint methods.

API References:
https://api.highcharts.com/class-refere ... s#addPoint
https://api.highcharts.com/class-refere ... es#setData

Let me know how that worked!
Best regards
Kamil Musiałowski
Highcharts Developer
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

I can manage to get my code working on online platforms, but this is my code. How do I use setdata?

Code: Select all




  Azure_table_qyery_cycle() {
    console.log("Azure_table_qyery_cycle()");





    this.Azure.HTTP_GET_AZURE_TABLE(this.Azure.node_sensor_selected, this.azureTableQuery)
      .subscribe(data => {
        data.headers.keys();
        console.log("azure tables");
        console.log(data.body.value);
        console.log(data.headers.headers);
        console.log(data.headers.keys());
        console.log(data.body.value.length);

        this.AzureTableStorageResult.push(...data.body.value);


        for (var i = 0; i < data.body.value.length; i++) {
          

          var local_time = this.Azure.local_time(moment(data.body.value[i].Timestamp))
          var timestampn = moment.utc(local_time).valueOf();

          this.PM.push(data.body.value[i].iot_mass_con_b)
          this.global_timestamp.push(data.body.value[i].Timestamp)
          //console.log(data.body.value[i].Timestamp)

          this.series[0].data.push([timestampn,data.body.value[i].iot_mass_con_b]);

          
          
        }
        
        
        

        this.plotdata.setData()

        this.plotdata = this.series[0].data.reverse();
        this.particulate_plot();


        console.log("Next Partition Key");
        var nextpartitionkey = data.headers.headers.get('x-ms-continuation-nextpartitionkey')[0];
        //console.log(nextpartitionkey);

        console.log("Next Row Key");
        var nextrowkey = data.headers.headers.get('x-ms-continuation-nextrowkey')[0];
        //console.log(nextrowkey);


        if ((typeof nextrowkey === "undefined") || (typeof nextpartitionkey === "undefined")) {
          console.log("No more Query");
        }
        else {
          var query_orginal = "$filter=PartitionKey%20ge%20'" + this.start_month_year + "'%20and%20PartitionKey%20le%20'" + this.current_month_year + "'and(%20iot_mass_con_a%20ge%2000%20or%20iot_num_con_b%20ge%2000)" + this.azureTableSelect;
          this.azureTableQuery = query_orginal + "&NextPartitionKey=" + nextpartitionkey + "&NextRowKey=" + nextrowkey
          setTimeout(() => { this.Azure_table_qyery_cycle() }, 1);
        }

      }, err => {
        console.log("ERROR");
        console.log(err);
      });
  }






  particulate_plot() {



    let particulte_chart = this.sensor_chart.stockChart('particulate_plot', {
      chart: {
        type: 'line',
        animation: true,
        backgroundColor: "transparent",
        Color: "black"
      },
      credits: {
        enabled: false
      },
      legend: {
        enabled: true,
        layout: 'horizontal',
        verticalAlign: 'bottom',
        floating: false,
      },
      rangeSelector: {
        selected: 1,
        inputEnabled: false,
        enabled: false
      },
      // title: {
      //   text: "Air Quailty",
      //   style: { "color": "#f4f5f8", "fontSize": "12px" }
      // },

      xAxis: {
        type: 'datetime',
        dateTimeLabelFormats: {
          day: '%e of %b',
        },

        title: {
          text: 'Date',
          style: { "color": "#ECEAF9", "fontSize": "12px" }

        },
        labels: {
          style: { "color": "#ECEAF9", "fontSize": "12px" }
        }

      },
      yAxis: {
        gridLineColor: '#bfd8dc',
        gridLineWidth: 2,
        min: 0,
        opposite: false,
        // title: {
        //   text: "Mass	\xb5g/m\xb3",
        //   style: { "color": "#010101", "fontSize": "12px" }
        // },
        // labels: {
        //   formatter: function () {
        //     return (this.value.toFixed(2) ? '' : '') + this.value.toFixed(2);
        //   },
        //   style: { "color": "#010101", "fontSize": "12px" }
        // },

        plotLines: [{
          label:
          {

            text: "EU Limit 40 \xb5g/m\xb3",
            style: { "color": "#010101", "fontSize": "12px" },
            align: 'center',
            y: -10,

          },
          color: 'red', // Color value
          value: 3, // Value of where the line will appear
          width: 5 // Width of the line   

        }],

      },

      navigator: {
        enabled: false
      },

      scrollbar: { enabled: false },

      series: [{
        data: this.plotdata,
        //data: this.series[0].data,
        zones: [{
          value: 30,
          color: '#0c8eca'
        }, {
          value: 60,
          color: '#eeb333'
        }, {
          value: 90,
          color: '#f0250a'
        }]

      }],

      "tooltip": {
        "pointFormat": "<b>{point.y:.2f}</b>",
        "shared": true
      },
    });
  }

kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

The setData method has to be called on series object. So you grab a desired series and then call this method on it. As a first argument it expects an array of data, the rest of them are optional booleans and the explanation for them you can find in the link below.

https://api.highcharts.com/class-refere ... es#setData

See demo example:
https://jsfiddle.net/BlackLabel/o4mq1h3s/
Kamil Musiałowski
Highcharts Developer
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

Can this function be called once data is downloaded via HTTP get from my other function?
kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

Sure, Highcharts object is globally accessible. So if the chart is created, you can access it everywhere by typing Highcharts.charts[0].series[0].setData().
Kamil Musiałowski
Highcharts Developer
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

Ok the issue is i am downloading data in small blocks.

I am getting the most recent data first and the lastest data .

So do I need to preform a revesere() ,before setting the data?
kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

Exactly. To put it simple, whenever you receive a new block of data you have to reverse it before appending it to the existing main data array, that you use in your series.
Kamil Musiałowski
Highcharts Developer
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

This is what I am calling every time I get the updated data.

Code: Select all

        this.particulte_chart.charts[0].series[0].setData(test_data[0].data.reverse()).
        this.particulate_plot();
But does not seems to work
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

Sorry this is my updated code:

Code: Select all



  particulate_plot() {



     let particulte_chart = this.sensor_chart.stockChart('particulate_plot', {
      chart: {
        type: 'line',
        animation: true,
        backgroundColor: "transparent",
        Color: "black"
      },
      credits: {
        enabled: false
      },
      legend: {
        enabled: true,
        layout: 'horizontal',
        verticalAlign: 'bottom',
        floating: false,
      },
      rangeSelector: {
        selected: 1,
        inputEnabled: false,
        enabled: false
      },
      // title: {
      //   text: "Air Quailty",
      //   style: { "color": "#f4f5f8", "fontSize": "12px" }
      // },

      xAxis: {
        type: 'datetime',
        dateTimeLabelFormats: {
          day: '%e of %b',
        },

        title: {
          text: 'Date',
          style: { "color": "#ECEAF9", "fontSize": "12px" }

        },
        labels: {
          style: { "color": "#ECEAF9", "fontSize": "12px" }
        }

      },
      yAxis: {
        gridLineColor: '#bfd8dc',
        gridLineWidth: 2,
        min: 0,
        opposite: false,
        // title: {
        //   text: "Mass	\xb5g/m\xb3",
        //   style: { "color": "#010101", "fontSize": "12px" }
        // },
        // labels: {
        //   formatter: function () {
        //     return (this.value.toFixed(2) ? '' : '') + this.value.toFixed(2);
        //   },
        //   style: { "color": "#010101", "fontSize": "12px" }
        // },

        plotLines: [{
          label:
          {

            text: "EU Limit 40 \xb5g/m\xb3",
            style: { "color": "#010101", "fontSize": "12px" },
            align: 'center',
            y: -10,

          },
          color: 'red', // Color value
          value: 3, // Value of where the line will appear
          width: 5 // Width of the line   

        }],

      },

      navigator: {
        enabled: false
      },

      scrollbar: { enabled: false },

      series: [{
        //data: this.plotdata,
        data:[1,2,3],
        //data: this.series[0].data,
        zones: [{
          value: 30,
          color: '#0c8eca'
        }, {
          value: 60,
          color: '#eeb333'
        }, {
          value: 90,
          color: '#f0250a'
        }]

      }],

      "tooltip": {
        "pointFormat": "<b>{point.y:.2f}</b>",
        "shared": true
      },
    });
  }
  
  

  Azure_table_qyery_cycle() {
    console.log("Azure_table_qyery_cycle()");





    this.Azure.HTTP_GET_AZURE_TABLE(this.Azure.node_sensor_selected, this.azureTableQuery)
      .subscribe(data => {
        data.headers.keys();
        console.log("azure tables");
        console.log(data.body.value);
        console.log(data.headers.headers);
        console.log(data.headers.keys());
        console.log(data.body.value.length);

        this.AzureTableStorageResult.push(...data.body.value);
        var test_data=[{ data: [] }, { data: [] }, { data: [] }];

        for (var i = 0; i < data.body.value.length; i++) {
          

          var local_time = this.Azure.local_time(moment(data.body.value[i].Timestamp))
          var timestampn = moment.utc(local_time).valueOf();

          this.PM.push(data.body.value[i].iot_mass_con_b)
          this.global_timestamp.push(data.body.value[i].Timestamp)
          //console.log(data.body.value[i].Timestamp)
          
          this.series[0].data.push([timestampn,data.body.value[i].iot_mass_con_b]);                    
          test_data[0].data.push([timestampn,data.body.value[i].iot_mass_con_b]);                    
          
        }
        console.log( this.series[0].data)
        //const data_Clone = {...this.series[0].data};         
        //console.log( data_Clone)
        //this.plotdata. =data_Clone.reverse();        
        
        //this.plotdata = test_data[0].data.reverse();        
        //console.log(this.plotdata)
        
        this.particulte_chart.HighCharts[0].series[0].setData(test_data[0].data.reverse()).
        this.particulate_plot();


        console.log("Next Partition Key");
        var nextpartitionkey = data.headers.headers.get('x-ms-continuation-nextpartitionkey')[0];
        //console.log(nextpartitionkey);

        console.log("Next Row Key");
        var nextrowkey = data.headers.headers.get('x-ms-continuation-nextrowkey')[0];
        //console.log(nextrowkey);


        if ((typeof nextrowkey === "undefined") || (typeof nextpartitionkey === "undefined")) {
          console.log("No more Query");
        }
        else {
          var query_orginal = "$filter=PartitionKey%20ge%20'" + this.start_month_year + "'%20and%20PartitionKey%20le%20'" + this.current_month_year + "'and(%20iot_mass_con_a%20ge%2000%20or%20iot_num_con_b%20ge%2000)" + this.azureTableSelect;
          this.azureTableQuery = query_orginal + "&NextPartitionKey=" + nextpartitionkey + "&NextRowKey=" + nextrowkey
          
            //if (this.tempCount <2)
            {
            setTimeout(() => { this.Azure_table_qyery_cycle() }, 1);
            }
            this.tempCount++
          
          
        }

      }, err => {
        console.log("ERROR");
        console.log(err);
      });
  }



I am trying to update the plot using the following command

Code: Select all

   this.particulte_chart.HighCharts[0].series[0].setData(test_data[0].data.reverse()).
        this.particulate_plot();
kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

Without having a working demo it's hard to guide you in a right direction with it. Also, we do not have official support for Azure - so I'm not familiar with its functionality.

Please check the following:

If you chart is stored in a variable called "particulte_chart" then to access the series on it, you have to ommit the charts property (since you are already referencing a chart). So the correct path would be particulte_chart.series[0].setData().
You can reference a chart by accessing the global object called Highcharts (and then you have to call property charts on it), or store the chart in a variable (just like you did in a code provided above) and then since it's already a chart object, you can access the series on it immediately.

Once you do that, instead of passing a test_data array, pass in a simple hardcoded array like [1,2,3,4]. Is is working? Does it throw any errors in the console?
Kamil Musiałowski
Highcharts Developer
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

Hi

Ok I am now using the following:

Code: Select all

        for (var i = 0; i < data.body.value.length; i++) {          
          var local_time = this.Azure.local_time(moment(data.body.value[i].Timestamp))
          var timestampn = moment.utc(local_time).valueOf();
          this.PM.push(data.body.value[i].iot_mass_con_b)
          this.global_timestamp.push(data.body.value[i].Timestamp)
          //console.log(data.body.value[i].Timestamp)          
          this.series[0].data.push([timestampn,data.body.value[i].iot_mass_con_b]);                    
        }        
        this.particulte_chart.series[0].setData(this.series[0].data.reverse())
The data is being updated onto the plot, but I am getting an error 15
https://assets.highcharts.com/errors/15/

It seems that when I perform reverse(), it also edits the original dataset so that when I preform another setData its using data which already been reversed.
djoshi
Posts: 61
Joined: Fri Mar 26, 2021 5:16 pm

Re: Data order with time stamp descending order issue

I found a working around, which seems to work, but can there be a better way?

Code: Select all

    this.particulte_chart.series[0].setData(this.series[0].data.reverse())
        this.series[0].data.reverse();
kamil.m
Posts: 892
Joined: Thu May 19, 2022 1:33 pm

Re: Data order with time stamp descending order issue

Like I've mentioned in the previous posts - it is better to not mutate the array that's in the Highcharts object, but reverse an array that's stored in a variable outside Highcharts, and then after it's reversed use setData (or addPoint) to add it to your chart.

So it would look like this:

Code: Select all

const dataArray = yourArrayFromAPI.reverse();
this.particulte_chart.series[0].setData(dataArray);
Kamil Musiałowski
Highcharts Developer

Return to “Highcharts Stock”