Handling LOTS of data

[mw-108]

Hello

What is the right approach to handle alot of measurement data in a LineSeries chart? I'm speaking about 3.5 mil X values. I already searched the forums here but didn't find a satisfying and up-to-date answer.

Currently on my VM (Win7 x32, 3GB RAM, 2 Core @ 4GHz) it takes approx. 2-3 minutes to populate and display the cart. But after that it is unresponsive. Cursor tool isn't shown, can't scroll, can't zoom, etc.

I upgraded to TChart Pro (XE7) recently and hoped that TChart is "intelligent" enough to handle that kind of issue on its own. But unfortunately it doesn't, unless I missed something. I already had a look at the "Real Time Demo" example, but the way it works it isn't useful for my.

I don't need to display all of the data. So I could cut the amount of measurement values by only displaying each 1.000th or 10.000th. But when I zoom in I need the full measurement data of that range. I saw there is an "OnZoom" event, but this is a TNotificationEvent(Sender: TObject) only, without any information about the zoom.

So how could I solve this?

Thank you very much.

[Yeray]

Hello,

First of all, please, try disabling Hover feature if it's enabled.
Also find some tips to improve the performance when drawing many points here:
http://www.teechart.net/support/viewtop ... 302#p67870

If you still find problems with it, please try to arrange a simple example project we can run as-is to reproduce the problem here.
Thanks in advance.

[mw-108]

Thank you for your fast reply.

However, I'm afraid this doesn't look good. I didn't try yet. But "Chart.Hover.Visible" is already "False". And from the conversation and screenshots it looks like the only way to deal with that problem is to uglify the line series so that it spends less time drawing? I already tried adding stuff and settings from the Realtime charting demo, it didn't do good. For instance when zooming in, the lines are drawn outside the chart when using "Series.LinePen.OwnerCriticalSection := nil;"

Don't understand why this is such a big deal for a company like yours? I mean optimizing a line chart isn't rocket science after all. If you have a screen resolution of 1280x1024 for instance, you obviously can't display 3.5 mil points on that X-axis. A good aproach would be to display each (3.500.000 / 1280)th point. And when zooming in, the rest will be drawn / shown / added.

At least it looks like thats the approach I have to realize next, because TChart costs $600 /year and can't handle that.
Nothing personal, but I really expected less headache for that pricing. An end-user (database programmer) shouldn't deal with such a thing when using your product.

Thank you.

[Yeray]

Hello,

it looks like the only way to deal with that problem is to uglify the line series so that it spends less time drawing?

No, the first tip in that post is to use DrawAllPoints property, that does something very similar to what you describe:

you have a screen resolution of 1280x1024 for instance, you obviously can't display 3.5 mil points on that X-axis. A good aproach would be to display each (3.500.000 / 1280)th point. And when zooming in, the rest will be drawn / shown / added.

DrawAllPoints feature avoids drawing repeated points at the same x pixel position. When you zoom in, the process is repeated so the points that were skipped will be considered again.
Also, DrawAllPointsStyle allows you to choose between two algorithms to decide how to handle the collisions: daFirst will draw only the first point at the conflictive x pixel position, while daMinMax will draw both the minimum and maximum values for that x pixel position.

[mw-108]

Ok, thanks. I will try that in a couple of minutes. Maybe I shot too early.

[mw-108]

Hm, I don't know whats wrong here. I looked through the Speed examples and from what I can see I did set the same properties my test project. But still I don't get even close to the performance of your demos.
I set the Chart to using OpenGL. But even setting GDI or GDI+ didn't boost performance. Neither did disabling custom axis labels and the cursor tool.

Can you please have a look at the attached demo project? I'm using XE7.

In our real project, there are 4 additional (Fast)LineSeries to display an expected measurement range. I also need the axis labels to be in the same format and a cursor tool to display details of a measurement point. So unfortunately there is no option to disable the custom axis labels and the cursor tool. Except you know a better / faster way?

Thank you very much.

[mw-108]

I modified the way he adds / displays the date time. And that gave a little performance boost indeed.

I attached the modified example.

[Yeray]

Hello,

I see you are forcing a chart repaint at the end of your ChartTool_CursorChange event. Note this repaints the chart every time you move the cursor.
I understand you need to do it to clear and repaint the rectangle you manually draw at OnAfterDraw, but having many points

An alternative could be XOR drawing your custom shape and text at ChartTool_CursorChange event instead of drawing it at OnAfterDraw.
I've tried to do something similar to what we do to draw the TCursorTool when it has FullRepaint set to false, but I haven't found how to XOR the Font for the texts.

[mw-108]

Hello Yeray,

thank you.

Yes, I tried a similar XOR approach before updating to Pro to draw the cursor (without text), but it had some weird side-effects when panning or zooming the chart. The last cursor wasn't always removed from the canvas so you ended up having "phantom" cursors. And that was one reason for me to upgrade to Pro, because of the CursorTool.

But unfortunately the AnnotationTool seems to be fixed at bottom or something. I followed an example given by you and modified it so that ChartTool2.Shape.Top and ChartTool2.Shape.Left are mapped to the mouse position in the chart, but that didn't work for me, the annotation was still stuck at the bottom axis: http://www.teechart.net/support/viewtop ... 64&start=0

Regarding the performance issue, disabling the CursorTool / OnChange-Event don't give much of a performance boost though.

However, this isn't my main issue anymore. I implemented the approach I talked about earlier, where I only draw each Nth point and then add additional points after zooming in. This works good so far. But now I'm stuck deleting these additional points when zooming out or even zooming further in. Just removing all the new XValueIndex I addded don't seem to work very well.

[mw-108]

Just for info: This topic can be marked as solved. Got the deleting issue fixed aswell.

Edit

I attached a small demo project to show what I did. In case someone else runs into a similar problem.

[Yeray]

Hello,

Just for info: This topic can be marked as solved. Got the deleting issue fixed aswell.

Great!

If you still need help with the positioning of the annotation or anything else, please don't hesitate to let us know.