Tech Study CPU issue

[DaveR]

In later copies of the TeeChart dll I have noticed a tendency for studies to draw OK, but as you keep adding market data to them, the memory and CPU usage starts spiking. I have removed MACD and ADX from the chart and am in the process of writing my own. Even Bollinger Bands is doing this. Sorry, the application is too large to make a small sandbox copy that does it. To duplicate it, I just have to drop in bollinger bands and then let the market data fill in for about 10 minutes. It will begin to spike the CPU after that.

Any chance you can look at it?

[Sandra]

Hello DaveR,

We will run some tests. Please could you tell us with what frequency and the total quantity of points that are being added to the Chart and at what volume level you begin to notice problems. Please could you tell us a little about the PC it is being run on (Windows version, CPU and RAM).

Thanks,

[DaveR]

This happens on many different machines, basically all customers who have it, customer support machines, and my own. My machine is a Win 7 Pro 64-bit with 8GB of RAM, I7 Processor, huge video card, huge drives, etc. I am compiling with Visual Studios 2010, .Net Framework 4, in 32-bit format. I am using the lastest TeeChart version available. On my I7-based machine, it is not as noticable since it has, essentially, 8 processors. On a single processor machine, the one processor spiking will cause the whole platform to become very sluggish and will ultimately crash it.

My charts will be financial market related data. I do not use the regular time-based horizontal scale. In order to selectively show or hide areas where there is no trade, the chart horizontal axis is just X number of points and the scale text is created by intercepting the GetAxisLabel event and setting the e.LabelText string manually.

If I do a 1-minute candle chart for the day up to now, say about 500 bars, and throw on a Bollinger Band study, everything will look normal. After about 10 minutes of updating the last bar during each minute and then adding a bar at each new minute, it will begin to spike. (On the other hand, I can put up any number of simple moving averages without having this issue.) New incomng data comes in at varying speeds. It may be 3 quotes in a second or a few hundred. I do not have AutoRefresh turned on, however. As data comes in the last bar in the CandelSeries is updated. A timer event will fire at a rate of about 4 times per second. If new data has arrived, the chart will be refreshed, meaning maximum refreshes is not all that high. Refreshing the chart includes scrolling the chart to the right for new bars if necessary and rescaling each series' Y axis as necessary. During this process, new data is not added to the chart. New data is only added while the chart is, basically, static. I think this problem is not the result of simply too many bars. It can happen on any size chart. The culprit is more likely to be something related to updating the chart over and over as new data is entered.

I wonder if this has something to do with studies that have multiple lines in the display. I have seen similar issues with MACD and ADX as well. The one thing each has in common is that they each display multiple lines.

I have rewritten Bollinger Bands and MACD. My own studies can be displayed on the charts just fine. The way I accomplish placing multiple lines on the screen for each of these studies is to declare an instance of my study function class for each visible line. But, only the first instance of the study actually calculates the results for the entire study. The second or third display lines are drawn by simply refering back to the first instance of the study and pulling their data from that one class. Data is calculated only once for each bar on the screen. After it is calculated, it is stored in a dictionary object. Any time the chart refreshes data on old bars, it simply reads the data that has already been calculated, not recalculating it. The only time the study is actually calculating anything is if it is working on the newest bar. (If you think about it, there is no reason to always recalculate the old data. Historic values will not change.) Despite the fact that chart updates seem to pull data very frequently, since I have already calculated most of that data, my studies are fairly light-weight.

As mentioned in a previous forum email, if I load a BollingerBand study, then invoke a Selection Tool on the chart and then move the mouse, as soon as the mouse enters the tChart.ClientRectangle, the whole program will crash very dramatically. I think you are already aware of that issue and are working on it. I mention it only because it might be related to the CPU issue. If there is some error trapping going on in the study, to much of it could be the cause of the CPU spike.

Here is code I use to add one of your BollingerBand studes to an existing chart. If you can see anything here that I am doing incorrectly, please let me know.

Dim line1 As New Steema.TeeChart.Styles.Line(TChart1.Chart)
Dim Bollinger1 As New Steema.TeeChart.Functions.Bollinger

Bollinger1.Deviation = Deviations
Bollinger1.Period = Period
Bollinger1.PeriodAlign = Functions.PeriodAligns.Last
Bollinger1.PeriodStyle = Functions.PeriodStyles.NumPoints

line1.Function = Bollinger1
line1.Color = c
line1.LinePen.Width = pWidth
Call SetLineStyle(line1, DashStyle)
line1.DataSource = CandleSeries1
line1.YValues.DataMember = "Close"
line1.CustomVertAxis = GreenAxis
line1.TreatNulls = Steema.TeeChart.Styles.TreatNullsStyle.Skip

Bollinger1.LowBandPen.Width = pWidth
Call SetLineStyle(Bollinger1.LowBandPen, DashStyle)
Bollinger1.LowBandPen.Color = c
line1.CheckDataSource()
Dim BBTag As String = "BB-" & GetRandomID()
line1.Tag = BBTag
line1.Title = "BB Upper"
Bollinger1.LowBand.Tag = BBTag
Bollinger1.LowBand.Title = "BB Lower"
Bollinger1.LowBand.TreatNulls = Styles.TreatNullsStyle.Skip

AddHandler line1.AfterDrawValues, AddressOf SeriesAfterDrawValues
AddHandler line1.BeforeDrawValues, AddressOf SeriesBeforeDrawValues
AddHandler Bollinger1.LowBand.AfterDrawValues, AddressOf SeriesAfterDrawValues
AddHandler Bollinger1.LowBand.BeforeDrawValues, AddressOf SeriesBeforeDrawValues

[DaveR]

I do have a follow-up question about updating the chart. Since I do not have AutoRepaint = True, after I update the last bar or add a new bar I have to call tChart.Refresh to force the chart to redraw. After that I see every series loop through the entire span of data. If I have 20K minute bars in the chart, that takes a lot of time and a lot of resources. Is there any way to tell it to refresh only the last X number of bars, or just the visible portion of the chart? That would sure speed up that process and make it much lighter weight.

[DaveR]

I found another study that seems to be sensitive the the Selector Tool. If I add one of these to a chart:

Dim AD1 As New Steema.TeeChart.Functions.CLVFunction

And then activate a selection tool, when I click the line in the function the chart turns white with a big X through it. The stack trace shows:

Overflow error.
at System.Drawing.Graphics.CheckErrorStatus(Int32 status)
at System.Drawing.Graphics.FillPolygon(Brush brush, Point[] points, FillMode fillMode)
at System.Drawing.Graphics.FillPolygon(Brush brush, Point[] points)
at Steema.TeeChart.Drawing.Graphics3DGdiPlus.Polygon(Point[] p)
at Steema.TeeChart.Drawing.Graphics3D.PolygonFour()
at Steema.TeeChart.Drawing.Graphics3D.Rectangle(Rectangle r, Int32 z)
at Steema.TeeChart.Tools.Selector.DrawSeriesHandles(Boolean AtMarks)
at Steema.TeeChart.Tools.Selector.DoDrawHandles()
at Steema.TeeChart.Tools.Selector.ChartEvent(EventArgs e)
at Steema.TeeChart.Chart.BroadcastToolEvent(EventArgs e)
at Steema.TeeChart.Chart.InternalDraw(Graphics g, Boolean noTools)
at Steema.TeeChart.Chart.InternalDraw(Graphics g)
at Steema.TeeChart.TChart.Draw(Graphics g)
at Steema.TeeChart.TChart.OnPaint(PaintEventArgs pe)
at System.Windows.Forms.Control.PaintWithErrorHandling(PaintEventArgs e, Int16 layer)
at System.Windows.Forms.Control.WmPaint(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

[Sandra]

Hello Daver,

Sorry for the delay, but we try to reproduce your problem here and doesn't achieve it. Would be very helpful, if you can arrange for us a simple code so we can reproduce your problem exactly here.

Thanks,

[DaveR]

I just uploaded a file to you using your Steema Software Support - Upload File page. The file is named TeeChartSandbox-DaveR.zip. It contains a VS2010 vb.net project that loads data from a text file and then starts a timer that adds random price data to the chart. It is basically a 1-minute bar chart of a stock index product. Every 60 random prices will make a new bar. There are three things to note from it:

1 - If you just run the program and let the data update the chart and you watch the resource usage for the program in Windows Task Manager, the CPU and memory used will stay relatively constant.

2 - If you click the button and add a Bollinger Band study to the chart and let the program run for 10 minutes you will see the CPU usage figures rising. Also the memory usage figures will slowly creep up. The difference is the presence of the study.

3 - If you click the "Add Selector" button which attaches a Selection Tool to the chart, as soon as you move the mouse cursor into the chart's client rectangle, it blows up.

Let me know if you have questions.

[Sandra]

Hello DaveR,

Many thanks for your project. We have achieved to reproduce your problem and we are working with it. We try to answer you asap.

Thanks,

[Sandra]

Hello DaveR,

Sorry for the delay. I inform you that finally we added your problem in the bug list report with number[TF02016256], so seems there is a memory leak when you combine Bollinger Function and ClipRectanble. We will try to fix it to upcoming maintenance releases of TeeChartFor.Net.

Thanks,

[DaveR]

Was this addressed in the June maintenance release? I did not see the ticket number in the updates file.

[Sandra]

Hello DaveR,

I am afraid, this problem isn’t still treated in last version of TeeChartFor.Net of July that is published this week. We will treat it upcoming maintenance releases of TeeChart.Net. I recommend you to be aware at this forum, our RSS news feed, twitter and facebook accounts for new release announcements and what's implemented on them. On the other hand, if you want, at the moment I suggest you comment the code you need to use ClipRectangle to work without problem of memory leak, we seems have found.

Thanks,

[DaveR]

Thank you, Sandra. One more question, are you looking in to the problem where activating the selection tool when that study is open causes the program to blow out? In the case of the Bollinger Band study, after you activate a selector tool and then move the mouse cursor into the chart client rectangle, the whole application blows. The ADX study (I think) also had a problem. If you show the ADX study, activate the selection tool, and then click a line in the ADX study, it would blow up in similar fashion.

[Sandra]

Hello DaveR,

I can reproduce your problem using your project, but not using my simple code where try to reproduce the problem with selector and it doesn't appears the problem for me, using my code. We will very helpful for us, if you can try to check if the problem occurs for you using my code, so, we can limiting the reasons of your problem appears.

Code: Select all

        public Form1()
        {
            InitializeComponent();
            InitialzieChart();
        }
        Steema.TeeChart.Styles.Candle candle;
        Steema.TeeChart.Styles.Line line1, line;
        Steema.TeeChart.Functions.Bollinger bollinger;
        Steema.TeeChart.Axis axis1;
        double tmpOpen, tmpClose;
        Random r;
        DateTime dt;
        bool withoutbollinger;
        private void InitialzieChart()
        {
            tChart1.Aspect.View3D = false;
            //Series
            candle = new Steema.TeeChart.Styles.Candle(tChart1.Chart);
            //Function
            line = new Steema.TeeChart.Styles.Line(tChart1.Chart);
            bollinger = new Steema.TeeChart.Functions.Bollinger();

            //  Set custom axes
            candle.VertAxis = Steema.TeeChart.Styles.VerticalAxis.Left;
            tChart1.Aspect.ClipPoints = false;
            candle.Pointer.InflateMargins = false;
            //Timer
            timer1.Tick += new EventHandler(timer1_Tick);
            //ClipPoints
            candle.BeforeDrawValues += new Steema.TeeChart.PaintChartEventHandler(series_BeforeDrawValues);
            candle.AfterDrawValues += new Steema.TeeChart.PaintChartEventHandler(series_AfterDrawValues);
           }
        private void UpdateBollinger()
        {
            //line.Function.Clear();
            line.DataSource = candle;
            line.Function = bollinger;
            bollinger.Deviation = 2;
            bollinger.Period = 14;
            bollinger.PeriodAlign = Steema.TeeChart.Functions.PeriodAligns.Last;
            bollinger.PeriodStyle = Steema.TeeChart.Functions.PeriodStyles.NumPoints;
            line.AfterDrawValues += new Steema.TeeChart.PaintChartEventHandler(series_AfterDrawValues);
            line.BeforeDrawValues += new Steema.TeeChart.PaintChartEventHandler(series_BeforeDrawValues);
        }
        void timer1_Tick(object sender, EventArgs e)
        {
            PopulateSeries();
        }

        void series_AfterDrawValues(object sender, Steema.TeeChart.Drawing.Graphics3D g)
        {
            tChart1.Graphics3D.ClearClipRegions();
        }

        void series_BeforeDrawValues(object sender, Steema.TeeChart.Drawing.Graphics3D g)
        {
            Steema.TeeChart.Styles.Series s = sender as Steema.TeeChart.Styles.Series;
            int left = s.GetHorizAxis.IStartPos;
            int right = s.GetHorizAxis.IEndPos;
            int top = s.GetVertAxis.IStartPos;
            int bottom = s.GetVertAxis.IEndPos;
            tChart1.Graphics3D.ClipRectangle(left, top, right, bottom);
        }

        private void button2_Click(object sender, EventArgs e)
        {
            tChart1[0].Clear();
            tChart1[1].Clear();
            withoutbollinger = true;
            timer1.Start();
            timer1.Interval = 1000;
        }
        private void PopulateSeries()
        {
            r = new Random();
            dt = DateTime.Now;
            tChart1.AutoRepaint = false;
            // show only 50 points - delete the rest
            while (candle.Count > 1000)
            {
                candle.Delete(0);
                line1.Delete(0);
            }
            if (candle.Count > 1000)
            {
                candle.Delete(0);
                line1.Delete(0);
            }
            else
            {
                for (int t = 0; t < 10; t++)
                {  //Candle;
                    tmpOpen = r.Next(100);
                    tmpClose = tmpOpen - r.Next(100);
                    candle.Add(dt, tmpOpen, tmpOpen + r.Next(50), tmpClose - r.Next(50), tmpClose);
                    dt = dt.AddDays(1);
                }
            }
            if (!withoutbollinger)
            {
                UpdateBollinger();
            }
            line.CheckDataSource();
            line.RefreshSeries();
            tChart1.AutoRepaint = true;
            tChart1.Refresh();
        }
        Steema.TeeChart.Tools.Selector selector1;
        private void button3_Click(object sender, EventArgs e)
        {
            if (selector1 != null)
            {
                tChart1.Tools.Remove(selector1);
               // selector1 = new Steema.TeeChart.Tools.Selector(tChart1.Chart);

            }
            else
            {
                selector1 = new Steema.TeeChart.Tools.Selector(tChart1.Chart);
            }
        }

Thanks,

[DaveR]

It looks like it may have to do with the frequency of updates. I made your timer interval adjustible. Once it starts updating fast, things spike. Memory usage cycles higher and higher until it stops.

When I run the chart without Bollinger Bands updating, the CPU usage and memory appear stable. If I add Bollinger bands, both begin to spike. On my 64-bit quad core machine, one processor core will peg out pretty consistantly. The memory reading cycles ever higher. This is running with the timer interval set to about 300, or updating 3 times per second. Keep in mind that stock and futures charts need to update very quickly, there is a ton of data coming in.

I rewrote Bollinger Bands myself and have a class that works. Basically, you have one class that handles each of the lines. The first class instantiated does the work and stores the data in collections. The second line is an instance of the same class, but it simply refers back to the original class to get its data. It really does no work. The net effect is that you get both lines, but you only have one set of calculations going.

In the following image you can see the chart of memory usage. You can see exactly where I turned on Bollinger Bands and where the program finally failed. I have uploaded the project file to your Upload Web page as TeeChartApp.zip.

TaskMgr.gif (68.16 KiB) Viewed 19327 times

[Sandra]

Hello DaveR,

Thanks for your explanation we have considered your modifactions of code to to solve the problem, but, would be very helpfull if you can send us the code where you rewrite of Bollinger function because it help us to achieve solve the bug number [TF02016256].

On the other hand, in previous post have attached for you a code, where the problem of selector tool doesn't appear for me. Also, I have checked your last project and the Selector Tool works fine. Can you tell us if you can reproduce the problem, with Selector Tool, in your machine and using your last application,TeeChartApp, because we can try to solve the problem you have in your first application?

Thanks,