TeePoly epsilon value

[tmac2]

TeeChart Pro v2018.24.180321, Delphi Tokyo 10.2.3.

TeePoly defines Float as Extended. The call in PolyFitting to GaussianFitting uses a fixed epsilon of 1.0e-15.

When TeePoly is compiled for x64, Extended is mapped to Double by the compiler. This causes the epsilon to have different results for Extended vs. Double. As seen in the System.Math unit, IsZero uses an epsilon based on the floating point type (as does TeEngine.IsZero).

Our code has a fitting process that fails in 32-bit mode (using extended) whereas it is fine in x64 (using double).

The change I made here to TeePoly is as follows:
- Deleted "uses Classes" from the interface (it is not used).
- Added "uses Math" to the implementation.
- Deleted the "Error" parameter from GaussianFitting.
- Where Error was used, changed to use Math.IsZero(). (This is in two places, and there is no need for the Abs() call when using IsZero.)

The PolyFitting procedure now works identically in 32 or 64-bit compilations, with float defined as Extended or Double.

[Marc]

Hello,

Many thanks for your checks on that and the feedback. We'll take your recommendations and feed them back into the stock routine.

Regards,
Marc Meumann

[tmac2]

Further testing seems to indicate that the original Error value for GaussianFitting is fine as long as it is defined based on the type of floating point value used. The Math.IsZero uses an epsilon that may not be optimal for the PolyFitting procedure.

With this concept, only these two changes to the code are made:

1) Define a constant for the Error value based on the size of Float:

Code: Select all

const
  FloatResolution = {$IF SizeOf(Float) = 10} 1E-19 {$ELSE} 1E-15 {$ENDIF} ;

2) Use FloatResolution in the call to GaussianFitting in PolyFitting:

Code: Select all

    if GaussianFitting(PolyDegree,M,B,Answer,FloatResolution)=0 then

[Marc]

Hello,

Thanks for the update. After reviewing this, it seems that the result-sensitivity could be affected according to the data content itself with respect to error condition being 1e-19 or 1e-15 (32 or 64 bit). Perhaps there's scope here for error to be a developer controlled variable. We'll consider that amongst the options.

Regards,
Marc