problem with floats and calculations
Albert van der Horst
albert at spenarnc.xs4all.nl
Sun Feb 28 13:43:51 CET 2010
In article <mailman.2549.1266184509.28905.python-list at python.org>,
Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>On Sun, 14 Feb 2010 10:33:54 +0100, Karsten Goen
><karsten.goen at googlemail.com> declaimed the following in
>> Maybe anyone can help me with this problem, I don't want to generate for
>> every possible user input a single formula. And also it should be possible
>> for a computer, my calculator at home does the same and is much smaller and
> Most ALL calculators that I've encountered use packed BCD internally
>for numeric data. That is -- two decimal digits (0-9) and a few special
>codes (mantissa sign, exponent sign, decimal point location, and
>exponent start). A lot of them (though strangely not HP) included "guard
>digits" -- they would display, say, 8 significant digits for results,
>but kept 10 digits internally, so that rounding errors wouldn't
>accumulate as rapidly. M$ Excel/Access/VisualBasic "money" data type
What is displayed has no influence on rounding errors.
Calculator vendors prevent puzzlement by non-knowledgeable users
by having guard digits. HP apparently deals with professional users,
so I think it not strange at all.
>carries four decimal places, on the assumption that only two are
>significant, and the last two are guards.
> Most ALL computers are using IEEE floating point (and the exceptions
>are still a binary floating point, not a decimal representation).
> The practice, which /used to be/ taught, is that one does not
>compare floating numbers for equality, but rather one compares the
>difference between floating numbers to be less than some epsilon value;
>epsilon chosen depending upon the significance needed.
> Rather than comparing
> a = b
> abs(a - b) < epsilon
> Since Decimal() also has "infinite" series values (1.0/3.0), an
>epsilon comparison may also be called for.
Not may be.
> Wulfraed Dennis Lee Bieber KD6MOG
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert at spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst
More information about the Python-list