[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: rounding errors

"Dominic R. Scales" wrote:

>   What gives? Is there any numerical math guy/gal out there
>   who can tell me how this happens? It seems to me, that
>   the accuracy of the second/third cast ist WAY off.
> 
>    a=double('2.56989')
>    b=double( 2.56989 )

You just have to define the 2.56989 as a double: b = 2.56989d :)

Remember that the floating point value 2.56989 is not the same as the
real number 2.56989000000000000..., but just some value around
2.5698900. Converting it to a double introduces more digits to the
right, but they are pretty random.

        Alex
-- 
  Alex Schuster     Wonko@weird.cologne.de          PGP Key available
                    alex@pet.mpin-koeln.mpg.de