[Ecls-list] Rounding bug in float
Raymond Toy
toy.raymond at gmail.com
Thu Mar 26 02:12:03 UTC 2009
Juan Jose Garcia-Ripoll wrote:
> We rely on the GMP library for bignums and rational operations. It
> seems not to be as precised as you would wish
>
>
>> (setq a (/ (1- (ash 1 100)) (ash 1 100)))
>>
> 1267650600228229401496703205375/1267650600228229401496703205376
>
>> (float a 1.0d0)
>>
>
> Breakpoint 9, cl_float (narg=2, x=0x3491f0) at num_co.d:64
> ecl_to_double (x=0x3491f0) at number.c:683
> (gdb) call (double)__gmpq_get_d(aux)
> $1 = 0.99999999999999989
>
Bummer. A little googling for mpq_get_d says it's supposed to truncate,
so I guess that explains the result you see. Too bad there doesn't seem
to be a way to get round-to-nearest.
Ray
More information about the ecl-devel
mailing list