Realworld computer problems: Floating-Point Arithmetic
Published: 2006-08-12
Category: Code
Tags:
Language: [English]
page views: 244

Do not trust floating points for any math operation where it is cucial to get even the smallest fraction right (calculate money) or relies on very small fractions.

Something I had to deal with when writing the GIS parts of Airtraffic. Due to binary math, frations for the comupter are NOT equal to the decimal frations that one expects... Demons are there.

My solution for Airtraffic was:

Create an object not unlike the "decimal" type in any decent SQL database. In python I created an object that stored the internal value with 3 32bit integer values. X, Y and Z. Each representing one part of the following formula:

X+Y/Z
Where Y 0; Z > 0;

It's not a perfect solution, there's still a loss in pression, but it's a lot less then using floating points. It was a mess to implement the math operations, but that was part of the challenge (i should have used some quality numerical libraries that come with Python instead, but what the heck!)

Good thing was: The rounding errors where predictable. Which is exactly the problem with normal floating points. You never know what is real and what are rounding errors.

What Every Computer Scientist Should Know About Floating-Point Arithmetic

The trouble with rounding floating point numbers | The Register