system decimal display

 From:  Michael Gibson
2445.34 In reply to 2445.32 
Hi Pilou, even though it is possible to do stuff with such a high amount of fractions or high numeric values, there are a lot of problems that can tend to come up. That's because the basic mechanism that a computer uses to calculate "floating point" numbers only has a limited precision since the numbers have to fit within a limited amount of memory.

It is possible to increase the level of precision with a different kind of approach, but it is usually only done for specialized software because each high precision number starts to take up a lot of memory, and also operations are much slower because they won't use the computer's built in floating point hardware.

With the standard system of floating point numbers, there will be a kind of rounding effect if you try to use values like 0.000000000000000000001 or 1000000000000000000000.23511241251412312312312321 - things very large or very small. It can work, but accuracy gets lost during calculations as numbers snap to the "nearest representable number" for the amount of memory that the number has to use to hold its data.

Floating point numbers have the greatest accuracy when they are somewhat closer to the value of 1.

That doesn't mean that you can't use a range of numbers, if you are dealing with numbers in a range of like 1000000 to 0.0000001 the error level for basic operations is pretty small.

But when you start to talk about tiny fractions or huge numbers of a scale much larger than that, those are usually better to do with software that is set up with specialized types of operations in mind to try and avoid accuracy loss.

- Michael