Has anyone happened the same with the calculator?
I have a logic where I add 2 dynamic numbers that are created depending on the intensity of L2 and R2 and I need to get the sum of both, but as you can see in the capture the sum is wrong, anyone knows why?
-
I have had similar accuracy problems with the calculator. I guess it's a problem of rounding the decimals...
I think I solved it using variables and adding them together, which are more precise.
Good Luck ;)
-
Dreams' Calculator gadget uses 32 bit floating point numbers, and when converted to decimal, they're not guaranteed to be accurate to more than 6 significant digits. Compared to the true value 1278431424 for the sum of 670588224 and 607843200, the value 1278431488 that you got is accurate to 7 significant digits, which is within the ball park.
If you want to experiment with how floating point numbers work, you might like to have a look at https://evanw.github.io/float-toy/.
Please sign in to leave a comment.