Skip to main content

Search

Has anyone happened the same with the calculator?

  • Nasic

    I have had similar accuracy problems with the calculator. I guess it's a problem of rounding the decimals...

    I think I solved it using variables and adding them together, which are more precise.

    Good Luck ;)

  • TheBeardyMan

    Dreams' Calculator gadget uses 32 bit floating point numbers, and when converted to decimal, they're not guaranteed to be accurate to more than 6 significant digits. Compared to the true value 1278431424 for the sum of 670588224 and 607843200, the value 1278431488 that you got is accurate to 7 significant digits, which is within the ball park.

    If you want to experiment with how floating point numbers work, you might like to have a look at https://evanw.github.io/float-toy/.

Please sign in to leave a comment.