I’ve been trying to code a little in Codea, and so far, my first project for playing around is to construct HP12C, but I’m having a hard time with large numbers.
For example, if I try to print(string.format(“%10.10f”,0.1)), instead of 0.1000000000, I get an 0.1000000015, which is quite weird for me. Also, if I try to print(string.format(“%d”,1234567890)), instead of 1234567890, I get an 1234567936…
So, what do you consider to be the best solution for working with such big numbers?
I tried to open a bug on issue tracker because the 1234567936 is really weird to me, but somebody replied some stuff and closed it.
numbers are stored in scientific notation: A x 10^B
In codea’s flavor of lua A is 25 bits long and B is 7 bits long (I’m probably off but whatever), which means A can be at most 2^25 ~ 32,000,000. So the maximum A has at most 8 digits, and no matter how big your number is, you’re only going to get 8 digits of precision. That’s why 1234567890 loses some of its least significant digits.
Also since it’s in binary, some numbers are hard to represent. Just like 1/3 is impossible to represent exactly in base 10, because it goes on forever, 1/10 is impossible to represent exactly in base 2 (when can you represent 1/n in base m?)
You could write some code to represent big numbers as arrays of digits, but then you’d have to implement all the basic operations yourself. Addition, multiplication, etc. It could get boring. Luckily you can find people who have already done it for us in most languages. Google for lua big numbers, and you’re bound to find one that you can reuse.
Hi @ruilov, that was exactly what I was thinking too, regarding recreating the basic operations but I was trying to avoid that. But if I want to stay close to HP12C functions, I must be as close as possible to its number precision…
Guess I’ll have to do that dirty work, but thanks a lot for the reply and also for the tips on how to eventually find some reusable code.
And thanks also for claryfing the specs for number data type in Codea… I tried to look for that with no success.
@Markborges
From my understanding, Codea uses 32 bit precision. That allows 7.22 digits of precision. That explains why 1234567890 comes out to 1234567936. The 1234567 is the 7 digits of precision and the 936 is the .22 digits of precision, which is a little meaningless. Looking up 32 bit precision on Wikipedia gives a good explanation. The response in the issue tracker gives a good explanation of what’s happening with 32 bit precision and what can be expected for calculations. It would be nice if Codea used 64 bit precision, but I guess 32 bit is good enough for what’s being done now.
@Markborges, doing your own can be fun too, even if you never thought about numerical computing. It will lead you to explore how to calculate sqrt, exp, sin, cos, or even pi^e. (speaking of which, there’s the cute old puzzle: what’s larger pi^e or e^pi?)
edit: although you can probably restrict yourself to just the rationals
