Is this a problem with the math.float() call?

Documentation for math.float states:

This function returns the largest integer smaller than or equal to value. This rounds a number down to the nearest integer. For example, math.floor(5.7) returns 5.

Below is my code for rounding a number to a nearest decimal place. The code works fine when I pass in a float that needs to be rounded. When I pass in a number that doesn’t need rounding my call to math.floor causes this code to round down. In this case my number goes from 1.1219 to 1.2189 when I try to round to four digits.

``````function setup()
f=Formatter(5)
f:set(1.1219)
print(f:get())
end
--===================================================
Formatter = class()

function Formatter:init(places)
-- you can accept and set parameters here
if places==nil or places<0 then
places=0
elseif places >8 then --arbitrary limit.
places=8
end
self.incr = places  -- save number of decimal places to preserve
self.num=0 -- initialize as a numeric value
end

function Formatter:set(float)
print("In set, float = " .. float)

round_num=float * 10^(self.incr+1)
round=round_num % 10
print ("In set, round_num = " .. round_num)
print ("In set, round = " .. round)

self.num=math.floor(float*10^self.incr) -- turn the number into an integer with self.incr decimal places

print("In set, calc = " .. float * (10^self.incr))
print("In set, calc with math.floor = " .. math.floor(float * (10^self.incr))) -- WHY IS THIS NUMBER different?
print("In set self.num = " .. self.num)

if round > 5 then      -- round up if trailing decimal place is greater than 5
self.num=self.num+1
end

self.num=self.num/10^self.incr -- return rounded decimal
end

function Formatter:get()
retval="" .. self.num
return retval
end
``````

@nfgdayton If all you’re doing is rounding a number to X decimal places, then the string.format command does it for you. Google string.format for all the options.

``````function setup()
v=1.1219
print(string.format("5 places %.5f",v))
print(string.format("4 places %.4f",v))
print(string.format("3 places %.3f",v))
print(string.format("2 places %.2f",v))
print(string.format("1 place  %.1f",v))
end
``````

@nfgdayton math.floor is working the way it should. The problem is the number 1.1219 isn’t 1.1219 . Some floating point numbers can’t be saved as exact. They might be just a little bit larger or smaller than what is printed. When 1.1219 is multiplied by 10000, it’s really 11218.999999999 . It just prints as 11219, so math.floor(1.1219*10000) prints as 11218.

I like this rounding function:

``````function math.round(number, places) --use -ve places to round to tens, hundreds etc
local mult = 10^(places or 0)
return math.floor(number * mult + 0.5) / mult
end
``````

Thanks to you both.

For anyone that stumbles on this thread:

I looked up the Lua users group NumbersTutorial after Dave’s answer. And then wrote some code to show another unexpected behavior with floating point numbers:

``````--This prints 11 as expected:
num= 1.1
num=num * 10
num2=string.format("%i",num)
print(num2)
-- This prints 110 as expected
num= 1.1
num=math.floor(num * 100)
num2=string.format("%i",num)
print(num)
-- But this raises an error when calling string.format:
num= 1.1
num=num * 100
num2=string.format("%i",num)
print(num)
``````

The lesson I’m taking away from this is twofold: first quit thinking like a Visual Basic circa 1990 programmer, and second to not count on having an integer without passing through a math.floor function.

@nfgdayton An easy way to get an integer is to use the integer divide, // . That fixes the error, see below.

``````    num=(num * 100) // 1     -- use // 1 to force an integer

or

num2=string.format("%i",num // 1)     -- use // 1 to force an integer.
``````

That is also by far the fastest way to get an integer