File IO

General
1

Wondering if anyone has a version of lua io for Codea that supports append mode. Project I’m working on samples data every ten seconds, and I need to save it. Quickly runs into issues when I start trying to save thousands on records.

2

@Mark I’m not sure what info you’re saving, but is it possible to read your file into a table at the start, add you new info to the table, then at the end write the table to the file.

3

@dave1707 I do save the info into a table as it’s recorded, but I need to save it against the possibility if the user closing the program, battery failure, ios hiccup, or (surely not) a crash.

4

Here’s what I’ve used. It also supports reading and writing tables. It’s from a few code snippets from the forums and some mods.


--# Main
function setup()
    FileIO.write("test.txt","Hello World\
","a")
    str = FileIO.read("test.txt")
    print(str)
    
end


--# FileIO
--FileIO

FileIO = {}
local Table = {}
local path = os.getenv("HOME").."/Documents/"


--[[Document
--Function: FileIO.write(file,txt)
--Parameters:
--    file (string) File to save. ex. test.txt
--    txt (string) String to save to file.
--    m (string) Write mode. w - Write, a - append, r - read.
--Return:
--]]
FileIO.write = function(file,txt,m)
    mode = m or "r"
    
   local f = io.open(path..file,mode)
    assert(f ~= nil)
    if type(txt) == "table" then
      -- local str = "##FORMAT: table\
"..Table.tostring(txt) 
        local str = Table.tostring(txt) 
        f:write(str)
    else
        f:write(txt)
    end
    f:close()
    print("File saved as: "..file)
end


--[[Document
--Function: FileIO.read(file)
--Parameters:
--    file (string) File to read. ex. test.txt
--Return:
--    str (string)
--]]
FileIO.read = function(file)
    local f = assert(io.open(path..file,"r"))
    local t = f:read("*all")
    local res = Table.totable(t)
    f:close()
    if res then return res else return t end

end



--[[Document
--Function: FileIO.readB(file)
--Parameters:
--    file (string) File to save. ex. test.txt
--Return:
--    t (string)
--]]
FileIO.readB = function(file)
    local hBinaryFile = io.open(FileIO..file, "rb");
    local sBinaryContent = "";
    if(hBinaryFile)then
        while(true)do
            local block = hBinaryFile:read(512); -- Read only 512 bytes per iteration
            if(not block)then break;end
            sBinaryContent = sBinaryContent..block
            break
        end
        hBinaryFile:close();
    end
    local t = {}
    for i=1,sBinaryContent:len() do
        t[i] = sBinaryContent:byte(i)
    end
    return t
end


--[[Document
--Function: FileIO.writeB(file,txt)
--Description: Writes a binary file.
--Parameters:
--    file (string) File to save. ex. test.txt
--    sBinaryContent (string)
--Return:
--]]
FileIO.writeB = function(file,sBinaryContent)
    -- Now lets output it again.
    local hBinaryOutput = io.open(FileIO.path..file, "wb");
    if(hBinaryOutput)then
         hBinaryOutput:write(sBinaryContent); --You could do it in parts, but oh.
         hBinaryOutput:close();
    end
end

Table.val_to_str  = function( v )
  if "string" == type( v ) then
    v = string.gsub( v, "\
", "\\\
" )
    if string.match( string.gsub(v,"[^'\"]",""), '^"+$' ) then
      return "'" .. v .. "'"
    end
    return '"' .. string.gsub(v,'"', '\\\"' ) .. '"'
  else
    return "table" == type( v ) and Table.tostring( v ) or
      tostring( v )
  end
end

Table.key_to_str = function( k )
  if "string" == type( k ) and string.match( k, "^[_%a][_%a%d]*$" ) then
    return k
  else
    return "[" .. Table.val_to_str( k ) .. "]"
  end
end

Table.tostring = function( tbl )

  local result, done = {}, {}
  for k, v in ipairs( tbl ) do
    table.insert( result, Table.val_to_str( v ) )
    done[ k ] = true
  end
  for k, v in pairs( tbl ) do
    if not done[ k ] then
      table.insert( result,
        Table.key_to_str( k ) .. "=" .. Table.val_to_str( v ) )
    end
  end
  return "{" .. table.concat( result, "," ) .. "}"
end

Table.totable = function(self)
    
local t = loadstring("return "..self)

if t then
return assert(t(),"Can not convert string to table.")
end
    
end
5

The above code does not support append for tables, at least as in it wont be able to read it back. I’ll have to add that in.

6

@Mark The other thing you could do is write multiple files. If a file reaches a certain size, start writing to another file. When your done, combine the seperate files into a single one and remove the others. If there’s a problem like you describe, then when you restart the program, if there are multiple files, combine them on startup.

7

Found the solution by following the example of the file I/O in Log4Codea. Thanks, @brookesi.

If yours had easy storage / retrieval of tables, @Briarfox, it would be dead perfect.

8

@Mark Thanks for posting that. For some reason I missed the “a+” for appending to a file. Or it went into my short term memory like a lot of other stuff I forget.

9

@Mark The above code stores and retrieves tables very easily. It just can’t append to the file with a table. It needs to overwrite the file.

tbl = {"filed1",field2="Codea rocks"}
FileIO.write("test.tbl",tbl,"w")
tbl = FileIO.read("test.tbl")
print(tbl.field2)
10

@Briarfox just noodling around… How about a PersistedTable class. Init it with the file info, then use it as a frame in place of normal table function.

Something along the lines of…

pTable = PersistedTable("myfile.txt")
pTable.insert(myTable, aRecord)
pTable.remove(myTable, 2)

with the PersistedTable class both handling normal table functions, and persistent storage. Plus a load function to restore a complete table.

11

@Mark already working on it :slight_smile: