File IO working (finally)

Finally I have text file IO working correctly. I will probably make a Git of it later but I will just paste in the code for now. It seems to be working perfectly and pretty fast now with changes. I was experimenting with JSON encoded tables and it worked great. Or long text files can be stored, or use it as a sort of code obfuscation, etc. Just paste it into a blank tab and voila.

--Codea File IO using images
--By Kyle Howard (Vega) 2012
--Change it, use it, distribute it, whatever

--Usage :
--myString = "blah blah blah"
--fileio:write(myString, "Documents:myFile")
--myLoadedString = fileio:read("Documents:myFile")
-- if myLoadedString ~= nil then print (myLoadedString) end

fileio = {}

function fileio:read(imgName)
    img = readImage(imgName)
    if img == nil then return nil end
    local r,g,b
    local dstr = ""
    local tx = {}
    for y=2, img.rawHeight-1,3 do
        for x=2,img.rawWidth-1,3 do
            r,g,b = img:rawGet(x,y)
            if r ~= 0 then table.insert(tx, string.char(r)) end
            if g ~= 0 then table.insert(tx, string.char(g)) end
            if b ~= 0 then table.insert(tx, string.char(b)) end
    dstr = table.concat(tx)
    return dstr

function fileio:write(str, imgName)
    if str == nil then return false end
    local rem = string.len(str)
    local len = rem
    local lines = 3
    local cols = 600
    if ContentScaleFactor == 2 then cols = 300 end
    while rem > 800 do
        lines = lines + 3
        rem = rem - 800
    local img = image(cols,lines)
    for x=1, img.rawWidth do
        for y=1,img.rawHeight do
    local p1, p2, p3
    local col = 1
    local row = 1
    local i = 1
    while i <= string.len(str) + 3 do
        p1 = 0
        p2 = 0
        p3 = 0
        p4 = 255
        if len >= i then p1 = string.byte(str,i) end
        if len >= i + 1 then p2 = string.byte(str,i+1) end
        if len >= i + 2 then p3 = string.byte(str,i+2) end
        row = row + 3
        if row > 598 then
            row = 1
            col = col + 3
        i = i + 3
    return true

And although this usage is already in comments, here is how to use it:

function setup()
    myString = "blah blah blah"
    fileio:write(myString, "Documents:myFile")
    myLoadedString = fileio:read("Documents:myFile")
    if myLoadedString ~= nil then print (myLoadedString) end

function draw()
    background(40, 40, 50)

By the way, for those who haven’t been following, this saves text into images. So don’t use a name for your text file that you are already using for an image file or it will overwrite it.

Let me know if you encounter a bug. Thanks.

cool… is it possible to name the file e.g. “Documents:myFile.txt” or so? maybe you could force the ending so that it’s not possible to accidently overwrite images?

Cool thing though… :smiley:

@Vega forgive my ignorance, but what is the advantage of this over the storage APIs?

well, one thing is that it can load stuff only when needed. For instance, if you have huge data needed for each level, you could package files with the game and then open and load the data as needed rather than loading everything up front. I am thinking I will use your .WAV library to convert audio files to text, then use this to convert those to images. Then I can only load those audio files as needed and my load time will not be affected at all.

@Vega I guess I will need to look deeper into the storage APIs since I assumed they were demand loaded.

My point is: how are you going to get your data into storage in the first place? I presume you will have it all as text as part of the program, then on first run it is going to load it all into storage. So if you have a ton of data, isn’t your program huge and slow to load?

Well yes you could put them in strings in your app and then save it to project data… and then delete the string. OR you could have code that downloads the data via http.get() and that code it only uncommented when you have new data.

The latter is how I am loading data into my projects.

Alright, well, I am all done arguing the usefulness of flat file storage. If anyone wants to use it, it is here. I am sure there are various ways to do lots of things, this gives you one more way to do storage of data.

@Vega please don’t think I wasn’t saying it wasn’t useful. I genuinely needed to know.

No matter what other factors are at play, it is a easier way to load files onto the iPad (especially large files) if you don’t have a server running; and there are pros and cons to either approach

Sorry, @JockM, it seemed like you were arguing that it wasn’t useful. And all I am saying is: maybe it is not. But if anyone does find it useful, here it is for your use.

Now I am going to spend a couple hours of my Sunday afternoon working on some Codea games. Happy coding to you.

Of course, if nothing else it is a nifty hack. But I think it is more than that, I would need to run some tests but I think this approach is more space efficient for binary data. And it has the nice advantage that you can enumerate them using spriteList()(currently there is no way to get a list of storage keys alas).

But it never hurts to understand the strengths and weaknesses of any approach.

I can think of a very useful (potential) use, albiet one that could probably be accomplished by wrapping the regular persistent stuff - letting code written that expects regular file i/o to be present to work without modification.

I want regular file i/o as part of Codea proper, not so much because I need it (although I have wanted it at times), but because having it would mean that code libraries out there that expect it wouldn’t need to be tweaked much. The more we can do to enable stock lua stuff to be imported and used without modification, the more useful that large corpus of work becomes for us.

Way back when, Codea didn’t have fonts. I (and others) implemented fonts using line drawing and dots, much to TLL’s surprise (and delight, I think). I like to believe that doing so underscored to TLL how much we wanted the feature, and how useful it would be, and caused them to perhaps implement it earlier than they would have otherwise? I see this in the same way - it may be a stunt, but it’s pretty damn cool, and it demonstrates a gap that someone felt the need to fill.

@Bortels that is a really good point. I think there is an opportunity to create a codea wrapper for Lua’sio.functions. One could make it pluggable so that it could use @Vega system and/or the storage APIs

I have been playing with this file io as an image thing a bunch
as I need to export animation data .
Inspired by Vega’s code I did some research on the PNG format
and it’s compression is based on rows , so i coded up an experiment
writing out the data vertically as a PNG 1 pixel wide.
Low and behold it works , no need for the extra pixel padding!
I do not have an ipad 3 however so I don’t know how it works on retina displays.

Another curious thing is that codea seems to draw the PNG bottom up so when I get
the image into love2d (doing love2d <-> codea) I need to read it back top down.
But other than that it works great!

Also Vega, a trick I read in (Lua Gems maybe?) for table concat.

local tx = {}

local str = “”

for y=2, img.rawHeight-1,3 do
    for x=2,img.rawWidth-1,3 do
        r,g,b = img:rawGet(x,y)
        if r ~= 0 then  str = str..string.char(r) end
        if g ~= 0 then str = str..string.char(g) end
        if b ~= 0 then str = str..string.char(b) end
table.insert( tx, str)
table.insert( tx, "")
dstr = table.concat(tx)

To avoid all the table inserts.
Why use a table at all? you might ask, well mainly because sending the
raw string to loadstring() causes an eof error and doing the table
concat thing fixes that.

Nice fix, @felbar. I never tried writing the pixels vertically. That will save a lot of disk space. I will rewrite my version to fix that.

BTW, string concatinations are extremely slow compared to table concatinations. That is the primary reason I use table.concat. Try it out with a very large data file (say 20,000 bytes) and you will see noticeable delay using the string concat.

lol, well serves me right for trusting a book and not testing myself!
BTW, can you test this on a retina device?