Codea crashes during saveText

Hi Codea people,

I’ve got a Codea crash problem :frowning:

I’m working on a small scientific app - it collects data about the way a subject moves their finger or stylus to follow a moving line. The subject will follow the line for a set number of movements - say 50 - which last for random time periods - usually between 1 and 3 seconds. Once the subject has followed all the movements, the trail is complete and the program re-draws a “start” button so a new trial can be started.

Every time the draw() function is called, the app saves data into a table. In a function sub-called by the Main draw() function I have:

        -- Insert data into main data table:
        table.insert(self.dataBuffer, {ElapsedTime, ti, self.current.i,
                                       self.t.x, self.t.y, self.lastTouch.x,
                                       self.lastTouch.y, tpos, dx, p.mirrx,
                                       CurrentTouch.state, self.lapse,
                                       UserAcceleration.x, UserAcceleration.y,
                                       UserAcceleration.z, 0,
                                       maskOn})

After a variable amount of time (the moving line moves a fixed number of times, but spends a randomised amount of time at each location), a function is called to output the content of self.dataBuffer into a Dropbox file:

In a method called PBPath:endAnimation (called when the trial is complete) the code is something like this:

        self.file = "Dropbox:somefile.txt"
        local txt = self:header()
        for i, l in ipairs(self.dataBuffer) do
            txt = txt..table.concat(l, "\\t").."\
"
        end
        -- Crash occurs within saveText:
        saveText(self.file, txt)

Unfortunately, saveText causes a Codea crash (I believe this as I have debugged the program with debug calls which output one Dropbox file per call). It crashes more often when the program has been running for more than a single trial. It does so more often when the number of movements in a trial is 50, when compared with 10 movements/trial. The amount of data produced and stored in dataBuffer and the local txt variable in a trial containing 50 steps is no more than 1 MB.

I’m keen to either find a workaround or a suitable debugging approach. I’m not near a Mac right now, but later on I plan to have a look at the iPad Air 2’s crash logs to see if they shed any light.

I tried building a sort of dummy program, by taking the real program and cutting out the animation so that it simply does the table.insert() with some dummy data, followed by the save step. That also crashes, but more intermittently than the real app. It allows you to run the program for as long as you like, adding 10 entries to self.dataBuffer on each call to draw, displaying the length of dataBuffer on the screen as you go. It sometimes crashes while it’s filling dataBuffer, and if dataBuffer is allowed to become greater than about 10000 entries, the process of populating the local txt variable appears to hang, but I’ve not yet been patient enough to allow it more than about 10 minutes for this - Codea doesn’t crash in this case though.

I have colleagues who have seen similar crashy behaviour with various other versions of this program, all of which share the same “save to dataBuffer” then “saveText from dataBuffer/local txt”.

All the best,

Seb James

@sebjames - welcome to the forum (and it really helps to have a well written problem description like this).

I’m not an expert on saving text - others may have better ideas - but there are a couple of things you can try. One is to use Lua’s text file functions, which may be more robust. Here is dave1707’s code for this.

function setup()
    local file = os.getenv("HOME").."/Documents/Dropbox.spritepack/myFile1.txt"
    txt="abcdefghijklmnopqrstuvwxyz\
"
    
    if file ~=nil then
        writeData(file)    
        readData(file)
    end
end

function writeData(file)
    print("----- start write -----")
    wFd = io.open(file,"w")
    wFd:write(txt)    -- write some letters from txt
    for z= 1,20 do
        wFd:write(z.." ")    -- write some numbers
    end
    wFd:close()
    print("----- stop write -----\
")
end

function readData(file)
    print("----- start read -----")
    rFd = io.open(file,"r")
    while 1 do
        data=rFd:read()
        if data==nil then
            rFd:close()
            break
        end
        print(data)
    end
    print("----- stop read -----\
")
end

I suspect too, that doing all the string concatenations before you save to disk, may cause problems if the string gets too long.

Is there any reason why you wouldn’t save the data for each movement as a string rather than as a subtable, then you can simply use a single string.concat statement to create the final string, before saving?

Ah, fantastic - I had the impression that I couldn’t use the regular lua file i/o (on iOS|with Dropbox files) and in any case, this saveText approach was coded by the original author of the program.

I’ll try regular file i/o, because I’ll be able to write line by line to the file (which is the way I’d normally approach this).

Thanks for the help!

Seb

Good luck!

I guess you know that having saved the data to your Dropbox folder in Codea, you need to manually sync the folder, in order to access the files from another device or PC.

You should be able to use saveText with Dropbox.

Try removing .txt from the name, self.file = "Dropbox:somefile"

@Ignatz: I found out about the manual syncing, so yes, that’ ok. @yojimbo2000: The saveText function works - it just crashes when the program puts too much data into txt or the dataBuffer. That said, it shouldn’t - afaik there should only be resource limits on the size of a string or table in Lua and I’m not hitting those on this device.

So, I tried @Dave1707’s file code. It doesn’t seem to work for me. I get the following error:

error: [string “-- file access…”]:31: attempt to index a nil value (global ‘wFd’)

@sebjames Here’s something that might work. I’m not saving everything you were because I didn’t know what it was, but I can drag my finger around the screen for 20 seconds without a problem saving the data. This is just an example of saving the data, so I don’t have everything you were trying to do in your code. You can look at the Dropbox result file to see what was saved.

function setup()
    tab={}
end

function draw()
    background(40, 40, 50)
    fill(255)
    for a,b in pairs(tab) do
        ellipse(b.tx,b.ty,5)
    end
end

function touched(t)
    if t.state==BEGAN or t.state==MOVING then
        table.insert(tab, {et=ElapsedTime,tx=t.x,ty=t.y,px=t.prevX,py=t.prevY,
        dx=t.deltaX,dy=t.deltaY,st=t.state,
        uax=UserAcceleration.x,uay=UserAcceleration.y,uaz=UserAcceleration.z})           
    end
    if t.state==ENDED then
        str=""
        for a,b in pairs(tab) do
            str=str.."==========\
"
            for c,d in pairs(b) do               
                str=str..c.."="..d.."\
"
            end
        end
        print("save started")
        saveText("Dropbox:result",str)
        print("save ended")
    end
end

Ok, If I stick an assert around the io.open() call, I get a more useful error:

error:[string "-- file access..."]:24: /private/var/mobile/Containers/Data/Application/219FAD1D-F8BF-4959-8C0B-6C8791DC27DE/Documents/Dropbox.spritepack/myFile1.txt: No such file or directory

Seems I can’t create that file…

Change the spritepack to assets. spritepack was changed to assets a few Codea releases ago. If you want to use saveText, see my code above.

    local file = os.getenv("HOME").."/Documents/Dropbox.assets/myFile.txt"

@dave1707 I think assets is just in Codea 2.3.2 (the beta)? I might have lost track of this though. @sebjames , if you’re not beta-testing Codea 2.3.2, try Dropbox.assetpack. Also, as I suggested earlier, try saveText without .txt

@yojimbo2000 You’re probably right. I’m loosing track of what’s changed where.

It’s “Dropbox.assetpack” in the current release version of Codea, 2.3.1(47). Now to see if that approach solves the crashing.

@dave1707 - thanks for the suggestion - I think that the code you just posted is more or less the same thing that our program is already doing - the one that causes the crashy behaviour!

Thanks everyone, using regular Lua file i/o into the Dropbox.assetpack directory seems to have sorted out the crashing problems.

I think there’s an underlying bug associated with large strings and the saveText function, but I don’t have a nice standalone demonstration of the problem other than the rather long app (which I can’t share at present as I’m not the original author and it’s not yet open source, though it may be in future).

All the best,

Seb

@sebjames how big were the strings you were saving? (string:len() = number of bytes).

I created a 50 count for loop that would read a 1.27MB text file into a string and save it again. It had no trouble doing the read or save which took about 5 seconds to do it. The only thing I can think is wrong is constantly adding something to the end of a string which would probably use up a lot of memory causing a crash.

Yes, I suspect it’s caused by repeated concatenation.

@yojimbo2000 Probably about 1MB at maximum. It was also crashing when the string length was shorter than this (~675 kbytes). I was concatenating lines of about 150-200 bytes in length, so there were a few thousand concatenations per run.