Persistent Data

Ooh - my last post brings me to something new.

Is there any way to persist data between runs? High Scores, preferences, whatever.

I don’t see one. It would be REALLY handy.

As an API, a simple table that would save itself between runs would do the trick.

I think we’ll end up doing a simple key->value API for this.

Lua bindings for the iOS embedded sqlite would be terrific.

true, embedded sqlite is slow as a turtle, tables are speedy :slight_smile:

+1 for simple - a key-value store seems sufficient. I can think of two good reasons to use SQLite - SQL, and being compatible with (and so being able to read/modify) databases from apps that use SQLite already - and neither are really applicable to the Codea use-case on the ipad (apple is NEVER going to let this app muck around with the data from other apps…)

Dylan has just added local and project data storage to Codea. A quick explanation of what each one is:

Local storage: This is stored on your device on a per-project basis. Good for things like high scores – these won’t be passed on if you ever share or give the project to someone else. They are local to your device only.

Project storage: This data is stored inside your project folder and will be sent with your project if you share it. It is stored as a Data.plist file inside your Project.codea package. I guess you could use it for procedurally generated data that you want to keep, level data, things like that.

This is the API:

-- Local storage
readLocalData( key [, defaultValue] )
saveLocalData( key, value )
clearLocalData()

-- Project storage    
readProjectData( key [, defaultValue] )
saveProjectData( key, value )
clearProjectData()

An example of how you might use it:

-- Highscores
function setup()
    highscore = readLocalData("highscore", 0) -- default to 0 if no highscore set
    -- more setup...
end

function draw()
    -- some awesome game

    if gameState == GAMEOVER then
        if currentScore > highscore then
             saveLocalData( "highscore", currentScore )
             -- congratulate player
        end
    end
end

Way cool. I was only thinking of local storage - project storage was something I hadn’t thought about. Are you going to expose either of these in the editor? And are they 8-bit clean?

Expose data as tabs within the editor? Not for this release, but it’s an interesting thought, especially for project data.

I’m not sure what you mean by 8-bit clean. Can you explain?

Can I save binary chunks? I expect this is “yes”, but gotta ask. Strings in lua are 8-bit clean - if I put in, say, compiled font data - will I get out what I put in byte-for-byte?

Sorry - I’m old. Way back in the day, it wasn’t assured, and you had to encode things to make sure you were ok. Which - I can do, if need be.

The problem, of course, with 8-bit clean is that an editor or display for it in a tab will be more difficult; how do you properly display an ascii null (00)? or other ‘unprintable’ characters, in such a manner that a human can use them? “\r” type shenanigans, perhaps.

The stores can only store numbers and strings at the moment (strings cannot have null characters as they are converted to c-strings, but it will check and throw an error in this case).

To be honest storing binary data didn’t occur to me! It should be relatively easy to add. Perhaps using something like base64 encoding is fine instead though.

You can use escape characters to define null characters and whatnot (‘\0’ etc) in lua.

In my particular case, I’m thinking font data - going full-on binary, I could compress it (rather than simply picking an efficient encoding) and halve the size.

But I already have base64 stuff pre-rolled (it turns out it doesn’t reduce my source data by that much over my home-rolled encoding, so I didn’t use it), so that’s fine.

I’m glad I asked :slight_smile:

Is there a limit to size? Can I put in a 1 meg string? I guess this is all going to a plist, and I’ll have those limitations, whatever they are. Not complaining! The world is limitations - knowing what they are and working to overcome them is what life does.

Yeah its using NSUserDefaults (for Local) and a plist file (Project) for storage, so whatever the limits are on those! I’m guessing they can get fairly large though (but it would slow project loading a fair bit if its too big). All the values are stored in RAM while the project is running, so its probably possible to store too much in there and fill the RAM up (but unlikely).

+1 for binary data storage…

binary data would be nice. I’ve stored mesh data in image object as a hack, its a bit mesdy, but you can see what color your data has! :slight_smile:

You can store binary data using a string and the save data methods. Lua does not use null terminated strings, so you can store whatever characters you want, including 0 in there.

I seem to crash codea whenever I save a string that’s got a zero in it…strange.

I think the post http://www.twolivesleft.com/Codea/Talk/discussion/1692/saveglobaldata#Item_10 is the solution to this discussion.

Codea may not care - but does the plist format? I suspect there are ‘fatal’ combinations of binary data you could put into a string and save to global storage. (I am thinking carriage returns and line feeds may bollix it up)

io.* and files are probably the right long-term solution; just not available enough yet.