Managing text-based assets. Pros and cons of GlobalData/ ProjectData vs ReadText/ saveText

Hi

Does anyone have a file manager for browsing files saved in global/ project/ local data?

There’s this one, but it doesn’t work too well (the buttons don’t seem to do anything, the columns are too narrow to see the full file name etc), perhaps because it hasn’t been updated for several years.

http://codea.io/talk/discussion/1738/tutorial-18-loading-and-saving-complicated-tables-to-global-data

Also, is it possible to query within Codea, the size of files (in bytes/ kb) saved in global/ project/ local data?

I thought I’d check whether anyone has one before writing one myself.

thanks

@yojimbo2000 the problem is that there is no function to list the files present inside a directory. Or, there is one in lua io library, but apple rejects this function. So you cant use someone else’s code to list your files.

You have listGlobalData etc for the global data (most of my files are saved in global data to be honest). And if a file-manager program was made a dependency of another program, I think the projectData commands within the file-manager dependency would access the data of the host program (I need to test this. But certainly the projectTabs commands behave this way).

Just did a quick check. Yes, a dependency does read the projectData and localData of the host program calling the dependency.

I’m just after something fairly light for browsing/ deleting stuff saved with saveGlobalData, saveProjectData, saveLocalData etc. Being able to check file sizes would be a nice bonus, but I’m not sure if it’s possible.

I do a lot of work with blender models, so my global data has now got tonnes of models, so I need a GUI, something a bit quicker than deleting files one-by-one with saveGlobalData(filename, nil). It wouldn’t be hard to make, I was just assuming that someone must have written something along these lines already.

To get the file size, just read each file and count the bytes. I already have something that lets you scroll thru the names of global, project, or local data. I’m seeing if I can add a browser to it.

Are all Lua strings one-byte character encodings? Is it just as simple as length of string = number of bytes?

I believe so

@yojimbo2000 I think Unicode character and Emojis are multibyte characters.

Or, perhaps instead of saving files with saveGlobalData, I should save them to Documents or Dropbox with saveText, then I’ll be able to see/ browse/ edit/ delete them in the regular Codea asset picker, and I wouldn’t need a globalData browser/ file-picker.

Tbh I’m a bit vague about this, can someone explain to me what the differences / pros-and-cons are of using the GlobalData commands vs the readText/ saveText commands? Perhaps readText/ saveText etc is meant to be for files you want to be more accessible/ editable, whereas the GlobalData is intended for files you want to be somewhat out of the way as it were?

how to delete singlebyte chracters or multibyte chracters in a mixed text?regex?

Having done a bit more testing, it seems that readText only works for files with the extension .txt, it can’t see, or load, for instance, a text file with extension .lua, or .lua.txt.

If you add two extensions, eg “MyFile.lua.txt”, at first that seems to work: you can see and select the file in the asset picker, and you can edit it by pressing the little pencil icon (a really nice feature, btw, for making quick edits to text assets). Unfortunately however, readText fails to read files with two extensions, it just returns nil. It has to be “MyFile.txt” to work properly. This is a shame I think. The two extensions trick would be an easy way of including all sorts of text-based assets.

This is a problem for working with 3D files, as although they’re basically text files, you’d want to know whether it was .obj, .ply, .mtl etc. I guess you could get round it with a prefix or suffix, eg “OBJ_myModel.txt”, “myModel_OBJ.txt” etc

What do people think? It would be very handy to be able to select 3D models in the asset picker, if they were saved in this way (you’d only be able to see the model name and its text source of course, not an actual preview of the model). And it would save me from having to writing a browser/ deleter for GlobalData.

It would make the Blender → Codea workflow much quicker. Batch changing the extension to .txt and adding an appropriate prefix wouldn’t be that hard (you might even get the Blender exporter to do that for you. Or you could set up an automated folder action with Hazel or Automator to process .obj, .mtl files).

Export from Blender straight into Dropbox/Apps/Codea/, then in Codea open the readText asset picker, navigate to Dropbox, hit sync, select the model you require.

@dave1707 @SkyTheCoder I’ve been doing some more experimenting. The io.seek command gives you the size of a file. In my testing so far, it returns exactly the same figures as string.len. Also, string.len counts emoji as multi-byte characters, ie if you add one emoji to the file, the string.len increases by 4. So you might as well use string.len as setting up io.seek takes a bit of work:

local c = io.open(file, "r")
print(c:seek("end")) -- size of file in bytes
local data = c:read("*all")
print(data:len()) -- length of string. Returns same value as io.seek

Also (slightly off-topic), I only just realised this, but the string commands, like the io commands, can be used in object-oriented fashion, eg myString:len() -- same as string.len(myString). I must have read this at some point somewhere, but it didn’t sink in until just now.

@yojimbo2000 Thanks, learn something new everyday. One thing I was thinking about using saveText/readText. Someone can actually look at the file, change something, or even delete it. Whereas using saveGlobalData, that’s more protected and only a program can alter the file or the data. So it just depends on how you want to use it.

Yeah, I’ve realised that if I’m wanting to write a file manager for browsing/deleting global data, that probably means I should’ve been saving the files as assets instead of data, because then I could just use the built-in asset pickers. I think rather than building a manager for globalData I’ll start migrating my 3D models to text assets. I guess the text asset feature is a relatively new one (it arrived at the end of last year I think?) which is probably why I haven’t investigated it until now. It’s a shame that it doesn’t let you use file extensions besides .txt though.