Thoughts on I/O in general and Codea programming model

So, I was trying to do something today, and my experience has changed my view on I/O in codea - specifically, both sockets and file-type I/O. I’m putting this in “beta” because this one was specifically dealing with sockets, but frankly this would happen with anything that blocks (including, I suspect, persistent storage, if you did something pathological with it, as I may well do at some point)

I had this idea to set up a Codea app as a front-end display server for other code running on the iPad (or anywhere, really, if you wanted), perl in this case; the idea is perl would connect to a Codea app listening on a socket bound to localhost, and the Codea app would do what perl asked - say, drawing on the screen. it’s a way to get Codea language-agnostic, in a way. Not a new concept, it’s basically what an X-Server does. Was just doodling to see how it would work, won’t pretend it’d be more useful than just writing the code in lua (although I could use GD, for example, to read a compressed image, in effect extending the language).

What? Why yes, I am insane. Thank you for asking! (“The only difference between myself and a madman is that I am not mad!” – Salvador Dali)

So I have an echo server up, and I realized - in hindsight - what should be obvious: When I block on accept(), the main thread… blocks. That means draw() never returns, because it’s blocked, and the whole Codea hangs - you need to kill it entirely. Presumably if I connected to it fast enough (or, more accurately, frequently enough), this would be less of an issue, but bleagh - I don’t really want to block, ever, not in the paradigm we’re using here.

So now I’m looking at doing a select() in the draw loop, and only doing accept() if there’s data present, but the docs for Lua’s socket libraries point out that even if select() says data is ready for read at the socket… it might not be. (YAY!) and now you block() anyway. That’s Bunk, but that’s how it works (from what I’ve read - it may be old, and I may be able to work around it by reading nonblocking, but I get the idea from reading, again old stuff, that that may not work in lua due to implementation choices). And really - I don’t want to busy-poll (although I’m looping anyway so why not); what I really want is to dispatch a request, and have a callback, like draw() and touched().

To put it another way, for a normal http get, we don’t want a blocking content = http.get(URL) command - because if the site is slow or unresponsive, we’re hung, and can’t even break out. What we want is more javascripty (or dare I say, node.js’y), where we’d say “http.get(URL, callback)” and have Codea (or the libraries) call callback() asynchronously when the get is finished. Likewise, for a server, I don’t want to bind() then block on accept() - I want to bind with a callback, and when data is present on that port, I get the callback called with the data as a parameter.

So you know where I was saying “just give us sockets, we’ll work it all out”? Well - I still say that, but the end-goal has moved. If you give us a super duper http.get, and it blocks… it’s going to be… let’s say “less useful” than a http.get that registers a callback.

The same applies for file IO (or persistent storage IO), but those are far less likely to block for a noticeable amount of time.

So I’m going to futz around tonight (and likely on for days or weeks) with doing just that - making a socket/http interface (client first) that will register a callback, do the work (presumably with a helper function in draw() to give it some processing time, although I should look into the whole coroutines thing), and in the end presenting to the end-user the interface I suggest above: Say “go grab this resource”, then forget about it, and when the resource is ready, a callback gets called with the resulting data.

Why do I waste time with a feature that Apple may simply disallow? Because I can, baby - because I can.

Anyway - I just wanted to bring this up, if it wasn’t super-obvious to me, maybe it wasn’t to others. And, maybe more importantly, it does indicate a need to do something to shelter beginners from this; not necessarily remove access (!), but wrap it up with a more fitting interface for the callback oriented programming style Codea uses.

The plan for http.get has always been not to use the LuaSockets one. The plan has been to offer a blocking and non-blocking variety within the same function call. However given that network functionality is uncertain on the App Store, we can’t justify implementing it (perhaps in a mac version?).

Lol - did you just suggest an OS X version of Codea?

I think I like the idea.

You’re competing against Love2D there, and it’s not bad.

But if we could run the same code both places (a la the LoveCodea stuff) but without the different APIs, that would be fun, and frankly easier to develop (I love the ipad as a platform for dev, but sometimes it’s faster to sit here at my desk and use a real keyboard).

The socket talk above is simply feedback; if/when we get the status of the concept of network connectivity straightened out with Apple - we need to provide non-blocking calls for regular coders, because blocking doesn’t fit with the processing-type model, and coding non-blocking with the standard libraries is fraught with peril.