Beta Discussion - Sockets (1.2.8)

I’ve just uploaded 1.2.8 (build 1) to TestFlight. It includes the full LuaSockets API. I need your feedback on this.

You can find the LuaSocket documentation here:
http://w3.impa.br/~diego/software/luasocket/home.html (click reference)

Here’s a little sample code you can try. Let me know if there are any issues.

function setup()
    print(socket._VERSION)

    for k,v in pairs(socket) do
        print(k)
    end

    c = socket.connect( "www.google.com", 80 )
    c:send( "GET index.html HTTP/1.0\\r\
\\r\
" )

    while true do
        local s, status = c:receive()
        print(s)
        
        if status == "closed" then break end
    end
end

function draw()
end

I’ll mess with this today, for sure! I’m very happy you added all the helper libraries - that’s going to make code a lot easier to write, and read.

Other useful adjuncts would be zlib, png/jpg/gif decoders (and/or encoders), and openssl. But this much alone opens so many doors I almost don’t know where to begin.

Well, you knew that as soon you issued the socket libraries, I was going to ask for media rendering. Consider yourself asked…I want an audio and video rendering object now. Will play this evening. Very exciting.

Alternatively to the above:

b, c, h = socked.http.request("http://www.google.com")
print (c)
for k,v in pairs(h) do
    print(k,v)
end
print b

c is the return code (2## is ok), h is headers, and b is the body.

I’m giddy with delight. :slight_smile: I’m already using this in my prototypes to download font data - now I have to go figure out why my base64 encode/decode aren’t working. Idea is to see if the data required is in the persistent cache - and if not, fetch and store it.

PS. My original plan of putting shared assets (like the font data) on github has hit a snag - github only does https. If anyone knows a good place to put binaries that can be accessed by http - post please.
I am hosting them from home for the time being, but being in America, my bandwidth sucks :slight_smile:

Hmm - I don’t see socket.mime there (but socket.url is).

mime exists outside of socket. You can use mime.encode and so on.

I’ve been thinking about the inclusion of sockets a lot. In some ways it feels slightly inappropriate to include in Codea. Because it doesn’t offer a simple path to creating multiplayer projects, and doesn’t even expose the bluetooth multiplayer components of iOS.

Downloading remote data could be done better, as well, I think. Sockets are not very accessible.

A networking API appropriate to Codea should sit at a higher level. What do you think?

Off the top of my head, an API I’d actually want to use would look something like this:


-- Shows a UI listing all nearby iPads that are running Codea
multiplayer.showLobby( name )

-- Callback occurs when a user connects to your session
multiplayer.userConnected( id )

-- Callback occurs when a user sends data
multiplayer.receivedData( data, id )

-- Send data to all users or a specific user id
multiplayer.sendData( data [, id] )

-- Download arbitrary data from a URL
-- If a callback is provided the function is non-blocking
network.get( url [, dataReceivedCallback] )

-- Maybe the following as well
network.post( url, data ) 

The LuaSockets API is really nice, but it’s not very accessible to many users. What do you think about the above API?

I think so long as the lower level API is also available, there’s nothing wrong with presenting and suggesting a higher level interface. Case in point - I posted using socket.http above as a better alternative than doing http via raw sockets. But - one of the things I’d like to talk to is a redis server, and that’s going to require telnet level tcp sockets. Could I wrap it and hide it behind a RESTful http server? I guess - but we wouldn’t get much out of it other than an additional point of failure and slower speeds, and would likely break some thing (PUB/SUB come to mind). (I also have a concept floating around of talking to javascript on a browser via websockets).

I do have to say - when I think “multiplayer”, the last thing I think of is bluetooth - I think networked games. I play (or played) World of Warcraft on my desktop, Words with Friends on the ipad - networking. I really can’t think of a game I play against people using bluetooth, even on iOS - that’s for controllers (which would be insanely cool, by the way - If i could pair up with a wiimote or playstation controller, it’d be solid gold).

As for “simple path to creating multiplayer projects” - I wouldn’t define Codea so narrowly; it looks to me based on what Andrew and others are doing that there’s a large educational audience. Maybe I’m a dreamer, but other than very CPU or data intensive tasks, I don’t see why I can’t do with Codea on my iPad anything I wouldn’t do on most any computer I’ve used in the past. I’m actually considering using it for some custom monitoring/data-visualization tasks for my work, now that I can use sockets to connect to the real world. (Yes, I could do it in javascript. I probably should do it in javascript. But - I don’t like javascript, much. Lua is nicer.)

“Sockets are not very accessible” - undoubtedly true. But a lot of low level coding is like that - we build on top of the less accessible things to make it easier for those who follow. Heck - I was gearing up to write my own http routines, because all I was hoping for was simple sockets; getting http and dns and mime (which has base64 encode/decode, w00t) is all a great bonus. And I was going to do that because for most people who just want to hit google and query the weather (to make their flight sim match the real world), sockets is really too low a level to mess with.

The funny part is (I was just rereading the above) - the socket stuff you exposed isn’t really sockets - it’s a high level wrapper over the real ugliness of BSD sockets. I’ve written code that had to bind to sockets and accept() and the lot; that was ugly. This stuff is heaven compared to how it used to be.

Bluetooth multiplayer is the standard on iOS and mobile devices. Bluetooth is used to establish connections and locate players, WiFi is then used for data transfer. Use of WiFi like this is part of the Bluetooth standard.

I’ve written with C sockets as well, and it isn’t nice. LuaSockets are nicer but it is still quite a large library and isn’t really good at getting quick results with little effort.

Aside from a redis client, what else would you need LuaSockets for that couldn’t be represented in a simpler, higher level API?

Let me ask you - what games do you play multiplayer using bluetooth? I mean - most of the friends I play games with are miles and miles away (I tend to play turn-based games), and my local bluetooth experiences have been poor at best - I have yet to find a game that “just works” with bluetooth. (I’m actually curious - every bluetooth game I’ve tried has been horrid when it came to connectivity, which is probably a big part of why I don’t identify bluetooth with multiplayer gaming… if you found one that works, I wanna play it! :slight_smile: )

Let’s see - what might sockets (raw sockets) be used for:
redis (and others like it - memcached comes to mind), irc client, music/video streaming, websockets (with javascript peers), telnet (of course). Frankly - less and less as time goes by.

The idea isn’t so much “what do I want to do” as “what may come down the pipe 3 months from now where http isn’t the protocol”.

It would be perfectly reasonable (in my view) to say “The Lua Sockets layer is supported - details at this URL. We recommend users use the following higher level APIs if possible”, then go into detail about http and smtp and the like (or into an even higher multiplayer.* api if you have one).

I guess I’m confused - Is there some reason you’d want to hide the lower level APIs?

I play Street Fighter IV over Bluetooth. Aside from trying a few racing games, that’s the one I actually play.

I also play turn based games. Those require a server and push notifications, we can’t offer those features through Codea.

I’d want to hide the socket APIs for the following reasons:

  • There are an extra 8-or-so large Lua files to interpret each time your project runs
  • They add a lot to autocomplete and documentation
  • They are large and complex, very few users will want to use them

To me it would be akin to exposing low-level drawing functions, such as glDrawElements, to the user. I would never expose that function because it goes against the idea of getting things on the screen quickly.

I’m open to argument on including sockets. Perhaps multiplayer features are not really important and all we need is a http get/post API. Or maybe we really do need sockets.

if you weren’t open to argument, I wouldn’t be wasting my time :slight_smile:

Things to consider:

“Those games require a server” - so does http. If you aren’t concerned about providing a webserver (and you shouldn’t be) - you shouldn’t worry about it for sockets either. One of the things I was considering using redis for was matchmaking, because there’s no good way (yet) to say “please type the IP of the person you want to connect to”. If you want to add zeroconf/Bonjour (which is probably already there, somewhere, you’d just need to expose it) that’d be cool.

“8 large files to interpret each time your project runs” - until they take a measurable amount of time, that’s premature optimization. I push play on an empty project, boom it goes - so that burden isn’t something to worry about IMHO. If the situation changes, deal with it then. Frankly - most of my startup time, demonstrably, is doing things like building images; the way to make that faster is to give us real access to new spritepacks and binary storage. Besides - to provde http, you’re going to have to interpret and load the socket layer underneath it anyway - closing off access to the API won’t speed anything up.

“they add to autocomplete and documentation” - the API isn’t that expansive for autocomplete. As for docs - you don’t document the vast majority of the Lua language, nor should you be expected to (I was surprised to find strings there). I’m very comfortable with the in-program documentation covering only Codea-Specific things - by definition, everyone using Codea has a really nice web browser and can go look at the Lua docs, including docs on Sockets and the higher level APIs dependent on them. “Sockets are an advanced topic for advanced users - documentation is available here” with a link to the luasockets page is plenty good.

“They are large and complex” - granted - “very few users will want to use them” - I’m not sure about that. I see quite a few people asking for them (or more generally, for connectivity) on the forum (that and text), and I suspect there are more lurking.

As for exposing low level drawing functions as an analogy - consider it! So long as you aren’t preventing use of the high level ones, I see little harm in it.

I see your userbase, especially your early adopters, as “tinkerers”, and educators, and learners, and indeed players. We all, to a greater or lesser extent, don’t mind diving in a bit and getting our hands dirty, or we’d not have bought the app - people who aren’t interested in this are playing Angry Birds. Indeed - a lot of the really new users are using it as an opportunity to learn how to code; I wouldn’t underestimate what they will want to do, or what they can eventually understand. I can see a reasonable concern about newbies getting confused - but frankly, that’s what newbies do, get confused; if they don’t get confused about sockets, they’ll never learn about them, and that’s not desirable either (knowing how things work is powerful juju).

I’ve said it before, it bears repeating: Codea is my favorite meta-game. It’s cool worrying about making sure the early levels are easy enough for beginners - but there has to be lots of depth for the more advanced players to be kept occupied.

This wondermark comic is very relevant (and hanging on my wall): http://wondermark.com/tink8/ - The insides of things (including code!) are beautiful - let’s see what they look like.

Re-reading, one additonal point - at some point, someone mentioned exposing GLSL. You don’t get much lower level - AND EXCITING - than that. Low level is power. Low level is awesome demos, and games where people say “I didn’t think you could do that in an interpreted language”. GLSL isn’t easy, but many things worth doing aren’t easy - and that’s ok.

Bluetooth multiplayer - GameKit - is an API designed for matchmaking. That’s what Bluetooth is used for.

The last thing I want is for people to have to find and type IPs, or even think about IPs when trying to set up a multiplayer game.

Here’s a middle ground that I’m thinking about:

  • We add an API that exposes bluetooth matchmaking for multiplayer games
  • We add an incredibly simple and nice http get and post API
  • We add only the following sections from the socket API: sockets, sockets.tcp, sockets.udp

My immediate application is for educational purposes, and my personal interest is in in networked applications, media distribution, and games.

For my educational purposes, I am most interested in a simple, high level networking library that will work anywhere, any time, on any network. I don’t want to have students worrying about which network someone is on, or whether my buddy’s bluetooth is turned on. In that sense, I think the general high level networking (not bluetooth, which only local) would be more useful with a simpler API with fewer exception and special cases. But I wouldn’t limit it to client APIs. I’d also like to be able to run a web server components in Codea and have students be able to work on client-server architectures, games, etc…connect their pads to sensors and so on…

For my personal interests though, I am also interested in high performance streaming applications where I would like access closer to the metal, as it were, to streaming buffer management, and also to audio and graphic/video rendering. A big part of that is decoding, and accessing things like H.264 encode/decode APIs, and audio codecs and so on.

And the hacker in me (and in some of my more advanced students) also want the low level graphics APIs for sheer performance.

I know personally, for myself, a bluetooth API for matchmaking will go unused; I simply don’t physically gather with enough people that I’d use it (and so I’m unlikely to write code to use it). Multiplayer == internet for me, and for everyone I know. (A bluetooth API to support things like game controllers is a whole 'nother bucket of fish - I’d be all over that action. I’m old. I want physical buttons.)

(Bonjour is another matter - I could see using that. But woof - talk about fairly complex…)

If you supported sockets, sockets.tcp, sockets.udp, and a robust http API (more than get and post - I use RESTful APIs, and that’s the way it’s going in the real world - we need to be able to at least override the methods if it’s not properly supported), what’s left to drop? ftp? smtp? Sure, I guess - I’ll be blunt, neither of those protocols is secure, there are better ways to do those things nowdays, so I can’t see using either of them much. But most of the rest of it - mime (which I’m using now, it’s LOVELY) and DNS and URL are going to be prerequisites for a robust http anyway - so I don’t see you as winning much there. And - a robust web API is going to need https (and openssl), just sayin.

I suspect this is much ado about nothing - I have a feeling the reality is those who want sockets already know why they want sockets, and those who don’t won’t care anyway, until suddenly someday they are trying to do something and need sockets - and are happy to find it’s supported. My long-term hope/goal is we use the networking code (and frankly, probably the http networking code) to wrap up things like IPs and matchmaking and the like, and your typical “I’m gonna make multiplayer pong” guy won’t have to worry about it anyway - they’re not gonna see sockets, and they’re not gonna see http either.

Right, Multiplayer == Internet. But you wouldn’t want to have to write your own server, host it, just to make a multiplayer game in Codea. Turn based wouldn’t work due to lack of push. It seems like a lot of development for a prototype. Maybe I’m missing something.

If we’re talking internet multiplayer I would rather Two Lives Left hosted a server and we build a Codea API to create sessions, lobbies and matchmake on the server. The server location could be changed, but would default in the iPad app to our server. It would make it really easy for users.

For developing a prototype or idea it actually sounds like a lot of work to code at the socket level. It doesn’t seem to support the idea that I can just spend one or two hours coding and have a multiplayer, networked game at the end of the session.

I’m not that opposed to including sockets. It just seems hard to justify for prototyping quick ideas.

@alvelda what you describe sounds like a central server that maintains network sessions between Codea users. Zero configuration, one-line-of-code creation of sessions, lobbies, and so on.

Regarding video, I don’t think even we have access to the h264 decoder on iOS.

Actually - the game I’ve been working on for years, on and off and restarting frequently, is more of what used to be “play by mail” - think of the old BBS games, where each player logged on whenever (or simultaneously) to do their turns and give orders and such - and then daily the game ran, and made results available. So - push isn’t so much an issue there, but yes - there’s a server side component. I’d hate to say “Codea is great, unless you want to do a game that talks to a server”.

Thinking about it - turn based can work without push, you just don’t get notified. Unless you used one of the many free messaging apps that support push via an email gateway… hmm, suddenly smtp seems more useful than I thought it might be. :slight_smile: (Meh - I’d still use a web gateway, many ISPs block port 25 from consumer connections to cut down on the spam…)

The problem with a two lives server (and I’m not saying you shouldn’t do that if you want) is that again, at least for me, anything where I need/want to run my own code server-side would be out. There may be value in providing that service and a real simple API to use it - but I don’t see the value in excluding other valid ways to achieve similar goals.

As for “prototyping” - I didn’t see that in Codea’s description, and I don’t think most people using it are “prototyping”, ie. making a throwaway version so they can later go on to do ‘the real thing’. Again - it’s a valid use, but I think most of us have no intent to move the code elsewhere - indeed, that’s the attraction of a Codea Player app. What I’m writing isn’t a “prototype” - it’s the real thing, written for a small but growing audience, same as when I wrote code for the 8-bit Atari. That’s not to say I wouldn’t enjoy and use either a player app, or an open source version and going the whole App store route - but that’s not something that’s blocking me from what I want to do. To put it another way - if we never see a player, that’s a shame, but I’m content to say (I hope at some point) “To play the awesomest game of all, you gotta pony up 8 bucks for Codea”.

And of course - none of that (or little of that) is relevant to Sockets versus No-Sockets. My biggest pro-socket argument is simply “why not?”. The code is there, written, works great in the beta, and enables a ton of cool things to happen in the long run - and the only real drawbacks I see are “maybe it’ll confuse someone”? I mean - what you have, right now, in this beta, seems reasonable to me - the lack of autocomplete and in-app docs are minor (I don’t tend to use autocomplete much, and the only things I look at in-app docs for are the codea specifics, and that’s mainly because I can’t view them online in Chrome! I’d much rather have docs on my big screen, and be coding on the ipad anyway…) Seems a slam-dunk to me.