Functional proposal

Hi Beta Testers

I have been thinking about adding common functional programming operations to Codea as part of the standard library. Basically map, reduce, filter and so on. I find that I use these more and more in programming, avoiding explicit loops where possible. Not for performance, but for composeability and readability.

I’d like to include an established Lua-based functional library in an update, but wanted to get your thoughts.

(I’m looking specifically at this library http://rtsisyk.github.io/luafun/reference.html)

Some possible examples:

Apply a transform to all points

xform = matrix()
points = { ... }
px = map(function(p) 
        return xform * p
end, points)

Take the first two touches and convert them to vec2

touches = { ... }
coords = map(function(touch)
    return vec2(touch.x, touch.y)
end, take(touches,2))

Remove a touch with a specific ID from a list of touches

touch = ...
touches = filter(function(t)
    return touch.id ~= t.id
end, touches)

Draw 10 randomly positioned ellipses

each(function(x, y) 
    ellipse(x, y, 100)
end, take(10, zip(rands(0,WIDTH), rands(0,HEIGHT))))

There are many more simple composeable functions and generators available, and because the above library treats strings as iterable sequences you can use these functions on strings wherever you would use a table.

I haven’t yet used this particular library, but it seems well thought out and documented.

Great, reminds me of JavaScript. Hopefully at some point over Christmas I’ll get a chance to try that library out.

Edit… Though it says LuaJIT is highly recommended for performance reasons

just to give you some input:

  • i am not sure this improves readibility: the syntax looks a bit esoteric to me … more documentation to read and remember; also all these functions can be easily defined by whoever needs them. So if there is no performance gain, …
  • i see a touch example above; what i would really like about touch is having the touch object not be userdata but a real table, so i can add my own fields to it.

thanks for the explanation.
For slicing the lua ‘move’ command is doing it

@Jmv38 I forgot about table.move, though I think it only works for array-style (integer indexed) tables.

Note @Jmv38, table.move is a great example of the higher-level semantics that functional operations try to express. This proposal is to add the most common ones into the default namespace.

@Jmv38 These kinds of functions are very common in lots of other languages though, JavaScript etc. I wouldn’t call them esoteric.

@yojimbo2000 yes, LuaJIT would generate more optimal code. Unfortunately it’s diverged on Lua 5.2 and the JIT aspect is unavailable on iOS anyway.

@Jmv38 the argument for readability comes from the fact that instead of a generic loop, you are explicitly declaring your intent. When you say filter you are filtering a list, when you loop through a table and accumulate values to remove it’s a lot less clear.

Same goes for map. Seeing that means you are transforming a list from one type to another.

Same with slicing, e.g

local first5 = take(5, t)

vs.

local first5 = {}
local idx = 1
for k,v in pairs(t) do
    first5[k] = v
    idx = idx + 1
    if idx > 5 then
        break
    end
end

Edit: with table.move, though this only works for array-like tables

local first5 = {}
table.move(t, 1, 5, 1, first5)

(I’m sure there’s a better way to write the pure Lua version. Just can’t think of it right now and I’m typing on my phone.)

I looked thru the reference and it looks like someone with nothing better to do took existing functions and just made them different. Codea/Lua is small and easy to use. I don’t see the point in adding to its size by adding functions that already exits, especially functions that aren’t easy to understand what they’re supposed to be doing. If things are going to be added, add new functionality. There are functions that I requested years ago that I think would be useful and there’s nothing yet that makes use of the microphone. That’s just my opinion, keep it small and simple and keep adding new functions.

@dave1707 agreed on the microphone and having other “feature” type features. I am currently working on:

  • Mac OS X, iPhone, tvOS support
  • Split screen editing
  • Tokenised code snippets
  • Project folders/collections on the browser screen
  • Native editor tabs for GLSL shaders, project assets
  • Copy entire project to clipboard including encoded assets
  • Better tutorials
  • Rewriting the Lua lexer and parser in Swift (it’s currently not great)
  • iCloud document picker support
  • Voxel API from Codea Craft
  • ReplayKit support — this would allow the mic to be used in screen recordings

And past that I’d like to work on:

  • Debugger
  • Local git versioning
  • 3D model support
  • Bluetooth API
  • Audiobus / sound processing API

Not all of those are going to work out; but I want to assure you that I am working on things other than this proposal.

That said, what would you like to see in terms of API features and changes?

To get back to the functional library:

Please note that these functions aren’t made because someone had nothing better to do. These are well known, and well defined operations that have been around for a very long time. The ideas behind functional programming are to keep your program composed of small, clearly defined “pure” functions with no side effects.

Functional programming has its roots back in the 1930s with Lambda calculus, so it’s not a new thing. Many of the ideas in functional programming are becoming more and more popular because they are good ideas.

The reason I’m interested in them is because I have been using them more and more in my own code. It has had a positive effect on my thinking and my programming. I was hoping to encourage the same through Codea’s standard library.

As @yojimbo2000 says, modern versions of Javascript, the current version of Swift, among other languages, encourage the use of these functions. And once you start using them, generic for or while loops can feel unnecessarily complex and inelegant in comparison.

@Simeon Maybe after using those functions they would seem familiar. I’m not saying don’t add them, I just prefer simple and easy to read code. If I looked thru a program that used a bunch of those functions, I wouldn’t have the slightest idea of what was happening. I guess it’s just a matter of using them and getting used to their functionality as with anything else. As for new functions, I would still like to see one that returns a list of existing projects and one that allows the creation of a new project. Both of those would allow the backup and restore of projects. I’m currently using a plist to get the names of all the projects for backups, but that’s still a manual process by selecting projects as dependencies. I also have a program that allows me to search thru everyone of my projects for anything I want. That also uses a plist that requires manual updates. I have over 400 projects, so looking thru a dependency list to add new projects takes time. Your above list of new thing looks impressive, can’t wait.

What i meant by ‘esoteric’ is exactly what @dave described very clearly above. Note that i dont want to argue on the interest of these functions, i just wanted to give my pov to Simeon, because he was asking for it. I am sure that these function certainly make sense if other langages / people use them, so dont take my opinion for an ‘authorized’ one.

Talking about new functions, i would love a 100% autonomous app store project generator (one-touch-app-publication). It must be feasible. Even having a dev licence and a mac, publishing an app demands still a lot of reading and thinking, when i would prefer to focus on the codea part only.

It must be feasible

The App Store is a walled garden, and Apple is the gatekeeper/ ogre! If they say “you need a Mac, Xcode, and a dev store subscription to publish to the App Store” then there’s very little any of us can do about that.

“There but for the grace of Apple go I”, basically.

This is precisely why iOS IDEs like Codea and Pythonista are so very precious though, because they represent the iOS open source community that cannot (because of the App Store’s rules) otherwise exist. And even then, you can sense that Apple aren’t too happy about people sharing code, which is why they forced the Pythonista and Codea devs to remove “Open In” functionality etc.

I am delighted though at the list of features that @Simeon laid out above. OS X support is particularly intriguing (and a little unexpected), as it suggests that Codea will expand beyond the iOS platform. OS X is still, basically, an open platform (ie you can install and run code from anywhere), so that should mean that the future of Codea as a platform is a bit more secure.

At the risk of droning on like a boring old man, Codea reminds me of a wonderful IDE called Blitz Basic, first released on the Amiga over 20 years ago by some New Zealand developers called Acid. I was pleasantly surprised, a few years back, to discover that Blitz Basic is still going relatively strong all this time later, having outgrown the Amiga and been open-sourced, and now being used for mobile and web development. Whatever path Codea takes, it could I think be just as long-lived.

Merry Christmas all of you, and here’s to 2016!

@Jmv38 @dave1707 I used to feel pretty much exactly the way you do about these sorts of functions — unreadable and esoteric. @Dylan was the one who convinced me otherwise, and since coming to rely on them, I find them pretty valuable.

But yeah, I have some issues with Luafun. In particular I dislike that the function argument is passed first, instead of the iterator/table/string.

@yojimbo2000 we have a working runtime for OS X under the old single-threaded render model. Last night I started porting the latest runtime.

@Simeon I read a lot about functional programming lately and totaly second your idea of pushing in this direction through codea’s api’s!

I also love your “working on”-list! All great things that should (must) be added :wink: - I want it all!

Please keep working on Codea and shaping your vision. I think you are on the right track.