New UI library in Craft, how can I contribute?

General
1

I have been putting a lot of thought and work into a new GUI library, studying Cider and SODA. I recently found in the new craft files that there is a plan for a new built in UI library in the works, and I like some of the ideas in there and would prefer contributing toward something that will be useful to more people then toward making something only for myself, or something that competes with a built in lib.

How can I go about contributing to this new library? I am currently outlining a revison to my own API with a lot of thought on how to make it easy to customize and extend, as well as have loads of ready to use and customize assets, from color pickers, numbers, sliders/knobs, etc…

I initially started making my own solutions from scratch and I found Cider and SODA. I was then planning to use those, mainly SODA, as it looked pretty and had some great slider tricks. In the process of trying to figure out how to customize the stylizing of SODA, I found the API a bit confusing and challenging to track down where to modify the look of objects and themes.

looking back at Cider UI design tool, I feel like it would be so useful to have something like this, greatly expedites GUI design. The Cider API is also really easy to grasp and extend. This also seems to fit in with things like the new craft voxel design tools.

2

@AxiomCrux

I would very much like to see your thoughts in practice once you feel it’s ready. A standard UI library has been much discussed and, as you have seen, some significant progress made.

FWIW, I too had trouble parsing how to configure things in SODA, and I wrote a couple helper projects to assist me, and also hopefully to spread the use of SODA, which is just so darn pretty it certainly warrants boosterism. In case any of it is helpful to you, I’m pasting in the links.

The first is a limited WYSIWG tool, for generating SODA buttons specifically, called SODA Designer. What was of particular use to me was that once I configure a button it can generate the code to make that button, which then can be easily cut and pasted into another project:

The second employs a paradigm I implemented for myself in Swift, first, and it was enormously helpful to me there, because programmatically configuring UIView objects can also be somewhat hard to parse. It may officially go by another name but I it called “Spec”, and the idea is that you create an object that holds a set of values and can then spit out different UI elements that share those values. So if I want all my UI to have a red fill color, for instance, I make a spec with a red fill color, and then from that spec I can generate buttons and labels and what-have-you, each with a single command, and they’ll all have that same red fill. It makes having a unified UI very simple:

Also again, fwiw, which may very well be nothing, I’d like to share a non-SODA UI approach I began but didn’t finish that I called “Simple Buttons”. It had a couple paradigms that I thought were interesting, and I’m sharing it in case there’s any chance you do too. One paradigm is that you can directly manipulate button placement, by touch, as a project runs, and just drag the buttons wherever you want, but also you can deactivate drag mode and use them as just regular buttons. It would help a lot for composing overall UI layouts. The other paradigm is that it tried to make button creation as Codea-like as possible, in other words, you make a button the same way you make an ellipse: it’s a single simple command that goes in your draw() method and that works out-of-the-box with a project’s current fill and stroke values:

When I saw how awesome SODA was I tried to integrate it with the Simple Button approach, but in the end there was too much re-inventing the wheel for me, so I threw in the towel and just embraced the SODA way.

I realize all of this is somewhat off-topic, because I think what you really want is to work with the UI conventions Craft is establishing, and contribute to that, and none of this will help you there. But I’m sharing it just in case any of it proves inspiring to you.

3

@UberGoober thank you, I will check these out. While I do hope to contribute to the new UI, I was also hoping others might do as you have, and provide some interesting ideas worth considering. I have tried SodaSpec, and was hoping it would be a solution for me, but the reality I found was that what I wanted/needed was to customize the look or interactivity, or make new widgets that weren’t available in SODA.

I do agree with the concept that an API should feel consistent with Codea’s drawing of shapes, and just as easily set up. I do think there are some potential efficiency issues with more complex interactive elements when everything is encapsulated within the draw function. That actually leads me to one major thing I have wanted to offer for consideration:

My friend and collaborator Taylor from Audulus 3 modular synthesis app uses a vector drawing library called NANOVG, which is extremely efficient and fast render. I’ve had patches in audulus with far more elements than the most complex overloading 2D game I’ve ever seen running 60fps no problem. I’m currently seeing if I can port that to work with Codea somehow, as well as looking at a public domain aa 2D vector library in shadertoy. I’ll post any useful info I find here.

4

@AxiomCrux I like your ambition!

I agree that SodaSpec is quite constrained in terms of customizability.

I wanted it for myself because quite often I would go to make a SODA button and be lost thinking “now how the heck do I make one of these again?” I wanted a single-function-no-parameters solution, so that’s its overriding goal.

I’m not going to say SODA is infinitely customizable, but I don’t think it should be colored with the same brush as SodaSpec. You can customize a ton of things if you can decipher the parameter system (which takes a lot of trial and error).

Let me put forward, if you will, a question that might give you pause before you discard the possibility of extending SODA. SODA can enable dozens of different built-in interactions, like tab bars and scrollable dialogues and little on/off widgets and etc–many of which have been tuned to the point where they mimic functionality people expect from the normal iOS environment. The question is, how long will it take you, for whatever system you have in mind, to be as fully-featured as that? And as beautiful?

This is what gave me pause when I considered making my own UI system. There’s so much good work there, I thought, why re-do it and compete with it when I could just try to improve the things that are bothering me.

For the heck of it, why don’t you list some of the things you think SODA can’t do, and we can talk about a) whether or not it actually can, because there’s a chance of that, and b ) how hard it would be to create a whole new UI system just to implement that feature (or group of features) vs. extending SODA to give it that functionality.

5

And a point of curiosity, since I know SODA was inspired by Cider, but I know nothing about Cider: how hard do you think it would be to make SODA mimic Cider in the ways where it currently falls short?

6

‘You can customize a ton of things if you can decipher the parameter system (which takes a lot of trial and error).’

I suppose this is the crux of my issue, I can see that it’s powerful capable and refined, but in the process of trying to parse the details of where I would go about modifying fundamental attributes of the functionality and styling, and how to even set up objects in the SODA api as it is,

I spent more time shifting through the details trying to learn it and it felt tedious every time. I would go and make my own widget from scratch that took maybe 20 minutes of FUN and INSPIRED enjoyable creative coding. I just think the fundamental way SODA was written was meant to be a “beautiful and functional ready to go approach” which is totally valid and great for many people, but lacks elegance and adaptability.

So what I have been doing is studying the elements of Cider (especially the interface builder, and their very simple and elegant class hierarchy) and Soda (great details and overall beautiful, but lost the ease of access and simplicity).

I believe the widget / GUI API should be as simple as the current CODEA parameter.type(“name”,min,max,init) setup, and allows extensibility and is customizable but easy to learn and remember. As clever as the whole SODA Init table idea is, in practice it is different enough from the standard Codea techniques that it feels out of place.

I find myself always reflecting on what I have made and re-factoring it toward the end of keeping my classes elegant and logical structure hierarchy to the best of my ability. Like the iPad itself, in practice it is a rectangle of glass you touch, The elegance and simplicity Keeps the focus on the experience provided within it,

ergonomic design is all about a feedback cycle of expectation, discovery, and intuition:
How do you expect it will respond to your action?
How closely did it match that expectation?

If there is a disparity between expectation and what happens, now we need to switch gears to a much more complex for process of learning, try something else, or completely shift to a hunt for answers as to how to achieve a desired result, look into manual or forum or internet and that kills the flow, so to avoid that giant recursive troubleshooting hunt routine we need to keep the structure of things simple and direct, and keep refining.

Intuition = preemptive unconscious pattern-recognition and the ability to understand something instinctively, without the need for conscious reasoning.

————-
My real goal is that I want a to create a UI design tool, as part of a large undertaking I am working toward. A combination of cross platform hardware and software tools, a modular multimedia Creative platform that has multitiple levels of abstraction and allows everyone from complete beginners to advanced gurus access to creating audio/visual interactive performances and installations (think max msp/ pure data meets Arduino and packaged in a way that provides easy access to low level elements for customizing hardware and software.

But the element in question is I want an open ended widget design tool for UI creation, and CODEA / GLSL is playing a role in rapid prototyping this structure.

Because this aspect of my goal overlaps with something I feel would be extremely useful to the CODEA community, I thought it pragmatic to offer my ideas into CODEA’s new UI library. I think they are on the right track from what I see in the new craft files, but it is very early on so it’s the perfect time for me to help if I am allowed. I will be available to put forth more direct effort toward this and provide my solutions on this thread in the next weeks, it is also part of the shadertoy / interactive shader format / glsl library I am working on. I have made a ton of useful stuff, I am just sorting the details before I cause confusion providing intermittent versions.
————

Soda doesn’t actually do that much in terms of built-in interactions it’s mainly the gesture library.

What’s soda doesn’t do, is make an easy to use framework for creating CODEA UI widgets to spec. I don’t want a cookie cutter slider or button, I want everyone to have access to levels of abstraction for their own GUI desires. The only two places I see it as a major room for improvement in Codea are providing gesture support and UI (and a more open ended low level solution for sound which I brought up in another thread)

7

@AxiomCrux You’ve certainly put a lot of thought into this topic. The UI library in the Craft examples is really just a prototype for what might be a more extensive system for Codea down the road.

I’m happy to hear out any suggestions for how it could be improved or changed. Personally I’m a fan of the way Unity’s UI system works in the sense that all objects are laid out by using anchors, pivots and stretching. This way you can easily make resolution independent layouts but without the added complication of a linear constraint system. I agree that a gesture system would also be a good idea (similar to how iOS handles it but simplified).

8

@John I will check out unity’s UI system.

Also worth checking out is JUCE, cross platform IDE primarily used in the audio plugin world. It has a fantastic GUI layout tool, might be similar to what you like in unity. It features a WYSIWYG layout with handles, and an attribute editor for stylizing.

MAX 7 (cycling74.com modular programming tool made with JUCE) is also great reference in this regard, drag and drop widgets and attribute editor.

Indeed I had gotten the impression the UI library in the craft examples is in an early state which is why I thought it might be an ideal time to post and see if there is a way for me to help advance it’s progress.

I’ve been experimenting with glsl for certain widgets, trying to find a way to benchmark and see if it is a more efficient solution for some things.

I’ll gather some screenshots and code here soon.

9

Oh also @John I noticed that the new craft update makes use of ARkit, and CODEA is iOS specific, maybe it is worth considering wrapping some of the iOS built in touch gesture library and UIKIT as options that keep consistency with iOS style and functionality?

10

@AxiomCrux wow, it is ambitious like nobody’s business to do all that, and Godspeed! I’m curious where you are thinking of starting.

For example, if I might sketch out a kind of format or something, if it’s a given that you want to mimic the “parameter.type(…)” structure from Codea, I’d venture that you’ll need to start with writing some code that accepts that kind of call and draws a visible result. I do hope you’ll forgive me for stating the obvious!

I only mention it because it seems like you’re in a research phase right now, and I’m curious to see you put some of these ideas in motion, so if writing that sort of thing is not a road you’ve gone down yet, are you interested in drafting something like it as a start?