Hello,
I was waiting for a process to finish, and I had this great idea for coding on the iPad: coding with glyphs. So I went ahead and made a quick (yet messy) mockup here:
https://dl.dropboxusercontent.com/u/91357336/Forum%20Misc/VisualCode.png
Basically, you draw your symbol in the draw area, and it adds it to your cursor. If it is a function that you’re calling, a little yellow box shows up.
Each symbol is classified under the following classes: Drawing (triangle), Lua stuff (square), custom variables (where the user can draw the picture) (circle), and static variables (like screen width and height).
I’d like to make a version of this, but there are a few issues. First of all, this is basically a version of Codea with glyphs, so I’m wondering if @Simeon would allow this on the App Store. Second of all, it’d be really hard to do the gesture recognition. I’m looking into native iOS APIs for doing this. Third of all, it’d need a lot of optimization. Forth of all, lots of parts of coding would be hard to implement visually (like variables of functions being called; you can see my current, crappy implementation in the image). To export the project, I’d pretty much say “Go buy Codea and follow these instructions to export it”.
I really thing this would actually be faster than writing with an onscreen keyboard (and less mistakes, if the gestures are done well).
Feedback would be great.
I’ll be posting more ideas as I refine it.
Thanks!
@Zoyt, It seems like an awesome Idea, but most handwriting applications carry much hate for inaccuracy, and coding with gestures would be hard to do unless it is EXTREMLY optimized. IMHO, I would stick to coding with words, but to each their own, and like I said, Awesome Idea,
@CodeaNoob - By handwriting, do you mean pictographs? Anyways, I one of the cool concepts for this is that it’s a bridge way into Codea. This won’t have the fancy stuff like shaders and meshes. I wouldn’t be using it because I have a BlueTooth keyboard, but most people don’t. But as I look into it, I think the gestures will be done all through Obj-C, so that shouldn’t be as slow.
@Zoyt, what i’m saying is it might take 2-3 tries for someone to get the program to recognize the symbol, and get frustrated at the amount of time, while writing it will take shorter
As an example from the aeons of programming history, check this out:
life?{?1 ??.?3 4=+/,¯1 0 1?.?¯1 0 1?.???}
This is Conways ‘Game of Life’ written in a language called APL. One line of terse code performs something that would take many lines of code in other languages. It uses a fairly complex syntax, symbols, glyphs etc. as part of its rich set of coding constructs.
But…
It’s criticised as being difficult to understand and follow, as well as harder to learn (unless you’re from Tau Ceti!).
This is the challenge I suspect when you start introducing obscure glyphs and symbols, it can make it harder not easier to learn without a lot of thought. 
@andymac3d - The symbols are meant to look like what they mean, not random ones… Haha. Did you ever learn APL? I’ve heard of it before, but I didn’t even bother to try and learn it.
@Prynok - What do you mean by “click and drag”? Are you talking about the cursor, or typing? There will still be a cursor, you’ll just be drawing symbols instead of typing. In my opinion, it should be faster because there will be less error - coding was made for physical keyboards where you can see the physical keys. I’m looking into alternatives that aren’t as crappy as things like GameSalad.
@Zoyt - I worked for IBM many years ago and some of my egg-head colleagues were APL nuts. Fortunately, I managed to avoid learning it - but their screens when they were coding always reminded me of a control panel of a Klingon spaceship. Although they always swore it was super powerful when you got your head around the syntax/symbols. hmmm :-/
Anyway, the point is that some sort of symbolic language will always be tricky to develop I guess - obvious looking symbols and glyphs to one person might be completely meaningless and random to others. As anyone who has tried to design icons for UI’s will testify - its easy if you use some sort of concrete visual metaphor (e.g paintbrush, wastebasket etc) far more tricky with abstract concepts such as programming constructs. This isn’t to say it wouldn’t work, it just would require a bit of thought.
@andymac3d - True. If you noticed in my sketch, I had a section called “suggestions”. So what the user does first is dray the type of action they want (ex: square for Lua functions), then the user can scroll through the suggestions and finish the glyph, or tap on the suggestion. That’ll probably be one of the hardest tasks.
@Zoyt, What I think @Prynok is saying is sort of like having a sidebar of symbols for different things in coding, and dragging them in an order, the only example I can think if is Cargo-Bot’s drag and program aspect
@CodeaNoob, @Prynok - Ahh. Got ya. Personally, I don’t think so. If I do follow through with this, I probably would give the user that option, though.
@SkyTheCoder - I remember that, but (if I’m not mistaken), it’s only for basic objects like squares and circles. That being said, I probably won’t be following through with this.
@SkyTheCoder - As I think about it, I could do it with that, but it’d be super complex to get a simple shape.
This reminds me of the drag and drop system used in Game Maker. That actually helped me a lot, and made the introduction to the complementary programming language, GML, much easier.
I played with and modified @West 's code to do something similar - it has a builder and a consumer - but the amount of time when using the app (consuming) was quite slow, and (surprisingly, but not) it was sensitive to direction - a circle drawn to the left was different than a circle drawn to the right.
It did reasonably well though, until I started to actually try to process the results - even as a developer taking my time, the inconsistencies of fingers and directions, coupled with complex shapes, would only yield hits in the 60-75% accurate range.
To be fair, the shapes I was defining were like diamond, star, curved lines, etc. Typically 10 or so points. Circle and square tend to get confused with each other if you use much less than that.