Harmony.codea - a procedural drawing tool

I’ve a version too that works really fine for a single drawed line, so for example for the simple brush. that requires just to adjust the ending vertices of the previous segment to exactly fit the initial vertices of the new segment. the result is comparable to the one I’ve seen in your image, and works well also with my smooth implementation, The problem arise with complex brushes where the main line is drawn together to other segments, so I need a way to correctly handle each segment. tomorrow I’ll check a way.

Well, insomnia is a good friend of developers… At the moment the only solution I’ve found is to add the possibility to set an id when drawing a line, so to identify a single line made of several small segments. It seems to work pretty well with all the brushes (except the poor shaded brush that having no main line is unchanged…), at different thickness values and alpha values. I’ve applied the same solution also to not smoothed lines with nice results. Again, there’re conditions where the alpha draw fails, like with sharp corners (artefacts) or when the draw is too slow. Code is on GitHub

Gosh! I can see that I’m going to have to get up very early to get the better of you.

I’ve a version too that works really fine for a single drawed line, so for example for the simple brush. that requires just to adjust the ending vertices of the previous segment to exactly fit the initial vertices of the new segment.

At the moment the only solution I’ve found is to add the possibility to set an id when drawing a line,

That’s near-enough exactly what I do. I suspect the details differ but likely only in minor ways.

How do you handle the smoothing on lines? When the smooth flag is set then for line segments I only smooth the sides, whereas for whole lines (such as those introduced by the brush), I smooth all of the boundary.

My code’s now updated. I’ll download yours and update it on my iPad.

I’ve a few ideas for tweaking the brushes. How set-in-stone are the brushes? As in, if I want to tweak a definition a bit then should I make a new brush with a new name or can I edit the brush directly? Would users expect, say, the Chrome brush to behave in a very particular way, or would something close to it still be okay to be named Chrome?

Well, be clement with my code, I’ve worked on it half an hour tonight because I was unable to sleep, I’ve then cleaned it this morning (like removing the id used twice, uff saved by variable scope!).

If you look my solution it’s really simple. For each new segment I modify only the ending vertices of the previous segment. I know, the correct way should be to move also the starting vertices of the new quad to fit in the middle point, but having some practical test… I have not seen a significant difference. I’ve just updated github with a new meshcanvas where the line method is cleaned up and the 6 or 3 vertex adjust logic can be switched by commenting/uncommenting a section of code, if you want to give it a look.

I’m now putting your code on iPad in the while so I can check your work.

About the brushes… well, the names and the definitions are the one mrdoob decided when He implememented Harmony. Almost all the implementation I’ve seen of Harmony (there’re a lot of them, even as iOS app) use the same names with almost the same behaviour.

Consider that I’m not so sure myself about the implementation I’ve ported to codea because the original Harmony source code is no more on GitHub and I’m not so canfident that the js code I started from is now the original or an already slightly modified version :slight_smile:

Just one thing, trying your version it seems there’s some bug in different situation, related to the adjustment of the vertices:

1 sometimes it happens that some wrong vertices are generated when a new line begin and that can generate or a little adjustment in the ending of the previous line or worst a stretched triangle long triangle

2.similar thing happens when drawing a line slowly and frequently changing direction. I try to be more clear: start drawing very slowly from top to bottom, and in the meanwhile make like a zig zag, going left and then right, just a little. sometimes strange triangles are generated.

Moreover after a line is drawn often I can’t start drawing soon after, even if I move my finger nothing appears on the screen.

I’ve checked only with the simple brush

Hi, a suggestion: in your addQuad function (and in general in your code) you are used to address quad vertex in that way:

1 - 2
 |   |
 3 - 4

while in codea genric addRect/setRect functions, with an angle of 0 ( axis aligned) are so configured:

1 - 4
 | \\ |
 2 - 3

where 1 is vertex 1 and 4 and 3 is vertex 3 and 5.

I think it would be better to use the same convention of codea rects, that would make more easy to understand your code to all codea users, and moreover would allow to better mix codea add/set rects call with your addquad function.

Yes, but my way is better so Codea should adapt to it not the other way around.

I’m not being totally facetious here. Imagine drawing a cube. The natural way to label the vertices is by binary code with a three-digit binary number. Then the first bit corresponds to the x-axis, the second to the y-axis, and the third to the z-axis. It is then really easy to figure out which vertices lie on which face using modular arithmetic. With the “round the edges” scheme then the mathematics to get 1 and 4 matched, and 2 and 3 is complicated and it gets more complicated as you step up dimensions.

(If you look at the code for my Roller Coaster project in the examples, you’ll see an “addPlank” method there. I’m pretty sure I used the same scheme.)

  1. sometimes it happens that some wrong vertices are generated when a new line begin and that can generate or a little adjustment in the ending of the previous line or worst a stretched triangle long triangle

Fixed this. I needed to blank the saved previous segment when starting a new piece of the line and I had it for BrushEx but not for Brush (took me a while to understand the relationship between the brushes).

  1. similar thing happens when drawing a line slowly and frequently changing direction. I try to be more clear: start drawing very slowly from top to bottom, and in the meanwhile make like a zig zag, going left and then right, just a little. sometimes strange triangles are generated.

Aaagh. I suspect this is due to the normals to the lines ending up on the wrong side, or being of zero length, or some funny thing happening in the computation. I’ll have to dig into this to figure out what is causing it.

Well, I totally agree with you about the reasons for your implementation, and now I understand better why you avoid using codea add/ser rects facilities… i’d like to ear what @Simeon think about that!

Anyway considering that you’re writing a lib that works on codea, I still found pretty strange to see code that works with a different conventions, moreover making difficult to integrate faster codea api at least for rects, but if the pros are so many in other situations it could be the right choice.

Experimenting a little, I’m still not sure what’s causing the triangles but my current hypothesis is that it occurs when there’s a big angle shift in a very short space of time.

Here’s a wacky idea: instead of the path being a polygon through those points, imagine rather that the points are “where it aims for”. So it’s as if you’re dragging the end of the path through some viscous fluid with elastic connecting it to your finger. At any point, it feels a pull towards the last touch with a force somehow proportional to the distance away (so that it never gets too far away). That would smooth out the corners a little.

But probably that’s for another program.

In the meantime, it seems to be when the path suddenly changes direction. One option would be to put a cap on how far away that corner could be. If it’s beyond, say, twice the line thickness then it gets truncated.

Have you tried also my simplier solution? Doing some more test the 6 vertex adjustment version seems to be more stable (speaking about artefacts generation) than the 3 vertex adjustment one.
Even if less accurate, what do you think about the final result?

I fear that making a real good line implementation would require more than putting cap on segments (I also have already had some tries with bad results), the risk is to have holes or strange edges.

I fear that making a real good line implementation would require more than putting cap on segments (I also have already had some tries with bad results), the risk is to have holes or strange edges.

Yes, I agree. We could spend ages making a really good line mesh implementation but at the expense of having a fun program for people to play with.

Actually, I did think of one way to make it perfect - at least after the line had been finished: render it opaque to an image and then render the image to the canvas tinted with the right alpha. That would solve all the alpha issues.

With regard to the corners, I haven’t yet looked at yours but I will this evening. I had another idea which was that the problem is to do with when the line almost doubles back on itself, thus making the corner very sharp. So I could detect when this happens and add an extra point making the bend two not-so-sharp bends.

Ah, I forgot to mention one thing: the original Harmony implementation doesn’t allow to change thickness and alpha… :slight_smile: But at the end it seems that making a correct line mesh is now the most interesting part of the project…

Testing different solution I’ve found a very easy one that gives perfect lines but as drawn by a paintbrush. The result effect is very similar to the one I’ve seen in some paint software. The idea is to create each time a new quad where (I’m using codea convention) vertices 1,2 are vertices 4 and 3 of the previous one, and vertices 3 and 4 are calculated just as v1,2 displaced by the vector resulting by end - start.

pseudocode:

function line(s,e)
...
v1 = prev_v4
v2 = prev_v3
v3 = v2 + (e-s)
v4 = v1 + (e-s)
...
end

If you want to try it the result is really nice, even if not the one we’re looking for.

Make the line perfect in post processing could be a solution… but not a fair one for a painting software. Users would expect to see the line correctly drawn while drawing, not adjusting at the end!

I have a couple more of ideas I’d like to test before to give up, but I start thinking that only working on (in the worst case) large number of segments could give the results we would achieve.

That’s not a paintbrush, that’s a calligraphic pen. Imagine drawing a circle with that rule. It will draw two circles of the same radius with one displaced slightly relative to the other (depending on the starting point) and it will “fill” the region in between - exactly what it does will depend slightly on how slowly you draw the circle.

Or imagine that v2 = v1 + e2 (ie, v2 is vertically above v1) and you draw straight up. Your line will have no thickness.

I had a go at doing this a while ago - when Grant posted his project - and didn’t get very far (we’ve gotten further now than I did then). In the end, I went off on a tangent slightly. Rather than drawing arbitrary lines, I drew bezier cubics. I’ll dig out my code if you’re interested. The reason that was easier was that I had an actual parametrisation of the curve and so could triangulate it properly by “walking” along it placing triangles appropriately and ensuring that they didn’t overlap.

Making the line perfect in post processing isn’t so bad an idea. While it is being drawn then you draw it with one of the partial methods, then when it’s finished you do it properly. The partial implementations are good enough that the user can see what it will look like.

ooops, yes you’re right, it’s a calligraphic pen! If you can find your old code bezier bazed I would be interested to give it a look. It should be easy to try your idea hacking my meshcanvas class, adding a second image for a middle step of setContext after each stroke, only for strokes with alpha < 1. I could try in the morning if I have time and we can see the result.

Well, I had a try with your idea of post processing the line after the stroke is completed. Mmm it works fine for unwanted artefacts indeed, even if it’s strange to see the line adjusting at the end, but has a very bad side effect: if you voluntary draw a stroke that overlaps itself, like a ribbon, this method flat the alpha effect even in that case, and that’s bad.

Well, that’s the choice: no overlaps, or allow some unwanted overlaps. Actually, when drawing a path with something like SVG or PDF then this behaviour is what happens: the path’s opacity is entire, not on a pixel-by-pixel basis.

I also had a go at this and thought it worked well. Perhaps it could be an option. Another way to get the best-of-both worlds would be to allow a path to join to the previous one so that it renders to the canvas but doesn’t clear: a partial reset. I think I’ll have a go at that, it just means splitting the reset process in two.

But a perfect solution is going to be very hard to implement with all the possible edge cases. Imagine something like:


|\\ /
| X
|/ \\

where the thickness of the line is such that the triangle is just filled. Then the crossing, the X, should be overlaid but the triangle should not. That’s going to be very hard to get it right for.

I’ve updated my code, you also need to update the Library.

Changes of note:

  1. I’ve implemented the “save to temporary image” method for handling the alpha. To offset the negatives, …
  2. It is possible to join two path segments, either by a “move” or by a “line”. So long as you wait half a second between drawing them, they will be “baked” at different times and so the alphas won’t interact. (I hadn’t realised what the resetBrushStartStroke did before this: the “move” is effectively this being false).
  3. I’ve put in place a fix for double-backs in the path. If the angle is small, then there’s a modification to the routine to fix that triangle.
  4. My father had a go with this and managed to overload the touch handler so I’ve added a menu option to reset that (which is why you need the library update).

Have a play!

I have still not looked at the code and now I’m pretty tired, but tomorrow I’ll give it a look expecially for your fix for double-backs in the path.

But I’ve tried it and after that and after having read your post there’re some things not so clear to me:

  1. if I have understood right the half of a second before the line is postprocessed is a design choice, and the ‘line’ and ‘move’ options should allow to better handle the cons of this solution. Is it right? But the ‘move’ seems do nothing in particular, while the ‘line’ has a strange behaviour that i found pretty unconfortable because it adds lines as soon as i touch again the screen and I don’t understand how that could compensate the cons of the postprocessing lines solution. Moreover the postprocessing solution is already strange by itself, having half of a second waiting before it applies makes it more strange!

  2. Again about ‘move’: I’ve not understood your point about the resetBtushStartStroke param (I know, not a good name, but consider that first Harmony.codea version was developed in a couple of hour with 38 of fever… :)). Its purpose is to make each stroke indipendent from previous strokes for brushes that implement that neighbour points connection concept. You said that move = false is equivalent to resetBtushStartStroke = true, but again I don’t understand what that has to do with the posprocessing of the lines.

Now instead a couple of bugs:

  1. after you clean the canvas the first draw line is always drawn as opaque before to be post processed.

  2. when the ‘line’ option is set and you start to touch the screen here and there, it happens very often that after some touches (so after some extra lines are added) some of the just drawn lines disappear.

Ok, I’ve given a deeper look to your work. The line and move params seems to be exactly what i thouht.

‘move’ is equivalent to my resetBrushStartStroke, and I think it’s necessary for a good drawing experience but I still don’t catch the connection with the postprocessing solution.

‘line’ instead add a connection segment between the ending point of the last stroke and the start point of the new one. I also think I’ve figure out why you’ve implemented the line param, it should be to easily handle situation like a ‘ribbon’ where the desired effect is a continuos line where the alpha of the overlapping region is correctly blended. If I’ve rightly understood the purpose I think that even if it’s logically correct it would be hardly used in a real painting session because a little too tricky.

Moreover there’s a bug in the line behaviour implementation because it always generate an overlapping region where the previous stroke ends and the connection line begins. You can see it easily using the simple brush and trying to make a straight line by several taps (btw, I’ve realized only now that you have derived also the simple brush from the BrushEx class!)

About the disappearing lines bug, instead, I was wrong yesterday: that has nothing to do with the line param, the problem is the screen orientation. You allow any orientation while running the app so when you move from landscape to portrait WIDTH and HEIGHT are switched and you should handle that when you draw the mesh on your canvas image: the image in fact has the width and height of the starting orientation, so you should somehow handle the rotation.