Meshes: Rectangle draw order

I’m trying to simplify my code by drawing as many of the game objects as possible on single meshes. I have an update() function which I call to update the location of the rectangles making up a mesh, with various setRect commands.

I had assumed that the draw order of the rectangles would follow the order in which I called the setRect functions in my update() (with later setRect calls bringing that rectangle to the front). However, it doesn’t seem to work like that. As far as I can tell, the rectangles which are drawn in front are the ones which were added to the mesh most recently, is that right? If so, is there any way to override this behaviour? My alternative is to make sure I add ALL the rectangles at the start, in the order I want, even if they aren’t going to be used for ages and / or may not be used. Or to add another mesh. But I’m trying to avoid that.

Does the draw order of the objects need to change from frame to frame? i.e. are you using a not-quite-overhead forced perspective (like in the Small World asset pack), where draw order depends on y position? Or do you want the draw order for a given object to stay fixed? If it’s the latter case, what I’ve done before with this kind of texture atlas approach is to have 3 or 4 meshes representing the different layers of the game, eg mesh 1 and 2 for layers of background, mesh 2 for player, enemies and important gameplay elements, mesh 4 for anything that needs to pass in front of the player. You’re right that rects are drawn in the order they’re added, not set.

The order doesn’t need to change. There’s only about 25 rectangles involved, so it’s not that difficult for me just to add them all in one go, in the right order, but make sure they only become visible when needed.

I was mainly wondering if there was something I could do around z-coords to push certain ones to the front, but it looks like normal rectangles are simply 2d objects.

use translate to set z

translate(0,0,-2)
--draw back layer of rectangles
translate(0,0,1)  --move 1 pixel closer
--draw next layer of rectangles
translate(0,0,1) --and so on

@Ignatz sorry, just to be clear I’m talking about multiple rects on a single mesh, not the Codea rectangle function. That method only works where you’ve got multiple draw calls correct? Whereas I will have a single mesh draw.

Yes, you can’t manipulate the z of a rect vis-a-vis the other rects in the same mesh (except via some form of shader sorcery!). I think the texture atlas approach is most useful for something like a large tiled backdrop, or an animated sprite sheet, or anything where there’s a lot going on onscreen, such as a busy shoot-em-up, or something with lots of particle effects, where you really want to cut down your draw calls as much as possible.

Actually, you can set the z value of rects, but you gave to do it after each setRect, as setRect resets the z value to zero. Note that if you are using z ordering to sort the draw order, you have to do your own alpha channel clipping. Here’s an example. It’s probably not worth the effort. If the draw order for each object is fixed (ie not a forced perspective scenario), then I would go with one mesh for each plane of action.

-- setRectZ

function setup()
    print("touch the screen to toggle the z coord of the blue-tinted rect \
\
nb you'd have to do your own alpha channel clipping.")
    m = mesh()
    local img = readImage("Small World:Court")
    m.texture = img
    w, h = img.width, img.height
    p = {a=vec2(WIDTH*0.3, HEIGHT*0.5), b=vec2(WIDTH*0.6, HEIGHT*0.5)} --point a and point b
    m:addRect(p.a.x, p.a.y, w, h)
    local col = color(84, 174, 231, 255)
    m:setRectColor(1, col) --tint rect 1 blue so we can tell them apart
    m:addRect(p.b.x, p.b.y, w, h) 
    tween(2, p, {a=p.b}, {easing=tween.easing.cubicInOut, loop=tween.loop.pingpong}) --point a tweens to point b and back
    rectZ = 0
    parameter.watch("rectZ") --the z value we will toggle
end

function draw()
    background(40, 40, 50)
    
    m:setRect(1,p.a.x, p.a.y, w, h) --setRect resets z to 0
    setRectZ(m, 1, rectZ) --have to set z after setRect call
    m:draw()
    
end

function touched(t)
    if t.state==BEGAN then
        rectZ = 1 - rectZ --toggle z between 1 and 0
    end
end

function setRectZ(m, rec, z) --mesh, rect number, z value
    local off = (rec-1)*6 --convert rect number to vertex number
    for i=1,6 do --6 points in rect
        local v = m:vertex(off+i) --grab position
        m:vertex(off+i, v.x, v.y, z) --set z
    end
end

Ok, cos I have lots of procrastination to do, here’s a setRectZ function that uses the normal to set the z of the rect (and does alpha discarding). That way, when you call setRect, it doesn’t matter about the z being reset. You can just call the setRectZ function once, whenever you want the draw order to change.

-- setRectZ
--uses normal to set rectangle z, to stop setRect resetting z to zero
function setup()
    print("touch the screen to toggle the z coord of the blue-tinted rect ")
    m = mesh()
    m.shader = shader(RectZShader.vert, RectZShader.frag)
    local img = readImage("Small World:Court")
    m.texture = img
    w, h = img.width, img.height
    p = {a=vec2(WIDTH*0.3, HEIGHT*0.5), b=vec2(WIDTH*0.6, HEIGHT*0.5)} --point a and point b
    m:addRect(p.a.x, p.a.y, w, h)
    local col = color(84, 174, 231, 255)
    m:setRectColor(1, col) --tint rect 1 blue so we can tell them apart
    m:addRect(p.b.x, p.b.y, w, h) 
    tween(2, p, {a=p.b}, {easing=tween.easing.cubicInOut, loop=tween.loop.pingpong}) --point a tweens to point b and back
    rectZ = 0
    parameter.watch("rectZ") --the z value we will toggle
end

function draw()
    ortho(0, WIDTH, 0, HEIGHT,-2000,2000) --set view frustum to required depth
    background(40, 40, 50)   
    m:setRect(1,p.a.x, p.a.y, w, h) 
    m:draw() 
end

function touched(t)
    if t.state==BEGAN then
        rectZ = 2 - rectZ --toggle z between 2 and 0
        setRectZ(m, 1, rectZ)
    end
end

function setRectZ(m, rec, z) --mesh, rect number, z value
    local off = (rec-1)*6 --convert rect number to vertex number
    for i=1,6 do --6 points in rect
        m:normal(off+i, 0, 0, z) --set z using normal
    end
end

RectZShader={
vert=[[
uniform mat4 modelViewProjection;

attribute vec3 position;
attribute vec3 normal;
attribute vec4 color;
attribute vec2 texCoord;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    vColor = color;
    vTexCoord = texCoord;
    //z position taken fron normal
    gl_Position = modelViewProjection * vec4(position.xy, normal.z, 1.);
}
]],
frag=[[
precision highp float;

uniform lowp sampler2D texture;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;
    if (col.a<0.3) discard; //discard for alpha clipping
    else gl_FragColor = col;
}
]]
}

If you were doing forced perspective, you could also adapt the above shader so that the y coordinate also becomes the z coordinate. You might have to tweak the nearest/ furthest z parameters of the orthogonal command though.

@yojimbo2000 wow, that’s impressive stuff! While there are literally infinite ways to get code wrong, it’s good to know that there’s at least dozens of ways to fix any problem you might come across along the way :slight_smile:

@yojimbo2000 I’ve been experimenting with your shader. first of all: massive props. I really ought to learn how to write shaders. One question though: I don’t understand the alpha discarding, why is that necessary? I’ve got as far as working out how to remove alpha discarding from your shader, and see the mess it creates, I’m just wondering why.

I have in mind an isometric game where characters might go behind obstacles (so flexible z ordering crucial) but nonetheless I would ideally like the obstacle to become translucent in those circumstances, because accuracy of movement is crucial for the player.

the shader book here may help you, they aren’t as hard as they look

https://www.dropbox.com/sh/mr2yzp07vffskxt/AACqVnmzpAKOkNDWENPmN4psa

For the sort of game you are considering, the simplest is to have layered obstacles, eg all the obstacles on the front layer have z=0, the next later has z=5, and so on. Then you can easily position your player behind, or in between obstacles.

For translucence, maybe it’s easier to draw the player twice, once normally, behind the obstacle, so only part shows, then again, partly transparent, so you see the faint outline of the hidden part of the player against the obstacle.

For an isometric game, I’d use a full 3d projection (I recommend making it Z-up for easy conversion from an XY coordinate space) with an orthogonal projection. Eg:

http://codea.io/talk/discussion/6285/monument-valley-style-orthogonal-projection-3d

Re the need for discarding, alpha channels assume blending, and for a blend to work the underlying image must already have been drawn. So what you’re seeing is it’s blending through to the backdrop, instead of through to what we want the underlying image to be, creating an empty box around the image.

That makes sense now. Full 3d projection is a step up from what I was thinking, but @Ignatz 's suggestion is a really good one. It would be trivial for all my game objects to have a simultaneous ghost layer on top which only becomes apparent when the object is behind an obstacle. Thanks for the ideas, both. One final question. Using your z shader @yojimbo2000 there seems to be a constraint on z layers from -10 to 10, is that right? I assume that’s a limit of the system. It should be enough for my purposes, just checking there isn’t a way to access finer control

@epicurus101 yes, you need to set the view frustum to whatever depth you need, the last two variables are the near and far z:

ortho(0, WIDTH, 0, HEIGHT,-2000,2000) --orthogonal projection view frustum

Add it to the very start of the draw routine. I’ll edit the code above to add it in.

If the obstacle translucence is really important, then maybe you could experiment with reordering all of the objects each frame according to the y position with a table.sort (instead of manipulating the rectangles z using the rectZ shader). I’ve never tried this method, so I’m not sure how fast calling table.sort each frame would be, it would depend on how many obstacles there are.

If you have a large, scrolling game world (ie there are many more game objects in existence than are visible on screen an any point), then instead of table.sort, you could maybe have separate update and draw methods for each game object, and a draw table which is emptied each frame, into which you insert only those objects that are on screen, inserting them according to y position, and then just draw that each frame.

So in the update method for each object you’d do something like this (edit. Actually the insert according to y code needs a little work. Maybe table.sort of draw objects would be quicker):

if not outOfBounds(-self.w,-self.h,WIDTH+self.w, HEIGHT+self.h) then --check whether object is visible on screen
    drawObjects[#drawObjects+1] = self
    --[[ if #drawObjects>0 and self.y>drawObjects[1].y then --if y is higher than other objects
        table.insert(drawObjects, 1, self) --draw this object first
    else
       table.insert(drawObjects, self) --add to end of table
    end ]]--
end

and you main loop would be (edit, added table sort):

drawObjects={}
for i,v in ipairs(gameObjects) do
   v:update()
end
table.sort(drawObjects, function(a ,b) a.y>b.y end) 
for i,v in ipairs(drawObjects) do
    v:draw()
end

the above is untested, and probably needs some work

Edit 2. Just realised that this thread is about Rect draw order, and all of the above only applies to mesh draw order. What can I say. I’m having one of those mornings =P~

Ok, hopefully this comment will be more useful than the one above.

This uses the setRectZ shader from above to automatically set the z of each point to equal the y coordinate of the baseline of each rect, so as the rect moves up the screen, it automatically goes behind other objects it passes. As long as you’re ok with discard rather than translucency, this would work well for an isometric/forced perspective game:

If you have lots of objects moving around every frame, experiment with modifying setRectZ so that you store a normal table for the entire mesh, and set the entire normal table just before you draw the mesh. Could be quicker than setting a whole bunch of individual normals.

-- setRectZ
--uses normal to set rectangle z, to stop setRect resetting z to zero
function setup()
    m = mesh()
    m.shader = shader(RectZShader.vert, RectZShader.frag)
    local img = readImage("Small World:Court")
    m.texture = img
    w, h = img.width, img.height
    p = {a=vec2(WIDTH*0.5, HEIGHT*0.3), b=vec2(WIDTH*0.5, HEIGHT*0.6)} --point a and point b
    m:addRect(p.a.x, p.a.y, w, h)
    local col = color(84, 174, 231, 255)
    m:setRectColor(1, col) --tint rect 1 blue so we can tell them apart
    m:addRect(p.b.x, p.b.y, w, h) 
    tween(2, p, {a=vec2(WIDTH*0.5, HEIGHT*0.9)}, {easing=tween.easing.cubicInOut, loop=tween.loop.pingpong}) --point a tweens to point b and back
    rectZ = 0
    setRectZ(m,2,-p.b.y-h*0.5) --set z of rect 2 to inverse y of baseline
end

function draw()
    ortho(0, WIDTH, 0, HEIGHT,-2000,2000) --orthogonal projection
    background(40, 40, 50)   
    m:setRect(1,p.a.x, p.a.y, w, h) 
    setRectZ(m,1,-p.a.y-h*0.5) --set z to inverse y of baseline
    m:draw() 
end

function setRectZ(m, rec, z) --mesh, rect number, z value
    local off = (rec-1)*6 --convert rect number to vertex number
    for i=1,6 do --6 points in rect
        m:normal(off+i, 0, 0, z) --set z using normal
    end
end

RectZShader={
vert=[[
uniform mat4 modelViewProjection;

attribute vec3 position;
attribute vec3 normal;
attribute vec4 color;
attribute vec2 texCoord;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    vColor = color;
    vTexCoord = texCoord;
    //z position taken fron normal
    gl_Position = modelViewProjection * vec4(position.xy, normal.z, 1.);
}
]],
frag=[[
precision highp float;

uniform lowp sampler2D texture;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;
    if (col.a<0.3) discard; //discard for alpha clipping
    else gl_FragColor = col;
}
]]
}

Here’s a version that sets the entire normal table:


--# Main
-- setRectZ
--uses normal to set rectangle z, to stop setRect resetting z to zero
supportedOrientations(LANDSCAPE_ANY)
displayMode(OVERLAY)
function setup()
    m = mesh()
    m.shader = shader(RectZShader.vert, RectZShader.frag)
    local img = readImage("Small World:Court")
    m.texture = img
    w, h = img.width, img.height
    objects={}
    local u = WIDTH * 0.09
    for i=1,10 do
        objects[i]=Object(vec2(u*i, HEIGHT*0.1))
    end
    normals = {}
    for i=1,#m.vertices do
        normals[i]=vec3(0,0,1) --dummy variable, make sure normals table is right length
    end
end

function draw()
    ortho(0, WIDTH, 0, HEIGHT,-2000,2000) --orthogonal projection
    background(40, 40, 50)   
    for i,v in ipairs(objects) do
        v:update()
    end
    m.normals = normals --set entire normals table
    m:draw() 
end

function setRectZ(m, rec, z) --mesh, rect number, z value
    local off = (rec-1)*6 --convert rect number to vertex number
    for i=1,6 do --6 points in rect
      --  m:normal(off+i, 0, 0, z) --set z using normal
        normals[off+i].z = z --could be faster if setting z of every rect on mesh each frame
    end
end

RectZShader={
vert=[[
uniform mat4 modelViewProjection;

attribute vec3 position;
attribute vec3 normal;
attribute vec4 color;
attribute vec2 texCoord;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    vColor = color;
    vTexCoord = texCoord;
    //z position taken fron normal
    gl_Position = modelViewProjection * vec4(position.xy, normal.z, 1.);
}
]],
frag=[[
precision highp float;

uniform lowp sampler2D texture;

varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;
    if (col.a<0.3) discard; //discard for alpha clipping
    else gl_FragColor = col;
}
]]
}

--# Object
Object = class()

function Object:init(pos)
    self.pos = pos
    self.rect = m:addRect(pos.x, pos.y, w, h)
    m:setRectColor(self.rect, math.random(255), math.random(255), math.random(255))
    tween.delay(self.rect*0.2, function() tween(2, self, {pos=vec2(pos.x, HEIGHT*0.9)}, {easing=tween.easing.sineInOut, loop=tween.loop.pingpong}) end)
end

function Object:update()
    m:setRect(self.rect,self.pos.x, self.pos.y, w, h) 
    setRectZ(m,self.rect,-self.pos.y-h*0.5) --set z to inverse y of baseline
end