Having trouble coordinating between immediate mode drawing and Craft in the AR Face Demo

General
1

I have continued hacking the AR Face Demo. I am trying to draw a point cloud on the face based on walking the positions of the AR faceModel. I want it to track the face just like the textures in the demo do. Those textures stick just like the face grabbers in the Alien movies. :slight_smile:

I can get frustratingly close. The triangle mesh I use wraps around nicely. My problem is that I can’t seem to set up the camera correctly and my point cloud drifts all around yet stays “kinda” close. The rotation does look good. I don’t see how to make my viewing setup jive with what is happening down in the Craft used in the demo. I am hoping there is an api that lets me get the setup values I need.

I am certain I am just suffering the confusion that goes with being a new user. I am hoping someone in the community can guide me.

Below is the “hopefully” revealing subset of my code. Also attached is a picture of how close I can get when I position my iPad by hand just right.

function draw()
    scene:update(DeltaTime)
    scene:draw(WIDTH, HEIGHT)
    pushMatrix()
    setupCamera()
    drawFacePointCloud(mainEntity,triMesh)
    popMatrix()
    drawTarget(mainAnchor)

end

function setupCamera()
    local cz = 0.15    
    perspective()
    camera(0,0,cz,0,0,0,0,1,0)  
end

function drawFacePointCloud(face, mesh)
    if face == nil then return end
    local wp = face.worldPosition
    local wr = face.eulerAngles
    local positions = face.model.positions
    pushMatrix()
    translate(wp.x,wp.y,wp.z)
    rotate(wr.x,1,0,0)
    rotate(wr.y,0,1,0)
    rotate(wr.z,0,0,1)
    mesh:setColors(color(255, 255, 0, 128))
    for k,v in pairs(positions) do
        local p = v
        pushMatrix()
        translate(p.x,p.y,p.z)
        mesh:draw()
        popMatrix()
    end
    popMatrix()   
end
2

@DavidLeibs When you post code, put ~~~ on a line by itself before and after the code so it shows correctly. I added them to your code above.

3

@dave1707,
Thank you so much for making the code readable. I will now add the needed marks in the future.

4

@DavidLeibs I’m not exactly sure why it’s out of sync but it may have something to do with your camera settings and the way ARKit updates in a multithreaded manner.
That said you can also use craft to make a model that visualises each point using a quad and place that in the face transform hierarchy. I’ve made an example project that does this and it seems to work pretty well.

5

@John Thank you so much for pointing me in the right direction. Your solution is so beautifully direct it restored my childlike sense of wonder. :slight_smile: I had not realized that you could use position with an index and do surgery on the vertices inside a model. I had imagined having to do much worse and that drove me down the rathole of direct drawing and trying to deal with dueling cameras and a world of magic numbers.

I am still left with the question of just how I would I go about coordinating with the Craft camera after the scene:draw() has returned but since I am typically in control of that camera it isn’t really an issue that will keep me up at night.

Once again, thank you so much!