Detecting Touch in 3D Issue

Hi,

I’ve been using Ignatz’s excellent guides to help me get a basic understanding of 3D graphics, and have made some small progress on my project. One issue among the many that currently perplex me is detecting which object is “under” the touch. I’ve adapted the code from https://coolcodea.wordpress.com/2013/05/30/70a-touching-an-object-on-a-3d-screen/ and https://coolcodea.wordpress.com/2013/05/19/55-3d-creating-4-connected-walls/ in the code below.

The issue I get is that the foremost rectangle in my mesh isn’t always the one returned by the check. It works sometimes, but not others. If I record the video of this code in use, I can see the transparent walls of the cube that the check misses. I suspect I’m making a fundamental mistake, any help would be appreciated in getting me past this problem.

Thanks,

Time Trial.

-- 3D Test

-- Use this function to perform your initial setup
function setup()
    parameter.integer('cameraY',-1000,1000,0)
    parameter.number('Rotation',-2*math.pi,2*math.pi,0)
    tapped=false
    tapX,tapY=0,0
    r,g,b=0,0,0
    tapImg=image(WIDTH,HEIGHT)
    meshTable={}
    AddFourWalls(0,0,0)
end

-- This function gets called once every frame
function draw()
    -- This sets a dark background color
    if tapped==true then
        setContext(tapImg)
        clip(tapX-1,tapY-1,2,2)
    end
    background(40, 40, 50)
    perspective(45,WIDTH/HEIGHT)
    cameraX=math.sin(Rotation)*400
    cameraZ=(math.cos(Rotation)*400)
    camera(cameraX,cameraY,cameraZ,0,0,0,0,1,0)
    for i,m in pairs(meshTable) do
        pushMatrix()
        translate(m.pos.x,m.pos.y,-m.pos.z)
        rotate(-m.ang,0,1,0)
        m:draw()
        popMatrix()
    end
    if tapped==true then
        r,g,b=tapImg:get(tapX,tapY)
        print('Image ' .. r/20 .. ' tapped (r=' .. r .. ')')
        setContext()
        clip()
        tapped=false
    end
end
function AddFourWalls(x,y,z)
    local w,h=100,100
    local xx,yy,zz=x-w/2,y-h/2,z-w/2
    AddImage(x,y,zz,w,h,CreateImage(w,h,1),0)
    AddImage(xx,y,z,w,h,CreateImage(w,h,2),90)
    AddImage(x,y,zz+w,w,h,CreateImage(w,h,3),0)
    AddImage(xx+w,y,z,w,h,CreateImage(w,h,4),90) 
end
function CreateImage(pw,ph,px)
    local ci=image(pw,ph)
    setContext(ci)
    rectMode(CORNER)
    fill(px*20,0,0)
    rect(-1,-1,pw+2,ph+2)
    fill(255)
    fontSize(14)
    text(px,10,10)
    setContext()
    return ci
end
function AddImage(x,y,z,w,h,i,a)
    local m=mesh()
    m.texture=i
    local v,t={},{}
    local xx,yy,zz=-w/2,-h/2,0
    v[#v+1]=vec3(xx,yy,zz)  t[#t+1]=vec2(0,0)
    v[#v+1]=vec3(xx+w,yy,zz)  t[#t+1]=vec2(1,0)
    v[#v+1]=vec3(xx+w,yy+h,zz)  t[#t+1]=vec2(1,1)
    v[#v+1]=vec3(xx+w,yy+h,zz)  t[#t+1]=vec2(1,1)
    v[#v+1]=vec3(xx,yy+h,zz)  t[#t+1]=vec2(0,1)
    v[#v+1]=vec3(xx,yy,zz)  t[#t+1]=vec2(0,0)
    m:setColors(color(255))
    m.vertices=v
    m.texCoords=t 
    m.pos=vec3(x,y,z) --store x,y,z
    m.ang=a  --store angle
    table.insert(meshTable,m)
end
function touched(touch)
    if touch.tapCount == 2 and touch.state == ENDED then
        tapped=true
        tapX=CurrentTouch.x
        tapY=CurrentTouch.y
    else
        tapped=false
    end
end

Your code is making my head explode! I haven’t done anything with 3D is ages. But, if the tap is just shooting a ray down the Z-axis to see what it gets, it will only work if the the Z-axis is not skewed in the human view. You always have to adjust for the skewedness. (Not sure if this is making any sense, but I hope it helps. Or, I may just be babbling…)

The idea behind my approach is that in many cases it is impossibly complex to mathematically determine which object has been touched, especially if some of them have transparent pixels so you can touch one object through the one in front.

So what I do when a touch happens, is to use the next draw opportunity to make a special internal image of the screen, and to colour each object with a unique code, eg a number from 1 to 255 in the red pixel. Transparent pixels are discarded. Then I look up the pixel that was touched, in this internal image.

This achieves what I want, very accurately.

But if you want transparent objects to register a touch, you need to disable the shader code that discards transparent pixels, so that those pixels will be given the special colour code as well.

Hope this helps.

@Ignatz that’s a very clever approach.

@Ignatz whatever was left of my brain just leaked out… Wow, what a solution.

I remember my first 3D game was one I wrote for the TRS-80. It generated a maze and then gave you a 3D view to find your way out. You could leave markers in locations, too! Unfortunately, it ate all of my 16K RAM and there was no room left for the rest of Colossal Cave.

Thanks for the replies. @Ignatz, I’ll try your suggestion. I didn’t think there were any transparent pixels, though!

@syntonica, @Igntatz’s guides linked in the original post explain the concepts very well. His real world examples (especially the ‘moving the paper’ analogy for translate and rotate) really gave me my first understanding at all of 3D.

@syntonica first person i see here who started like me! Trs80 too. Lunar lander was my first game. 4 characters rocket, plus 1 for the thrust… Good old times.

Trs80! Now your talking! Death Maze 5000! Pixels the size of a house! The golden age of computing… Sigh :slight_smile:

@andymac3d golden age? Not quite sure… Remember that tape recorder? Iiihhihiiiihhhiiiiihhiiiii… We could actually listen the binary code, and the most gifted of us could mentally read the code from the sound… No, just kidding! :wink:

@Jmv38 Did you used to make music by putting a radio next to your computer and run different loops to make notes?

@Jmv38 I can recognize words from binary… I just need a few seconds to remember the order of the alphabet XD

Edit: 010000110100100001000101010001010101001101000101 I wrote that using my mind, and it successfully translates to CHEESE. http://www.roubaixinteractive.com/PlayGround/Binary_Conversion/Binary_To_Text.asp

I’m special, my mom says :3

Edit 2: (And 010000110110100001100101011001010111001101100101 for Cheese, proper capitalization)

@SkyTheCoder be careful They might be reading your posts… THEY will come and get you if THEY known your super powers! then THEY will keep you locked in a secret base and force you to work for THEM and you wont see your mom ever again! :wink:

@Jmv38 They? Them? :stuck_out_tongue:

Hush, THEY will hear us! :stuck_out_tongue:

@SkyTheCoder The NSA :smiley:

Thank god i’m Canadian!

(no offence to Americans)

How does that work if you have foreground and background objects? Which gets selected?

I went a different way with this: I put a cursor on the screen in 3D And used it as a mouse of sorts. When it’s 3D position was close to another existing 3D object, that object gets selected.

This solved the z-axis thing - sorta. As I moved in x-y, I made my up-down (y) axis emulate moving back in the z axis too. So I moved in 3D space for object / mesh selection.

@Ignatz If I understand it, look at how the 3D cursor works in Blender. This position is set by left-clicking. @aciolino Is this correct?

the quake1 engine has been open-sourced for a while. you might look there for how they determine if planes are in front of other planes from the view point. They use BSP trees to describe the level, tho.