If you have a mess of objects on a 3D scene, some in front of others, how do you know which object a user touched?
I understand the math can only get you so far, because touching the screen is like touching a window - it gives you a direction (from your eye to the touch point) but doesn’t tell you how far away the object is, beyond the window. So you have to test which object(s) intersect with that direction vector, and pick the nearest.
I’ve found a way that I think is simpler and which will give pixel perfect accuracy. This is how it works.
Your touchable objects need to be in their own, separate meshes (this is a good idea anyway, if they have transparent edges, because you need to sort them by distance from the camera), with a stored id number 1-255.
You draw them normally until you have a touch on the screen, and then you store the x,y position of the touch. On the next draw, you don’t draw to the screen, but to a hidden image, and (using a shader) any mesh with an id number is drawn with a single colour, being (i,0,0) where i is the id number - and transparent pixels are discarded. Objects without id numbers are drawn normally. When this is done, draw tests the pixel colour of the hidden image at the touch point, and it it is of the form (i,0,0), it extracts i, the id of the object that was touched. Then it goes back to drawing normally again.
Here is a demo with a number of numbered circles, which, when touched, pop up a message. There is also a library image which doesn’t have an id and doesn’t react to touches (except to say “you missed”).
What I like about this is that this should always work, there is only a minor impact on FPS because all this extra work only happens once, just after a touch. Also, the shader discards transparent pixels around each object, so you can touch objects which are behind others, very accurately (if you were using math instead, you would have to somehow check whether any object that intersects with the touch direction was touched in its transparent area, or not).
The code is here: https://gist.github.com/dermotbalson/5898835
It’s a bit messy at the moment, and a lot of it is just to create and draw all the objects. But if it’s useful, I’ll tidy it into something that’s easy to use.