3D flying in Codea [UPDATE 2]

Andrew Stacey (mostly) and I have spent the last couple of months trying to master 3D flying in Codea, and we’d like to share everything we’ve learned.

The difficulty is knowing how much knowledge to assume to start with, and what people might be interested in. Here’s what I’m thinking.

Assumed knowledge (that won’t need to be explained) - meshes and basic 3D graphics (ie what perspective does, what the camera settings are for, etc).

Knowledge we can share

  • how to use “quaternions” to do trouble free rotations in 3D (including a library with examples)
  • how to control a plane/spacecraft in 3D, using a joystick or iPad tilt (including a Flight library)
  • how to add features like different camera views, camera tracking options, radar screens, shooting
  • how to import 3D models from other graphics packages like Blender
  • how to wrap a sky image all the way around the screen for added realism

I’m tossing up the best way to present all this stuff. Perhaps an ebook is best.

It will take many hours of work and won’t be easy, so first I’d like to ask for genuine expressions of interest, just to be sure there are people who will want to use it. So please let me know below if you really think you would try it out.

The video below (posted previously) doesn’t use our final software, but gives you a good idea of what is possible. It uses two imported 3D models.
https://www.youtube.com/watch?v=V7cCZ9tZeg8

@Ignatz That is super awesome! I would really like this in ebook form. Your ebooks have been excellent. They got me going on shaders and I’d love to work through one on 3D. I’ve brushed up on perspective and using camera. I’d like to see this presented to the forum. A lot of good could come out of a project like this.

I bumped this because I put it in the wrong discussion category at first so nobody saw it :stuck_out_tongue:

@Ignatz i would be very interested in reading your ebook and grb the code. Understanding How to use the quaternions in a practical way is omething i would really like to do.

Url to the code? Sounds really useful!

I’m interested in quaternions, and specially how the calculations are done, and also the sky.

Thanks for the comments. Seeing as how you need to be pretty experienced to do this stuff, and you just want the code anyway, perhaps the best approach is a good set of demos, rather than an ebook that will take ages to write.

Let me know what you think of this project so far. It’s a “step by step” project, with a parameter that lets you choose which demo you want to see (push the output area down to see it at the bottom of the other parameters). If you get crazy results when flipping between demos, just restart, it should go back to your latest choice.

https://gist.github.com/dermotbalson/8780502

(Note it needs one image downloaded for the sky - see the Notes tab)

It still needs a demo of the special flight functionality that took Andrew and I a while to figure out, and also how to import 3D models, and more, but it shows basic rotation functionality, camera tracking and sky, using quaternions.

And I think you’ll agree the code is pretty small and not scary. But whatever you do, don’t look in the Quaternion tab. There be dragons in there.

I’ll include additional demos for the other stuff once I know I’m on the right track.

@Ignatz - I’m very interested in your and Andrews Quaternion lib - it’s really the ‘proper’ grown up way to do this sort of thing and a worthy addition to the 3d stuff demoed so far. Thank you.

I think a collection of demos would be more than adequate considering the audience may well be fairly small. Will you write some notes on your blog as well?

Once again - great work! :wink:

@ignatz thank you for sharing this gist. It is certainly an excellent asset to start 3d control.

Thanks Ignatz, found it really useful!

I’ve updated the code to this

https://gist.github.com/dermotbalson/8817757

which includes more demos on the use of the flying library and 3D models, and I’ve posted about it here

http://coolcodea.wordpress.com/2014/02/05/149-how-the-3d-flight-library-works/

There’s more to come, but this is most of the flying stuff…

One comment on the blog post:

Unfortunately, this is not easy when you’re working with quaternions, because it means finding out what roll angle you’re at, so you can reverse it. You can extract the roll angle from a quaternion, but it could suffer from gimbal lock.

It has absolutely nothing to do with the fact that we’re using quaternions to represent rotations, and even less to do with gimbal lock. It’s because we mix pitch and roll into a single rotation and once they’re mixed, it’s impossible to separate them cleanly. This would happen whatever the internal implementation.

@Andrew_Stacey - I agree that when you mix angles, you can’t separate them cleanly, but that wasn’t the point I was making. I was actually talking about gimbal lock.

The point is that if you rely on extracting angles from quaternions back into Euler values, you may run into gimbal lock (because Euler angles have discontinuities). That’s not a fault of the quaternions, but of Euler space itself.

What I meant by what I wrote, is that it’s a bad idea to work in Euler space at all. You can’t use quaternions to avoid gimbal lock and then somehow extract Euler angles that are also free from it. So you need to find another solution, as we did.

My reading of the phrase was that the problem is with quaternions, not with Euler angles: “this is not easy when you’re working with quaternions”. I agree that the problem is with Euler angles, but I’d say that the issue is more that the problem is not clearly defined: what does it mean to “cancel the roll” on a rotation? This implies that there is a unique property of a rotation called its “roll” that can be neatly removed.

This is not true, but it’s nothing to do with gimbal lock. It’s to do with the properties of the map from R^3 (space of three parameters) to SO(3) (space of rotations) that sends a triple of angles to the rotation that they represent. And the problem with this is not that this map isn’t invertible (it isn’t, but that can be gotten around) nor that it doesn’t have an important technical property to do with directions[1], but that when you convert a rotation back to Euler angles then the obvious path that cancels the roll might not be the right path to take.

This is exactly what we’re seeing with that “swoop”: we’ve split the rotation into two pieces: yaw and pitch-and-roll, but sometimes pitch-and-roll turns out to be yaw. The right solution is to figure out what the correct target should be in rotation space and then move to that. I’ve actually had an idea about that …

[1] I mention this “technical property with directions” merely to point out that gimbal lock has little to do with discontinuities and lots to do with directions. It is the fact that the directions get stuck that causes all the problems. There are issues with discontinuities with reversing the wrap-around map R → S^1 (the circle), but no issues with directions and so this doesn’t suffer from “gimbal lock”.

@Andrew_Stacey - I clarified the wording a bit, I hope.

Anyway, I think it’s pretty clear that 3D rotations are a lot trickier than they first appear!

If you want to cancel roll on your rotation, you could convert the quaternion to axis-angle and then just set the angle component to 0.

@Causeless, actually, that won’t work, for the reason set out in my post (link above).

I’ve updated the demos to include a 3D model, using tilt to fly, and adding a second plane and radar screen.

http://goo.gl/mnUEMb

Don’t forget that the Quaternion tab can do a lot more, but for that, you may need to consult Andrew_Stacey.

If you want to know more about things like skydomes, importing 3D models, or adding bullet effects, I’ve blogged about all of them on http://coolcodea.wordpress.com.

I’m probably going to leave it there for a while. This whole business has taken months and i want to take a break from trying to do any more mental rotations!

You didn’t explain why you couldn’t convert to axis-angle. You only talked about euler angles, which do suffer from gimbal lock, however axis-angle does not.

The only real downside with axis-angle is that it doesn’t perfectly interpolate, however you are only trying to interpolate the roll component which’ll interpolate fine.

@Causeless - I’m not an expert on this, so you may well be right. However, I have access to a number of books on building 3D engines and on mathmatics for graphics, and without exception, they use quaternions and don’t cover axis-angle at all. I realise that this doesn’t necessarily prove much!

If you’d like to write some code that implements axis-angle, I’ll be happy to include reference to that, but I’m not a mathematician, and since @Andrew_Stacey is, I was happy to take his advice (and code). You should really have this discussion with him.