Skeletal Animation Shader

This is the new one I’ve been playing with the last few days. You have a mesh for your object and then pin it to a skeleton which is just a bunch of transformation matrices. Then the vertex shader will move your mesh to match the way the skeleton is moved.

Movements and mesh are currently rubbish, my next steps are to make it so you can pull in richer meshes, and also make bone weights work right for joints, but I thought I’d post it for those interested.

Code here:

At the moment the shader isn’t efficient mainly because I can’t get it to feed a table of matrices to a “uniform mat4 bones[15]” on the shader…

This may not be quite best practice for this apparently common thing, since even though it’s common, I couldn’t find good code to steal of the interwebs so built it myself…

Very, very, very cool. Curious though. It opens up a whole new realm of 3D rigging! Good job!

The key is that it should give good performance even on a complex mesh, because it’s only the collection of Matrices for the bones that have to be passed to the shader each frame as the vertices are static.

There’s a lot more work to be done, at the moment I hard code single bone binding for the meshes just to get it working, for vertices around a joint you want to bind it with a weighting to 2 bones etc. I have plans for this, but it will take me a few more days to get that stuff together.


This looks absolutely brilliant, does it slow down at all? I definitely want to start making 3D since these shaders are coming out, makes it look great

So I’ve been playing with this and trying to do 1) automatic allocation of vertices to bones and 2) binding vertices to 2 bones for better joints. All I’ve been making is a mess.

So the current solution is fine for single bone binding, and manual allocation of vertices, but I have the hunch I’m doing it in a non-standard way.

Instead my next step is to look at how you can build a mesh in Blender and bring it into Codea (which I think @xavier on the forum has done previously) and then presumably in Blender you can build a skeleton system, so how to import that too. And if I can figure that all out, then I should be able to build the right kind of shader to go with it.

Performance wise, this should scale well to meshes with larger number of vertices, because the scale of calculation is based on the number of bones more than anything else.

Played a bit more… I actually got 2 bones and weightings working, I had to modify my bone structure a bit more, but it worked… don’t ask me to explain how the maths works, but it seems to…

I think my solution is now closer to a normal way of doing things, and I’ve removed an step I had to “skeletonise” the mesh, which felt a little hinky.

I think I’ll still look into the blender stuff, but rather than groking the blender skeleton stuff, I may build a tool to construct rigging in Codea for an arbitrary mesh. I’ll post it all if I get it working :wink:

Gist and video in first post updated. It now has more correct bone handling in the shader etc, and I set up a rigging tween walk scenario.

This is exciting stuff!!! That’s one step closer to making Codea a full-blown, immersive, 3D game engine.

Wait, you can use Blender models in Codea?

Yeah, haven’t done it myself as yet, but check out this thread, I think it’s a custom python script to turn the mesh into an image, and then Codea code to pull that in.