In Codea 4, mesh can load GLTF animation, here is a demo

[binaryblues]

I found that the mesh increase in Codea 4 supports a lot of file types, so I tried gltf. However, in this routine there is only a simple load and no animation, because I know very little about gltf. hopefully @John will provide some sample code, such as to get the roles moving.

These gltf are from the app: Anima Toon, it is very easy to control the character`s actions.

The code is very short, However, I suggest you download the project file, because it includes the GLTF file.

function setup()

    -- 使用基本场景的默认镜头
    scn = scene.default3d()
    scn.sky = nil
    -- scn.camera.farPlane =5000
    
    cam = scn.camera:get(camera)
    cam.farPlane = 50000
    print("cam: ", cam.farPlane)
    
    -- create a sun entity and add a directional light to it
    sunEntity = scn:entity("sun")
    sun = sunEntity:add(light.directional(vec3(-1,-1,-1)))
    sun.color = color(233, 150, 80)
    -- sun.clearColor = color(233, 150, 80)
    sun.intensity = 0.05   
    -- sun.direction =vec3(-100,-100,-100)
    sun.castShadows = false
    
    rig = scn.camera:add(camera.rigs.orbit)
    rig.distance = 50
    rig.angles.x = 25    
    rig.angles.y = 75
    
    -- 用于地面：新建 material.lit() 会自动打开 阴影，如何关闭? 
    checkers = material.lit()
    checkers.map = image.read(asset.builtin.Blocks.Brick_Grey)
    checkers.color = color(243, 243, 242)
    checkers.scaleOffset = vec4(0.125, 0.125, 0.0, 0.0)
    checkers.roughness = 0.5
    
    -- 地面
    ground = scn:entity("ground")
    ground:add(mesh.box(50, 0.1, 50))
    ground.y = -0.05
    ground.material = checkers

        
    robot = scn:entity("robot")    
    robot:add(mesh.read(asset.builtin.Blocky_Characters.Robot))
    robot.scale = vec3(.5,.5,.5)
    robot.position = vec3(10,0,-10)
    robot.material = checkers
    
    -- load Run.gltf (1 action)
    myChar = scn:entity("myChar")    
    myChar:add(mesh.read(asset.run))
    -- myChar.material = checkers
    myChar.position = vec3(0,0.5,0)    
    
    for k,v in pairs(myChar.mesh.animations) do 
        print(k,v)
    end
    
    -- load DogWalk.gltf (1 action)
    myPet = scn:entity("myPet")    
    myPet:add(mesh.read(asset.DogWalk))
    -- myPet.material = checkers
    myPet.position = vec3(0,0.5,-10)
    
    -- load DogWalk.gltf (3 actions)
    myPet1 = scn:entity("myPet1")    
    myPet1:add(mesh.read(asset.ExportDog))
    -- myPet.material = checkers
    myPet1.position = vec3(-5,0.6,-12)
    
    scene.main = scn
end

-- This function gets called once every frame
function draw()
    -- This sets a dark background color 
    background(40, 40, 50)
    rig.angles.y = time.elapsed*30
end

C4AnimationGltf.zip (404.7 KB)

The demo video:

Codea 4 scene

The original animation:

The walking man

A dog include 3 actions

[binaryblues]

In Build 349 the new feature “performance stats display” is very easy to use.

The frame rate is very low after loading gltf, and it seems that the bottleneck of processing is still on the CPU side.

[John]

Looking at your stats, 330 draw calls seems a little high for the number of models on screen. If you post the example project I can try to see what’s causing the performance issues

[binaryblues]

Thank you @John , Here is the project:

https://talk.codea.io/uploads/short-url/8BkgJgmYqrDtPivq69xgXJiKL1L.zip

I’m also confused because I’m only showing a static model, not making it move

[John]

I think that the mesh rendering needs optimisation but I also think those models are horribly optimised. Each dog has 58 submeshes, which are also skinned and also need to be rendered several extra times for shadows mapping. They could be one submesh each. It will be a good benchmark to stress test this kind of stuff though.

[binaryblues]

These gltf animations are all exported from this app: Anima Toon, which is a voxel animation editor. The export menu does not provide optimization options and may be exported according to the default configuration, I’ll see if I can optimize the models in blender.

[John]

Looking at the models in Blender, there’s an enormous number of empty objects (2844) which are slowing down mesh rendering by simply existing. I wonder what they are for?

[John]

I ran some additional post processing options in the mesh loader (removing redundant nodes, etc…) as well as some other fixes (cache coherency, memory alignment, mesh draw code fixes) and got your example running at 120fps with shadows

The robot texture seems to be missing so there’s still some bugs to be fixed but it’s a lot better than 5fps!

[John]

Also got the animation working:

In setup(): anim = myChar.mesh.animations[1]
In draw(): anim:apply(myChar, time.elapsed % anim.duration)

[binaryblues]

@John I didn’t expect Anima Toon to export a model like this. It looks like there’s a lot of potential for performance optimization when exporting models. I’m going to submit your findings to the developers and hope they can fix them.

You can do 120 frames! Yes! The Codea 4 character animation features in the performance is no problem, will give us a better 3D game development with Codea 4 experience! And it looks like this animation class encapsulates very well, use is also super simple and clear! Thank you again for your hard work!

[SugarRay]

@John, for Codea 4 have you (or are you) taking a look at the other model loaders (e.g. .obj, .fbx., .blend) and tuning them up, too? I had shared with you a while back a complex 3D model that Codea was having trouble rendering completely accurately and recently tried loading that model into the latest Codea 3.x App Store release and still had some trouble rendering the model accurately (e.g. .obj loader left out 1 key wall, .fbx loader didn’t show any textures, and .blend loader gave me an error and wouldn’t load). Yet, I can load and render the same 3D model accurately using standard desktop .obj, .fbx, and .blend loaders. The iOS “Sketch 3D” app is also able to load and render that 3D model example accurately (which I thought was based on the same assimp library that Codea was using). If all of these loaders are being updated and improved and, additionally can load .gltf models, I’ll be very excited to try the Codea 4.0 release when it comes out.

Thank you and have a happy upcoming holiday season.

Cc: @binaryblues

[John]

Codea should be able to load GLTF models now. Missing walls and textures might be due to backface culling and file path issues respectively.

I’ve noticed that GLTF is a bit more reliable. I am working on a new asset loading options parameter that will allow model importing to run various assimp options that may fix some issues or allow for extra settings, such as backface drawing, optimising mesh drawing and other stuff.

The best way to fix it is to have a set of simple test models that use various features in each supported format with a list of a bugs (like unit tests for models) and we can try fixing them one at a time

[SugarRay]

Thanks for the reply, @John. When you mentioned that Codea should be able to load GLTF, you mean the current release (3.7)? I have a version of the same model in GLTF format and Codea does load it but still misses the some of the walls, and, in addition, doesn’t apply any of the textures. So, to summarize, I have a complex 3D scene that I imported into Blender and then exported to:

.obj
.fbx
.gltf
.blend

Windows default 3D viewer renders the .obj, .fbx, .gltf formats of the 3D scene all correctly rendered without any missing walls. Blender loads and renders the .blend file correctly without any missing walls.

With Codea 3.7:

.obj file is loaded and rendered but with the missing wall I mentioned
.fbx file is loaded and rendered but with the missing wall and textures aren’t applied
.gltf file is loaded and rendered but with the missing wall and textures aren’t applied
.blend file is not loaded (gives error “expect userdata, got nil”)

Would it be helpful to upload links to that model in all of the different formats for testing or were you wanting to test a simpler model in the different formats first?

[John]

I think if you could provide a minimal example of a model that loads incorrectly. So something simple with hidden parts (again, this is most likely due to backface culling, so the model parts exist but aren’t drawn on both sides). The texture issue is probably the one that needs the to be looked at more closely. I suspect that model loading programs have a lot of special code to handle models that may actually be broken in some way but still draw them as properly

[SugarRay]

Thanks, @John, will do. To briefly clarify:

1. Will the model I described do or were you wanted a simpler model that doesn’t load correctly? If the latter, I’ll look around-- otherwise, I have the model ready to send.

2. Did you want the model or a Codea project incorporating the model that doesn’t load correctly or both?

3. Do you want the model in the different storage formats (e.g. .obj.,.fbx, .gltf) or just one type of format for now?

[John]

A minimal project that loads any model with those issues in one format is enough to start with, gltf is fine

[SugarRay]

Okay, @John. Here is a simple project with the 3D model (Egypt.gltf) that loads but doesn’t render one wall and doesn’t apply any textures:

CraftEgypt.zip

(I left a Cobra.obj object in the project as a comparison; since the project usually uses an .obj Egypt model the proportions are off but I think you’ll get the idea).

I’ve also attached a picture ow what the Egypt model is supposed to look like.

Fyi.

[John]

I looked into the Craft rendering issues with this model. There are some two-sided material flags that are in the GLTF model format. For the textures, they are actually embedded into the model in GLTF format, which isn’t handled by Craft but is handled by Codea 4. I can add the two-sided option to both runtimes, although the embedded textures are a bit more tricky to backport

[John]

@SugarRay

I got double-sided rendering working in Craft and Codea 4. The textures also load in Codea 4 so the result on your model looks like this:

Not exactly sure why it looks like gold, but there is some progress

[SugarRay]

Looks great, @John, thanks! You fixed both the missing wall problem and the missing textures.

So, Codea 4 and perhaps the next Codea release will allow embedded textures in GLTF? That would be great!

Would you like me to send the .fbx version of the model to see if you can get the .fbx importer to also render the additional wall and textures (and/or the .obj version to see why missing wall in that format) at your convenience?