An interesting feature of craft: post effects

binaryblues · January 4, 2022, 10:01am

One of the interesting things that I found about craft.camera ——post effects.

It offers three post effects:

craft.bloomEffect()
craft.depthOfFieldEffect()
craft.selectionEffect()

the third one I don’t know how to use, and the other two are great, but there’s not a lot of discussion here about post effects, probably due to the lack of documentation, I only found two posts about the Craft Post Effect:

https://codea.io/talk/discussion/9849/is-it-possible-to-turn-on-the-bloo

https://codea.io/talk/discussion/12392/how-to-add-post-processing#latest

But it’s really interesting,

here’s the sample code:

-- CraftCameraTest

function setup()
    -- Create a new craft scene
    scene = craft.scene()
    scene.ambientColor = color(218, 158, 79)
    scene.sky.active = false
    
    -- Setup camera and lighting
    scene.sun.rotation = quat.eulerAngles(125, 125, 0)
    
    pb = craft.bloomEffect()
    pd = craft.depthOfFieldEffect()
    ps = craft.selectionEffect()

    parameter.boolean("enable",true, function() pb.enabled = enable end)
    parameter.integer("iterations",1,100,5, function() pb.iterations =iterations end)
    parameter.number("threshold",-1.0,3.0,2.0,function() pb.threshold = threshold end)
    parameter.number("softThreshold",-1.0,3.0,1.5,function() pb.softThreshold = softThreshold end)
    parameter.number("intensity", 0, 10, 0.06,function() pb.intensity = intensity end)

    -- Helper class for interactive camera  
    myViewer = scene.camera:add(OrbitViewer, vec3( 2.0,  0.5,  0.5), 8, 1, 400)
    myViewer.rx = 45
    myViewer.ry = -45
    
    c = scene.camera:get(craft.camera)
    c.hdr = true
    c.colorTextureEnabled = true
    c:addPostEffect(pb)
    c:addPostEffect(ps)
        
    myChest = Chest()    
end

function update(dt)
    -- Update the scene (physics, transforms etc)
    scene:update(dt)
    myChest:update()
end

-- Called automatically by codea 
function draw()
    update(DeltaTime)
    scene:draw()	
end

Chest = class()

function Chest:init()
    e = self.entity
    self.base = scene:entity()
    self.base.parent = e
    self.base.position = vec3(0.5, 0.3, 0.5)
    local r1 = self.base:add(craft.renderer, craft.model.cube(vec3(0.8,0.6,0.8)))
    r1.material = craft.material(asset.builtin.Materials.Specular)
    r1.material.diffuse = color(133, 79, 30)
    
    self.root = scene:entity()
    self.root.position = vec3(1.5, 0.3, 0.5)
    local r0 = self.root:add(craft.renderer, craft.model.cube(vec3(0.8,0.2,0.8),vec3(0.3,0.1,0.4)))
    
    self.top = scene:entity()
    self.top.parent = self.root
    self.top.position = vec3(0.1, 0.6, 0.1)
    local r2 = self.top:add(craft.renderer, craft.model.cube(vec3(0.8,0.2,0.8),vec3(0.4,0.1,0.4)))
    r2.material = craft.material(asset.builtin.Materials.Specular)
    r2.material.diffuse = color(66, 47, 30, 255)
    self.angle = 0
    self.y = 0.6
    
    self.ball = scene:entity()
    self.ball.parent = self.root
    self.ball.position = vec3(0, 0.3, 0.5)
    local r3 = self.ball:add(craft.renderer, craft.model.icosphere(0.2))
    r3.material = craft.material(asset.builtin.Materials.Specular)
    r3.material.diffuse = color(133, 124, 30)
    
    self.obj = scene:entity()
    self.obj.parent = self.root
    self.obj.position = vec3(1.5, 0.3, 0.5)
    self.obj.scale = vec3(0.5, 0.5, 0.5)
    local r4 = self.obj:add(craft.renderer, craft.model(asset.builtin.Primitives.RoundedCube))
    r4.material = craft.material(asset.builtin.Materials.Specular)
    r4.material.diffuse = color(30, 133, 101)
    
    self.floor = scene:entity()
    self.floor.parent = e
    self.floor.position = vec3(0.1, 0.6, 0.1)
    local r5 = self.floor:add(craft.renderer, craft.model.plane(vec2(1.2,1.2),vec3(0.6,0.0,0.6)))
    r5.material = craft.material(asset.builtin.Materials.Specular)
    r5.material.diffuse = color(46, 66, 30)
    
    touches.addHandler(self, -1, false)
end

function Chest:update()
    self.root.rotation = quat.eulerAngles(0,  0, self.angle)
    self.top.rotation = quat.eulerAngles(0,  0, self.angle*8)
    self.ball.rotation = quat.eulerAngles(0,  0, self.angle)
    self.obj.rotation = quat.eulerAngles(0,  0, self.angle*10)
    self.floor.rotation = quat.eulerAngles(0,  0, self.angle)
    
    scene.debug:line(vec3(0.1, 0.6, 0.1-1),vec3(0.1, 0.6, 0.1+1),color(47, 239, 5))
    scene.debug:line(vec3(1.5, 0.3, 0.5-1),vec3(1.5, 0.3, 0.5+1),color(239, 5, 5))
    
    local a = self.ball.worldPosition + vec3(0,0,-1)
    local b = self.ball.worldPosition + vec3(0,0,1)
    scene.debug:line(a, b,color(239, 5, 199))
    
    local a = self.top.worldPosition + vec3(0,0,-1)
    local b = self.top.worldPosition + vec3(0,0,1)
    scene.debug:line(a, b,color(16, 239, 5))
    
    local a = self.obj.worldPosition + vec3(0,0,-1)
    local b = self.obj.worldPosition + vec3(0,0,1)
    scene.debug:line(a, b,color(239, 233, 5))
end


function Chest:interact()
    if not self.open then
        self.open = true
        tween(1.6, self, {angle = 90, y=10}, {easing=tween.easing.backOut,loop = tween.loop.once })
    else
        self.open = false
        tween(1.6, self, {angle = 0, y=0.6}, tween.easing.cubicIn)
    end
end


function Chest:touched(touch)
    if touch.state == BEGAN then
        self:interact()
    end
end

Here is the demo videos:

https://youtu.be/ZX40cdd0kg8

https://youtu.be/rqdV8xcr2pw

Bri_G · January 4, 2022, 11:12am

@binaryblues - nice demo, could you explain what you mean by post effects. Is it a collection of light effects ? Or is it a physical effect like physics on a bouncing ball?

binaryblues · January 4, 2022, 12:00pm

@Bri_G Glad to try to explain it: the concept of post-effects is commonly used in movies, but ultimately it needs to be done on a computer. I call them post effects because in Codea Craft it’s called PostEffects:

craft.bloomEffect()
craft.depthOfFieldEffect()
craft.selectionEffect()
craft.camera:addPostEffect(effect)

And unity has a similar feature, Bloom is simply a kind of shimmering, it makes light shine, the effect of light overflows.
Common formats for images are JPG, PNG, etc. . These images are called LDR (low dynamic range images) . These images can only record RGB 0-255, corresponding to 0-1. In other words, RGB only in the 0-1 picture, called LDR.
Conversely, if the image can be less than 0 or more than 1, we call it HDR (high dynamic range image) . HDR/TIF/EXR is one of the best image formats for HDR.

Our real world is a colorful HDR world. Colors beyond LDR are not stored in the photo, so they are exposed to solid color.

Bloom is a must-have for 3D gaming, with a beam of light hitting the box at 100 degrees and showing up as pure white at best. It’s only when you open bloom that you realize how much light he’s spilling out. This is especially true for game effects. It’s all fire and electricity and, at worst, it’s a transparent glowing object, and if you take out bloom, you’ll see that it’s just moving tiles floating on top of a piece, and it’s not going to work very well. A proper Bloom adds a lot to the final effect.

Back to our Craft, using the post effect, All the models in the scene will give a brilliant picture effect.

Bri_G · January 4, 2022, 1:22pm

@binaryblues - thanks for the explanation. Thought it could be a graphic extension designed to introduce realism.

Also thanks for putting labels on LDR and HDR for me, I always thought that HDR ws just LDR exaggerated to by enhancing contrast.

binaryblues · January 4, 2022, 1:53pm

@Bri_G Not at all. I also learned a lot from your technical discussions.

Steppers · January 4, 2022, 2:57pm

I think they’re technically called ‘Post-Processing Effects’ as they’re applied after geometry is rendered and they only act on the rendered 2D image.

They tend to be special effects or shaders that are applied to a rendered image to enhance it in some way.

Bloom tends to identify bright pixels in the image and let them ‘bleed’ into surrounding pixels.

Other post-processing effects like blurs (depth of field), screen-space ambient occlusion, edge detect, etc are applied similarly.

binaryblues · January 4, 2022, 3:21pm

@Steppers Thanks for your very good complementary explanation!

Btw. Craft’s depth of field is very good, the only downside is it’s gonna be slow.

Steppers · January 4, 2022, 3:24pm

@binaryblues I wasn’t trying to detract from your comment btw
All good info!

binaryblues · January 4, 2022, 3:32pm

@Steppers I know what you mean, don’t worry, it’s normal that people don’t always see eye-to-eye, and I feel that with my $10-a-year translation software, I can understand what you’re saying with basic accuracy

piinthesky · January 4, 2022, 3:50pm

@binaryblues the depth of field is interesting, shame it is so slow. What is the selectionEffect i didn’t notice what difference it makes! Do selectionEffect and depthOfField have any parameters to play with?

binaryblues · January 4, 2022, 4:02pm

@piinthesky Well, the depth of field effect is like taking a picture with a camera with a large aperture. It looks fun, but it’s too slow.
Depth of field is only a Boolean parameter: enabled, the selection effect has no parameters, I do not know what the selection effect will be, like you, perhaps only developers know these details.
I found a doc repo of @John on GitHub in a previous post and discovered the post effects feature based on the content.

https://codea.io/talk/discussion/12633/complete-craft-reference#latest

https://github.com/JohnTM/Codea-Docs

The LuaBinding code:

LuaBinding(module)
        .beginClass<PostEffect>("postEffect")
        .endClass()
        
        .beginExtendClass<BloomPostEffect, PostEffect>("bloomEffect")
        .addConstructor(LUA_SP(BloomPostEffectPtr), LUA_ARGS())
        .addProperty("enabled", &BloomPostEffect::isEnabled, &BloomPostEffect::setEnabled)
        .addProperty("iterations", &BloomPostEffect::getIterations, &BloomPostEffect::setIterations)
        .addProperty("threshold", &BloomPostEffect::getThreshold, &BloomPostEffect::setThreshold)
        .addProperty("softThreshold", &BloomPostEffect::getSoftThreshold, &BloomPostEffect::setSoftThreshold)
        .addProperty("intensity", &BloomPostEffect::getIntensity, &BloomPostEffect::setIntensity)
        .endClass()

        .beginExtendClass<DepthOfFieldPostEffect, PostEffect>("depthOfFieldEffect")
        .addConstructor(LUA_SP(DepthOfFieldPtr), LUA_ARGS())
        .addProperty("enabled", &DepthOfFieldPostEffect::isEnabled, &DepthOfFieldPostEffect::setEnabled)
        .endClass()
        
        .beginExtendClass<SelectionPostEffect, PostEffect>("selectionEffect")
        .addConstructor(LUA_SP(SelectionPostEffectPtr), LUA_ARGS())
        .endClass();

The last one is SelectionPostEffect, it only has a Constructor without any property.

John · January 6, 2022, 4:32am

Hey guys

I’ll add some documentation for the bloom post processing effect. The depth of field effect was more of an experiment and is very inefficient (uses a single pass circle of confusion based blur). The selection effect is used by Shade for drawing selection outlines and picking and isn’t currently supported for other uses

binaryblues · January 6, 2022, 6:01am

@John Thank you very much for your comments, and I hope you will be able to add some sample code in future documentation so that more people will try out these features, which took a while to implement after all, and the visual effects are good, if the code has been hidden in the unknown is too bad.