custom shader for craft models (changed from previous title)

Codea Craft
1

I have an image that i use in my app as texture for a craft sphere. It draws circles at specific places around the surface of the sphere. At the moment to change the colour of the sphere i set the diffuse property to the desired colour.

In the future i would like to be able to set the colour of the background of the image separately from the rest of the image, i.e. the background colour would vary from sphere to sphere, while the colour of the circles do not change. This i cannot do by setting the diffuse property as it is global to all of the texture.

I have about 2000 spheres onscreen so the implementation needs to be efficient. I think probably the best way to do this would be to implement the texture as a Shade shader with an input for the background colour. I started trying to do this within Shade, but i confess i am lost and i do not know how to proceed. Could someone @John, @Simeon please give me some hints on the best way to do it? I am familiar with glsl shaders and did implemented it as a glsl shader before adopting craft.

Below is the codea function i use to generate the image/texture:

function makeDOMImage(backCol)
--    print(backCol)
    local i,j 
    local img=image(255,255)
    
    local mushroom_size = 0.23
    local pmt_size      = 0.05
    local ring_size     = 0.07
    
    local twelveth  = 0.083333
    local sixth     = 0.1666667
        
    local p2=pmt_size*pmt_size
    local r2=ring_size*ring_size
    
        for i=1,img.width do
        for j=1,img.height do
            
            local col=backCol
            
            local phi_prime = i/img.width                
            local theta     = j/img.height 
            local up        = 0.0
            
            if theta<0.5 then --force theta > 0.5
                phi_prime=phi_prime+twelveth
                theta=1.0-theta
                up=100.0        
            end
                    
            phi_prime=math.fmod(phi_prime, sixth)
            if phi_prime>twelveth then phi_prime=sixth-phi_prime end
                    
            local a =math.sin(theta*3.1415)
            a = a*a*4.0
                    
            local A = a*phi_prime*phi_prime
            local tminusp=twelveth-phi_prime
            local B = a*tminusp*tminusp
                    
            --the theta's of the dom positions are: 
            -- 0.980875, 1.2706, 1.872738, 2.162463, 2.579597, 3.1415923073180982
                    
            local omt=1.0-theta
            local d1=omt*omt
                    
            local tm8=theta-0.82111139
            d1 =math.min(d1, up+A+tm8*tm8)  
                    
            local tm7=theta-0.687549 
            local d1=math.min (d1, B+tm7*tm7)
                    
            local tm6= theta-0.59587   
            local d1 =math.min(d1, A+tm6*tm6)   
                    
            if (d1<r2) then  col = vec4(0.5, 0.5, 0.5, 1.0)*255 end  --ring
                
            if (d1<p2) then col = vec4(0.9, 0.3, 0.3, 1.0)*255 end   --pmt
                    
            if (j/img.height<mushroom_size) then col = vec4(0.5, 0.5, 0.5, 1.0)*255 end --mushroom

            thiscol=color(col.r,col.g,col.b,col.a)
            
            img:set(i, j, thiscol)  
        end
    end
    return img
end
2

i wonder if this is a candidate for instancing

3

@RonJeffries Do you have a simple example of instancing.

4

sorry, no. simeon posted a not-very simple one.

5

@RonJeffries Look what I found among my projects. I totally forgot about this and just stumbled upon it. It draws 40 instances of mesh rects moving across the screen. I don’t remember how this is supposed to work.

viewer.mode=FULLSCREEN

function setup()
    m = mesh()
    m:addRect(0,0,10,10)
    m:addRect(50,50,10,10)
    m:setColors(color(255,0,0,100))
    m.shader = shader(vert, frag)
    numInstances = 40
    val=0
end

function draw()
    background(40, 40, 50)
    val=val+1
    m.shader.val=val
    m:draw(numInstances)
end

vert=[[
    #extension GL_EXT_draw_instanced: enable     
    uniform mat4 modelViewProjection;
    attribute vec4 position;
    attribute vec4 color;    
    varying lowp vec4 vColor;
    uniform highp float val;
    void main()
    {   vColor = color;
        float xOffset = val;
        float yOffset = float (gl_InstanceIDEXT) * 12.+100.;
        vec4 offset = vec4(xOffset, yOffset, 0, 0);
        gl_Position = modelViewProjection * (position+offset);
    }
    ]]

frag=[[    
    varying lowp vec4 vColor;
    void main()
    {   gl_FragColor = vColor;
    }
    ]]
6

@RonJeffries Here’s another one I found that I posted years ago. It’s 185,000 mesh rects. It looks like it’s 500 instances of 370 mesh rects.

PS. Run in portrait mode.

supportedOrientations(PORTRAIT_ANY)
displayMode(FULLSCREEN)

function setup()
    print(370*500)
    m = mesh()
    for z=1,370 do
        m:addRect(z*2,0,1,1)
    end
    m:setColors(color(255,255,0))
    m.shader = shader(vert, frag)
    numInstances = 500
    xVal=10
end

function draw()
    background(40, 40, 50)
    m.shader.xVal=xVal
    m:draw(numInstances)
end

vert=[[
    #extension GL_EXT_draw_instanced: enable     
    uniform mat4 modelViewProjection;
    attribute vec4 position;
    attribute vec4 color;    
    varying lowp vec4 vColor;
    uniform highp float xVal;
    void main()
    {   vColor = color;
        float xOffset = xVal;
        float yOffset = float (gl_InstanceIDEXT) * 2.+10.;
        vec4 offset = vec4(xOffset, yOffset, 0, 0);
        gl_Position = modelViewProjection * (position+offset);
    }
    ]]

frag=[[    
    varying lowp vec4 vColor;
    void main()
    {   gl_FragColor = vColor;
    }
    ]]
7

@RonJeffries Heres 54,000 mesh rects moving across the screen.

viewer.mode=FULLSCREEN

function setup()
    print(100*540)
    m = mesh()
    for z=1,100 do
        m:addRect(z*2,0,1,1)
    end
    m:setColors(color(255,255,0))
    m.shader = shader(vert, frag)
    numInstances = 540
    xVal=10
end

function draw()
    background(40, 40, 50)
    xVal=xVal+1
    m.shader.xVal=xVal
    m:draw(numInstances)
end

vert=[[
    #extension GL_EXT_draw_instanced: enable     
    uniform mat4 modelViewProjection;
    attribute vec4 position;
    attribute vec4 color;    
    varying lowp vec4 vColor;
    uniform highp float xVal;
    void main()
    {   vColor = color;
        float xOffset = xVal;
        float yOffset = float (gl_InstanceIDEXT) * 2.+10.;
        vec4 offset = vec4(xOffset, yOffset, 0, 0);
        gl_Position = modelViewProjection * (position+offset);
    }
    ]]

frag=[[    
    varying lowp vec4 vColor;
    void main()
    {   gl_FragColor = vColor;
    }
    ]]
8

thanks guys, but i am heavily invested in the craft environment and would like that the lighting of the sphere reacts to the craft lighting.

I had an idea…i thought i might be able to use the vertex colors to set the background color on the sphere and then overlay a texture to define the colors of the circles on the sphere, with the texture being transparent everywhere except for the circles. Here i am hoping that in the transparent regions the vertex color would show through-do you think that should work?

I tried to set the vertex colors of a craft icosphere, but the visualised sphere still stays white-any ideas why? Does craft display vertex colors?

viewer.mode=OVERLAY

function setup()
        
    scene = craft.scene()    
    scene.sky.material.sky=color(40, 35, 244)
    scene.sky.material.horizon=color(15, 199, 250)

    assert(OrbitViewer, "Please include Cameras (not Camera) as a dependency")  
    viewer = scene.camera:add(OrbitViewer, vec3(0), 25, 0, 2000)
    
    dom1=scene:entity()
    dom1.position=vec3(0,0,0)
    dom1.scale=vec3(1,1,1)*10

    dom1.model = craft.model.icosphere(1)
    dom1.material = craft.material(asset.builtin.Materials.Standard)  
    local vCount=dom1.model.vertexCount
    local iCount=dom1.model.indexCount
    print("icount",iCount)
    print("vcount",vCount)
    local vcolor=color(233, 80, 174)
    for i =1,vCount do
        print("before ",dom1.model.colors[i] )
        dom1.model:color(i,vcolor)
        print("after ", dom1.model.colors[i] )
    end
end

function update(dt)
    scene:update(dt)
end

function draw()
 --   background(33, 38, 245)  
    update(DeltaTime)
    scene:draw() 
end
9

fantastic stuff Dave, thanks!

10

Arrrghhh, as i use the same model for every sphere the ‘background’ vertex color will be the same for every sphere. So the idea does not work for my case!

While testing this i did manage to change the vertex colors and superimpose upon it a transparent texture- so it would work if one was using multiple models. Note that i was only able to ‘see’ the vertex color when using a ‘specular’ material, with the ‘standard’ material the vertex colors do not appear. This is why the code at the begining of this thread did not work. @John is that intended?

11

@piinthesky I have no clue what you’re trying to do, but it sounds like you have a bunch of spheres on another sphere and you’re trying to control what’s on each of the spheres. If that’s wrong, can you explain more what you’re trying to do.

Here’s an example where I have 5,112 spheres placed on a larger sphere. Pressing the Fast parameter will change all the spheres to blue fast. If the Slow parameter is pressed, then the spheres will change to blue at about 60 per second. I guess textures can be changed on each sphere also.

PS. Try changing to this “pt.model = craft.model.icosphere(size,5)” in createSphere1and do the slow change. This creates 20,472 spheres.

viewer.mode=STANDARD

function setup()
    cnt=0
    assert(OrbitViewer, "Please include Cameras (not Camera) as a dependency")
    tab={}
    tab2={}
    scene = craft.scene()
    skyMaterial=scene.sky.material
    skyMaterial.sky=color(255)
    skyMaterial.horizon=color(255)
    scene.sun.rotation=quat.eulerAngles(0,0,0)
    v=scene.camera:add(OrbitViewer, vec3(0,0,0), 100, 0, 300)
    createSphere1(vec3(0,0,0),20)
    print("spheres  "..#pt.model.positions)
    for a,b in pairs(pt.model.positions) do
        found=false
        for z=1,#tab do
            if b.x==tab[z].x and b.y==tab[z].y and b.z==tab[z].z then
                found=true
                break
            end
        end
        if not found then        
            createSphere2(b.x,b.y,b.z)
            table.insert(tab,{x=b.x,y=b.y,z=b.z})
        end
    end
    parameter.action("Fast",fast)
    parameter.action("Slow",slow)
end

function fast()
    changeFast=true
end

function slow()
    changeSlow=true
end

function createSphere2(x,y,z)
    local s=scene:entity()
    s.position=vec3(x,y,z)
    s.model = craft.model.icosphere(.5,1)
    s.material = craft.material(asset.builtin.Materials.Specular)
    s.material.diffuse=color(255,255,0)
    table.insert(tab2,s)
end

function createSphere1(p,size)
    pt=scene:entity()
    pt.position=vec3(p.x,p.y,p.z)
    pt.model = craft.model.icosphere(size,4)
    pt.material = craft.material(asset.builtin.Materials.Standard)
    pt.material.diffuse=color(255,0,0)
end

function draw()
    update(DeltaTime)
    scene:draw()
    collectgarbage()
    if changeFast then
        for z=1,#tab2 do
            tab2[z].material.diffuse=color(0,0,255)
        end

    end
    if changeSlow then
        if cnt<#tab2 then
            cnt=cnt+1
            tab2[cnt].material.diffuse=color(0,0,255)
        end
    end
    end

function update(dt)
    scene:update(dt)
end
12

@dave1707 thanks alot, indeed that could be a way to do it, albeit expensive in the number of entities.

13

@piinthesky Even though there’s 5,112 spheres reported, only 2,563 are being used because of the duplicates. I eliminate them in setup with the table tab. The 5,112 are the number of vertices and vertices get used multiple times to create the icosphere. If I didn’t remove the duplicates, then there would be multiple spheres at each position which would cause problems when manipulating them. The above code runs at 60 FPS on my iPad, so things aren’t being slowed down. As for the number of entities, apparently it’s not a problem.

14

So i finally managed to do what i wanted…
I have a texture for a craft sphere in which i wanted to replace the background color while keeping the rest of the texture (the texture adds circles to a sphere).
This is done by a craft shader which replaces the transparent pixels with the desired background color.

In general, the ability to use an inline code shader with craft models is quite powerful.

-- Custom Shader

customShaderPhysical = {
name = "Custom Physical",

options =
{
USE_COLOR = { true },
USE_LIGHTING = { true },
STANDARD = { true },
--PHYSICAL = { true },
--ENVMAP_TYPE_CUBE = { true },
--ENVMAP_MODE_REFLECTION = { true },
--USE_ENVMAP = { false, {"envMap"} },
},

properties =
{
--envMap = { "cubeTexture", "nil" },
--envMapIntensity = { "float", "0.75" },
refactionRatio = { "float", "0.0" },
map = {"texture2D", nil},
backcol={"vec3", vec3(0.,0.,0.) }
},

pass =
{
base = "Surface",

blendMode = "disabled",
depthWrite = true,
depthFunc = "lessEqual",
renderQueue = "solid",
colorMask = {"rgba"},
cullFace = "back",

vertex =
[[
void vertex(inout Vertex v, out Input o)
{
}
]],

surface =
[[
uniform sampler2D map;
uniform vec3 backcol;
        
void surface(in Input IN, inout SurfaceOutput o)
{
o.diffuse = texture(map, IN.uv).rgb;
if (texture(map, IN.uv).a==0.0 ) o.diffuse.rgb=backcol; 
o.roughness = 0.1;
o.metalness = 0.0;
o.emission = vec3(0.0, 0.0, 0.0);
o.emissive = 0.0;
o.opacity = 1.0;
o.occlusion = 1.0;
}
]]
}
}


function setup()

    domImage=makeDomImage(color(0,0,0,0))
--[[    
    -- pseudoMesh extension 
    extendModel()
    --pseudoMesh sphere
    refsphereModel=craft.model.sphere({radius=1, number=10,faceted=false
                                        ,axes={vec3(0,0,1),vec3(1,0,0),-vec3(0,1,0)} })
--]]    
    
    -- Create a new craft scene
    scene = craft.scene()    
    scene.sky.material.sky=color(40, 35, 244)
    scene.sky.material.horizon=color(15, 199, 250)
    
    assert(OrbitViewer, "Please include Cameras (not Camera) as a dependency")  
    viewer = scene.camera:add(OrbitViewer, vec3(0), 25, 0, 2000)
    
    
    -- Add the shader definitions to the rendering system
    craft.shader.add(customShaderPhysical)    
    
--[[    
    local e1 = scene:entity()
   e1.model = craft.model.icosphere(1,3)
--    e1.model=craft.model("Dropbox:DOMSimplified_blend.obj")                     

    e1.material = craft.material(asset.builtin.Materials.Standard)
    e1.material.map = domImage
    e1.x = 2
    e1.z = 10
--]]
        
    backcol=color(233, 80, 118)
    local e2 = scene:entity()
    e2.model=craft.model.icosphere(1,3)
    e2.material = craft.material("Custom Physical")    
    e2.material.map = domImage
    e2.material.backcol=backcol
    e2.x = -2
    e2.z = 10
    
    backcol=color(80, 92, 233)
    local e3 = scene:entity()
    e3.model = craft.model("Primitives:Sphere")
--    e3.model = refsphereModel
    e3.material = craft.material("Custom Physical")    
    e3.material.map = domImage
    e3.material.backcol=backcol
    e3.x = -6
    e3.z = 10
    
end
    
function update(dt)
    scene:update(dt)
end
    
function draw()
    update(DeltaTime)
    scene:draw()    
--   sprite(domImage, WIDTH/2, HEIGHT/4)  
end
15

@piinthesky Is this supposed to work.

16

@dave1707 yes! but you need to combine with the makeDomImage function i posted at the begining of this thread. As both the icosphere and primitive:sphere have problems with their vertices the mapping of the texture on the sphere is not perfect.

The important point here is that one can implement vertex and fragment shaders on craft models without having to use Shade- just like we did previously with meshes pre-craft.

17

@John is there a way to implement ‘specular’ material with your custom shader approach?

18

@piinthesky Looks like you’re having trouble putting a texture on an icosphere. Here’s a program I created long ago that textures an icosphere with no problems. I included a map for Earth. The arguments to the function createSphere(pos,size,level,flat,ins,image) is

Pos= x,y,z position of sphere
Size= size of sphere
Level= Level of the icosphere. Doesn’t work with level 0
Flat= draws flat surface of triangles (true or false)
Ins= Allows you to see the sphere from the inside (true or false)
Image= texture for icosphere. The width should be twice the height for best results.

19

@dave1707 - takes me back to the reason you built that routine - Craft didn’t seem to texture the sphere properly. Do you know if that issue has been corrected?

20

@Bri_G Putting an image on an icosphere still has problems. There’s a jagged area running from the top to the bottom of the sphere. That’s why I wrote my own routine that correctly wraps an image on an icosphere.