Shadow Map- Help

Hi! I have begun to experiment with light and other effects and have been trying to import them into projects. Recently, I discovered a shadow shader made by @SpaceMonkey that had been released one year ago. I started to copy and paste a bit to make my own version but somehow it did’nt work. I would be pleased if someone could point out what’s wrong in this code:

function setup()
    parameter.number("k",-100,50,50)
    lp=vec3(0,10,50)
    --simage is the shadowMap
    simage=image(1000,1000)
    sm=matrix()
    updatemesh()
end

function draw()
    lp.z=k
    setContext(simage)
    background(0, 0, 0, 0)
    perspective(50)
    camera(lp.x,lp.y,lp.z,0,0,25)
    m.shader=shader(shad.shadow1vertex,shad.shadow1frag)
    m.shader.lp=lp
    m.shader.modmat=modelMatrix()
    sm=viewMatrix()*projectionMatrix()
    md:draw()
    setContext()
    background(127, 127, 127, 255)
    perspective(50)
    camera(30,50,50,30,0,0)
    --camera(lp.x,lp.y,lp.z,0,0,0)
    m.shader=shader(shad.shadow2vertex,shad.shadow2frag)
    m.shader.lm=sm
    m.shader.lp=vec4(lp.x,lp.y,lp.z,1)
    m.shader.modmat=modelMatrix()
    m.shader.shadowMap=simage
    m:draw()
    resetMatrix()
    ortho()
    viewMatrix(matrix())
    sprite(simage,0,0,300,300)
end

function updatemesh()
    m=mesh()
    m.vertices={vec3(0,0,-20),vec3(0,0,20),vec3(40,0,20),vec3(40,0,20),vec3(40,0,-20),vec3(0,0,-20),
    vec3(0,0,0),vec3(40,0,0),vec3(40,20,0)}
    m:setColors(color(255, 255, 255, 255))
    md=mesh()
    md.vertices=m.vertices
end

shad={shadow1vertex=[[
uniform highp mat4 modelViewProjection;
attribute vec4 position;
varying vec4 wp;
uniform mat4 modmat;
void main()
{
    //convert the vertex into world coordinates (same as the light)    
    wp=position*modmat;
    gl_Position = modelViewProjection * position;
}
]],
shadow1frag=[[
#extension GL_EXT_shader_framebuffer_fetch : require
precision lowp float;
uniform mediump vec3 lp;
varying mediump vec4 wp;
void main()
{
    //if (!gl_FrontFacing) discard;
    highp vec3 tlp = wp.xyz/wp.w;
    float col = max(1.0 - length(tlp.xyz - lp.xyz)/200.0,gl_LastFragData[0].x);
    gl_FragColor = vec4(col, col, col, 1.0);
}
]],
shadow2vertex=[[
uniform mat4 modelViewProjection;
uniform mat4 modmat;
uniform mat4 lm;
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
const mat4 ScaleMatrix = mat4(0.5, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0,0.5, 0.0, 0.5, 0.5, 0.5, 1.0);
varying vec4 wpos;
varying vec4 lipos;
void main()
{
    vColor = color;
    vTexCoord = texCoord;
    wpos=position*modmat;
    lipos=ScaleMatrix*lm*wpos;
    gl_Position = modelViewProjection * position;
}
]],
shadow2frag=[[
precision highp float;
uniform lowp sampler2D texture;
uniform lowp sampler2D shadowMap;
uniform vec4 lp;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
varying vec4 lipos;
varying vec4 wpos;
void main()
{
    //if (!gl_FrontFacing) discard;
    highp vec3 tlp = lipos.xyz/lipos.w; //varying
    float shadowMapDepth = texture2D(shadowMap, tlp.xy).r;
    
    //calculate distance to compare to our shadow map
    //1.004 <- avoid shadow acne compared to 1.0
    float lightDistance = 1.004 - length((wpos.xyz/wpos.w)-lp.xyz)/30.0; //uniform
    
    //if the depth from the shadow map is higher (closer) t
    //han the current depth just use ambience otherwise normal ADS
    //additionally we'll check whether we are out of 
    //bounds on the texture as the shadow map only covers 60 degrees
    //Note that these comments are made by 
    if (shadowMapDepth > lightDistance && tlp.x < 1.0 && tlp.x > 0.0) {
        gl_FragColor =vec4(0,0,0,1);
    }
    else {
        gl_FragColor =vec4(1.,1.,1.,1.);
    }
}
]]}

@MMGames - that is a lot of work to understand and debug this code, and shaders are not easy, so you will be lucky if anyone has the time to help you.

What I suggest you do is try yourself to isolate the problem, because it could be in any of two shaders, how those shaders are implemented, the Codea 3d implementation, or the chosen image.

For example - you are drawing two meshes. Try drawing them one at a time, with and without their shader. If the problem disappears (or doesn’t) it gives you clues where the cause lies. So I would simplify your program, taking bits out until the problem goes away, then home in on whatever caused the change.

Hm… well, the first pass works fine. It gives me a working shadowMap. I think the problem exists in the second shader. It might be that it can’t read the shadowMap properly or that my shadow matrix modifies the vertex position wrong in some way. I might continue experiment myself, but help is still appreciated

@MMGames you also overwrite m (the mesh variable) in updateMesh() so you’ll lose the shader as soon as you do that, it’s also quite expensive to create a mesh again and again.

@Luatee - he only calls UpdateMesh in setup, so that is ok

@MMGames - I assume you tried running spacemonkey’s code successfully. Have you analysed all the changes you’ve introduced? There is no shortcut to debugging something like this, because graphic glitches are hard to interpret, and so I believe you need to painstakingly isolate the problem.

@Ignatz Ahh my bad, I didn’t take a full look.

@Ignatz I will do that. Thanks for the help

@MMGames - sorry it isn’t easier to help you. I know the pain. I have sometimes spent several days trying to track down subtle bugs. :-??

@Ignatz you can say that again, I just spent a week digging through my code to figure out why something wasn’t working, and it was the difference between “suit=i” and “suit=self.suits[i]”

@Doge - lol, we’ve all been there. I think it’s worst with 3D graphics, though, because it’s so hard to figure out what is causing the mess on the screen!

Thanks everyone for the help. Found the problems that caused it to fail.

Can we see the working code also. :slight_smile:

I would also want to have time to experiment a bit more with shadows. :slight_smile:

@tnlogy Here is the code. I did not include any light effects at all, so it’s hard to tell where objects are! :slight_smile:

function setup()
    parameter.watch("1/DeltaTime")
    parameter.number("k",-50,50,0)
    lp=vec3(-20,20,0)
    simage=image(500,500) --resolution of shadowMap
    sm=matrix()
    m=mesh()
    m.vertices={vec3(0,0,0),vec3(0,0,40),vec3(40,0,40),vec3(40,0,40),vec3(40,0,0),vec3(0,0,0),
    vec3(0,0,0),vec3(0,0,10),vec3(0,10,10),
    vec3(10,0,0),vec3(10,0,10),vec3(10,10,10)}
    m:setColors(
    color(60, 255, 0, 255))
    md=mesh() --md is a copy of the original mesh and is used in the first pass
    md.vertices=m.vertices
end

function draw()
    lp.z=k --move the light
    setContext(simage)
    background(0, 0, 0, 0)
    perspective(50)
    camera(lp.x,lp.y,lp.z,0,5,0)
    sm=viewMatrix()*projectionMatrix()
    md.shader=shader(shad.shadow1vertex,shad.shadow1frag)
    md.shader.lp=lp
    md.shader.modmat=modelMatrix()
    md:draw()
    setContext()
background(202, 202, 202, 255)
    perspective(50)
    camera(-40,30,30,15,0,0)
    m.shader=shader(shad.shadow2vertex,shad.shadow2frag)
    m.shader.lm=sm
    m.shader.lp=vec4(lp.x,lp.y,lp.z,1.0)
    m.shader.modmat=modelMatrix()
    m.shader.shadowMap=simage
    m:draw()
end

shad={shadow1vertex=[[
uniform highp mat4 modelViewProjection;
attribute vec4 position;
varying vec4 vPos;
uniform mat4 modmat;
void main()
{  
    vPos=position*modmat;
    gl_Position = modelViewProjection * position;
}
]],
shadow1frag=[[
#extension GL_EXT_shader_framebuffer_fetch : require
precision lowp float;
uniform mediump vec3 lp;
varying mediump vec4 vPos;
void main()
{
    //if (!gl_FrontFacing) discard;
    highp vec3 tlp = vPos.xyz/vPos.w;
    float col = max(1.0 - length(tlp-lp)/200.0,gl_LastFragData[0].x);
    gl_FragColor = vec4(col, col, col,1.0);
}
]],
shadow2vertex=[[
uniform mat4 modelViewProjection;
uniform mat4 modmat;
uniform mat4 lm;
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
const mat4 ScaleMatrix=mat4(0.5, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0,0.5, 0.0, 0.5, 0.5, 0.5, 1.0);
varying vec4 wpos;
varying vec4 lipos;
void main()
{
    vColor=color;
    vTexCoord=texCoord;
    wpos=position*modmat;
    lipos=ScaleMatrix*lm*wpos;
    gl_Position=modelViewProjection*position;
}
]],
shadow2frag=[[
precision highp float;
uniform lowp sampler2D texture;
uniform lowp sampler2D shadowMap;
uniform vec4 lp;
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
varying vec4 lipos;
varying vec4 wpos;
void main()
{
    //if (!gl_FrontFacing) discard;
    float hadow;
    highp vec3 tlp = lipos.xyz/lipos.w; //varying
    float shadowMapDepth=texture2D(shadowMap, tlp.xy).r;
    float lightDistance=1.004-length((wpos.xyz/wpos.w)-lp.xyz)/200.0; //uniform
    if (shadowMapDepth>lightDistance && tlp.x<1.0 && tlp.x>0.0) hadow=0.3;
    else hadow=1.0;
    vec4 col=vColor*hadow;
    col.a=1.0;
    gl_FragColor=col;
}
]]}