Draw scaling mystery? (Plus much faster Gaussian blur)

yojimbo2000 · September 3, 2015, 9:50am

OK this is a real head scratcher.

I’ve managed to greatly speed up (and simplify) my Gaussian blur shader by scaling all of the drawing down by 0.25. When the result is up scaled again x4, GLSL smoothing adds to the blurriness. It looks great, and on my iPad Air can handle the fullscreen at 60 fps (with 14x14 = 196 samples per pixel for the blur effect).

BUT here’s the weird thing. I’m not actually upscaling the image x4. I didn’t get that far in my test. In the code below, I expected to see the output image to still be width/0.25, height/0.25. Those are the dimensions of the rect that it is drawing (tap the screen to compare the image sizes. The blurred one should be the same size as the input test one). But, for some reason, it is drawing to the full width and height of the screen! How can this be? My code is doing what I eventually want it to do, but it is doing it before I’ve got there?

If anyone can solve this mystery I’ll be really grateful.

(See the comments to lines 47 and 49 in Main)


--# Main
--Gaussian blur
--adapted by Yojimbo2000 from http://xissburg.com/faster-gaussian-blur-in-glsl/ 
    
function setup()
    downSample = 0.2
    local dimensions = vec2(WIDTH, HEIGHT) * downSample --down sampled
    
    blurTex = {} --image textures
    blurMesh = {} --meshes
    for i=1,2 do --2 passes, 1: horizontal, 2: vertical
        blurTex[i]=image(dimensions.x, dimensions.y)   
        local m =mesh()
        m.texture=blurTex[i]
        m:addRect(dimensions.x/2, dimensions.y/2,dimensions.x, dimensions.y)
        m.shader=shader(Gaussian.vs[i], Gaussian.fs)
        blurMesh[i] = m
    end
    unblurred=mesh() --mesh w/o the blur shader
    unblurred.texture=blurTex[1]
    unblurred:addRect(dimensions.x/2, dimensions.y/2,dimensions.x, dimensions.y) --(WIDTH/2,HEIGHT/2,WIDTH,HEIGHT) --
    showBlur=true  --blur toggle
    pos={x1=200, y2=200, x3=800}
    tween(2, pos, {x1=800, y2=800, x3=200}, {easing=tween.easing.cubicInOut, loop=tween.loop.pingpong})
    movement = true 
    profiler.init()
    print("tap to toggle blur shader")
end

function draw()
    if movement then
        setContext(blurTex[1])
        pushMatrix()
        scale(downSample)
        background(50)
        
        sprite("SpaceCute:Rocketship",pos.x1,HEIGHT-200)
        sprite("Small World:Store Extra Large",pos.x3,200,300)
        sprite("SpaceCute:Beetle Ship",WIDTH/2,pos.y2)
        popMatrix()
    end
    
    if showBlur then --draw blur if required
        setContext(blurTex[2])
        blurMesh[1]:draw() --pass one, offscreen
        setContext()
        blurMesh[2]:draw() --pass two. Why does this draw fullscreen??...
    else
        setContext()
        unblurred:draw() --...it should draw at the downsampled size, like the unblurred mesh?
    end
    profiler.draw()
end

--touching toggles blur on and off
function touched(t)
    if t.state==ENDED then showBlur=not showBlur end
end

profiler={}

function profiler.init(quiet)    
    profiler.del=0
    profiler.c=0
    profiler.fps=0
    profiler.mem=0
    if not quiet then
        parameter.watch("profiler.fps")
        parameter.watch("profiler.mem")
    end
end

function profiler.draw()
    profiler.del = profiler.del +  DeltaTime
    profiler.c = profiler.c + 1
    if profiler.c==10 then
        profiler.fps=profiler.c/profiler.del
        profiler.del=0
        profiler.c=0
        profiler.mem=collectgarbage("count", 2)
    end
end
--# Gaussian
Gaussian = {
vs = { -- horizontal pass vertex shader
[[
uniform mat4 modelViewProjection;

attribute vec4 position;
attribute vec2 texCoord;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_Position = modelViewProjection * position;
    vTexCoord = texCoord;
    v_blurTexCoords[ 0] = vTexCoord + vec2(-0.028, 0.0);
    v_blurTexCoords[ 1] = vTexCoord + vec2(-0.024, 0.0);
    v_blurTexCoords[ 2] = vTexCoord + vec2(-0.020, 0.0);
    v_blurTexCoords[ 3] = vTexCoord + vec2(-0.016, 0.0);
    v_blurTexCoords[ 4] = vTexCoord + vec2(-0.012, 0.0);
    v_blurTexCoords[ 5] = vTexCoord + vec2(-0.008, 0.0);
    v_blurTexCoords[ 6] = vTexCoord + vec2(-0.004, 0.0);
    v_blurTexCoords[ 7] = vTexCoord + vec2( 0.004, 0.0);
    v_blurTexCoords[ 8] = vTexCoord + vec2( 0.008, 0.0);
    v_blurTexCoords[ 9] = vTexCoord + vec2( 0.012, 0.0);
    v_blurTexCoords[10] = vTexCoord + vec2( 0.016, 0.0);
    v_blurTexCoords[11] = vTexCoord + vec2( 0.020, 0.0);
    v_blurTexCoords[12] = vTexCoord + vec2( 0.024, 0.0);
    v_blurTexCoords[13] = vTexCoord + vec2( 0.028, 0.0);
}]],
-- vertical pass vertex shader
 [[
uniform mat4 modelViewProjection;

attribute vec4 position;
attribute vec2 texCoord;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_Position = modelViewProjection * position;
    vTexCoord = texCoord;
    v_blurTexCoords[ 0] = vTexCoord + vec2(0.0, -0.028);
    v_blurTexCoords[ 1] = vTexCoord + vec2(0.0, -0.024);
    v_blurTexCoords[ 2] = vTexCoord + vec2(0.0, -0.020);
    v_blurTexCoords[ 3] = vTexCoord + vec2(0.0, -0.016);
    v_blurTexCoords[ 4] = vTexCoord + vec2(0.0, -0.012);
    v_blurTexCoords[ 5] = vTexCoord + vec2(0.0, -0.008);
    v_blurTexCoords[ 6] = vTexCoord + vec2(0.0, -0.004);
    v_blurTexCoords[ 7] = vTexCoord + vec2(0.0,  0.004);
    v_blurTexCoords[ 8] = vTexCoord + vec2(0.0,  0.008);
    v_blurTexCoords[ 9] = vTexCoord + vec2(0.0,  0.012);
    v_blurTexCoords[10] = vTexCoord + vec2(0.0,  0.016);
    v_blurTexCoords[11] = vTexCoord + vec2(0.0,  0.020);
    v_blurTexCoords[12] = vTexCoord + vec2(0.0,  0.024);
    v_blurTexCoords[13] = vTexCoord + vec2(0.0,  0.028);
}]]},
--fragment shader
fs = [[precision mediump float;
 
uniform lowp sampler2D texture;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_FragColor = vec4(0.0);
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 0])*0.0044299121055113265;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 1])*0.00895781211794;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 2])*0.0215963866053;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 3])*0.0443683338718;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 4])*0.0776744219933;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 5])*0.115876621105;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 6])*0.147308056121;
    gl_FragColor += texture2D(texture, vTexCoord         )*0.159576912161;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 7])*0.147308056121;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 8])*0.115876621105;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 9])*0.0776744219933;
    gl_FragColor += texture2D(texture, v_blurTexCoords[10])*0.0443683338718;
    gl_FragColor += texture2D(texture, v_blurTexCoords[11])*0.0215963866053;
    gl_FragColor += texture2D(texture, v_blurTexCoords[12])*0.00895781211794;
    gl_FragColor += texture2D(texture, v_blurTexCoords[13])*0.0044299121055113265;
}]]
}

UberGoober · January 3, 2016, 3:12pm

@yojimbo2000 : honestly reading this kind of code is a bit beyond my skills right now. But I do have a thought.

Since OpenGL always uses the 0-1 scale for all its parameters, it might be quite easy to inadvertently feed a scaled-down image to an OpenGL process and have that process treated as full-size.

I.e. Something like this:
[WARNING: PSEUDOCODE!]

Image = fullSize()
SmallImage = halfSize()
-- hypothetical OpenGL function: 
ProcessedFullSize = openGLDoSomethingWithImage(fullSize)
ProcessedHalfSize = openGLDoSomethingWithImage(HalfSize)
ProcessedFullSize.size == ProcessedHalfSize.size

As I understand it, without any scale value passed to it, openGLDoSomethingWithImage will treat each image as having a scale value of 1, so in each case whatever it returns will always have the same size.

Again, the code is too advanced for me to locate where such a thing might be happening, but perhaps this is a lead to finding it.

Ignatz · January 3, 2016, 3:14pm

@yojimbo2000 - adding setContext() after line 44 fixes it. It seems you need to close one context before starting another.

yojimbo2000 · January 3, 2016, 3:41pm

@Ignatz gosh you’re right. Thanks!

Ok, here’s the code all fixed up. The performance is now very good, because both passes of the blur are performed on an image of the downsampled size. One interesting thing: it’s better to choose a downsample factor that is not a clean factor of 1. Eg 0.15 or 0.2 is better than 0.25. The reason why, is that you can perceive a pixelization effect with a factor of 0.25, that is obscured if you choose a non-even factor.


--# Main
--Gaussian blur
--adapted by Yojimbo2000 from http://xissburg.com/faster-gaussian-blur-in-glsl/ 
    
function setup()
    downSample = 0.15
    local dimensions = {vec2(WIDTH, HEIGHT) * downSample, vec2(WIDTH, HEIGHT)}--down sampled
    
    blurTex = {} --image textures
    blurMesh = {} --meshes
    for i=1,2 do --2 passes, 1: horizontal, 2: vertical
        blurTex[i]=image(dimensions[1].x, dimensions[1].y)   
        local m =mesh()
        m.texture=blurTex[i]
        m:addRect(dimensions[i].x/2, dimensions[i].y/2,dimensions[i].x, dimensions[i].y)
        m.shader=shader(Gaussian.vs[i], Gaussian.fs)
        blurMesh[i] = m
    end
    unblurred=mesh() --mesh w/o the blur shader
    unblurred.texture=blurTex[1]
    unblurred:addRect(WIDTH/2,HEIGHT/2,WIDTH,HEIGHT) --(dimensions.x/2, dimensions.y/2,dimensions.x, dimensions.y) --
    showBlur=true --blur toggle
    pos={x1=200, y2=200, x3=800}
    tween(2, pos, {x1=800, y2=800, x3=200}, {easing=tween.easing.cubicInOut, loop=tween.loop.pingpong})
    movement = true 
    profiler.init()
    print("tap to toggle blur shader")
end

function draw()
    if movement then
        setContext(blurTex[1])
        pushMatrix()
        scale(downSample)
        background(50)
        
        sprite("SpaceCute:Rocketship",pos.x1,HEIGHT-200)
        sprite("Small World:Store Extra Large",pos.x3,200,300)
        sprite("SpaceCute:Beetle Ship",WIDTH/2,pos.y2)
        popMatrix()
        setContext()
    end
    
    if showBlur then --draw blur if required
        setContext(blurTex[2])
        blurMesh[1]:draw() --pass one, offscreen
        setContext()
        blurMesh[2]:draw() --pass two. fullscreen
    else
        unblurred:draw() 
    end
    profiler.draw()
end

--touching toggles blur on and off
function touched(t)
    if t.state==ENDED then showBlur=not showBlur end
end

profiler={}

function profiler.init(quiet)    
    profiler.del=0
    profiler.c=0
    profiler.fps=0
    profiler.mem=0
    if not quiet then
        parameter.watch("profiler.fps")
        parameter.watch("profiler.mem")
    end
end

function profiler.draw()
    profiler.del = profiler.del +  DeltaTime
    profiler.c = profiler.c + 1
    if profiler.c==10 then
        profiler.fps=profiler.c/profiler.del
        profiler.del=0
        profiler.c=0
        profiler.mem=collectgarbage("count", 2)
    end
end
--# Gaussian
Gaussian = {
vs = { -- horizontal pass vertex shader
[[
uniform mat4 modelViewProjection;

attribute vec4 position;
attribute vec2 texCoord;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_Position = modelViewProjection * position;
    vTexCoord = texCoord;
    v_blurTexCoords[ 0] = vTexCoord + vec2(-0.028, 0.0);
    v_blurTexCoords[ 1] = vTexCoord + vec2(-0.024, 0.0);
    v_blurTexCoords[ 2] = vTexCoord + vec2(-0.020, 0.0);
    v_blurTexCoords[ 3] = vTexCoord + vec2(-0.016, 0.0);
    v_blurTexCoords[ 4] = vTexCoord + vec2(-0.012, 0.0);
    v_blurTexCoords[ 5] = vTexCoord + vec2(-0.008, 0.0);
    v_blurTexCoords[ 6] = vTexCoord + vec2(-0.004, 0.0);
    v_blurTexCoords[ 7] = vTexCoord + vec2( 0.004, 0.0);
    v_blurTexCoords[ 8] = vTexCoord + vec2( 0.008, 0.0);
    v_blurTexCoords[ 9] = vTexCoord + vec2( 0.012, 0.0);
    v_blurTexCoords[10] = vTexCoord + vec2( 0.016, 0.0);
    v_blurTexCoords[11] = vTexCoord + vec2( 0.020, 0.0);
    v_blurTexCoords[12] = vTexCoord + vec2( 0.024, 0.0);
    v_blurTexCoords[13] = vTexCoord + vec2( 0.028, 0.0);
}]],
-- vertical pass vertex shader
 [[
uniform mat4 modelViewProjection;

attribute vec4 position;
attribute vec2 texCoord;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_Position = modelViewProjection * position;
    vTexCoord = texCoord;
    v_blurTexCoords[ 0] = vTexCoord + vec2(0.0, -0.028);
    v_blurTexCoords[ 1] = vTexCoord + vec2(0.0, -0.024);
    v_blurTexCoords[ 2] = vTexCoord + vec2(0.0, -0.020);
    v_blurTexCoords[ 3] = vTexCoord + vec2(0.0, -0.016);
    v_blurTexCoords[ 4] = vTexCoord + vec2(0.0, -0.012);
    v_blurTexCoords[ 5] = vTexCoord + vec2(0.0, -0.008);
    v_blurTexCoords[ 6] = vTexCoord + vec2(0.0, -0.004);
    v_blurTexCoords[ 7] = vTexCoord + vec2(0.0,  0.004);
    v_blurTexCoords[ 8] = vTexCoord + vec2(0.0,  0.008);
    v_blurTexCoords[ 9] = vTexCoord + vec2(0.0,  0.012);
    v_blurTexCoords[10] = vTexCoord + vec2(0.0,  0.016);
    v_blurTexCoords[11] = vTexCoord + vec2(0.0,  0.020);
    v_blurTexCoords[12] = vTexCoord + vec2(0.0,  0.024);
    v_blurTexCoords[13] = vTexCoord + vec2(0.0,  0.028);
}]]},
--fragment shader
fs = [[precision mediump float;
 
uniform lowp sampler2D texture;
 
varying vec2 vTexCoord;
varying vec2 v_blurTexCoords[14];
 
void main()
{
    gl_FragColor = vec4(0.0);
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 0])*0.0044299121055113265;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 1])*0.00895781211794;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 2])*0.0215963866053;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 3])*0.0443683338718;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 4])*0.0776744219933;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 5])*0.115876621105;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 6])*0.147308056121;
    gl_FragColor += texture2D(texture, vTexCoord         )*0.159576912161;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 7])*0.147308056121;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 8])*0.115876621105;
    gl_FragColor += texture2D(texture, v_blurTexCoords[ 9])*0.0776744219933;
    gl_FragColor += texture2D(texture, v_blurTexCoords[10])*0.0443683338718;
    gl_FragColor += texture2D(texture, v_blurTexCoords[11])*0.0215963866053;
    gl_FragColor += texture2D(texture, v_blurTexCoords[12])*0.00895781211794;
    gl_FragColor += texture2D(texture, v_blurTexCoords[13])*0.0044299121055113265;
}]]
}

UberGoober · January 6, 2016, 6:28pm

Should I use this instead of the blur code from Soda?

yojimbo2000 · January 6, 2016, 6:45pm

I think this pretty much is the blur code from Soda?

yojimbo2000 · January 6, 2016, 7:31pm

Actually, @UberGoober (or anyone else with an iPad 3), could you run the code I posted above and let me know what FPS you get please? Thanks.

stevon8ter · January 6, 2016, 7:38pm

@yojimbo2000 I tried it an got between 37 and 40 fps (ipad 3)

37 being the longest and then an ocasional jump to 39.9/40 (ocasional as in 1/4th or 1/3th of the total time [which would make 37fps be 3/4th or 2/3th], just trying to make it as clear as possible )

yojimbo2000 · January 6, 2016, 8:47pm

@stevon8ter thanks! Not quite as good as I’d hoped. On the other thread, @UberGoober was saying he got 50 on the iPad 3 for the version that just does 5 samples per pass.