Pixelated camera

General
1

Stuck in the sofa with a cold, I made a pixelated camera. Have only managed to make it work with a square mesh for no, and it doesnt work for a 1x1 for some reason, doubkes the size.

I use a shader to calculate the average value for each pixel.


--# Main
function setup()
    noSmooth()
    spriteMode(CORNER)
    m = mesh()
    m.shader = quadShader
    m.texture = CAMERA
    m:addRect(.5,.5,1,1)
    -- width of pixelated image
    parameter.integer("pw",1,64,32, updatePw)
    -- calculate average of steps*steps pixels
    parameter.integer("steps",1,20,10)
end

function updatePw()
    pi = image(pw,pw)
end

-- This function gets called once every frame
function draw()
    background()
    m.shader.size = 1/(pw*2)
    m.shader.steps = steps
    resetMatrix()
    setContext(pi)
    scale(pw)
    m:draw()
    setContext()
    resetMatrix()
    sprite(pi,0,0,WIDTH)
end


--# Shader
quadShader = shader([[
uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec2 texCoord;
varying highp vec2 vTexCoord;

void main() {
    vTexCoord = texCoord;
    gl_Position = modelViewProjection * position;
}

]],[[
precision highp float;
uniform lowp sampler2D texture;
varying highp vec2 vTexCoord;
uniform float steps;
uniform float size;

void main() {
    float delta = size*2./steps;
    vec4 c = vec4(0.,0.,0.,0.);
    for(float y=-size; y <= size; y += delta) {
        for(float x=-size; x <= size; x += delta ) {
            c += texture2D(texture, vTexCoord + vec2(x, y));
        }
    }
    gl_FragColor = c / (steps*steps);
}
]])
2

Here’s a really lightweight pixel camera I made for use in applications :slight_smile:

displayMode(OVERLAY)
supportedOrientations(LANDSCAPE_ANY)
function setup()
    print("Hello World!")
    spriteMode(CORNER)
    img = image(100,100)
    parameter.integer("pixels",1,500,60,function() rebuild(pixels) end)
end
function rebuild(x)
    img = image(x,x)
end
function draw()
    noSmooth() 
    background(40, 40, 50)
    setContext(img)
    sprite(CAMERA,0,0,img.width,img.height)
    setContext()
    sprite(img,WIDTH-HEIGHT,0,HEIGHT,HEIGHT)
end
3

@tnlogy Interesting. I’ve been working on something where I wanted a pixelation effect. In the end, I decided that a shader would be too slow. What I did instead was to draw the desired segment of the image to a single pixel and then read that pixel. This pixel would be the average colour in that segment.

But I never tested out the shader possibility. What’s it like on speed?

4

Hmm, will that result in the average value, or just a random color?

The speed is quite ok. My plan is to make an algorithm that can get the average pixel color and how big the difference is in the area, maybe using delta-E, which seems to be some standard algorith for it. Then use if for a division algorith to pixelate the most interesting areas.

5

But maybe this solution is better. I moved the color calculation to the vertex shaderinstead. Still it will need to do it 6 times per pixel, but it’s more flexible than having to write to an image first. Runs smoothly.


--# Main
function setup()
    parameter.integer("size",1,64,16,updateSize)
    -- calculate average of steps*steps pixels
    parameter.integer("steps",1,20,10)
end

function updateSize()
    m = mesh()
    m.shader = quadShader
    m.texture = CAMERA
    for y=1,size do
        for x=1,size do
            addPixel(x,y,size)            
        end
    end
end

function addPixel(x,y,size)
    m:addRect(x-.5,y-.5,1,1)
    local i = m.size/6
    local ux,uy = x/(size+1), y/(size+1)
    m:setRectTex(i, ux,uy,0,0)
end

function draw()
    background()
    m.shader.size = .02
    m.shader.steps = steps
    scale(WIDTH/size)
    m:draw()
end


--# Shader
quadShader = shader([[
uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec2 texCoord;
uniform lowp sampler2D texture;
varying lowp vec4 vColor;
uniform float steps;
uniform float size;

void main() {
    float delta = size*2./steps;
    vec4 c = vec4(0.,0.,0.,0.);
    for(float y=-size; y <= size; y += delta) {
        for(float x=-size; x <= size; x += delta ) {
            c += texture2D(texture, texCoord + vec2(x, y));
        }
    }
    vColor = c / (steps*steps);
    gl_Position = modelViewProjection * position;
}

]],[[
varying lowp vec4 vColor;

void main() {
    gl_FragColor = vColor;
}
]])
6

@tnlogy could this shader be used to say create the pixelation effect in the ebay app when pulling out the side menu?

7

@tnlogy Actually, I think that colour interpolation is what is meant to happen when you draw a large sprite to a small region. I just tried it out and it is what happens. And since it is built in, it would be the fastest method. Your method has a lower sampling size, at least potentially, which might speed it up at the expense of accuracy. Also, I’d worry about adding together all those vec4s. The return of a texture2D call is a lowp vec4. If I add a lowp to a highp, is the result automatically converted to highp? I don’t know.

8

I think it is converted, hard to find info on it though, but the result looks ok.

Good to know how to used the built in version, will use it when I’m not interested in controlling the sampling rate and calculating the color difference in the sampling.

Btw, I get no shader errors any more, something is broken?