NV (Night Vision)

One thing to note: This does NOT just whiten the image! It retains hue and saturation, it just automagically makes it look bright as day!

I was playing around with shaders a lot the other day, and was working on a project called Cinemator, which I might release in a while, but basically, I was fiddling with getting the average of the sampled color, and multiplying the color by the average. Unintentionally, I found a way to make a legitimate night vision shader. It takes the average of the super dark room, adds 20 to it when using the front camera, 80 with the back, and multiplies the color by that. Boom. Bright as day. Somehow, super simple.

Code (long press on New Project):

--# Main
-- NV (Night Vision)

-- Use this function to perform your initial setup
displayMode(STANDARD)

function setup()
    parameter.boolean("CameraFront", true, function(b)
        if b then
            cameraSource(CAMERA_FRONT)
        else
            cameraSource(CAMERA_BACK)
        end
    end)
    parameter.action("Update Camera if Frozen", function()
        if CameraFront then
            cameraSource(CAMERA_BACK)
            tween.delay(1, function()
                cameraSource(CAMERA_FRONT)
            end)
        else
            cameraSource(CAMERA_FRONT)
            tween.delay(1, function()
                cameraSource(CAMERA_BACK)
            end)
        end
    end)
    parameter.number("darkness", 0, 1, 0)
    
    m = mesh()
    m.texture = CAMERA
    m.shader = shader(Shaders.NV.vS, Shaders.NV.fS)
    
    rIdx = m:addRect(0, 0, 0, 0)
    
    saveProjectInfo("Author", "SkyTheCoder")
    saveProjectInfo("Description", "A night vision camera, to really see in the dark! By SkyTheCoder.")
    
    print("-- INFO --")
    print("## PARAMETERS ##")
    print("CameraFront")
    print("__")
    print("On will use the front camera, off will use the back camera.")
    print("____/")
    print("Update Camera if Frozen")
    print("__")
    print("Sometimes the camera will freeze if the iPad locks or exits the app. Press this button to resume live input.")
    print("____/")
    print("darkness")
    print("__")
    print("Ranges from 0 to 1. 0 is a pitch black room, 1 is a daylight-bright room. Changes how much the shader brightens the room.")
    print("____/")
    print("## NOTES ##")
    print("The back camera does not work as well as the front camera. It's a different camera, and its color does not match with the front camera. The shader is optimized for the front camera, and no matter how much I tweak it, the back camera just isn't as good with it.")
    print("__")
    print("The original input is just there to amaze you. If you can, find a pitch black room, and spot the difference. The original can be solid black while the modified is bright as day.")
    print("__")
    print()
    print("Have fun! -Sky")
end

-- This function gets called once every frame
function draw()
    -- This sets a dark background color
    background(63)
    
    -- Here we set up the rect texture and size
    m.texture = CAMERA
    local cw, ch = spriteSize(CAMERA)
    local backCamDiv = 6
    if CameraFront then
        m:setRect(rIdx, 3 * WIDTH/4, HEIGHT/2, cw / 2, ch / 2)
    else
        m:setRect(rIdx, 3 * WIDTH/4, HEIGHT/2, cw / backCamDiv, ch / backCamDiv)
    end
    
    -- Configure out custom uniforms for the NV shader
    m.shader.darkness = darkness
    m.shader.backCam = not CameraFront
    
    -- Draw the mesh
    m:draw()
    
    if CameraFront then
        sprite(CAMERA, WIDTH / 4, HEIGHT /  2, cw / 2, ch / 2)
    else
        sprite(CAMERA, WIDTH / 4, HEIGHT /  2, cw / backCamDiv, ch / backCamDiv)
    end
    
    fill(255)
    
    font("Copperplate-Light")
    fontSize(48)
    
    text("ORIGINAL", WIDTH / 4, 3 * (HEIGHT / 4))
    text("MODIFIED", 3 * (WIDTH / 4), 3 * (HEIGHT / 4))
    
    fontSize(124)
    
    text("NV", WIDTH / 2, 7 * (HEIGHT / 8))
    
    fontSize(48)
    
    text("SKY", WIDTH / 12, HEIGHT / 24)
end

--# Shaders
Shaders = {
NV = {
vS = [[
//
// A night vision vertex shader
//

//This is the current model * view * projection matrix
// Codea sets it automatically
uniform mat4 modelViewProjection;

//This is the current mesh vertex position, color and tex coord
// Set automatically
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;

//This is an output variable that will be passed to the fragment shader
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    //Pass the mesh color to the fragment shader
    vColor = color;
    vTexCoord = texCoord;
    
    //Multiply the vertex position by our combined transform
    gl_Position = modelViewProjection * position;
}

]],
fS = [[
//
// A night vision fragment shader
//

//Default precision qualifier
precision highp float;

//This represents the current texture on the mesh
uniform lowp sampler2D texture;

//How dark the room is. 0 is darkest, 1 is brightest
uniform float darkness;

uniform bool backCam;

//The interpolated vertex color for this fragment
varying lowp vec4 vColor;

//The interpolated texture coordinate for this fragment
varying highp vec2 vTexCoord;

void main()
{
    //Sample the texture at the interpolated coordinate
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;
    
    float average = col.r + col.g + col.b; //Add up the red, green, and blue values
    average /= 3.0; //Divide it by three to get the average
    
    if (backCam) {
        average += (1.0 - darkness) * 80.0; //Add a lot to the average depending on how dark the room is.
    } else {
        average += (1.0 - darkness) * 20.0; //Add a lot to the average depending on how dark the room is.
    }
    
    //Set the output color to the texture color
    if (backCam) {
        gl_FragColor = vec4(col.r * (average / 2.0), col.g * average, col.b * (average / 1.5), col.a); //Multiply the color by the modified average to boost the brightness while retaining hue and saturation. (Not just making the image closer to white!)
    } else {
        gl_FragColor = col * average; //Multiply the color by the modified average to boost the brightness while retaining hue and saturation. (Not just making the image closer to white!)
    }
}

]],
},
}

Steps: Find a dark room, or close all the curtains or something, so you can hardly see. Start up the project. Choose your camera. Be amazed.

Whoa, Looks great! You could make this a standalone app! (Maybe add in a greenish tinge for sci-fi fans)

A good aspect about this to me was that it kept the coloring. I could add a fun option to make it green, but for practical use it’s much better in color. At a weather fest thing I got to try on real army standard night vision goggles. They had to have a cap on it literally a few pixels big so it wouldn’t blind me. They were big and heavy. And everything was green. An iPad is light weight, and this retains color. Much better. :)>-

Best part: You don’t go blind if you use it in daylight! Just a white screen :stuck_out_tongue:

Very interesting, nice! :slight_smile:

Wow this is brilliant! Better than any night vision app I’ve used well done.

Nice work!

Amazing app!

I wish i had an ipad2 or higher to test this… :((

Right, iPad 1 doesn’t have a camera… I might put up a few screenshots.

@Jmv38 - You could export it into the runtime and put it on your iPhone (if you have one). Or you could take a photo in a dark room with a camera and put that as the input photo.

@SkyTheCoder great work. Love how much it boosts the brightness.

(Note: The reason it works better with the front camera is likely because your face is being illuminated by the iPad screen.)

@Simeon I actually did find the whole face illuminating thing in the Shader Lab while I was working on it, and I found on the bindings screen it didn’t illuminate myself, and it worked fine. The back camera is supposed to be much better, and has different ways of getting the colors, and I found that when using it with the same configuration as the front, it was still black. I had to add 80 instead of 20 in order to get it bright enough, and then it had a purple tint, so I lowered the R and B values. Later I found that gave it a green tint, but decided I wouldn’t bother with it now. It still retains some color.