Color gradients

I am working on on a program that colorizes grayscale images with color gradients.
The first hurdle was to create good color gradients.
Why? What’s wrong with normal gradients?
The problem with normal color gradients like in photoshop or illustrator, is that they make linear transitions from one color stop to the next. In your artwork or photo that shows up as oversaturated stripes. I would find myself manually adding more color stops to the gradient to make the transitions more gradual. That is a tedious process. You can also use the duo tone system, but that is even more cumbersome.
This program tackles the problem with a different approach.
The classic color gradients can be seen as a 3D polygon through the rgb color cube. Starting from the black point drawing straight lines to the color stops to end at the white point. A grayscale gradient would be a straight diagonal line from black to white.
This program changes the polygon into a Bézier curve at starts at the black point, but then curves around the gray diagonal in the middle.
The program takes three colors, a pure color, a shade and highlight as input and uses them as attractor points. Non of the attractor colors are actually one the curve, but they sequential pull the curve away from the gray center line.

The next issue was to compensate for luminosity. Colors have dramatically different luminosity compared to there rgb values, so I had to find a way to redistribute the color stops according to their perceived lightness.

This is my first Codea program after many years of not coding and I’m still on a steep learning curve.
Feel free to point out where my code is clunky or old school.
For instance the colorize function changes the test image pixel by pixel. Is there a way to replace that function with a shader?

I also have a few other questions, but first I’ll post the code and show you what it does.

--# Main
-- colorize
-- This program takes three colors, a pure color, a highlight and a shade, to
-- produce a smooth gradient from black to white that gives a natural
-- looking range of colors from dark to light.
-- The approach is different from regular gradient functions like in photoshop.
-- normally you would build up a gradient with white and black in the end stops
-- and put in color stops somewhere in between and make liniar transitions
-- from one to the next. In the 3d rgb color cube that would look like a
-- polygon with straight lines and hard edges at every stop.
-- Those hard edges show up in your artwork as ugly spikes of oversaturated 
-- stripes. I made this program to come up with something better that looks
-- more natural.
-- This program takes out those hard edges. Instead of a polygon, it approaches 
-- the gradient as a bezier curve through the rgb color cube. The gradient goes
-- from black to white and inbetween it is pulled away from the gray diagonal 
-- line, by the three input colors that work as atractor points. 
-- Similar to the curve tools that you find in Inkscape or Illustrator, 
-- the line never touches the atractors at any point in the line.
-- After that it redistributes the color stops to compensate for their perceived 
-- luminosity.


-- Use this function to perform your initial setup
function setup()
    parameter.color("highlight",color(255,255,128))
    parameter.color("purecolor",color(255,0,0))
    parameter.color("shade",color(0,0,127))
    
    n=255 -- the number of color stops
-- first build a small test image.    
    photo=image(240,240) 
    setContext(photo)
    for i=25,255 do
        fill(i)
        ellipse(120+i/6,120+i/5,255-i)   
    end
    setContext()

    range=image(5,1000) -- this image stores the color range.
    white=color(255) -- white and black are used as constants.
    black=color(0)
    photo=decolorize(photo) -- the test image is actually grayscale already
    photo2=photo -- photo2 will contain the colorized image. 
    imageX=WIDTH/2-240 -- the image can be dragged around. 
    imageY=HEIGHT/2-240 -- for future purpose of building a drawing program.
    print("Change the colors to create different color ranges.")
    print("Tap the colored gradient bar on the right to apply the range to the image.")
end

-- This function gets called once every frame
function draw()
    background(127, 127, 127, 255)
    noStroke()
    spriteMode(CORNER)
    sprite(photo2,imageX,imageY,500)
    setContext(range)
    dg(0,1,shade,purecolor,highlight) -- build up the gradient map in range.
    setContext()    
    sprite(range,WIDTH-40,0,40,HEIGHT) -- put the range on the screen on the right.
    -- indicate the atractor points next to the range for user reference.
    drwAtractors(shade,purecolor,highlight)
end
function luminence(c) -- this fuctions returns the perceived lightness of a color
    -- the mathematical equasion is highly debated on forums, but this one got 
    -- the most consensus.
    return math.sqrt(0.299*c.r*c.r + 0.587*c.g*c.g + 0.114*c.b*c.b)/255
end
-- I didn't use the b3 and b4 bezier functions with resp. 1 and 2 attractors. 
-- but if you want to make your own version with on or two attractors, it is there.
function b3(t,p0,p1,p2) 
    return math.pow((1-t),2)*p0
            +2*t*(1-t)*p1
            +math.pow(t,2)*p2
end
function b4(t,p0,p1,p2,p3) 
        return  math.pow((1-t),3)*p0
                +3*t*math.pow((1-t),2)*p1
                +3*math.pow(t,2)*(1-t)*p2
                +math.pow(t,3)*p3
end
function b5(t,p0,p1,p2,p3,p4) -- this is the bezier function actually used
        return  math.pow((1-t),4)*p0
                +4*t*math.pow((1-t),3)*p1
                +6*math.pow(t,2)*math.pow(1-t,2)*p2
                +4*math.pow(t,3)*(1-t)*p3
                +math.pow(t,4)*p4
end
function dg(t0,t1,c1,c2,c3)
        local k0=b5(t0,black, c1,c2, c3,white)
        local k1=b5(t1,black, c1,c2, c3,white)
        local l0=luminence(k0)
        local l1=luminence(k1)
        if l1>l0 then
            if l1-l0 < 1/n then
                fill(k0)
                rect(0,l0*1000,5,l1*1000)
            else
                local t3=t0+(t1-t0)/2
                dg(t0,t3,c1,c2,c3)
                dg(t3,t1,c1,c2,c3)
            end
        end
end
function drwAtractors(c1,c2,c3)
    -- this function plots the atractor points next to the gradient on the screen
    -- according to their luminence to get an idea where they influence the 
    -- gradient most. Yellows tend to pull up and blues pull down quite 
    -- considerably.
    local k0=b5(1/4,black, c1,c2, c3,white)
    local k1=b5(1/2,black, c1,c2, c3,white)
    local k2=b5(3/4,black, c1,c2, c3,white)
    local l0=luminence(k0)
    local l1=luminence(k1)
    local l2=luminence(k2)
    fill(black)     
    rect(WIDTH-60,0,20,HEIGHT)  
    fill(c1)
    ellipse(WIDTH-50,HEIGHT*l0,20)
    fill(c2)
    ellipse(WIDTH-50,HEIGHT*l1,20)
    fill(c3)
    ellipse(WIDTH-50,HEIGHT*l2,20)
end

function touched(t)
    if t.x> WIDTH-40 then 
        if t.state==ENDED 
        then   
            photo2=colorize(photo)
        else
            photo2=photo
        end
    end
    if t.state== MOVING -- to drag the image around
    then
        imageX=imageX+ t.deltaX
        imageY=imageY+ t.deltaY
    end
    
end

function decolorize(img)
    -- this function turns a photo into black and white.
    -- it doesn't just average the rgb values but actually calculates 
    -- luminosity.
    local r,g,b,a,l,t,c,k
    img2 = img:copy()
    for x= 1, img2.width do
        for y = 1, img2.height do
            r,g,b,a = img:get(x,y)
            l=math.floor(255*luminence(color(r,g,b)))
            img2:set(x,y,l,l,l,a)
            end    
    end
    popMatrix()
    popStyle()
    return img2
end
function colorize(img)
    -- this function replaces the grays in the image with a color from the
    -- color range pixel by pixel in two for loops.
    local r,g,b,a,l,t,c,k
    img2 = img:copy()
    for x= 1, img2.width do
        for y = 1, img2.height do
            r,g,b,a = img:get(x,y)
            l=math.floor(999*r/255)+1
            
            r,g,b=range:get(3,l)
            img2:set(x,y,r,g,b,a)
            end       
    end
    popMatrix()
    popStyle()
    return img2
end

Hmm, how do I paste my code into a post?
Ah, got it.

Here is a video. http://youtu.be/o7EctkaGXII

a shader can absolutely do this for you, as I’m sure someone will demonstrate…but nice work anyway

Here’s an example of a colour gradient shader:

http://codea.io/talk/discussion/6772/multiple-step-colour-gradient-shader

It just uses the GLES smoothstep function, which I believe is a hermite curve, so not smooth enough for your needs I imagine. It wouldn’t be too hard to throw a different algorithm into the fragment shader though.

Thanks @Ignatz! I think I figured it out with your ebook.
Just made my first fragment shader.

//
// A basic fragment shader
//

//Default precision qualifier
precision highp float;

//This represents the current texture on the mesh
uniform lowp sampler2D texture;
uniform lowp sampler2D range;
//The interpolated vertex color for this fragment
varying lowp vec4 vColor;

//The interpolated texture coordinate for this fragment
varying highp vec2 vTexCoord;

void main()
{
    //Sample the texture at the interpolated coordinate
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;

    //Set the output color to the texture color
    col= texture2D( range, vec2(0.5,col));
    gl_FragColor = col;
}

The trick is to feed it the color range image as a second texture and use it as a lookup table for the first.
Thanks for putting me on the trail of the smoothstep function and the hermite curve, @yojimbo2000. That will come in handy in situations where I want to force the curve through anchor points. I didn’t have an algorithm for that.
More to experiment.

@tyxs - that’s great, well done!

Have fun! =D>

All done. Now all time critical functions have been replaced by shaders.
Changing the parameter colors immediately changes the image.


--# Main
-- colorize
-- This program takes three colors, a pure color, a highlight and a shade, to
-- produce a smooth gradient from black to white that gives a natural
-- looking range of colors from dark to light.
-- The approach is different from regular gradient functions like in photoshop.
-- normally you would build up a gradient with white and black in the end stops
-- and put in color stops somewhere in between and make liniar transitions
-- from one to the next. In the 3d rgb color cube that would look like a
-- polygon with straight lines and hard edges at every stop.
-- Those hard edges show up in your artwork as ugly spikes of oversaturated 
-- stripes. I made this program to come up with something better that looks
-- more natural.
-- This program takes out those hard edges. Instead of a polygon, it approaches 
-- the gradient as a bezier curve through the rgb color cube. The gradient goes
-- from black to white and inbetween it is pulled away from the gray diagonal 
-- line, by the three input colors that work as atractor points. 
-- Similar to the curve tools that you find in Inkscape or Illustrator, 
-- the line never touches the atractors at any point in the line.
-- After that it redistributes the color stops to compensate for their perceived 
-- luminosity.


-- Use this function to perform your initial setup
function setup()
    parameter.color("highlight",color(255,255,128))
    parameter.color("purecolor",color(255,0,0))
    parameter.color("shade",color(0,0,127))
    
    n=255 -- the number of color stops
-- first build a small test image.    
    photo=image(500,500) 
    setContext(photo)
    for i=25,500 do
        fill(255*i/500)
        ellipse(250+i/5,250+i/5,500-i*0.95)   
    end
    setContext()
   -- photo=readImage() -- insert your own image to experiment
    range=image(5,1000) -- this image stores the color range.
    white=color(255) -- white and black are used as constants.
    black=color(0)
    imageX=WIDTH/2 -- the image can be dragged around. 
    imageY=HEIGHT/2-- for future purpose of building a drawing program.
    print("Change the colors to create different color ranges.")
    setupmeshshader()
end

-- This function gets called once every frame
function draw()
    background(127, 127, 127, 255)
    noStroke()
    spriteMode(CORNER)
    setContext(range)
    dg(0,1,shade,purecolor,highlight) -- build up the gradient map in range.
    setContext() 
    m.shader.range=range
    m.texture = photo
    local cw,ch = spriteSize(photo)
    m:setRect(rIdx, imageX,imageY, cw, ch)
    m:draw()
    sprite(range,WIDTH-40,0,40,HEIGHT) -- put the range on the screen on the right.
    -- indicate the atractor points next to the range for user reference.
    drwAtractors(shade,purecolor,highlight)
end
function luminence(c) -- this fuctions returns the perceived lightness of a color
    -- the mathematical equasion is highly debated on forums, but this one got 
    -- the most consensus.
    return math.sqrt(0.299*c.r*c.r + 0.587*c.g*c.g + 0.114*c.b*c.b)/255
end
-- I didn't use the b3 and b4 bezier functions with resp. 1 and 2 attractors. 
-- but if you want to make your own version with on or two attractors, it is there.
function b3(t,p0,p1,p2) 
    return math.pow((1-t),2)*p0
            +2*t*(1-t)*p1
            +math.pow(t,2)*p2
end
function b4(t,p0,p1,p2,p3) 
        return  math.pow((1-t),3)*p0
                +3*t*math.pow((1-t),2)*p1
                +3*math.pow(t,2)*(1-t)*p2
                +math.pow(t,3)*p3
end
function b5(t,p0,p1,p2,p3,p4) -- this is the bezier function actually used
        return  math.pow((1-t),4)*p0
                +4*t*math.pow((1-t),3)*p1
                +6*math.pow(t,2)*math.pow(1-t,2)*p2
                +4*math.pow(t,3)*(1-t)*p3
                +math.pow(t,4)*p4
end
function dg(t0,t1,c1,c2,c3)
        local k0=b5(t0,black, c1,c2, c3,white)
        local k1=b5(t1,black, c1,c2, c3,white)
        local l0=luminence(k0)
        local l1=luminence(k1)
        if l1>l0 then
            if l1-l0 < 1/n then
                fill(k0)
                rect(0,l0*1000,5,l1*1000)
            else
                local t3=t0+(t1-t0)/2
                dg(t0,t3,c1,c2,c3)
                dg(t3,t1,c1,c2,c3)
            end
        end
end
function drwAtractors(c1,c2,c3)
    -- this function plots the atractor points next to the gradient on the screen
    -- according to their luminence to get an idea where they influence the 
    -- gradient most. Yellows tend to pull up and blues pull down quite 
    -- considerably.
    local k0=b5(1/4,black, c1,c2, c3,white)
    local k1=b5(1/2,black, c1,c2, c3,white)
    local k2=b5(3/4,black, c1,c2, c3,white)
    local l0=luminence(k0)
    local l1=luminence(k1)
    local l2=luminence(k2)
    fill(black)     
    rect(WIDTH-60,0,20,HEIGHT)  
    fill(c1)
    ellipse(WIDTH-50,HEIGHT*l0,20)
    fill(c2)
    ellipse(WIDTH-50,HEIGHT*l1,20)
    fill(c3)
    ellipse(WIDTH-50,HEIGHT*l2,20)
end

function touched(t)
    
    if t.state== MOVING -- to drag the image around
    then
        imageX=imageX+ t.deltaX
        imageY=imageY+ t.deltaY
    end
    
end


--# Shader
function setupmeshshader()
m = mesh()
    m.texture = photo
    
   
 rIdx = m:addRect(0, 0, 0, 0)
m.shader = shader(
    [[//This is the current model * view * projection matrix
// Codea sets it automatically
uniform mat4 modelViewProjection;

//This is the current mesh vertex position, color and tex coord
// Set automatically
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;

//This is an output variable that will be passed to the fragment shader
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;

void main()
{
    //Pass the mesh color to the fragment shader
    vColor = color;
    vTexCoord = texCoord;
    
    //Multiply the vertex position by our combined transform
    gl_Position = modelViewProjection * position;
}
]],[[

//Default precision qualifier
precision highp float;
const vec4 lumcoeff = vec4(0.299,0.587,0.114,0.);
//This represents the current texture on the mesh
uniform lowp sampler2D texture;
uniform lowp sampler2D range;
//The interpolated vertex color for this fragment
varying lowp vec4 vColor;

//The interpolated texture coordinate for this fragment
varying highp vec2 vTexCoord;

void main()
{
    //Sample the texture at the interpolated coordinate
    lowp vec4 col = texture2D( texture, vTexCoord ) * vColor;

    //Set the output color to the texture color
    lowp float a =col.a;
    lowp float l= dot(col,lumcoeff);
    col= texture2D( range, vec2(0.5,l));
    col.a = a;
    gl_FragColor = col;
}]])
end

Awesome! That certainly is a very smooth gradient, looks gorgeous. Putting your own images in is a lot of fun.

I have no idea how the maths works! I’m curious about your background. Is this for a pro imaging application?

If you’re interested in curves that go through control points, I also have a GLES version of the Catmull-Rom curve.

A couple of thoughts. Does the gradient range map need to be built every cycle? You could perhaps add a callback to the color pickers so that it just gets built when the colors are changed? Also, the range gradient image could just be 1 pixel wide, couldn’t it?

@yojimbo Thanks! I’ve been a hobby programmer for years, but totally out of shape these days. I used to make stuff in basic and had formal training in Pascal.
I’m also a hobby artist. In real life I do other stuff. This experiment in Codea is born out of frustration with all painting and sketching programs. I have a latent ambition to launch a drawing app, but let’s be realistic. I’m chewing on every line of code in my limited spare time. Codea doesn’t even have direct access to the photo album. If I can make an awesome drawing tool on the iPad for my own use, I’m a happy coder and artist.

So the frustration that fuels this program is the way that all drawing programs deal with color. They don’t offer a logical way to colorize gray drawings or photos. Have you ever tried to colorize a black and white photo in Photoshop? It’s a struggle and there are no simple guidelines. And the results are usually disappointing. It’s not that I don’t know how to do it. I know several ways to color a picture in Photoshop. I’m just not satisfied with any of the available solutions.

So this Codea experiment is to test some of my ideas how to do it better.
One of the problems Is that the rgb color system doesn’t take a colors luminosity into account. Luminosity is the perceived lightness of a color. In the rgb color system red, green and blue build up to white in 255 equal steps. But in our visual system, they don’t contribute to the luminosity of white in equal amounts. Blue only adds a fraction of light to a color and green adds the most. Green and red together, making yellow come very close to white. So when you change a gray pixel into a color, you actually change the luminosity of that pixel dramatically.

My idea is to make drawing or imaging tools that allow you to change a color and at he same time keep the luminosity unchanged unless the user wants to change it. I think this program is a good start and I’m very pleased to lean about the shader possibilities this weekend!

My next step will be to allow the user to paint on the image with this gradient. So that you can color different areas with different gradients.
Another idea is to create a painting tool where you pick a color range instead of just one color. And paint a picture from scratch.

Curves that force through control points will be useful for another idea that I have in mind, but I’ll get back to you when I come to that.
The range image should probably not be built up in the draw function every time. But I haven’t figure out how to structured my events properly, and they don’t slow down the program noticeably, so for now… Good idea to put it in a callback function. But I might as well start thinking about a user interface.

And yes, one pixel wide should do it, but I wondered if the edges don’t blur? And a height of a 1000 is probably more then necessary as well.

Making progress. Just a idea of where I’m heading.
http://youtu.be/omxhfRxs3WY

Looks good! How many pixels is that canvas?

Normal iPad screen size. I havent bothered showing screen edges yet. But it scales and rotates with pinch gesture.