Can I edit a shader so that it gives me the information of a pixel(which satisfies a condition). Like, say I want it to give me the x,y coordinates of a pixel whose colour is (1.,1.,1,1.) or the colour of a pixel whose x,y coordinates are width/2 and height/2 . Can we set values for global variables using the shader lab?
I think the answer is no. The only output you can get from a shader is what it renders. So you can come up with some scenarios by rendering to a buffer (using setContext()) and using that output as a way of getting info out.
For your second scenario, give me the color at a particular coordinate, that could be done, you just pass it the x,y and get it to do a texture lookup and render the color out. But it would be easier to just use image:read() or whatever the Codea function is.
I’ve been exploring motion detection and trying to find ways to get information out of an image efficiently without querying every pixel value. As spacemonkey says, you could draw your mesh to an image instead of the screen, with setContext, but then you have to read all the pixels in the image anyway.
Here is code for a shader that draws an image only for pixels that are changing (ie there is movement). You get a blue image on the screen.
https://gist.github.com/dermotbalson/5772101
However, the best way I found (and I think the most common approach generally) was to sample points with image:get(x,y), eg only look at every 20th pixel, but this will only work where you are looking for chunks of colour.
Thanks @Spacemonkey. @Ignatz I’ve been reading through your tutorial and then started to think about alternative ways of detecting motion. I wrote a code which detects me using colour of my skin. But that was very simple, what I couldn’t understand was how to get the position of the hand from the picture. One more thing @Ignatz I realiased right now that the camera in your program should be flipped acroos the Y-axis that is left should be right and right should be left. Else it works perfectly.
I did some more work on it, and calculated the average x,y position of all the pixels in my hand, and stored it. I can then see how that position changes at the next redraw. Usually it will change in both x and y, so you need to set a minimum threshold so you ignore small differences.
@Ignatz very nice. It crashed mine though… I think the issue is that you create the mesh every draw cycle, and I presume this blows out the memory somehow…
Below modified to declare the mesh once up front, I also added a debug view of the camera in bottom right. Had to do a little messing around because the camera doesn’t start working for a few frames…
-- Motion shader
--shows only moving pixels on screen
function setup()
cameraSource(CAMERA_FRONT)
parameter.integer("FPS",0,60,6)
img0=nil
print("move the ipad around, wave your hand")
m2=mesh()
m2.shader = shader(MoveShader.vertexShader, MoveShader.fragmentShader)
debugMesh = mesh()
debugMesh:addRect(WIDTH-200, 100,400,200)
debugMesh:setRectTex(1,0,0,1,1)
end
function draw()
background(200)
local img=image(CAMERA)
for i=1,10000 do
end
if img0~=nil then --take picture
if m2.size == 0 then
u=m2:addRect(WIDTH/2,HEIGHT/2,img0.width,img0.height)
m2:setRectTex(u,0,0,1,1)
end
m2.texture=img0
m2.shader.texture2=img
m2:draw()
debugMesh.texture =img
debugMesh:draw()
end
img0=img
FPS=FPS*.9+.1/DeltaTime
end
MoveShader = {
vertexShader = [[
//
// A basic vertex shader
//
//This is the current model * view * projection matrix
// Codea sets it automatically
uniform mat4 modelViewProjection;
//This is the current mesh vertex position, color and tex coord
// Set automatically
attribute vec4 position;
attribute vec4 color;
attribute vec2 texCoord;
//This is an output variable that will be passed to the fragment shader
varying lowp vec4 vColor;
varying highp vec2 vTexCoord;
void main()
{
//Pass the mesh color to the fragment shader
vColor = color;
vTexCoord = texCoord;
//Multiply the vertex position by our combined transform
gl_Position = modelViewProjection * position;
}
]],
fragmentShader = [[
//
// A basic fragment shader
//
//Default precision qualifier
precision highp float;
//This represents the current texture on the mesh
uniform lowp sampler2D texture;
uniform lowp sampler2D texture2;
//The interpolated vertex color for this fragment
varying lowp vec4 vColor;
//The interpolated texture coordinate for this fragment
varying highp vec2 vTexCoord;
void main()
{
//Sample the texture at the interpolated coordinate
lowp vec4 col = texture2D(texture,vTexCoord) ;
lowp vec4 col2 = texture2D(texture2,vTexCoord) ;
lowp float v1=col.r*.71+col.g*.21+col.b*.08;
lowp float v2=col2.r*.71+col.g*.21+col.b*.08;
//Set the output color to the texture color
if (abs(v1-v2)>0.2) gl_FragColor = vec4(0.,0.,1.,0.5)* vColor; //blue
else discard; //gl_FragColor = col* vColor;
}
]]}
Yeah, I had trouble with the camera not working for a couple of frames, had to put a while loop in to deal with that
Maybe we should report that as a bug
@Ignatz, tried your new code the FPS always stays above 50, and that’s awesome =D>. How’d you do that. If I just create a sprite using the camera my FPS drops to 30. Why does that happen??
I’m facing this weird problem! While working on getting a solution to increase the FPS in the motion detection program i found out that If I increase the size of the sprite using CAMERA then the frame rate drops significantly. Am I doing something wrong?? Just can’t understand.
Here’s the code.
--# Main
cameraSource(CAMERA_FRONT)
function setup()
print("keep sensitvity below camwidth")
timer = 0
X,Y,reds,greens,blues = {},{},{},{},{}
MoveX={}
XX=100
parameter.number("FPS",0,60,60)
parameter.integer("camwidth",10,500,10)
parameter.integer("sensitivity",5,50,5,changeSense)
end
function CreateImage()
img = image(camwidth,camwidth)
local w,h=img.width,img.height
setContext(img)
sprite(CAMERA,w/2,h/2,w,h)
setContext()
end
function draw()
background(40,40,50)
CreateImage()
if img~=nil then
sprite(img,WIDTH/2,HEIGHT/2,img.width,img.height)--in motion detection X,Y
MotionAndDirection(Left,Right)
end
pushStyle()
strokeWidth(5)
stroke(255, 255, 255, 255)
noFill()
ellipse(XX,HEIGHT-100,60)
popStyle()
FPS = .9*FPS+.1/DeltaTime
end
function Left()
XX=100
end
function Right()
XX=WIDTH-100
end
function changeSense()
X,Y,reds,greens,blues = {},{},{},{},{}
moving=false
end
--# MotionAndDirection
function MotionAndDirection(leftMove,rightMove)
local sense = sensitivity
if #reds==2*sense^2 then --pixels change reset
X,Y,reds,greens,blues = {},{},{},{},{}
moving=false
end
if not moving then
if timer<ElapsedTime-.5 then
if #MoveX>=2 then
if MoveX[1]>MoveX[#MoveX] then
rightMove()
MoveX={}
elseif MoveX[1]<MoveX[#MoveX] then
leftMove()
MoveX={}
else
MoveX={}
end
end
timer=ElapsedTime
sound(SOUND_PICKUP, 26546)
end
end
local accuracy = 40 --half the range of value for which colour will change
for j = 1,sense do
for i=1,sense do
red,green,blue = img:get(img.width/sense*i,img.height/sense*j)
table.insert(X,img.width/sense*i)
table.insert(Y,img.height/sense*j)
table.insert(reds,red)
table.insert(greens,green)
table.insert(blues,blue)
end
end
if #reds>sense^2 then
for i=1,sense^2 do
if reds[sense^2+i]>reds[i]+accuracy and greens[sense^2+i]>greens[i]+accuracy and --pixel change
blues[sense^2+i]>blues[i]+accuracy or
reds[sense^2+i]<reds[i]-accuracy and greens[sense^2+i]<greens[i]-accuracy and
blues[sense^2+i]<blues[i]-accuracy then
pushStyle()
fill(255,0, 0, 255)
ellipse(WIDTH/2-img.width/2+X[i],HEIGHT/2-img.height/2+Y[i],5)--make dots, from main X,Y
popStyle()
table.insert(MoveX,X[i])
moving=true
end
end
end
end
Presumably it’s because you’re drawing more pixels.
In answer to your previous question, I think the way I increased the frame rate was to analyse movement every N frames instead of every frame.
by drawing more pixels do you mean more pixels whose colour i heck has changed or not, then i think not. cause i used only 25 such pixels but yeah it could be cause i used to update the camera every frame not every X frames.