Can I somehow pass an array to a shader, or get acces to its contents?
Its an array of up to maybe 60 images and I want to read their pixel colours.
@Kirl Instead of passing 60 images to the shader, pass 1 at a time. If I’m not mistaken, you can’t get any info back from a shader. So if you read the pixel colors, what info are you after.
@Kirl To read information and retrieve data from shaders, look at the buffer in the reference book. Basically saying myMesh:buffer("variableName"), you can get a variable from a shader.
The array saves the last 60 frames from the camera and I make a new image by combining parts from those images. I have it working without a shader by copying regions and/or individual pixels from those images, but that’s terribly slow as expected.
So every frame I make one new image from all of those 60 images.
I was hoping I could maybe do everything inside of a shader to speed things up?
@CamelCoder Do you have a small example of using buffer to retrieve info from a shader. @Kirl What are you trying to do by making a new image from 60 images.
You can make some trippy effects by drawing parts of the screen later than others. I use a noise where white is the most recent frame and black 60 frames back, or from top to bottom where the top is now and the bottom 60 frames back.
It works, but it’s too slow.
So I’m not trying to read stuff out from the shader (although that sounds useful) but rather get that array of 60 images into the shader. If that’s at all possible.
@Kirl If I understand shaders correctly, which I may not, they work on one image at a time. You don’t send an array of images to it, you send 1 image to it. That doesn’t mean you can’t send 60 images to it one after the other. I’ll have to look thru my shader examples, but I might have something where i take an image and create a different image seperate from the first. If that’s the case, the 60 images can be sent one after the other and all of them can be use to create another seperate image. I’ll let you know what I find.
@Kirl I found the shader I was thinking of. I’m passing 2 images to it and any pixels that are different by an amount I choose, I turn them yellow. I use the camera and every 2 images gets sent to the shader to compare them. So if I can pass 2 images to the shader, maybe you can do 60. I’ll have to play with it more to see what happens if I try more images.
@Kirl Are you trying to do something like this. This uses the camera and each frame part of the image is moved down a section of the screen. You can change nbr to change the number of sections shown.
displayMode(FULLSCREEN)
function setup()
nbr=30
h=HEIGHT//nbr
imgTab={}
for z=1,nbr do
table.insert(imgTab,image(WIDTH,h))
end
end
function draw()
background(40, 40, 50)
collectgarbage()
for a,b in pairs(imgTab) do
sprite(b,WIDTH/2,HEIGHT-(h*a)+h/2)
end
img=image(CAMERA)
if img~=nil then
imgTab[1]=img:copy(0,HEIGHT-h,WIDTH,h)
for z=nbr,2,-1 do
imgTab[z]=imgTab[z-1]:copy(0,0,WIDTH,h)
end
end
fill(255)
text(1/DeltaTime//1,WIDTH/2,HEIGHT-30)
end
@Kirl Here’s an example of passing multiple images to a shader. The only problem is an error on the line m.shader.tbl trying to create a table. I don’t know shaders well enough to know why, so maybe someone else can fix it. The second version sends 3 seperate images to the shader. That one works.
This version sends a table to the shader, but causes an error.
function setup()
img1=readImage("Planet Cute:Enemy Bug")
img2=readImage("Planet Cute:Character Horn Girl")
img3=readImage("Planet Cute:Star")
m=mesh()
m:addRect(300,300,img1.width,img1.height)
m.shader=shader(Shader.vs,Shader.fs)
m.shader.tbl={img1,img2,img3} -- this line causes an error
end
function draw()
background(40, 40, 50)
m:draw()
end
Shader =
{ vs = [[
uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec2 texCoord;
varying highp vec2 vTexCoord;
void main()
{ vTexCoord = texCoord;
gl_Position = modelViewProjection * position;
}]],
fs = [[
uniform lowp sampler2D tbl[3]; // 3 = number of images
varying highp vec2 vTexCoord;
mediump vec4 col1;
void main()
{
col1=vec4(0.,0.,0.,0.);
for (lowp int z=0;z<3;z++) // 3 = number of images
col1 = col1 + texture2D( tbl[z], vTexCoord);
gl_FragColor = col1/3.; // 3 = number of images
}]]
}
This version sends 3 different images to the shader and combines them.
function setup()
img1=readImage("Planet Cute:Enemy Bug")
img2=readImage("Planet Cute:Character Horn Girl")
img3=readImage("Planet Cute:Star")
m=mesh()
m:addRect(300,300,img1.width,img1.height)
m.shader=shader(Shader.vs,Shader.fs)
m.shader.t1=img1
m.shader.t2=img2
m.shader.t3=img3
end
function draw()
background(40, 40, 50)
m:draw()
end
Shader =
{ vs = [[
uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec2 texCoord;
varying highp vec2 vTexCoord;
void main()
{ vTexCoord = texCoord;
gl_Position = modelViewProjection * position;
}]],
fs = [[
uniform lowp sampler2D t1;
uniform lowp sampler2D t2;
uniform lowp sampler2D t3;
varying highp vec2 vTexCoord;
mediump vec4 col1;
void main()
{
mediump vec4 col1 = texture2D( t1, vTexCoord);
mediump vec4 col2 = texture2D( t2, vTexCoord);
mediump vec4 col3 = texture2D( t3, vTexCoord);
gl_FragColor = (col1+col2+col3) / 3.;
}]]
}
thanks dave! I’ll look into your examples! =)
Below is an example of the kind of effects I’d like to do with shaders, the effect doen’t really work at these speeds. It can be a great effect with a static camera especially, anything that moves will deform while the rest of the scene appears normal.
-- Cam
-- Use this function to perform your initial setup
function setup()
cameraSource(CAMERA_BACK)
frames = {}
frameBuffNr = 30
parameter.boolean("toggleMode", true)
parameter.integer("frameBuffNr", 1, 60, frameBuffNr)
parameter.number("waves", 0, .1, .01)
pic = image(400,400) -- eventual resolution
end
function p2pNoise(pic1, pic2)
rx,ry = pic1.width/pic2.width, pic1.height/pic2.height
for i=1, pic2.width do
for j=1, pic2.height do
fi = (noise(i*waves, j*waves)+1)/2 * #frames //1 +1 -- frame index
c = color( frames[fi]:get(i*rx//1, j*ry//1) )
pic2:set(i,j, c)
end
end
sprite(pic, WIDTH/2, HEIGHT/2 ,WIDTH, HEIGHT)
end
function p2pScan(pic1, pic2)
steps = frameBuffNr
step = HEIGHT/steps
for i=1, HEIGHT, step do
local r = i/HEIGHT
fi = math.ceil(r * #frames) -- (math.sin(r * math.pi) +1)/2 * #frames //1 +1 --
il = frames[fi]:copy(1,math.ceil(r*pic1.height), pic1.width, math.ceil(pic1.height/steps))
sprite(il, WIDTH/2, i+math.ceil(pic1.height/steps)/2, WIDTH, step)
end
end
-- This function gets called once every frame
function draw()
-- This sets a dark background color
background(40, 40, 50)
-- Do your drawing here
collectgarbage()
while #frames > frameBuffNr do
table.remove(frames)
end
img = image(CAMERA)
if img ~= nil then
table.insert(frames, 1, img)
if toggleMode == true then p2pScan(img, pic)
else p2pNoise(img, pic) end
sprite(img, 100,100, 200,200)
end
text(1//DeltaTime, WIDTH/2,HEIGHT-50)
end
@Kirl Tried your latest code. Keeps crashing after a few seconds. I thought you needed collectgarbage(), but you already have it. Added a collectgarbage() inside the while loop and that seemed to fix the crashes. This runs at 0 to 5 frames/sec on my iPad Air.
EDIT: Still crashes, just takes longer for it to happen.
@Kirl Here an example using a shader. I take an image and randomly shift lines of 10 pixels back and forth. You can add code to move the lines more smoothly if you want. This runs a 6 FPS on my iPad Air.
supportedOrientations(PORTRAIT_ANY)
displayMode(FULLSCREEN)
function setup()
offset={}
m=mesh()
m:addRect(WIDTH/2,HEIGHT/2,WIDTH,HEIGHT)
m.texture=readImage("Cargo Bot:Startup Screen")
m.shader=shader(vShader,fShader)
end
function draw()
background(40, 40, 50)
for z=1,100 do
offset[z]=math.random()/50
end
m.shader.offset=offset
m:draw()
fill(255,0,0)
text(1/DeltaTime//1,WIDTH/2,HEIGHT-50)
end
vShader = [[
uniform mat4 modelViewProjection;
attribute vec4 position;
attribute vec2 texCoord;
varying highp vec2 vTexCoord;
void main()
{ vTexCoord = texCoord;
gl_Position = modelViewProjection * position;
} ]]
fShader = [[
uniform lowp sampler2D texture;
varying highp vec2 vTexCoord;
uniform lowp float offset[100];
lowp float xx;
void main()
{ lowp int d=0;
{ for (highp float z=0.; z<=1.; z=z+.01)
{ if (vTexCoord.y>z && vTexCoord.y<z+.01)
xx=offset[d];
d=d+1;
}
if (vTexCoord.x+xx>=1.0)
xx=xx-1.0;
}
gl_FragColor=texture2D( texture,vec2(vTexCoord.x+xx,vTexCoord.y));
} ]]
Thanks dave, I’ll play around with it.
BTW, if it crashes you have to lower the framebuffnr before running (this is the nr of camera frames it saves), I was surprised I managed save up to 30-60 images myself.
This reminds me of a related question I had; is it possible to set the resolution of the actual camera image? So that I can lower the size of the frames that I store in the array?
@Kirl As far as I know, you can’t change anything with the camera. But you can copy a section of the screen to a smaller size.
function setup()
end
function draw()
background(0)
collectgarbage()
img1=image(CAMERA)
if img1~=nil then
img=img1:copy(100,100,400,400)
sprite(img,WIDTH/2,HEIGHT/2)
end
end