I was doing some simple tests of using the camera to capture motion (writing a little program that measures head-bob as a means of tracking running speed). But using the camera to generate images for analysis was causing frequent crashes.
Bring it down to its essence, and it fails within twenty seconds. Looking at the documents, the slowness of this operation is noted, and yes, I’m doing here exactly what the help docs say don’t do, so in a sense, the bug is mine.
But it makes little sense to me why this is sure doom.
-- ImageCameraTest
function setup()
cameraSource(CAMERA_FRONT)
end
function draw()
background(40, 40, 50)
img = image(CAMERA)
end
While this is not…
function setup()
cameraSource(CAMERA_FRONT)
end
function draw()
background(40, 40, 50)
sprite(CAMERA)
end
Is there a way to capture camera pixels in a way that lets me query them quickly?
@Mark I don’t think img=image(Camera) is meant to do it 60 times per second by having it in draw. Try this, tap the screen to capture an image. You could set a timer to do it automatically.
function setup()
cameraSource(CAMERA_BACK)
print"keep tapping screen to capture image."
end
function draw()
background(40, 40, 50)
if img~=nil then
sprite(img,WIDTH/2,HEIGHT/2)
else
sprite(CAMERA,WIDTH/2,HEIGHT/2)
end
end
function touched(t)
if t.state==BEGAN then
img=image(CAMERA)
end
end
You’re creating a full-sized image 60 times a second (or thereabouts). Even though you’re reusing the img variable, the image itself won’t get removed from memory until the next garbage collection, so the images will stack up in memory and you’re reaching saturation.
Instead of creating a new image each time, try reusing old images. Something like:
@Mark, i think garbage collector is just not keeping up with the images you are creating every frame. You should probably only keep a small number of latest frames - just enough for motion analysis, and discard the rest, by forcing garbage collection periodically.
For example, here, only 10 last captures are kept in the table, and i’m calling collectgarbage() explicitly to recover the memory:
--# Main
-- ImageCameraTest
frames = {}
head = 0
N = 10
function setup()
cameraSource(CAMERA_FRONT)
end
function draw()
background(40, 40, 50)
-- keep N last frames
frames[head + 1] = image(CAMERA)
head = (head + 1) % N
-- periodically force garbage collection
if head == 0 then
collectgarbage()
end
end
@Mark I wrote a little program that compares one image to the next and they weren’t the same. I have the camera in draw(), so the 2 images are 1/60 second apart. Even if I cover the camera so both images are black, comparing the pixels of the first image and the second image result in a difference. I wonder if there should be a percentage of difference before the images are considered different.
I did this a while back, and took pixel samples, eg every 50th pixel across and down. But that was to detect movements of a uniformly colored object in my hand.
If you’re outside, just about every pixel is going to change colour at every frame, and the complexity of real life scenery is going to make tracking orientation extremely difficult.
There are plenty of apps that track running, I wonder how they do it.
@Mark If you want to track running speed, have you looked at the location functions. There’s location.speed which measures speed in meters per second. I haven’t tried them because my speed on the couch is usually 0.
EDIT: I guess the runner would have to be carrying the device though. So forget this option.
@dave1707 and @Ignatz. I use a combination of location and accelerometer for outside running, I’m looking at the camera for a very specific purpose – treadmills. I’m thinking the user props the iPad or iPhone somewhere so that they can see the screen, then, using the front camera, I measure points scattered over the screen in a grid. I look for pixels being “hidden” and “revealed” to come up with a steps-per-minute pace and turn that into a (rough) approximation of pace.
Of course, the runner on the treadmill knows how fast they are going, but that’s okay. I’m not trying to provide feedback in mph. I’m trying to move their avatar along a path.
@Mark I wonder if this might be better handled by a shader? The image analysis could happen in there (where it has no problem analysing the camera stream at full speed). The output of the shader could be a single colour indicating the height of the head, and it would be fairly cheap to read the single colour out for use in analysis.