Image to string (bytes) without save/read ? Possible but takes too much time : unusable.

Is there a way to get the string (bytes array) of an image (codeaimage) without using saveImage and read image with io ?

I want to be able to do :

local mime = require 'mime'
local stream

function setup()
  stream = image(WIDTH, HEIGHT)
end

function draw()
  setContext(stream)
  -- some drawings
  setContext()

  -- stream:asString() doesn't exist and will return the image as a string
  local encodedImage = mime.b64(stream:asString())
  -- ...
end

--[[
stream:asString() need to return the equivalent of
saveImage("Project:stream", stream)

local f = io.open(os:getenv("HOME") .. '/Documents/'..PROJECTNAME..'.codea/stream.png', 'r')
local streamAsString = f:read('*a')
f:close()

stream:asString() <=> streamAsString
]]--

Thanks in advance !

@HyroVitalyProtago You could use r,g,b,a=image:get(x,y). You would write r g b a values to the file. You also need to write the width and height values so you know the size of the picture when you read the file. I might be able to show an example if I have one otherwise I need to create it.

@HyroVitalyProtago Here’s an example to save the width, height and the r,g,b,a values of an image. I haven’t done anything to read it and create the image. You would read the file to get the width and height, then keep reading and use image:set to create the image.

function setup()
    img=readImage("Platformer Art:Block Grass")
    w=img.width
    h=img.height
    createFile()
end

function createFile()
    file = os.getenv("HOME").."/Documents/Dropbox.assets/test.txt"
    f=io.open(file,"w") -- open file and delete anything in it.
    f=io.open(file,"a+")    -- open file for append
    f:write("("..w..","..h..")")    -- write the width and height
    for x=1,w do
        for y=1,h do
            r,g,b,a=img:get(x,y)
            str=string.format("(%f,%f,%f,%f)",r,g,b,a)
            f:write(str)    -- append the r,g,b,a values (r,g,b,a)
        end
    end
    f:close()   -- close the file
end

@HyroVitalyProtago In my above code, I didn’t take into consideration that a retina screen uses 1/2 pixel positions.

@HyroVitalyProtago Here’s some code that will create an image file and read it back. It accounts for retina images. I don’t like the imageCreate function, maybe I’ll redo it later.

function setup()
    img=readImage("Platformer Art:Block Special")
    w=img.rawWidth
    h=img.rawHeight
    
    createFile()
    readFile()
    createImage()
end

function draw()
    background(0)
    sprite(img,WIDTH/2,HEIGHT/4)
    if img1~=nil then
        sprite(img1,WIDTH/2,HEIGHT/2)
    end    
end

function createFile()
    print("create file")
    file = os.getenv("HOME").."/Documents/Dropbox.assets/test.txt"
    f=io.open(file,"w") -- open file and delete anything in it.
    f=io.open(file,"a+")    -- open file for append
    f:write("("..w..","..h..")")    -- write the width and height
    for x=1,w do
        for y=1,h do
            r,g,b,a=img:rawGet(x,y)
            str=string.format("(%f,%f,%f,%f)",r,g,b,a)
            f:write(str)    -- append the r,g,b,a values
        end
    end
    f:close()   -- close the file
    print("create file done")
end

function readFile()
    print("read file")
    file = os.getenv("HOME").."/Documents/Dropbox.assets/test.txt"
    a=io.open(file,"r")
    fil=a:read("a")
    a:close()   -- close the file
    print"read file done"
end

function createImage()
    print("create image")
    tab={}
    xx=""
    w,h=nil,nil
    for z=1,#fil do
        v=string.sub(fil,z,z)
        if v=="(" then
            r,g,b,a=nil,nil,nil,nil
            xx=""
        elseif v=="," or v==")" then
            val=tonumber(xx)
            if w==nil then
                w=val
            elseif h==nil then
                h=val
            elseif r==nil then
                r=val
            elseif g==nil then
                g=val
            elseif b==nil then
                b=val
            elseif a==nil then
                a=val
                table.insert(tab,vec4(r,g,b,a))               
            end
            xx=""
        else
            xx=xx..v
        end
    end
    img1=image(w,h)
    cnt=0
    for x=1,w do
        for y=1,h do
            cnt=cnt+1
            img1:rawSet(x,y,tab[cnt].x,tab[cnt].y,tab[cnt].z,tab[cnt].w)            
        end
    end
    print("create image done")
end

@dave1707 thanks for this responsiveness! But maybe I wasn’t clear on my goal.

  • I have an image generated at the beginning (the setup thing)
  • At each update, I override it with current drawings captured with setContext
  • At the end of each update, I want to be able to have the image as a string to encode it (with base64 for example) and send it over the network

My problem is that I don’t want to save/read my image because it takes too long. IO operations cost a lot.

I haven’t tried yet, but I think it will be also not enough fast to iterate on each pixel to write it into a string, and I don’t think it will produce the same string as the one returned when we read the image file because it’s not the same encoding.

For the moment, I’ve something very similar to the last thing posted here : https://codea.io/talk/discussion/8503/best-way-to-upload-an-image

But because I want to stream my screen, write and read images takes too long just to get the image as a string.

In other words, how to do this without saving/reading a file? Is there a way to access the underlying structure of a codeaimage? I want a kind of saveImage into a string, not a file.

local mime = require 'mime'
function imageToBase64(img)
    saveImage("Documents:tempImage", img)
    local file = io.open(os.getenv("HOME").."/Documents/tempImage.png","r")
    local content = file:read("*a")
    file:close()
    return mime.b64(content)
end

@HyroVitalyProtago I guess I don’t inderstand what you’re trying to do. So here’s another attempt. The function createStr() will take whatever image you have and convert it to a string. You just need to pass it the width, height and image name. What I have in setup is just my example, you’ll have to pass your image info. The width, height and r,g,b,a values will be in the string str separated by commas. You can then do what you want with the string. Does this help any or am I still clueless.

function setup()
    pic=readImage("Platformer Art:Block Grass")
    wid=pic.rawWidth
    hei=pic.rawHeight
    createStr(wid,hei,pic)
end

function createStr(w,h,img)
    tab={}
    str=""
    table.insert(tab,string.format("%d,%d,",w,h))
    for x=1,w do
        for y=1,h do
            r,g,b,a=img:rawGet(x,y)
            w=string.format("%d,%d,%d,%d,",r,g,b,a)
            table.insert(tab,w)
        end
    end
    str=table.concat(tab)
end

@HyroVitalyProtago I think you want the output string to contain the hex values from 00 to ff. Then you encode it with base64 and send it somewhere. So if a pixel has the values of 255 128 255 255 then the string would contain ff 80 ff ff . Is that what you’re after, a string of hex values. If so then I’ll modify the above code.

@dave1707 Thanks again for the response. But I want a string output equivalent to read file of saved image

function setup()
  pic = readImage("Platformer Art:Block Grass")
  assert(imageToString1(pic) == imageToString2(pic)) -- I don't think this is true, but I haven't tried on my ipad yet : my goal is a imageToString1 that return the same thing that imageToString2 without using IO
end

-- @todo find a function that satisfied the assert predicate in setup
function imageToString1(img)
    local w,h = img.rawWidth, img.rawHeight
    tab={}
    str=""
    table.insert(tab,string.format("%d,%d,",w,h))
    for x=1,w do
        for y=1,h do
            r,g,b,a=img:rawGet(x,y)
            w=string.format("%d,%d,%d,%d,",r,g,b,a)
            table.insert(tab,w)
        end
    end
    str=table.concat(tab)
end

function imageToString2(img)
  saveImage("Documents:tempImage", img)
  local file = io.open(os.getenv("HOME").."/Documents/tempImage.png","r")
  local content = file:read("*a")
  file:close()
  return content
end

@HyroVitalyProtago From what you show above, you want to create a string of an image that would be the same as the file created by saveImage. If that’s what you want, then you’re wasting your time. saveImage creates a file in png file format. In the function imageToString1(), after you created the string str using rawGet, then you would have to convert that string to a string in png file format. So basically you would be doing a lot of processing on your own that would most likely take a lot longer than just doing the saveImage and io. I’ll keep looking into this, but I don’t think there’s a better way.

@dave1707 That’s it. Thanks for your response. If you found anything, don’t hesitate!

@HyroVitalyProtago Something else I didn’t mention is the png file is also compressed. That means you would not only have to write (find) code to convert it to a png file, you would also have to write (find) code to compress the data.

Just out of curiosity, I’ve tried this lib : https://github.com/wyozi/lua-pngencoder. I didn’t even try to find out if the encoding was good. It really takes a lot longer than with IO.

@dave1707 true! So I will tag this discussion as closed. If the encoding already takes too long, I can’t even imagine adding compression. In any case, thanks again for the support!