Downloading and Uploading Gists

This program allows to easily download a Github gist and turn it into a Codea project. You can also do the reverse operation: take one of your Codea projects, and upload it as a new anonymous gist.

Special thanks to:

  • @Jmv38, who provided feedback for the initial version, pointed out the need for tab-splitting, and also suggested the program flow for the upload operation
  • @HyroVitalyProtago, for the idea of automatic downloading of Dkjson library

--# Main
-- Gist uploader/downloader
-- v2.0 by juce, Jmv38 and HyroVitalyProtago

local function iter(obj)
    if not obj.name then
        return
    end
    local data = obj.data
    local name, pos = obj.name, obj.pos
    local s, e, next = data:find("\
[-][-]# ([%w_]+)[^\
]*\
.", pos)
    obj.name, obj.pos = next, e
    return name, data:sub(pos, s and s-1 or nil)
end

local function tabs(data)
    local s, e, name = data:find("^%s*[-][-]# ([%w_]+)[^\
]*\
.")
    local obj = {
        pos = e or 1, 
        name = name or "Main",
        data = data,
    }
    return iter, obj
end

function saveLink(link)
    local name = "myGists"
    local tab = ""
    for i,v in ipairs(listProjectTabs()) do
        if name == v then tab = readProjectTab(v) end
    end
    tab = tab .. "-- " .. link .. "  -- project: \
"
    saveProjectTab(name,tab)
end

function getDkjson(cb)
    if not readGlobalData("Dkjson") then
        status, msg = "", "Downloading dkjson..."
        http.request("https://gist.githubusercontent.com" ..
            "/HyroVitalyProtago/5965767/raw/" ..
            "73facb82eda4c92393c51535f8dd08728e25555d/Dkjson.lua",
            function(data)
                saveGlobalData("Dkjson", data)
                msg = ""
                cb()
            end)
    else
        cb()
    end
end

function setup()
    colors = {
        red = color(177, 49, 49, 255),
        yellow = color(203, 209, 60, 255),
        green = color(96, 181, 47, 255)
    }
    url, c = "", colors.yellow
    
    -- load dkjson library (download, if necessary)
    getDkjson(function()
        assert(loadstring(readGlobalData("Dkjson")))()
        assert(json.encode)
        assert(json.decode)
        -- display button menu
        menu()
    end)
end

function menu()
    -- Download gist via link in pasteboard
    parameter.action("DOWNLOAD: Paste gist link", function()
        url = pasteboard.text
        parameter.clear()
        parameter.action("Download gist", function()
            if not url:match("/raw") then
                url = url .. "/raw"
            end
            http.request(url, function(data)
                msg = "Downloaded. Splitting into tabs ..."
                tween.delay(1, function()
                    saveProjectTab("myGists", nil)
                    for tabname,tabdata in tabs(data) do
                        saveProjectTab(tabname, tabdata)
                    end
                    msg, c = "Success!", colors.green
                    parameter.action("Quit", function()
                        close()
                    end)
                end)
            end, function(err)
                msg, c = err, colors.red
                parameter.action("Quit", function()
                    close()
                end)
            end)
        end)
    end)

    -- Upload data from pasteboard to gist
    parameter.action("UPLOAD: Create new gist", function()
        local data = {
            description = 'Gists Codea Upload',
            public = true,
            files = {
                ['Project.lua'] = {
                    content = pasteboard.text
                }
            }
        }
        parameter.clear()
        msg = "Starting upload ..."
        http.request('https://api.github.com/gists', function(res)
            local link = "https://gist.github.com/anonymous/" .. json.decode(res).id
            msg, c = "Success!\
" .. link, colors.green
            pasteboard.copy(link)
            saveLink(link)
            print("link copied in the pasteboard and in tab myGists")
            parameter.action("View gist", function()
                openURL(link, true)
            end)
            parameter.action("Quit", function()
                close()
            end)
        end, function(err)
            msg, c = err, colors.red
            parameter.action("Quit", function()
                close()
            end)
        end,
        { method = 'POST', data = json.encode(data) })
    end)    
end

function draw()
    background(18, 18, 19, 255)
    fill(c)
    textWrapWidth(WIDTH-100)
    fontSize(35)
    if msg then
        text(msg, WIDTH/2, HEIGHT/2)
    else
        text(url, WIDTH/2, HEIGHT/2)
    end
end

How to use it:

  • Create a project named “Gist” and paste the code above into its Main tab. (you only need to do this step once really)

When you want to download a gist:

  1. Copy the gist link. (For example: this awesome xfc library by jmv38 and others: https://gist.github.com/anonymous/3017b8ab57a5649ad34f )
  2. In Codea, find the “Gist” project, and duplicate it. Give it a name that is relevant to the code that you are about to download.
  3. Run that duplicated project, and push “DOWNLOAD: Paste gist link” button. The link that you copied into pasteboard (in step 1) should be displayed. Push “Download gist” button to start downloading. Once download is complete, and you see “Success!” message, hit the “Quit” button, and now your duplicated project should have the downloaded code.

When you want to upload a Codea project as a new gist:

  1. Long-press on the project button in Codea, the select “Copy” option from the menu that appears
  2. Run “Gist” project, push “UPLOAD: Create new gist” button.
  3. Once upload is complete, it will display the gist link and copy it to pasteboard. You can then “View gist” or “Quit”.

NOTE that the code above is the stable version (v2.0). It already implements the suggestions and feautures that are discussed further down.

The very first version, which started the discussion below, can be found here:
https://gist.github.com/anonymous/2dd7e010ec19c5ece2fd

The user and all related content has been deleted.

@juce nice job! However, there stil a bit work needed: you must split the downloaded code in separate tabs and create them. Tabs can be found in the code because they are split with following syntax: ‘–# tabname’. In the example of my library you must split the tabs because:

  • the tab ‘data’ must be separate.
  • the tab ‘main’ must be separate from other tabs, otherwise you cant use the library with a dependency (the main tab is ignored).
    Thanks!

@juce here is your code where i have added a tab splitter:
Note that the ‘main’ tab always remain in first position, so this must be compatible with your code (it is, unless you put code outside the function that use code defined in the previous tabs)


--# Main
-- gistLoad

-- Gist downloader

function splitAnsSaveProjectTabs(str)
    print("---- installing new version ---")
    local a,b,c,d,tabName,tabContent
    local plainText = true
    local start = 0
    local tabMarker = "--".."# " -- this string cannot appear in the code!
    repeat
        a,b = str:find(tabMarker,start+1,plainText)
        c,d = str:find("\
",b+1,plainText)
        tabName = str:sub(b+1,c-1)
        local e,f = str:find(tabMarker,d+1,plainText)
        if e then e = e -1 end
        tabContent = str:sub(d+1,e)
        saveProjectTab(tabName, tabContent)
        print( tabName .. " : installed"  )
        start = e
    until start == nil
end

function setup()
    url, c = "", color(203, 209, 60, 255)
    parameter.action("Paste gist url", function()
        url = pasteboard.text
        parameter.action("Download", function()
            if not url:match("/raw") then
                url = url .. "/raw"
            end
            http.request(url, function(data)
                splitAnsSaveProjectTabs(data)
                msg, c = "Success!", color(96, 181, 47, 255)
                parameter.action("Quit", function()
                    close()
                end)
            end, function(err)
                msg, c = err, color(177, 49, 49, 255)
                parameter.action("Quit", function()
                    close()
                end)
            end)
        end)
    end)
end

function draw()
    background(18, 18, 19, 255)
    fill(c)
    textWrapWidth(WIDTH-100)
    fontSize(35)
    if msg then
        text(msg, WIDTH/2, HEIGHT/2)
    else
        text(url, WIDTH/2, HEIGHT/2)
    end
end

@Jmv38, thanks for the feedback and the code. My version with tab-splitting is below (i was assuming the magic “–# TABNAME” splitter had to be at the beginning of a line, but maybe that’s not true…) In terms of speed, both versions feel about the same.


--# Main
-- Gist downloader

local function iter(obj)
    if not obj.name then
        return
    end
    local data = obj.data
    local name, pos = obj.name, obj.pos
    local s, e, next = data:find("\
[-][-]# ([%w_]+)[^\
]*\
.", pos)
    obj.name, obj.pos = next, e
    return name, data:sub(pos, s)
end

function tabs(data)
    local s, e, name = data:find("^%s*[-][-]# ([%w_]+)[^\
]*\
.")
    local obj = {
        pos = e or 1, 
        name = name or "Main",
        data = data,
    }
    return iter, obj
end

function setup()
    url, c = "", color(203, 209, 60, 255)
    parameter.action("Paste gist url", function()
        url = pasteboard.text
        parameter.action("Download", function()
            if not url:match("/raw") then
                url = url .. "/raw"
            end
            http.request(url, function(data)
                msg = "Downloaded. Splitting into tabs ..."
                tween.delay(1, function()
                    for tabname,tabdata in tabs(data) do
                        saveProjectTab(tabname, tabdata)
                    end
                    msg, c = "Success!", color(96, 181, 47, 255)
                    parameter.action("Quit", function()
                        close()
                    end)
                end)
            end, function(err)
                msg, c = err, color(177, 49, 49, 255)
                parameter.action("Quit", function()
                    close()
                end)
            end)
        end)
    end)
end

function draw()
    background(18, 18, 19, 255)
    fill(c)
    textWrapWidth(WIDTH-100)
    fontSize(35)
    if msg then
        text(msg, WIDTH/2, HEIGHT/2)
    else
        text(url, WIDTH/2, HEIGHT/2)
    end
end

@juce great! I thought you needed a little help to split the tabs, but clearly you didn’t!

@juce since you master the subject, let me challenge you again :wink: : could you add a button ‘create gist’ to your program? What i mean is:

  • the pasteboard may contained a copyied project. (i prefer this vs entering the project name).
  • taping the button :
    1/creates the anonymous gist (author gist is too much fuss, i think).
    2/copies the gist adress in the pasteboard (so i can paste it directly where i want, in the forum for instance).
    That would be great!

Ha… Well, i certainly needed the fedback :slight_smile: Made me go and read the wiki page about Codea project format, which i didn’t know about. Appreciate you posting your version of the splitting too - it is always cool to see different ways to do the same task. In fact, i find yours rather easier to read than mine.

@Jmv38, sure, this sounds like a useful feature. Will certainly do. (Although, probably in the morning - it’s almost 2am my time now…)

cant wait to see your achievement!

So, here it is. Works, but i had to cheat a little bit


--# Main
-- Gist uploader/downloader

local function iter(obj)
    if not obj.name then
        return
    end
    local data = obj.data
    local name, pos = obj.name, obj.pos
    local s, e, next = data:find("\
[-][-]# ([%w_]+)[^\
]*\
.", pos)
    obj.name, obj.pos = next, e
    return name, data:sub(pos, s)
end

function tabs(data)
    local s, e, name = data:find("^%s*[-][-]# ([%w_]+)[^\
]*\
.")
    local obj = {
        pos = e or 1, 
        name = name or "Main",
        data = data,
    }
    return iter, obj
end

function setup()
    url, c = "", color(203, 209, 60, 255)
    -- Download gist via link in pasteboard
    parameter.action("Paste gist url", function()
        url = pasteboard.text
        parameter.clear()
        parameter.action("Download", function()
            if not url:match("/raw") then
                url = url .. "/raw"
            end
            http.request(url, function(data)
                msg = "Downloaded. Splitting into tabs ..."
                tween.delay(1, function()
                    for tabname,tabdata in tabs(data) do
                        saveProjectTab(tabname, tabdata)
                    end
                    msg, c = "Success!", color(96, 181, 47, 255)
                    parameter.action("Quit", function()
                        close()
                    end)
                end)
            end, function(err)
                msg, c = err, color(177, 49, 49, 255)
                parameter.action("Quit", function()
                    close()
                end)
            end)
        end)
    end)
    -- Upload data from pasteboard to gist
    parameter.action("Upload new gist", function()
        data = pasteboard.text
        parameter.clear()
        http.request('http://gist-proxy.aws.mapote.com:8888/gists', function(link)
            msg, c = "Success!\
" .. link, color(96, 181, 47, 255)
            parameter.action("Copy link", function()
                pasteboard.copy(link)
                parameter.action("Quit", function()
                    close()
                end)
            end)
        end, function(err)
            msg, c = err, color(177, 49, 49, 255)
            parameter.action("Quit", function()
                close()
            end)
        end,
        { method = 'POST', data = data })
    end)
end

function draw()
    background(18, 18, 19, 255)
    fill(c)
    textWrapWidth(WIDTH-100)
    fontSize(35)
    if msg then
        text(msg, WIDTH/2, HEIGHT/2)
    else
        text(url, WIDTH/2, HEIGHT/2)
    end
end

The cheating part is that instead of speaking directly to Github Gist API, i go via a small proxy service, which i just wrote. The reason being that github api uses JSON for requests and responses, but Codea doesn’t have native JSON support. So i made a simple web service, which creates new gists when you POST something to it.

Alternatively, I could’ve used one of the pure-Lua JSON parsers out there, but i’m yet to find one with acceptable performance for large documents…

@Juce you are amazing!
Have you seen this. json lib included in several projects posted here?
here it is: https://gist.github.com/anonymous/95cb507e0d8ca45c74ad
this is an old program ‘autogist’. Ive just created the gist with your program… lol!

@Jmv38, funny, i also just downloaded your gist. So i guess it kinda works :wink:

The json lib in Autogist - i don’t think i saw that one before. Thanks for the link! I’ll give it a try with some big documents, perhaps it is better than the ones i tried in the past…

(The ideal way to solve it would be to add native json.encode and json.decode to Codea 2.x. That would be awesome, actually… Should be trivial for TLL folks to do, and plenty of choices for a fast JSON parser written in C)

@juce you know, i think there shouldnt be any problem with big strings: lua is very optimized for string processing.

@juce i think your program is now really the best tool around to share code the easiest way possible!
The idea of just duplicating your project to the desired name and then just running it… it is brilliant by its simplicity!

@juce, i agree with @Jmv38, it is brilliant

Thank you guys. Glad it’s useful.

@juce - if it’s finished, to make it easy for new people to use it, could you please write up final instructions for installing and using it, along with the final code and a simple example of use?

What a lot of people do is to put all this in the original post at the top, so it always has the latest code, instructions etc.

@Ignatz, good idea. Thanks.
I updated the first post with the final code, and more detailed instructions on how to use it.
The final code is also available here:

Gist uploader/downloader v1.0:
https://gist.github.com/anonymous/ca31311e08e68cbed8ae

=D>