Codea-SCM: basic source control for your projects

Hi all. This is something I had initially built for myself, but after some polishing and refactoring it seems to be ready to be used by others (if there’s interest).

The basic idea is this:
You can version/source control your Codea projects by pushing the code to remote repositories on hosting sites, such as Github, Bitbucket, Gitlab, and so for. There are 2 parts to the system: a Codea app that you would run in your Codea workspace on iPad, and a web service that runs on my server at https://codea-scm.aws.mapote.com , which does the communications with repo hosting sites, and basically acts as a proxy for your source control actions.

The web site has install instructions and more detailed info, which is recommended to take a look at:
https://codea-scm.aws.mapote.com/

But if you are eager to get started, without reading any further, here’s the bootstrap installer for you:

--# Main
-- codea-scm bootstrap installer

function setup()
    displayMode(STANDARD)
    print("Installing codea-scm ...")
    http.request("https://codea-scm.aws.mapote.com/install",
        function (data, status, headers)
            assert(loadstring(data))()
            print("Installation complete. SUCCESS!")
            print("Launch the project again to start using codea-scm.")
            tween.delay(2.0, close)
        end,
        function (err)
            print("PROBLEM downloading: " .. err)
        end)
end

function draw()
    background(37, 38, 50, 255)
end

Possible uses:

  • versioning and backups:
    We are using true source control system, so yep we get the versioning from it. Backups are taken care off somewhat indirectly - by using remote repositories hosted at GitHub and other well known sites, (Presumably, they look after their storage systems and back them up regularly :slight_smile: )

  • coding on both iPad and your laptop/desktop:
    Some types of changes are just much easier to do on a computer. Vim, full keyboard, etc… You can do that, then commit and push your changes to the repository, then pull them to the iPad - to continue working there.

  • code sharing:
    The nice thing about repo hosting sites - such as GitHub and Bitbucket - they already have the infrastructure built around users and projects, and the logic and the UI all figured out. By hosting your repositories on one of those sites, you can leverage their access control mechanisms to share your Codea projects with just friends or everybody (or no one, if you don’t want to).

Limitations:

  • Right now only git is supported, although i tried to keep the web service and the app generic enough, so that other scm’s can be added in the future, if needed/wanted. For instance, Mercurial (Hg) and Subversion should not be hard to add support for.

  • You are limited to “Diff”, “Push”, and “Pull” operations. Anything more complex, you’d still need to do on your laptop/desktop. (Which is probably better kept that way anyway)

  • Only your source tabs are pushed/pulled. Assets, such as sprites, are not versioned. I still need to figure out a way to handle those.

###P.S.
Codea-SCM works on Codea 1.5.5 and Codea 2.0. However, with 2.0, there is a caveat to be aware of - related to http.request. It seems that in Codea 2.0, if you restart the project, http.request stops working (all requests just hang). Since Codea-SCM uses http.request quite a bit, the workaround is: don’t restart the project, simply close it, and then press the “play” button again. Then all should be good.

Very cool! I’ll have to try it out when I have a chance.

How does this differ from Codea Community?

@juce thanks for sharing! Same question as jakAttak?
ps: welcome on board!

@JakAttak, @Jmv38 - While I have the same question, I do think it’s good to have alternatives to everything.

Guys, thanks for comments!

About Codea Community and how this is different (EDITED for clarity):

If i’m not mistaken, Codea Community is more about code sharing and community aspects of it, while my project is focusing primarily on source control, delegating other things - such as account management, sharing, access control - to the remote repository hosting sites (Github, Bitbucket, etc.).

Both can coexist happily, i think.

in case anyone actually cares… just a quick update: the Codea-SCM web server now has updated OpenSSL library and a new re-keyed SSL certificate, so that it is not vulnerable to the Heartbleed bug

@juce First of all, THANK YOU so much for creating this! I’ve been experimenting with the various methods of backing up Codea projects, but Codea-SCM is so far my favourite. It took a bit of time to work out how to get Bitbucket to play with SSH keys, but once that was sorted, this was very easy to use.

@JakAttak @Jmv38 @Zoyt I would say that the main difference from Codea community is that using a Git service gives you industry-standard version control.

For others who are reading who might not be familiar with version control (I’m fairly new to it too)

  1. make a change to your code.

  2. run the Codea-SCM programme

  3. it compares the code in Codea with the remote copy online. If there are differences, the circle next to the project name turns blue.

  4. If you press the project name, you will then see all of the differences in the app. I’m really impressed that you can see the diffs in-app! You see your code, with lines you’ve added in green, deletions in red (I don’t think CodeaCommunity supports diffs?)

  5. Press push to send your revisions up to your online repo, add a comment about what’s changed

  6. You can then browse all of the changes you’ve made over time on the page of your client.

  7. If your repo is synced to a client on your computer, they’ll also automatically appear there. (Is there any way to retrieve Codea Community projects from outside Codea?)

  8. If you need to restore, press pull in the Codea-SCM app, and choose from every previous version you’ve pushed.

Regarding the various websites that offer GIT, Github is good if want to share everything (you need a paid account to have a private space), and I recommend BitBucket if you want private space for free.

I love CodeaCommunity too, and consider it an absolute essential for sharing and learning, which should be the first thing all new Codea users install. But for large projects, standardised cross-platform versioning and backup is really important, and Codea-SCM is IMO an incredibly useful and easy to use implementation of that.

Thank you to all the people who make these awesome tools.

@yojimbo2000, thanks for kind words.
I’m glad someone finds this useful. I haven’t been here for a while, so only noticed your comment now. Was thinking lately if i should renew the SSL cert for the site or let it expire… I guess i’ll renew for another year. :slight_smile:

@juce Can you see usage stats for the site, is it popular? It deserves to be. I think it’s a fantastic service, thank you for creating it. I love being able to see my Codea projects as repositories on my Mac with full versioning etc.

On another thread someone said that they were uncomfortable with having a third party site as the conduit between Codea and GitHub/BitBucket etc, and I suppose that’s a legitimate concern. How secure would you say Codea-SCM is?

@yojimbo2000, it is not popular :slight_smile: Which kinda makes sense, because there isn’t really a whole lot of people that need it. 8 different folks have used the system since July 2014.

About security: yes, i can certainly understand the concern of using a third party site. It is something that people think about, and in fact, it is something they should always be thinking about. I have a small FAQ section on the web site, which talks about security:
https://codea-scm.aws.mapote.com/#Security

To summarize here: i would say Codea-SCM is probably as secure as it can be given the nature of what it is doing. First key moment is that a person’s username/password on Github/Bitbucket is never used, and is not known to Codea-SCM at any moment in time, so there’s no way it can compromise it. Instead, SSH keys are used, which are always disposable: you can delete them at any time, and revoke access to a particular keypair in Github/Bitbucket account settings. Perhaps the easiest way to visualize how Codea-SCM works is this:

If iPad had a git client and an SSH client, then theoretically, we could do a direct SSH connection:

Codea app on iPad <—SSH—> Github

In that case your SSH private key stays on iPad and never leaves it. That’s the ideal scenario, but at the moment it’s not possible.

So, what Codea-SCM doing instead is this:

Codea app on iPad <----HTTPS—> Codea-SCM web service <—SSH—> Github

Your private SSH key travels from iPad to Codea-SCM web service via secure HTTPS connection, and is then used to establish an SSH connection to Github. The private key is only briefly stored on disk while connection to Github is established, and then is immediately deleted.

Anyways, this is getting too long, but perhaps my main point is that it is secure, in terms of communication channels. Information travelling between your iPad and Github/Bitbucket is always protected by encryption via HTTPS and SSH. Of course, if the Codea-SCM server is hacked and broken into, then the keypairs can be watched and stolen (provided that the attacker gains root access), but that’s true about pretty much any service :). The good news in that case is that because these keypairs are disposable SSH keys, it is not as bad as if, for example, Codea-SCM had kept real usernames and passwords.

I forgot to mention that if anyone wants to run their own copy of Codea-SCM web service locally - that’s pretty easy to do an any Linux machine or a Mac. This way you eliminate the “third party site” concern completely.

@juce thank you for answering my question in such a detailed way. Personally I find this the best back-up and source control service for Codea, thanks for making it!

What you’re saying about running it locally is also very interesting, are there instructions somewhere for that?

@yojimbo2000, you’re very welcome. i’m glad there is a real user of the system, other than myself :wink:

truth be told, there aren’t any instructions for running the web service locally. The code will need some clean-up work (it’s also Lua, btw) and writing some minimum docs, and uploading to Github, which i was thinking of doing anyway… but haven’t gotten to yet.

@juce you’ve made a great tool, I just don’t think many Codea users are that deep in to programming that they would need version control. I will try to get it set up when I’m not bed ridden ill as it looks very useful for keeping up to date back ups.

Of course people need version control, they just don’t know it yet :stuck_out_tongue:

Lots of people ask about backup, and version control is the best form of backup

@yojimbo2000 and everyone else who’s using this:
i fixed a bug in Serializer code. The problem was that if the code in one of your tabs had repeated closing brackets - like for example:

a[b[i]]

then that tab would be incorrectly truncated when pushed to git repository.

I would recommend upgrading to v1.1. This is how to do it:

  1. delete your Codea-SCM project. (Node that this will not actually delete the project’s local data, which is where the names and the git-remotes for the “linked” projects are stored, so you won’t lose those)
  2. copy the bootstrap code from the first post above, long-press “Add New Project” and choose “Paste Into Project”. Name the new project with the same name as you had for the one you deleted in step 1.
  3. Run it. It will download the updated version and then stop.
  4. Done. Now, when you run it again, you should see the “v1.1” in the right bottom corner. All your “linked” projects should also be there, as before.

Thanks for that @juce , upgrade was very easy, now on 1.1. I’ve never seen that double square bracket bug though (and I have lots of shaders in double brackets).

And going back to your earlier post, I’d be interested in seeing the server code, if you do have time to post it. Someone could set up a mirror if you ever decided not to renew the licence

@yojimbo2000, yep, i will get the server code ready and available. Hopefully, sometime this week.

about the bug: i should clarify: it was only happening when you had closing double brackets somewhere in the code ( ]] ), but not the opening ones. If there were also two opening brackets ( [[ ) anywhere in that tab, then the Serializer worked correctly.

@yojimbo2000,

The server code is now available here:

https://bitbucket.org/juce/codea-scm

Hopefully, things are more or less self-explanatory, but if you decide to set it up locally and run into problems, i can help. Shouldn’t be too much grief though…