Working Copy 2.6 (full on-device Git source control), now has "Export to Codea"

[yojimbo2000]

Around the same time iOS 9 came out, Working Copy, a full iOS Git client, got an update that introduced some exciting new features. One of these is a WebDAV server. This means it is possible to have full two-way communication between Codea and Working Copy.

Announcing…

Working Copy Codea Client (WebDAV edition)

https://github.com/Utsira/WorkingCopyCodeaClient

It’s still beta, but I’ve been doing all of my versioning with it for a while now (I am in the Working Copy beta program). eg all the Soda versioning was done with it, it seems fairly solid.

A lot of the inspiration for this came from the excellent Codea-SCM by @juce

From the readme:

WCCC: Working Copy Codea Client

A Codea program that connects Codea to Working Copy, a full-featured iOS Git client.

##Features

• Browse all of your repositories from within Codea

• Link remote files and folders to Codea projects for two-way version control

• Switch branches in Working Copy, or checkout an earlier version of your project, then pull that version into Codea

• Easily import multi-file repositories that don’t have installers with WCCC’s “Copy as single file” feature, which concatenates the remote files using Codea’s “paste into project” format

• SHA1 authentication verifies write operations and warns you if you are in danger of overwriting data

Brings the full power of Git source control and social coding to Codea. When you switch to a different branch of your repository, or checkout a previous version, those changes are reflected in the Working Copy WebDAV. This makes it easy to switch between different branches of a project, or to pull a previous version of the project into Codea. All of this is done locally, on device. No web connection is needed. When you get back online, you can merge your changes with your remote repositories on GitHub or BitBucket. WCCC’s support for Codea’s paste-into-project format means that you can easily install multi-file Git repositories in Codea.

Installation

Getting everything installed is a bit fiddly. But, you only have to follow these steps once, and then you’re flying.

1. You need the apps Codea and Working Copy installed on your iPad.

2. Install Soda in Codea (this handles the GUI of WCCC).

3. Install WCCC in Codea: Copy the entire contents of Working Copy Codea Client installer.lua to the clipboard. In Codea project screen, long-press “+ Add New Project”, and select “Paste into project”.

4. Make Soda a dependency of WCCC: In the code editor of the WCCC project, press the + sign in the top-right corner and put a check in the box next to “Soda”.

5. WCCC uses 2 technologies to communicate with Working Copy: x-callback URLs and WebDAV. These both need to be turned on in Working Copy settings.

• Turn on x-callbacks and copy the key to your clipboard:

  

• Currently, there is no support for digest authentication in Codea (I’m looking into whether this can be implemented now that Codea has sockets). Therefore, you will have to clear both the username and the password fields, and turn off remote connections. This means that Working Copy’s WebDAV will only run on-device (ie only other apps on your iPad can see it). It is highly recommended that you do not turn on remote connections without a password and username set. The URL should be http://localhost:8080/

6. The first time you run WCCC you will get a dialog prompting you to enter your x-callback key and the WebDAV URL. Paste the key into the text entry field. WCCC will then wake the WebDAV. Switch back to Codea and start working.

###About WebDAV
When WorkingCopy is in the background, the WebDAV server is shut off after a few minutes. WCCC detects this and automatically wakes the WebDAV up again. When this happens you’ll see a prompt telling you that the WebDAV needs to be woken up.

When you press “Activate WebDAV”, Working Copy will be foregrounded or opened, and the WebDAV started. When you see a blue “Connect to WebDAV server at” message in WorkingCopy, you can switch back to Codea, eg with a four-finger app-switch gesture, or, on iOS 9, by pressing the “Back to Codea” button in the top-left corner

Usage

Browse your Working Copy repositories by selecting folders and files in the “Finder” pane on the left.

WCCC supports two modes of working with repositories:

1. Single Project Repositories.

This is recommended for working with large projects. The repository houses a single Codea project. The project’s tabs are saved as individual .lua files in a folder called /tabs/. The project’s Info.plist file is saved to the root of the repository. This file contains the order that the tabs are in in Codea.

Actions available in Single Project repositories:

Copy as a single file

Multiple lua files are concatenated together in “paste into project” format. If WCCC sees an Info.plist file in the root of the repository, it reads that first to determine the order to concatenate the lua files in. If additional lua files are found not mentioned in Info.plist, these will be added to the end of the concatenate-ed file. If no Info.plist file is found, WCCC will concatenate the files in the order in which they appear in the repository (nb this could be wrong, and you might have to reorder the tabs once you paste the project in Codea). WCCC expects to find all of the .lua files in one location (ie a single folder on the root, or just in the root itself), and will only search folders one layer past the root of the repository

Link/ Relink

To properly push and pull, you need to link the repository to a Codea project. As currently there is no way to get the list of Codea projects from within Codea, you will have to type in the Codea project name (case sensitive). WCCC remembers the link between the repository path and the Codea project, so you only have to type in this name once. Once linked, push and pull become available as actions

Push

Pushes the linked project to the repository. Info.plist is pushed to the root, and the tabs are pushed to a /tabs/ folder. If you have pushed to the remote before, SHA1 authentication is used to check whether the remote has changed since you last pushed. Following the write, a further SHA1 verification checks that the push succeeded (issue: the verification frequently fails if files have been renamed or deleted. Switching to Working Copy, then back to Codea and pushing again seems to fix this). If the project is very large, generating the SHA1 keys may take a second or two. A switch to Working Copy button will then open a commit window in Working Copy. This allows you to verify what has changed, select which files to commit, write a commit message, and push to a remote repository if there is one.

Pull

Pulls the content of the remote repository into the Codea project. If you have pushed to the remote before, SHA1 authentication is used to check whether you have changed the Codea project since you last pushed. Following the pull, the operation is verified with SHA1 authentication.

Push installer to root

Saves an additional Installer.lua file to the root of the repository. The installer contains all of the project’s tabs, concatenated together into a single file using Codea’s “paste-into-project” format. This makes it easy for people not using WCCC to install your project, by copying the contents of the installer, and then long pressing on the “add project” button in Codea’s main screen, and selecting “paste into project”. The installer files that you used to install Soda and WCCC were created in this way.

2. Multiple Project Repositories.

For smaller projects that don’t require an entire repository, you can set a repository to multiple project mode. Codea projects are saved to the root of the repository as single files using Codea’s “paste-into-project” format. Add a new project to the repository, or select a file in the finder to bring up actions for the file

Actions available for files in Multiple Project repositories:

Copy

Places the file being viewed in the clipboard

Link/ Relink

Link the file to a Codea project. Enables Push

Push as single file

Pushes the project to the file as a single file in Codea’s “paste-into-project” format

Pull

(not yet implemented)

[Rodolphe]

Impressive!!!

[Kirl]

Looks quite handy, thanks for sharing. =)

[yojimbo2000]

@Rodolphe the other thing to point out, which might be particularly interesting for you, is that Working Copy also works really well with Codea export files (ie completely ignoring my client program here). So you long-press the project in Codea, hit export, do your settings, then, in the share dialog that appears, select Working Copy as the target. Working Copy deals with zip files very intelligently. A dialog will appear giving you options to unzip as a new repository, or as a folder into an existing repository etc. I haven’t experimented yet with then linking the Working Copy repo to the xcode source control, but it could be one way of triangulating your project in Codea, the latest Codea runtime, and the changes you’ve made in xcode.

So you can actually do quite a lot just using Codea and Working Copy and regular iOS sharing features.

The main feature that my client program adds, is the process in the reverse direction, pulling multifile projects in to Codea (and of course getting multifile projects out of Codea where you don’t want the entire runtime and everything else that goes into the xcode export).

[yojimbo2000]

I added a short tutorial to the readme which might be handy for those new to Git

Tutorial

Cloning a repository and pulling the files into Codea

A quick tutorial. We’re going to use WCCC to manage its own source code, so that when updates are posted, they can easily be pulled into Codea. We’re going to do this by cloning this repository. A clone is a local copy of a repository that can easily be kept in sync with the online repository. This can be done in just 3 steps.

1. Copy the URL for this repository. Git repositories have special addresses that end in .git, but, the regular web URL for repositories on GitHub can be used as an alias for the git address, so go ahead and copy the URL in the Safari address bar (or, hitting Copy in the action menu also copies the URL)

2. Clone the repository in Working Copy. Launch Working Copy. If you are in a repository, press the back button until you get back to the root. At the top of the left-hand file browser pane, press + to add a new repository, and then press “Clone”.
   
   Working Copy should recognise that you have a GitHub URL in the pasteboard, and will offer to clone the repository. Tap the blue message (if this blue prompt does not appear, just paste the URL into the URL field and press clone). The WCCC repository will be cloned in Working Copy.

3. Use WCCC to link the clone in Working Copy to the project in Codea. Launch WCCC. You will probably have to switch back and forth between Working Copy and WCCC to wake the WebDAV. In WCCC, use the left-hand file browser panel to find the WCCC repository that you just cloned in Working Copy and select it. Press the “Link” button, and in the link dialog that opens, enter whatever name you called WCCC in Codea, and press “Link”.
   
   WCCC will remember the connection between the Working Copy repository and the Codea project. You can now press “Pull” whenever you want to pull the latest version of WCCC into Codea.

[LoopSpace]

Do you happen to know how much of this I could use with the free version of WCCC? It would be nice to try-before-I-buy to see if I would find this useful.

[yojimbo2000]

The IAP unlocks push to remote (eg to your account on BitBucket, GitHub, or any git enabled remote). So you’ll be able to pull from remotes, and do on-device Git with the free version.

[piinthesky]

Tried to adopt wccc, but get an error message??

Main 17: attempt to index a nil value (global sha1)

Do i understand correctly that if i do not purchase the working copy iap, the code is only saved locally on the ipad, so it is not secure against the ipad failing?

[yojimbo2000]

@piinthesky thanks for bringing this error to my attention, and apologies for not spotting this. This was due to a problem in the installer, which I’ve now fixed. The installer now saves all the tabs before attempting to load them, in case an error (or in this case, the “please make Soda a dependency” assert) interrupts the save. Please reinstall with the new installer:

--# Main

local url = "https://raw.githubusercontent.com/Utsira/WorkingCopyCodeaClient/master/WorkingCopyWebDAV.codea/"
 
local function install(data)
    --parse plist into list of tab files
    local array = data:match("<key>Buffer Order</key>%s-<array>(.-)</array>")
    local files = {}   
    for tabName in array:gmatch("<string>(.-)</string>%s") do
        table.insert(files, {name = tabName})
    end   
    --success function
    local function success(n, name, data)
        if not data then alert("No data", name) return end
        print("Loaded "..n.."/"..#files..":"..name)
        files[n].data = data
        for _,v in ipairs(files) do
            if not v.data then 
                return --quit this function if any files have missing data
            end
        end
        --if all data is present then save...
        for i,v in ipairs(files) do
            saveProjectTab(v.name, v.data)
            print("Saved "..i.."/"..#files..":"..v.name)
        end
        for i,v in ipairs(files) do --load...
            load(v.data)() 
        end
        setup() --...and run
    end
    --request all the tab files
    for i,v in ipairs(files) do 
        local function retry(error) --try each file twice, in case of time-outs
            print(error, v.name.." not found, retrying")
            http.request(url..v.name..".lua", function(data) success(i, v.name, data) end, function(error2) alert(error2, v.name.." not found") end)
        end
        http.request(url..v.name..".lua", function(data) success(i, v.name, data) end, retry)
    end
end
http.request(url.."Info.plist", install, function (error) alert(error) end)

To answer your second question, yes, you need to buy the full version to push to a remote server (GitHub, BitBucket etc).

[yojimbo2000]

There’s a couple of things to note here:

First of all, v 2.6 of Working Copy introduces a fantastic “Export to Codea” function. This is absolutely the easiest way to pull a multi-file repo into Codea. Also, the current version of Codea added a very nice zipped project export feature. Working Copy has great support for zipped folders, so this is a very nice way to push projects to Working Copy.

So with these two features, clients like my code here, WCCC, are a lot less necessary. For many users, Codea’s “zipped project export”, plus Working Copy’s “Export to Codea”, will be enough to push and pull from repos, without having to install a client and set it up.

There are some advantages to using WCCC though. Mainly, “paste-into-project” is only possible with a new project, you can’t paste into an existing one. This could be a problem if you’re trying to restore an earlier build but you have project data or assets in your project folder. WCCC’s pull command will overwrite the tabs in the existing project, so it’s useful if you need to roll back your code, but you want to otherwise keep the project folder as is (TBH I just put all assets in Dropbox, I never use the project asset folder). Secondly, WCCC will remember which repos in Working Copy are connected to which Codea projects, which is useful. Finally, if you rename or delete a tab, WCCC will remove the old tab from the repo, whereas if you’re using zipped project export to push those kinds of changes, the old tab will need to be deleted by hand in the repo.

[RonJeffries]

i guess I’m missing some key thing. ive been using Codea SCM for a project. I wanted to try WCCC and Working Copy. (Have the full version of WC.) I set up working copy, cloned the repo. set up WCCC, hooked to it. seemed ok. selected pull and it says “pull verify failed”. The local shows all kinds of plist things, the remote looks like just tab contents.

What am I missing?

[RonJeffries]

@yojimbo2000 see above, thanks!

[yojimbo2000]

@RonJeffries Thanks for the report. I’m thinking about taking the SHA1 verification out of the project altogether, or using it more sparingly, as I think it’s too heavy a tool for write verification. With larger projects it takes too long to produce the keys (maybe I’m impatient), and it often returns “verify failed” messages when, in fact, the operation ran perfectly well (this seems to happen particularly if a tab has been renamed or deleted). Also, I’ve been using WCCC for all my projects for months now and have never had any data integrity issues.

A couple of questions for you:

Did the pull operation succeed (despite the “pull verify failed” message)?

I wasn’t clear what repo you’re trying to pull from. Is it one that contains an Info.plist file in the same folder as all of the .lua tabs? WCCC is now designed to work with “flat” repositories, like the .codea folders that Codea now exports (either with the “Xcode export” or “zipped project folder”). Codea-SCM on the other hand places the tabs one folder deeper than the Info.plist. You can easily convert a Codea-SCM repo to a WCCC one by putting the Info.plist file into the tabs folder.

Or, WCCC will still work if it doesn’t find an Info.plist, it just won’t know what order to write the tabs in. Tab order is only important if:

• you have class inheritance (sub-classes must be to the right of super-classes)
• you execute code in other tabs at compile-time (ie outside of any function)
• you have sub-tables spanning several tabs (eg defining UI = {} in one tab, then UI.button ={} in another)

[RonJeffries]

@yojimbo2000 thanks for reply.

If it went ahead and pulled and then put up the bad verification, then I guess it succeeded: Codea SCM shows it as green. I had assumed that the verification, whatever it is, preceded doing anything.

It is a project with all the tabs as produced from Codea-SCM. So I gather the two are using incompatible formats for the repo. That probably explains everything.

Sounds like I’d best stick with one or the other. Thanks!

[yojimbo2000]

I switched to the flat structure, with the plist and the tabs in the same folder, because this is how Codea exports repositories, and it is also several orders easier to write the WebDAV code if you’re not having to jump into and search through sub-folders.

[RonJeffries]

So incompatible with Codea SCM, right? Thanks!

[yojimbo2000]

The worst thing that would happen if you used Codea-SCM to pull in a repo created by Codea’s repo export, is that it might think the Info.plist is one of the tabs and write it to one (I haven’t tried this so can’t say for sure). So you’d have to delete it that tab or comment it out before attempting to run the code. And I guess your tabs wouldn’t be in order. So, yeah, probably not ideal to switch back and forth between the two systems.

[RonJeffries]

I will have to think about that. If I understand you, you’re saying that WCCC adheres more closely to Codea’s export repo format than Codea-SCM?
Thanks,

[yojimbo2000]

Yes. Both the repo export options in Codea, the Xcode one and the zipped project, put the Info.plist in the same folder as the lua tabs. Seeing as Working Copy handles the importing of zip folders so well, it seemed like the best method to use, especially when Codea added the “zipped project export” option.

So I have my Xcode project in Working Copy on the iPad. I make changes in Codea, push via Working Copy to BitBucket. Then, in Xcode on the Mac, I can pull the changes in. It’s a very smooth workflow, and I can still have full source control when I don’t have an internet connection. Having a full Git client that is local to the iPad is incredibly useful IMO, as it means it’s not just push and pull, you can fork, have multiple branches and so on (though remerging a branch never quite seems to go that smoothly in Working Copy).

Also, don’t forget that Working Copy now has a built-in “Export to Codea” function. This is particularly useful the first time you’re bringing a project into Codea.

[binaryblues]

Thanks for your great work! I love it!