Contact

Subscribe via Email

Subscribe via RSS/JSON

Categories

Recent Posts

Creative Commons Attribution 4.0 International License
© Rakhesh Sasidharan

Elsewhere

Using Git and Dropbox (along with GitHub but working tree in Dropbox)

Here’s something I tried out today. I don’t think it will work out in the long term, but I spent time on it so might as well make note of it here in case it helps someone else. 

I have multiple machines – one at office, couple at home – and usually my workflow involves having whatever I am working on being present in a local folder (not in Dropbox) managed by Git, with changes pushed and pulled to GitHub. That works fine with the caveat that sometimes I forget to push the changes upstream and leave office/ home and and then I am unable to work at home/ office because my latest code is on the other machine while GitHub and the machine I am on don’t have those changes.

One solution would be to have a scheduled task (or some hook to PowerShell ISE) that automatically commits & pushes my changes to GitHub periodically (or whenever I click save). I haven’t explored those but they seem wasteful. I like my commits to be when I make them, not run automatically, resulting in lots of spurious entries in my history.

My idea is to have the working tree in Dropbox. This way Dropbox takes care of syncing my latest code across all my machines. I don’t want to move the Git repository into Dropbox for fear of corruption/ sync conflicts, so I’ll keep that out of Dropbox in a local folder on each machine. My revised workflow will be that I am on a particular machine, I’ll make some changes, maybe commit & push them … or maybe not, then I’ll go to the other machine and I can continue making changes there and push them off when I am happy. Or so I hope …

Setting it up

Consider the case of two machines. One’s GAIA, the other’s TINTIN. Let’s start by setting up a repository and working tree on GAIA. 

Straight-forward stuff. My working tree is in Dropbox, Git repository is in a local folder. I make a commit and push it to GitHub. So far so good. 

Now I get to TINTIN. 

Thanks to Dropbox TINTIN already has the working tree. But it doesn’t have the Git repository (as that’s not a part of Dropbox). So even though the working tree appears on TINTIN I can’t issue any Git commands against it. 

The one time workaround is to pull the repository and working elsewhere and simply copy the repository to where the working tree in Dropbox expects it (c:/Users/Rakhesh/Git/Test.git in the above example)

Here’s what the xcopy switches do:

  • /K Copies attributes. Normal Xcopy will reset read-only attributes.
  • /E Copies directories and subdirectories, including empty ones.
  • /H Copies hidden and system files also.
  • /I If destination does not exist and copying more than one file, assumes that destination must be a directory.

I use xcopy because the Git repository is hidden and I can’t use move. The various switches ensure xcopy copies everything over.

Update: While writing this post I realized the above is an over-kill. All I need to do on TINTIN is the following:

Simple! This creates a bare-repository in the expected location.

Now I open the config file in the Git repository and add a pointer to the working tree (not necessary, but is a good practice). That is, I add to the [core] section:

Update2: If you create a bare repository as above remove from the [core] section (if it exists):

Note: I had some trouble with the bare repository idea a couple of times so I wouldn’t recommend it. Specifically, it didn’t pick up some folder changes. If it doesn’t work for you, use the approach I went with initially. 

Now TINTIN too is in sync with GAIA and GitHub. 

Good!

Changing things

Let’s add a file and commit it on GAIA. And while we are at it push it to GitHub. 

Once again, thanks to Dropbox TINTIN gets the changes in the working tree. But does it have the history? No! And because of that it will think there are changes to commit:

I can’t simply commit this change on TINTIN and push it upstream because that would conflict with the server:

That’s because while both GAIA and TINTIN because from the same commit point, now GAIA (and upstream) are at commit A which is a descendant of the starting point, while TINTIN is at commit B which is also a descendant of the same starting point. A and B are siblings, so if the push were to succeed the previous commit and its history would be lost. (These are known as non-fast-forward updates and the git-push help page has a diagram explaining this). 

I can’t simply pull the changes from upstream either as the same conflict applies there too!

One workaround is to stash the changes on TINTIN – which is fine in this case because they are not really changes, we have already captured it upstream and on GAIA – and then pull the changes. (I could also commit and then pull changes from GitHub, merge-ing them). 

Note that TINTIN has the correct history too. 

Update: If git pull complains about changes it is also possible to do a git reset as below: 

This is what I do nowadays. What git reset --hard does is that it resets the working tree and HEAD to the commit upstream (origin/master), effectively replacing all the tracked local files with versions from upstream. 

Changing things (part 2)

Now on to the scenario for which I am mixing Dropbox and GitHub. I make changes on GAIA and commit these, but forget to push them upstream. 

When I get to TINTIN, the changes are there but once again it isn’t aware of the commit. And unlike in the previous case, it can’t pull any changes from GitHub as it too does not know of any changes. All I can do is add the changed file and commit it. Note that this will be a different commit from the one at GAIA (which TINTIN is not aware of). 

And then I push it upstream when I am done working on TINTIN:

Back at GAIA the next day, when I check the status I’ll see the following:

This is because as far as GAIA knows I had made a commit but didn’t push it to GitHub. So it tells me my local branch is ahead by 1 commit. Git never checks the remote end in real time when I issue a git status. It merely reports based on the cached information it has. 

I can’t of course push this commit to GitHub because that has long moved on. We are back to a non-fast-forward update scenario here because GitHub has a commit from TINTIN which is a sibling to the commit from GAIA that we are now trying to push.  

Thankfully, in this particular case I can do a git pull and it will do a merge of the commit from GAIA and GitHub. A merge is possible here because the commits have a common ancestor and changes can be incorporated without any conflicts. 

And just to confirm, GAIA has the whole history, including a commit indicating the merge:

I can now push this to GitHub (and later pull to TINTIN). 

That’s it!

Wrapping up

As you can see having your working tree in Dropbox, with the Git repository in a local folder elsewhere, is possible but involves some minor tweaking around. I am going to try this for some of my repositories going forward, and will update this post if I come across any gotchas. 

Fringe cases

Occasionally when I am on one of my machines and do a git pull I get the following:

In the above case I had created a new folder (called Update-Hosts) on another machine and moved files to it. The machine I am currently on has these changes via Dropbox but isn’t aware of them from a Git point of view. So I did what Git suggests – remove this folder and git pull again. I won’t lose anything because the remote copy has the latest changes anyway.

You can see it’s picked up the new directory and moves/ renames.

Using Git and Dropbox (along with GitHub)

I use Git mostly with GitHub I forget Git is an independent tool of its own. You don’t really need to push everything to GitHub to make it public. 

Usually I have my local Git repository somewhere on my computer, pushing to a remote repository in GitHub. What I’d like to do is have the local repository in Dropbox too because I stash most of my things in Dropbox. The way to do this is to have a new repository in Dropbox and to push to that also along with the GitHub one. So I continue working in the local repository as before, but when I push I do so to GitHub and the copy in my Dropbox. Nice, eh!

Here’s how I go about doing that.

1) Create a new repository as usual:

2) Change to this directory, add some files, and commit:

3) Create a new repository in GitHub:

newrepo

GitHub helpfully gives commands on how to push from the existing repository to GitHub, so let’s do that (with two slight differences I’ll get to in a moment)

The differences are that (a) I don’t call the remote repository origin, I call it github, and (b) I don’t set it as the upstream (if I set it as upstream via the -u switch then this becomes the default remote repository for push and pull operations; I don’t want this to be the default yet). 

4) Create a new repository in Dropbox.

Note that this must be a bare repository. (In Git (v 1.7.0 and above) the remote repository has to be bare in order to accept a push. A bare repository is what GitHub creates behind the scenes for you. It only contains the versioning information, no working files. See this page for more info). 

5) Add the Dropbox repository as a remote to the local repository (similar command as with the “github” remote):

I am calling this remote “dropbox” so it’s easy to refer to. 

6) Push to this repository:

(Note to self: I don’t set this as upstream. Hence the lack of a -u switch. If I set it as upstream then this repository becomes the new default repository if you issue a git push command without any remote name). 

7) Create a new remote call “origin” and add both the Dropbox and GitHub remotes to it:

8) And push to this remote, setting it as the new upstream:

That’s it! From now on a git push will push to both Dropbox and GitHub. And I can push individually to either of these remotes by calling them by name: git push dropbox or git push github.

When I am on a different machine I just clone from either Dropbox or GitHub as usual, for example:

Note that I change the name from origin (the default) to dropbox (or github).  

Then I add the other remote repository:

And as before add a remote called origin and make it upstream:

There’s no real advantage in having two remote repositories like this, but I was curious whether I can set something up along these lines and thought I’d give it a go. Two blog posts I came across are this and this. They take a similar approach as me and are about storing the repository in Dropbox only (rather than Dropbox and GitHub). 

Dropbox to OneDrive and back

I am a Dropbox user. Got all my music, wallpapers, documents, portable programs, etc in Dropbox. I don’t store any sensitive stuff in it, just non-sensitive stuff.

Past few days I had been thinking of moving from Dropbox to OneDrive. Why? Because 1) Dropbox is very expensive compared to the competition, 2) OneDrive is already built into Windows 8 and newer any way, and 3) I love the “smart files” feature of OneDrive.

Smart files is a neat feature really. In a corporate environment you would have worked with offline/ cached files – smart files is very similar to that. What happens with smart files is that all your OneDrive files are visible in Explorer, but by default they are not downloaded to the computer. Only when you open the file, or choose to have it available offline, are the files download. I find this useful as this way I don’t have to resort to “selective sync” like I do with Dropbox. Instead, I can see everything on all my computers but only download the files I need as and when I open them. Of course, on some computers I would download all the files for offline use so I can use them even when I am not connected to the Internet.

I didn’t want to jump into OneDrive straight away so I started with moving some of my documents over, a few music folders, and also all my portable programs. Initially I felt everything was going well but after I moved to another laptop did I realize the experience is not as smooth as Dropbox. Couple of things:

  1. Dropbox files and folders have a marker that show whether the file or folder is in sync or to be synced. OneDrive has no such thing.
  2. The system tray icon of Dropbox offers better feedback. It shows me what’s happening, how may items remain, etc. The OneDrive icon doesn’t show all that. Right clicking the OneDrive folder icon gives a percentage status but it isn’t as detailed as Dropbox.
  3. Smart files are good but they can bit you unknowingly. Case in hand: I copied my portable programs to OneDrive from my desktop. Then I logged in to my laptop, saw that all these portable programs were visible, and so tried copying them elsewhere. Maybe it’s because I use TeraCopy, the program was just stuck on the first file. No error message, no status, nothing. Cancelled and tried again a couple of times but got the same result each time. Then I realized what the problem was.

    On the laptop the files were not really present as they were only links to the real files elsewhere. But since there’s no marker to indicate this (except the status field of a file) I hadn’t realized this. I imagine OneDrive was trying to download the files in the background so TeraCopy could copy and that’s why it was stuck. Not good!

I think ultimately it comes down to Dropbox being longer in this field and so its software is more refined and “experienced”. Dropbox has other great features too which I have no idea whether OneDrive (or even Copy, Google Drive, etc) support. Features such as incremental uploads (I modify an MP3 file tag, only the changed bits are uploaded, not the whole file again), LAN syncing (if my desktop and laptop are online together at home, and all my files are uploaded to Dropbox, they are not downloaded again to the desktop or laptop; rather, the files are copied from laptop or desktop to the other over LAN).

So that’s it. Back to Dropbox!