9.1 Git and Other Systems - Git as a Client

The base command in Git for all the Subversion bridging commands is git svn. It takes quite a few commands, so we’ll show the most common while going through a few simple workflows.

It’s important to note that when you’re using git svn, you’re interacting with Subversion, which is a system that works very differently from Git. Although you can do local branching and merging, it’s generally best to keep your history as linear as possible by rebasing your work, and avoiding doing things like simultaneously interacting with a Git remote repository.

Don’t rewrite your history and try to push again, and don’t push to a parallel Git repository to collaborate with fellow Git developers at the same time. Subversion can have only a single linear history, and confusing it is very easy. If you’re working with a team, and some are using SVN and others are using Git, make sure everyone is using the SVN server to collaborate – doing so will make your life easier.

To demonstrate this functionality, you need a typical SVN repository that you have write access to. If you want to copy these examples, you’ll have to make a writeable copy of an SVN test repository. In order to do that easily, you can use a tool called svnsync that comes with Subversion.

To follow along, you first need to create a new local Subversion repository:

Then, enable all users to change revprops – the easy way is to add a pre-revprop-change script that always exits 0:

You can now sync this project to your local machine by calling svnsync init with the to and from repositories.

This sets up the properties to run the sync. You can then clone the code by running:

Although this operation may take only a few minutes, if you try to copy the original repository to another remote repository instead of a local one, the process will take nearly an hour, even though there are fewer than 100 commits. Subversion has to clone one revision at a time and then push it back into another repository – it’s ridiculously inefficient, but it’s the only easy way to do this.

Now that you have a Subversion repository to which you have write access, you can go through a typical workflow. You’ll start with the git svn clone command, which imports an entire Subversion repository into a local Git repository. Remember that if you’re importing from a real hosted Subversion repository, you should replace the file:///tmp/test-svn here with the URL of your Subversion repository:

This runs the equivalent of two commands – git svn init followed by git svn fetch – on the URL you provide. This can take a while. If, for example, the test project has only about 75 commits and the codebase isn’t that big, Git nevertheless must check out each version, one at a time, and commit it individually. For a project with hundreds or thousands of commits, this can literally take hours or even days to finish.

The -T trunk -b branches -t tags part tells Git that this Subversion repository follows the basic branching and tagging conventions. If you name your trunk, branches, or tags differently, you can change these options. Because this is so common, you can replace this entire part with -s, which means standard layout and implies all those options. The following command is equivalent:

At this point, you should have a valid Git repository that has imported your branches and tags:

Note how this tool manages Subversion tags as remote refs. Let’s take a closer look with the Git plumbing command show-ref:

Git doesn’t do this when it clones from a Git server; here’s what a repository with tags looks like after a fresh clone:

Git fetches the tags directly into refs/tags, rather than treating them remote branches.

Now that you have a working directory, you can do some work on the project and push your commits back upstream, using Git effectively as an SVN client. If you edit one of the files and commit it, you have a commit that exists in Git locally that doesn’t exist on the Subversion server:

Next, you need to push your change upstream. Notice how this changes the way you work with Subversion – you can do several commits offline and then push them all at once to the Subversion server. To push to a Subversion server, you run the git svn dcommit command:

This takes all the commits you’ve made on top of the Subversion server code, does a Subversion commit for each, and then rewrites your local Git commit to include a unique identifier. This is important because it means that all the SHA-1 checksums for your commits change. Partly for this reason, working with Git-based remote versions of your projects concurrently with a Subversion server isn’t a good idea. If you look at the last commit, you can see the new git-svn-id that was added:

Notice that the SHA-1 checksum that originally started with 4af61fd when you committed now begins with 95e0222. If you want to push to both a Git server and a Subversion server, you have to push (dcommit) to the Subversion server first, because that action changes your commit data.

If you’re working with other developers, then at some point one of you will push, and then the other one will try to push a change that conflicts. That change will be rejected until you merge in their work. In git svn, it looks like this:

To resolve this situation, you can run git svn rebase, which pulls down any changes on the server that you don’t have yet and rebases any work you have on top of what is on the server:

Now, all your work is on top of what is on the Subversion server, so you can successfully dcommit:

Note that unlike Git, which requires you to merge upstream work you don’t yet have locally before you can push, git svn makes you do that only if the changes conflict (much like how Subversion works). If someone else pushes a change to one file and then you push a change to another file, your dcommit will work fine:

This is important to remember, because the outcome is a project state that didn’t exist on either of your computers when you pushed. If the changes are incompatible but don’t conflict, you may get issues that are difficult to diagnose. This is different than using a Git server – in Git, you can fully test the state on your client system before publishing it, whereas in SVN, you can’t ever be certain that the states immediately before commit and after commit are identical.

You should also run this command to pull in changes from the Subversion server, even if you’re not ready to commit yourself. You can run git svn fetch to grab the new data, but git svn rebase does the fetch and then updates your local commits.

Running git svn rebase every once in a while makes sure your code is always up to date. You need to be sure your working directory is clean when you run this, though. If you have local changes, you must either stash your work or temporarily commit it before running git svn rebase – otherwise, the command will stop if it sees that the rebase will result in a merge conflict.

When you’ve become comfortable with a Git workflow, you’ll likely create topic branches, do work on them, and then merge them in. If you’re pushing to a Subversion server via git svn, you may want to rebase your work onto a single branch each time instead of merging branches together. The reason to prefer rebasing is that Subversion has a linear history and doesn’t deal with merges like Git does, so git svn follows only the first parent when converting the snapshots into Subversion commits.

Suppose your history looks like the following: you created an experiment branch, did two commits, and then merged them back into master. When you dcommit, you see output like this:

Running dcommit on a branch with merged history works fine, except that when you look at your Git project history, it hasn’t rewritten either of the commits you made on the experiment branch – instead, all those changes appear in the SVN version of the single merge commit.

When someone else clones that work, all they see is the merge commit with all the work squashed into it, as though you ran git merge --squash; they don’t see the commit data about where it came from or when it was committed.

Branching in Subversion isn’t the same as branching in Git; if you can avoid using it much, that’s probably best. However, you can create and commit to branches in Subversion using git svn.

To create a new branch in Subversion, you run git svn branch [new-branch]:

This does the equivalent of the svn copy trunk branches/opera command in Subversion and operates on the Subversion server. It’s important to note that it doesn’t check you out into that branch; if you commit at this point, that commit will go to trunk on the server, not opera.

Git figures out what branch your dcommits go to by looking for the tip of any of your Subversion branches in your history – you should have only one, and it should be the last one with a git-svn-id in your current branch history.

If you want to work on more than one branch simultaneously, you can set up local branches to dcommit to specific Subversion branches by starting them at the imported Subversion commit for that branch. If you want an opera branch that you can work on separately, you can run:

Now, if you want to merge your opera branch into trunk (your master branch), you can do so with a normal git merge. But you need to provide a descriptive commit message (via -m), or the merge will say “Merge branch opera” instead of something useful.

Remember that although you’re using git merge to do this operation, and the merge likely will be much easier than it would be in Subversion (because Git will automatically detect the appropriate merge base for you), this isn’t a normal Git merge commit. You have to push this data back to a Subversion server that can’t handle a commit that tracks more than one parent; so, after you push it up, it will look like a single commit that squashed in all the work of another branch under a single commit. After you merge one branch into another, you can’t easily go back and continue working on that branch, as you normally can in Git. The dcommit command that you run erases any information that says what branch was merged in, so subsequent merge-base calculations will be wrong – the dcommit makes your git merge result look like you ran git merge --squash. Unfortunately, there’s no good way to avoid this situation – Subversion can’t store this information, so you’ll always be crippled by its limitations while you’re using it as your server. To avoid issues, you should delete the local branch (in this case, opera) after you merge it into trunk.

The git svn toolset provides a number of commands to help ease the transition to Git by providing some functionality that’s similar to what you had in Subversion. Here are a few commands that give you what Subversion used to.

The git svn tools are useful if you’re stuck with a Subversion server, or are otherwise in a development environment that necessitates running a Subversion server. You should consider it crippled Git, however, or you’ll hit issues in translation that may confuse you and your collaborators. To stay out of trouble, try to follow these guidelines:

If you follow those guidelines, working with a Subversion server can be more bearable. However, if it’s possible to move to a real Git server, doing so can gain your team a lot more.

First, you need to install git-remote-hg. This basically entails dropping its file somewhere in your path, like so:

…assuming ~/bin is in your $PATH. Git-remote-hg has one other dependency: the mercurial library for Python. If you have Python installed, this is as simple as:

If you don’t have Python installed, visit https://www.python.org/ and get it first.

The last thing you’ll need is the Mercurial client. Go to https://www.mercurial-scm.org/ and install it if you haven’t already.

Now you’re ready to rock. All you need is a Mercurial repository you can push to. Fortunately, every Mercurial repository can act this way, so we’ll just use the "hello world" repository everyone uses to learn Mercurial:

Now that we have a suitable “server-side” repository, we can go through a typical workflow. As you’ll see, these two systems are similar enough that there isn’t much friction.

As always with Git, first we clone:

You’ll notice that working with a Mercurial repository uses the standard git clone command. That’s because git-remote-hg is working at a fairly low level, using a similar mechanism to how Git’s HTTP/S protocol is implemented (remote helpers). Since Git and Mercurial are both designed for every client to have a full copy of the repository history, this command makes a full clone, including all the project’s history, and does it fairly quickly.

The log command shows two commits, the latest of which is pointed to by a whole slew of refs. It turns out some of these aren’t actually there. Let’s take a look at what’s actually in the .git directory:

Git-remote-hg is trying to make things more idiomatically Git-esque, but under the hood it’s managing the conceptual mapping between two slightly different systems. The refs/hg directory is where the actual remote refs are stored. For example, the refs/hg/origin/branches/default is a Git ref file that contains the SHA-1 starting with “ac7955c”, which is the commit that master points to. So the refs/hg directory is kind of like a fake refs/remotes/origin, but it has the added distinction between bookmarks and branches.

The notes/hg file is the starting point for how git-remote-hg maps Git commit hashes to Mercurial changeset IDs. Let’s explore a bit:

So refs/notes/hg points to a tree, which in the Git object database is a list of other objects with names. git ls-tree outputs the mode, type, object hash, and filename for items inside a tree. Once we dig down to one of the tree items, we find that inside it is a blob named “ac9117f” (the SHA-1 hash of the commit pointed to by master), with contents “0a04b98” (which is the ID of the Mercurial changeset at the tip of the default branch).

The good news is that we mostly don’t have to worry about all of this. The typical workflow won’t be very different from working with a Git remote.

There’s one more thing we should attend to before we continue: ignores. Mercurial and Git use a very similar mechanism for this, but it’s likely you don’t want to actually commit a .gitignore file into a Mercurial repository. Fortunately, Git has a way to ignore files that’s local to an on-disk repository, and the Mercurial format is compatible with Git, so you just have to copy it over:

The .git/info/exclude file acts just like a .gitignore, but isn’t included in commits.

Let’s assume we’ve done some work and made some commits on the master branch, and you’re ready to push it to the remote repository. Here’s what our repository looks like right now:

Our master branch is two commits ahead of origin/master, but those two commits exist only on our local machine. Let’s see if anyone else has been doing important work at the same time:

Since we used the --all flag, we see the “notes” refs that are used internally by git-remote-hg, but we can ignore them. The rest is what we expected; origin/master has advanced by one commit, and our history has now diverged. Unlike the other systems we work with in this chapter, Mercurial is capable of handling merges, so we’re not going to do anything fancy.

Perfect. We run the tests and everything passes, so we’re ready to share our work with the rest of the team:

That’s it! If you take a look at the Mercurial repository, you’ll see that this did what we’d expect:

The changeset numbered 2 was made by Mercurial, and the changesets numbered 3 and 4 were made by git-remote-hg, by pushing commits made with Git.

Git has only one kind of branch: a reference that moves when commits are made. In Mercurial, this kind of a reference is called a “bookmark,” and it behaves in much the same way as a Git branch.

Mercurial’s concept of a “branch” is more heavyweight. The branch that a changeset is made on is recorded with the changeset, which means it will always be in the repository history. Here’s an example of a commit that was made on the develop branch:

Note the line that begins with “branch”. Git can’t really replicate this (and doesn’t need to; both types of branch can be represented as a Git ref), but git-remote-hg needs to understand the difference, because Mercurial cares.

Creating Mercurial bookmarks is as easy as creating Git branches. On the Git side:

That’s all there is to it. On the Mercurial side, it looks like this:

Note the new [featureA] tag on revision 5. These act exactly like Git branches on the Git side, with one exception: you can’t delete a bookmark from the Git side (this is a limitation of remote helpers).

You can work on a “heavyweight” Mercurial branch also: just put a branch in the branches namespace:

Here’s what that looks like on the Mercurial side:

The branch name “permanent” was recorded with the changeset marked 7.

From the Git side, working with either of these branch styles is the same: just checkout, commit, fetch, merge, pull, and push as you normally would. One thing you should know is that Mercurial doesn’t support rewriting history, only adding to it. Here’s what our Mercurial repository looks like after an interactive rebase and a force-push:

Changesets 8, 9, and 10 have been created and belong to the permanent branch, but the old changesets are still there. This can be very confusing for your teammates who are using Mercurial, so try to avoid it.

Git and Mercurial are similar enough that working across the boundary is fairly painless. If you avoid changing history that’s left your machine (as is generally recommended), you may not even be aware that the other end is Mercurial.

It is simple to use. It is enough to clone a Bazaar repository prefixing it by bzr::. Since Git and Bazaar both do full clones to your machine, it’s possible to attach a Git clone to your local Bazaar clone, but it isn’t recommended. It’s much easier to attach your Git clone directly to the same place your Bazaar clone is attached to — the central repository.

Let’s suppose that you worked with a remote repository which is at address bzr+ssh://developer@mybazaarserver:myproject. Then you must clone it in the following way:

At this point, your Git repository is created but it is not compacted for optimal disk use. That’s why you should also clean and compact your Git repository, especially if it is a big one:

Bazaar only allows you to clone branches, but a repository may contain several branches, and git-remote-bzr can clone both. For example, to clone a branch:

And to clone the whole repository:

The second command clones all the branches contained in the emacs repository; nevertheless, it is possible to point out some branches:

Some remote repositories don’t allow you to list their branches, in which case you have to manually specify them, and even though you could specify the configuration in the cloning command, you may find this easier:

Since you are working on a project managed with Bazaar, you shouldn’t create a .gitignore file because you may accidentally set it under version control and the other people working with Bazaar would be disturbed. The solution is to create the .git/info/exclude file either as a symbolic link or as a regular file. We’ll see later on how to solve this question.

Bazaar uses the same model as Git to ignore files, but also has two features which don’t have an equivalent into Git. The complete description may be found in the documentation. The two features are:

As a consequence, there are two different situations to consider:

Whatever the case is, you will have to remain vigilant against any change of .bzrignore to make sure that the .git/info/exclude file always reflects .bzrignore. Indeed, if the .bzrignore file were to change and contained one or more lines starting with "!!" or "RE:", Git not being able to interpret these lines, you’ll have to adapt your .git/info/exclude file to ignore the same files as the ones ignored with .bzrignore. Moreover, if the .git/info/exclude file was a symbolic link, you’ll have to first delete the symbolic link, copy .bzrignore to .git/info/exclude and then adapt the latter. However, be careful with its creation because with Git it is impossible to re-include a file if a parent directory of that file is excluded.

To fetch the changes of the remote, you pull changes as usually, using Git commands. Supposing that your changes are on the master branch, you merge or rebase your work on the origin/master branch:

Because Bazaar also has the concept of merge commits, there will be no problem if you push a merge commit. So you can work on a branch, merge the changes into master and push your work. Then, you create your branches, you test and commit your work as usual. You finally push your work to the Bazaar repository:

Git’s remote-helpers framework has some limitations that apply. In particular, these commands don’t work:

Since Git’s and Bazaar’s models are similar, there isn’t a lot of resistance when working across the boundary. As long as you watch out for the limitations, and are always aware that the remote repository isn’t natively Git, you’ll be fine.

Perforce provides a product called Git Fusion (available at http://www.perforce.com/git-fusion), which synchronizes a Perforce server with Git repositories on the server side.

Git-p4 is a two-way bridge between Git and Perforce. It runs entirely inside your Git repository, so you won’t need any kind of access to the Perforce server (other than user credentials, of course). Git-p4 isn’t as flexible or complete a solution as Git Fusion, but it does allow you to do most of what you’d want to do without being invasive to the server environment.

Git-p4 makes it possible to use a Git workflow with a Perforce server, and it’s pretty good at it. However, it’s important to remember that Perforce is in charge of the source, and you’re only using Git to work locally. Just be really careful about sharing Git commits; if you have a remote that other people use, don’t push any commits that haven’t already been submitted to the Perforce server.

If you want to freely mix the use of Perforce and Git as clients for source control, and you can convince the server administrator to install it, Git Fusion makes using Git a first-class version-control client for a Perforce server.