Appendix B: Embedding Git in your Applications

Advanced Functionality

Libgit2 has a couple of capabilities that are outside the scope of core Git. One example is pluggability: Libgit2 allows you to provide custom ``backends'' for several types of operation, so you can store things in a different way than stock Git does. Libgit2 allows custom backends for configuration, ref storage, and the object database, among other things.

Let’s take a look at how this works. The code below is borrowed from the set of backend examples provided by the Libgit2 team (which can be found at https://github.com/libgit2/libgit2-backends). Here’s how a custom backend for the object database is set up:

git_odb *odb;
int error = git_odb_new(&odb); 

git_odb_backend *my_backend;
error = git_odb_backend_mine(&my_backend, /*…*/); 

error = git_odb_add_backend(odb, my_backend, 1); 

git_repository *repo;
error = git_repository_open(&repo, "some-path");
error = git_repository_set_odb(repo, odb); 

(Note that errors are captured, but not handled. We hope your code is better than ours.)

• 1 Initialize an empty object database (ODB) frontend,'' which will act as a container for the backends'' which are the ones doing the real work.
• 2 Initialize a custom ODB backend.
• 3 Add the backend to the frontend.
• 4 Open a repository, and set it to use our ODB to look up objects.

But what is this git_odb_backend_mine thing? Well, that’s the constructor for your own ODB implementation, and you can do whatever you want in there, so long as you fill in the git_odb_backend structure properly. Here’s what it could look like:

typedef struct {
    git_odb_backend parent;

    // Some other stuff
    void *custom_context;
} my_backend_struct;

int git_odb_backend_mine(git_odb_backend **backend_out, /*…*/)
{
    my_backend_struct *backend;

    backend = calloc(1, sizeof (my_backend_struct));

    backend->custom_context = …;

    backend->parent.read = &my_backend__read;
    backend->parent.read_prefix = &my_backend__read_prefix;
    backend->parent.read_header = &my_backend__read_header;
    // …

    *backend_out = (git_odb_backend *) backend;

    return GIT_SUCCESS;
}

The subtlest constraint here is that my_backend_struct’s first member must be a `git_odb_backend structure; this ensures that the memory layout is what the Libgit2 code expects it to be. The rest of it is arbitrary; this structure can be as large or small as you need it to be.

The initialization function allocates some memory for the structure, sets up the custom context, and then fills in the members of the parent structure that it supports. Take a look at the include/git2/sys/odb_backend.h file in the Libgit2 source for a complete set of call signatures; your particular use case will help determine which of these you’ll want to support.

Other Bindings

Libgit2 has bindings for many languages. Here we show a small example using a few of the more complete bindings packages as of this writing; libraries exist for many other languages, including C++, Go, Node.js, Erlang, and the JVM, all in various stages of maturity. The official collection of bindings can be found by browsing the repositories at https://github.com/libgit2. The code we’ll write will return the commit message from the commit eventually pointed to by HEAD (sort of like git log -1).

new Repository(@"C:\path\to\repo").Head.Tip.Message;

GTRepository *repo =
    [[GTRepository alloc] initWithURL:[NSURL fileURLWithPath: @"/path/to/repo"] error:NULL];
NSString *msg = [[[repo headReferenceWithError:NULL] resolvedTarget] message];

pygit2.Repository("/path/to/repo") # open repository
    .head                          # get the current branch
    .peel(pygit2.Commit)           # walk down to the commit
    .message                       # read the message

Further Reading

Of course, a full treatment of Libgit2’s capabilities is outside the scope of this book. If you want more information on Libgit2 itself, there’s API documentation at https://libgit2.github.com/libgit2, and a set of guides at https://libgit2.github.com/docs. For the other bindings, check the bundled README and tests; there are often small tutorials and pointers to further reading there.

Getting Set Up

There are a number of ways to connect your project with JGit and start writing code against it. Probably the easiest is to use Maven – the integration is accomplished by adding the following snippet to the <dependencies> tag in your pom.xml file:

<dependency>
    <groupId>org.eclipse.jgit</groupId>
    <artifactId>org.eclipse.jgit</artifactId>
    <version>3.5.0.201409260305-r</version>
</dependency>

The version will most likely have advanced by the time you read this; check https://mvnrepository.com/artifact/org.eclipse.jgit/org.eclipse.jgit for updated repository information. Once this step is done, Maven will automatically acquire and use the JGit libraries that you’ll need.

If you would rather manage the binary dependencies yourself, pre-built JGit binaries are available from https://www.eclipse.org/jgit/download. You can build them into your project by running a command like this:

javac -cp .:org.eclipse.jgit-3.5.0.201409260305-r.jar App.java
java -cp .:org.eclipse.jgit-3.5.0.201409260305-r.jar App

Plumbing

JGit has two basic levels of API: plumbing and porcelain. The terminology for these comes from Git itself, and JGit is divided into roughly the same kinds of areas: porcelain APIs are a friendly front-end for common user-level actions (the sorts of things a normal user would use the Git command-line tool for), while the plumbing APIs are for interacting with low-level repository objects directly.

The starting point for most JGit sessions is the Repository class, and the first thing you’ll want to do is create an instance of it. For a filesystem-based repository (yes, JGit allows for other storage models), this is accomplished using FileRepositoryBuilder:

// Create a new repository
Repository newlyCreatedRepo = FileRepositoryBuilder.create(
    new File("/tmp/new_repo/.git"));
newlyCreatedRepo.create();

// Open an existing repository
Repository existingRepo = new FileRepositoryBuilder()
    .setGitDir(new File("my_repo/.git"))
    .build();

The builder has a fluent API for providing all the things it needs to find a Git repository, whether or not your program knows exactly where it’s located. It can use environment variables (.readEnvironment()), start from a place in the working directory and search (.setWorkTree(…).findGitDir()), or just open a known .git directory as above.

Once you have a Repository instance, you can do all sorts of things with it. Here’s a quick sampling:

// Get a reference
Ref master = repo.getRef("master");

// Get the object the reference points to
ObjectId masterTip = master.getObjectId();

// Rev-parse
ObjectId obj = repo.resolve("HEAD^{tree}");

// Load raw object contents
ObjectLoader loader = repo.open(masterTip);
loader.copyTo(System.out);

// Create a branch
RefUpdate createBranch1 = repo.updateRef("refs/heads/branch1");
createBranch1.setNewObjectId(masterTip);
createBranch1.update();

// Delete a branch
RefUpdate deleteBranch1 = repo.updateRef("refs/heads/branch1");
deleteBranch1.setForceUpdate(true);
deleteBranch1.delete();

// Config
Config cfg = repo.getConfig();
String name = cfg.getString("user", null, "name");

There’s quite a bit going on here, so let’s go through it one section at a time.

The first line gets a pointer to the master reference. JGit automatically grabs the actual master ref, which lives at refs/heads/master, and returns an object that lets you fetch information about the reference. You can get the name (.getName()), and either the target object of a direct reference (.getObjectId()) or the reference pointed to by a symbolic ref (.getTarget()). Ref objects are also used to represent tag refs and objects, so you can ask if the tag is ``peeled,'' meaning that it points to the final target of a (potentially long) string of tag objects.

The second line gets the target of the master reference, which is returned as an ObjectId instance. ObjectId represents the SHA-1 hash of an object, which might or might not exist in Git’s object database. The third line is similar, but shows how JGit handles the rev-parse syntax (for more on this, see Branch References); you can pass any object specifier that Git understands, and JGit will return either a valid ObjectId for that object, or null.

The next two lines show how to load the raw contents of an object. In this example, we call ObjectLoader.copyTo() to stream the contents of the object directly to stdout, but ObjectLoader also has methods to read the type and size of an object, as well as return it as a byte array. For large objects (where .isLarge() returns true), you can call .openStream() to get an InputStream-like object that can read the raw object data without pulling it all into memory at once.

The next few lines show what it takes to create a new branch. We create a RefUpdate instance, configure some parameters, and call .update() to trigger the change. Directly following this is the code to delete that same branch. Note that .setForceUpdate(true) is required for this to work; otherwise the .delete() call will return REJECTED, and nothing will happen.

The last example shows how to fetch the user.name value from the Git configuration files. This Config instance uses the repository we opened earlier for local configuration, but will automatically detect the global and system configuration files and read values from them as well.

This is only a small sampling of the full plumbing API; there are many more methods and classes available. Also not shown here is the way JGit handles errors, which is through the use of exceptions. JGit APIs sometimes throw standard Java exceptions (such as IOException), but there are a host of JGit-specific exception types that are provided as well (such as NoRemoteRepositoryException, CorruptObjectException, and NoMergeBaseException).

Porcelain

The plumbing APIs are rather complete, but it can be cumbersome to string them together to achieve common goals, like adding a file to the index, or making a new commit. JGit provides a higher-level set of APIs to help out with this, and the entry point to these APIs is the Git class:

Repository repo;
// construct repo...
Git git = new Git(repo);

The Git class has a nice set of high-level builder-style methods that can be used to construct some pretty complex behavior. Let’s take a look at an example – doing something like git ls-remote:

CredentialsProvider cp = new UsernamePasswordCredentialsProvider("username", "p4ssw0rd");
Collection<Ref> remoteRefs = git.lsRemote()
    .setCredentialsProvider(cp)
    .setRemote("origin")
    .setTags(true)
    .setHeads(false)
    .call();
for (Ref ref : remoteRefs) {
    System.out.println(ref.getName() + " -> " + ref.getObjectId().name());
}

This is a common pattern with the Git class; the methods return a command object that lets you chain method calls to set parameters, which are executed when you call .call(). In this case, we’re asking the origin remote for tags, but not heads. Also notice the use of a CredentialsProvider object for authentication.

Many other commands are available through the Git class, including but not limited to add, blame, commit, clean, push, rebase, revert, and reset.

Further Reading

This is only a small sampling of JGit’s full capabilities. If you’re interested and want to learn more, here’s where to look for information and inspiration:

• The official JGit API documentation can be found at https://www.eclipse.org/jgit/documentation. These are standard Javadoc, so your favorite JVM IDE will be able to install them locally, as well.

• The JGit Cookbook at https://github.com/centic9/jgit-cookbook has many examples of how to do specific tasks with JGit.

Advanced Functionality

go-git has few notable advanced features, one of which is a pluggable storage system, which is similar to Libgit2 backends. The default implementation is in-memory storage, which is very fast.

r, err := git.Clone(memory.NewStorage(), nil, &git.CloneOptions{
    URL: "https://github.com/src-d/go-git",
})

Pluggable storage provides many interesting options. For instance, https://github.com/src-d/go-git/tree/master/_examples/storage allows you to store references, objects, and configuration in an Aerospike database.

Another feature is a flexible filesystem abstraction. Using https://godoc.org/github.com/src-d/go-billy#Filesystem it is easy to store all the files in different way i.e by packing all of them to a single archive on disk or by keeping them all in-memory.

Another advanced use-case includes a fine-tunable HTTP client, such as the one found at https://github.com/src-d/go-git/blob/master/_examples/custom_http/main.go.

customClient := &http.Client{
	Transport: &http.Transport{ // accept any certificate (might be useful for testing)
		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
	},
	Timeout: 15 * time.Second,  // 15 second timeout
		CheckRedirect: func(req *http.Request, via []*http.Request) error {
		return http.ErrUseLastResponse // don't follow redirect
	},
}

// Override http(s) default protocol to use our custom client
client.InstallProtocol("https", githttp.NewClient(customClient))

// Clone repository using the new client if the protocol is https://
r, err := git.Clone(memory.NewStorage(), nil, &git.CloneOptions{URL: url})

Further Reading

A full treatment of go-git’s capabilities is outside the scope of this book. If you want more information on go-git, there’s API documentation at https://godoc.org/gopkg.in/src-d/go-git.v4, and a set of usage examples at https://github.com/src-d/go-git/tree/master/_examples.

Further Reading

• The official API documentation is available at https://www.dulwich.io/apidocs/dulwich.html

• Official tutorial at https://www.dulwich.io/docs/tutorial has many examples of how to do specific tasks with Dulwich