Building Go projects with gb
gb is a new build tool for Go created by Dave Cheney. It address the problem of reproducible builds. Building the same functional binary anywhere at any time is a problem of dependency management. Of knowing exactly which library version to use, and having it at hand.
gb is a radical tool compared to current dependency solutions, which work with the existing Go toolchain and idioms.
gb takes a different approach, it replaces not only dependency management tools, but the Go build tools themselves. Instead of using
go build or
go install you would use
Dependency management in Go has always been a little wonky, a little weird. From the beginning there was
go get but no other specific tools or process.
go get infers a location from the import path about where libraries live, and grabs them. Placing them ever so lovingly in the
src directory along side project code.
go get knows enough about DVCS to grab the HEAD code, but not much more. There are no options for specifying which, if any, versions should be used, or commit hashes to checkout. The idea was to simply keep everything up to date manually. Keeping everything at HEAD sounds nice, but in practice it has proved problematic.
Realizing this wasn't a good long-term solution, the Go Team called upon the community to develop ideas and play with solutions. A wonderful idea indeed; practicing and playing will give us better solutions than just pie in the sky thinking would. The results of this are now coming to fruition, the Go Team has called for Vendoring to be the community's solution to dependency management. They called for a consensus to be reached on a file format for tools doing to dependency management to interoperate.
The idea behind
gb's solution for reproducible builds is based on the idea that the
go get command, and associated handling of dependencies, is fundamentally flawed.
gb suggests a new way to handle dependencies and build Go applications.
go build because it has to.
gb stores Go code in two places; first in $GOPATH/src where
main.main() is, this represents the project, and then in $GOPATH/vendor/src where 3rd party dependencies are kept.
gb looks in both of these location when building the project binary, where
go build does not.
gb works on projects. A project is “defined as any directory that has a
$GOPATH/src are two examples, each one it's own project.
gb a project is an end goal, not to be consumed by other code. Not to be imported and used. Projects are not go get'able. They are where
main() lives, they compile to a binary you intend to run.
gb does not use a configuration file, the source and code structure alone is enough to build a project. Dave Cheney is not opposed to having one; in fact he would rather have one than not. But opted to go without due to lack of consensus by the community on the format.
Respect the Source
gb does not do any import rewriting. This is because it has two separate locations for code.
gb checks in both
vendor/src when looking for the source of an import, where
go build does not. Tools like godeps have to rewrite import paths for its vendored libraries otherwise the build when fail when
go build can't find them.
I like that I don't have to do import rewriting for my dependency's dependencies. I want to use a library and never have to change it from what the upstream devs intended. This hands-off approach carries over to canonical import paths as well. They are left untouched.
I really like that 3rd party code is separated out into its own directory, and all libraries are flatly organized in that directory. I can look at
vendor/src and see all of my dependencies. I don't have to go hunting and pecking for a vendored package of a vendored package.
The main goal for
gb is reproducible builds. It manages this by vendoring. Which takes libraries the project needs and makes them a part of the
gb project, and commits them to the projects source control repository. Vendored packages are stored in
Any new checkout of the project will have all dependencies needed to build. If one is so daring there you can use git submodules/subtrees to store the dependeny’s history.
gb, in it’s current state, does not manage dependency or which version are used. Rather it delegates this responsibility to
gb-vendor, a plugin to the main
gb-vendor itself is not special; it is a wrapper around
go get -d. It’s only purpose is to grab dependencies that are go-get’able and drop them into the project’s vendor directory at
gb-vendor is a quick proof of concept that Dave threw together. It lacks many features of a tradition DVCS; which puts it in a tricky position. What is
gb-vendor’s role in managing dependency version and how much work should it do before pawning off to a proper DVCS.
gb is straightforward to use. First you vendor your dependencies, then you build your application.
or you can use
gb build all to build everything in your
A note about
gb vendor; it was a quick plugin that Dave threw together to make his talk go smother. It may or may not stay around. What it actually does is in flux, and its future really depends on how developers use it, and want to use it. There is an open ticket on github for discussing `gb vendor’.
You can find detailed examples in the getting started documentation on github.
A turn in the road
gb. I've played with it for a couple of days, and it aligns with my personal preferences for Go development and project structure; more so than other dependency solutions currently available. It feels light-weight. It gets out of my way, and lets me manage my code without interfering. It feels good.
gb feels like a departure of sorts; but it isn't so different than the core Go assumptions about code structure. Rather it looks at them in a different way. I can't help but feel that this is some sort of significant moment in Go history. One we will look back upon; with reverence or trepidation I don't know, only time will tell. For now, though, I will use it to build my projects.