Go Dependencies Considered Harmful

In the wake of the ongoing vendoring discussions within the Go community, I think that almost all Go projects should be able rely on having few, mature, dependencies, regular builds (almost all CI systems support some sort of cron for regular builds), and good test coverage to ensure they stay on top of their third-party dependencies.

Mature third party libraries don’t change their API hugely, and suddenly. Libraries such as github.com/lib/pq and github.com/gorilla/mux that are cornerstones of many large Go projects simply don’t become different libraries with completely different APIs overnight.

Using immature and unstable libraries is a mistake no matter how you look at it - if the library is still under development or not well maintained (i.e. breaks API compatibility every other week), why are you using it for a production application? It’s entirely the developer’s responsibility to review any third party dependency they introduce to their application - you wouldn’t introduce a random SaaS into your codebase without checking it out / asking around about it first, would you?

It’s also more than doable to build large, production applications without using any third-party code at all, for a great example of this, see Netflix’s Rend, a memcached proxy that manages data chunking and L1 / L2 caches. The core of nats, a high performance messaging system, gnatsd, is another great example with only 3 dependencies outside the main repo, on two golang.org/x/net/crypto/ packages, and github.com/nats-io/nuid, all of which are very stable/controlled dependencies.

In the event that you absolutely have to use an unstable or immature library within your application, e.g. for an exotic file format - Go supports /internal/... packages, into which you can copy these sort of dependencies, independently of any tooling or trust in the original maintainers, while ensuring they’re not importable outside your application. As Rob Pike says, a little bit of copying is better than a little bit of dependency - the ultimate way to be happy with your dependencies is to eliminate them.

Vendoring dependencies solves a particular problem (no matter the language) where having non-absolutely-reproducible builds (not just build artifacts), not being able to deploy while GitHub is down for an hour (do your CI/CD systems work independently of GitHub, too?), and fixing small, rare, API changes will cost you thousands of $CURRENCY. Otherwise, it is not an answer to using poorly maintained libraries that cause your builds to break every few hours because of your choice in libraries and maintainers. In all likelihood, vendoring is a solution to a problem you simply don’t face.

Some sort of “package manager” is equally not the solution to the problems that a lack of discipline around managing third party dependencies creates. Developers of libraries and applications alike should hold themselves to a higher standard of technical excellence; designing stable APIs and avoiding dependency trees that grow out of control. Otherwise, a tool to make things easier to manage will only mask these problems, not solve them.

Follow me on twitter, @fortytw2, or shoot me an email if you think I’ve said anything interesting here and want to chat more