GitHub Dependabot in Action

I’ve used this awesome tool on 20 open-source projects over the last two years. Here’s my opinion.

Typical problems Dependabot solves range from updating direct upstream dependencies when they get a patch release to propagating the security vulnerabilities your upstream projects have already fixed. If you have minimal time and don’t have a whole team of people to manage dependencies, Dependabot is a solution for you. If you have more than 10–20 dependencies in a project with more than 30–50 thousand lines of code, you must take open-source dependencies seriously. Taking them seriously is also proportional to the number of repositories you have.

But disregard that last line: you must take open-source dependencies seriously, no matter what. If you want to read my detailed opinion about it, please read the following article .

What Does the Workflow With Dependabot Look Like?

Make sure you have enough automated testing coverage. This is the first and most important thing because every time there is a new upstream change anywhere in your dependency tree, the tool creates a pull request into your repository. It works the same way for public and internal repositories.

Great, now what? Frequently there’s more than one PR from Dependabot, so it’s recommended to put off at least 15–30 minutes to code review each incoming change. The first thing you need to pay attention to is the changelog that’s part of the description. This should give you an initial idea of whether this change breaks your project.

I would also recommend reading the full diff of upstream change because it will provide you with two benefits. First, you will become more familiar with the things you depend on. The second is that you’ll know precisely what was changed, giving insight into what might have caused the potential breakage. I have learned it the hard way, so you don’t have to.

Unit test coverage greater than 70% is recommended before you get Dependabot integrated. The more test coverage, the greater the chance of catching any issue at a dependency bump. You can track code coverage with services like codecov.io . It automates reporting of coverage changes through pull request comments.

Usually, Dependabot provides more than one pull request every time it runs. I recommend merging them sequentially — one after another. This will give you confidence that there is no global dependency conflict. The more unit test coverage you have, the more confident you will accept these changes.

Squashing is the best way to merge because each dependency bump appears as a single commit in the source tree. Great, you’ve merged one PR out of many. Now, what to do with other dependency bumps? It usually detects changes in the main upstream branch and tries to recreate or rebase a pull request automatically. However, I don’t find it happening fast enough, so I update the branch manually.

I haven’t yet found an easy GitHub public REST API to automate it. But I don’t complain much yet; it doesn’t take more than 1–2 hours a week to keep all dependencies up to date everywhere.

How often should you schedule it to run? My recommendation will be a daily cycle: this will ensure security vulnerabilities are patched fast enough and spread your code review time evenly across the week. If the upstream project changes more frequently than you manage to bump the versions, fear not: the automation is smart enough to handle even that.

Where Does Dependabot Break?

So, how much unit test coverage should the project have to be ready for this kind of automation ? It’s so difficult to say. For example, I’m the lead maintainer of a relatively large Terraform provider with 10 million downloads. It has 90% unit test coverage with all of the external contributions. Once it introduced a change from the upstream SDK, the unit tests passed just fine.

A few days later, I saw a bug report that the newest release version crashed with a segmentation fault. Luckily, I release the project almost every other week, and it’s straightforward to bisect the changes to the most probable candidates. If you release so often, your change surface is typically small. It was immediately apparent that the reason for the crash occurred after reviewing the diff in the upstream change — my favorite null pointer exception, where developers forget to program defensively and don’t check if objects are correctly initialized. I reverted the bump immediately and created a new release after the integration tests had passed. The moral of the story:

If you depend on other projects with lower unit test coverage, you should hope for the best and plan for the worst. 90% coverage cannot save you from situations like this one. What will save you is the speed of how fast can you do a new release confidently.

The Dependabot is excellent for the open-source ecosystem because it makes it more secure and vibrant. Every project with it is alive (or looks live on the surface). Start using it right now.For more content, follow the author on Medium, browse nfx on GitHub, or @nf_x on Twitter.