GitHub pull request analysis helps fix the leak

by fabrice bellingard|

    If you follow SonarSource, you are probably aware of a simple and yet powerful paradigm that we're using internally: the water leak concept. That is how we've been working on a daily basis at SonarSource since a couple of years already, using various features of SonarQube like "New Issues" notifications"Since previous version" differential period, and quality gates. These features allows us to make sure that no technical debt is introduced on new code. More recently, we have developed a brand new plugin to go even further in this direction: the SonarQube GitHub Plugin.

    Analysing GitHub pull requests to detect new issues

    At SonarSource, we use GitHub to manage our codebase. Every bug fix, improvement, and new feature is developed in a Git branch and managed through a pull request on GitHub. Each pull request must be reviewed by someone else on the team before it can be merged into the master branch. Previously, it was only after the merge and the next analysis (every master branch is analysed on our internal SonarQube instance several times a day) that SonarQube feedback was available, possibly leading to another pull request-review cycle. "Wouldn't it be great" we thought, "if the pull request could be reviewed not only by a teammate, but also by SonarQube itself before being merged?" That way, developers would have the opportunity to fix potential issues before they could be injected into the master branch (and reported on the SonarQube server).

    This is what we achieved with the new SonarQube GitHub Plugin. Basically, every time a pull request is submitted by a member of team, the continuous integration system launches a SonarQube preview analysis with the parameters to activate the GitHub plugin, so that:

    1. When the SonarQube analysis starts, the GitHub plugin updates the status of the pull request to mention that there's a pending analysis
    2. Then SonarQube executes all the required language plugins
    3. And at the end, the GitHub plugin:
      • adds an inline comment for each new issue,
      • adds a global comment with a summary of the analysis,
      • and updates the status of the pull request, setting it to "failed" if at least one new critical or blocker issue was found.

    Here's what such a pull request looks like (click to enlarge):

    Thanks to the GitHub plugin, developers get quick feedback as a natural, integrated part of their normal workflow. When a GitHub analysis shows new issues, developers can choose to fix the issues and push a new commit - thus launching a new SonarQube analysis. But in the end, it is up to the developer whether or not to merge the branch into the master, whatever the status of the pull request after the analysis. The SonarQube GitHub plugin provides feedback, but the power remains where it belongs - in the hands of the developers.

    What's next?

    Now that the integration with GitHub has proven to be really useful, we feel that doing a similar plugin for Atlassian Stash would be valuable, and writing it should be quite straightforward.

    Also, analysing pull requests on GitHub is a great step forward, because it gives early feedback on incoming technical debt. But obviously, developers would like to have this feedback even earlier: in their IDEs. This is why in the upcoming months, we will actively work on the Eclipse and IntelliJ plugins to make sure they allow developers to efficiently fix issues before commit and adopt the "water leak" approach wholesale. To achieve this target, we'll update the plugins to run SonarQube analyses in the blink of an eye for instantaneous feedback on the code you are developing.