Kill the Noise! to Change Gear in our Code Analyzers

by freddy mallet|

    Over the past few weeks, you may have noticed that most of our product news about code analyzers contained a mention of a “Kill The Noise!” project. We initiated this project at the beginning of the year to sharpen the detection of issues of type "Bug" on certain code analyzers: SonarJS, SonarC#, SonarJava and SonarCFamily. In simpler words, it means that our objective with this project is to make sure that when a SonarQube users clicks on “Bugs” on a project homepage he will be able to fix real and nasty bugs instead of trying to figure out whether the issues he is looking at are real bugs or not.

    That may sound like an obvious and mandatory behavior for code analyzers to be extremely sharp when reporting bugs, but do you actually know any analyzer on the market that has at least 90% accuracy?

    Over the past two years, we have developed the technology to do path-sensitive DFA (data flow analysis) on C#, Java, JavaScript, C and C++. This technology allows us to go through all execution paths while symbolically simulating the execution of each statement. With help of those DFA engines, we’re able to spot tricky bugs like this one in the Linux kernel:

    And then we realized two things :

    • A lot of our older rules of type "Bug" were not really finding obvious bugs, but were more reinforcing good coding practices that help make the code more robust, and therefore reliable. Here are few examples: Non empty switch cases should end with unconditional break statement or Two branches in a conditional structure should not have the same implementation. Those rules spot some real quality issues but most of the time such implementations were done on purpose, and don’t actually lead to unexpected behavior in production.
    • Some rules combined the detection of bugs and quality issues. That was for instance the case for the rule to detect unconditionally true/false conditions. Sounds strange? Here is the reasoning: when a branch is fully unreachable due to an unconditionally false condition there is no doubt about the fact that it is a bug. But when a (sub)condition is unconditionally true, it might be on purpose just to make the code more self-explanatory: Boom! A rule that finds both bugs and quality issues

    So we made an effort of reclassification and split of rules. Obviously we made all this while keeping the hunt for the infamous false-positives.

    At the end we are all working at SonarSource towards this ultimate goal to have code analyzers being at least 90% accurate when raising issues of type "Bug" and we are not far from making this dream become a reality. Obviously any feedback on the latest versions of SonarJS, SonarC#, SonarJava and SonarCFamily is highly welcome to help us Kill the Noise!