SonarSource is taking Python analysis by storm in 2020

by g. ann campbell|

Why Python?

For a lot of years, SonarSource implemented a few Python rules, calculated metrics, and displayed coverage, but mainly we just wrapped PyLint and called it a day. There's no real shame in that; PyLint is a mainstay and a lot of folks rely on it. But now we've realized that "okay" isn't good enough. So we've set a goal for 2020. By the end of the year, SonarQube, SonarCloud, and SonarLint should be your go-tos for Python Code Quality and Security. In other words, our Python analysis will be Kicking Asp and Taking Names. (See what I did there? :-D)

"But wait", I hear you saying. "The Python ecosystem is already littered with linters. What makes you think you have something to add?"

Great question! First, we'll offer fast, highly accurate analysis with minimal configuration, and plenty of really valuable rules that other tools don't offer (more on that in a minute). We'll present the issues raised by those rules in our highly-evolved UX. It's so good that some Python developers had already adopted SonarQube purely to see their code coverage. And while our coverage presentation is good, our issues display is even better. Precise issue locations mean we guide you to the exact symbol or expression in question. And when the primary location isn't enough to understand the issue, we highlight the secondary locations you need to understand what's going on (you’ll see an example of that in a minute).

There are a lot more benefits that come out of the box with SonarQube and SonarCloud, and I'll get to them. But first, let's get back to Kicking Asp and Taking Names. 

Beefing up our analysis engine

How will we get there? Well, we've started by improving the underlying technology. We've implemented:

  • Type inference - so we know for example when a variable holds a string versus a boolean.
  • A control flow graph - so we can follow the execution path through the code - for instance to recognize an infinite loop
  • A project-level symbol table - so we know how many parameters your functions take, even when they're in different modules.

In other words, we've worked hard to make the analyzer really understand your code. That let's us implement rules like these:

  • S2190 - Recursion should not be infinite
  • S2159 - Silly equality checks should not be made
  • S5727 - Comparison to None should not be constant
  • S930 - The number and name of arguments passed to a function should match its parameters

Of course, those are just a downpayment. We're planning support for keyword arguments, plus other valuable rules, like:

  • Iterable unpacking, "for-in" loops and "yield from" should use an iterable object
  • Item operations should be done on objects supporting them
  • Loops with at most one iteration should be refactored
  • Operators should be used on compatible types

In addition to the more than 100 rules that are already available, we've also added more than a dozen rules related to exception handling, such as S5712 - Some special methods should return "NotImplemented" instead of raising "NotImplementedError".

Python Code Security

It doesn't stop at Code Quality (Bugs and Code Smells). We're tackling Python Code Security (Security Hotspots and Vulnerabilities) this year too. Already we support Django and Flask in rules covering seven of the OWASP Top 10 categories: A1, A2, A3, A5, A6, and A7, including nearly a dozen injection-detection (A1) rules; and more in the works. (More on this later; watch this space.)

In addition to beefing up the underlying technology to offer great new rules, we'll also offer good integration; we don't suffer from "Not Invented Here", so adding SonarQube or SonarCloud to your build means you don't have to give up what you're used to. For instance, pycodestyle is the standard style checker. It would be silly for us to reimplement those rules, so we'll import from pycodestyle (and others, of course) via Flake8. 

Of course, we'll make all this available in both SonarQube and SonarCloud, with the same sophisticated issue inspection interface, Security Hotspot review, and metric drilldowns already available for all the other languages. Plus, there'll be detailed rule descriptions to help you understand the issues we raise and why you want to fix them, PR decoration in commercial editions, and automatic analysis of the other languages in your project.

Similarly, SonarLint - either standalone or connected to your project in SonarQube or SonarCloud - gives you in-IDE checking so you can avoid ever committing new issues in the first place.

It's been nearly eight years since SonarSource first ventured into Python static analysis. For most of that time it was secondary at best. But this year, it comes into its own. This year, it'll start Kicking Asp and Take Names. Count on it.