Checkmk: Remote Code Execution by Chaining Multiple Bugs (3/3)

by stefan schiller|

This is the third and last article in the Checkmk - Remote Code Execution by Chaining Multiple Bugs series (first article, second article). Within the series of articles, we take a detailed look at multiple vulnerabilities we identified in Checkmk and its NagVis integration, which can be chained together by an unauthenticated, remote attacker to fully take over the server running a vulnerable version of Checkmk.

In the last article, we evaluated the ability of an attacker to forge arbitrary LQL queries. This allows the attacker to exfiltrate monitoring data and issue external Nagios commands, which can be leveraged to delete arbitrary files. We could demonstrate that this ability could be combined with a file race condition to bypass the authentication of the NagVis component. 

In this third and last article, we complete our deep dive into the technical details of the vulnerability chain. At this point, the attacker has gained access to the NagVis component. Based on this, we will outline how the attacker can escalate this access to the Checkmk GUI itself by exploiting an authenticated file read vulnerability in NagVis.

At last, we take a detailed look at an authenticated code injection vulnerability in Checkmk, which forms the final step to remote code execution.

Technical Details

We start this section by briefly recapping the vulnerabilities and exploitation chain. After this, we look at the arbitrary file read vulnerability in NagVis and the code injection vulnerability in Checkmk.

Exploitation Chain

As a reminder, the following picture summarizes the exploitation chain enabling an unauthenticated attacker to gain remote code execution:

In the last two articles, we covered the first two vulnerabilities (1, 2) and an arbitrary file deletion, which can be exploited by an unauthenticated attacker to gain access to the NagVis component. Within this article, we determine how an attacker can escalate to the Checkmk automation user by exploiting an authenticated arbitrary file read in NagVis (3). With access to the Checkmk automation user, an attacker can ultimately gain code execution by exploiting a code injection vulnerability in Checkmk’s watolib (4):

Arbitrary File Read in NagVis

After an attacker has gained access to NagVis, the exposed attack surface is greatly increased because authenticated endpoints can now be accessed. For one of these endpoints, our automatic scan with SonarCloud discovered an interesting path injection vulnerability:

Try it by yourself in SonarCloud!

The endpoint is implemented in the CoreModGeneral class. This class offers different actions which an authenticated user can trigger. One of these actions is called getHoverUrl:

share/nagvis/htdocs/server/core/classes/CoreModGeneral.php

class CoreModGeneral extends CoreModule {
   ...
   public function handleAction() {
       $sReturn = '';

       if($this->offersAction($this->sAction)) {
           switch($this->sAction) {
               ...
               case 'getHoverUrl':
                   $sReturn = $this->getHoverUrl();
               break;
           ...

Within the getHoverUrl method, getCustomOptions is called to retrieve user-provided GET and POST parameters. In this case, the parameter url is retrieved, which is supposed to be an array containing URLs. For each provided URL, a new NagVisHoverUrl object is created. The response, which is stored in $arrReturn, contains the requested URL (url) as well as the string representation of the NagVisHoverUrl object (code):

share/nagvis/htdocs/server/core/classes/CoreModGeneral.php

   private function getHoverUrl() {
       $arrReturn = Array();

       // Parse view specific uri params
       $aOpts = $this->getCustomOptions(Array('url' => MATCH_STRING_URL));

       foreach($aOpts['url'] AS $sUrl) {
           $OBJ = new NagVisHoverUrl($this->CORE, $sUrl);
           $arrReturn[] = Array('url' => $sUrl, 'code' => $OBJ->__toString());
       }

       $result = json_encode($arrReturn);
       ...
       return $result;
   }

Within the constructor of the NagVisHoverUrl class, the method readHoverUrl is called.

This method uses file_get_contents to retrieve the requested URL:

share/nagvis/htdocs/server/core/classes/NagVisHoverUrl.php

   private function readHoverUrl() {
       ...
       if(!$content = file_get_contents($this->url)) {
           throw new NagVisException(l('couldNotGetHoverUrl', Array('URL' => $this->url)));
       }
       ...
       $this->code = $content;
   }

Since an authenticated user can fully control the URLs provided, the getHoverUrl action can be used to read arbitrary files by using the file:/// scheme.

This vulnerability further increases the attacker’s ability to read arbitrary files accessible by the webserver user. The impact depends on the presence of accessible files with sensitive content. Unfortunately, for automation users, these files exist.

Checkmk Automation Users

Checkmk provides two types of user accounts: normal users and automation users. A normal user has a regular password and can log in to the GUI. An automation user can be used as a convenient way to automate certain activities that would normally be done via the GUI. Instead of a regular password, an automation user is authenticated by an automation secret. This secret can usually not be used to log in to the GUI but is provided as an additional GET parameter to the accessed endpoint.

The default automation user is called automation and is preconfigured with a random secret. The hash of this secret and the hash of regular passwords are by default stored in an htpasswd file:

Though, the secret is additionally stored in a plaintext file, which is called automation.secret:

Since the file contains the plaintext secret, the aforementioned arbitrary file read vulnerability can be leveraged by an attacker to retrieve it without requiring to crack the hash stored in the htpasswd file.

Although this secret can be used to access authenticated endpoints, it cannot be used to log in to the GUI with it. Let’s have a look at the corresponding code. When a user logs in, the function check_credentials is called:

checkmk/cmk/gui/userdb/htpasswd.py

   def check_credentials(self, user_id: UserId, password: str) -> CheckCredentialsResult:
       ...
       if self._is_automation_user(user_id):
           raise MKUserError(None, _("Automation user rejected"))
       ...

As we can see, the function _is_automation_user checks if the provided user_id corresponds to an automation user. If that is the case, an error is raised, and the GUI login fails. This is what the _is_automation_user function looks like:

checkmk/cmk/gui/userdb/htpasswd.py

   def _is_automation_user(self, user_id: UserId) -> bool:
       return Path(cmk.utils.paths.var_dir, "web", str(user_id), "automation.secret").is_file()

Accordingly, the presence of the automation.secret file is used in order to determine if the user is an automation user.

By leveraging the Linefeed Injection vulnerability and the Nagios PROCESS_FILE command outlined in the second article, an attacker has not only the ability to read arbitrary files but also to delete them. This means that the attacker can delete the automation.secret file after reading it. Since the login process verifies the provided credentials via the htpasswd file and the automation.secret file is not present, the automation user is assumed to be a normal user, and access to the GUI is granted:

After the successful login, an attacker can exploit an authenticated code injection vulnerability.

Code Injection watolib auth.php

In order to seamlessly integrate NagVis into Checkmk, a file called auth.php is generated, which contains information about users, roles, and groups present in the Checkmk GUI. This file is updated when the corresponding data changes (e.g., user settings) by a function called _create_auth_file. This function loads the required data and calls _create_php_file:

checkmk/cmk/gui/watolib/auth_php.py

def _create_auth_file(callee, users=None):
   if users is None:
       users = userdb.load_users()
   ...
   _create_php_file(callee, users, get_role_permissions(), groups)

Within _create_php_file the content of the auth.php file is created and written to disk. In order to format the user data, the function _format_php is called:

checkmk/cmk/gui/watolib/auth_php.py

def _create_php_file(callee, users, role_permissions, groups):
   # Do not change WATO internal objects
   nagvis_users = copy.deepcopy(users)
   ...
   content = """<?php
// Created by Multisite UserDB Hook (%s)
global $mk_users, $mk_roles, $mk_groups;
$mk_users   = %s;
...
?>
""" % (
       callee,
       _format_php(nagvis_users),
       ...
   )

   store.makedirs(_auth_php().parent)
   store.save_text_to_file(_auth_php(), content)

The function _format_php converts the given data into the corresponding PHP representation. Data of type str is inserted into a single-quoted string. Single quotes within the data itself are escaped by prepending a backslash (\) to prevent the string context can be escaped:

checkmk/cmk/gui/watolib/auth_php.py

def _format_php(data, lvl=1):
   s = ""
   ...
   elif isinstance(data, str):
       s += "'%s'" % data.replace("'", "\\'")
   ...

The replacement does not take into account that the data can contain a backslash itself, followed by a single quote (\'). When encountering this sequence, the single quote is prepended by a backslash, which is escaped by the already present backslash (\\'). This way the string context can be escaped and arbitrary PHP code can be injected into the file.

An attacker can exploit the vulnerability after authenticating with the default automation user and then changing the profile settings. After the auth.php file is automatically updated, it contains the attacker-injected PHP code. The attacker now only needs to access the NagVis component, which includes the auth.php file and executes the injection code.

Patch

The arbitrary file read vulnerability was patched in NagVis 1.9.34, which was integrated into Checkmk version 2.1.0p11 by limiting the requested scheme to http and https:

nagvis/share/nagvis/htdocs/server/core/classes/NagVisHoverUrl.php

  private function readHoverUrl() {
      ...
      $aUrl = parse_url($this->url);
      if(!isset($aUrl['scheme']) || $aUrl['scheme'] == '' || ($aUrl['scheme'] != 'http' && $aUrl['scheme'] != 'https'))
          throw new NagVisException(l('problemReadingUrl', Array('URL' => $this->url, 'MSG' => l('Not allowed url'))));
      ...

The code injection vulnerability was patched with Checkmk version 2.1.0p11 by escaping both single-quote characters and backslash characters (commit):

checkmk/cmk/gui/watolib/utils.py

def format_php(data: object, lvl: int = 1) -> str:
   """Format a python object for php"""
   s = ""
   ...
   elif isinstance(data, str):
       s += "'%s'" % re.sub(r"('|\\)", r"\\\1", data)
   ...

Timeline

DateAction
2022-08-22We report all issues to Checkmk.
2022-08-23Vendor confirms all issues.
2022-08-29NagVis patched version 1.9.34 is released.
2022-08-30Checkmk version 2.1.0p11 is released containing NagVis 1.9.34.

Summary

In this last article in the series, we detailed an authenticated, arbitrary file read vulnerability in NagVis, which enables an attacker to gain access to the Checkmk automation user. We further took a look at how Checkmk identifies automation users. This revealed that an attacker could leverage the arbitrary file deletion once more to gain access to the Checkmk GUI. This access can further be leveraged to exploit a code injection vulnerability in Checkmk’s watolib.

The arbitrary file read vulnerability is caused by a missing validation of the URL scheme. The impact of this vulnerability is greatly increased because the automation secret is stored in plaintext. Whether it be a file or a database, sensitive values, which can directly be used by an attacker to gain more privileges, should not be stored in plaintext. These sensitive values can for example be passwords, authentication tokens, or password reset tokens.

Dynamic code generation, like creating PHP files, can be very dangerous and should be avoided if possible. There is no built-in method that escapes values in the context of code generation for another language. Thus a custom implementation is required, and some cases can easily be missed. The outlined code injection vulnerability showed that a single mistake in the escaping implementation directly leads to code execution.

Series Wrap-Up

This article completes the Checkmk - Remote Code Execution by Chaining Multiple Bugs series. The series showcased how an attacker successively gained more abilities and access by chaining one vulnerability after another.

In general, web applications have become more secure in the past few years. Vulnerabilities instantly leading to remote code execution are far less common. This requires attackers to leverage less impactful vulnerabilities and chain them together. These chains are often only possible because the security precautions tend to be lower the higher the level of authentication.

The assumption that an attacker lacks a particular ability is dangerous and can quickly lead to a domino effect when an initial security boundary is breached. It is essential to apply security on all layers. Even one seemingly unimportant, additional security check can mitigate one link in an exploit chain and thus break the whole chain.

This is why Sonar believes in the Clean Code approach, which embeds security as an integral part of the development. Handling security issues should not be a painful aftermath. Directly addressing and preventing these when the code is being developed saves time, work, and frustration. Our unique Clean as You Code approach addresses issues upfront, and no new issues end up in the released code. If you haven’t discovered the power of the Sonar solution yet, you can learn more here.

Finally, we would like to highlight the professional reaction of the Checkmk team. There are security issues in each and every software. The difference is how these issues are dealt with. All of our reported issues were quickly verified, handled with absolute transparency, and fixed by providing comprehensive patches. Thank you!

Related Blog Posts