3.3. Application Hardening

The Concept of Application Hardening

Application Hardening is a security feature designed to prevent exploitation of various types of vulnerabilities in software applications (that have not been patched yet or the patch has not been developed yet). Kerio's Application Hardening is based a capability of stopping even attacks targeting new or unpublished vulnerabilities (as opposed to signature based Intrusion Detection Systems).

To explain how Application Hardening works, it is necessary to understand how application vulnerabilities are exploited. Vulnerabilities are mostly introduced by programmers that fail to check properly the input data entering into the application. If an application contains vulnerability, it can be exploited by an attacker. In the worst case, the attacker can take over the control of the computer on which the vulnerable application is running. In other words, a vulnerable application becomes a backdoor to the system that can be exploited by hackers or an automatically spreading Internet worms.

However, the vast majority of Internet worms and hackers exploiting the system through a vulnerable application must perform certain actions that are relatively easily distinguishable and detectable. For example, to take control over the system, the attacker must copy and launch a backdoor program on the vulnerable system. Kerio ServerFirewall's Application Hardening feature offers defense mechanisms designed to recognize and stop these actions.

Application Hardening in Kerio ServerFirewall is implemented in form of rules defining which applications should be protected and how. The rules must be defined by the Administrator. The reason why Application Hardening cannot be applied globally on all applications is that not every application running on the OS is suitable for this type of protection.

Application Hardening provides three types of defense mechanisms explained below.

The three defenses

1st Defense: Process spawning control

One of the methods used by hackers or worms to break into the system is to trick the vulnerable application into spawning executable files of their choice on the system. The Process spawning control defense uses the fact that in most cases, the application does not need the ability to launch other executables for its proper function. By taking away the process spawning ability from the application, hackers will not be able to perform process spawning attack.

2nd Defense: Executable files protection

Another method used to break into the system is to trick the vulnerable application into modifying or creating executable files of their choice on the system (to copy the worm's main code). Then the worm gets eventually executed and activated.

The Executable files protection defense is based on the fact that in most cases, the application does not need to create or modify executable files for its proper function. By disallowing the application to modify executable files, hackers will not be able to perform attacks tampering with executable files on the system.

3rd Defense: System tampering protection

Yet another method used to break into the system is to trick the vulnerable application into modifying special sensitive areas of the operating system and taking advantage of those modifications. Those sensitive areas for example include Windows Registry keys used to control launching of applications upon system startup, the system.ini and win.ini files etc. The System tampering protection defense is based on the fact that in almost all cases normal applications do not need to perform such operations for their proper function. By not allowing applications to modify special areas of the OS, hackers will not be able to perform attacks tampering with the sensitive special areas of the system.

In the Application Hardening rule, all protections of this kind are grouped under a single option.

Exclusions

Some applications may have legitimate needs to perform actions that are normally disallowed by the enabled defenses. For example, an FTP service process may need the ability to create and modify executable files in its upload directory. For these situations, the Application Hardening allows definition of per-process exceptions for Process spawning control and Executable files protection defenses. The exception can be set specifically for the exact action of the application so that the effectiveness of the defense is not compromised.

Logging is important

Actions captured by Application Hardening rule can be logged into a log file. It is recommended that the logging is enabled for all created rules. If a hacking attempt is detected, the log file contains valued information about the event.

Setting up hardening rules

The key approach when setting up application hardening rules is:

  • Identify applications that are potentially at risk because they process data from insecure sources (web servers, mail servers...).

    The Network Status page can be of great help in identifying those processes.

  • Create hardening rules for the respective application processes and activate the defense mechanisms. Define exceptions, if necessary.

If there is any uncertainty about what defenses should be enabled for individual processes or what exceptions they needs, it is possible to enable only logging (without activated defenses) to learn the legitimate needs of the process and set the defenses based on the log accordingly.

The online Knowledge Base can be also helpful to set up the optimal rules for particular processes.

Example

Example of using Application Hardening to secure Internet Information Services 6.0:

IIS 6.0 consists of two relevant processes: the inetinfo.exe and w3wp.exe. The inetinfo.exe is a common process for many network services offered by the IIS and the w3wp.exe is a specialized process handling just the WWW service.

Both processes are handling data from potentially insecure sources so the Application Hardening defenses should be enabled for both of them. That can be achieved by creating one rule for inetinfo.exe, another for w3wp.exe and enabling all three defenses for both rules.

By enabling all three defenses, the inetinfo.exe and w3wp.exe processes will not be allowed to spawn other executables, modify executable files and modify OS sensitive areas neither. As a result, hackers or worms deploying these exploitation techniques will be stopped.

Application Hardening also allows flexible definition of exceptions for situations when the protected process has a legitimate need to perform actions disallowed by the defenses. In the case of IIS, that can happen most likely in two situations:

  1. Let's assume that the IIS is also hosting a web application written in Perl programming language. In order to process the web pages written in Perl, the w3wp.exe process needs the ability to spawn perl.exe. That can be achieved by creating an exception for the perl.exe executable in the rule definition. With this exception defined, the w3wp.exe will be allowed to spawn perl.exe, but no other process.

  2. Another example concerns the executable file protection and the IIS FTP service (when the FTP service enabled). The FTP service is implemented in the inetinfo.exe process. If the inetinfo.exe is not allowed to create or modify executable files, then the FTP users will not be allowed to upload executable files. To solve that, the inetinfo.exe process can be allowed to create/modify executable files in specified directories.