1 SECURITY OF THE SECURITY TOOLS FuHau Hsu

1 SECURITY OF THE SECURITY TOOLS Fu-Hau Hsu National Central University

Background 2 Plenty of critical personal information is stored in computers Many commercial activities are proceeded through computers and networks.

Results 3 Computers become a major battlefield between attackers and computer users.

Solutions 4 Diverse security tools, have been developed to handle this critical work. anti-virus software auto-patch mechanisms firewall intrusion detection systems and so on

New Problem 5 The complexity of these security tools increases. The security tools become attack targets. The security of the security tools also becomes a crucial issue.

Disable Anti-virus Software 6 Research has shown that diverse approaches have been developed to disable different antivirus software.

Problems (1) 7 How to guarantee the normal operation of antivirus software.

Problems (2) 8 If attackers can disable anti-virus software, is it possible that they can also change the behavior of the anti-virus software through replacing, injecting, or hooking code?

Impacts of Anti-virus Parasites 9 Given the fact that most, if not all, computers allow only one anti-virus application to be installed on them at one time, an attacker that can control the anti-virus application of a computer can fully command the computer without being detected.

Auto-Patch 10 How could a computer guarantee that the auto -patch code it executes is not malicious code?

Impacts 11 If the correctness of auto-patch code cannot be protected, instead of being a tool to seal the security breaches of a computer, auto-patch may become an efficient channel for attackers to intrude a computer or spread malicious code.

12 Antivirus Software Shield against the Antivirus Terminator

Purpose 13 We proposed ANtivirus Software Shield (ANSS) to protect antivirus software from being terminated by malware.

14 Arms Race between Malware Writers and Antivirus Programmers The simplest way for a computer user to secure his computer is to install antivirus software on his computer. 81% of computer users utilize antivirus software to protect their computers, which means over a hundred million computer users use antivirus software in the USA alone. Attackers attempt to break any defense mechanism by masquerading social engineering

15 Malware Self-Defense Techniques (1) Disorder signature-based virus detection methods by Transforming Encoding Compressing Packing malware code. Avoid executing themselves when they are monitored by debuggers hypervisors.

16 Malware Self-Defense Techniques (2) Unmeaning bytes may also be inserted into malware to hinder static malware analysis. More aggressive and dangerous malware may even directly disrupt the functionality of security solutions, especially antivirus software

Antivirus Terminator 17 Malware that terminates the execution of antivirus software without the consciousness of the antivirus software users is called an antivirus terminator.

Who Protect Antivirus? 18 The most effective solution to antivirus terminators is antivirus software itself.

The Problem 19 Even though antivirus companies claim that they have implemented protection mechanism to protect their products against antivirus terminators, our experimental results shown that techniques utilized by antivirus terminators to terminate antivirus software still could terminate antivirus software.

20 Approaches to Terminate Antivirus Software

Null Debugger Method 21 An antivirus terminator can use API Debug. Active. Process to attach itself to an antivirus process as a debugger to control it. Then right after invoking Debug. Active. Process, the antivirus terminator exits. Because the controller of the antivirus process does not exist, the process will crash immediately.

Dll Unloading Method 22 An antivirus terminator can use API Zw. Unmap. View. Of. Section to unload some important dll files, such as ntdll. dll, from an antivirus process to erase some portions of the virtual address space of the antivirus process. ntdll. dll defines many common used windows Native APIs, hence, once the dll file is unloaded from the antivirus process and an API in the dll file is invoked, the process will crash.

Process Termination Method 23 An antivirus terminator gets the handle of an antivirus process by calling API Open. Process with the process ID of the antivirus process. Then the antivirus terminator can use APIs Nt. Terminate. Process or Zw. Terminate. Process to terminate the antivirus process in the kernel level.

Close Message Method 24 An antivirus terminator can use API Find. Window to search all windows running on the system to find the window matching the name of an antivirus software window (e. g. avguard of antivir or kavsvc of kaspersky). Then the antivirus terminator continues sending messages, such as WM_CLOSE or WM_QUIT, to the related antivirus process by APIs Send. Message or Post. Message until the process is terminated.

Mouse Simulator Method 25 An antivirus terminator can use API Send. Input to counterfeit a series of mouse events which lead to the suspension of an antivirus process. E. g. : An antivirus terminator may find the icon of an antivirus process first. Then it forges a series of mouse events to move the cursor to the right lower corner of the screen, press the right button to expand a work menu, and then chose to suspend an antivirus process. This approach was designed by us. Hence, we have not found an antivirus terminator in the wild utilizing this approach to terminate antivirus software.

26 Registry Modification Method (1) An antivirus terminator can modify the registry so that a NULL debugger will be attached to an antivirus process when the antivirus process begins its execution. The above steps will stop the execution of the antivirus process.

Registry Modification Method (2) 27 Besides, an antivirus terminator can also modify the registry to delete antivirus related processes from the startup process list; hence, the system will be booted without the protection of an antivirus process. API Zw. Open. Key can be used to open a registry key. And API Zw. Set. Value. Key can be used to modify a registry key value.

Thread Termination Method 28 An antivirus terminator can use API Terminate. Thread to terminate threads of an antivirus process one by one till the antivirus process stops.

Table I: Methods Utilized by Various Malware to Terminate Antivirus Software 29

30 Combine Two or More Terminating Approaches As Table I shown, attackers may combine two or three techniques together to strengthen their malware.

Market Share Survey 31

Most Popular Antivirus Software 32 5 famous brands of antivirus software were used in our tests. Avira Antivir Personal Kaspersky Anti-virus 2012 Norton Antivirus 2011 ESET NOD 32 Antivirus 4 AVAST Free Antivirus 6.

Terminator Program 33 For each of the above methods we wrote a program, called terminator program, to see whether the approach alone can terminate antivirus. None of them deems the above 7 programs as malware.

Impact of Terminator Programs 34 The results show that if a piece of malware bundles itself with one or more of the first 6 terminator programs; it becomes an effective antivirus terminator. As a result, we believe antivirus terminators will become a severe threat to many computer systems which are usually protected by antivirus.

Frangible APIs 35 The Windows native APIs that are executed when the above 7 techniques are used to terminate antivirus software called frangible APIs. The native APIs may not be called directly by antivirus terminators, but are called by the APIs utilized by the antivirus terminators.

List of Frangible APIs 36

Intercept Windows API Calls 37 ANSS monitors invocations to frangible APIs. Frangible APIs calls with parameters that will stop or suspend antivirus software blocked by ANSS directly. ANSS uses SSDT (System Service Descriptor Table) hooking to intercept and examine invocations of frangible APIs.

System Service Dispatch Table 38 The Windows kernel provides various system services for application programs to use. Inside the kernel the addresses of these system services are stored in the SSDT, Ki. Service. Table.

Usage of SSDT 39 When a program issues a system call to execute a system service, ki. System. Service looks up the SSDT to find the address of the corresponding system service. Then ki. System. Service uses the address to invoke the system service.

SSDT Hooking 40 Through SSDT hooking, ANSS modifies some function addresses stored in the Ki. Service. Table service descriptor table and replaces them with ANSS API handlers. After an ANSS API completes its task, it invokes the original API.

Execution Flow of a System Call with and without SDDT Hooking 41

ANSS Structure 42 ANSS consists of three components: interceptor, filter, and blocker.

Interceptor 43 When a program is going to invoke a frangible API, the interceptor intercepts the execution flow of the program and transfers the execution flow to the filter before the code of a frangible API is executed. ANSS utilizes SSDT hooking to accomplish the above task.

Frangible APIs and ANSS APIs 44 For each frangible API, there is a corresponding ANSS API in the ANSS filter. In other words, an invocation to a frangible API is transferred to its corresponding ANSS API first through SSDT hooking.

Filter 45 Based on some build-in rules, the filter examines the parameters of the API call to filter out the intention to abuse the API that will terminate or suspend antivirus software. For a normal API call, the filter transfers the execution flow back the original frangible API code.

Blocker 46 An API call with a malicious intention is further passed to the blocker which returns an error code, ACCESS_DENY, as the output of the API call. Besides, the blocker prints out an error message to notify users that it just blocked an attack on the antivirus software.

Filter Rules 47 The ANSS filter utilizes various rules to prevent frangible APIs from being executed with malicious parameters.

Antivirus Process Handle Rule 48 The ANSS filter restrains an application from obtaining the process handle of an antivirus program, because once an attacker obtains the handles of an antivirus program, the attacker can terminate the program easily. To get the process handle of an application program, attackers can use API Open. Process which utilizes frangible API Nt. Open. Process inside the kernel land to finish its work.

Nt. Terminate. Process Rule 49 The filter does not allow an application to use Nt. Terminate. Process to terminate antivirus software.

Debug. Active. Process Rule 50 The filter does not allow an application to utilize Debug. Active. Process to attach itself as a debugger of antivirus software. Debug. Active. Process uses frangible API Nt. Debug. Active. Process in the kernel address space to accomplish its task.

Post. Message Rule 51 The ANSS filter prohibits a program from sending messages, such as WM_CLOSE or WM_QUIT, to antivirus software, because an antivirus terminator can use API Post. Message to send the above messages to the antivirus software to stop it. API Post. Message sends messages through the kernel land frangible API Nt. User. Post. Message.

Registry Key Rule 52 The filter disallows an application to modify or delete Windows Registry keys to protect antivirus software, because attackers can use registry APIs, like Nt. Delete. Key, Nt. Set. Value. Key, and Nt. Create. Key to disable antivirus software.

53 Nt. Unmap. View. Of. Section Rule The ANSS filter forbids an application to use Nt. Unmap. View. Of. Section to unload a. dll file from antivirus, because an antivirus terminator can use the frangible API to stop antivirus software.

Send. Input Rule 54 When the foreground process is antivirus software, the ANSS filter inhibits an application to use API Send. Input to deliver mouse events to the process, because an antivirus terminator can use API Send. Input to imitate a series of action leading to the termination of antivirus software.

Limitation 55 ANSS uses SSDT hooking to intercept frangible APIs calls. If after ANSS is installed in a computer, an antivirus terminator can bypass the detection of the antivirus software in the computer and hook the system too, then ANSS may no longer work as expected.

Solution 56 To provide a thorough protection, ANSS needs to work with the anti-hooking mechanism which is a well studied issue in the literature.

Evaluation - Compatibility 57 We made diverse experiments to test whether ANSS influences the operations of normal applications.

58 Compatibility Evaluation Samples We use false positive tests to evaluate the influence caused by ANSS upon normal applications. There are two types of applications used in our tests. The first type of application is antivirus software. Any non-antivirus application is classified as the second type.

59 Antivirus Compatibility Evaluation For antivirus software, we installed ANSS first. Then we manually disabled the antivirus software, and manually enabled the antivirus software to check whether these operations proceeded as usual. Besides, we also updated the virus definition files made regular system scan shut down the operating system. All operations proceeded as usual.

Result 60

61 Non-Antivirus Compatibility Evaluation We installed ANSS in 8 our daily used computers to test if ANSS influences their normal operations. Then we used these computers as usual.

62 Hardware Specification of Test Hosts These computers had installed Microsoft Windows XP Professional Service Pack 3 with Intel 2. 66 GHz Core 2 Quad CPU Q 9400 1. 96 GB RAM.

63 Software Used in the Test Hosts Besides, lots of popular applications, such as Firefox Windows Live. Messenger i. Tunes Oracle VMVirtual Machine Avira Anti. Vir Personal Microsoft Office Adobe Reader.

Result 64 After 6 months’ tests, none of the applications or computers was disabled, crashed, or blocked by ANSS. In other words, ANSS did not influence the operations of applications and computers during our test periods.

Evaluation - Functionality 65 We made diverse experiments to test whether ANSS can protect antivirus software.

66 Functionality Evaluation Samples Then we used the 18 antivirus terminators that we abstracted from 2063 malware samples to test the 5 famous brands of antivirus software: Avira Antivir Personal Kaspersky Anti-virus 2012 Norton Antivirus 2011 ESET NOD 32 Antivirus 4 AVAST Free Antivirus 6

Challenge 67 However, the virus definition files of some of the above antivirus software have already had the signatures of the antivirus terminators used in our tests. Hence, before the antivirus terminators execute, they will be detected and blocked by antivirus software.

Solution 68 But our experiments only want to test whether the mechanisms that antivirus terminators use to terminate antivirus software can really close antivirus software. Hence, when making our experiments, we temporarily did not allow antivirus software to scan files so that we can accurately test the terminating mechanisms without interference.

Result 69 Experimental results show that every antivirus terminator can terminate some of these antivirus programs. However, if these antivirus programs were protected by ANSS, none of the antivirus terminators can close them.

Survivability of Various Antivirus Programs with Or without ANSS 70

71 Performance Evaluation Environment We used Windows benchmark tool called Pass. Mark Performance Test to test the CPU and memory performance overhead of ANSS on a Windows XP system with Intel 2. 66 GHz Core 2 Quad CPU Q 9400 448 MB RAM.

72 Performance Overhead Measurement Performance overhead measurement was performed under two different system setups. In the first setup, we installed a Windows XP system which did not execute ANSS. In the second setup, we installed a Windows XP system that executed ANSS. Under each setup, we executed Pass. Mark a hundred times. For each benchmark, we use its average value to represent its final value.

CPU Performance Overhead 73

Memory Performance Overhead 74

Conclusion 75 Experimental results show that many commercial antivirus programs will be closed by antivirus terminators. However, when being protected by ANSS, antivirus software can function as usual even under the attacks of antivirus terminators. ANSS introduces at most 0. 42% CPU performance overhead and 1. 77% memory write performance overhead. Therefore, ANSS is an effective and efficient solution against antivirus terminators.

76 Thanks