Abstract
Most existing virtual machine introspection (VMI) technologies analyze the status of a target virtual machine under the assumption that the operating system (OS) version and kernel structure information are known at the hypervisor level. In this paper, we propose a model of virtual machine (VM) security monitoring based on memory introspection. Using a hardware-based approach to acquire the physical memory of the host machine in real time, the security of the host machine and VM can be diagnosed. Furthermore, a novel approach for VM memory forensics based on the virtual machine control structure (VMCS) is put forward. By analyzing the memory of the host machine, the running VMs can be detected and their high-level semantic information can be reconstructed. Then, malicious activity in the VMs can be identified in a timely manner. Moreover, by mutually analyzing the memory content of the host machine and VMs, VM escape may be detected. Compared with previous memory introspection technologies, our solution can automatically reconstruct the comprehensive running state of a target VM without any prior knowledge and is strongly resistant to attacks with high reliability. We developed a prototype system called the VEDefender. Experimental results indicate that our system can handle the VMs of mainstream Linux and Windows OS versions with high efficiency and does not influence the performance of the host machine and VMs.
1. Introduction
Cloud computing has become a dominant computing paradigm over the past several years. The flexibility and scalability offered by cloud providers have led to increasingly more services being outsourced to the cloud. Because of the proliferation of cloud computing, cloud-based systems are becoming an increasingly attractive target for malware [1–3]. The most dangerous threat to the cloud is vitalization security, and one type of potential attack on a cloud virtualization system is neighbor attacks [4, 5]. In an Infrastructure-as-a-Service (IaaS) cloud, an adversary could design and run a rootkit or malware that could alter the normal behavior of the legitimate guest operating system (OS) by either modifying the system call table, interrupting descriptor table, or some other critical OS data structure [6]. Later, such malignant software can acquire the control of a virtual machine (VM) by evading existing defense mechanisms. This could not only affect the compromised customer but also attack the cloud infrastructure and other cohosted customers directly. For example, by exploiting the software vulnerabilities within the hypervisor source code, virtualized environment neglected operations manipulation (VENOM) attacks, which allow an attacker to break out of a guest VM and access the underlying hypervisor, have been performed [7]. Protecting VMs from advanced, sophisticated malware or threats is a highly exigent task for cloud service providers [8].
Currently, there are two prominent approaches to malware detection in virtualized infrastructures, namely, in-VM and VMI-based malware detection. In-VM detection systems are agent-based and signature-dependent and run within the target guest VM. They are inadequate for thwarting emerging advanced malware attacks [9] and their functionalities are restricted only to a single system. In a virtualized environment, a hypervisor is able to manage multiple guest OSs [10]. Protecting individual guest OSs using a host-based intrusion detection system (IDS) or antimalware solution is ineffective. To overcome this problem, VMI has emerged as a fine-grained technique that uses the underlying hypervisor to provide complete visibility of the running state of the VMs [11–15]. The main motivation behind VMI is to scrutinize any abnormal changes that occur during the running state of the VM. Monitoring the running state of the VM without compromising performance as well as without the knowledge of the VM is an active research topic. VMI’s main challenge lies in the difficulty of converting low-level byte values into high-level semantic states of the monitored VM’s OS.
This paper addresses the problem of securely monitoring VMs in a server consolidation scenario where multiple VMs run on a host machine. Our primary research contribution is the architecture of a system that can monitor and protect VMs in real time. The main contributions of this paper are as follows:(i)A model of VM security monitoring based on memory introspection is proposed. In contrast to traditional monitoring technology, this model alters the pattern of installing a proxy in the VM. That is, compared with existing VMI technology, there is no need to install a proxy in a virtual machine monitor (VMM) or host. Using a hardware-based approach to acquire the physical memory of the host machine in real time, the security of the host machine and VM can be diagnosed. This pattern has strong resistance to attacks with high reliability and will not influence the performance of the host machine or VM.(ii)A novel approach for VM memory forensics based on VMCS is proposed. Using this approach, running VMs can be detected automatically and the running state information of a VM can be reconstructed. The problem of the semantic gap can be solved. Compared with previous memory introspection technologies, the obtained information is more accurate and comprehensive.(iii)Based on the memory forensics result of a host machine and VM, malicious behavior within the host machine and VM can be detected. Specifically, by drawing the parent-child process relationship graph of the host machine and VM, VM escape can be discovered.(iv)Using the above techniques, we developed a VM defense system called VEDefender which presents the following features: it is transparent to guest machines; it is hard to access, even from a compromised VM; it can collect data, analyze them, and find malicious activities within the host machine and VM; and, at present, it supports Windows and Linux guest machines. These features are leveraged to examine the state of the monitored VMs periodically and detect running malicious processes.
The remainder of this paper is organized as follows. Section 2 provides background information and surveys work related to our work. The proposed framework and detailed technologies based on memory introspection are described thoroughly in Sections 3 and 4. In Section 5, two aspects of the proposed tool are evaluated: effectiveness and performance. The final section presents the conclusions and indicates opportunities for future research in this area.
2. Related Work
Different terminologies are applied to the virtualization framework. We adopt the following terminologies throughout this paper: a guest VM (GM) is a VM running on a given hypervisor. The guest OS is an OS system running on a particular GM. For convenience, the physical address of a host machine is abbreviated as HPA (host physical address). Corresponding to this terminology, the physical address of a VM is abbreviated as GPA (guest physical address), and the virtual address of a VM is abbreviated as GVA (guest virtual address).
2.1. Virtualization Technology
Before presenting the details of our architecture, Intel virtualization technology (VT-x) is introduced briefly.
The main characteristic of Intel VT-x is the support for two new virtual machine extension (VMX) modes of operation. When VMX is enabled, the processor can be either in VMX root mode or in VMX nonroot mode. The behavior of the processor in VMX root mode is similar to classic protected mode, except for the availability of a new set of instructions, called VMX instructions. In contrast, nonroot mode is limited, even when the CPU is running in ring 0. Hence, the VMM can inspect and intercept operations on critical resources without modifying the code of the guest OS. Moreover, as nonroot mode operation supports all four IA-32 privilege levels, guest software can run in the original ring it was designed for. A processor that has been turned on in normal mode can be switched to VMX root operation by executing a “vmxon” operation. The VMM running in root mode sets up the environment and initiates the VM by executing the “vmlaunch” instruction [16].
As shown in Figure 1, Intel VT-x technology defines a data structure called VMCS, which embeds all the configuration information needed to capture the state of the VM or resume its execution. The VMCS consists of six logical groups: the guest-state area, host-state area, VM-execution control fields, VM-exit control fields, VM-entry control fields, and VM-exit information field. The various control fields determine the conditions under which control leaves the VM (VM exit) and returns to the VMM and further defines the actions that need to be performed during VM entry and VM exit operations. Various events may cause a VM exit, and the processor can also exit the VM by executing a “vmcall” instruction [17].
2.2. Existing Malware Detection Methods in Virtualized Infrastructure
Existing malware detection approaches usually involve two steps: first, monitoring hooks are placed at different points within the virtualized infrastructure, and then a regularly updated attack signature database is used to determine attack presence [18]. Some of the relevant work is described below.
2.2.1. In-VM Approach to Malware Detection
LARES inserts hooks in a guest VM and protects its guest component by using the hypervisor for memory isolation with the goal of supporting active monitoring [19]. In addition, some solutions consist of an in-VM agent running within the guest VM and a remote scrutiny server monitoring the VM’s behavior. CloudAV, a cloud-based antivirus system featuring multiple antivirus engines, includes a lightweight host agent and a network service to protect the guest VMs against attacks [20]. This mode has the advantages of rich abstractions and fast speed.
2.2.2. VMI-Based Malware Detection
VMI was initially suggested in 2003. It was defined as a method of inspecting a VM from the outside for analyzing the software running on it [21]. Over the past few years, concrete contributions to VMI have been made, and various methods have been suggested to inspect VM data from the outside [22–24]. As mentioned above, the difficulty in interpreting the low-level bits and bytes of a VM into the high-level semantic state of a guest OS is called the “semantic gap problem” [25–28]. It is very difficult to derive a complete view of a guest OS from outside a GM without knowledge of the hardware architecture or guest OS [29]. At present, there are five major approaches to bridging the semantic gap: manual, debugger-assisted, compiler-assisted, binary analysis-assisted, and guest-assisted approaches.
There are different possible events related to a GM. These events can be grouped for introspection at various levels [30] such as memory introspection, system events introspection, system call introspection, interrupt requests introspection, I/O device driver introspection, and live process introspection.
Memory introspection using this classification is our research emphasis. Memory introspection deals with live memory forensics, which aims to extract the semantic knowledge from the live memory of a running computer system. When the OS is running, the main memory contains all the important data structures, including process control blocks, registry entries, loadable kernel modules, kernel data structures, and page tables. The main memory also contains pages related to data segments and code segments of the process being executed. Information related to the OS can be retrieved by examining the content of main memory [31]. In fact, modern rootkits manipulate memory to avoid detection and can thus be identified by inspecting the same memory contents. Although criminals tend to avoid leaving any evidence in persistent storage, it is extremely hard for them to completely remove their footprints in memory. Consequently, memory is the best place to inspect the current status of the target machine at a specific time [32–34].
According to the operation environment, current memory forensics methods can be divided into the following categories [35]: methods running in the OS kernel, methods based on hardware [36, 37], methods based on an independent OS kernel, methods based on SMM [38], methods based on software VMMS, and methods based on hardware virtualization.
XenAccess was developed as a monitoring library for the Xen hypervisor. The purpose of this library is to provide memory and disk monitoring capabilities for both open-source and closed-source OSs. A whole-system binary code extractor called Virtuoso generates out-of-box code for use in VMI [39]. Using Virtuoso, developers are able to create VMI programs to monitor VMs running a variety of different OSs. Using LibVMI, a hypervisor-assisted detection scheme was proposed to extract key process features from the processes running within VMs [40]. In addition, HYPERSHELL is a practical hypervisor-layer guest OS shell that has all of the functionality of a traditional shell but offers better automation, uniformity, and centralized management [41]. Finally, HyperCheck is a hardware-assisted tampering detection framework designed to protect the integrity of hypervisors and operating systems [38].
2.2.3. Graph-Based Event Correlation
Coping with malware is becoming increasingly challenging, given its relentless growth in complexity and volume. One of the most common approaches is to use machine learning techniques to automatically learn models and patterns behind such complexity as well as to develop technologies for keeping pace with the development of novel malware [42]. Among these techniques, graph-based event correlation has been presented to detect attacks within critical infrastructures [43]. To identify the sequence of events within a network, graph-based event correlation represents the events from network logs obtained as sequences in a graph. Given a collection of logs from different points within the network, these events are correlated in a graph with the event features, which include timestamp, source, and destination IP.
By collecting events from different sources within the network, a temporal graph model is generated to derive event correlations. Relationships between files are represented as a graph for malware detection. Then, based on the constructed graphs, a belief propagation algorithm is proposed to detect previously unknown malware [44].
2.3. Limitations of Existing Approaches
As mentioned above, in-VM solutions cannot resist advanced attacks. Comparatively, VMI solutions offer many advantages because they run at a higher privilege level and are isolated from attacks. Further, they are one layer below the guest OS and interpose the guest OS events. However, to solve the semantic gap problem, VMI solutions must perform additional address translation and world switching that traps the hypervisor for security checks and monitoring. Execution VMI tools may affect the performance of host machine.
However, current memory introspection solutions only analyze the target VM and their memory analysis capabilities depend on the functionality provided by the VMM. Further, it is difficult to detect VM escape and these technologies cannot resist attacks from the host or malicious VMMs. More trustworthy VMI techniques are needed.
In addition, the aim of the current graph-based event correlation solutions is analysis of the network data. They hence do not analyze the behavior of data inside the VM.
3. Threat Model
The threat model we adopt in this paper considers powerful attacks and malware that can be performed or injected inside GMs at runtime. When running, guests can be subject to viruses, code injection, buffer overrun, and all other possible kinds of intrusion. The intruder might exploit zero-day vulnerabilities to gain root privileges remotely and affect the kernel and applications. In contrast, the attacker does not have physical access to the host machine and hence cannot perform any hardware-based attacks.
Based on the above threat model, the main requirements of a security model for the virtualization environment are described below.
RQ1 Transparency. The system should minimize visibility from the VMs; that is, potential intruders should not be able to detect the presence of a monitoring system.
RQ2 Immunity to Attacks from the Guest. The host system and sibling guests should be protected from attacks from a compromised guest.
RQ3 Validity. The system should be able to determine adequate active information of the running VMs.
PQ4 Minimum Performance Impact. The main goal in virtualization is to share resources between available guests. The system should place as little burden as possible on the hypervisor and real hardware resources.
PQ5 Generality. The system should be adaptive and monitor the mainstream Linux and Windows kernel VMs.
4. System Design and Technology Details
In this section, our defense system named VEDefender is presented and the reasons for the technologies chosen to build the VEDefender prototype are discussed. The most relevant implementation details are also provided. The prototype was implemented on top of a kernel-based virtual machine (KVM), a VMM that leverages hardware virtualization technology.
4.1. VEDefender: A Hypervisor-Based Defense System for VMs
The basic idea used to develop our defense system is that the information of not only the host machine but also the GMs is involved in the memory of the host machine (Figure 2). It has five main components:(i)Host machine memory capture (HMMC): this component allows the memory content of the host machine to be obtained for specified addresses and lengths in real time(ii)Host machine memory analysis (HMMA): this component provides analysis of the host machine memory; thus, the running state information of the host machine can be acquired, including running processes, open files, loaded modules, and network connections(iii)GM behavior analysis (GMBA): based on the system information of the host machine, the running guest machine can be detected. Address translation from the GPA to HPA and from the GVA to HPA can be implemented. Using the address translation method, GM memory contents can be obtained and any changes can be monitored periodically. The memory analysis methods corresponding to the OS version are put into effect and the guest machine running state can be reconstructed(iv)Abnormal behavior detection (ABD): this component allows abnormal behavior inside the guest machine to be discovered, including hidden processes, malicious code injection, and VM escape(v)Intelligent response (IR): when abnormal states are detected, some necessary actions are taken to deal with the abnormal VM, including creating backup copies of the target VM and migrating the target VM to the host machine in a specific virtual network
All these components operate while the guest OS is running without interrupting or suspending its execution.
4.2. HM Memory Capture and Analysis
There are two ways to obtain the physical memory of the host machine: software-based approaches and hardware-based approaches. Hardware-based memory acquisition methods need additional hardware devices that should be postinstalled in the target system. However, compared with the software solutions, the hardware-based approaches can hardly be bypassed or identified by antiforensic malware because they run below the operating system. Accordingly, we designed a hardware device in the form of a PCI expansion card that can dump the system memory in direct memory access (DMA) mode. The device includes a USB controller, PCI bridge controller, USB interface, PCI Express interface, complex programmable logic device, power module, program storage, clock module, and reset module. The entire installation procedure is described as follows: the device is added to the target computer as a PCI bridge and then the computer loads the corresponding driver automatically. After the device is enumerated successfully, the system controller responds to the command from the USB interface and manages PCI bridge controller to read the memory data.
The device can circumvent the OS and runs separately from the processor when it is working. Hence, it barely affects the processor performance and has a light footprint in the target system memory. To realize runtime defense and save storage space, we retrieve the memory content of the appointed address and length on demand instead of capturing the entire memory contents as a memory dump. The device is suitable for common hardware architectures in which PCI Express is supported.
The memory acquisition method we developed inherently provides the foundation for quick attack detection. It runs separately from the processor and barely affects the processor performance.
To extract useful information from the memory content, kernel symbols are essential. As of now, there are three main ways to obtain these symbols:(i)One can copy /proc/kallsyms or System.map and parse the file. The main difference between these two files is that /proc/kallsyms contains not only kernel symbols but also symbols exported from modules. System.map contains only kernel symbols.(ii)Symbols can be extracted from the kernel’s executable file.(iii)Symbols are loaded in the memory when the kernel boots up. Kernel symbols can be recovered using the addresses of the kallsyms_addresses, kallsyms_num_syms, kallsyms_names, kallsyms_token_table, and kallsyms_token_index symbols. Using the method described in [45], kernel version and kernel symbols are obtainable.
As shown in Figure 3, several essential symbols are picked out to analyze the host machine. The init_task symbol is used to extract running process information and processes related to VMs can be found. In the KVM virtualization environment, each VM is represented as a process named qemu-system-i386 or qemu-system-x86_64, for 32-bit and 64-bit systems, respectively. Furthermore, from the detailed parameters of the process, we can parse out the detailed VM configuration information, such as disk size, disk path, vcpu number, disk format, and memory size. The modules symbol can be used to traverse the doubly linked module list. Of these modules, the kvm.ko module is used to provide the core virtualization infrastructure. Of the symbols exported from kvm.ko module, the vm_list symbol can be used to detect the running VMs. The concrete methods to perform this task are described in the next section. Moreover, the rt_hash_mask, rt_hash_table, and net_namespace_list symbols are used to obtain the information about network configuration and network connections; the boot_cpu_data symbol is used to gain CPU information; the log_buf symbol corresponds to system log and debug information; and the iomem_resource symbol reflects the available physical address space of the target system.
During the analysis procedure, precise knowledge of the structure layout information is also essential. Because the Linux kernel is highly configurable, users can specify a large number of configuration options that affect the kernel build process. For example, sched information only exists in the task_struct structure when the CONFIG_SCHEDSTATS macro or the CONFIG_TASK_DELAY_ACCT macro is set. That is, the structure elements can vary significantly, even within the same kernel version, and it is hence not accurate to specify structure member information and memory layout. The method of obtaining the task_struct layout is described as follows:(i)The relationships among task_struct (TS), files_struct (FILES), fs_struct (FS), nsproxy (NS), and signal_struct (SGS) structures are shown in Figure 4. It can be seen that TS has five pointers, one each to FS, FILES, TS, NS, and SGS.(ii)The init_task variable from the System.map file represents the TS structure of the swapper process. From the address of init_task, the offsets of the files, fs, nsproxy, signal, and comm members are obtained by searching for the value of the init_files, init_fs, init_signals, init_nsproxy, and swapper strings.(iii)The values of tasks point to tasks elements for another process. The content of the address from the offset of the comm variable is obtained. If the content is a character string, the offset of the tasks is valid. Otherwise, the process continues to scan the TS structure.(iv)According to the features of the TS structure, the tgid variable is close to the pid variable (see (1)). Furthermore, both variables are between the comm and tasks variables (see (2)), and their values are equal in most cases. Based on these features, the offset of the pid variable can be obtained.
4.3. GM Behavior Analysis
The VM detection algorithm and analysis method, which deeply analyze the kernel structures related to KVM, are shown in Figure 5. The vm_list symbol points to a doubly linked list, each entry of which corresponds to a KVM structure. The KVM structure contains the global information of the corresponding VM, such as physical memory region, vcpus information, page table, memory management unit (MMU), I/O bus information, and the dirty translation lookaside buffer. Among this information, vcpus points to the kvm_vcpu structure, which is a part of the vcpu_vmx structure. In the vcpu_vmx structure, virtual CPU information is included, such as the anomaly information, interrupt event, and the loaded_vmcs pointer, which is relative to the VMCS structure currently used in this vcpu. The kernel data structures of different OSs and system configurations can vary. The primary problem is how to identify the offset of the vm_list variable in the KVM structure and the loaded_vmcs variable in the vcpu_vmx structure.
The offset of the vm_list variable in the KVM structure can be retrieved by analyzing the vm_stat_get function, which is defined in the kvm_main.c file. The vm_stat_get function definition in the 3.10.0-229.4.2.el7.x86_64 system, is shown in Figure 6.
Combined with the function definition, we decompiled the binary code loaded in memory and the partial recompilation results are listed in Algorithm 1.
| ||||||||||||||||
The third parameter used by the list_for_each_entry function is the vm_list variable. Here, it can be seen that the offset of the vm_list variable in the KVM structure is 0x9F8.
In the same way, by analyzing the “if (vmx->loaded_vmcs->cpu != cpu)” statement in the vmx_vcpu_load function, we can obtain the offset of the loaded_vmcs variable in the vcpu_vmx structure, which is 0x3B00 (see Algorithm 2).
| ||||||||||||||
So far, the VMCS information of each active VM is available. As mentioned above, the VMCS has a data area for guest status, which maintains the CPU states before a VMEXIT event happens. These states include control registers (CR0, CR3, and CR4), the debug register, RSP, RIP, RFLAGS, segment registers, and extended page table (EPT) pointer. EPT is an address space protection technique supplied by Intel VT-x. The traditional MMU translates the GVA directly to the HPA by looking up the guest page table. However, while the EPT is enabled, GVA will first be translated into the GPA through the guest page table. Then, the GPA is translated to the HPA using the extension page table. The extension page table, adopted in the memory dump phase, can prevent the guest OS from illegally accessing protected physical memory. Using the value of the EPT pointer, GVA and GPA can be translated into an HPA. In other words, given an appointed physical or virtual address of a VM, its memory content can be acquired.
Once the memory has been acquired, memory analysis for VM may be implemented using different methods for various OSs. For a Windows system, a memory analysis method based on the kernel processor control region (KPCR) structure is useful for identifying detailed OS information. The KPCR structure contains kernel-specific variables and pointers to additional control structures. With the help of kernel variables, the running process, loaded modules, network connections, and running services of the VM can be obtained.
For Linux systems, a way to automatically determine the system version is missing in existing solutions. In this section, a novel approach to identifying the precise kernel version without any prior knowledge is presented. As the system starts up, the crash_save_vmcoreinfo_init() function is triggered to initialize the content of vmcoreinfo_data, which includes general kernel crash information such as the kernel version, page size, and symbol information. In the vmcoreinfo_data region, the character string “OSRELEASE=” is the first string; moreover, “SYMBOL(swapper_pg_dir)=” and “SYMBOL(_stext)=” character strings are also involved. By searching for these three strings, the address of vmcoreinfo_data can be located and detailed information about the kernel version can also be obtained. Then, using the methods described in Section 4.1, symbols can be obtained and therefore the running processes, loaded modules, network connections, and other active state information can also become available.
From the above description, we can see that our analysis method does not depend on fixed addresses during reconstructing the running state of host machine and VMs. As a result, the utilization of address space layout randomization (ASLR) does not influence our method. In fact, our experiments are made under the condition that the ASLR setting is on.
4.4. Abnormal Behavior Detection
A malicious behavior analysis for the host machine and VM is executed simultaneously. So far, running processes are our analysis target and the following aspects are carried out.
First, processes hidden by rootkits are detected using cross-validation. For Windows, two lists of processes are used: one for the scheduling and one for the tracking. A suspicious process whose object is removed from the tracking list will be invisible while still active. Besides, every thread corresponds to a data structure named ETHREAD, and all ETHREAD structures are connected by a doubly linked list. Every thread belongs to a process and, using the thread list, a process list can be retrieved. More specifically, we can summarize the characteristics of the process structures and scan memory for process objects. Thus, a cross analysis of these three process lists is executed to find hidden processes. Similarly, for Linux systems, process lists can be analyzed in diverse ways, including via doubly linked lists, hash tables, and feature retrieval. Hash table can be acquired using the pid_hash symbol. Besides, each process corresponds to a task_struct type structure, and the structure characteristics should be determined. The following rules can be applied to make this judgment for 32-bit systems (see Algorithm 3).
| ||||||||||
Second, to reveal the relationships among the analysis results obtained above, an event correlation graph is constructed. In the graph, the parent-child processes are connected with lines and a corresponding point related to every process is established for the relevant information, including open files, network connections, and registry information. Every process is represented in the following format: <pid, processname, ppid, openfiles, network connection, registry information>
The above data are converted into the JSON (JavaScript Object Notation) format and saved in a JSON data file. Using a visualizer named D3.js, process relationship diagrams can be described.
Third, by comparative analysis on the process relationship diagrams of different moments, we can determine the concrete behavior and loading procedure of the malicious program. In Section 4.1, we mentioned that qemu-system-i386 and qemu-system-x86_64 in the host machine are special processes corresponding to VMs. Hence, these two processes are the focus of our analysis and their state changes can be used to analyze the VM escape behavior. During the analysis for VENOM vulnerability, we found that a new terminal process with root authority is started after a successful suspicious VM escape to the host machine using VENOM. The parent process of the terminal process is a process named qemu-system-i386 that does not exist in the process list, which means it is a hidden process or a hidden thread. In contrast, we find that the hidden object has the same tgid and different pid with a process named qemu-system-i386 in the process list, and its parent process is the same process named sudo. More specifically, the qemu-system-i386 process corresponds to the suspicious VM. This behavior pattern does not exist in a normal VM process relationship and can be used to discover a VM escape.
4.5. Intelligent Response
Once unexpected behaviors are detected, the whole memory information and disk information are completely preserved. For VMs, the memory access algorithm is described as shown in Algorithm 4.
| ||||||||||||||||||||
It can be seen that, compared with memory acquisition for host machine, address translation from GPA to HPA is an extra step. These backup copies can only be used for forensics. To prevent further attack spread, our measure is to migrate the target VM to the host machine in a specific virtual network which cannot be implemented only depending on physical memory read operations. In other words, to migrate a virtual machine, more permissions are needed such as memory write or root permission in the host machine. In [46], the major issues of virtual machine live migration are discussed. Preparation time, downtime, resume time, page transferred, total migration time, and application degradation are usually used to measure the performance of live migration. To migrate a VM from one host to another in a trusted manner, a Trusted Virtual Machine Migration Protocol (TVMMP) was proposed, which can guarantee the coherence and continuity of the trusted status during the VM migration and provide secure aids for the trusted migration of a VM in the IaaS platform [47]. Using this protocol, a vulnerable or aggressive VM can be migrated to a specific virtual network that can block some suspicious commands.
5. Evaluation and Experimental Results
Based on the techniques described above, we developed a VM defense system called VEDefender that includes a PCI device and a terminal program. Our evaluation in this section aims to verify the effectiveness and efficiency of VEDefender. Effectiveness means that VEDefender can determine the OS version and reconstruct the running state of the host machine, detect running VMs and reconstruct their running state without any prior knowledge, determine hidden processes within the host machine and VMs, and build a process relationship diagram and analyze the VM escape behavior. Comparatively, the efficiency is the time needed to implement the above functions.
To achieve these objectives, the test procedure comprises the following stages: Preinstall the PCI device on the target host machine. Design the monitoring strategy by setting the monitor interval, save directory, and other parameters. Boot the host machine and start VEDefender to monitor changes in the host machine. Start the VMs one by one. Implant rootkits in the GMs and hide several processes. Exploit VENOM vulnerability to escape from the VM to the host machine.
5.1. Testbed Setup
For the experiments, two servers with Intel Xeon E5-2620 CPUs at 2.00 GHz along with 16 GB RAM were chosen as the target host machines. The machines ran Linux kernel versions 3.10.0-327 (64-bit) and 3.2.0-24 (32-bit), respectively. A virtualization environment was set up using KVM on the host machine to enable multiple guest VMs to be run. The OS versions of the VMs are five popular operating systems: Windows 7, Windows XP, Centos 7, Fedora 19, and Ubuntu 12.
5.2. Effectiveness
After each GM system is started, VEDefender can detect the running state of the VM. First, the corresponding VMCS addresses, disk paths, and operation system versions of each VM are obtained. Based on this information, running processes, loaded modules, network connections, and the registry (for the Windows GM) can be parsed and the process relationship diagram is constructed. By tracing the parent-child relationships between processes, some hidden and exit processes can be detected. Because the process relationship diagram is too large to display, a partial diagram is shown in Figure 7. It can be seen that cmd.exe, LWS.exe, BaofengPlatform.exe, and jusched.exe are hidden processes and runonce.exe is an exit process. In Figure 8, the relationships between process and network connections are presented.
To evaluate trustworthiness of our technology, we injected publicly available real-world rootkits on both the Windows and Linux GMs. For example, we injected the FU-rootkit into the Windows XP GM and used FU-rootkit to hide the calc.exe process. As shown in Figure 9, a cross-check with the processes list obtained from different patterns confirmed that the calc.exe process was hidden.
To analyze the VENOM vulnerability, the following experiments were performed. The kernel version of the host machine was 3.2.0-24 (32-bit), and then we installed two GMs with Linux versions 3.2.57 and 3.2.0-24. The target GM is the first one. First, we wrote a large amount of data to the DATA_FIFO register in the floppy disk controller. The command line “sudo qemu-system-i386 –had poc_rc1.img –usbdevice tablet –m 512 –enable-kvm” was executed in the host machine to boot the GM. Second, the poc program in the GM was executed to trigger GM crash. Then, we checked the log of host machine to find the vulnerable address. Third, we booted the GM again and rewrote the poc code. The poc code was implemented and the vulnerability was activated. Up to this point, the VM escape proceeded well.
We monitored the running state of the host machine and GMs and performed a contrastive analysis. The initial analysis result indicates that there are two qemu-system-i386 processes running in the host machine corresponding to two GMs. The partial process relationship diagram is shown in Figure 10. Every qemu-system-i386 process has no child process and the group_leader member in the task_struct structure points to itself. Using the method described in Section 4.2, the running state information of the GMs is also obtainable.
After VM escape has been executed, the process relationship of host machine has been changed. As shown in Figure 11, process qemu-system-i386 with pid 15781 has not changed. However, for process qemu-system-i386 with pid 15994, its parent process has two child processes. The new process is not in the doubly linked process list and has a child process named sh. Furthermore, its group_leader member in the task_struct structure points to the process qemu-system-i386 with pid 15994. Except for this abnormal behavior, address transition exception occurs when analyzing the target GM process. Using these two rules, we can conclude that a VM escape exists in the GM related to qemu-system-i386 with pid 15994.
For the various platforms, there are several vulnerabilities that may cause VM escape. A part of the vulnerabilities we have collected are listed in Table 1.
Some of these vulnerabilities exploit stack overflows, and others exploit vulnerabilities existing in the network card drivers or shared directories for directory traversal. VENOM was chosen as target in our experiment because it has widespread influence. VENOM exploits the bug in QEMU’s virtual floppy disk controller (FDC). This vulnerable FDC code is used in numerous virtualization platforms and appliances, notably Xen, KVM, VirtualBox, and the native QEMU client. Our detection method is based on the existence of anomalies in the relationship between the physical machine processes and the existence of anomalies in the virtual machine address translation. These two rules are used to determine whether there is an escape phenomenon. The proposed method may detect other escape attacks with similar behavior patterns that have the following two features:(i)Generally, a new process or thread is created to execute an illegitimate intent after VM escape behavior has been implemented.(ii)VM running status, which includes memory management, address translation, and other indicators, is affected.
However, further experiments need to be done to prove our assumption, and this forms an important part of our future work.
5.3. Performance Evaluation
In this section, the time cost for different phases is evaluated. As illustrated in Figures 12–14, for each additional VM, the detection time increases by about 50 ms. Moreover, 2–80 ms was required for OS version identification and 12–65 ms was needed to obtain the process and module information of the VMs. In addition, experiments were performed to test the execution efficiency of LIBVMI, which can also be used to obtain the process and module information of the VM. As shown in Figure 15, 200–600 ms was required for the acquisition procedure. The experimental results prove that VEDefender can obtain running state information of the PMs and VMs and demonstrate that its execution process is efficient.
The monitor interval was set to 5 seconds and the monitor content included running processes, loaded modules, and network connection of both the host machine and GMs. During the monitoring procedure, the changes in running GMs could be obtained by traversing the vm_list list in the host machine. There is no need to determine the OS version more than once for a running VM. From Figures 12–14, we can calculate that every duration time is less than 500 ms.
The experimental results prove that VEDefender can deal with VMs with a wide range of OS versions and demonstrate that its execution time is acceptable.
5.4. Comparisons with Existing Security Approaches
Actaeon [48] is designed to analyze hypervisor structures in physical memory dumps by summarizing the characteristics of several fields of the VMCS structure, scanning the entire memory for valuable VMCS structure, and recognizing the address space of VMs. It can be seen that Actaeon needs a memory snapshot of the host machine to search for existing VMs. According to the experiment, about 90 seconds are required to obtain a 4 GB memory image and 65 seconds are needed to find the available VMCS structures. Our approach has overcome this limitation by monitoring the state of the host machine periodically and detecting the existence of VMs precisely.
XenAccess is a monitoring library for the Xen hypervisor and does not provide active monitoring. Similar to LibVMI, its memory analysis capabilities are quite limited and the validity of the acquired information relies on the hypervisor. In contrast, our solution can detect suspicious processes or modules running in the host machine. We chose adore-ng and suekit as our detection objects. By cross-checking the process lists and loaded modules, implanted rootkits can be detected. Furthermore, according to the abnormal behavior pattern in the host machine, VM escapes may be detected. Moreover, our method can resist hypervisor attacks.
In another study, a lightweight introspection framework called Pathogen [49] was proposed to streamline the detection of the exact OS version and data structure definitions. This framework is in an early stage of development and supports the analysis of a limited number of data structures and further lacks active data structure monitoring capabilities.
6. Conclusions
Using memory introspection, this paper proposed a novel dynamic malware analysis framework that periodically scrutinizes the state of the introspected system to detect hidden, dead, and dubious processes in the monitored VM without modifying the guest OS kernel at the host level. A VM detection method based on the VMCS was put forward, and VM memory information can also be obtained. Combined with memory analysis technology, the semantic gap problem can be solved. We implemented a prototype called VEDefender in the KVM environment.
Most out-of-VM solutions can perform similar functionality, including identifying malware and moving the monitoring functionality out of the VM. However, the differences of our system are explained as follows:(i)VEDefender has a high level of isolation. The raw data are obtained in the DMA mode, which circumvents the host OS. This means that the acquisition method does not rely on the guest machine and VMM. This pattern can defend against attacks from not only the guest OS but also the VMM.(ii)The acquisition part of the whole system is a hardware device and the analytical part exists outside of the host machine. We have no need for an account in the host machine or VM and we do not have any need to install the software inside the host machine or VM. Hence, VEDefender is completely transparent to VMs and the VMM and does not affect their performance. Compared with LibVMI, the memory content we obtain does not depend on the VMM and is more reliable. More importantly, VEDefender provides a trusted execution environment, which means an attacker controlling the system cannot interfere with the analysis and cannot tamper with the results. Based on these features, we think that VEDefender is strongly resistant to attacks with high reliability.(iii)By analyzing the memory content of the host machine and VMs mutually, VM escape can be detected by the abnormal behavior pattern in the host machine. So far, we have not found relevant literature to solve the problem of VM escape detection.(iv)When malicious behavior has been detected, VEDefender is then able to access the whole physical memory of the host machine without restriction. According to the techniques described above, the physical memory of a VM can also be retrieved and saved as evidence.(v)Using the above techniques, VEDefender can introspect a live OS without the prior knowledge of the VM. It can be also integrated into other IDS solutions and has good extensibility.
Moreover, our system uses a PCI expansion card to acquire the memory content of the host machine, and the device should be preinstalled in the target machine. It may not be feasible to install a PCI expansion card in all kinds of systems and the additional cost for attaching additional hardware may not be acceptable in several deployments.
When VM escape is detected, the malicious VM can be identified. However, concrete attack behavior inside a malicious VM still needs further analysis in future work. There are a few vulnerabilities that can be used to realize VM escape behavior, such as CVE-2015-7504, CVE-2015-3247, and CVE-2017-4934. In the next steps of our research, we will perform a further general study of loopholes in the virtualization system and build our knowledge base for efficient analysis.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grants nos. 61572297 and 61602281), the Shandong Provincial Natural Science Foundation of China (Grants nos. ZR2016YL014, ZR2016YL011, ZR2014FM003, and ZY2015YL018), the Shandong Provincial Outstanding Research Award Fund for Young Scientists of China (Grants nos. BS2015DX006 and BS2014DX007), Shandong Provincial Key Research and Development Program of China (Grant no. 2017CXGC0706), and the Shandong Academy of Sciences Youth Fund Project, China (Grant no. 2015QN003).