Carnegie Mellon University
`
`CARNEGIE INSTITUTE OF TECHNOLOGY
`
`THESIS
`
`SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
`
`FOR THE DEGREE OF Doctor of Philosophy
`
`TITLE
`
`Efficient Hypervisor Based Malware Detection
`
`
`
`
`
`PRESENTED BY
`
`Peter F. Klemperer
`
`ACCEPTED BY THE DEPARTMENT OF
`
`Electrical and Computer Engineering _
`
`
`
`_
`
`,
`
`PAS -20/FL
`
`THVISOR
`PROFESSOR
`DATE
`>
`
`4) pce [pet
`/ 之 -/ 7-2
`
`7
`DEPARTMENT HEAD
`DATE
`
`cad
`
`1
`
`APPROVED BY THE COLLEGE COUNCIL
`
`
`DEAN
`DATE
`
`WIZ, Inc. EXHIBIT - 1014
`WIZ, Inc. v. Orca Security LTD. - IPR2024-00220
`
`WIZ, Inc. EXHIBIT - 1014
`WIZ, Inc. v. Orca Security LTD.
`
`

`

`Efficient Hypervisor Based Malware Detection
`
`Submitted in partial fulfillment of the requirements for
`
`the degree of
`
`Doctor of Philosophy
`
`in
`Electrical and Computer Engineering
`
`Peter Friedrich Klemperer
`
`B.S., Computer Engineering, University of Illinois at Urbana-Champaign
`M.S., Electrical and Computer Engineering, University of Illinois at Urbana-Champaign
`
`Carnegie Mellon University
`Pittsburgh, PA
`
`May 2015
`
`

`

`Copyright c(cid:13) 2014 Peter Friedrich Klemperer
`
`Copyright © 2014 Peter Friedrich Klemperer
`
`Copyright © 2014 Peter Friedrich Klemperer
`
`Copyright © 2014 Peter Friedrich Klemperer
`
`

`

`i A
`
`bstract
`
`Recent years have seen an uptick in master boot record (MBR) based rootkits that load before the
`
`Windows operating system and subvert the operating system’s own procedures. As such, MBR
`
`rootkits are difficult to counter with operating system-based antivirus software that runs at the same
`
`privilege-level as the rookits. Hypervisors operate at a higher privilege level than the guests they
`
`manage, creating a high-ground position in the host. This high-ground position can be exploited to
`
`perform security checks on the virtual machine guests where the checking software is isolated from
`
`guest-based viruses. The efficient introspection system described in this thesis targets existing vir-
`
`tualized systems to improve security with real-time, concurrent memory introspection capabilities.
`
`Efficient introspection decouples memory introspection from virtual machine guest execution,
`
`establishes coherent and consistent memory views between the host and running guest, while main-
`
`taining normal guest operation. Existing introspection systems have provided one or two of these
`
`properties but not all three at once.
`
`This thesis presents a new concurrent-computing approach – high-performance memory snap-
`
`shotting – to accelerating hypervisor based introspection of virtual machine guest memory that
`
`combines all three elements to improve performance and security. Memory snapshots create a co-
`
`herent and consistent memory view of the guest that can be shared with the independently running
`
`introspection application. Three memory snapshotting mechanisms are presented and evaluated for
`
`their impact on normal guest operation.
`
`Existing introspection systems and security protection techniques that were previously dis-
`
`missed as too slow are now be enabled by efficient introspection. This thesis explains why ex-
`
`isting introspection systems are inadequate, describes how existing system performance can be im-
`
`proved, evaluates an efficient introspection prototype on both applications and microbenchmarks,
`
`and discusses two potential security applications that are enabled by efficient introspection. These
`
`applications point to efficient introspection’s utility for supporting useful security applications.
`
`

`

`Acknowledgments
`
`My sincerest thanks to the members of my committee. Bryan D. Payne, Mahadev Satyanarayanan,
`
`Greg Ganger, and especially my adviser James C. Hoe. Thank you for the conference calls, markups,
`
`difficult questions, letters of reference, and countless feedback. James, thank you for your support
`
`in completeing this task we undertook together. You stood by me while I found my way through
`
`this process.
`
`Thank you to the CUPS lab for taking me in and teaching me the ways of usability, especially
`
`Professors Lorrie Cranor and Lujo Bauer. I look forward to applying that knowledge throughout my
`
`career and for that I will be forever grateful.
`
`Thank you to the members and staff of the Parallel Data Lab, but specifically to Joan Digney
`
`and Karen Lindenfelser, who always work so hard to make it all happen. The retreat provided
`
`a wonderful audience that was receptive to crazy new ideas and offered unparalleled networking
`
`opportunities.
`
`I thank my fellow Hamerschlag Hall A-level labmates: Berkin Akin, Matthew Beckler (hon-
`
`orary), Adam Hartman (honorary), Eric Chung, Aaron Kane, Yoongu Kim, Brett Meyer (honorary),
`
`Peter Milder, John Porche (honarary), Eriko Nurvitadhi Marie Nguyen, Michael Papamichael, Mal-
`
`colm Taylor, Yu Wang, Gabe Weisz, Guanglin Xu, and Milda Zizyte. The countless hours spent
`
`discussing projects over coffee were invaluable. Also thanks to my labmates in the CUPS lab,
`
`particulary Rebecca Hunt Balebako, Michelle Mazurek, Manya Sleeper, Rich Shay, Blase Ur, and
`
`Kami Vaniea. You all were my second home at CMU. Patrick, Brett, and Peter demonstrated how
`
`to be an excellent graduate student to me, even if I have not always lived up to their examples.
`
`To my friends Casey Canfield and Michael Taylor, I will always remember our times together in
`
`Pittsburgh fondly. To the members of the Gigahurtz softball and hockey teams, we did not always
`
`ii
`
`

`

`iii
`
`win, but we definitely competed. To Brent Povis, Ryan Pocratsky, Jason Fox, and Shane Rice, thank
`
`you for teaching me golf, the sport of Andrew Carnegie. All of you fine folks kept me relaxed and
`
`sane in the midst of the stress and trials these past six years.
`
`To my parents Walter and Diane Klemperer who have supported me in so many ways. Thank
`
`you for the encouragement to undertake my Ph.D., for all the phone calls (please forgive me for all
`
`the phone calls I did not make also), the visits, and the wonderful holidays. I know I can always
`
`count on your love and support.
`
`Finally, to my wonderful wife, Kristen, who began this journey with me and has given me so
`
`much love and support to follow it though. We make a really great team.
`
`The research described in this thesis was made possible by the National Science Foundation via
`
`grant #DGE-0903659 and the Bertucci Graduate Fellowship.
`
`

`

`Contents
`
`1
`
`Introduction
`
`1.1 Motivation .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`1.2 Memory Introspection .
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`1.3 Achieving Efficient Introspection . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`1.4 Thesis Contributions
`
`1.5 Thesis Organization .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2 Background
`
`2.1 Hypervisor Background .
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.1.1 Memory Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.1.2 KVM Hypervisor .
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.2 Hypervisor Based Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.2.1 Hypervisor Based Introspection for Security . . . . . . . . . . . . . . . . .
`
`2.2.2
`
`Introspection Software: VMware VProbes . . . . . . . . . . . . . . . . . .
`
`2.2.3
`
`Introspection Software: LibVMI . . . . . . . . . . . . . . . . . . . . . . .
`
`2.3 Detecting the Mebroot Rookit with Introspection . . . . . . . . . . . . . . . . . .
`
`2.3.1 Mebroot Threats
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.3.2 Mebroot Virus Family . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`2.3.3 Differential Mebroot Network Traffic Analysis . . . . . . . . . . . . . . .
`
`2.4 Background Summary .
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`3 Key Ingredients for Efficient Introspection
`
`3.1 Memory Introspection .
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`iv
`
`1
`
`1
`
`2
`
`2
`
`4
`
`4
`
`6
`
`6
`
`6
`
`9
`
`9
`
`9
`
`10
`
`11
`
`11
`
`12
`
`13
`
`13
`
`16
`
`17
`
`17
`
`

`

`v
`
`CONTENTS
`
`3.2 Developing Requirements for Efficient Introspection . . . . . . . . . . . . . . . .
`
`3.2.1
`
`Pausing is too slow so we need coherency . . . . . . . . . . . . . . . . . .
`
`3.2.2
`
`Parallelism without coherency is insufficient
`
`. . . . . . . . . . . . . . . .
`
`3.2.3 Efficient Introspection: Parallelism with Coherency . . . . . . . . . . . . .
`
`3.3 Requirements for Efficient Introspection . . . . . . . . . . . . . . . . . . . . . . .
`
`3.3.1 Requirement 1: Native Memory Introspection Performance . . . . . . . . .
`
`3.3.2 Requirement 2: Coherent Memory Views . . . . . . . . . . . . . . . . . .
`
`3.3.3 Requirement 3: Normal Guest Performance . . . . . . . . . . . . . . . . .
`
`3.4 Existing Introspection Platforms Inadequate . . . . . . . . . . . . . . . . . . . . .
`
`3.5 Summary .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4
`
`Implementing Efficient Introspection by Snapshotting
`
`4.1 High Performance Snapshotting . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.1.1
`
`Stop-and-Copy Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.1.2 Delta-Copy Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.1.3
`
`Pre-Copy Snapshot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.1.4
`
`Snapshotting Mechanism Guidance . . . . . . . . . . . . . . . . . . . . .
`
`4.2 KVM/QEMU Hypervisor Modifications . . . . . . . . . . . . . . . . . . . . . . .
`
`4.2.1 KVM Host Linux Kernel Module . . . . . . . . . . . . . . . . . . . . . .
`
`4.2.2 QEMU Modification Details . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.3 The LibVMI Project Modifications . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.3.1 LibVMI API Changes
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`4.4 Example Minimal LibVMI Application . . . . . . . . . . . . . . . . . . . . . . .
`
`5 Application Benchmark Evaluation
`
`5.1 Benchmark Testing Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.2 Application Benchmarks .
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.2.1 Kernel Build .
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.2.2 ClamAV Antivirus Scan . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.2.3 Apache Web Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.2.4 Netperf Network Performance . . . . . . . . . . . . . . . . . . . . . . . .
`
`18
`
`18
`
`19
`
`19
`
`20
`
`20
`
`20
`
`21
`
`21
`
`23
`
`24
`
`24
`
`25
`
`26
`
`28
`
`29
`
`29
`
`30
`
`30
`
`32
`
`33
`
`35
`
`37
`
`37
`
`38
`
`39
`
`39
`
`44
`
`44
`
`

`

`CONTENTS
`
`5.2.5 Weka Machine Learning . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`5.3 Application Benchmarking: Winners & Losers
`
`. . . . . . . . . . . . . . . . . . .
`
`6 Microbenchmark Evaluation
`
`6.1 Why Microbenchmarking? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.2 Microbenchmark Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.2.1 Application Runtime Microbenchmark . . . . . . . . . . . . . . . . . . .
`
`6.2.2 Memory Load Microbenchmark . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3 Microbenchmark Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3.1
`
`Stop-Copy Snapshot Evaluation . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3.2 Delta-Copy Snapshot Evaluation . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3.3 Drifting Load Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3.4
`
`Pre-Copy Snapshot Evaluation . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.4 Microbenchmark Evaluation: Key Results . . . . . . . . . . . . . . . . . . . . . .
`
`6.4.1
`
`Snapshot Frequency Most Significant Influence on Guest Performance . . .
`
`6.4.2 Delta-Copy Snapshot Offers Superior Performance . . . . . . . . . . . . .
`
`6.4.3 Unaccounted Snapshot Stop-Time . . . . . . . . . . . . . . . . . . . . . .
`
`6.4.4 Dirty Page Tracking is Cheap . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.4.5
`
`Introspection Impact on Guest
`
`. . . . . . . . . . . . . . . . . . . . . . . .
`
`6.4.6
`
`Stop-Copy Snapshotting Impacts Only Guest Runtime . . . . . . . . . . .
`
`6.4.7 Dirty Page List Synchronization is Expensive . . . . . . . . . . . . . . . .
`
`6.5 Efficient Introspection Performance Guidance . . . . . . . . . . . . . . . . . . . .
`
`7 Potential Applications
`
`7.1
`
`Introspection Application Performance Goals . . . . . . . . . . . . . . . . . . . .
`
`7.2 Potential Application: Antivirus Signature Memory Scan . . . . . . . . . . . . . .
`
`7.2.1
`
`Previous Antivirus Capability . . . . . . . . . . . . . . . . . . . . . . . .
`
`7.2.2 Antivirus with Efficient Introspection . . . . . . . . . . . . . . . . . . . .
`
`vi
`
`49
`
`52
`
`53
`
`53
`
`54
`
`55
`
`56
`
`57
`
`58
`
`69
`
`79
`
`83
`
`92
`
`92
`
`92
`
`92
`
`93
`
`93
`
`94
`
`94
`
`95
`
`97
`
`97
`
`98
`
`98
`
`99
`
`7.2.3
`
`Performance Evaluation of Antivirus with Efficient Introspection . . . . . . 100
`
`7.3 Potential Application: Network Integrity Manager . . . . . . . . . . . . . . . . . . 101
`
`7.3.1
`
`Previous Network Scanning Capability.
`
`. . . . . . . . . . . . . . . . . . . 101
`
`

`

`vii
`
`CONTENTS
`
`7.3.2 NetIM with Efficient Introspection . . . . . . . . . . . . . . . . . . . . . . 103
`
`7.3.3
`
`Performance Evaluation of NetIM with Efficient Introspection . . . . . . . 104
`
`7.4 Application Summary .
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
`
`8 Related Work
`
`9 Conclusions
`
`108
`
`112
`
`9.1 The Need for Efficient Snapshotting . . . . . . . . . . . . . . . . . . . . . . . . . 112
`
`9.2 Summary of Work .
`
`.
`
`.
`
`9.3 Concluding Remarks .
`
`9.4 Future Work .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
`
`

`

`List of Tables
`
`2.1 The available mebroot threats from the 2011 Symantec report with NetIM and
`
`DiskIM results and observation notes.
`
`. . . . . . . . . . . . . . . . . . . . . . . .
`
`13
`
`2.2 Network packet capture from both the uninfected host and Mebroot infected guest.
`
`Bold IP addresses indicate traffic only captured by the host. Further analysis indi-
`
`cated that the packets sent to the bolded IP addresses were DNS name resolution
`
`related.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`15
`
`6.1 Memory access pattern summary for the Kernel Build, ClamAV Antivirus Scan,
`
`Apache Web Server, Netperf Network Performance, Bonnie++ Disk Performance,
`
`and Weka Machine Learning Application Benchmarks. The approximate dirty page
`
`working set size for each application is listed for the complete run of the Application
`
`Benchmark and the dirty page working set size for the Application Benchmark when
`
`it is sampled at 1 Hz.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`54
`
`6.2 Memory Bandwidth Microbenchmark performance with dirty page tracking enabled
`
`and disabled for a range of configurations.
`
`. . . . . . . . . . . . . . . . . . . . .
`
`93
`
`6.3 Memory Bandwidth Microbenchmark (lmbench) performance with dirty page syn-
`
`chronization performed with various frequencies. Only dirty page synchronization
`
`was performed, no memory was copied. The highlighted figures indicate an ob-
`
`served performance impact at 4 Hz for the 512 MB lmbench write and 2/4 Hz for
`
`the 1024 MB lmbench write.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`95
`
`viii
`
`

`

`List of Figures
`
`2.1 Depiction of two-level page mapping for two virtual machine guests, their pro-
`
`cesses on the same host, and shadow page table mappings. Figure borrowed with
`
`modification from VMware Performance Evaluation of Intel EPT Hardware As-
`
`sist. http\protect\kern+.2222em\relax//www.vmware.com/pdf/
`
`Perf_ESX_Intel-EPT-eval.pdf . . . . . . . . . . . . . . . . . . . . . .
`
`2.2 Network Integrity NDIS Block Diagram . . . . . . . . . . . . . . . . . . . . . . .
`
`2.3 Offline packet capture traces of (2.3(a)) an infected virtual machine guest and (2.3(b))
`
`7
`
`12
`
`an uninfected virtual machine guest. For each virtual machine guest, a view from
`
`within the guest OS and from outside the guest OS, at the host, are presented. A
`
`large number of extra packets can be observed in the infected host PCAP trace, that
`
`are not observed in the infected guest PCAP trace or either of the uninfected traces.
`
`14
`
`3.1 High Performance Memory Introspection Requirements . . . . . . . . . . . . . . .
`
`21
`
`4.1 Shared Memory Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`25
`
`4.2 Snapshot performance timelines for Stop-and-Copy 4.2(a), Delta-Copy 4.2(b), Pre-
`
`Copy 4.2(c). Worse performance is indicated as darker red and no-impact is indi-
`
`cated in green.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`26
`
`4.3 VMI Tools operation block diagram showing Introspection VM, guest being intro-
`
`spected upon, hypervisor, and hardware. Figure borrowed with modification from
`
`VMI Tools website. http://code.google.com/p/vmitools/ . . . . . .
`
`32
`
`ix
`
`

`

`LIST OF FIGURES
`
`x
`
`5.1 Block diagram describing the application benchmark testing procedure. In Tests #1
`
`and #2 introspection completes successfully before the next snapshot period begins.
`
`Test #3 fails because introspection could not complete before the scheduled start of
`
`the next snapshot period.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`38
`
`5.2 Chart illustrating the Kernel Build Application Benchmark under (a) Stop-Copy and
`
`(b) Delta-Copy snapshotting regimes. (Continued on next page.)
`
`. . . . . . . . . .
`
`40
`
`5.2
`
`(Continued from previous page.) Chart illustrating the Kernel Build Application
`
`Benchmark under (a) Stop-Copy and (b) Delta-Copy snapshotting regimes.
`
`. . . .
`
`41
`
`5.3 Chart illustrating the ClamAV Scan Application Benchmark under (a) Stop-Copy
`
`and (b) Delta-Copy snapshotting regimes. (Continued on next page.)
`
`. . . . . . .
`
`42
`
`5.3
`
`(Continued from previous page.) Chart illustrating the ClamAV Scan Application
`
`Benchmark under (a) Stop-Copy and (b) Delta-Copy snapshotting regimes.
`
`. . . .
`
`43
`
`5.4 Chart illustrating the Apache Web Server Application Benchmark under (a) Stop-
`
`Copy and (b) Delta-Copy snapshotting regimes. (Continued on next page.)
`
`. . . .
`
`45
`
`5.4
`
`(Continued from previous page.) Chart illustrating the Apache Web Server Appli-
`
`cation Benchmark under (a) Stop-Copy and (b) Delta-Copy snapshotting regimes.
`
`.
`
`.
`
`.
`
`. .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`46
`
`5.5 Chart illustrating the Netperf Network Performance Application Benchmark under
`
`(a) Stop-Copy and (b) Delta-Copy snapshotting regimes. (Continued on next page.)
`
`47
`
`5.5
`
`(Continued from previous page.) Chart illustrating the Netperf Network Perfor-
`
`mance Application Benchmark under (a) Stop-Copy and (b) Delta-Copy snapshot-
`
`ting regimes.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`48
`
`5.7
`
`(Continued from previous page.) Chart illustrating the Weka Machine Learning Ap-
`
`plication Benchmark under (a) Stop-Copy and (a) Delta-Copy snapshotting regimes.
`
`51
`
`6.1 Microbenchmark Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.2 Microbenchmark Parameter Rundown . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.3 Microbenchmark Parameter Rundown . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.4 Microbenchmark Stop-Copy No-Load Accounting . . . . . . . . . . . . . . . . .
`
`6.5 Microbenchmark Stop-Copy No-Load Varying Frequency . . . . . . . . . . . . . .
`
`55
`
`57
`
`59
`
`60
`
`61
`
`

`

`xi
`
`LIST OF FIGURES
`
`6.6 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.6 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.7 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.7 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.7 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.7 Microbenchmark Stop-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . .
`
`6.8 Microbenchmark Parameter Rundown . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.9 Microbenchmark Delta-Copy No-Load Accounting . . . . . . . . . . . . . . . . .
`
`6.10 Microbenchmark Delta-Copy No-Load Varying Frequency . . . . . . . . . . . . .
`
`6.11 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.11 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.12 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.12 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.12 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.12 Microbenchmark Delta-Copy Guest-Load Varying WSS . . . . . . . . . . . . . .
`
`6.13 Memory access pattern comparison and the effect of various write patterns on dirty
`
`page creation. All three patterns write 1024 MB into the buffer but in different
`
`ways: pattern (a) writes 1024 MB into a static 1024 MB window, pattern (b) writes
`
`1024 MB into two overlapping 512 MB drifting windows, pattern (c) writes 1024
`
`MB total into sixteen overlapping 64 MB drifting windows. Each of these patterns
`
`results in different dirty page list sizes with corresponding effects on Delta-Copy
`
`snapshot memory copy overhead.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.14 Drifting Write Load .
`
`6.14 Drifting Write Load .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`6.15 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`6.15 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`6.16 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`6.16 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`6.16 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`6.16 Microbenchmark Pre-Copy Guest-Load Varying WSS . . . . . . . . . . . . . . . .
`
`62
`
`63
`
`64
`
`65
`
`66
`
`67
`
`69
`
`70
`
`71
`
`72
`
`73
`
`75
`
`76
`
`77
`
`78
`
`80
`
`81
`
`82
`
`85
`
`86
`
`88
`
`89
`
`90
`
`91
`
`

`

`LIST OF FIGURES
`
`xii
`
`6.17 Impact of introspection .
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`94
`
`6.18 Efficient introspection delta-copy snapshot performance heat map for all tests pre-
`
`sented in this thesis. *Note: no tests were observed with snapshot period 128.0 and
`
`less than 64 MB of dirty pages but performance in this regime will be 100%. . . . .
`
`96
`
`7.1 LibVMI benchmarks (kernel symbol translation, virtual address translation, read
`
`memory chunks, and read memory byte-by-byte) comparing performance between
`
`three interfaces: Xen Zero-Copy, KVM/QEMU One-Copy Socket, and KVM/QEMU
`
`Serial Socket.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`98
`
`7.2 Block diagram showing the host-based Antivirus software performing memory hash-
`
`ing on the introspected guest. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
`
`99
`
`7.3 Efficient introspection microbenchmark performance heat map overlayed with An-
`
`tivirus specific limitations.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
`
`7.4 Network Integrity Module Block Diagram . . . . . . . . . . . . . . . . . . . . . . 102
`
`7.5 Block diagram showing the host-based NetIM software performing differential anal-
`
`ysis comparing the outgoing packets passed by the guest firewall with the packets
`
`observed leaving the guest.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
`
`7.6 Chart illustrating the memory requirements to buffer outgoing network packets for
`
`analysis by the NetIM.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
`
`7.7
`
`efficient introspection performance heat map overlayed with NetIM specific limita-
`
`tions. .
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`.
`
`. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
`
`

`

`

`

`Chapter 1
`
`Introduction
`
`This thesis presents efficient introspection, a technique that supports the development of security
`
`applications in virtualized environments. Memory introspection is the process of examining virtual
`
`machine guest memory from the high-ground of the virtual machine host. The security application
`
`executes in a seperate environment, isolated by virtualization from the potentially infected guest.
`
`Previously existing memory introspection techniques provided some, but not all, of the follow-
`
`ing three essential properties for efficient virtual machine introspection: coherent memory access,
`
`native introspection performance, and normal guest operation. Efficient introspection combines all
`
`three of these properties through high-performance memory snapshotting.
`
`1.1 Motivation
`
`In 2011 four new master boot record (MBR) based rootkits were released: TIDSERV.M, Fispboot,
`
`Alworo, and SMITNYL. Each of these rootkits infects the master boot record so that they will be
`
`loaded, with kernel privileges, before the Windows operating system (OS). Loading before the OS
`
`allows the virus to hide itself by subverting the kernel’s own procedures. The TIDSERV.M-based
`
`Auleron botnet also uses its kernel level privileges to steal login, password and credit card data
`
`from the network traffic of infected systems. The threat posed by the Auleron botnet to banking
`
`data is difficult to counter with OS-based antivirus software. OS-based antivirus runs with the same
`
`privilege-level as the rootkits. The antivirus is vulnerable to tainting and spoofing by the rootkits,
`
`rendering detection much more difficult. Virtual machine (VM) based antivirus protects OSes from
`
`1
`
`

`

`1.2. Memory Introspection
`
`2
`
`a high-ground position with a higher privilege than the viruses.
`
`1.2 Memory Introspection
`
`Memory introspection is a technique for hypervisor based security detection where the VM guest
`
`is inspected (or introspected) by the hypervisor. Existing introspection implementations typically
`
`mask memory access performance deficiencies with limited checking or long detection latencies.
`
`Slow memory access performance limits an introspection application’s ability to check the entire
`
`memory state of a VM guest. To counter memory access performance limits, introspection appli-
`
`cation designers may try to seek out opportunities to investigate only smaller, more critical parts
`
`of the guest state so that the overall impact of slow memory access is minimized. Other works
`
`may periodically sample the guest memory, minimizing the memory access at any given sample,
`
`but over time build up a statistical knowledge of the overall guest state. While these contributions
`
`are certainly valuable, accelerating memory access capability directly enables new classes of in-
`
`trospection applications that were previously dismissed as too slow or resource intensive. Newly
`
`developed introspection applications will utilize the increased memory access support to provide
`
`real-time security protections against VM resident malware.
`
`Inefficient introspection mechanisms not only limit introspection applications, but they can ad-
`
`versely effect guest performance. Guests that are taken off-line for checking or that have to compete
`
`for resources with inefficient introspection platforms may display poor performance. End-users may
`
`not tolerate poor performance and disable the introspection-based mitigation entirely, no matter the
`
`threats posed by rootkits.
`
`1.3 Achieving Efficient Introspection
`
`Increasing introspection performance requires answering two important research questions. First,
`
`how fast are the memory access performance requirements for a useful, real-time virtualization-
`
`based detection tool? Native memory performance is certainly a reasonable baseline but invites the
`
`second research question: can native speed be reached? How can the performance bottlenecks be
`
`understood, explained, and removed? This thesis answers these questions.
`
`

`

`3
`
`Chapter 1. Introduction
`
`This thesis presents a concurrent-computing approach to accelerating hypervisor introspection
`
`of virtual machine guest memory. Existing virtual machine introspection tools are extended to pro-
`
`vide parallelism by snapshotting guest memory. Snapshotting guest memory will allow the efficient
`
`introspection system to:
`
`• decouple memory introspection from virtual machine guest execution,
`• establish coherent and consistent memory views between the host-based introspection appli-
`cations and the running guest,
`• provide intelligent memory translation for native speed access to introspected memory spaces.
`
`Existing introspection systems have provided one or two of these properties but not all three at
`
`once. This thesis will present a new real-time, concurrent-computing approach to accelerate hyper-
`
`visor based introspection of virtual machine guest memory, which I call efficient introspection, that
`
`combines all three elements to improve performance and security.
`
`The resulting system will add efficient introspection memory access capability while maintain-
`
`ing native guest performance. These efficiency increases will provide security system designers with
`
`greater flexibility to maintain guest performance and interactivity while increasing security check-
`
`ing capability. Efficiency is demonstrated through application and microbenchmark evaluation as
`
`well as through several potential introspection application investigations. Application benchmarks
`
`show that normal guest performance can be maintained under a variety of introspection scenarios.
`
`Microbenchmarking provides a systematic exploration of the introspection scenarios and helps ex-
`
`plain why the application benchmarks perform well. The potential application discussion compares
`
`the efficient introspection to previously available introspection platforms and provides guidance for
`
`introspection application developers who are concerned with performance.
`
`Detection techniques that had been formerly dismissed as too slow have been reevaluated in the
`
`context of efficient introspection and shown to be viable. This thesis explores two such techniques:
`
`memory-signature antivirus detection and network packet differential analysis. Specific limitations
`
`in introspection platforms had limited the utility of these application but they are now made possible
`
`with efficient introspection. These new techniques enabled by high performance memory introspec-
`
`tion will increase protection for securing critical computing applications.
`
`

`

`1.4. Thesis Contributions
`
`1.4 Thesis Contributions
`
`The main contributions in this thesis are summarized as follows:
`
`4
`
`• Developing three requirements critical to the implementation of efficient introspection: co-
`herent memory access, native memory introspection performance, and normal guest perfor-
`
`mance.
`
`• Open-Source release of the efficient introspection prototype as an element of the LibVMI [1]
`introspection project.
`
`• Evaluation of the performance impact of efficient introspection on application benchmarks
`reveals normal guest operation.
`
`• Systematic exploration of introspection scenarios through microbenchmarking explains the
`behavior of the application benchmarks.
`
`• Identification of key factors in efficient introspection application performance impact pro-
`vide guidance for potential introspection application developers in the form of performance
`
`estimation techniques.
`
`• Examination of potential applications of efficient introspection: memory-signature antivirus
`scanning and network packet differential analysis.
`
`These steps towards defining and realizing efficient memory introspection leave several inter-
`
`esting research directions unaddressed. These open issues are summarized in Section 9.4.
`
`1.5 Thesis Organization
`
`Outline The rest of this prospectus is arranged as follows: Chapter 2 presents background on hyper-
`
`visors, malware, and hypervisor based malware detection; Chapter 3 expands the concept of efficient
`
`introspection and how it will improve detection performance; Chapter 4 presents high-performance
`
`snapshotting mechanisms and describes their extension to LibVMI introspection platform; Chap-
`
`ter 5 discusses the evaluation of the efficient introspection prototype with application benchmarks;
`
`

`

`5
`
`Chapter 1. Introduction
`
`Chapter 6 discusses the systematic microbenchmark evaluation of the efficient introspection proto-
`
`type; Chapter 7 discusses potential security applications enabled by efficient introspection; Chap-
`
`ter 8 is related work; and Chapter 9 concludes the dissertation.
`
`

`

`Chapter 2
`
`Background
`
`In this chapte