Authors: Thanh Nguyen (Nvidia), Meni Orenbach (Nvidia), and Ahmad Atamli (Nvidia)
DFRWS USA 2022
Abstract
Live system call traces provide essential information in analyzing modern malware. Prior work demonstrated how system call traces can be used to differentiate benign from malicious applications. For example, ransomware invokes file system API to remove user access to its sensitive data, and ask for a ransom to restore the access privileges.
Unfortunately, current methods and tools focus on offline reconstruction using memory dumps of the entire system. While it is possible to use such methods in live analysis by pausing execution and tracking system calls, it severely hinders the system performance. In this paper, we present the design and implementation of our method to trace system calls in Linux-based systems. We show how using our method allows obtaining and analyzing the system calls performed by real ransomware. To build a system call trace for a given process, we first retrieve the process context from a raw live memory image. Next, we analyze this context to identify the last invoked system call. We repeat the process and get a full trace.
We prototype our approach as a LibVMI tool using only memory read APIs and compare the accuracy of our method with that of strace, an intrusive and dedicated system call tracer. We show that we achieve the same level of accuracy, yet, without impacting the performance of the traced program. Further, we demonstrate how non-intrusive tracing has a better performance compared to trap-based tracing when using LibVMI traps and interrupts APIs to hook system call events. Finally, we provide an analysis of system calls performed by mature ransomware and demonstrate how our extracted traces can be used to identify them.