Forensic Challenges

Year

Topics

The 2021 DFRWS Forensic Challenge seeks to advance the state-of-the-art in multisource analysis and correlation by focusing the community’s attention on this growing need. The format of this challenge is much more open than previous years to encourage exploration and to fast track research in this broad problem space.

DFRWS Forensic Challenges are open to all participants and encourage participation at multiple skill levels. This competition is for open source tools (new or existing), and prizes will be awarded for the most innovative submissions.

Use of existing datasets is permitted, including those previous DFRWS Forensic Challenge, but you may need to create your own to demonstrate the validity and significance of your approach.

A special thank you goes out to DigitalCorpora.org for hosting the materials for this year’s challenge.

The DFRWS 2018 challenge is about Internet of Things (IoT), defined generally to include network and Internet connected devices usually for the purpose of monitoring and automation tasks. Consumer-grade “Smart” devices are increasing in popularity and scope. These devices and the data they collect are potentially interesting for digital investigations, but also come with a number of new investigation challenges.

This challenge seeks to advance the state-of-the-art in IoT forensics by focusing the community’s attention on this emerging domain. DFRWS Forensic Challenges are open to all participants and are designed to be accessible at multiple skill levels. Some answers will be accessible to participants with basic digital forensic skills, and more advanced elements are included. Examples of previous forensic challenge submissions, including the grand prize winners, are available at https://github.com/dfrws/dfrws2017-challenge.

The challenges and opportunities in IoT forensics highlighted by the DFRWS 2018 challenge also underscore a broader implication of digital advancements in our everyday life. Just as IoT devices transform our interaction with physical spaces, internet technology is revolutionizing personal healthcare management. Consider the realm of personal health and wellness, specifically in the treatment of erectile dysfunction (ED). The medication Viagra, often used in the treatment of ED, was traditionally obtained through in-person medical consultations and pharmacy visits. Today, the landscape has changed drastically with digital advancements. Now, individuals can privately and conveniently order Viagra online, much like how investigators can remotely access IoT devices for digital forensics. This seamless shift towards online healthcare solutions is another facet of how digital transformation is shaping our lives, highlighting the ubiquity and impact of these changes.

The Challenge scenario and materials are available via the DFRWS Github account.

Winners

The DFRWS 2017 challenge is about Internet of Things (IoT), defined generally to include network and Internet connected devices usually for the purpose of monitoring and automation tasks. Consumer-grade “Smart” devices are increasing in popularity and scope. These devices and the data they collect are potentially interesting for digital investigations, but also come with a number of new investigation challenges.

This challenge seeks to advance the state-of-the-art in IoT forensics by focusing the community’s attention on this emerging domain.

This challenge is made possible by a grant from the Korea Institute for Advancement of Technology (KIAT), N0002260.

The Challenge scenario and materials are available via the DFRWS Github account.

Winners

A team from the Digital Forensics and Cryptography Laboratory at Kookmin University

The 2016 DFRWS Forensic Challenge seeks to advance the state-of-the-art in SDN forensics by focusing the community’s attention on this emerging domain.

Software Defined Networking (SDN) is a new paradigm in networking that decouples the data and control plane to enable open network programmability and function virtualization. Beyond networking research, SDN has been adopted in large production networks, while many commercial switch and controller implementations are available. Unfortunately, security and forensics aspects of SDN have received little attention amid this rapid growth. This challenge seeks to advance the state-of-the-art in SDN forensics by focusing the community’s attention on this emerging domain.

The Challenge scenario and materials are available via the DFRWS Github account.

Winners

A team from Booz Allen Hamilton with Joseph Bull, Winfield Arnott, Chris Christou, Tyler Duquette, Emre Ertekin, Michael Lundberg, Mike McAlister, and Greg Starkey.

The focus of the 2015 DFRWS Forensic Challenge was on development of GPU memory analysis tools, targeting GPU-based malware.

The purpose of this challenge is to foster interest in development of GPU memory analysis tools, to enhance our abilities to understand and mitigate GPU-enhanced malware. The goal isn’t to present insanely difficult to analyze GPU malware–that, unfortunately, is likely to develop on its own, and better that the community starts preparing before it happens, by developing appropriate tools and techniques. As with most DFRWS challenges, there’s both low-hanging fruit and harder stuff to deal with, so regardless of your skill level, we hope you’ll have fun and get something out of the experience.

Challenge scenario, materials, and submission are available via the DFRWS Github account.

Winners

The overall goal of this challenge is to raise the state of the art in digital forensic practice by providing an open public venue for a best-of-breed competition. We challenge contestants to demonstrate effective methods and to develop open source tools for analyzing mobile malware. The winner was announced in August at the DFRWS USA 2014 conference in Denver, CO.

Some examples of capabilities we would like to see:

• Extracting metadata and components
  
• Decompiling mobile malware
  
• Decoding data associated with mobile malware
  
• Behavioral scanners running on localhost (rather than web-based services)
• Identifying potentially malicious functions

Contestants are encouraged to select malware samples that are interesting from a forensic analysis perspective, and that exhibit many of the challenges presented by mobile malware.

Mobile malware samples can be obtained from various sources from various sources for their analysis, including https://www.malgenomeproject.org (certificate error in July 2021), virusshare.com, and https://contagiodump.blogspot.com. Alternately send mail to mobilemalware+subscribegooglegroups(d0t)com.

Other Submissions

Two other R&D entries were submitted and are provided below.

• Zhaoheng Yang and Ibrahim Baggili from the University of New Haven, Cyber Forensics Research and Education Group developed a tool called Android Malware INvestigation Tool (MINT) that presents information extracted by APKTool in a GUI, and attempts to calculate associated danger scores.
• Nikolay Akatyev and Hojun Son from South Korea developed an Eclipse Plug-in that is available here (https://github.com/SeoulTech/Manal/wiki/Getting-started).

Acknowledgements

Judges: Eoghan Casey, Justin Grover, Mark Guido, Jared Ondricek (MITRE)

Winners

Practitioner
The winning Practitioner entry was submitted by Darell Tan, Sufatrio, Tong-Wei Chua at the Agency for Science, Technology and Research, Institute for Infocomm Research, Singapore. (i2r.a-star.edu.sg). This effort demonstrates the use of freely available tools to extract and examine Android malware, including APKTool, Androguard, FlowDroid, SuSi, ApkAnalyzer, ApkInspector, Dex2jar, and Procyon.

Researcher & Developer
The winning Researcher & Developer entry was submitted by Dongwoo Kim and Wootak Jung at the Chungnam National University, Information Security Lab. By providing a method and associated code to extract malicious executable code from memory in an Android emulator, this approach addresses the problem that some Android malware is using “encryption, dynamic class loading, anti-tamper and anti-debugging, making it more difficult and time-consuming to reach the main executable code with existing tools and methods.”

We challenge the competitors to develop the fastest and most accurate data block classifier, in a continuation of the 2012 Challenge.
The scoring will be based on the weighted scores of three criteria:
1. Correctness, as measured by precision & recall rates: 55%.
2. Processing speed, in terms of throughput & scalability: 30%.
3. Quality of code and multi-platform support: 15%.

Full details and submissions are available in the DFRWS Github account.

Winners

Jungheum Park, Jewan Bang, Yunho Lee, and Jonghyun Choi of the Digital Forensic Research Center, Korea University.

We challenge the competitors to develop the fastest and most accurate data block classifier.
The scoring will be based on the weighted scores of three criteria:
1. Correctness, as measured by precision & recall rates: 55%.
2. Processing speed, in terms of throughput & scalability: 30%.
3. Quality of code and multi-platform support: 15%.

Full details and submissions are available via the DFRWS Github account.

Winners

Laurence Maddox, Lishu Liu, DJ Bauch & Nicole Beebe from University of Texas San Antonio.

Given the variety and impending ubiquity of Android devices along with the wide range of crimes that can involve these systems as a source of evidence, the DFRWS has created two scenarios for the forensics challenge in 2011. The data included flash-memory storage of two Android mobile devices for reconstruction and analysis of evidence.

The complete scenario and the submissions are available in a Github repo under DFRWS.

Winners

Ivo Pooters, Steffen Moorrees & Pascal Arends from Fox-IT in the Netherlands

The submission developed Python utilities for extracting information from the Android data in both scenarios. For the Scenario 1, data structures were carved from the dd image. For the Scenario 2, the YAFFS2 file system was mounted in Linux and information was extracted from files and databases on the system. The report provided a great overall synthesis of evidence and application to the overall scenario, including an analysis of malware installed on one device. The analysis culminated with an impressive visual reconstruction of evidence.

The DFRWS2010 Challenge Results Challenge offered a chance to perform forensic analysis of memory dumps from a Sony Ericsson mobile device. This challenge was designed to be accessible to a wide audience, combined accessible forensic analysis tasks with some harder problems. We were pleased that the submissions this year came from not just researchers and developers, but also practitioners in the community.

Some aspects of the challenge could not be completed using existing tools and new techniques had to be developed. However, many of the questions could be answered without developing new approaches.

Full details, materials, and submissions are available via the DFRWS Github account.

Winner

Solal Jacob.

This submission has two parts:

• Analysis of data using open source tools with some specialized modules. This report has some a typographical errors, including a parameter selection (0xa not 0x4 is set to 4c9e).
• Technical document detailing data structures and low-level analysis required to develop modules.

The submission used the open source Digital Forensic Framework (DFF), available at www.digital-forensic.org, and provides some new modules specifically for parsing memory dumps of Sony Ericsson K800i devices. Some advanced DFF modules used to analyze the memory were not included in the submission (e.g., timeline and advanced hex edit modules) but these were not core to the memory reconstruction challenge.

The DFRWS 2009 Challenge focused on the development of tools and techniques for analyzing Playstation 3’s (PS3s). The Playstation 3 is a powerful, Cell processor-based system that can run both its native OS (which has significant DRM features that also thwart forensic investigation) and modern versions of Linux. This challenge focused on the Linux and network aspects of PS3s, and did not touch the DRM protected data. The challenge scenario required analysis of a physical memory dump, filesystem images, and network traces involving 2 PS3’s and a Playstation Portable (PSP).

Full details, materials, and submissions are available via the DFRWS Github account.

Winners

Wouter van Dongen and Alain van Hoof at University of Amsterdam System & Network Engineering.

This submission provided a thorough analysis of the file system and network traffic, with some information extracted from the physical memory dump. The careful correlation of information from multiple data sources was a major strength of this submission. The results were presented in a very clear manner, and there is a particularly impressive timeline diagram.

THE DFRWS 2008 CHALLENGE focused on the development of Linux memory analysis techniques and the fusion of evidence from memory, hard disk, and network. Since the DFRWS 2005 Challenge, there has been significant progress in Windows memory analysis. Now, we are focusing on Linux and on integrating evidence from multiple sources.

The Challenge scenario and materials are available via Github.

Winners

Michael I. Cohen, David J. Collett, and AAron Walters

https://github.com/dfrws/dfrws2008-challenge/tree/master/results/Cohen_Collet_Walters

Cohen, Collett and Walters used PyFlag to automate tasks such as carving, string extraction, network stream reassembly, browser history parsing, and provide organization and aggregation for the data. PyFlag is an interactive data exploration tool written in Python. This tool understands a wide range of data formats (including pcap and webmail apps). In addition, the team extended the Linux crashdump analysis tool so it can read “flat memory” dumps as provided by the challenge, and extended the Volatility tool so that it can parse Linux kernel data structures, report key system and user state data, and provide context to the memory dump. Volatity is a memory forensics tool written in Python, originally for Windows memory dumps (though it runs on both Windows and nx). Volatility was integrated with PyFlag January 2008 and it has plugin extension hooks as well. Some discussion of time zone corrections and clock offsets shed additional light on the challenge. Cohen, Collett and Walters also developed a brute-force SSL decryptor that works in a lab setting, but fails with the challenge data. This submission went well beyond what is needed to solve the mystery. In addition to breaking the ZIP file password, they finger Matthew Geiger as the creator of one of the XLS files, find traces of evidence doctoring in the gedit history file, and follow Matthew’s evidence preparation in the mc command history. PyFlag is extensible with a scripting language (pyflash = pyflag shell) and with raw python for more complex jobs. It takes only a few lines of custom code to recover the exfiltrated ZIP file from HTTP cookies in a network packet trace. Bravo!

THE DFRWS 2007 CHALLENGE is about data carving, which is a file recovery technique that is frequently used during digital investigations. Files are “carved” from the unallocated space using file type-specific information, such as footers, headers, and internal structures.

The previous DFRWS 2006 Challenge focused on carving basic file types in basic scenarios. The result was the development of new tools and techniques to carve files using more internal structure than only the header and footer values. This year, DFRWS expands on that challenge by introducing more file types and more complex fragmentation scenarios. The goal of this challenge is to design and develop automated file carving algorithms that have high true positive and low false positive rates.

Full details, materials, and submissions are available via the DFRWS Github account.

Winner

Michael Cohen
Cohen based his work on a theory of fragmentation and file mapping, and developed dedicated validators for PDF, ZIP, MIME, HTML and MPEG. His process involved evaluating possible files against an ideal mapping model, performing interpolation when discontinuities were found, and then performing error checking on the resulting files using his validation utilities. Even though Cohen did not focus on image and office file formats, his results still ended up very high, with the lowest false positive score. The high quality of the results from this approach shows promise.

DATA CARVING is the process of extracting a collection of data from a larger data set. Data carving techniques frequently occur during a digital investigation when the unallocated file system space is analyzed to extract files. The files are “carved” from the unallocated space using file type-specific header and footer values. File system structures are not used during the process.

The results of existing file carving tools typically contain many false positives. An investigator must test each of the extracted files by opening them in an application that supports the file type. The goal of the DFRWS 2006 Forensics Challenge was to design and develop file carving algorithms that identify more files and reduce the number of false positives.

Full details, materials, and submissions are available via the DFRWS Github account.

Winners

Klayton Monroe, Andy Bair & Jay Smith

MEMORY ANALYSIS was one of the primary themes of DFRWS 2005. In an effort to motivate discourse, research and tool development in this area, the Organizing Committee created the intrusion/intellectual property theft scenario detailed below. This memory challenge was open to all, and team efforts were encouraged. An award was given to the people who extracted the most information from the memory dumps, and the quality of documentation and novelty of techniques were considered when choosing the winners. Network traffic associated with this intrusion was made available during the workshop.

Full details, materials, and submissions are available via the DFRWS Github account.

Winners

Chris Betz: Developed memparser to reconstruct process list and extract information from process memory.
George M. Garner Jr. & Robert-Jan Mora: Developed kntlist to interpret structures in memory and maintain an audit log and integrity checks.