About Past Issues Editorial Board

KAIST
BREAKTHROUGHS

Research Webzine of the KAIST College of Engineering since 2014

Spring 2025 Vol. 24
Computing

A new bug-finding system to Identify Kernel Out-of-Order Concurrency Bugs

March 4, 2025   hit 294

A KAIST research team has developed a groundbreaking system to detect concurrency bugs in CPUs based on out-of-order execution such as Apple M3 processor. Using this newly developed automated technology, the research team identified a total of 11 new bugs in Linux, including several critical ones that could cause severe damage to systems in the Linux kernel, an operating system used for servers

Workflow overview of the system. (1) Profiling memory accesses of operating system. (2) Planning how to test out-of-order concurrency bugs. (3) Running in-vivo test to identify bugs.

The research team led by Professors Youngjin Kwon and Inshik Shin has developed an operating system bug detection system capable of identifying and diagnosing concurrency bugs caused by out-of-order execution on ARM servers and devices equipped with Apple processors—all without requiring physical CPU hardware.

 

Concurrency bugs occurs when an application executed in multi-core CPU interacts in unintended and unpredictable ways. Particularly, concurrency bugs arising from out-of-order execution are notoriously difficult to detect through developer intuition or conventional testing, often remaining undiscovered for years even by expert developers. If exploited by attackers, these bugs could lead to system hacking or operational failures. To address this, the research team designed and implemented an automated testing system capable of identifying such bugs without requiring any manual effort.
Figure 1: two key techniques to emulate out-of-order execution in OEMU

A notable feature of the developed bug detection technology is its ability to accurately and efficiently emulate such Apple processors called OEMU (Out-of-order EMUlator) as shown in the above figure. This means that even without access to Apple processors or ARM servers, a commonly used Intel CPU-based server can run the operating system in an emulated environment to detect these bugs.

 

The team further integrated their technique with fuzzing methods for finding concurrency bugs in kernels. Fuzzing, a widely used technique employed by companies like Google to detect operating system bugs, works by randomly generating input values to identify bugs and errors during execution.
Figure 2: newly detected concurrency bugs in recent Linux Kernels

Using this newly developed technology, the research team identified and reported 11 new concurrency bugs in the latest Linux operating system, providing patches to Linux developers. These bugs were found across critical kernel components, including TLS code used for encrypted networks, posing significant security risks.

 

The team’s technology is not limited to Linux; it is also applicable to other operating systems, such as Android, commonly used on mobile devices, and Windows. Its potential applications are expected to expand significantly in the future. This research was recognized as a Best Paper at the 2024 Symposium on Operating Systems Principles (SOSP), the most prestigious conference in the field of systems research. The research outcomes are publicly available as open source:

https://github.com/casys-kaist/ozz