Blue Screen of Death

A software update crashed millions of computers on July 19, 2024, causing widespread flight cancellations. I happened to be flying that day, and while waiting in the terminal, I started thinking about kernels.

Written in the days after the July 19, 2024 incident.

My flight was luckily only delayed by two hours, but I felt for those whose journeys were more severely disrupted. That sympathy dissipated when Delta notified me four hours before my return flight a few days later that they had canceled it. The cause was a widespread failure that Microsoft estimated affected 8.5 million Windows devices running CrowdStrike, security software that CrowdStrike says is used by more than half of Fortune 500 companies.

The incident

At 04:09 UTC on July 19, CrowdStrike released an update for its Falcon sensor software on Windows systems. A defect in this update caused these systems to crash, triggering the Blue Screen of Death (BSOD). Affected machines could enter a crash-and-restart loop that required manual recovery.

CrowdStrike's later root-cause analysis described a mismatch between the Rapid Response Content supplied to the sensor and the number of input fields the Content Interpreter expected. The resulting out-of-bounds read caused a Windows crash. The problematic content arrived through CrowdStrike's update system, so affected sensors received it without a normal operating-system update cycle.

Like many endpoint security products, CrowdStrike's Falcon sensor includes a kernel-mode driver so it can observe and enforce controls below ordinary applications. This level of access introduces the risk of an application crashing the entire system (or worse), and after that occurred on July 19 and I was left to my own devices in the airport terminal, I started thinking about the variations of kernels out there and what could be done to prevent this from happening again while also realizing I was severely understudied on the topic.

Kernels

A kernel is the core of an operating system, providing essential services to other programs. There are many different types of kernels, including monolithic, micro, hybrid, exo, and nano, all with their pros and cons.

Type	Description	Pros	Cons
Monolithic kernels	Keep many operating-system services in kernel space	Direct communication and low overhead	A fault in privileged code can crash or compromise the system
Microkernels	Move many services to isolated user-space processes	Smaller privileged core and stronger fault isolation	More communication overhead and system-design complexity
Hybrid kernels	Combine a microkernel-derived design with more services in kernel space	Practical balance chosen by systems such as Windows NT and XNU	Retain a large privileged attack and failure surface
Exokernels	Securely expose low-level resources so applications can build abstractions	Application-specific control	Research-oriented design with a demanding programming model
Nanokernels	A loosely used term for a minimal privileged hardware layer	Small scope	No single standard definition or general-purpose feature set

Multi-tier isolation

What follows is a learning experiment from someone new to the domain, not a demonstrated fix. With that caveat, a multi-tier kernel architecture could in principle prevent higher-level applications from having unrestricted access to the kernel:

• Core OS functions operate at the lowest, most privileged level.
• Essential drivers and security software run at a slightly higher level with restricted access.
• Application-level software operates at the highest level with minimal kernel access.

During my brief research, I found academic work discussing three different multilevel security kernel architectures. The authors conclude that the choice of architecture depends on the specific requirements of a system or deployment scenario.

The goal is to prevent one component failure from taking down the system. A least-privilege design can reduce blast radius, but this sketch does not show that a different kernel architecture would have prevented the Falcon failure. Endpoint security also faces detection, compatibility, and performance tradeoffs when work moves outside the kernel.

SimpleOS: a prototype implementation

After reading about kernels, I decided to prototype SimpleOS as a way to learn. Since it was my first kernel, I made it monolithic to understand standard practices.

To start, SimpleOS featured the following:

1. Monolithic kernel design
2. Interrupt handling system with custom handler support
3. Memory management with paging and simple heap allocation
4. Basic multitasking using round-robin scheduling
5. Essential x86 structures (GDT, IDT) and initialization

SimpleOS was a study project built to understand those kernel mechanics directly.