Blue Screen of Death

A software update crashed millions of computers on July 19, 2024, causing widespread flight cancellations. I happened to be flying that day, and while waiting in the terminal, I started thinking about kernels.

Written in the days after the July 19, 2024 incident.

My flight was luckily only delayed by two hours, but I felt for those whose journeys were more severely disrupted. That sympathy dissipated when Delta notified me four hours before my return flight a few days later that they had canceled it. The cause was a widespread failure that Microsoft estimated affected 8.5 million Windows devices running CrowdStrike, security software that CrowdStrike says is used by more than half of Fortune 500 companies.

The Incident

At 04:09 UTC on July 19, CrowdStrike released an update for its Falcon sensor software on Windows systems. A defect in this update caused these systems to crash, triggering the infamous Blue Screen of Death (BSOD), which signals that the computer is incapacitated and requires a manual reboot.

The root cause was a problematic modification to a configuration file, Channel File 291, which handles screening named pipes. This led to an out-of-bounds memory read, causing an invalid page fault, which was forced onto each device by an auto-update from CrowdStrike. In simpler terms, the program tried to access memory it shouldn't, causing a crash, and this update was forced upon millions of systems, ultimately bringing them into a BSOD state and rendering them temporarily useless.

Like many security products, CrowdStrike's Falcon sensor operates at the kernel level, where it can watch the whole system. This level of access introduces the risk of an application crashing the entire system (or worse), and after that occurred on July 19 and I was left to my own devices in the airport terminal, I started thinking about the variations of kernels out there and what could be done to prevent this from happening again while also realizing I was severely understudied on the topic.

Kernels

A kernel is the core of an operating system, providing essential services to other programs. There are many different types of kernels, including monolithic, micro, hybrid, exo, and nano - all with their pros and cons.

Type	Description	Pros	Cons
Monolithic Kernels	Run all OS services in kernel space	High performance, efficient	Large size, potential system-wide crashes
Microkernels	Most services run in user space	Enhanced stability and security	Lower performance
Hybrid Kernels	Balance between monolithic and microkernels	Optimized performance and modularity	Increased complexity
Exokernels	Gives programs direct control instead of relying on kernel	High performance, efficient	Increased complexity
Nanokernels	Offer bare minimum services for hardware management	Minimal attack surface, specialized	Limited functionality, not general-purpose

Multi-Tier Kernels

What follows is a learning experiment from someone new to the domain, not a demonstrated fix. With that caveat, a multi-tier kernel architecture could in principle prevent higher-level applications from having unrestricted access to the kernel:

• Core OS functions operate at the lowest, most privileged level.
• Essential drivers and security software run at a slightly higher level with restricted access.
• Application-level software operates at the highest level with minimal kernel access.

During my brief research, I found academic work discussing three different multilevel security kernel architectures. The authors conclude that the choice of architecture depends on the specific requirements of a system or deployment scenario.

The goal is to prevent a single issue from compromising the entire system or affecting core OS functions. A least-privilege design along these lines might have limited the impact of a bug like the one in Falcon's Channel File 291 - though shipping security software outside the kernel has real detection and performance trade-offs that this sketch does not address.

SimpleOS: A Prototype Implementation

After reading about kernels, I decided to prototype SimpleOS as a way to learn. Since it was my first kernel, I made it monolithic to understand standard practices.

To start, SimpleOS featured the following:

1. Monolithic kernel design
2. Interrupt handling system with custom handler support
3. Memory management with paging and simple heap allocation
4. Basic multitasking using round-robin scheduling
5. Essential x86 structures (GDT, IDT) and initialization

SimpleOS was a study project built to understand those kernel mechanics directly.