An operating system kernel is the core software layer that mediates between user programs and hardware, managing resources and enforcing protection boundaries. Kernels come in three primary architectural styles: monolithic (all OS services run in a single privileged address space), microkernel (only minimal services run in kernel mode; others run as user-space servers), and hybrid (a pragmatic blend used by macOS and Windows). The architectural choice determines performance characteristics, reliability, and extensibility tradeoffs. Understanding kernel structure is the foundation for understanding every other OS concept.
Compare Linux (monolithic) and macOS/Mach (hybrid microkernel) side-by-side. Draw the privilege boundary and identify which services cross it. Then explore what a system call looks like at the assembly level.
If you have studied instruction set architecture, you know that CPUs provide privileged instructions that only certain code is allowed to execute — operations like disabling interrupts, accessing I/O ports, or modifying page tables. The kernel is the software that runs with these privileges. Everything else — your web browser, your text editor, the shell — runs in user mode with restricted access. The kernel is the gatekeeper: when a user program needs to do something privileged (read a file, send a network packet, allocate memory), it must ask the kernel through a system call, which switches the CPU from user mode to kernel mode, performs the operation, and switches back.
The fundamental design question in OS architecture is: how much code runs in kernel mode? A monolithic kernel puts everything — process scheduling, memory management, file systems, device drivers, networking — into a single large program running in kernel space. Linux is the canonical example. The advantage is performance: since all kernel components share the same address space, they can call each other directly with ordinary function calls, with no overhead from context switches or message passing. The disadvantage is reliability and security: a bug in any kernel component (especially device drivers, which are the largest and least-tested part of most kernels) can crash or corrupt the entire system. A single faulty video driver can bring down the whole OS.
A microkernel takes the opposite approach: only the absolute minimum runs in kernel mode — typically just inter-process communication (IPC), basic scheduling, and memory management. Everything else — file systems, device drivers, networking — runs as separate user-space server processes. When a program needs to read a file, it sends a message to the file system server via the kernel's IPC mechanism, the server performs the operation, and sends the result back. This isolation means a crashing driver only brings down its own process, not the entire system. The seL4 microkernel has even been formally verified — mathematically proven to be free of certain classes of bugs. The classic disadvantage is performance: what was a single function call in a monolithic kernel now requires multiple context switches and message copies. However, modern microkernels have narrowed this gap significantly through careful IPC optimization.
In practice, most widely-used operating systems are hybrids. Windows NT and macOS both use a design where a relatively large kernel provides core services (more than a pure microkernel) but some components run in user space (more modular than a pure monolithic kernel). macOS is built on the Mach microkernel but bundles a BSD subsystem into kernel space for performance. The choice between these architectures is not about one being objectively better — it is about which tradeoffs matter for a given system. Embedded and safety-critical systems favor microkernels for their isolation guarantees. General-purpose desktop and server systems favor monolithic or hybrid designs for their performance. Understanding this architectural spectrum is the foundation for every OS topic that follows, because whether you are studying process management, memory management, or file systems, the answer to "how does this work?" always starts with "it depends on the kernel architecture."