Inside Windows Process Creation: What Really Happens After You Double-Click an EXE

Have you ever wondered what happens in that split second between double-clicking an executable and seeing your application window appear on screen? That seemingly instantaneous moment actually involves a complex orchestration of system components, memory management, and security checks that most developers never see. Today, we'll pull back the curtain on Windows internals and trace the complete journey from a simple API call to running code.

What You'll Need to Follow Along

Required Tools (All Free):

• Process Monitor (ProcMon) - Download from Microsoft Sysinternals
• Process Explorer - Also from Sysinternals
• Visual Studio Community or any C++ compiler
• PowerShell (built into Windows)
• Windows Performance Toolkit (optional, for advanced tracing)

Test Subject: We'll use Windows Notepad (notepad.exe) as our guinea pig since it's simple and available on every Windows system.

Stage 1: The Shell Receives Your Click

When you double-click notepad.exe in Windows Explorer, you're not directly communicating with the operating system kernel. Instead, you're talking to the Windows Shell, which is the explorer.exe process that provides the desktop environment and file management interface. This distinction is crucial because the shell acts as an intermediary that interprets your actions and translates them into appropriate system calls.

The moment your mouse button releases after that double-click, Windows Explorer springs into action. But it doesn't immediately know what to do with the file you've clicked. Instead, it must consult the Windows Registry to determine how to handle this particular type of file. This process is called file association lookup, and it's fundamental to how Windows knows which application should open which files.

Experiment 1: Watching Shell Activity

Let's observe this process in real-time using Process Monitor, a powerful tool that shows us exactly what's happening behind the scenes:

1. Open Process Monitor:
   • Download ProcMon from Microsoft Sysinternals
   • Run as Administrator (this gives us access to system-level activities)
   • Set filter: Process Name is explorer.exe
2. Watch the Magic:
   • Double-click on any .txt file (this will launch notepad)
   • Observe the burst of activity in ProcMon

What's Happening Behind the Scenes: The shell performs a sophisticated lookup process in the Windows Registry. When you double-click a .txt file, explorer.exe doesn't inherently know that text files should open with Notepad. Instead, it follows a chain of registry entries to determine the appropriate action.

First, Windows looks up the file extension in the registry:

HKEY_CLASSES_ROOT\.txt → txtfile

This tells the system that files with the .txt extension are associated with a file type called "txtfile". But this is just the first step. The system then looks up what should happen when someone wants to "open" a txtfile:

HKEY_CLASSES_ROOT\txtfile\shell\open\command → %SystemRoot%\system32\NOTEPAD.EXE %1

This registry entry contains the actual command that should be executed, with %1 being a placeholder for the filename you double-clicked.

PowerShell Investigation

You can explore this process yourself using PowerShell commands that query the same registry information the shell uses:

This investigation reveals how flexible and configurable Windows file handling is. Every file extension can be associated with different applications, and users can change these associations to customize their experience. The shell handles all this complexity transparently, making the simple act of double-clicking a file work exactly as users expect.

Stage 2: CreateProcess API - The Gateway to Process Creation

Once the shell has determined that Notepad should be launched to handle your text file, it needs to actually create the new process. This is where the CreateProcess API comes into play. The CreateProcess function serves as the primary gateway between user-mode applications and the kernel's process management subsystem, but it's much more sophisticated than it might initially appear.

When the shell calls this function, it's not just saying "run this program" - it's providing a comprehensive blueprint that includes security settings, memory requirements, user interface preferences, and dozens of other configuration options that will determine exactly how the new process behaves.

Understanding the CreateProcess Parameters

Let's examine what happens when we create a process programmatically. The following code demonstrates the same process creation that occurs when you double-click a file, but with full visibility into the parameters being used:

When this code executes, CreateProcess doesn't immediately create a process. This might seem counterintuitive, but the function actually performs extensive validation and preprocessing first. The function carefully examines each parameter to ensure they're valid and consistent with system security policies. For instance, it verifies that the specified executable file exists and that the calling process has permission to execute it.

The STARTUPINFO structure contains detailed specifications about how the new process's main window should appear, what its standard input and output handles should be, and various other user interface and I/O settings. The PROCESS_INFORMATION structure will be filled in by CreateProcess to provide information about the newly created process, including its unique process ID and the handle to its primary thread.

The function also examines the creation flags to ensure they're compatible with each other and with the system's current security policy. Some combinations of flags are mutually exclusive, while others require special privileges that the calling process might not possess. This validation phase typically takes only a few hundred microseconds, but it's crucial for preventing malicious or buggy applications from compromising system stability.

Stage 3: Transition to Kernel Mode - The System Call Boundary

After completing its user-mode preparations, CreateProcess must transition from user mode to kernel mode to perform the actual process creation. This transition represents one of the most fundamental concepts in operating system design: the separation between user space (where applications run) and kernel space (where the operating system itself operates).

In user mode, applications have limited access to system resources and cannot directly manipulate critical system data structures. This restriction is enforced by the processor's hardware memory protection mechanisms, which prevent user-mode code from accessing kernel memory or executing privileged instructions. When an application needs the operating system to perform a privileged operation like creating a process, it must request this service through a system call.

The transition occurs through a carefully controlled mechanism that switches the processor from user mode (Ring 3) to kernel mode (Ring 0). This switch is not just a simple function call - it's a fundamental change in the processor's execution context that enables access to privileged system resources while maintaining security and stability.

Observing the Transition with WinDbg

To see this in action, we can use WinDbg to debug a simple program like Notepad:

Open WinDbg (x64 version on 64-bit Windows): 
Load `notepad.exe` → Debugger immediately breaks at initial breakpoint.  

This happens before Notepad’s `WinMain` executes → we’re paused during process initialization.

Check call stack (`k`): 
    Shows functions starting with `Ldr...` (from ntdll!Ldr`).  
    These belong to the Windows Loader, responsible for loading the EXE/DLLs.  
    Confirms: we’re still inside the loader, not yet in Notepad code → window not visible.

bp ntdll!NtCreateFile

mov r10, rcx
mov eax, 55h
test byte ptr [SharedUserData+0x308],1
jne ntdll!NtCreateFile+0x15
syscall
ret
int 2Eh
ret

*mov r10, rcx: Move first arg (handle) to r10 → syscall convention requires it.

*mov eax, 55h: Load system service number.

        Each syscall is identified by a number, not name.
        On your OS: 0x55 = NtCreateFile.

*test [SharedUserData+0x308],1: Reads flag from a fixed memory page (0x7FFE0000).
       If set → system uses int 2Eh instead of syscall.
       Normally syscall is used (faster).
       Flag is set if Credential Guard / VBS is enabled → hypervisor prefers int.

*syscall: CPU transitions from user mode (ring3) → kernel mode (ring0).
       At this moment → execution enters the System Service Dispatcher inside           ntoskrnl.exe.
      ret: Execution returns when kernel finishes → result in RAX.

The System Call Mechanism

CreateProcess internally calls NtCreateUserProcess, which is the native system call responsible for process creation. This native API represents the true interface to the Windows kernel, while the Win32 CreateProcess function is essentially a wrapper that provides a more convenient interface for application developers. Understanding this distinction is crucial because it reveals the layered architecture of Windows, where documented APIs build upon undocumented native system services.

When the system call executes, several important things happen simultaneously. The processor saves the current execution context, switches to kernel mode, and transfers control to a system call dispatcher within ntoskrnl.exe, the Windows kernel. This dispatcher validates the system call number and parameters, then routes the request to the appropriate kernel subsystem for processing.

Kernel-Mode Processing Begins:Once execution has transitioned to kernel mode, the Windows kernel takes complete control of the process creation sequence. The system call handler performs additional validation of all parameters passed from user mode, ensuring they don't contain malicious values or represent security risks. This validation is crucial because kernel-mode code operates with elevated privileges and must exercise extreme caution when handling data from user mode.

Windows kernel is a very deep and interesting story; we will cover this in another blog in the future.

Stage 4: Creating the Process Object - The Foundation of Execution

Creating the Process Object

When Windows creates a process, it doesn’t just run a program—it builds a complete environment for it. At the heart of this is the EPROCESS kernel object, which represents the process inside the operating system. This structure stores everything Windows needs to manage the process: process IDs, parent relationships, security tokens, memory information, handles, and pointers to related objects. Each new process gets a unique ID, and its security token is usually inherited from the parent process, controlling what the process can access.

Alongside this, the kernel sets up the process’s virtual address space, which can be massive on 64-bit systems—up to 128 TB. Initially, this space is mostly empty except for structures needed to initialize the process.

Loading the Executable Image: Bringing Code into Memory

Windows parses the executable’s Portable Executable (PE) headers to understand the layout of code, data, and read-only sections. The loader maps the file into memory using virtual memory techniques, loading sections like .text (executable code), .data (read/write globals), and .rdata (read-only data) with proper memory protections.

Address Space Layout Randomization (ASLR) ensures that these sections are placed at unpredictable addresses, making it harder for attackers to exploit predictable memory locations.

A process without threads cannot execute code, so Windows must create at least one thread to serve as the primary execution context for the new process. Thread creation involves allocating and initializing several data structures, including the kernel's ETHREAD object and the user-mode Thread Environment Block (TEB).

The ETHREAD structure contains all the information the kernel needs to manage the thread, including its current state, priority, processor affinity, and execution context. The execution context includes the values of all processor registers, which define what instruction the thread will execute and what data it can access when it runs. Initially, these registers are set up so that the thread will begin execution at the process's entry point.

The Final Leap: Jumping to the Entry Point

After all DLLs have been loaded and initialized, Windows finally transfers control to the application's code. This happens by setting up the primary thread's execution context to begin at the executable's entry point, which is specified in the PE headers. For most applications, this entry point is not the main() function that developers write, but rather a runtime startup function provided by the compiler.

The runtime startup function performs additional initialization specific to the programming language and runtime environment. For C and C++ applications, this includes initializing the C runtime library, setting up global constructors, parsing command line arguments, and preparing the environment variables. Only after this initialization is complete does the runtime call the developer's main() function.

At this moment, the process creation journey is complete. What began as a simple CreateProcess API call has resulted in a fully functional process with its own virtual address space, security context, and execution thread. The process is now ready to execute application code and respond to user input.

The Timeline: Understanding Performance Implications

The entire process creation sequence typically takes anywhere from 10 to 100 milliseconds on modern hardware, depending on the complexity of the application and the number of DLLs it requires. Simple console applications with few dependencies can be created very quickly, while complex applications with many DLL dependencies may take significantly longer.

Understanding this timeline helps explain why some applications start faster than others. Applications that minimize their DLL dependencies, use delay-loading for optional components, or perform lazy initialization can reduce their startup time significantly. Conversely, applications that load many large DLLs or perform extensive initialization in their DllMain functions will start more slowly.

Conclusion:

Creating a process may seem simple, but it’s actually a carefully coordinated sequence of kernel and user-mode operations. From the API call to kernel object creation, memory setup, DLL loading, and thread initialization, each step ensures the process runs securely and efficiently.

Understanding this process is valuable for developers, security researchers, and system administrators. It explains performance behavior, security mechanisms, and even malware analysis. The next time you launch a program, you’re witnessing decades of operating system engineering executed in just milliseconds—a true testament to both complexity and elegance.