kGraft

kGraft is a feature of the Linux kernel that implements live patching of a running kernel, which allows kernel patches to be applied while the kernel is still running. By avoiding the need for rebooting the system with a new kernel that contains the desired patches, kGraft aims to maximize the system uptime and availability. At the same time, kGraft allows kernel-related security updates to be applied without deferring them to scheduled downtimes.^[1][3] Internally, kGraft allows entire functions in a running kernel to be replaced with their patched versions, doing that safely by selectively using original versions of functions to ensure per-process consistency while the live patching is performed.^[4]

kGraft is developed by SUSE, with its source code licensed under the terms of versions two and three of the GNU General Public License (GPL).^[3][5] In April 2014, kGraft was submitted for inclusion into the Linux kernel mainline,^[4] and the minimalistic foundations for live patching were merged into the Linux kernel mainline in kernel version 4.0, which was released on April 12, 2015.^[6]

Internally, kGraft consists of two parts – the core kernel module executes the live patching mechanism by altering kernel's inner workings, while userspace utilities prepare individual hot patch kernel modules from source diffs. Live kernel patching is performed at the function level, meaning that kGraft can replace entire functions in the running kernel with their patched versions, while relying on the mechanisms and infrastructure established by ftrace to "route around" old versions of functions. No changes to the kernel's internal data structures are possible; however, security patches, which are one of the natural candidates to be used with kGraft, rarely contain changes to the kernel's data structures.^[3][4][7]

While applying hot patches, kGraft does not require a running kernel to be stopped for patched versions of functions to be introduced into it. Instead of replacing functions atomically, kGraft provides consistent "world views" (or "universes") to userspace processes, kernel threads and interrupt handlers, which are monitored during their execution so the original versions of patched kernel functions can continue to be used. To accomplish that, kGraft maintains original versions of patched functions in a read-copy-update (RCU) fashion, and dynamically selects between the original and patched versions depending on which process, kernel thread or interrupt handler executes them. More specifically, original versions of functions continue to be used‍—‌at the time when a hot patch is applied‍—‌for processes currently executing within the kernel space, for kernel threads until they reach their completion points, and for currently executing interrupt handlers.^[3][4][8][9]

Due to its design, kGraft does not introduce additional latency while applying hot patches. As the downside, original versions of patched kernel functions may be required to be maintained for extended periods of time in case there are processes that remain for too long within the kernel space; for example, a process may wait for I/O on a network socket. Also, as both original and patched versions of functions are allowed to be executed in parallel, troubles may arise if they use kernel's internal data structures in different ways.^[4][9]

"World views" or "universes", maintained by kGraft in a read-copy-update fashion^{[3]: 14, 15, 17}

Without additional logic in place, not replacing patched functions atomically could lead to inconsistencies.

Each process is monitored so it executes a patched function consistently within a single system call.

After everything migrates to a new "universe", trampoline-style checks are no longer needed.

SUSE announced kGraft in January 2014^[10] and released it publicly in March 2014 under the terms of the GNU General Public License version 2 (GPLv2) for the kernel part, and under the terms of version 3 (GPLv3) for the userspace part.^[1][5] It was released shortly after Red Hat released its own live kernel patching implementation called kpatch.^[11] kGraft aims to become merged into the Linux kernel mainline, and it was submitted for the inclusion in April 2014.^[4][9]

kGraft was made available for SUSE Linux Enterprise Server 12 on November 18, 2014, as an additional feature called SUSE Linux Enterprise Live Patching.^[12][13]

Minimalistic foundations for live kernel patching were merged into the Linux kernel mainline in kernel version 4.0, which was released on April 12, 2015. Those foundations, based primarily on the kernel's ftrace functionality, form a common core capable of supporting hot patching by both kGraft and kpatch, by providing an application programming interface (API) for kernel modules that contain hot patches and an application binary interface (ABI) for the userspace management utilities. However, the common core included into Linux kernel 4.0 supports only the x86 architecture and does not provide any mechanisms for ensuring function-level consistency while the hot patches are applied.^[6][14][15]

Since April 2015, there is ongoing work on porting kGraft to the common live patching core provided by the Linux kernel mainline.^[15] However, implementation of the required function-level consistency mechanisms has been delayed because the call stacks provided by the Linux kernel may be unreliable in situations that involve assembly code without proper stack frames; as a result, the porting work remains in progress as of September 2015^[update]. In an attempt to improve the reliability of kernel's call stacks, a specialized sanity-check stacktool userspace utility has also been developed.^[16][17]

• Dynamic software updating, a field of research focusing on upgrading programs while they are running
• kexec, a method for loading a whole new Linux kernel from a running system
• Ksplice and KernelCare, other Linux kernel live patching technologies developed by Ksplice, Inc. (later acquired by Oracle) and CloudLinux, respectively

• Official website