Patching strategy¶

Most servers that are broken into aren't attacked with anything clever. They run a version of some software with a vulnerability that was published, fixed and packaged weeks or months earlier, and nobody installed the fix. Once a fix is public, the change it makes shows attackers exactly where the weakness was, so the time between a fix and working exploits can be days. Patching promptly closes that window; it's the single most effective routine task in host security.

Patching also changes running systems, and every change can break something. A patching strategy is the balance between those two risks: how quickly updates go in, how they're tested, how you know they took effect, and how you go back if one goes wrong.

Where security fixes come from¶

Upstream projects fix vulnerabilities; distributions turn the fixes into packages. Vulnerabilities are tracked by CVE identifiers (CVE-2026-12345), which let you match one issue across upstream releases and every distribution's packages.

How a fix reaches you depends on the distribution's release model.

Stable releases (Debian, Ubuntu LTS, RHEL and its rebuilds) keep each package at the version it shipped with, and backport security fixes into it. A Debian 13 system keeps its OpenSSH major version for the life of the release; a security update changes only the package revision. That is why "the version number looks old" isn't evidence of a vulnerability on these systems: the fix is in the package changelog, not in the upstream version. These distributions publish security advisories for each fix:

Distribution	Advisories	Security updates come from
Debian	Debian Security Advisories (DSA) and Debian LTS Advisories (DLA), with the security tracker at `security-tracker.debian.org`	The `debian-security` archive (`trixie-security` for Debian 13)
Ubuntu	Ubuntu Security Notices (USN)	The `-security` pocket (`resolute-security` for 26.04)
RHEL, AlmaLinux, Rocky Linux	Red Hat Security Advisories (RHSA), rebuilt as ALSA and RLSA	The normal repositories; advisories are attached to updates as metadata, which `dnf` can filter by

Rolling releases (Arch, Gentoo) don't backport. A security fix arrives as the next upstream version of the package, together with everything else that changed upstream. Arch publishes advisories on its security tracker, and Gentoo publishes Gentoo Linux Security Advisories (GLSA), which glsa-check can test a system against. On these systems, staying secure and staying current are the same thing, so the strategy is about updating often and carefully rather than about selecting security fixes.

An installed update isn't always an applied one¶

Installing a package replaces files on disk. Programs that are already running keep using the old code they loaded into memory until they're restarted. Depending on what was updated, applying the fix needs one of these:

What was updated	What applies it
A program that isn't running (a command-line tool)	Nothing: the next run uses the new version
A daemon, such as `sshd` or `nginx`	Restarting that service; many packages do this themselves on upgrade on Debian, Ubuntu and the RHEL family, while Arch and Gentoo leave it to you
A shared library, such as OpenSSL or glibc	Restarting every process that has the old library loaded; for glibc or systemd that's effectively a reboot
The kernel, or CPU microcode	A reboot into the new kernel, or a live patch

Processes still using deleted, replaced files are detectable: they keep the old file open, and /proc/<PID>/maps shows it as (deleted). Tools built on that idea tell you what needs restarting: needrestart on Debian and Ubuntu (which Ubuntu installs by default and runs after every apt operation), checkrestart from debian-goodies on Debian, and dnf needs-restarting on the RHEL family. On Ubuntu, packages that need a reboot also create the file /run/reboot-required.

A kernel update is the clearest case: until you reboot, uname -r shows the old kernel, and the system is still exposed to whatever the update fixed. Scheduling those reboots is part of the strategy, not an afterthought. A fleet of servers that never reboots is running old kernels, whatever the package manager says.

Live patching¶

Live patching applies fixes to a running kernel without a reboot. A live patch is a kernel module that redirects calls from the vulnerable functions to corrected ones, using the kernel's livepatch infrastructure. Commercial services provide these patches for supported kernels: for example Canonical Livepatch for Ubuntu, and kpatch for RHEL.

Live patching is for buying time, not for replacing reboots. Only some fixes can be expressed as a live patch (changes to data structures generally can't), patches are produced only for selected vulnerabilities and kernels, and live patches stack up until the next real kernel update. It lets you fix a serious kernel flaw today and reboot at the next maintenance window, instead of choosing between an emergency reboot and staying exposed.

Automatic or manual¶

Applying security updates automatically is a sound default for most stable-release servers: their security updates are small, backported, tested for that release, and far more likely to prevent a compromise than to break anything. Debian and Ubuntu provide unattended-upgrades for this (Ubuntu enables it by default); the RHEL family provides dnf-automatic. Both can be limited to security updates and told whether to reboot.

Rolling releases are different. An Arch update can bring a new major version of anything, and occasionally needs manual steps announced on the project's news page; a Gentoo update compiles packages, can require configuration-file merges, and depends on USE flags and profile changes. Updating these systems unattended risks leaving them half-updated or unable to boot, with nobody watching. The safer pattern is frequent, attended updates, with automation limited to downloading packages or checking advisories and telling a human.

Whichever you choose, someone has to notice failures. An automatic update job that has been failing silently for three months is worse than none, because everyone assumes the systems are patched.

Staging and rollback¶

For anything that matters, updates should reach a less important system first. A typical rhythm on stable releases is to apply updates to test or staging machines as soon as they appear, and to production a few days later, unless an advisory is urgent enough to skip the wait. In a fleet, the same idea applies as rings: a few canary machines, then a fraction of the fleet, then the rest.

Rollback means having a way back before you need it:

Package downgrade. dnf history undo reverts a whole transaction on the RHEL family, as long as the old packages are still available. apt can install an older version explicitly if the archive still has it; Arch keeps old packages in its cache (/var/cache/pacman/pkg/), and Gentoo can rebuild an older version if it's still in the tree or a binary package was kept.
Kernel fallback. Distributions keep the previous kernel or kernels installed, so a kernel that won't boot can be bypassed from the bootloader menu.
Snapshots. Filesystem or volume snapshots (Btrfs, LVM, or the hypervisor's) taken before an update give a complete rollback, including configuration.
Rebuild. Systems built from configuration management can be rebuilt on known-good versions, which makes a bad update a nuisance rather than an outage.