What broke

On the morning of November 4, unattended-upgrades installed kernel 7.0.0-42 on a home server that had been running 7.0.0-38. Nothing happened until the next reboot, which was a hard power reset a day later. After it, every PostgreSQL container started and then sat at 100% CPU. The APIs that depend on them went into crash loops and one public endpoint began returning 502.

Ruling out the usual suspects

Restarting the containers changed nothing, and neither did pulling the current postgres:15-alpine image. The CPU was burning in the gosu step of the entrypoint, the small tool that drops privileges before PostgreSQL starts, so it was not even getting as far as the database.

Reproducing it without the application

The useful step was taking PostgreSQL out of the picture. Running docker run --entrypoint gosu postgres:15-alpine postgres id finished in 0.37 seconds on the old kernel and had not finished after 15 seconds on the new one. It is a one-line test that needs no database, no volume and no network, and it points at the host rather than at anything inside the container.

Getting back without making it worse

I used grub-reboot to boot the old kernel exactly once, which leaves the default untouched, and confirmed every container came up healthy. Only then did I make it permanent. The GRUB default now names the 7.0.0-38 entry, and apt-mark hold covers linux-generic, linux-image-generic and linux-headers-generic, so unattended-upgrades cannot move the kernel forward again. I kept the 7.0.0-42 package installed, because a hold only blocks newer versions and I want the broken one available for testing.

What I still do not know

I never confirmed the root cause. It could be the kernel, the container runtime or an incompatibility between them, and all I know is that changing the kernel changes the result. That is enough to operate safely, but not enough to close the incident. The pin stays until a newer kernel passes the same 15 second test, and that test is what I will use to decide when to lift it.

  • Boot a rollback kernel once with grub-reboot before changing any default.
  • Reproduce with the smallest command that shows the fault, outside the application.
  • Hold the kernel packages, keep the bad version installed, and write down what is still unknown.