Ceph Squid End of Life on Proxmox: The Upgrade Order

Ceph's release page gives Squid an estimated end of life of 31 October 2026, and on a Proxmox cluster that date is the easy part of the Ceph upgrade. Proxmox builds Tentacle, the release that replaces it, only for Proxmox VE 9. The August security releases also bring a new cephx key type that every daemon and client is meant to move to, so a cluster that was clean before the upgrade can report HEALTH_ERR straight after it.

CloudSpinx builds and runs Proxmox clusters with Ceph storage for organizations in Kenya. For this post we counted what Proxmox actually ships: the Ceph, pve-manager and kernel builds in its public no-subscription and test repositories, read from the package indexes and the pve-manager packages themselves. That count settles the upgrade order for a cluster still on Proxmox VE 8. We did not run the upgrade for this post, so everything about how the software behaves is taken from Proxmox's and Ceph's documentation, and from the helper's source code where the post says so.

What Ceph Squid end of life changes

Nothing stops on 1 November. The cluster keeps serving disks. What ends is upstream maintenance: the Ceph releases page files an end-of-life release under archived releases, which "do not receive bug fixes or backports", and Reef went there in March with 18.2.8. Squid's last listed build is 19.2.7.

The reason not to wait is the August round of security fixes. Squid 19.2.6 and Tentacle 20.2.4 shipped together with four CVE fixes, one of them CVE-2025-30156, which Ceph describes as "an authentication bypass in CephX caused by misuse of AES-CBC". Tentacle is no long stay either. Its own estimated end of life is 1 June 2027, seven months after Squid's, so budget the next hop the way you would a Kubernetes minor: as a recurring line, not an event.

Which Ceph builds Proxmox ships for PVE 8 and PVE 9

Proxmox builds each Ceph release for a Debian suite, and each Proxmox VE major sits on one suite: bookworm for 8, trixie for 9. Reef and Quincy have no trixie build. Tentacle has no bookworm build. Squid is the only release built for both, so it is the bridge every Ceph cluster on Proxmox VE 8 has to cross.

The Proxmox VE 8 side is thinner than it looks. Its no-subscription repository stops at Squid 19.2.5, one point release short of the CVE fix. 19.2.6 sits only in the test repository, next to pve-manager 8.4.22, which carries Proxmox's key migration helper; 8.4.21, the newest pve-manager in no-subscription, does not. The newest kernel series for Proxmox VE 8 is 6.14, and Proxmox says kernel clients need a running 7.0 or newer for the new keys. Proxmox VE 9.2 defaults to 7.0. Proxmox's own table puts the end of Proxmox VE 8 in August 2026, which closes the question. We read only the public repositories, because the enterprise ones need a subscription key, so none of this is a claim about them.

The order for a Ceph upgrade on Proxmox

From Reef on Proxmox VE 8 it is three upgrades and two rounds of key rotation. A Squid cluster still on Proxmox VE 8 starts at step 2, one already on 9.2 at step 3, and a Quincy cluster goes to Reef first.

1. Reef to Squid, still on Proxmox VE 8. The 8 to 9 guide makes it a precondition: "a hyper-converged Ceph cluster installed directly in Proxmox VE must run Ceph 19.2 Squid".
2. Proxmox VE 8 to 9.2, with the Ceph repository switched to Squid for trixie, which pulls 19.2.6 or later, and every node rebooted into the 7.0 kernel. The health errors can appear once the first upgraded node with a monitor reboots, and the guide says to carry on with the upgrade if nothing else is wrong.
3. The cluster's own keys, with Proxmox's helper. This clears the two health errors.
4. Squid to Tentacle: the Ceph repository switched to Tentacle, noout set, packages upgraded on every node, monitors restarted one node at a time until ceph mon dump reports min_mon_release 20 (tentacle), then managers, then OSDs one node at a time, then any CephFS MDS daemons, then ceph osd require-osd-release tentacle and noout unset.
5. Storage and admin keys: staged, every client refreshed, confirmed, and the old cipher switched off.

Clear the errors in step 3 before step 4, because the Tentacle guide starts from a cluster that is "healthy and working". Its stated minimum is pve-manager 9.1.4, while Proxmox's September announcement asks for 9.2 and its key migration docs ask for 9.2.19, so 9.2.19 is the floor that matters.

Take Tentacle at 20.2.4-pve4 or later. A plain apt full-upgrade installs 20.2.4-pve5, the newest in Proxmox's public Tentacle repositories, but older builds down to 20.2.0 are still listed there, 20.2.4-pve1 to pve3 among them, and step 5 needs at least 20.2.4-pve3 on every monitor, pve4 to tell refreshed clients from stale ones. Ceph's release notes also warn that a move from Squid 19.2.6 to an older Tentacle "may fail", because "Tentacle daemons older than v20.2.4 fail to start."

If your OSDs were created under Octopus or earlier, read the Squid to Tentacle guide's known issue first. It reports META usage growing on those OSDs after the move, with a manual RocksDB compaction, one OSD at a time and off-peak, as a temporary workaround, or recreating each affected OSD, one at a time.

A Reef cluster on Proxmox VE 8 has two upgrades to finish, Ceph to Squid and Proxmox VE to 9.2, before the move to Tentacle even starts. If you would rather know which steps your nodes need before the first one, paste pveversion -v from one node into the notes of the scoping form.

Why Ceph shows HEALTH_ERR after 19.2.6

Squid 19.2.6 and Tentacle 20.2.4 added six health checks around the old aes cipher, covering keys, tickets and whether monitors still accept it, and two of them carry error severity: AUTH_INSECURE_SERVICE_KEY_TYPE and AUTH_INSECURE_SERVICE_TICKETS. Proxmox's documentation is plain that this "does not mean that storage access or a Ceph service has failed". The fix is a new key type, aes256k, which Ceph calls the first new key type CephX has ever had.

The cluster's own keys go first. Run the helper on one node, read the dry run, then apply the same selection:

/usr/share/pve-manager/migrations/pve-cephx-rotate-service-keys --rotate-cluster-keys
/usr/share/pve-manager/migrations/pve-cephx-rotate-service-keys --rotate-cluster-keys --apply

It rotates the manager, metadata server, OSD, monitor, bootstrap, crash and encrypted OSD lockbox keys, restarting services one at a time when needed, monitors included. Storage keys and client.admin stay as they are, existing tickets stay valid and the old cipher stays enabled, so no guest's credentials have changed yet. The warnings left after this are about client keys and the allowed ciphers, plus one about rotating service keys that clears by itself within a few hours. Proxmox allows muting them while an old client waits for an upgrade, and the same page says what a mute buys: "Muting does not resolve the condition."

Off Proxmox, Ceph's announcement says cephadm automates this rotation "except for client keys", which leaves the same last step as below.

The clients that keep the old key

Step 5 is the one that breaks late. A staged key leaves both keys valid and every client keeps working, so nothing fails yet. The client-key warning stays until confirmation, and the cipher warnings until the old cipher is switched off. After confirmation the old key stops authenticating, and in Proxmox's words, "Existing IO can appear to work until a reconnect and then fail."

What holds an old key, per Proxmox's cephx migration section, is a longer list than the VMs. Rebooting a VM from inside the guest does not refresh it, so live-migrate it or stop and start it: "A guest reboot is not enough." A container on RBD always uses the kernel client, and a kernel client needs a running 7.0 kernel for the new key, one more reason the Proxmox VE 8 hop comes first. Backups, restores, disk imports and clones "can retain the key they started with", so confirmation goes outside the window your Proxmox Backup Server jobs run in. Then there is everything outside Proxmox: a Kubernetes cluster mounting RBD volumes, or a server with an image mapped, which needs the staged key copied from /etc/pve/priv/ceph/<storage>.keyring and a restart.

The helper's source draws that line itself. Before it accepts --confirm-all-clients-refreshed it refuses while "recorded client(s) ... are still connected", and a comment in the code names what is left: "consumers no run can see: disconnected ones, and copies of the key on hosts Proxmox VE does not manage." On our reading, a client connected on the old key holds the confirmation up, and one that is switched off at that moment does not, which makes it the one that cannot authenticate when it comes back. Keep /etc/pve/priv/cephx-key-migration.json until the migration is complete and every client has been refreshed, and protect it, because it holds the old keys.

A three-node cluster running only VMs, with nothing outside Proxmox touching Ceph, does not need us. Follow Proxmox's upgrade guides and its cephx section, let the helper print each next step, and do it yourself.

The expensive case is a Proxmox VE 8 cluster on Reef with containers on Ceph and a backup server or a Kubernetes cluster reading from it, because that is three upgrades and a confirmation that waits on the slowest client. Send us pveversion -v and ceph versions from one node, ceph health detail, and every machine outside the cluster that maps or mounts Ceph, through the Proxmox scoping form or on WhatsApp at +254 713 403 044. We come back with the hops your cluster needs in order, the clients that will hold the confirmation up, and one figure for doing the upgrade with you, node by node. There is no charge for that answer and no obligation after it.