CalyxOS HSM Signing: A Blueprint for Open-Source Security

The Problem: Software Keys Are a Ticking Time Bomb

When you install an Android OS update, your device verifies the cryptographic signature to confirm the software came from who it claims. If an attacker obtains the signing key, they can push malicious updates that your phone will trust without question. For a privacy-focused operating system like CalyxOS — used by journalists, activists, and security-conscious individuals — a key compromise wouldn't just be embarrassing. It could be life-threatening.

Historically, CalyxOS (like most AOSP-based projects) stored signing keys as files on disk. This is the norm across the Android ecosystem, but it's a fundamentally fragile approach. A single breach, a compromised developer machine, a leaked backup — and the keys are gone forever. You can't "un-steal" a file. There's no revocation mechanism for Android verified boot keys. Once they're out, you're starting over with new keys and asking every user to factory reset.

In their FOSDEM 2026 talk, the CalyxOS team detailed how they redesigned their entire signing infrastructure around hardware security modules. The result is a system that's more secure, more auditable, and — crucially — reproducible by other open-source projects.

Why YubiHSM 2? The Pragmatic Choice

CalyxOS evaluated the full spectrum of HSM options: cloud-based solutions like AWS CloudHSM, enterprise appliances from Thales and Entrust, mid-range options like Nitrokey NetHSM, and smaller hardware tokens. They chose the YubiHSM 2 — a $300 USB device — as their starting point.

This is a fascinating decision. In our comprehensive HSM guide, we cover how enterprise HSMs from Thales and Entrust cost $30,000–$90,000. CalyxOS chose a device that costs 0.3% of that. Why?

• Immediate availability — no lengthy procurement cycles or vendor negotiations
• Open-source tooling — Yubico's SDK and tools are fully open-source, aligning with CalyxOS's mission
• Auditability — Trail of Bits could audit the entire ceremony and toolchain
• Migration path — they explicitly designed for future migration to a more robust solution
• Cost — a non-profit can actually afford multiple units for redundancy

The lesson here is profound: perfect shouldn't be the enemy of good. A $300 HSM properly deployed is infinitely more secure than a $90,000 appliance you can't afford sitting in a Thales catalog.

Key Wrapping: Solving the Storage Problem

The YubiHSM 2 has severely limited storage — it can't hold all the signing keys an Android build requires (unique keys per device model, per component). CalyxOS solved this with key wrapping:

• A wrap key lives permanently inside the HSM
• Signing keys are encrypted ("wrapped") by the wrap key and stored externally as files
• When a key is needed, it's imported into the HSM and decrypted inside the secure boundary
• The plaintext signing key never exists outside the HSM

Shamir's Secret Sharing: No Single Point of Trust

The wrap key is the crown jewel. If the HSM dies and the wrap key is lost, all signing keys are gone. CalyxOS needed a backup strategy that didn't recreate the original problem (a key file someone could steal).

Their solution: Shamir's Secret Sharing (SSS), a 1979 algorithm that splits a secret into n shards where any k shards can reconstruct it, but k-1 shards reveal nothing. CalyxOS uses a 5-of-3 scheme: five shards distributed to five trusted individuals, any three of whom can reconstruct the wrap key.

This is elegant operationally:

• No single person can compromise the signing infrastructure
• The project survives the loss of two shard holders (hit by a bus, goes rogue, loses their shard)
• Recovery requires an in-person ceremony — you can't be social-engineered over email

The Key Ceremony: Security Theater Done Right

One subtle challenge: the YubiHSM 2 doesn't natively support Shamir's Secret Sharing in firmware. The official yubihsm-setup tool briefly holds the complete wrap key in memory before splitting it. This means the ceremony environment itself becomes a security boundary.

CalyxOS designed formal key provisioning ceremonies with strict controls to ensure key material can't leak during generation or restoration. The entire ceremony package — scripts, procedures, threat model — was audited by Trail of Bits, one of the most respected security firms in the industry.

This is where CalyxOS raises the bar for open-source projects. Most projects treat key management as an afterthought. CalyxOS treated it as infrastructure worth investing in professional auditing. The audit report is publicly available.

What Other Projects Can Learn

1. HSMs Are No Longer Enterprise-Only

The YubiHSM 2 at ~$300 puts hardware key protection within reach of any project. If CalyxOS can do it, so can LineageOS, GrapheneOS, /e/OS, postmarketOS, and every custom ROM project.

2. Key Wrapping Is the Pattern for Scale

Any project that manages many keys (F-Droid signs thousands of apps) can use the same wrap-key pattern. You don't need an HSM with massive storage — you need one that can protect a single master key.

3. Formal Ceremonies Matter

The ceremony concept — air-gapped machines, multiple witnesses, documented procedures, external audit — is transferable to any project's key generation process. You don't need to hire Trail of Bits (though it helps). Just document your process and have peers review it.

4. Transparency Is a Feature

CalyxOS published their criteria, their process, and their audit results. This transparency is itself a security feature — it allows the community to verify claims and identify weaknesses that the team might have missed.

Looking Forward

CalyxOS explicitly noted they built in a migration path to a "better or final signing solution." This suggests they may eventually move to a network-attached HSM or more robust hardware as the project grows. The key wrapping architecture makes this transition straightforward — the wrapped keys don't change, only the device holding the wrap key does.

We'd love to see this approach become a standard across the Android custom ROM ecosystem. Imagine a shared HSM infrastructure that multiple privacy-focused ROM projects could use — a community signing service, if you will. The cryptographic patterns CalyxOS has established make this technically feasible.

For a deeper understanding of HSM technology, architecture, and the broader market landscape, read our Definitive Guide to Hardware Security Modules.