6 hours ago
A customer building a FedRAMP-compliant deployment found that Ubuntu Pro 22.04 FIPS container images were silently failing on standard, mainline Linux kernels, the kind used by managed Kubernetes environments such as AWS EKS and Fargate. The fix required a patch that preserved FIPS compliance without triggering months of recertification. This post explains what happened, why it matters for anyone running FIPS workloads in containerized cloud environments, and what Canonical changed as a result.
The problem: a kernel flag that doesn’t exist in the cloud
The customer was in the middle of a large FedRAMP migration across multiple platforms. Their containerized workloads relied on Ubuntu Pro 22.04 FIPS images, and the containers were failing in ways that pointed to a fundamental incompatibility rather than a configuration mistake.
The root cause was a hard dependency in libgcrypt20-fips on GRND_RESEED_ONLY, an Ubuntu-specific kernel flag introduced as part of Canonical’s FIPS implementation. The flag does not exist in standard mainline Linux kernels. Managed Kubernetes offerings from major cloud providers run mainline kernels, not Ubuntu FIPS kernels. When libgcrypt20-fips attempted to call getrandom() using GRND_RESEED_ONLY, the syscall returned an error on those kernels, and the container failed.
This wasn’t a misconfiguration. A fully FIPS-compliant host, a correctly built container, and an image identical to one working on a Jammy FIPS kernel would still fail, because the package was calling a syscall the underlying kernel didn’t support.
The customer arrived at this diagnosis themselves, supported by public bug reports and their own internal testing. They came to Canonical with specific package names, specific error conditions, and specific references, including bug 2055825.
Why this matters beyond one customer
This is a common architecture for any organisation under FedRAMP, FISMA, or DoD compliance requirements running Ubuntu Pro FIPS images on managed Kubernetes environments with mainline kernels. The assumption that a FIPS-certified image behaves consistently regardless of the underlying kernel is reasonable; before this fix, it wasn’t reliably true.
Escalation and investigation
Canonical Support escalated to the Canonical FIPS team, and Matthew, a Canonical support engineer, led the technical investigation.
The core challenge was finding a fix that did two things simultaneously:
FIPS certification is not a property you can alter lightly. Changes to cryptographic packages can invalidate certification, requiring months of revalidation through NIST’s (National Institute of Standards and Technology) CMVP (Cryptographic Module Validation Program) process. Any fix that broke certification would simply trade one blocker for another for customers with active compliance obligations.
Matthew’s patch resolved both constraints. It modified the packages’ behaviour when GRND_RESEED_ONLY was unavailable, allowing them to fall back correctly without compromising the cryptographic guarantees that underpin FIPS compliance.
Testing and release
Canonical provided the patched packages to the customer via a private PPA for validation. The customer tested the containers against their target environments and confirmed the fix worked.
Canonical then released the packages to the jammy-fips-updates and noble-fips-updates repositories. Customers running Ubuntu Pro 22.04 or 24.04 FIPS container images on standard cloud kernels should ensure their systems are pulling from jammy-fips-updates or noble-fips-updates respectively.
What this means for FIPS container deployments
No single layer told the full story here: the container looked correct, the host was FIPS-compliant, and the failure only showed up in the gap between a package assumption and a kernel that didn’t support it. That’s the part worth remembering: FIPS compliance in cloud-native environments depends on the interaction between certified packages and the underlying kernel, not just on using certified images.
Canonical Support provides access to expert engineers across the full open source stack, including the FIPS team. If you are working through a FIPS or FedRAMP deployment and running into compatibility issues, contact us or visit our Support page.
How a missing kernel flag broke FIPS-certified containers in managed Kubernetes
A customer building a FedRAMP-compliant deployment found that Ubuntu Pro 22.04 FIPS container images were silently failing on standard, mainline Linux kernels, the kind used by managed Kubernetes environments such as AWS EKS and Fargate. The fix required a patch that preserved FIPS compliance without triggering months of recertification. This post explains what happened, why it matters for anyone running FIPS workloads in containerized cloud environments, and what Canonical changed as a result.
The problem: a kernel flag that doesn’t exist in the cloud
The customer was in the middle of a large FedRAMP migration across multiple platforms. Their containerized workloads relied on Ubuntu Pro 22.04 FIPS images, and the containers were failing in ways that pointed to a fundamental incompatibility rather than a configuration mistake.
The root cause was a hard dependency in libgcrypt20-fips on GRND_RESEED_ONLY, an Ubuntu-specific kernel flag introduced as part of Canonical’s FIPS implementation. The flag does not exist in standard mainline Linux kernels. Managed Kubernetes offerings from major cloud providers run mainline kernels, not Ubuntu FIPS kernels. When libgcrypt20-fips attempted to call getrandom() using GRND_RESEED_ONLY, the syscall returned an error on those kernels, and the container failed.
This wasn’t a misconfiguration. A fully FIPS-compliant host, a correctly built container, and an image identical to one working on a Jammy FIPS kernel would still fail, because the package was calling a syscall the underlying kernel didn’t support.
The customer arrived at this diagnosis themselves, supported by public bug reports and their own internal testing. They came to Canonical with specific package names, specific error conditions, and specific references, including bug 2055825.
Why this matters beyond one customer
This is a common architecture for any organisation under FedRAMP, FISMA, or DoD compliance requirements running Ubuntu Pro FIPS images on managed Kubernetes environments with mainline kernels. The assumption that a FIPS-certified image behaves consistently regardless of the underlying kernel is reasonable; before this fix, it wasn’t reliably true.
Escalation and investigation
Canonical Support escalated to the Canonical FIPS team, and Matthew, a Canonical support engineer, led the technical investigation.
The core challenge was finding a fix that did two things simultaneously:
FIPS certification is not a property you can alter lightly. Changes to cryptographic packages can invalidate certification, requiring months of revalidation through NIST’s (National Institute of Standards and Technology) CMVP (Cryptographic Module Validation Program) process. Any fix that broke certification would simply trade one blocker for another for customers with active compliance obligations.
Matthew’s patch resolved both constraints. It modified the packages’ behaviour when GRND_RESEED_ONLY was unavailable, allowing them to fall back correctly without compromising the cryptographic guarantees that underpin FIPS compliance.
Testing and release
Canonical provided the patched packages to the customer via a private PPA for validation. The customer tested the containers against their target environments and confirmed the fix worked.
Canonical then released the packages to the jammy-fips-updates and noble-fips-updates repositories. Customers running Ubuntu Pro 22.04 or 24.04 FIPS container images on standard cloud kernels should ensure their systems are pulling from jammy-fips-updates or noble-fips-updates respectively.
What this means for FIPS container deployments
No single layer told the full story here: the container looked correct, the host was FIPS-compliant, and the failure only showed up in the gap between a package assumption and a kernel that didn’t support it. That’s the part worth remembering: FIPS compliance in cloud-native environments depends on the interaction between certified packages and the underlying kernel, not just on using certified images.
Canonical Support provides access to expert engineers across the full open source stack, including the FIPS team. If you are working through a FIPS or FedRAMP deployment and running into compatibility issues, contact us or visit our Support page.
The problem: a kernel flag that doesn’t exist in the cloud
The customer was in the middle of a large FedRAMP migration across multiple platforms. Their containerized workloads relied on Ubuntu Pro 22.04 FIPS images, and the containers were failing in ways that pointed to a fundamental incompatibility rather than a configuration mistake.
The root cause was a hard dependency in libgcrypt20-fips on GRND_RESEED_ONLY, an Ubuntu-specific kernel flag introduced as part of Canonical’s FIPS implementation. The flag does not exist in standard mainline Linux kernels. Managed Kubernetes offerings from major cloud providers run mainline kernels, not Ubuntu FIPS kernels. When libgcrypt20-fips attempted to call getrandom() using GRND_RESEED_ONLY, the syscall returned an error on those kernels, and the container failed.
This wasn’t a misconfiguration. A fully FIPS-compliant host, a correctly built container, and an image identical to one working on a Jammy FIPS kernel would still fail, because the package was calling a syscall the underlying kernel didn’t support.
The customer arrived at this diagnosis themselves, supported by public bug reports and their own internal testing. They came to Canonical with specific package names, specific error conditions, and specific references, including bug 2055825.
Why this matters beyond one customer
This is a common architecture for any organisation under FedRAMP, FISMA, or DoD compliance requirements running Ubuntu Pro FIPS images on managed Kubernetes environments with mainline kernels. The assumption that a FIPS-certified image behaves consistently regardless of the underlying kernel is reasonable; before this fix, it wasn’t reliably true.
Escalation and investigation
Canonical Support escalated to the Canonical FIPS team, and Matthew, a Canonical support engineer, led the technical investigation.
The core challenge was finding a fix that did two things simultaneously:
- Made libgcrypt20-fips, gnutls28, and ubuntu-fips work correctly on standard kernels without the GRND_RESEED_ONLY flag
- Preserved FIPS compliance without requiring a full recertification cycle
FIPS certification is not a property you can alter lightly. Changes to cryptographic packages can invalidate certification, requiring months of revalidation through NIST’s (National Institute of Standards and Technology) CMVP (Cryptographic Module Validation Program) process. Any fix that broke certification would simply trade one blocker for another for customers with active compliance obligations.
Matthew’s patch resolved both constraints. It modified the packages’ behaviour when GRND_RESEED_ONLY was unavailable, allowing them to fall back correctly without compromising the cryptographic guarantees that underpin FIPS compliance.
Testing and release
Canonical provided the patched packages to the customer via a private PPA for validation. The customer tested the containers against their target environments and confirmed the fix worked.
Canonical then released the packages to the jammy-fips-updates and noble-fips-updates repositories. Customers running Ubuntu Pro 22.04 or 24.04 FIPS container images on standard cloud kernels should ensure their systems are pulling from jammy-fips-updates or noble-fips-updates respectively.
What this means for FIPS container deployments
No single layer told the full story here: the container looked correct, the host was FIPS-compliant, and the failure only showed up in the gap between a package assumption and a kernel that didn’t support it. That’s the part worth remembering: FIPS compliance in cloud-native environments depends on the interaction between certified packages and the underlying kernel, not just on using certified images.
Canonical Support provides access to expert engineers across the full open source stack, including the FIPS team. If you are working through a FIPS or FedRAMP deployment and running into compatibility issues, contact us or visit our Support page.
How a missing kernel flag broke FIPS-certified containers in managed Kubernetes
A customer building a FedRAMP-compliant deployment found that Ubuntu Pro 22.04 FIPS container images were silently failing on standard, mainline Linux kernels, the kind used by managed Kubernetes environments such as AWS EKS and Fargate. The fix required a patch that preserved FIPS compliance without triggering months of recertification. This post explains what happened, why it matters for anyone running FIPS workloads in containerized cloud environments, and what Canonical changed as a result.
The problem: a kernel flag that doesn’t exist in the cloud
The customer was in the middle of a large FedRAMP migration across multiple platforms. Their containerized workloads relied on Ubuntu Pro 22.04 FIPS images, and the containers were failing in ways that pointed to a fundamental incompatibility rather than a configuration mistake.
The root cause was a hard dependency in libgcrypt20-fips on GRND_RESEED_ONLY, an Ubuntu-specific kernel flag introduced as part of Canonical’s FIPS implementation. The flag does not exist in standard mainline Linux kernels. Managed Kubernetes offerings from major cloud providers run mainline kernels, not Ubuntu FIPS kernels. When libgcrypt20-fips attempted to call getrandom() using GRND_RESEED_ONLY, the syscall returned an error on those kernels, and the container failed.
This wasn’t a misconfiguration. A fully FIPS-compliant host, a correctly built container, and an image identical to one working on a Jammy FIPS kernel would still fail, because the package was calling a syscall the underlying kernel didn’t support.
The customer arrived at this diagnosis themselves, supported by public bug reports and their own internal testing. They came to Canonical with specific package names, specific error conditions, and specific references, including bug 2055825.
Why this matters beyond one customer
This is a common architecture for any organisation under FedRAMP, FISMA, or DoD compliance requirements running Ubuntu Pro FIPS images on managed Kubernetes environments with mainline kernels. The assumption that a FIPS-certified image behaves consistently regardless of the underlying kernel is reasonable; before this fix, it wasn’t reliably true.
Escalation and investigation
Canonical Support escalated to the Canonical FIPS team, and Matthew, a Canonical support engineer, led the technical investigation.
The core challenge was finding a fix that did two things simultaneously:
- Made libgcrypt20-fips, gnutls28, and ubuntu-fips work correctly on standard kernels without the GRND_RESEED_ONLY flag
- Preserved FIPS compliance without requiring a full recertification cycle
FIPS certification is not a property you can alter lightly. Changes to cryptographic packages can invalidate certification, requiring months of revalidation through NIST’s (National Institute of Standards and Technology) CMVP (Cryptographic Module Validation Program) process. Any fix that broke certification would simply trade one blocker for another for customers with active compliance obligations.
Matthew’s patch resolved both constraints. It modified the packages’ behaviour when GRND_RESEED_ONLY was unavailable, allowing them to fall back correctly without compromising the cryptographic guarantees that underpin FIPS compliance.
Testing and release
Canonical provided the patched packages to the customer via a private PPA for validation. The customer tested the containers against their target environments and confirmed the fix worked.
Canonical then released the packages to the jammy-fips-updates and noble-fips-updates repositories. Customers running Ubuntu Pro 22.04 or 24.04 FIPS container images on standard cloud kernels should ensure their systems are pulling from jammy-fips-updates or noble-fips-updates respectively.
What this means for FIPS container deployments
No single layer told the full story here: the container looked correct, the host was FIPS-compliant, and the failure only showed up in the gap between a package assumption and a kernel that didn’t support it. That’s the part worth remembering: FIPS compliance in cloud-native environments depends on the interaction between certified packages and the underlying kernel, not just on using certified images.
Canonical Support provides access to expert engineers across the full open source stack, including the FIPS team. If you are working through a FIPS or FedRAMP deployment and running into compatibility issues, contact us or visit our Support page.