Skip to content

feat: support S3 SSE-C (customer-provided encryption keys) - #1017

Open
schaurian wants to merge 2 commits into
cloudnative-pg:mainfrom
schaurian:feat/s3-sse-c
Open

feat: support S3 SSE-C (customer-provided encryption keys)#1017
schaurian wants to merge 2 commits into
cloudnative-pg:mainfrom
schaurian:feat/s3-sse-c

Conversation

@schaurian

@schaurian schaurian commented Jul 19, 2026

Copy link
Copy Markdown

Summary

Adds support for S3 Server-Side Encryption with Customer-provided keys (SSE-C) to the ObjectStore resource, closing #646.

S3-compatible providers such as Hetzner Object Storage do not offer bucket-managed encryption (SSE-S3 / SSE-KMS) and only support SSE-C. Without this, backups and WALs can only be stored unencrypted at rest on those providers.

Users can now reference a secret holding a base64-encoded 256-bit AES key via a new sseCustomerKey field:

apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
spec:
  configuration:
    destinationPath: "s3://BUCKET_NAME/path/to/folder"
    endpointURL: "https://fsn1.your-objectstorage.com"
    s3Credentials:
      accessKeyId: { name: aws-creds, key: ACCESS_KEY_ID }
      secretAccessKey: { name: aws-creds, key: ACCESS_SECRET_KEY }
      sseCustomerKey: { name: aws-sse-c, key: key }

What changed

The field is defined in the shared barman-cloud library and flows through the embedded BarmanObjectStoreConfiguration, so the plugin change is intentionally small:

File Change
go.mod / go.sum Point barman-cloud at the version carrying sseCustomerKey (see below)
internal/cnpgi/operator/specs/secrets.go Include sseCustomerKey in CollectSecretNamesFromCredentials, so the instance Role covers a key kept in its own Secret (by @andrein, with test)
config/crd/bases/…_objectstores.yaml Regenerated CRD (adds sseCustomerKey)
manifest.yaml Regenerated consolidated installer
web/docs/object_stores.md New "Server-Side Encryption with Customer Keys (SSE-C)" section
.wordlist.txt Spellcheck entries for the new docs section

No controller logic changes are needed: the library injects --sse-customer-key file://… into every barman-cloud-* command (backup, wal-archive, wal-restore, restore, backup-list, backup-delete, check-wal-archive) and materializes the key from its secret, so all read/write paths are covered.

sseCustomerKey cannot be combined with data.encryption / wal.encryption: barman-cloud rejects --sse-customer-key together with --encryption. barman-cloud#284 adds that check to ValidateBackupConfiguration; note that this plugin does not currently call ValidateBackupConfiguration for ObjectStore objects, so today the combination is only caught at run time (documented as such). Happy to wire the validation into the plugin in a follow-up if you'd like it enforced at admission.

Dependency

This PR depends on cloudnative-pg/barman-cloud#284. Until that is merged and released, go.mod carries a temporary replace directive pointing at the PR head:

replace github.com/cloudnative-pg/barman-cloud => github.com/schaurian/barman-cloud v0.5.2-0.20260908142745-14f89b687e74

Once barman-cloud#284 is released, the replace should be dropped and the require bumped to the released version.

Runtime use requires Barman ≥ 3.20.0 (--sse-customer-key option), which the sidecar image pins since v0.15.0 — this branch is rebased on main accordingly.

Testing

  • Full task ci run locally: commitlint, spellcheck, go test (envtest), uncommitted-drift check (CRD / manifest / API docs regenerated via the repo's dagger tasks), docusaurus build, and the ephemeral-k3s e2e suite — 10 passed, 0 failed, 4 skipped (the GCS specs disabled pending barman-cloud-backup 3.20.0 should respect STORAGE_EMULATOR_HOST during multiplepart backup EnterpriseDB/barman#1218). golangci-lint reports only gomoddirectives on the temporary replace line above, which goes away once the replace is dropped.
  • CRD and manifest.yaml diff to exactly the new field.
  • Field-tested by @andrein against a Ceph RGW store that supports only SSE-C, with CloudNativePG 1.30.0 (continuous WAL archiving with Encryption: SSE-C objects, base backup, backup-list, retention via backup-delete, serverRecoveryWindow, and recovery into a new Cluster via externalClusters[].plugin) — see the comment below.

Closes #646

🤖 Generated with Claude Code

@schaurian
schaurian marked this pull request as ready for review August 9, 2026 16:14
@schaurian
schaurian requested a review from a team as a code owner August 9, 2026 16:14
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. enhancement New feature or request go Pull requests that update go code labels Aug 9, 2026
@andrein

andrein commented Sep 8, 2026

Copy link
Copy Markdown

Tested at adb7045 with main merged in for Barman 3.20.0, against a Ceph RGW
object store that supports only SSE-C, with CloudNativePG 1.30.0 and
cloudnative-pg/barman-cloud#284 at a300841.

Two changes were required:

  1. sseCustomerKey is not collected by CollectSecretNamesFromCredentials, so
    the instance Role does not cover a key stored in its own Secret, as in the
    aws-sse-c docs example, and credential resolution fails with a forbidden
    error. fix: grant the instance role access to the SSE-C key secret schaurian/plugin-barman-cloud#1 against this branch adds it with a test.
  2. The branch is based on v0.14.0, whose sidecar pins Barman 3.19.1;
    --sse-customer-key exists from 3.20.0, pinned on main since v0.15.0.

With those:

  • key materialised in the sidecar and passed to every barman-cloud command,
    including backup-list and backup-delete
  • WAL archiving continuous; objects report Encryption: SSE-C, 400 without the key
  • base backup completed with Backup.status populated
  • ObjectStore.status.serverRecoveryWindow populated
  • retention applied on the primary
  • recovery into a new Cluster through externalClusters[].plugin reproduced
    the source database

schaurian and others added 2 commits September 8, 2026 16:36
Surface the new `sseCustomerKey` field on `s3Credentials` through the
ObjectStore CRD so users can enable Server-Side Encryption with
Customer-provided keys (SSE-C). This is required by S3-compatible
providers that only support SSE-C for encryption at rest, such as
Hetzner Object Storage.

The field flows through the embedded BarmanObjectStoreConfiguration
from the barman-cloud library, so this change is limited to bumping the
dependency, regenerating the CRD and the consolidated manifest, and
documenting usage in the object stores guide.

Depends on cloudnative-pg/barman-cloud#284 (temporarily pinned via a
replace directive until that change is released).

Closes cloudnative-pg#646

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Florian Schauer <florian@schauer.to>
The sidecar reads credential secrets through a Role whose resource
names are collected from the credential references. Without the new
reference in that list, a key kept in its own secret, as in the docs
example, is forbidden and every barman-cloud command fails.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Andrei Nistor <andrei@nistor.tech>
@schaurian

Copy link
Copy Markdown
Author

Thanks for the end-to-end run and the RBAC fix, @andrein — merged schaurian#1 into this branch.

Updated the branch (eed228a):

  • Rebased onto main, so the sidecar carries Barman 3.20.0 and the branch no longer conflicts.
  • go.mod replace now points at the current head of feat: add support for S3 SSE-C (customer-provided encryption keys) barman-cloud#284 (14f89b6), which includes your ValidateBackupConfiguration change.
  • CRD, manifest.yaml and API docs regenerated with the repo's dagger tasks; the diff against main is exactly the new field.
  • Docs: the SSE-C section no longer claims the field is independent of encryption — it now states that sseCustomerKey cannot be combined with data.encryption / wal.encryption — and the version note points at Barman ≥ 3.20.0 (sidecar since v0.15.0) instead of the barman issue.
  • Added the new docs words to .wordlist.txt; spellcheck was the other reason the first CI run failed.

Local task ci on this head: commitlint, spellcheck, go test, uncommitted-drift check, docusaurus build and the ephemeral e2e suite (10 passed, 0 failed, 4 skipped — the GCS specs) all pass. golangci-lint reports only gomoddirectives for the temporary replace, which disappears once barman-cloud#284 is released and the replace is dropped.

One observation: the plugin does not call ValidateBackupConfiguration anywhere, so the new barman-cloud check does not currently guard ObjectStore objects — an object store with both sseCustomerKey and data.encryption is still accepted and fails at the first backup. The docs describe that behaviour. If you'd like it rejected at admission, I'm happy to wire the validation into the plugin in a follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request go Pull requests that update go code size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature request] Support for S3 SSE-C - Server-Side Encryption with Customer-provided keys

2 participants