Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/infrahub-backup/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ version: 1.6.0
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "2.4.1"
appVersion: "2.6.0"
14 changes: 11 additions & 3 deletions charts/infrahub-backup/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,13 @@ Backup and restore Helm chart for Infrahub on Kubernetes
| cronJob.successfulJobsHistoryLimit | int | `3` | Number of successful job history to keep |
| cronJob.suspend | bool | `false` | Suspend the CronJob (useful for maintenance) |
| cronJob.timeZone | string | `""` | Time zone for `backup.schedule` and `restore.schedule`, as an IANA name such as "Europe/Zurich". Empty uses the time zone of the kube-controller-manager, usually UTC. |
| externalDatabase | object | `{"allowRestore":false,"insecureTLS":false,"neo4jAddress":"","neo4jBackupPort":"","postgresAddress":"","timeout":""}` | Settings for a Neo4j or PostgreSQL database that runs outside the Infrahub namespace. They have no effect on a database that runs in the namespace. The backup and restore pods run in the Infrahub namespace, so infrahub-backup runs `neo4j-admin`, `cypher-shell` and the PostgreSQL client in its own container, and creates no Job and no Secret. Needs infrahub-backup 2.6.0 or later. |
| externalDatabase.allowRestore | bool | `false` | Allow a restore to overwrite an external database. Sets `INFRAHUB_ALLOW_EXTERNAL_RESTORE` on the restore containers only. A restore of an external Neo4j database also needs `restore.storage.type: s3`: infrahub-backup copies the artifact under `seed/` in `restore.storage.s3.bucket`, and the Neo4j servers download it from there. So the credentials in `restore.storage.s3.secretName` must be able to write to that bucket, also with `restore.storage.s3.key`, and the Neo4j servers must be able to read from it. |
| externalDatabase.insecureTLS | bool | `false` | Connect to an external database without verifying its certificate, and accept a connection with no encryption when the deployment has TLS disabled. Sets `INFRAHUB_EXTERNAL_DB_INSECURE_TLS`. An external Neo4j server with TLS disabled needs it. |
| externalDatabase.neo4jAddress | string | `""` | Neo4j hosts to back up from, as "host[:port][,host[:port]...]". Sets `INFRAHUB_NEO4J_ADDRESS`. Empty uses the address the Infrahub server is configured with. For a Neo4j cluster, list every member, so that the backup can use another member when one is down. |
| externalDatabase.neo4jBackupPort | string | `""` | Port of the Neo4j backup service on every host, not the Bolt port. Sets `INFRAHUB_NEO4J_BACKUP_PORT`. Empty uses the infrahub-backup default, 6362. |
| externalDatabase.postgresAddress | string | `""` | PostgreSQL endpoint of the task manager database, as "host[:port]". Sets `INFRAHUB_POSTGRES_ADDRESS`. Empty uses the address the task manager is configured with. |
| externalDatabase.timeout | string | `""` | Time limit for each operation against an external database, with a unit such as "4h" or "90m". Sets `INFRAHUB_EXTERNAL_DB_TIMEOUT`. Empty uses the infrahub-backup default, 2h. |
| extraEnv | list | `[]` | Extra environment variables for the backup and restore containers, appended after the ones the chart sets. For example, trust a private CA for an S3 endpoint with a self-signed or internal certificate: `[{name: SSL_CERT_FILE, value: /etc/ssl/custom/ca.crt}]` together with the volume below. |
| extraVolumeMounts | list | `[]` | Extra volume mounts for the backup and restore containers: `[{name: s3-ca, mountPath: /etc/ssl/custom, readOnly: true}]` |
| extraVolumes | list | `[]` | Extra volumes for the backup and restore pods, for example a ConfigMap holding a CA certificate: `[{name: s3-ca, configMap: {name: s3-ca}}]` |
Expand All @@ -59,13 +66,14 @@ Backup and restore Helm chart for Infrahub on Kubernetes
| rbac | object | `{"create":true}` | RBAC configuration |
| rbac.create | bool | `true` | Create Role and RoleBinding for pod exec permissions |
| resources | object | `{"requests":{"cpu":"100m","memory":"256Mi"}}` | Resource limits and requests |
| restore | object | `{"enabled":false,"mode":"job","options":{"excludeTaskmanager":false,"extraArgs":[],"migrateFormat":false,"sleep":"3m"},"schedule":"0 4 * * *","storage":{"local":{"filename":"infrahub_backup_latest.tar.gz"},"path":"/infrahub_backups","s3":{"bucket":"","endpoint":"","key":"","latest":false,"prefix":"","region":"us-east-1","secretName":""},"type":"s3"}}` | Restore configuration |
| restore | object | `{"enabled":false,"mode":"job","options":{"excludeTaskmanager":false,"extraArgs":[],"migrateFormat":false,"neo4jClusterOnlineTimeout":"","sleep":"3m"},"schedule":"0 4 * * *","storage":{"local":{"filename":"infrahub_backup_latest.tar.gz"},"path":"/infrahub_backups","s3":{"bucket":"","endpoint":"","key":"","latest":false,"prefix":"","region":"us-east-1","secretName":""},"type":"s3"}}` | Restore configuration |
| restore.enabled | bool | `false` | Enable restore functionality (mutually exclusive with backup) |
| restore.mode | string | `"job"` | Restore mode: "job" for one-shot, "cronjob" for scheduled (e.g. a nightly prod -> staging sync) |
| restore.options | object | `{"excludeTaskmanager":false,"extraArgs":[],"migrateFormat":false,"sleep":"3m"}` | Restore options |
| restore.options | object | `{"excludeTaskmanager":false,"extraArgs":[],"migrateFormat":false,"neo4jClusterOnlineTimeout":"","sleep":"3m"}` | Restore options |
| restore.options.excludeTaskmanager | bool | `false` | Exclude task-manager database from restore |
| restore.options.extraArgs | list | `[]` | Extra arguments to pass to the restore command |
| restore.options.migrateFormat | bool | `false` | Migrate backup format from older versions |
| restore.options.neo4jClusterOnlineTimeout | string | `""` | With a Neo4j cluster, how long the restore waits for the database to come online on every server, such as "15m". Passes `--neo4j-cluster-online-timeout`, which needs infrahub-backup 2.4.1 or later. Empty uses the infrahub-backup default. |
| restore.options.sleep | string | `"3m"` | Sleep duration before restore when using local storage (allows time to copy files) |
| restore.schedule | string | `"0 4 * * *"` | Schedule for CronJob mode (cron expression) |
| restore.storage | object | `{"local":{"filename":"infrahub_backup_latest.tar.gz"},"path":"/infrahub_backups","s3":{"bucket":"","endpoint":"","key":"","latest":false,"prefix":"","region":"us-east-1","secretName":""},"type":"s3"}` | Storage configuration |
Expand All @@ -77,7 +85,7 @@ Backup and restore Helm chart for Infrahub on Kubernetes
| restore.storage.s3.endpoint | string | `""` | S3 endpoint URL (for non-AWS S3-compatible storage) |
| restore.storage.s3.key | string | `""` | S3 key (path) to the backup artifact (exactly one of key or latest must be set) |
| restore.storage.s3.latest | bool | `false` | Restore the newest backup under bucket/prefix instead of naming one (exactly one of key or latest must be set) |
| restore.storage.s3.prefix | string | `""` | S3 key prefix the backups live under (only used with latest; match the source's backup.storage.s3.prefix) |
| restore.storage.s3.prefix | string | `""` | S3 key prefix the backups live under (with latest, match the source's backup.storage.s3.prefix). A restore of an external Neo4j database also copies the artifact under `<prefix>/seed/`, so it needs write access to the bucket |
| restore.storage.s3.region | string | `"us-east-1"` | S3 region |
| restore.storage.s3.secretName | string | `""` | Name of Kubernetes Secret containing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY |
| restore.storage.type | string | `"s3"` | Storage type: "s3" or "local" ("cronjob" mode requires "s3") |
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added `externalDatabase` values for a Neo4j or PostgreSQL database that runs outside the Infrahub namespace: `allowRestore`, `insecureTLS`, `timeout`, `neo4jAddress`, `neo4jBackupPort` and `postgresAddress`. Each one sets the matching infrahub-backup environment variable. These values need infrahub-backup 2.6.0 or later.
1 change: 1 addition & 0 deletions charts/infrahub-backup/changelog/+image-2.6.0.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Bumped the `infrahub-backup` image to `2.6.0`, which the `externalDatabase` values need. It backs up and restores a Neo4j Enterprise or PostgreSQL database that runs outside the Infrahub namespace. The backup and restore pods of this chart run in the Infrahub namespace, so `infrahub-backup` runs the database tools in its own container, creates no Job and no Secret, and needs no change to the Role of the chart. See the [infrahub-backup 2.5.0](https://github.com/opsmill/infrahub-backup/releases/tag/v2.5.0) and [2.6.0](https://github.com/opsmill/infrahub-backup/releases/tag/v2.6.0) release notes.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added `restore.options.neo4jClusterOnlineTimeout`, which sets how long a restore waits for the database to come online on every server of a Neo4j cluster. It needs infrahub-backup 2.4.1 or later.
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A restore Job or CronJob with `restore.storage.s3.key` now also sets `INFRAHUB_S3_BUCKET`, `INFRAHUB_S3_PREFIX` and `INFRAHUB_S3_ENDPOINT`. A restore of an external Neo4j database copies the archive into this bucket, under `seed/`, for the Neo4j servers to download, so its credentials need write access to the bucket. The archive download does not change.
43 changes: 41 additions & 2 deletions charts/infrahub-backup/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,12 @@ Return S3 environment variables for restore
*/}}
{{- define "infrahub-backup.restoreS3EnvVars" -}}
{{- if eq .Values.restore.storage.type "s3" }}
{{- if .Values.restore.storage.s3.latest }}
{{- /*
The bucket, prefix and endpoint are set even when the key names the archive: a
restore of an external Neo4j database copies the archive into this bucket for
the Neo4j servers to download. The download of a named key reads the bucket
from its s3:// address and ignores these.
*/}}
- name: INFRAHUB_S3_BUCKET
value: {{ .Values.restore.storage.s3.bucket | quote }}
{{- if .Values.restore.storage.s3.prefix }}
Expand All @@ -115,7 +120,6 @@ Return S3 environment variables for restore
- name: INFRAHUB_S3_ENDPOINT
value: {{ .Values.restore.storage.s3.endpoint | quote }}
{{- end }}
{{- end }}
- name: INFRAHUB_S3_REGION
value: {{ .Values.restore.storage.s3.region | quote }}
{{- if .Values.restore.storage.s3.secretName }}
Expand All @@ -133,6 +137,38 @@ Return S3 environment variables for restore
{{- end }}
{{- end }}

{{/*
Return the environment variables for databases that run outside the Infrahub
namespace. Call with (dict "root" $ "restore" true) from a restore container.
*/}}
{{- define "infrahub-backup.externalDatabaseEnvVars" -}}
{{- $ext := .root.Values.externalDatabase -}}
{{- if and .restore $ext.allowRestore }}
- name: INFRAHUB_ALLOW_EXTERNAL_RESTORE
value: "true"
{{- end }}
{{- if $ext.insecureTLS }}
- name: INFRAHUB_EXTERNAL_DB_INSECURE_TLS
value: "true"
{{- end }}
{{- with $ext.timeout }}
- name: INFRAHUB_EXTERNAL_DB_TIMEOUT
value: {{ . | quote }}
{{- end }}
{{- with $ext.neo4jAddress }}
- name: INFRAHUB_NEO4J_ADDRESS
value: {{ . | quote }}
{{- end }}
{{- with $ext.neo4jBackupPort }}
- name: INFRAHUB_NEO4J_BACKUP_PORT
value: {{ . | quote }}
{{- end }}
{{- with $ext.postgresAddress }}
- name: INFRAHUB_POSTGRES_ADDRESS
value: {{ . | quote }}
{{- end }}
{{- end }}

{{/*
Return backup command arguments
*/}}
Expand Down Expand Up @@ -173,6 +209,9 @@ Return restore command arguments
{{- if .Values.restore.options.migrateFormat }}
- --migrate-format
{{- end }}
{{- with .Values.restore.options.neo4jClusterOnlineTimeout }}
- --neo4j-cluster-online-timeout={{ . }}
{{- end }}
{{- if eq .Values.restore.storage.type "local" }}
{{- if .Values.restore.options.sleep }}
- --sleep={{ .Values.restore.options.sleep }}
Expand Down
1 change: 1 addition & 0 deletions charts/infrahub-backup/templates/cronjob-backup.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ spec:
{{- if eq .Values.backup.storage.type "s3" }}
{{- include "infrahub-backup.s3EnvVars" . | nindent 16 }}
{{- end }}
{{- include "infrahub-backup.externalDatabaseEnvVars" (dict "root" $ "restore" false) | nindent 16 }}
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 16 }}
{{- end }}
Expand Down
1 change: 1 addition & 0 deletions charts/infrahub-backup/templates/cronjob-restore.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
{{- include "infrahub-backup.restoreS3EnvVars" . | nindent 16 }}
{{- include "infrahub-backup.externalDatabaseEnvVars" (dict "root" $ "restore" true) | nindent 16 }}
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 16 }}
{{- end }}
Expand Down
1 change: 1 addition & 0 deletions charts/infrahub-backup/templates/job-backup.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ spec:
{{- if eq .Values.backup.storage.type "s3" }}
{{- include "infrahub-backup.s3EnvVars" . | nindent 12 }}
{{- end }}
{{- include "infrahub-backup.externalDatabaseEnvVars" (dict "root" $ "restore" false) | nindent 12 }}
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
Expand Down
1 change: 1 addition & 0 deletions charts/infrahub-backup/templates/job-restore.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ spec:
fieldRef:
fieldPath: metadata.namespace
{{- include "infrahub-backup.restoreS3EnvVars" . | nindent 12 }}
{{- include "infrahub-backup.externalDatabaseEnvVars" (dict "root" $ "restore" true) | nindent 12 }}
{{- with .Values.extraEnv }}
{{- toYaml . | nindent 12 }}
{{- end }}
Expand Down
37 changes: 36 additions & 1 deletion charts/infrahub-backup/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@ restore:
s3:
# -- S3 bucket containing the backup
bucket: ""
# -- S3 key prefix the backups live under (only used with latest; match the source's backup.storage.s3.prefix)
# -- S3 key prefix the backups live under (with latest, match the source's backup.storage.s3.prefix). A restore
# of an external Neo4j database also copies the artifact under `<prefix>/seed/`, so it needs write access to the
# bucket
prefix: ""
# -- S3 key (path) to the backup artifact (exactly one of key or latest must be set)
key: ""
Expand All @@ -107,11 +109,44 @@ restore:
excludeTaskmanager: false
# -- Migrate backup format from older versions
migrateFormat: false
# -- With a Neo4j cluster, how long the restore waits for the database to come online on every server, such as
# "15m". Passes `--neo4j-cluster-online-timeout`, which needs infrahub-backup 2.4.1 or later. Empty uses the
# infrahub-backup default.
neo4jClusterOnlineTimeout: ""
# -- Sleep duration before restore when using local storage (allows time to copy files)
sleep: "3m" # default should be lower than Helm's default install timeout of 5m
# -- Extra arguments to pass to the restore command
extraArgs: []

# -- Settings for a Neo4j or PostgreSQL database that runs outside the Infrahub namespace. They have no effect on a
# database that runs in the namespace. The backup and restore pods run in the Infrahub namespace, so infrahub-backup
# runs `neo4j-admin`, `cypher-shell` and the PostgreSQL client in its own container, and creates no Job and no Secret.
# Needs infrahub-backup 2.6.0 or later.
externalDatabase:
# -- Allow a restore to overwrite an external database. Sets `INFRAHUB_ALLOW_EXTERNAL_RESTORE` on the restore
# containers only. A restore of an external Neo4j database also needs `restore.storage.type: s3`: infrahub-backup
# copies the artifact under `seed/` in `restore.storage.s3.bucket`, and the Neo4j servers download it from there.
# So the credentials in `restore.storage.s3.secretName` must be able to write to that bucket, also with
# `restore.storage.s3.key`, and the Neo4j servers must be able to read from it.
allowRestore: false
# -- Connect to an external database without verifying its certificate, and accept a connection with no
# encryption when the deployment has TLS disabled. Sets `INFRAHUB_EXTERNAL_DB_INSECURE_TLS`. An external Neo4j
# server with TLS disabled needs it.
insecureTLS: false
# -- Time limit for each operation against an external database, with a unit such as "4h" or "90m". Sets
# `INFRAHUB_EXTERNAL_DB_TIMEOUT`. Empty uses the infrahub-backup default, 2h.
timeout: ""
# -- Neo4j hosts to back up from, as "host[:port][,host[:port]...]". Sets `INFRAHUB_NEO4J_ADDRESS`. Empty uses the
# address the Infrahub server is configured with. For a Neo4j cluster, list every member, so that the backup can
# use another member when one is down.
neo4jAddress: ""
# -- Port of the Neo4j backup service on every host, not the Bolt port. Sets `INFRAHUB_NEO4J_BACKUP_PORT`. Empty
# uses the infrahub-backup default, 6362.
neo4jBackupPort: ""
# -- PostgreSQL endpoint of the task manager database, as "host[:port]". Sets `INFRAHUB_POSTGRES_ADDRESS`. Empty
# uses the address the task manager is configured with.
postgresAddress: ""

# -- Container image configuration
image:
repository: registry.opsmill.io/opsmill/infrahub-backup
Expand Down
2 changes: 1 addition & 1 deletion tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ chart changed (see `.github/workflows/ci.yml`):
|--------------------------|-----------------|----------------|
| `infrahub` | `infrahub` | Deploys the community chart; a `BuiltinTag` round-trips through the Infrahub SDK. |
| `infrahub-enterprise` | `enterprise` | Same SDK check against the enterprise chart. |
| `infrahub-backup` | `backup` | Seeds a tag, runs the backup Job to an S3 bucket served by Garage, deletes the tag, runs the restore Job, asserts the tag is back. The Jobs reach Garage over HTTPS with a certificate issued by a private CA that the test generates, and trust it through `extraEnv`, `extraVolumes` and `extraVolumeMounts`. A restore without these values must fail certificate verification. |
| `infrahub-backup` | `backup` | Seeds a tag, runs the backup Job to an S3 bucket served by Garage, deletes the tag, runs the restore Job, asserts the tag is back. The Jobs reach Garage over HTTPS with a certificate issued by a private CA that the test generates, and trust it through `extraEnv`, `extraVolumes` and `extraVolumeMounts`. A restore without these values must fail certificate verification. `test_infrahub_backup_external_neo4j.py` deploys Infrahub with `neo4j.enabled=false`, pointed at a Neo4j 2025.10.1 Enterprise server in the `neo4j-external` namespace. The backup Job must capture that database with the tools in its own container and create no Job and no Secret. The restore Job, with `externalDatabase.allowRestore`, must bring back the data from before the backup. |
| `infrahub-observability` | `observability` | Deploys Infrahub + the stack together (tracing → Tempo, Alloy scraping Infrahub), drives API traffic, then asserts via Grafana that Infrahub metrics (Prometheus) and Infrahub traces (Tempo) are queryable. |
| `infrahub-mcp` | `mcp` | Deploys Infrahub with the MCP sub-chart behind a shared ingress and drives it over the streamable-HTTP transport. |

Expand Down
Loading
Loading