added docker compose for running local arma network run with grafana by moran-abilea · Pull Request #1203 · hyperledger/fabric-x-orderer

moran-abilea · 2026-09-09T14:33:22Z

related to #449

Running Arma with Prometheus and Grafana inside docker compose

it runs an Arma sample network together with Prometheus and
Grafana, and watch the metrics of a load test on a dashboard while it runs.

By default it runs a network of four parties with two shards — each party has a
router, two batchers, a consensus and an assembler node — and drives it with a five
minute load test. The number of parties, the number of shards, the duration and the
load are all configurable.

Everything runs as the user that invokes the scripts.

Copilot

🟡 Changes recommended

The documented build and startup flow fails, and the dashboard contains a non-working metric query.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a local Docker Compose example running Arma with Prometheus, Grafana, and a configurable load test.

Changes:

Adds setup, cleanup, and Docker build scripts.
Adds Prometheus/Grafana provisioning and monitoring dashboard.
Documents local usage and configuration.

File summaries

File	Summary
`node/examples/grafana/scripts/run_sample.sh`	Generates configuration and starts the sample network; composed node paths currently prevent services from starting.
`node/examples/grafana/scripts/clean_sample.sh`	Stops containers and removes generated data.
`node/examples/grafana/scripts/build_docker.sh`	Builds the Arma image; the documented build command and image tag are currently incorrect.
`node/examples/grafana/README.md`	Documents setup and operation; its Grafana volume guidance does not match the Compose configuration.
`node/examples/grafana/grafana/datasource.yaml`	Provisions Prometheus as Grafana’s data source.
`node/examples/grafana/grafana/dashboards.yaml`	Configures dashboard provisioning.
`node/examples/grafana/grafana/arma-dashboard.json`	Defines monitoring panels; one batcher latency query uses the wrong metric namespace.
`node/examples/grafana/compose.yaml`	Defines Prometheus, Grafana, and submitter services; Grafana data is not persisted.

Review details

Suppressed comments (2)

node/examples/grafana/README.md:95

The README says edits live in a Grafana volume, but this compose file declares no Grafana data volume; only the read-only provisioning and dashboard bind mounts are present. Dashboard edits therefore live in the container's ephemeral filesystem and are lost when the container is recreated or removed by cleanup. Please either add a persistent volume and explain its lifecycle, or document that edits must be exported before teardown without calling it a volume.

Edits live in Grafana's volume, which `clean_sample.sh` removes, so export the JSON over `grafana/arma-dashboard.json` to keep one.

node/examples/grafana/compose.yaml:30

The Grafana data directory is not persisted: these mounts only provide read-only provisioning files, and no volume is mounted at /var/lib/grafana. Consequently dashboard edits made through the UI are lost when the container is recreated, contrary to the README's claim that edits live in Grafana's volume. Add a named Grafana data volume (and keep the cleanup script removing it) if UI edits are meant to survive a restart.

      - /tmp/arma-sample/grafana/provisioning:/etc/grafana/provisioning:ro
      - /tmp/arma-sample/grafana/dashboards:/etc/grafana/dashboards:ro

Files reviewed: 7/8 changed files
Comments generated: 3
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

moran-abilea · 2026-09-20T08:38:18Z

+cd "$(dirname "${BASH_SOURCE[0]}")/../.."
+bash ./scripts/build_docker.sh


fixed as suggested,
also not written here in the review note but also fixed as suggested: added correct links in the readme.md

moran-abilea · 2026-09-20T08:38:22Z

+chmod a+x "${SAMPLE_DIR}"
+chmod -R a+rX "${SAMPLE_DIR}/grafana" "${SAMPLE_DIR}/prometheus.yml"
+
+"${COMPOSE[@]}" up -d


checked it, it's wrong
nothing to fix here

moran-abilea · 2026-09-20T08:38:27Z

+                      },
+                      "spec": {
+                        "editorMode": "code",
+                        "expr": "sum by (party_id, shard_id) (\r\n  batch_ledger_append_latency_seconds_sum\r\n)\r\n/\r\nsum by (party_id, shard_id) (\r\n   batch_ledger_append_latency_seconds_count\r\n)",


checked it, it's wrong
nothing to fix here

moradna · 2026-09-17T07:36:22Z

+
+# The same "arma" image the other examples use.
+cd "$(dirname "${BASH_SOURCE[0]}")/../.."
+bash ./scripts/build_docker.sh


I don't think this wrapper file is needed, you can use the existing node/examples/scripts/build_docker.sh directly

fixed as suggested

moradna · 2026-09-17T08:42:40Z

+# The generator leaves the operations port unset, which means a random one, and binds it
+# to localhost. Prometheus scrapes it from another container and needs to know it in
+# advance. Every node has its own container, so one port serves them all.


maybe this comment can be shorter, for example Expose a fixed operations port so Prometheus can scrape the nodes.

fixed as suggested

moradna · 2026-09-17T08:43:51Z

+mkdir -p "${SAMPLE_DIR}/grafana/provisioning/datasources" \
+    "${SAMPLE_DIR}/grafana/provisioning/dashboards" \
+    "${SAMPLE_DIR}/grafana/dashboards"
+cp "${EXAMPLE_DIR}/grafana/datasource.yaml" "${SAMPLE_DIR}/grafana/provisioning/datasources/"
+cp "${EXAMPLE_DIR}/grafana/dashboards.yaml" "${SAMPLE_DIR}/grafana/provisioning/dashboards/"
+cp "${EXAMPLE_DIR}/grafana/arma-dashboard.json" "${SAMPLE_DIR}/grafana/dashboards/"


Do we need to copy the grafana files into /tmp? They can be mounted directly from compose.yaml instead

I tried mounting these files directly from the repository, and it simplifies the script. However, it makes the example dependent on the checkout permissions. With umask 027, the files are 640 and Grafana fails to read the bind mounts.

We could chmod the repository files before starting Compose, but I'd prefer not to modify permissions in the user's checkout. An init container also works, but adds a service and named volumes just to stage three files.

So I kept the small staging step in /tmp, and added a comment explaining that it intentionally normalizes permissions for the container.

moradna · 2026-09-17T08:50:17Z

+# Scrape every node of the network. The hosts come from the deployment, so to change the
+# number of parties or shards edit ../config/example-deployment.yaml and ../compose.yaml.


I don't think this comment is needed

removed entierly

moradna · 2026-09-17T11:16:10Z

+```
+
+The number of parties and shards comes from the network itself, so changing it means
+editing [../config/example-deployment.yaml] and


this looks like a link, but it doesn't have a target

alread fixed, with the rest of the links

moradna · 2026-09-17T11:31:57Z

+### Working on a New Metric Methodology
+
+1. Add the metric in `node/<role>/metrics.go`, keeping `party_id` and `shard_id` among its
+   `LabelNames` as the existing metrics do. 
+   Its Prometheus name is `<namespace>_<name>`.
+2. Rebuild the image and start again: `build_docker.sh`, `clean_sample.sh`, `run_sample.sh`.
+3. Confirm it is exported, in the Prometheus UI or straight from a node:
+   `docker exec arma-grafana-prometheus-1 wget -qO- http://router.p1:8080/metrics | grep <name>`
+4. Add a panel in Grafana, selecting `${DS_PROMETHEUS}` as its data source rather than the
+   Prometheus data source itself, so the dashboard stays independent of this setup.
+5. Export it over `grafana/arma-dashboard.json` and document the metric in [docs/monitoring/metrics.md].


I think this section is a bit out of scope here.

section removed entierly

Signed-off-by: Moran Abilea <moran.abilea@ibm.com>

moran-abilea requested review from moradna and tock-ibm September 9, 2026 14:33

moran-abilea self-assigned this Sep 9, 2026

moran-abilea force-pushed the dockerize-grafana branch 3 times, most recently from 1a83e2c to 6b0df3a Compare September 10, 2026 07:39

moran-abilea marked this pull request as draft September 10, 2026 09:22

moran-abilea marked this pull request as ready for review September 10, 2026 10:41

moran-abilea force-pushed the dockerize-grafana branch 5 times, most recently from b105104 to 184028f Compare September 16, 2026 11:19

moradna requested a lite review from Copilot September 17, 2026 08:21

Copilot started reviewing on behalf of moradna September 17, 2026 08:21 View session

Copilot AI reviewed Sep 17, 2026

View reviewed changes

moran-abilea force-pushed the dockerize-grafana branch 3 times, most recently from eaf18a1 to 0acdc3a Compare September 20, 2026 08:38

moradna reviewed Sep 22, 2026

View reviewed changes

moran-abilea force-pushed the dockerize-grafana branch 3 times, most recently from 8d56e83 to 967fce4 Compare September 22, 2026 12:46

moran-abilea requested a review from moradna September 22, 2026 12:46

moran-abilea added 2 commits September 24, 2026 14:39

Add docker compose for running local arma network run with grafana

625fe64

Signed-off-by: Moran Abilea <moran.abilea@ibm.com>

fix review notes

2af5d4f

Signed-off-by: Moran Abilea <moran.abilea@ibm.com>

moran-abilea force-pushed the dockerize-grafana branch from 967fce4 to 2af5d4f Compare September 24, 2026 11:39

		cd "$(dirname "${BASH_SOURCE[0]}")/../.."
		bash ./scripts/build_docker.sh

		# Scrape every node of the network. The hosts come from the deployment, so to change the
		# number of parties or shards edit ../config/example-deployment.yaml and ../compose.yaml.

Conversation

moran-abilea commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (2)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

moran-abilea commented Sep 9, 2026 •

edited

Loading