feat: add grafana alert test action [DX-5147] - #1651
Conversation
|
👋 Tofel, thanks for creating this pull request! To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team. Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks! |
ea2b7dd to
70a1e50
Compare
…ontractkit/.github into dx-5147-grafana-alert-gate-action
There was a problem hiding this comment.
🟡 Changes recommended
The README and action implementation are materially inconsistent (e.g., ctf-ref/build-from-source vs released binary), and the current runner/date behavior needs alignment to avoid broken executions.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
Adds a new composite GitHub Action intended to act as a CD quality gate by recording Grafana alert state over a release/test window and then checking whether any watched alert was “bad” during that window.
Changes:
- Introduces
ci-grafana-alert-testcomposite action wiring to rungrafana-alertcheckinrecordandcheckmodes. - Adds documentation describing how to integrate the action into a workflow and how the gate behaves.
File summaries
| File | Description |
|---|---|
| actions/ci-grafana-alert-test/README.md | Documents the new action’s purpose, usage, and operational expectations. |
| actions/ci-grafana-alert-test/action.yml | Implements the composite action: input validation, window resolution, binary install, record/check execution, summary output, and artifact upload on failure. |
Review details
Suppressed comments (1)
actions/ci-grafana-alert-test/README.md:28
- The usage example includes
ctf-ref, but action.yml does not define this input, so consumers following the example will pass an unknown input and still be running the default behavior (downloaded v0.1.0).
```yaml
- uses: smartcontractkit/.github/actions/ci-grafana-alert-test@ci-grafana-alert-test/v1
with:
mode: record
grafana-url: ${{ vars.GRAFANA_URL }}
grafana-token: ${{ secrets.GRAFANA_TOKEN }}
alerts: |
My Service Latency
My Service Error Rate
ctf-ref: <commit-sha>
- Files reviewed: 2/2 changed files
- Comments generated: 5
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
1b9b781 to
a3389da
Compare
There was a problem hiding this comment.
🟡 Changes recommended
The action currently has avoidable security/robustness gaps (unsafe $GITHUB_OUTPUT write and no integrity verification of the downloaded executable) that should be addressed before relying on it as a CD quality gate.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Review details
Suppressed comments (2)
Previously missed (2) — in code that hasn't changed since the last review.
actions/ci-grafana-alert-test/action.yml:7
- The action description hard-codes the grafana-alertcheck version (
v0.1.0), but the executable version is already controlled viaRELEASE_VERSIONlater in the file. This creates a second source of truth and will drift on version bumps (and it contradicts the README claim that the version is pinned in one place).
actions/ci-grafana-alert-test/action.yml:367 - This action writes several outputs using
>> "$GITHUB_OUTPUT". To match the preferred pattern used elsewhere in this repo and avoid subtle redirection/escaping issues, append outputs via| tee -a "$GITHUB_OUTPUT"(grouping them in a single block also keeps the step easier to maintain).
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Lite
Create an action that allows to validate whether Grafana Alerts were firing during some time period.
Test run: https://github.com/smartcontractkit/griddle-app-test-service/actions/runs/34370101552/job/102528834799