Skip to content

🤖 Update Benchmark Results - #6936

Merged
thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update
Sep 29, 2026
Merged

thomhurst merged 1 commit into
mainfrom
automated-benchmarks-update

Conversation

@thomhurst

Copy link
Copy Markdown
Owner

Automated Benchmark Update

This PR updates the benchmark documentation with the latest results from the Speed Comparison workflow.

Benchmarks Produced

Individual benchmark artifacts are available for download:

  • benchmark-DataDrivenTests
  • benchmark-AsyncTests
  • benchmark-ScaleTests
  • benchmark-MatrixTests
  • benchmark-MassiveParallelTests
  • benchmark-SetupTeardownTests
  • benchmark-BuildTime
  • benchmark-summary (aggregated overview)

Runtime Benchmarks

  • AsyncTests
  • DataDrivenTests
  • MassiveParallelTests
  • MatrixTests
  • ScaleTests
  • SetupTeardownTests

Build Benchmarks

  • BuildTime
  • 2026-09-29
  • Ubuntu Latest • .NET SDK 10.0.401

Changes

  • Updated benchmark data in docs/static/benchmarks/latest.json
  • Updated historical trends in docs/static/benchmarks/historical.json
  • Regenerated benchmark documentation in docs/docs/benchmarks/index.md
  • Updated benchmark summary in docs/static/benchmarks/summary.json

Workflow Run

  • Run ID: 36608749549
  • Triggered: workflow_dispatch
  • Date:

🤖 This PR was automatically created and will be merged automatically once CI checks pass.

@thomhurst thomhurst added automated benchmarks documentation Improvements or additions to documentation ignore-for-release labels Sep 29, 2026
@thomhurst
thomhurst deployed to Pull Requests September 29, 2026 18:13 — with GitHub Actions Active
@thomhurst
thomhurst deployed to Pull Requests September 29, 2026 18:13 — with GitHub Actions Active
@thomhurst
thomhurst deployed to Pull Requests September 29, 2026 18:13 — with GitHub Actions Active
@thomhurst
thomhurst merged commit 4345ee3 into main Sep 29, 2026
24 checks passed
@thomhurst
thomhurst deleted the automated-benchmarks-update branch September 29, 2026 18:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T18:20:06.593572Z 5b9ba94 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Review

Automated benchmark refresh, docs/data only (README table, benchmark markdown, static JSON). No source or engine code is touched, so there are no correctness, AOT or API concerns.

  • The README table and versions (TUnit 1.72.0) are consistent with the run date (2026-09-29).
  • Relative ordering is unchanged: TUnit (AOT) is fastest and TUnit beats the other frameworks in every scenario.
  • Absolute times for every framework moved together, up in most scenarios and down in the scale suite. That looks like CI runner noise rather than a regression.
  • I did not validate the JSON files programmatically, since a shell loop was blocked in this environment. I only spot-checked the diff.

LGTM.

@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Low risk] Benchmark results and documentation updated with new measurements.

The PR appears safe to merge, with a non-blocking history-retention issue worth correcting.

Findings

  1. P2 Duplicate history date ▶

Summary

The PR refreshes benchmark measurements and TUnit version labels across the README, benchmark pages, and published JSON data.

  • The generated history adds a duplicate same-day entry, reducing distinct-date coverage under its retention cap.

Reviews (1) · Last reviewed commit: "chore: update benchmark results"

Comment on lines 359 to 360
"date": "2026-09-29",
"environment": "Ubuntu"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Duplicate history date This update adds a second entry for 2026-09-29. The history keeps only the latest 90 entries, so the duplicate takes a slot and pushes out an older distinct date. Repeated same-day runs will further reduce the dates available for future trend analysis; the history generator should replace or deduplicate same-day entries.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5b9ba947b2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +354 to +357
{
"date": "2026-09-29",
"environment": "Ubuntu"
},

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid appending duplicate same-day history records

When the benchmark workflow runs more than once on the same UTC date, as happened here after f197760, this adds a second indistinguishable 2026-09-29 entry. Because the producer subsequently truncates the array with slice(-90), the duplicate also evicts a genuine older data point, reducing historical coverage without recording a new time point; replace or deduplicate the current day's entry before truncation.

Useful? React with 👍 / 👎.

This branch was successfully deployed

1 active deployment
Pull Requests — 5b9ba947 Deployed Sep 29, 2026 by thomhurst via modularpipeline (ubuntu-latest) #19623
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated benchmarks documentation Improvements or additions to documentation ignore-for-release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant