[Ti-Tai Wang (titaiwangms)]

Description

Add default-off, op-owned CUDA MoE source fusion on top of #34022.

• Recognize the bounded FP32 dense SwiGLU source graph: raw-logit TopK, selected-logit Softmax, per-expert projections and weighted sum. Support ONNX 18 / Microsoft 1 imports, E=2..64, and k=1..E with conservative shape/alignment/integer bounds.
• Reuse PatternRewriter, the existing CUDA capability callback, exact typed-target probing, owned offer snapshots and deferred exact-region replacement. Preserve immutable learned banks; do not introduce a second matcher or support registry.
• Gate ep.cuda.runtime_fusion.moe independently and require actual SM80+ hardware before offering FP32 MoE.
• Batch matcher roots to prepare plans/indexing once per scan. Keep op-owned factories lightweight and materialize declarations only after configuration/admission.
• Validate graph-replacement equivalence, not the numerical accuracy of existing kernels: compare source/fused output with TF32 disabled and fused/direct-target output in both TF32 modes. Both modes retain source execution, CUDA placement, target-attribute, shape and finite-output checks.

No production kernel changes, expanded existing-pilot matching scope, default enablement or initializer transformations are included.

Earlier review follow-ups: all five findings in the earlier assessed Copilot review are addressed:

• Bound N to 65535 * 1024 for the existing CUDA routing fallback's grid-Y limit. Metadata-only tests cover the limit and one past it, including no target lookup or graph mutation on rejection.
• Apply a cheap necessary root check: unassigned ONNX Add with a right input produced by ONNX Mul. The existing matcher remains authoritative. Tests cover 10,000 unrelated Adds, positive roots, and structural near misses.
• Document application-side ConstantFolding;TransposeOptimizer disabling where needed to preserve the admitted immutable-bank topology. The MoE flag does not change ordinary optimizer behavior.
• Refresh factory/eager-catalog/lazy-parsing semantics, opt-in PatternLimits ceilings, and batched visitor behavior.
• Normally merge [Fusion] Add op-owned contrib source declarations v0 #34022's declaration/binding/schema-linkage/output-mapping fixes while retaining the MoE mathematical profile.

Numerical integration scope: remove only the TF32-on source/fused blocking assertion, rather than increasing its tolerance or changing an existing kernel. Source MatMul can use TF32, whereas the existing FP32 MoE dispatch uses SIMT. This is a cross-kernel floating-point-path comparison, not a replacement-wiring accuracy contract.

The original TF32-off source/fused bound remains atol=1e-5, rtol=2e-4. Fused/direct consistency retains exactly that bound with TF32 both disabled and enabled. All 88 distinct fixture combinations and their 264 source/fused/direct sessions remain after deduplicating repeated E=2/k=2 inputs, including the original source execution in TF32-on mode. No independent high-precision kernel-accuracy gate is added. The numerical-profile description and CUDA fusion documentation now state this scope consistently.

For provenance, the earlier head 2453a226a4 completed the original CUDA matrix with one TF32-on source/fused assertion failure at E=64, k=1, N=7, scaled inputs, element 144: difference 0.0020274445414543152, former bound 0.0019924522042274475. The latest passing matrix uses the explicitly revised integration scope; it is not evidence that this cross-kernel difference or kernel accuracy has been fixed.

Latest review fixes: addressed all three inline items in review 5851521402.

• Decline FP32 source fusion when the existing target's float-sized workspace would round downward; preserve exact/upward-rounded sizes. Twelve metadata-only cases cover the reported 448-byte shortfall and both affected buffers, without huge allocations or kernel edits.
• Validate exact configuration values before pre-SM80 decline through an approved append-only, metadata-only ProviderHost virtual. Selection and validation reuse the same factory/config resolver; no graph access, materialization, target lookup, new matcher or public C API is introduced.
• Keep the matrix independently filterable/shardable. After the test-only cleanup, it contains 88 distinct gtests / 264 sessions, removing only the repeated E=2/k=2 combinations while retaining CUDA placement checks and the approved numerical gates.

Motivation and Context

Extend the op-owned source-declaration path to MoE without a second EP-wide rewrite registry. Keep the existing ScaledSiLU and CausalConvWithState pilots unchanged while testing a larger family of alternatives.

This PR depends on #34022 and remains in native stack #34023 with ordered history #34021 (merged), #34022, #34086. Updates preserve published history; no force-push or default enablement.

Current test-only cleanup (2026-11-11)

Parent: 04f2a6e58cb216fc182b8955b48018d448eed66f; combined stack: b776d13387af86c78036e48262ca8b7cdd05f1f9. Normal commits/merge and fast-forward publication only. The final committed test contents match the validated source overlay byte-for-byte; production code, kernels, matching scope, default-off behavior and numerical gates are unchanged.

Remove owner-policy English-reason coupling and isolate malformed/duplicate registered-schema and legacy-key failures. Complete the other declarations so missing-policy rejection cannot mask schema-key validation.

Deduplicate the CUDA matrix from 96 to 88 distinct cases / 264 source-fused-direct sessions: compiled and executed the actual old/new parameter generators using ORT containers, proving identical distinct configurations and unique parameter names. Remove the diagnostic-only k-choice field. Consolidate the nine near-miss mutations at E=8/16/64 on production discovery, retaining no-lookup/no-mutation assertions and the separate fixed-candidate wrong-k regression. Remove one duplicate host E=2/k=2 fixture, unasserted timing output and the exact 63-plan implementation assertion; single-index preparation and matching-result checks remain.

Validation: passed on the combined source overlay: fresh CPU rebuild and 45/45 targeted host tests, zero failures/errors/skips; actual CUDA test translation-unit compilation with the configured CUDA-build C++ flags; canonical CLANGFORMAT and whitespace checks. The compiled old/new generator comparison passed (96 generated cases versus 88 unique cases, equal distinct configurations). Temporary generator source/executable were removed afterward.

Exact commands and results:

# PASS: rebuild the two changed host test objects and relink
cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/host/Release --target onnxruntime_test_all --parallel 2
# PASS: 45/45, from the build output directory
cd /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/host/Release
./onnxruntime_test_all --gtest_filter='MoE*:ContribSourceReplacementPolicyTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/test-cleanup-host.xml
# PASS: actual old/new C++ parameter generators, no GPU use
python /home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/validate_test_cleanup_cases.py

Not rerun for this test-only cleanup: GPU numerical execution, full/cross-platform suites, and standalone schema consumers. The older 386-host/98-GPU results below are historical, not reruns on these heads. The 45-test run used the combined stack, not a separate parent-only build. No new-head CI success is claimed.

Previous CI policy validation after syncing main 0b22d27a85 (before test-only cleanup)

Parent: 5362926ba8374fa250943346647c9269212db011; combined stack: e63f8268ea5d7c4113c861ef4d9a1418694ce12a. Normal merges only; no history rewrite.

Upstream #33901 added MatMulNBitsQLoRAMlp-1 and MatMulNBitsQLoRAQkv-1. Both now author PendingTodo beside their schema definitions; neither is added to the frozen legacy baseline or advertised as an executable source. The existing regression covers both exact keys, unqualified variants, CPU/CUDA/WebGPU, rejection of version 2, and rejection after removing either policy. This follow-up does not edit kernels, matching scope or default-off behavior.

Passed: 386/386 CPU-build host tests on the exact clean combined commit, including full inventory and the extended owner-policy regression; 2/2 standalone schema consumers; all 3 disabled-contrib syntax probes (bert_defs.cc, contrib_defs.cc, training_op_defs.cc, with both training macros); canonical CLANGFORMAT and diff whitespace checks. The host/schema builds used the newly pinned Eigen archive (SHA1 18eca78e961ac5e9123ccdff5035ab340dcc5e11), isolated from the earlier Eigen 3.4 cache. At most four compilation jobs. Source-tree identity and artifact hashes were checked before publication.

Exact build/test commands (all passed; tests run from their respective build output directories):

# PASS: schema-build
cmake --build /mnt/data/titaiwang/runtime-fusion/review-fixes-schema-validation-20261110/Release --parallel 4 --target ort_opschema_lib_test ort_opschema_lib_disabled_contrib_test
# PASS: schema-consumers
ctest --test-dir /mnt/data/titaiwang/runtime-fusion/review-fixes-schema-validation-20261110/Release -R '^ort_opschema_lib(_disabled_contrib)?_test$' --output-on-failure --no-tests=error --timeout 60
# PASS: runtime-build
cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/host/Release --parallel 4 --target onnxruntime_test_all
# PASS: host
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/host/Release/onnxruntime_test_all '--gtest_filter=MoE*:ContribSourceReplacementPolicyTest.*:UnitSwishHostTest.*:CUDARuntimeFusionHostTest.*:PatternRewriterTests.*:PatternComponentTests.*:PatternCapabilityTests.*:ReplacementComponentTests.*:CheckAndBindComponentTests.*:MatchComponentTests.*:FunctionTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/fusion-ci-20261111T1809/validation-3/host.xml

Exclusions for that previous main-sync follow-up: no GPU workload or CUDA kernel rebuild was performed for this metadata-only follow-up. The 98-case CUDA result below belongs to the earlier pre-main-sync tree and is not qualification of the new merged head. Windows/macOS/Android/Web/plugin execution and full upstream integration coverage remain CI responsibilities; new-head CI is not claimed green. The bounded two-file authored diff was reviewed directly; the upstream commit set was not independently re-reviewed.

Previous review-fix validation before the main sync

Previous review-fix validation: 9dcc6903bba388e5d7a5d2506db1a88e4a50fb52

Parent-only overlay: 358/358 host, 2/2 real CUDA EP, 2/2 standalone schema consumers passed. That run preceded only extraction of an unchanged test DSL literal into a constant.

The final combined stack passed 386/386 host, 98/98 real CUDA EP tests (96 MoE cases / 288 sessions plus the two existing pilots), 2/2 standalone schema consumers, and disabled-contrib syntax probes for bert_defs.cc, contrib_defs.cc, and training_op_defs.cc with both training macros. CUDA execution used idle physical GPU 3, H200/sm90, under the fail-closed guard; zero gtest failures/errors/skips. The newly built task-owned CUDA provider was loader-verified and artifact hashes stayed unchanged during execution. Four build jobs, NVCC threads=1.

Commits were created after validation; their merged tree was verified byte-for-byte against the compiled final source overlay before normal atomic publication. These are source-tree-qualified runs, not a claim that embedded build revision strings were regenerated after committing. The validation tree is clean at the published merge head.

Commands below ran from the existing build-output directories with the existing CUDA/cuDNN loader paths. Each completed successfully:

timeout --signal=TERM --kill-after=15s 14400s cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release --parallel 4 --target onnxruntime_test_all onnxruntime_providers_cuda onnxruntime_provider_test
# Passed.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_test_all '--gtest_filter=MoE*:ContribSourceReplacementPolicyTest.*:UnitSwishHostTest.*:CUDARuntimeFusionHostTest.*:PatternRewriterTests.*:PatternComponentTests.*:PatternCapabilityTests.*:ReplacementComponentTests.*:CheckAndBindComponentTests.*:MatchComponentTests.*:FunctionTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/review-fixes-20261111/validation-combined-2/host.xml
# Passed.
timeout --signal=TERM --kill-after=15s 1800s cmake --build /mnt/data/titaiwang/runtime-fusion/review-fixes-schema-validation-20261110/Release --parallel 4 --target ort_opschema_lib_test ort_opschema_lib_disabled_contrib_test
# Passed.
ctest --test-dir /mnt/data/titaiwang/runtime-fusion/review-fixes-schema-validation-20261110/Release -R '^ort_opschema_lib(_disabled_contrib)?_test$' --output-on-failure --no-tests=error --timeout 60
# Passed.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_provider_test '--gtest_filter=CudaRuntimeFusionTest.UnitSwishScalarFloatBitwise:CudaRuntimeFusionTest.ExtendedOptimizersNumericsAndCudaDispatch:MoE/CudaMoERuntimeFusionTest.SourceFusedDirectTF32OffAndOn/*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/review-fixes-20261111/validation-combined-3/cuda.xml
# Passed.

Canonical lintrunner --take CLANGFORMAT <changed C++ paths> and git diff --check passed in both working trees. Lintrunner used the existing virtual environment plus PYTHONPATH=/usr/lib/python3.12/site-packages; the earlier adapter blockage is resolved without installing dependencies. The task-created driver cache was removed.

Previous exclusions: actual pre-SM80 execution is not qualified on H200; host tests cover metadata validation without materialization/lookup and the actual-device regression was extended for a future pre-SM80 run. Actual Windows/MSVC, full training/plugin suites, other GPU architectures, authentic fixtures and whole-model accuracy/performance were not run locally. No new high-precision kernel-accuracy gate is added. New-head CI is reported separately, not claimed green. The earlier Android comparison below is historical, not a new-head size measurement. No new full-team/model-diversity review coverage is claimed.

Historical validation and CI follow-ups (earlier heads only)

Earlier CUDA validation baseline: c4d2e5d38870b8890f9be75facc2f52c0184b355

• CUDA build: passed, four build jobs and NVCC threads=1. Build logs confirm recompilation of both changed C++ translation units.
• Targeted host regressions: 41/41 passed, zero failures/errors/skips, covering MoE and source-replacement policy tests.
• Complete MoE CUDA matrix: passed, one gtest containing 96 fixture combinations / 288 session executions, zero failures/errors/skips, gtest time 584.175 seconds. This is the complete matrix, not the reduced diagnostic fixture.
• Physical GPU 4, H200/sm90, CUDA 13.3. Loader logs verified the task-owned CUDA provider; source cleanliness and ELF hashes were checked.
• The task-local GPU guard's completion-race handling passed 7/7 regression tests. Live-workload ownership, foreign-PID rejection, device identity, and fail-closed malformed/unavailable telemetry checks remain enforced.
• All 20 protected worktrees remained unchanged. The new validation driver cache was removed.
• Changed C++ ranges passed clang-format 18 and git diff --check. Canonical lint could not complete because lintrunner_adapters is missing; no dependencies were installed.

Commands ran with the existing virtual environment activated, from /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release, using task-owned CUDA/cuDNN loader paths. The actual CUDA matrix used physical GPU 4 under the fail-closed resource guard, with 30-second per-query telemetry timeouts and a 30-minute workload budget. Sampled telemetry is not a GPU lease.

cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release --parallel 4 --target onnxruntime_test_all onnxruntime_providers_cuda onnxruntime_provider_test
# Passed.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_test_all --gtest_filter='MoE*:ContribSourceReplacementPolicyTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/moe-fusion-integration-20261109/tf32-gate-c4d2e5d388/host.xml
# Passed: 41 tests, zero skips.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_provider_test --gtest_filter=CudaRuntimeFusionTest.MoESourceFusedDirectTF32OffAndOn --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/moe-fusion-integration-20261109/tf32-gate-c4d2e5d388/cuda.xml
# Passed: one complete matrix gtest, 96 fixture combinations / 288 sessions, zero skips.

Earlier review-fix baseline, not a fresh current-head rerun: d8595ee30d passed 382/382 host regressions and 95/95 provider regressions, explicitly excluding the then-failing MoE numerical matrix. The c4d2e5d baseline above covers that complete matrix under the approved scope. Parent #34022 independently passed its CUDA build, 356 host tests, 95 provider tests, and standalone schema consumer.

Earlier CI compatibility follow-up: 2a14039f224f48e3677f9724ba1ee22d45592187

Normally merge #34022's schema-macro portability fix. Update only the test assertions: preserve strict checks of the seven replacement attributes and absent swiglu_limit, while permitting only the compatible upstream swiglu_limit_after_activation INT default 0. Its presence does not enable clipping; wrong types, nonzero values, or other extra attributes still fail. No kernel, matcher, admission, default-off, or numerical-gate changes.

• Exact new-head CUDA build and host tests: 41/41 passed against the original schema.
• New-default compatibility: 41/41 host tests passed with a task-local schema-only overlay copying exactly the upstream attribute from the captured CI merge ref. The complete CUDA matrix also passed on precursor 08839905fd90ef959b55bd930e57a23aa771f2d6 before the final schema-only follow-up: 96 fixture combinations / 288 source/fused/direct sessions, one gtest, 648.04 seconds, physical GPU 4, zero failures/errors/skips. Loader logs verify the task-owned CUDA provider and artifact hashes stayed unchanged during execution.
• The final follow-up changes only the schema-only helper and standalone consumer. Full-runtime preprocessing of the changed header is identical to the matrix-qualified precursor; the final head's new-default host tests were rerun, not the unchanged CUDA matrix.
• The compatibility overlay imported no upstream kernel changes and was removed after validation. Clean exact-head CUDA artifacts were rebuilt afterward. This is explicit new-default compatibility coverage, not qualification of the entire synthetic CI merge tree.

Exact-head commands, from the existing build-output cwd and loader environment:

timeout --signal=TERM --kill-after=15s 14400s cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release --parallel 4 --target onnxruntime_test_all onnxruntime_providers_cuda onnxruntime_provider_test
# Passed.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_test_all '--gtest_filter=MoE*:ContribSourceReplacementPolicyTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/fusion-ci-20261110T221446/validation-moe-final/host.xml
# Passed.

That follow-up's final-head host validation with only the upstream schema-default overlay:

timeout --signal=TERM --kill-after=15s 14400s cmake --build /mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release --parallel 4 --target onnxruntime_test_all onnxruntime_providers_cuda onnxruntime_provider_test
# Passed.
/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_test_all '--gtest_filter=MoE*:ContribSourceReplacementPolicyTest.*' --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/fusion-ci-20261110T221446/validation-overlay-host-final/host.xml
# Passed.

Complete matrix on runtime-identical precursor 08839905fd90ef959b55bd930e57a23aa771f2d6 with that overlay:

/mnt/data/titaiwang/runtime-fusion/moe-fusion-integration-20261109/cuda/Release/onnxruntime_provider_test --gtest_filter=CudaRuntimeFusionTest.MoESourceFusedDirectTF32OffAndOn --gtest_output=xml:/home/titaiwang/.copilot/session-state/07b6ecde-d659-4b29-aab6-78ba683dbbb3/files/fusion-ci-20261110T221446/validation-overlay/cuda.xml
# Passed.

Changed C++ ranges match clang-format 18 and pass git diff --check. Canonical lintrunner was attempted for the changed files but was then blocked by missing lintrunner_adapters; no dependencies were installed.

Known upstream Android baseline failure: the exact upstream main commit d7ab0d271b0399d889c6b9540cacf04a84b74333 fails the same gate: section total 1,611,530 bytes versus threshold 1,610,778, exceeding it by 752 bytes. #34022's captured earlier CI build reports the same total, on-disk size, and all 27 section sizes. This measured size failure is already present upstream; these fixes do not raise the threshold or claim the gate is green.

Exclusions recorded for that earlier follow-up: new-head full/cross-platform CI, including actual Windows/MSVC, is pending. The diagnosed 8-versus-7 local assertions are corrected and covered under both schema variants, but CI is not claimed all-green. Other GPU architectures, actual pre-SM80 execution, optional authentic fixtures, whole-model accuracy/performance, and earlier fixed-model review gaps remain unqualified. TF32-on cross-kernel accuracy remains outside this integration gate. This follow-up received direct bounded review with no actionable findings, not another whole-team review.

[Copilot]

🟡 Changes recommended

The reported H200 run has 307 failures with TF32 enabled even when MoE fusion is disabled, leaving the new provider test failing on supported hardware until baseline-specific tolerances are validated and the full matrix is rerun.

1 open issue

• Set a validated TF32 error budget for the H200 provider test · New

What changed in this PR

Adds default-off CUDA MoE source fusion to the op-owned declaration framework introduced in #34022, reusing the existing matcher, capability callbacks, and MoE kernel.

Changes:

• Recognizes bounded FP32 dense SwiGLU graphs with conservative shape checks, exact target probing, and an SM80+ hardware gate.
• Adds lazy declaration factories and batched-root matching while preserving learned weight banks.
• Adds host and CUDA tests, documents supported boundaries, and clarifies numerical-semantics guidance.

File	Description
onnxruntime/​test/​providers/​cuda/​cuda_runtime_fusion_test.cc	Adds numerical, placement, and pre-SM80 rejection tests.
onnxruntime/​test/​optimizer/​moe_sources_test.cc	Tests matching, admission, ownership, stale offers, and pattern limits.
onnxruntime/​test/​optimizer/​contrib_source_replacement_policy_test.cc	Validates MoE declaration scope and numerical metadata.
onnxruntime/​core/​providers/​cuda/​cuda_execution_provider.cc	Adds MoE selection with an actual-device SM80+ gate.
onnxruntime/​core/​optimizer/​pattern_rewriter.h	Declares configurable pattern limits and batched matching.
onnxruntime/​core/​optimizer/​pattern_rewriter.cc	Implements bounded larger patterns and scan-local preparation.
onnxruntime/​core/​optimizer/​cuda_runtime_fusion.h	Defines MoE configuration and rule identifiers.
onnxruntime/​core/​optimizer/​cuda_runtime_fusion.cc	Collects factories and materializes declarations after admission.
onnxruntime/​core/​optimizer/​cuda_runtime_fusion_internal.h	Exposes internal factory-based selection.
onnxruntime/​core/​graph/​contrib_ops/​source_replacement.h	Adds the mathematical-semantics numerical profile category.
onnxruntime/​core/​graph/​contrib_ops/​source_replacement_policy.cc	Accepts the new numerical profile category.
onnxruntime/​contrib_ops/​cuda/​fusion/​scaled_silu_sources.cc	Adds a lightweight ScaledSiLU factory.
onnxruntime/​contrib_ops/​cuda/​fusion/​moe_sources.h	Declares the fixed-configuration structural test helper.
onnxruntime/​contrib_ops/​cuda/​fusion/​moe_sources.cc	Implements MoE patterns, guards, snapshots, probing, and replacement.
onnxruntime/​contrib_ops/​cuda/​fusion/​cuda_source_replacements.h	Declares MoE sources and shared factory metadata.
onnxruntime/​contrib_ops/​cuda/​fusion/​causal_conv_with_state_sources.cc	Adds a lightweight causal-convolution factory.
docs/​design/​Extensible_Runtime_Fusion.md	Extends the declaration design with bounded MoE support.
docs/​Cuda_Runtime_Fusion.md	Documents MoE configuration, eligibility, and validation boundaries.
cmake/​onnxruntime_optimizer.cmake	Includes MoE host companions in the optimizer build.
AGENTS.md	Adds operator and fusion numerical-semantics guidance.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

The MoE rewrite can admit shapes whose replacement exceeds CUDA’s grid-Y limit and fails at execution.

6 open issues

• Set a validated TF32 error budget for the H200 provider test
• Document optimizer settings needed to preserve MoE topology · New
• Cap admitted N to avoid exceeding CUDA grid-Y limit · New
• Filter candidate roots before full pattern matching · New
• Document factory registration and lazy rule materialization · New
• Update pattern reference for enlarged limits and batched scans · New

_{Review effort: Max}

[Copilot]

🟡 Changes recommended

Accepted large MoE dimensions can underallocate the FC1 buffer by 448 bytes due to float-based workspace sizing, allowing gated activation to corrupt adjacent workspace and routing scales.

3 open issues

• Prevent workspace sizing overflow for large expanded sizes · New
• Validate fusion setting before skipping unsupported hardware · New
• Split the CUDA matrix into independently runnable parameterized tests · New

6 resolved since last review

• Set a validated TF32 error budget for the H200 provider test
• Filter candidate roots before full pattern matching
• Cap admitted N to avoid exceeding CUDA grid-Y limit
• Document optimizer settings needed to preserve MoE topology
• Update pattern reference for enlarged limits and batched scans
• Document factory registration and lazy rule materialization

_{Review effort: Max}