[Rui Ren (rui-ren)]

Scope and dependency

Extend native merge stack #33967 above #33994. This is not the alternative
operator-consolidation PR #33940, and does not replace the existing stack.
The existing five PRs and their source heads are unchanged.

Parent: #33994 at a8ff5642f5c85519561964928006413a4fde7859.
Candidate: 1b5063c4af651e12bf955bc978c9312e2734a9f9.

Bring the nine previously isolated native optimization commits onto the actual
merge-stack tip, with six changed files. Preserve MatMulPacked, MoEPacked,
their packed-weight layouts and precision options, Mamba2Verify, and the
standalone/integrated router introduced by #33994. No graph conversion,
requantization, schema addition, GenAI inference change, or acceptance tuning.

Kernel changes

• NVFP4: vectorized group decoding, paired BF16 operand construction and
  half-chunk staging for eligible large-N Hopper projections, with supported
  offset/stride fallbacks retained.
• Q8: aligned packed-record windows while retaining scale precision and the
  fallback for unsupported alignment.
• Dense IQ2: reconstruct complete BF16 operands without changing the existing
  contiguous K partitions and cancellation/reduction order.
• IQ2 experts: exact magnitude-codebook reconstruction and launch-selected
  aligned FC1 activation loads. Preserve FP32 scale reconstruction, sign before
  BF16 rounding, FC1/FC2 warp partitions, squared-ReLU boundaries and FC2 loading.
• Use [CUDA] Extract Bolt projected-sigmoid routing into MoERouterPacked #33994's shared LoadPacked8 fallback after its router extraction rather
  than restoring the removed private router helper.

All nine commits retain provenance, signoffs and coauthor trailers. They were
ported without merging or rewriting the existing stack.

Actual-stack H100 throughput

This is the actual stack-tip comparison, not the earlier private-branch
versus #33940 comparison. That distinction matters: the remote stack tip did
not yet contain the optimized native kernels.

Input tokens	Stack ordinary tok/s	Candidate ordinary tok/s	Stack k=7 tok/s	Candidate k=7 tok/s
178-254	54.36	92.42	223.84	355.42
427-932	54.20	92.08	185.24	293.62
1028-1663	54.25	92.04	201.48	318.91

Approximately 70% ordinary and 58-59% k=7 improvement in this controlled
comparison. This does not claim that all remaining latency is resolved.
The separate #33940 equivalent-package result of approximately 109 ordinary /
370 k=7 tok/s on the longest bin is not a measurement of this candidate;
transferring that remaining gain is still outstanding.

Protocol: original frozen 30 real prompts, ten per bin, unchanged warmups and
per-prompt mode order, natural EOS/512-new-token cap, two reversed-order passes,
same physical H100 per baseline/candidate shard. Eight logical shards ran in
waves over GPUs 1,3,4,5,6,7. All admitted runs passed the foreign-client and
resource guards. No profiler ran in the timing window; clocks were not locked.
Rates pool decoded tokens over elapsed decode time, excluding the first token,
not arithmetic means of per-request TPS.

Correctness and build validation

• Exact same native model package, packed bytes and GenAI libraries in both arms.
  Graph hashes match the preserved source-value audit; full package-file hashes
  were recorded again. No exported checkpoint/package was modified.
• 22 native CUDA regressions passed, zero skips, covering projections,
  complete operand rounding/cancellation, IQ2 production widths/codebook scales,
  routing and the standalone router. Changed C++ files passed CLANGFORMAT;
  git whitespace checks passed.
• 1,782 complete live tensor/state hashes per arm, bitwise equal and finite
  in ordinary and k=7 modes, also equal to the preserved quantized control.
• Independent full LCB145 ordinary/k=7 probes passed all 512 generated tokens
  and complete counters.
• The accepted timing run passed 240 complete measured generation sequences
  against frozen ordinary-control tokens. Within-mode complete speculative
  counter dictionaries matched between arms and repeats.
• Isolated incremental rebuild on the archived clean [CUDA] Extract Bolt projected-sigmoid routing into MoERouterPacked #33994 build; every
  affected compilation unit/header dependent was rebuilt and the provider/test
  binary relinked with the production flags. Untouched objects were reused:
  this is not a new clean full-source build.
• CUDA 12.8.93, driver 580.159.04, H100 80GB. Common corrected GenAI source:
  c0fd01914265d5157d6da1376c38b4df521a76c5
  (microsoft/onnxruntime-genai#2916).
  Both arms use that same source; its correctness fix is not credited with the
  kernel speedup.

Accepted statistics SHA256:
0b9b7e2385b2dd9d7e5f7d459c65ef74338b5f707d1536454e7e27fb9a407c0f.

Remaining qualification

Keep draft. No independent HF/benchmark-suite accuracy rerun, longer-context
qualification, other-GPU-architecture qualification, full-CI-green or merge-ready
claim. This validates the pinned #33994 lineage plus this patch, not an
unmeasured rebase onto the subsequently advanced lower #33953 head.
The original packages, earlier evidence, other PRs and native stack membership
are preserved; only this new follow-up is intended for append.