Repository navigation

[CUDA] Speed up native Bolt packed projections and IQ2 experts - #34002

Merged

Tianlei Wu (tianleiwu) merged 13 commits into

rui-ren/bolt-standalone-router-on-33969-20261108from

rui-ren/bolt-packed-latency-on-33994-20261109

Nov 12, 2026

Merged

[CUDA] Speed up native Bolt packed projections and IQ2 experts#34002
Tianlei Wu (tianleiwu) merged 13 commits into
rui-ren/bolt-standalone-router-on-33969-20261108from
rui-ren/bolt-packed-latency-on-33994-20261109

Commits on Nov 12, 2026
- Vectorize NVFP4 projection group decoding without changing BF16 arithmetic
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  558e317
- Pack NVFP4 BF16 projection operands with qualified aligned loads
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  eb00be9
- Improve IQ2 expert fragment reconstruction without changing arithmetic
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  9c44ad0
- Pack exact IQ2 expert magnitudes without runtime integer conversion
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  38e00a3
- Stage NVFP4 half-chunk loads for large-N BF16 projections
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  25c749e
- Load Q8 packed operands through aligned vector windows
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  a5575bd
- Optimize dense IQ2 BF16 operand reconstruction
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  888843d
- Align packed BF16 regressions with the public scale contract
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  88d7b40
- Reduce IQ2 expert-up operand reconstruction overhead
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  d101263
- Share IQ2 decoding and harden packed CUDA regression coverage
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  07dd477
- Address NVFP4 architecture and regression-test review feedback
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  4bfd567
- Migrate IQ2 operand oracle to full-score consumer inputs
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  8f53dc5
- Fix full-score routing in the IQ2 alignment regression
  
  Rui Ren (rui-ren)
  and
  Copilot
  committed
  beb3482

Commits

Commits on Nov 12, 2026