Skip to content

[GPU] remove defensive handle about oc-broadcast - #38934

Merged
clee30 merged 3 commits into
openvinotoolkit:masterfrom
Kotomi-Du:remove_oc_broadcast_handle
Oct 22, 2026
Merged

clee30 merged 3 commits into
openvinotoolkit:masterfrom
Kotomi-Du:remove_oc_broadcast_handle

Conversation

@Kotomi-Du

Copy link
Copy Markdown
Contributor

Details:

  • [GPU] Refactor for FC Compressed  #38697 added handling for an OC-broadcast decompression scale/zero-point ([1, G] / [G, 1]) in the oneDNN FullyConnected implementation: get_grouped_mask() clears the OC bit from the grouped scale/zp mask whenever the OC dim of the scale/zp is 1.
  • That check didn't consider whether the weights' N is also 1, in which case the scale isn't really broadcast. For example, u4 weights [1, 2048] with a transposed scale [1, 16] (group size 128) lost the OC bit; for a 2D FC that leaves mask = 1, K only. oneDNN's gemm doesn't support that mask, so oneDNN falls back to the reference ocl:ref:any matmul, a performance regression for this dynamic-shape FC.
  • On re-evaluation, the OC-broadcast handling is overly defensive. The standard weight-compression flow (NNCF/optimum) always produces per-OC scale/zp [N, G, 1], which ConvertFullyConnectedToFullyConnectedCompressed reshapes to [N, G], so the scale/zp OC dim always matches the weights' N.
  • This PR removes get_grouped_mask() and restores the full grouped mask for grouped scale/zp in both create() and load(), i.e. the behavior before [GPU] Refactor for FC Compressed  #38697. It also removes the compressed_int8_broadcast_scale_zp unit tests, which covered only the removed path.

Tickets:

AI Assistance:

  • AI assistance used: yes
  • Used AI to trace oneDNN matmul implementation selection (gemm_t vs ref_t), analyze the scale/zp mask computed by get_grouped_mask(), check where scale/zp shapes come from in the compression pipeline, and draft this description.

@Kotomi-Du
Kotomi-Du requested review from a team as code owners October 17, 2026 04:43
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Oct 17, 2026
@Kotomi-Du Kotomi-Du changed the title remove defensive handle about oc-broadcast [GPU] remove defensive handle about oc-broadcast Oct 17, 2026

@sungeunk sungeunk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Fixed perf issue on B70

@clee30
clee30 added this pull request to the merge queue Oct 22, 2026
@p-durandin p-durandin added this to the 2026.5 milestone Oct 22, 2026
Merged via the queue into openvinotoolkit:master with commit ee2e4b5 Oct 22, 2026
192 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants