[e-ddykim]

Details:

• Follow-up to [GPU] remove defensive handle about oc-broadcast #38934 to fix incorrect results when grouped weight decompression scales and zero points are broadcast across output channels (N > 1), while preserving the valid N=1 grouped-mask correction.
  • Restore the OC-broadcast mask for grouped scales and zero points in both oneDNN FC creation and cache loading.
  • Determine weights OC from the logical weights rank and orientation used by the matmul descriptor.
  • Retain the full grouped mask for single-OC weights, preserving the N=1 GEMM selection.
  • Restore and extend regression coverage for parameter orientations, rank-2/rank-3 inputs, fresh/imported networks, per-OC controls, and dynamic U4 N=1 cases.
  • Add public Parameter-weight graph conversion tests for both weight orientations.

Tickets:

• 197314

AI Assistance:

• AI assistance used: yes

[Kotomi-Du]

@e-ddykim may I ask why we need to address this? do we really have such case in real scenario? The standard weight-compression flow (NNCF/optimum) always produces per-OC scale/zp [N, G, 1], which ConvertFullyConnectedToFullyConnectedCompressed reshapes to [N, G], so the scale/zp OC dim always matches the weights' N.

[sungeunk]

LGTM

[e-ddykim]

@e-ddykim may I ask why we need to address this? do we really have such case in real scenario? The standard weight-compression flow (NNCF/optimum) always produces per-OC scale/zp [N, G, 1], which ConvertFullyConnectedToFullyConnectedCompressed reshapes to [N, G], so the scale/zp OC dim always matches the weights' N.

@Kotomi-Du The reshape preserves the scale/ZP tensor’s OC dimension; it does not expand it to match the weights’ N. Therefore, [N,G,1] becomes [N,G], but [1,G,1] becomes [1,G] even when the weights have N > 1. OC broadcasting remains.

[Kotomi-Du]

@e-ddykim may I ask why we need to address this? do we really have such case in real scenario? The standard weight-compression flow (NNCF/optimum) always produces per-OC scale/zp [N, G, 1], which ConvertFullyConnectedToFullyConnectedCompressed reshapes to [N, G], so the scale/zp OC dim always matches the weights' N.

@Kotomi-Du The reshape preserves the scale/ZP tensor’s OC dimension; it does not expand it to match the weights’ N. Therefore, [N,G,1] becomes [N,G], but [1,G,1] becomes [1,G] even when the weights have N > 1. OC broadcasting remains.

do we really have such real scenario? What I observed is oneDNN doesn't have such optimized kernel and will use ocl:ref:any which is very slow. Based on this observation, shall we address the broadcast in transformation stage to get better performance?