[Kotomi-Du]

Description

For weights [64, N, K] and zero points [64, N, 1], the old code incorrectly read N as the group count but accidentally selected a grouped mask that preserved per-expert zero points. After the PR #38697 correctly identified one group along K, the existing rank check selected a per-channel mask that ignored the expert axis, causing all experts to reuse the first expert’s zero points. The fix explicitly checks whether weights have a batch/expert dimension and retains the grouped mask for 3D weights, including during cache loading.

Stage	IFM-group axis read	ngroups	Zero-point mask	Expert handling
Before PR #38697	1 (incorrect)	N (incorrect)	7, grouped	Preserved by accident
After PR #38697, before fix	2 (correct)	1 (correct)	4, per-channel	Expert axis ignored
After current fix	2 (correct)	1 (correct)	7, grouped	Expert axis preserved

Details

Affected topology:

• Weights: [64, N, K], with transpose_b = true.
• Zero points: [64, N, 1], with transpose_b_zp = true.
• Assume N > 1 and K > 1, so the weights' non-unit dimension count is 3.

Before the PR #38697: correct expert handling by accident

The old code chose the zero-point IFM axis using the number of non-unit zero-point dimensions:

// Zero-point shape: [64, N, 1]
dzp_rank = count_if(dims > 1);             // 2, because the trailing 1 is ignored
dzp_ifm_dim_idx = weights_transposed
                     ? dzp_rank - 1
                     : dzp_rank - 2;     // 1
ngroups = dzp_layout.get_dim(1);          // N, not the actual group count of 1

Because ngroups != 1, the code selected the grouped branch:

attr->set_zero_points(DNNL_ARG_WEIGHTS, grouped, {group_size, 1}, dzp_data_type);

For a 3D FC, grouped = (1 << 3) - 1 = 7. This mask includes the expert dimension, so different experts retain their own zero points. The observed case worked, but the axis selection and inferred group count were wrong.

After the PR #38697, before the fix: correct axis exposes the mask-selection bug

The new orientation-aware helper correctly identifies the IFM-group axis:

get_ifm_dim_idx(dzp_layout, true, 3);  // 2
ngroups = dzp_layout.get_dim(2);      // 1

However, the per-channel condition still used the non-unit zero-point rank:

// dzp_rank is still 2 for [64, N, 1].
if (ngroups == 1 && dzp_rank <= 2) {
    attr->set_zero_points(DNNL_ARG_WEIGHTS, per_oc, {}, dzp_data_type);
}

It didn't land to the condition which has group attribute.

After the fix in current PR

Mask selection now explicitly checks whether the weights have a batch/expert dimension:

const bool has_batch_dim = weight_rank > 2;
if (ngroups == 1 && !has_batch_dim) {
    attr->set_zero_points(DNNL_ARG_WEIGHTS, per_oc, {}, dzp_data_type);
} else {
    attr->set_zero_points(DNNL_ARG_WEIGHTS, grouped, {group_size, 1}, dzp_data_type);
}

For [64, N, K], has_batch_dim is true. Even with ngroups == 1, the code keeps the grouped mask 7 and preserves per-expert zero points. Ordinary 2D per-channel weights still use the per-channel branch.

The same batch-dimension check is applied to scale and zero-point attributes during cache loading, keeping grouped handling consistent with fresh compilation.

Ticket

• CVS-198171

AI Assistance:

• Yes
• Used AI to find the root cause; engineer manually build OV- GenAI to validate the fix

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.
Some pipeline(s) encountered errors during trigger evaluation.

[Copilot]

🟡 Changes recommended

The new mask selection expects 1,024 zero points from a 16-value buffer for shared-expert inputs, risking invalid reads or incorrect results.

1 open issue

• Select zero-point broadcast mask for shared transposed weights · New

What changed in this PR

This PR addresses a GPU oneDNN compressed fully connected regression exposed by #38697, preserving per-expert quantization parameters for 3D weights with one group along K.

Changes:

• Uses the weights’ batch dimension to retain grouped zero-point handling.
• Applies batch-aware scale and zero-point mask selection during cache loading.
• Adds regression tests covering expert counts, parameter orientations, caching, and group counts.

File	Description
src/​plugins/​intel_gpu/​tests/​unit/​test_cases/​fully_connected_gpu_test.cpp	Adds parameterized per-expert zero-point accuracy tests.
src/​plugins/​intel_gpu/​src/​graph/​impls/​onednn/​fully_connected_onednn.cpp	Updates quantization mask selection for compilation and cache loading.

_{Review effort: Balanced}

[andrew-k-park]

Please fix the conflicts with master.

[andrew-k-park]

I reviewed the PR and also tested it on A770, the same GPU as CI GPU_ARC_RPL. I built the GPU plugin from your PR head, loaded it into the private-build wheels of the PR base (8f0a334), and ran the deepseek-ocr-2 INT8_ASYM model with the CI WWB command:

• WWB similarity: 0.712 on the base, 0.893 with your PR (WW42: 0.885, CPU: 0.897).

LGTM

[e-ddykim]

Please check these regressions --> e-ddykim@f51e408

[Kotomi-Du]

Please check these regressions --> e-ddykim@f51e408

Thanks for the review. I saw the same concern raised by Copilot review as below.

However, I don't think we need to take it because there is no such real case for now. The previous code never take care such case before as well. So, I don't think it is a regression introduced by this PR. Here are additional two concerns:

1. sometimes a small change of this logic to adapt to new case may bring another regression. It's a learning in [GPU] Refactor for FC Compressed  #38697 where we introduced logic to treat new broadcast case but it introduces regression then [GPU] remove defensive handle about oc-broadcast #38934.
2. technically, as zero points shared across experts, we should expect frontend（transformation) to broadcast across experts first. Otherwise, it may come to the same concern I raised in this PR [GPU] Fix OC-broadcast FC decompression masks #39106

In general, I am more conservative and tend to not add new case coverage to complicate the code. If you still think it is a good approach to go, I can take care in the next PR to cover new case you posted. I suggest not to put in this PR.

[Kotomi-Du]

build_jenkins

[e-ddykim]

@Kotomi-Du The reason I called this a regression is that the test passes on the current master branch but fails when this PR is applied. However, if you're confident that this scenario cannot occur in practice, I don't think we need to take any additional action here.

[Kotomi-Du]

@Kotomi-Du The reason I called this a regression is that the test passes on the current master branch but fails when this PR is applied. However, if you're confident that this scenario cannot occur in practice, I don't think we need to take any additional action here.

understood, I guess the regression track is as below, so basically my refactor introduces regressions of MOE model while make it pass for zp share across expert case by accident.

Stage	zp share across expert case
Before PR #38697	failed
After PR #38697, before fix	pass
After current fix	failed