[imargulis]

What does this PR do?

Extends the PagedAttention export of DFlash drafts to the rest of the DFlash family, so that every DFlash-family draft runs in OpenVINO GenAI continuous batching from one exported IR: DFlash, DFlash v2, and DeepSpec DSpark drafts with or without a Markov head, including the MiniCPM-V 4.7 DSpark drafts.

Changes to the export:

• DFlash v2 (DFlash2DraftModel, e.g. incoai/Qwen3.8-27B-DFlash2):
  • The draft uses the per-row backbone of the DFlash drafts. Its grouped dynamic convolution shifts the taps along the packed block rows and masks them to the block of the row they produce, so a tap never reaches another sequence's block after the PagedAttention conversion.
  • The candidate selector is exported as the draft's Markov head, openvino_markov_head_model.xml with markov_head_type=dflash2. It is the packed Top-K lattice of the DSpark Markov head, with transitions gated by the draft hidden states and no dense step. Like the DSpark Markov head, it is never weight-compressed and keeps the checkpoint precision unless --weight-format fp16/fp32 is given.
  • The backbone gets dflash/version=2 and dflash/gpu_min_activations_scale_factor=32 instead of the decoder default ACTIVATIONS_SCALE_FACTOR=8. OpenVINO GenAI compiles the draft on GPU with at least that scale factor.
• DSpark: to the exporter, DSpark drafts are DFlash v1 drafts, with the candidate position offset (0) and the Markov head taken from their config. The Markov head model drops the dflash/interface tag: markov_head_type names the head, and runtimes find its computations and read their results by port name.
• Block rows: the Qwen3 and Muse-Glimmer drafts write them with index_put_ instead of item assignment. Item assignment strips the leading unit dimension of a one-row value, so when the dummy token_type_ids hold a single block row, the trace freezes that shape and the exported draft fails on any other block length.
• Sliding layers: the PyTorch draft keeps the whole past in its cache, as the exported graph does, and applies the window through its attention mask. DynamicCache(config=...) kept only the last window of keys, so the second cached step built its mask over more keys than the cache held.
• CLI: DFlash v2 drafts are loaded natively like the other DFlash drafts, and the task is now also inferred from a local draft checkpoint, so --task is no longer needed.
• Docs: Qwen3.8-27B-DFlash2 in the supported architectures, and the files a DFlash v2 export produces.

Runtime support: openvinotoolkit/openvino.genai#4847. On GPU, the paged draft needs PagedAttention with token_type_ids in the mixed stage and the Markov head needs the radix TopK, both in the OpenVINO nightly from 2026.5.0.dev20261028.

How to test

Environment (Python 3.11+). Until openvinotoolkit/openvino.genai#4847 is merged, OpenVINO GenAI is built from its branch, which needs CMake 3.26+ and a C++ compiler:

python -m venv dflash-env
source dflash-env/bin/activate  # Windows: dflash-env\Scripts\activate
python -m pip install --upgrade pip

# optimum-intel. MiniCPM-V 4.7 targets also need transformers>=5.18.
pip install "optimum-intel[openvino] @ git+https://github.com/huggingface/optimum-intel.git"

# OpenVINO and OpenVINO Tokenizers nightly, and OpenVINO GenAI with DFlash continuous batching
pip install --pre -U openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install --pre "openvino-genai @ git+https://github.com/imargulis/openvino.genai.git@dflash-exp-markov" --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

Export a DSpark draft and its target. Drafts need neither --task nor --trust-remote-code:

optimum-cli export openvino --model Qwen/Qwen3-4B --weight-format int8 Qwen3-4B-int8-ov
optimum-cli export openvino --model deepseek-ai/dspark_qwen3_4b_block7 dspark_qwen3_4b_block7-ov

Run with OpenVINO GenAI:

import openvino_genai as ov_genai

draft = ov_genai.draft_model("dspark_qwen3_4b_block7-ov", "GPU")
pipe = ov_genai.LLMPipeline("Qwen3-4B-int8-ov", "GPU", draft_model=draft)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 256
config.num_assistant_tokens = 7

result = pipe.generate(["Write a Python function that checks whether a number is prime."], generation_config=config)
print(result.texts[0])
print("accepted tokens:", result.extended_perf_metrics.get_num_accepted_tokens())

The other drafts export the same way:

# DFlash
optimum-cli export openvino --model z-lab/Qwen3.6-27B-DFlash --weight-format int4 Qwen3.6-27B-DFlash-ov
# DFlash v2
optimum-cli export openvino --model incoai/Qwen3.8-27B-DFlash2 --weight-format int4 Qwen3.8-27B-DFlash2-ov
# DeepSpec DFlash draft, a DSpark draft without a Markov head
optimum-cli export openvino --model deepseek-ai/dflash_qwen3_4b_block7 dflash_qwen3_4b_block7-ov
# MiniCPM-V 4.7 DSpark
optimum-cli export openvino --model openbmb/MiniCPM-V-4.7-1B-DSpark MiniCPM-V-4.7-1B-DSpark-ov

The Qwen3.6, Qwen3.8 and MiniCPM-V 4.7 targets are multimodal and have linear attention: they run with VLMPipeline.

Validation

Intel Core Ultra X7 358H, Arc B390 GPU.
HumanEval, greedy; the 27B and 35B targets 5 prompts x 256 new tokens, the others 20 prompts x 2048.

Draft	Target	num_assistant_tokens	Target alone, tok/s	With draft, tok/s	TPOT speedup	Tokens per step
z-lab/Qwen3.6-27B-DFlash, int4	Qwen3.6-27B, int4	15	7.13	58.74	8.01x	12.12
incoai/Qwen3.8-27B-DFlash2, int4	Qwen3.8-27B, int4	7	7.53	41.49	5.50x	7.40
deepseek-ai/dspark_qwen3_4b_block7	Qwen3-4B, int8	7	25.32	94.07	3.69x	5.48
deepseek-ai/dflash_qwen3_4b_block7 (no Markov head)	Qwen3-4B, int8	7	25.29	87.48	3.43x	5.08
openbmb/MiniCPM-V-4.7-1B-DSpark	MiniCPM-V-4.7-1B, int8	15	94.03	178.32	1.88x	3.98
openbmb/MiniCPM-V-4.7-35B-A3B-DSpark	MiniCPM-V-4.7-35B-A3B, int4	15	40.41	113.38	2.65x	9.90

Tests

The DFlash v2 tests use a new tiny draft, optimum-intel-internal-testing/tiny-random-qwen3-dflash2, a draft for tiny-random-qwen3 laid out like the published DFlash v2 checkpoints: block_size in a nested dflash_config, grouped-query attention with one KV head, a sliding and a full-attention layer, random base kernels that reach the previous rows, and non-default output_multiplier, final_logit_softcapping and input_embedding_scale. They cover the checkpoint keys, config normalization and validation, the grouped dynamic convolution against a reference implementation, and the exported package: the rt_info of both IRs and the Markov head's Top-K lattice on packed blocks against a reference formula, also from a bf16 checkpoint.

For DFlash and DFlash v2, the draft IR converted to PagedAttention matches the PyTorch draft over several steps of sequences packed into one call, and each input is read only in its own rows. A draft without a Markov head (DFlash, DeepSpec DFlash) still exports a single IR. The CLI test of the Markov head package runs on the DSpark and the DFlash v2 drafts with default, INT8, INT4 and FP16 weights, and the task is inferred for every draft architecture.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.