[imargulis]

What does this PR do?

Adds export of DSpark drafts, DFlash drafts with a Markov head that adjusts each drafted token's logits based on the previous token. Until now, drafts with a Markov head (markov_rank > 0) were rejected.

The draft backbone is exported as for DFlash. The Markov head is exported next to it as one model, openvino_markov_head_model.xml, that holds two computations over the same weights: a dense Markov step and a Top-K candidate scorer. OpenVINO GenAI selects one of them when it loads the draft, so the export has no DSpark-specific options. Only the vanilla Markov head is supported for now. It is never weight-compressed, so it keeps its original precision unless --weight-format fp16/fp32 is given.

Runtime support: openvinotoolkit/openvino.genai#4660. The draft backbone follows the current DFlash draft contract, which is planned to change with continuous batching support for DFlash-family drafts. The Markov head model is built for compatibility with continuous batching already: the candidate scorer takes the draft blocks of several requests packed back to back.

Also:

• MiniCPM-V 4.7 language models get the hidden-state metadata needed to serve as targets for DFlash-family drafts.
• DFlash and DSpark drafts are exported without a tokenizer, since they run with the target's tokenizer.

How to test

Environment (Python 3.11+). Until openvinotoolkit/openvino.genai#4660 is merged, OpenVINO GenAI is built from its branch, which needs CMake 3.26+ and a C++ compiler:

python -m venv dspark-env
source dspark-env/bin/activate  # Windows: dspark-env\Scripts\activate
python -m pip install --upgrade pip

# optimum-intel from this PR. Loading the draft through remote code needs diffusers and einops.
MiniCPM-V 4.7 needs transformers>=5.18
pip install "optimum-intel[openvino] @ git+https://github.com/imargulis/optimum-intel.git@dspark-preview"
pip install "transformers>=5.18"

# OpenVINO and OpenVINO Tokenizers nightly, and OpenVINO GenAI with DSpark support
pip install --pre -U openvino openvino-tokenizers --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install --pre "openvino-genai @ git+https://github.com/imargulis/openvino.genai.git@dspark-preview" --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly

Export the target and the DSpark draft:

optimum-cli export openvino --model openbmb/MiniCPM-V-4.7-1B --task image-text-to-text --weight-format int8 minicpm-v-4.7-1b-int8-ov
optimum-cli export openvino --model openbmb/MiniCPM-V-4.7-1B-DSpark --task text-generation-with-past --trust-remote-code minicpm-v-4.7-1b-dspark-ov

Once #1983 is merged, the draft export needs neither option for a Hub model ID: DFlash and DSpark drafts are then loaded directly instead of through remote code, and their task is inferred. A local checkpoint folder still needs --task text-generation-with-past, because the task cannot be inferred from a local folder.

optimum-cli export openvino --model openbmb/MiniCPM-V-4.7-1B-DSpark minicpm-v-4.7-1b-dspark-ov

Run with OpenVINO GenAI:

import openvino_genai as ov_genai

# MiniCPM-V 4.7 has linear attention: the scheduler needs at least num_assistant_tokens + 2 linear-attention blocks.
scheduler_config = ov_genai.SchedulerConfig()
scheduler_config.num_linear_attention_blocks = 17

# Top-64 candidate scorer by default; markov_head_top_k=16 changes K, markov_head_top_k=0 selects the dense Markov walk.
draft = ov_genai.draft_model("minicpm-v-4.7-1b-dspark-ov", "GPU")
pipe = ov_genai.VLMPipeline("minicpm-v-4.7-1b-int8-ov", "GPU", draft_model=draft, scheduler_config=scheduler_config)

config = ov_genai.GenerationConfig()
config.max_new_tokens = 256
config.num_assistant_tokens = 15

result = pipe.generate("Write a Python function that checks whether a number is prime.", generation_config=config)
print(result.texts[0])
print("accepted tokens:", result.extended_perf_metrics.get_num_accepted_tokens())

On GPU, the Top-K candidate scorer needs openvinotoolkit/openvino#38845 to be fast. With the current nightly, its TopK over the vocabulary falls back to a generic GPU kernel, and the draft is slower than the target alone.

Tests

Validated end to end with OpenVINO GenAI on MiniCPM-V-4.7-1B and its DSpark draft: with both computations, the outputs and accepted tokens are identical to those of the previous revision's separate Markov head models, which were also validated with MiniCPM-V-4.7-35B-A3B-DSpark. Unit tests use a tiny generated DSpark checkpoint and cover numerical parity of both computations, packed draft blocks of different lengths, changing K, metadata, precision, CLI export, and drafts without a Markov head still exporting as plain DFlash drafts. A GenAI test runs the tiny DSpark draft with a tiny Qwen3 target through the continuous batching pipeline and checks that greedy outputs match the target alone; the preview models workflow runs the DSpark tests against the GenAI nightly.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[Copilot]

🟡 Changes recommended

CLI dtype overrides break checkpoint-precision preservation, and the fixture cache can reuse stale artifacts.

2 open findings

• Preserve BF16 Markov constants through CLI loading · New
• Validate cached fixtures before reuse · New

What changed in this PR

Adds OpenVINO export support for DSpark drafts with vanilla Markov heads and MiniCPM-V 4.7 targets.

Changes:

• Exports dense or Top-K Markov-head sidecar models.
• Adds DSpark validation, metadata, CLI options, and precision handling.
• Adds extensive export, parity, compression, and CI coverage.

File	Description
optimum/​exporters/​openvino/​dflash_utils.py	Defines DSpark configuration validation and interfaces.
optimum/​exporters/​openvino/​model_patcher.py	Implements the DSpark backbone and Markov exporters.
optimum/​exporters/​openvino/​model_configs.py	Adds Markov model export contracts.
optimum/​exporters/​openvino/​input_generators.py	Generates Markov export inputs.
optimum/​exporters/​openvino/​convert.py	Splits and annotates DSpark exports.
optimum/​exporters/​openvino/​base.py	Adds precision-preservation configuration.
optimum/​exporters/​openvino/​__main__.py	Integrates DSpark export and tokenizer handling.
optimum/​commands/​export/​openvino.py	Adds --dspark-top-k.
tests/​openvino/​utils_tests.py	Creates tiny DSpark fixtures.
tests/​openvino/​test_dflash.py	Tests parity, metadata, and precision.
tests/​openvino/​test_exporters_cli.py	Tests CLI packages and tokenizer omission.
docs/​source/​openvino/​models.mdx	Documents supported DSpark checkpoints.
.github/​workflows/​test_openvino_preview_models.yml	Adds MiniCPM-V metadata validation.

🧠 Review effort: Balanced

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[rkazants]

Please also provide proper PR descriprion with full standalone code snippets for models export, inference using Openvino GenAI. How to set up environment to test it

[rkazants]

btw, did you cherry-pick my changes for CB support?

[imargulis]

Sure, will do so when GenAI PR is also ready

[imargulis]

No. It’s a SDPA-only version that is subject to change together with DFlash2 support which is expected to come with CB Note that the backbone is a pure DFlash.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[imargulis]

@rkazants

Two follow-ups for after this PR and #1983 merge.

1. Rename Eagle3DummyGenerator. It now generates the inputs of Eagle-3, DFlash and DSpark drafts, so the name is misleading. What these drafts share is that they are grafted onto the target model: they read its hidden states. I propose GraftedDraftDummyGenerator and GraftedDraftVLMDummyGenerator (for Eagle3VLMDummyGenerator), with the term defined in the docstring. It touches several model configs, so I'd keep it out of this PR.

2. DSpark drafts with a pruned vocabulary. Several published DSpark drafts (which is also true for some DFlash drafts) score only part of the target vocabulary, as Eagle-3 drafts do:

Draft	Draft vocabulary	Target vocabulary
RedHatAI/Qwen3.6-35B-A3B-speculator.dspark	32,000	248,320
novita/kimi-k2.6-dspark	32,000	163,840
mpashkovskii/dspark-qwen3-8b	32,000	151,936
Qwen3.6-27B-DSpark-49k	49,167	248,320

This PR doesn't support them. It supports the DeepSpec format of the MiniCPM-V 4.7 drafts, which reuse the target's lm_head and whose Markov tables cover the full vocabulary (their draft_vocab_size equals vocab_size). The first three drafts above are also in the speculators format, which differs beyond the vocabulary:

• The config has no top-level model_type. Its auto_map points to a config.py that imports the speculators package, the Qwen3 settings are nested in transformer_layer_config, and the target layers are listed in aux_hidden_state_layer_ids. The architecture is DSparkDraftModel (novita, mpashkovskii) or Qwen3DSparkModel (RedHat, the same name as the DeepSpec format). All these can be normalized.
• The weights include the draft's own lm_head [32000, H], d2t [32000] (draft id to target id, as in Eagle-3), t2d [V], a full copy of embed_tokens [V, H], and a confidence head [1, H + 256]. The Markov head is split between the two vocabularies: markov_w1 [V, 256] is looked up by the previous target token, and markov_w2 [32000, 256] gives a bias over the draft vocabulary.

With this PR, these checkpoints fail when loaded (missing model_type, unknown architecture, or a markov_w2 shape mismatch) rather than producing a draft that proposes wrong tokens.

Supporting them changes the draft contract with GenAI: the draft applies its own lm_head, the Markov head's Top-K runs over the draft vocabulary, and the ids it returns are mapped to target ids through d2t. Eagle-3 already does this mapping in both repositories: the export outputs d2t, and GenAI maps the draft's ids. It is worth doing: for Qwen3.6 the output layer and the Top-K shrink by about 7.75x. speculators' DSpark config extends its DFlash config, so speculators-format DFlash drafts probably share this layout, and the two are best designed together with continuous batching support.

Not sure who's the decision maker whether the above should be supported.