fix(structured_output): fall back to prompt-based schema when the native path cannot consume it

[Harsh23Kashyap]

Summary

Fixes #40907 — when a model plugin's YAML declares structured-output in its features list but the provider plugin does not actually consume the JSON schema, the schema was silently dropped from the request and the response came back unconstrained. The fix detects this case at the native-vs-prompt branch and falls through to the prompt-based schema injection path, which already works correctly for models that do not declare the feature.

Root cause

api/core/llm_generator/output_parser/structured_output.py:125-128 (pre-fix):

if model_schema.support_structure_output:
    model_parameters = _handle_native_json_schema(
        provider, model_schema, json_schema, model_parameters_with_json_schema, model_schema.parameter_rules
    )
else:
    _set_response_format(model_parameters_with_json_schema, model_schema.parameter_rules)
    prompt_messages = _handle_prompt_based_schema(
        prompt_messages=prompt_messages,
        structured_output_schema=json_schema,
    )

The branch decision is taken on support_structure_output alone. The native path inside _handle_native_json_schema (lines 222-228) sets json_schema in the request body and then conditionally sets response_format only if a response_format parameter rule lists json_schema as an option. For a provider whose API has no response_format concept (concrete instance: claude-sonnet-4-6 in langgenius/anthropic v0.3.26, filed separately as langgenius/dify-official-plugins#3671), no such rule exists. Nothing is set, the provider plugin does not read json_schema, and the schema is dropped on the wire. The else branch (prompt-based fallback) had already been excluded by the if/else, so a model that declared the feature produced worse results than one that did not: the latter would still get prompt-based schema guidance.

The reporter's second observation — that the editor indicator reads served model metadata while the runtime branches on the plugin's YAML declaration — is out of scope for this PR. Aligning the editor indicator to the runtime predicate is a sensible follow-up that reuses the same predicate.

Changes

• api/core/llm_generator/output_parser/structured_output.py — added the can_use_native_schema predicate that requires BOTH support_structure_output=True AND a response_format parameter rule that lists ResponseFormat.JSON_SCHEMA. If the native path cannot actually consume the schema, fall through to the prompt-based branch (which already works). An INFO log is emitted on the fallback path so the operator can see which models are silently using prompt-based schema injection. The native and prompt-based branches themselves are unchanged.
• api/tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py — 2 new tests cover the two regression paths. Both assert the prompt-based fallback ran by checking the request body (json_schema is not set) and the prompt_messages (a SystemPromptMessage containing the schema payload is injected).

Out of scope (deliberate)

• Editor indicator alignment. The reporter's second cause — that the editor indicator reads served model metadata while the runtime branches on the plugin's YAML declaration — would require touching the editor component. This PR fixes the runtime silent-drop, which is the user-visible behavior; the editor follow-up is a separate concern that should reuse the same can_use_native_schema predicate so the UI and runtime agree.
• Provider plugin fixes. claude-sonnet-4-6 in langgenius/anthropic v0.3.26 should be fixed in the plugin repo (the reporter filed langgenius/dify-official-plugins#3671). Once that ships, the runtime predicate still works correctly.
• Other model types that read json_schema without going through response_format. If a future provider natively consumes json_schema via a custom key (not response_format), the current predicate would still route them to the prompt-based branch. The runtime behavior would still be correct (the schema would be applied via prompt injection), just less efficient. If such providers appear, the predicate can be extended to allow-list them; not adding speculative hooks for non-existent providers.

Testing

• uv run python -m pytest tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py — 25/25 pass (was 23/23; 2 new tests added).
• uv run ruff check on both files — All checks passed.
• uv run ruff format --check — clean on both files.
• uv run pyrefly check core/llm_generator/output_parser/structured_output.py — 0 diagnostics.
• pyrefly check on the test file is not gated (file is in tests/unit_tests/pyrefly.toml project-excludes per "Existing strict-mode debt").
• /consult-kiro (openai/gpt-5 --variant low) confirmed: "Fix looks correct and minimal; it prevents silent schema drops by gating the native path on real JSON Schema consumability and otherwise cleanly falls back to prompt injection." Suggested the optional INFO log; applied.

Risk

Low. The change is to a single branch in a single function. The new predicate is a strict tightening: any model that was previously routed to the native path but where the native path silently dropped the schema will now be routed to the prompt-based path. The prompt-based path already works correctly for models that do not declare the feature, so the user-visible behavior strictly improves (the schema is at least attempted) for any model in the buggy state. Models that were correctly using the native path are unaffected.

Impact

• Users: structured output for affected models (e.g. claude-sonnet-4-6 in langgenius/anthropic v0.3.26) now applies the schema via prompt injection. The user enables structured output in the editor; the request now goes out with the schema in the system prompt and a response_format: json_object hint, so the model is at least asked to conform. Once the plugin-side fix lands, the runtime predicate will route these models back to the native path automatically.
• Maintainers: a 30-line production diff with 2 new tests, scoped to a single file. The predicate is small enough to read in one pass and the comment block explains exactly why it exists.

Follow-up opportunities

• Align the editor indicator to the runtime predicate so the UI and runtime agree on which models can use native structured output.
• Consider a similar gate for the _set_response_format fallback path: if the model has no response_format rule that lists JSON or JSON_OBJECT, the fallback also sets nothing on the wire. Less critical (the prompt-based injection still applies the schema) but worth a comment if the maintainer wants to be consistent.

[github-actions]

Pyrefly Diff

base → PR

--- /tmp/pyrefly_base.txt	2026-09-01 07:30:08.570988852 +0000
+++ /tmp/pyrefly_pr.txt	2026-09-01 07:29:59.686927301 +0000
@@ -5720,6 +5720,10 @@
    --> tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py:192:16
 ERROR `in` is not supported between `str` and `None` [not-iterable]
    --> tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py:203:16
+ERROR `not in` is not supported between `Literal['{{schema}}']` and `None` [not-iterable]
+   --> tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py:358:17
+ERROR `in` is not supported between `Literal['"type": "object"']` and `None` [not-iterable]
+   --> tests/unit_tests/core/llm_generator/output_parser/test_structured_output.py:359:17
 ERROR Argument `() -> Session` is not assignable to parameter `session_factory` with type `sessionmaker[@_]` in function `sqlalchemy.orm.scoping.scoped_session.__init__` [bad-argument-type]
   --> tests/unit_tests/core/llm_generator/test_llm_generator.py:37:31
 ERROR Object of class `NoneType` has no attribute `split` [missing-attribute]

[github-actions]

Pyrefly Type Coverage

Metric	Base	PR	Delta
Type coverage	61.78%	61.78%	-0.00%
Strict coverage	61.38%	61.38%	-0.00%
Typed symbols	43,005	43,005	0
Untyped symbols	26,777	26,780	+3
Modules	3290	3290	0