feat(providers): add current GPT budgets to OpenAI and Azure registries - #539
Conversation
- Add 13 entries per provider: GPT-5 generation at 400K/128K (5, mini, nano, 5.1, 5.1-codex, 5.1-codex-max, 5.2) and the 1.05M generation at 1.05M/128K (5.4 corrected from 1M, 5.5, 5.6-luna/terra/sol, 5.6 alias of Sol, 6-astra last); numbers verified against official OpenAI model pages, sources cited inline - Azure mirrors stock IDs since deployments resolve by model name - Annotate -codex entries as Responses-API-only; omit base gpt-5.3 (no general release), gpt-4.1 and o-series (unverified) - Extend both provider test suites with gpt5/flagship lookup loops - No code or default-model changes - Verified: 125 passed / 9 skipped (pre-existing); ruff + format + diff-check clean Signed-off-by: Yoseph Zuskin <zuskinyoseph@gmail.com> Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <noreply@opencode.ai>
rng1995
left a comment
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Reviewed current head 2190e7e2f6a3dae2117690311df20eed06b7c9b8. The official OpenAI model documentation supports the proposed token budgets, and all required checks pass. One coverage gap remains: gpt-5.4 is newly added to the Azure registry but omitted from the new Azure flagship regression loop, so that entry can regress while CI stays green. Add it to the 1,050,000/128,000 assertions.
The required test change and mergeStateStatus=BEHIND block merging.
Signed-off-by: Yoseph Zuskin <zuskinyoseph@gmail.com>
- Add gpt-5.4 to test_metadata_flagship_generation: it was the one new Azure entry with 1,050,000/128,000 budgets but no lookup coverage, so a typo or removal would have left CI green - Verified post upstream/main merge: provider suites 192 passed / 12 skipped; ruff + diff-check clean Signed-off-by: Yoseph Zuskin <zuskinyoseph@gmail.com> Co-Authored-By: OpenCode Muse Spark 1.3 Free (1M context) <noreply@opencode.ai>
rng1995
left a comment
rng1995
left a comment
There was a problem hiding this comment.
[SkillSpector Review]
Re-reviewed current head 095e5cb51a304fa60ba7c71d0d9f1e63d4e8072f against the prior requested change, current provider registries, complete tests, thread reply, and exact-head checks.
The missing Azure regression is resolved: gpt-5.4 is now included in the 1,050,000-context/128,000-output flagship loop, while the separate GPT-5 generation loop covers the 400,000-context entries. The OpenAI and Azure registry values and their focused tests are internally consistent. I found no remaining required changes.
All six exact-head checks pass. GitHub reports the PR as mergeable but BLOCKED; branch-protection requirements still govern merging.
Priority: P1 — accurate model budgets directly affect prompt truncation and output allocation.
|
Hi @rng1995, any further changes required or is this ready to merge? Thanks in advance! |
Merging NVIDIA#539 left gpt-5.6-luna, gpt-5.6-terra, and gpt-5.6-sol mapped twice in the OpenAI provider registry: once in the canonical 1.05M generation block and again in the pricing block added here. YAML loaders disagree on duplicate keys -- most keep the last mapping, stricter ones reject the document -- so the resolved budget was loader-dependent and the two copies could drift apart unnoticed. Keep one mapping per model, the canonical ones, and move the pricing comments alongside them. That leaves a comment-only diff for this file; every mapping in it is now NVIDIA#539's. The lookup loop added to test_metadata_known_model repeated assertions that test_metadata_flagship_generation already makes, and could not have caught the duplication in the first place, since both copies resolved to the same values. Replace it with coverage that value-only assertions cannot provide: - test_each_model_is_mapped_once reads keys via yaml.compose across the root and all bundled registries. yaml.safe_load collapses a repeated key to the last one, which is why a green suite missed this. - test_root_registry_replaces_bundled_yaml exercises the repo-root file as a SKILLSPECTOR_MODEL_REGISTRY override, pinning the ids only it carries and the fact that an override replaces the bundled YAML rather than merging with it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: chinmay1819 <chinmaydeshpande34@gmail.com>
2 participants
Add current GPT model budgets to the OpenAI and Azure registries
Problem
The bundled
openairegistry knows onlygpt-5.4(at a stale 1M context),and the
azure_openairegistry stops at the GPT-4 generation, so anyonescanning with a current GPT model — including GPT-6 Astra for complex code —
falls back to the 128K default budget and risks truncated or over-allocated
token windows.
Fixes: #538
Approach
provider (
openai+azure_openai), all numbers verified 2026-09-14against official OpenAI model pages.
gpt-5,gpt-5-mini,gpt-5-nano,gpt-5.1,gpt-5.1-codex,gpt-5.1-codex-max,gpt-5.2(the-codexentries annotated Responses-API-only, since this provider speaks chat
completions).
gpt-5.4(corrected from 1M),gpt-5.5,gpt-5.6-luna,gpt-5.6-terra,gpt-5.6-sol, thegpt-5.6alias (documented alias of Sol), and
gpt-6-astralast.budgets apply. Omitted deliberately: base
gpt-5.3(no general releaseexists — only deprecated/sunset
chatandcodexvariants),gpt-4.1and o-series (unverified), default-model bumps (behavior change, separate
proposal).
test_metadata_gpt5_generation+test_metadata_flagship_generationloops in both provider suites.
Verification
tests/unit/test_providers.py+tests/unit/test_new_providers.py:125 passed, 9 skipped (pre-existing skips).
ruff check+ruff format --check: clean on all touched files.git diff --check: clean.Risks
revises a window, the entry needs a matching update (each block cites
its source and verification date).
Signed-off-by(maintainer: verify on push).