Skip to content

[bulker]: Preserve per-request bulk create statuses - #4911

Draft
Xichen96 wants to merge 1 commit into
sonic-net:masterfrom
Xichen96:dev/xichenlin/fix-objectbulker-create-status
Draft

Xichen96 wants to merge 1 commit into
sonic-net:masterfrom
Xichen96:dev/xichenlin/fix-objectbulker-create-status

Conversation

@Xichen96

@Xichen96 Xichen96 commented Sep 22, 2026

Copy link
Copy Markdown
Contributor

What I did

Added optional per-request create-status output to ObjectBulker and associated each bulk result with the request that queued it. NeighborContext now receives its own next-hop create status, and NeighOrch consumes that status directly instead of looking it up by the returned object ID.

The OID-keyed create-status map and create_status(oid) accessor are removed. Existing callers that do not request a status remain source-compatible.

Why I did it

The old status path lost the relationship between a bulk-create request and its result:

  1. A host-route MUX standby-to-active transition queued neighbor and next-hop creates.
  2. In synchronous mode, SAI-Redis preallocated a VID for the next-hop request before syncd executed it.
  3. If the next-hop create failed, SAI-Redis returned both the failed request's VID and its per-object failure status.
  4. ObjectBulker stored the status under that returned VID, but reset the caller's failed output OID to SAI_NULL_OBJECT_ID.
  5. NeighOrch then requested create_status(SAI_NULL_OBJECT_ID).
  6. Because the null key was absent, unordered_map::operator[] inserted a zero-initialized value. Numeric zero is SAI_STATUS_SUCCESS.
  7. NeighOrch therefore recorded a null local next-hop OID, incremented next-hop CRM and router-interface reference accounting, marked the neighbor hardware-configured, and allowed the MUX transition to complete.

The previous unit mock did not populate returned object IDs. Failed requests therefore appeared under the null key and accidentally hid the production VID-to-null mismatch.

The correction stores the caller's optional status pointer beside each queued request, initializes it to SAI_STATUS_NOT_EXECUTED, initializes the SAI status array to the same value, and copies statuses[i] directly back to request i. A failed or unwritten next-hop result can no longer become success through an unrelated OID lookup.

This change deliberately does not redesign SAI_STATUS_ITEM_ALREADY_EXISTS behavior or MUX transition logic.

How I verified it

  • Added ObjectBulker tests for distinct failed VIDs, zero output IDs, mixed results across multiple bulk chunks, successful creation, and an unwritten status.
  • Added NeighOrch tests proving failed and unexecuted next-hop creates do not install or account a null next hop.
  • Added an end-to-end host-route MUX regression using the normal SAI bulk response shape: aggregate SAI_STATUS_FAILURE, a nonzero returned VID, and per-object SAI_STATUS_INSUFFICIENT_RESOURCES.
  • On unmodified master, that regression produced an active MUX state, a cached null next-hop OID, hw_configured=true, and incorrect CRM and router-interface reference increments.
  • The same regression passes with this change and verifies rollback removes the successfully created neighbor and restores both neighbor and next-hop accounting.
  • Before rebasing onto the latest master, the complete mock-test binary passed 1034/1034 tests.
  • Exact-head Azure.sonic-swss CI is pending on this draft PR.

Details if related

This is an independent ObjectBulker status-integrity correction. Follow-up handling for a trustworthy next-hop SAI_STATUS_ITEM_ALREADY_EXISTS result without a recoverable OID is intentionally out of scope.

Work item tracking
  • Microsoft ADO (number only): 39809156

Associate each ObjectBulker create result with the request that queued it instead of the returned object ID. This prevents failed preallocated VIDs from being lost when the caller OID is reset to null and keeps unwritten results as NOT_EXECUTED.

Pass the next-hop result through NeighborContext so NeighOrch cannot manufacture success from a missing null-OID lookup. Add focused, chunked, and end-to-end regression coverage.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: a41b9f81-e913-42b1-aa01-5cd553b97791
Signed-off-by: Xichen Lin <lukelin0907@gmail.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

No unresolved blocking issues were identified.

Review effort: Lite
Findings: None

What changed in this PR

Preserves per-request bulk-create statuses so failed next-hop creations are not treated as successful.

Changes:

  • Adds request-associated status tracking to ObjectBulker.
  • Updates NeighOrch to consume direct next-hop statuses.
  • Adds bulker, neighbor, and MUX rollback regression tests.
File Description
tests/​mock_tests/​neighorch_ut.cpp Tests failed and unexecuted next-hop handling.
tests/​mock_tests/​mux_rollback_ut.cpp Tests MUX rollback behavior.
tests/​mock_tests/​bulker_ut.cpp Tests status propagation scenarios.
orchagent/​neighorch.h Initializes next-hop state safely.
orchagent/​neighorch.cpp Handles failed and unexecuted next-hop creates.
orchagent/​bulker.h Tracks and propagates per-request create statuses.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants