Add Major system leak as an aggregation of multiple Minor leaks in BMC - #2541
Open
judyjoseph wants to merge 8 commits into
Open
judyjoseph wants to merge 8 commits into
judyjoseph wants to merge 8 commits into
Conversation
Collaborator
|
/azp run |
|
No pipelines are associated with this pull request. |
Collaborator
|
/azp run |
|
No pipelines are associated with this pull request. |
judyjoseph
commented
Sep 9, 2026
| ``` | ||
| config liquid-cool major-leak-threshold <N> | ||
|
|
||
| - N : integer, minimum 2 (a single Minor leak is MINOR_SYSTEM_LEAK). Stored in system_major_leak_num_min_sensors of LEAK_CONTROL_POLICY, overriding the platform-provided value. |
Contributor
Author
There was a problem hiding this comment.
Need to add a check in code to make sure it is atleast 2
judyjoseph
commented
Sep 9, 2026
| +--------------------------------------+-------------------------------------------+-------------------------------+ | ||
| | 1 Critical leak | CRITICAL | CRITICAL_SYSTEM_LEAK | | ||
| | 2 or more leaks (any severity) | Any Severity | CRITICAL_SYSTEM_LEAK | | ||
| | 1 Minor leak staying for MAX-T secs | MINOR | CRITICAL_SYSTEM_LEAK | |
Contributor
Author
There was a problem hiding this comment.
there was a comment from community as to whether we need to change this from CRITICAL --> MAJOR system leak
Collaborator
|
/azp run |
|
No pipelines are associated with this pull request. |
Contributor
Author
|
Fraser, @vivekverma-arista @roger-nexthop for review as well - cannot find you in reviewers list |
Contributor
|
Thanks for the updates, Judy. This looks good to me. |
Collaborator
|
/azp run |
|
No pipelines are associated with this pull request. |
Collaborator
|
/azp run |
|
No pipelines are associated with this pull request. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Introduce a MAJOR severity level for liquid-cooling leak handling, alongside the existing MINOR and CRITICAL levels, for both locally-detected system leaks and external Rack Manager alerts.
Key changes:
• Add SYSTEM_LEAK_MAJOR_EVENT and RACK_MGR_MAJOR_EVENT to the BMC event table and renumber the Switch-Host state-transition table accordingly (both default to syslog_only, no power-off).
• Define MAJOR_SYSTEM_LEAK in the thermalctld severity algorithm: triggered when MAX-N or more concurrent Minor leaks are present. Precedence is CRITICAL > MAJOR > MINOR, evaluated top-down.
• Replace the fixed "2 or more Minor leaks" rule with a configurable MAX-N threshold. Default comes from a new platform API get_major_leak_num_min_sensors() (LiquidCoolingBase) - returning 0 means the platform does not support MAJOR classification. Users can override via system_major_leak_num_min_sensors in LEAK_CONTROL_POLICY.
• Add CLI config liquid-cool major-leak-threshold and extend leak-action to accept major.
• Update LEAK_CONTROL_POLICY DB schema and show platform leak control-policy output with the new major actions and threshold (threshold line omitted when the platform returns 0).
• 1 Minor sensor staying as Minor for MAX-T secs also moved to SYSTEM_MAJOR instead of CRITICAL as earlier