Bug:[PDDF][PSU] BMC/IPMI read failure incorrectly reported as PSU absent due to get_presence() mapping "N/A" to False

[ssithaia-ebay]

Is it platform specific

generic

Importance or Severity

High

Description of the bug

On a SONiC device using the PDDF platform framework (Celestica DS4000), intermittent PSU absence/recovery warnings are observed even though the PSUs are not physically removed.
The issue was observed for both PSU1 and PSU2.

Analysis:

• Initial analysis indicates that an intermittent BMC/IPMI command failure is being interpreted by the PDDF PSU presence logic as False (PSU not present).
• As a result, a transient failure to retrieve PSU presence information from the BMC is reported as a physical PSU removal. When the next BMC poll succeeds, the PSU is immediately reported as inserted again.
• Therefore, the current implementation does not distinguish between:
  • PSU confirmed absent
  • PSU confirmed present
  • PSU presence status unavailable because the BMC/IPMI read failed

Observed Behavior
On August 29, 2026, multiple PSU1 absence/recovery events were observed.
Examples:
Time Event Duration
10:07 PSU1 reported absent and then present ~6 seconds
13:46 PSU1 reported absent and then present ~30 seconds
14:46–14:48 PSU1 reported absent and then present ~1 min 20 sec

• Similar intermittent behavior has also been observed for PSU2.
• There was no corresponding physical PSU removal during these events.
• The warnings clear automatically once a subsequent BMC/IPMI read succeeds.

Code Flow
PSU presence is periodically polled by psud approximately every 2 seconds:
pmon / psud
|
| PSU2.get_presence()
v
pddf_psu.py::get_presence()
|
| get_attr_name_output("PSU2", "psu_present")
v
pddfapi.py
|
| pddf-device.json BMC command
| ipmitool raw 0x04 0x2D 0x3B
| index = 2
| mask = 0x01
v
raw_ipmi_get_request()
For a successful BMC response:
ipmitool raw 0x04 0x2D 0x3B

Example:
00 c0 01 80
^^
index 2
The configured mask is applied:
01 & 01 = 1
which produces:
status = "1"
The PDDF plugin contains a value map similar to:
{
"1": true,
"0": false
}
Therefore:
"1" -> True -> PSU present
"0" -> False -> PSU absent
However, when the BMC/IPMI command fails, times out, or does not return usable data, raw_ipmi_get_request() returns:
"N/A"
The resulting flow becomes:
BMC/IPMI read failure
|
v
raw_ipmi_get_request()
|
v
"N/A"
|
v
"N/A" not present in valmap
|
v
get_presence() returns False
|
v
psud interprets False as physical PSU absence
|
v
PSU absence warning
When the next poll succeeds:
BMC response OK
|
v
status = "1"
|
v
True
|
v
PSU reported inserted again
This produces misleading PSU removal/reinsertion events even though the hardware was never physically removed.

Steps to Reproduce

The issue occurs intermittently when the BMC/IPMI command used to retrieve PSU presence fails or returns an invalid/unavailable result.

Actual Behavior and Expected Behavior

Actual Behavior
Currently:
Valid BMC response, presence bit = 1
-> True

Valid BMC response, presence bit = 0
-> False

BMC timeout/error/invalid response
-> "N/A"
-> value not found in valmap
-> False
Therefore both of the following cases become indistinguishable to psud:
PSU genuinely absent -> False
BMC communication failure -> False
This can generate false PSU absence and recovery notifications.

Expected Behavior
A BMC/IPMI communication failure should not automatically be interpreted as:
PSU present = False
The software should distinguish between the following states:
Present
Absent
Unknown / read failure
Only an explicit, successfully decoded BMC response indicating that the PSU presence bit is 0 should result in a PSU-absent state.
A timeout, malformed response, command failure, exception, or "N/A" result should be treated as a sensor/read failure rather than confirmation that the PSU is physically absent.

Relevant log output

Output of show version, show techsupport

show platform summary

Platform: x86_64-cel_silverstone_v2-r0
HwSKU: DS4000
ASIC: broadcom
ASIC Count: 1
Serial Number: R4023B2F044513GD200427
Model Number: R4023-F9003-A0
Hardware Revision: 4

Attach files (if any)

ds4000_bmc_20260916T070339Z.zip