Repository navigation
esp32 uart: Fix read blocking on a stale event - #2428
Conversation
A read that finds data takes every buffered byte, including the bytes of UART_DATA events still queued. When the listener then handles one of those events, uart_read_bytes waits forever for bytes that were already delivered, blocking the scheduler: with a peer that then goes quiet (a modem after ATH), the task watchdog resets the chip. Read only what the buffer holds, without waiting, and skip an event that finds it empty. Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
3e9ccb1 to
d326239
Compare
bettio
left a comment
There was a problem hiding this comment.
Just a small issue.
Also, not introduced by this PR, so it might be an additional comment or a new open issue: the blocking read in uart_driver_do_read is the other half of this race.
uart_driver_do_read still calls uart_read_bytes(..., count, portMAX_DELAY) after sampling count with uart_get_buffered_data_len. On the default dual-core (SMP) builds the listener on the other scheduler can consume part of those bytes in between (a uart:read/2 timeout does cancel_read + read while a data event is being handled), so that call can still park a scheduler thread until the peer sends more, which after ATH is never. A 0 timeout there too, registering the reader with safe_update_reader_data when nothing was read (as the USJ branch in the same function already does) and building the binary from the returned length, would close it.
With a 0 tick timeout uart_read_bytes can return -1 (rx mutex busy) or
fewer bytes than counted (the other scheduler drained the buffer), and
the result was dropped: {ok, Bin} went out with uninitialized bytes.
uart:read/2 had the same race with portMAX_DELAY: after counting the
buffered bytes, the listener on the other scheduler could take some,
and the read then waited for the peer to send more.
Both paths now read into a temporary buffer without waiting, build the
binary from the returned length, and keep the reader registered when
nothing was read. The UART and USB-Serial/JTAG branches share it.
Signed-off-by: Gabriel Mancini <gabriel.mancini@gmail.com>
|
Thanks! I fixed Tested:
|
The ESP32 UART listener reads
event.sizebytes for eachUART_DATAevent withportMAX_DELAY. Butuart:read/1,2reads every buffered byte when it finds data (uart_get_buffered_data_len), including the bytes ofUART_DATAevents still waiting in the queue. When the listener later handles one of those events, the buffer holds fewer bytes than the event announces (often none), anduart_read_bytesblocks the scheduler until that many new bytes arrive. If the peer goes quiet, nothing ever arrives: without SMP the whole VM stops, and the task watchdog resets the chip.How we hit it: a SIM7600 modem left in PPP data mode keeps sending LCP frames. The application sends
+++to return the modem to command mode, thenATH. The modem goes quiet, a stale event is handled, and the task watchdog (both cores idle, the AtomVM task blocked) resets the board. It happened the first time we tried it, on an ESP32 (no SMP build, release-0.7 based; our application sets a 30 s task watchdog).The fix clamps the count to what the buffer holds, reads without waiting, and skips an event that finds the buffer empty (the reader stays registered for the next one). The USB Serial/JTAG path already reads what is available and is unchanged.
Tested on hardware (ESP32, LilyGO T-Internet-COM with a SIM7600SA on UART2): the same sequence now returns to command mode and dials again (twice from data mode, plus several dials from command mode, with the modem then carrying a PPP link to NervesHub). I did not find a way to reproduce this in QEMU, since it needs a peer on a second UART; happy to add a test if you have a preferred harness for it.