comm: synchronize the rail barrier with putValue flags instead of GIN indexed signals

[KeitaW]

Problem

Above 22 NVLink domains, buffer construction fails on EP_HOST_ASSERT(gin_indexed_signals_cnt >= num_rdma_ranks - 1) in nccl.cu. The rail barrier in gin_barrier_wo_local_sync gives every rail peer a dedicated indexed signal, and with the default 11 GIN contexts the per-context budget is (256 - 2 * 11) / 11 = 21 signals, so the assert holds up to 22 domains and fails at 23.

#5 lifts that ceiling by turning the rail barrier into a counting barrier on one reserved signal. Review of #5 objected that reserving the signal offsets every data-path signal id by one and restructures the unordered arm's resource request around the reservation. This PR makes the other change: the rail barrier stops using indexed signals at all, so nothing in the data path has to move.

Change

Arrival is a 4-byte putValue of a round number into a slot the sender owns on each peer, and the wait polls the peers' slots with ld_acquire_sys until each holds the current round. The slot table sits at the end of WorkspaceLayout, indexed by (barrier tag, phase, peer). Round numbers come from device-local send and receive counters in the same workspace, one pair per (tag, index), the same arrangement the World branch already uses through the signal shadow, so no value is plumbed from the host and no gpu_barrier call site changes.

Slots alternate between two phases so that each slot has at most one write in flight. A putValue overwrites where a SignalInc accumulated, and overwrites do not commute, so rounds k and k+2 must not share a slot under an unordered transport. A rank issues its round k+2 write only after every peer has reported round k+1, which each peer does only after reading the rank's round k value, so the round k write to that phase slot has already been delivered. The wait compares with flag == round: under that lockstep the slot holds exactly round once the arrival is delivered, and an exact compare turns a broken requirement into a timeout at the offending round, with the stale value in the print.

The flag is posted through a NCCLGin handle on QP 0 with CTA-scoped sharing, the same context the previous signal protocol used, rather than through the caller's channel handle. The World instantiation is unchanged.

The phase argument rests on three requirements that hold at every call site today and are written at the rail branch: every rank runs the same number of rail barriers per tag, two rail barriers on one tag are separated by a rank-wide join (a grid sync or a kernel boundary), and the workspace is never re-zeroed after construction. Nothing enforces them at compile time or at runtime beyond a static_assert that each instantiation's tag fits the slot table.

Commits: 828697d (the barrier), 730fcc6 and 5de6dea (tag check, exact compare, the written requirements), 33e8de0 (QP 0), 5315cc2 (host assert for the next limit, see below).

What stays as it is on main

deep_ep/include/deep_ep/impls/, csrc/kernels/elastic/, qp_mapping.cuh, gin_resource_alloc.cuh and handle.cuh are byte-identical to main; the diff is comm.cuh, layout.cuh, nccl.cu and a comment in buffer.hpp. The GIN resource request is unchanged, and both arms print gin_context_cnt=11, gin_indexed_signals_cnt=21, num_qp=11 with EP_BUFFER_DEBUG=1; the difference is that the barrier no longer consumes any of those signals. WorkspaceLayout::get_num_bytes() grows by 256 KiB (16 tags x 2 phases x 1024 ranks x 4 bytes of flags, and the same again for the counters). The symmetric workspace is rounded up to 2 MiB, and the GPU-side allocation is 14 MiB before and after; only the pinned host mirror grows.

Validation

Everything below ran on four nodes of eight H200 GPUs over EFA (Libfabric_GDAKI v14 GIN plugin), EP32, --num-sms=24 --num-allocated-qps=11, hidden 7168, top-8 of 256 experts, against a baseline built from main at 54fffef over the same libnccl.

• test_ep on the unordered hybrid kernels passes correctness in every cell, 13 cells at an earlier commit and one at 5315cc2, with DeepEP rail barrier timeout absent from every pod's log. These runs were also the first execution of the SM90 sender path.
• test_barrier (1000 Buffer::barrier() rounds per timed call) passes at 5315cc2 with 22.63 to 23.04 us per barrier across the four nodes. An earlier pair of single runs measured 22.9 us for the flags against 24.9 us for the indexed signals; one run each, so this is not offered as an effect size.
• Combine timing at 4096 tokens is not reported: a baseline-versus-baseline comparison moved the same image by 763 us between runs at that shape, which is larger than any arm difference, so that harness cannot separate the arms.
• EP_HYBRID_KERNEL=ordered fails at buffer construction on main and on this branch alike, with GIN strong signals are required, but the GIN plugin does not support them, so the ordered hybrid kernels never reach any rail barrier on this plugin.