Linux Socket Buffer Settings: Measure the Applied Value Instead of Trusting the Request
Read back SO_RCVBUF and SO_SNDBUF after setting them, then qualify the real traffic path. On Linux the requested value can be capped, is doubled for internal accounting and is subject to a minimum. A successful setsockopt call therefore does not prove that the requested payload capacity was installed or that the receiver can survive the intended burst.
This guide pins Linux v6.12 socket-option implementation and Linux man-pages 6.12. The concrete workload discussion concerns UDP receive buffering. TCP autotuning and Unix-domain socket-specific behavior need separate qualification. A host check used one unbound owned UDP socket and transmitted no packets.
Record three different quantities
Keep the requested application value, the returned option value and observed traffic behavior in separate fields. Add socket family/type, kernel version, relevant effective limits and whether the application explicitly set the option. Do not report the returned number as measured throughput or usable application bytes.
The socket option reference explains that Linux doubles SO_RCVBUF and SO_SNDBUF requests to account for bookkeeping and returns that doubled value. The v6.12 implementation first applies the normal receive/send maxima, bounds arithmetic, and enforces its minimum accounting values. Thus both large and very small requests can produce a result different from a naive two-times expectation.
For a hypothetical request of 64 KiB that is admitted without a limiting clamp and lies above the minimum, a 128 KiB readback is consistent with the documented accounting. It does not mean 128 KiB of UDP payload can necessarily be queued. Each datagram consumes additional kernel storage, and actual admission depends on the protocol path.
Inspect effective limits without changing them
Read the relevant rmem_max and wmem_max context through the supported deployment inspection path. Record the namespace and product configuration rather than copying a default from a different machine. Compare the requested values with those limits and the returned socket values.
The normal SO_RCVBUF and SO_SNDBUF paths cap requests rather than using a generic “request too large” failure as the only signal. Treat readback as part of setup verification. A wrapper that logs only setsockopt success can conceal a configuration that was never applied at the intended size.
SO_RCVBUFFORCE and SO_SNDBUFFORCE are privileged alternatives with CAP_NET_ADMIN requirements. They are not the diagnostic default in this procedure. Do not change system limits or add privilege merely because a burst test loses packets; first establish whether the bottleneck is socket capacity, consumer service time or an earlier receive stage.
Build a datagram workload contract
Specify payload size distribution, datagrams per burst, spacing, maximum reader pause and the application’s acceptable loss policy. A byte-rate average is insufficient when a short burst arrives while the reader is unavailable. Keep kernel receive buffering separate from any userspace queue that stores already received messages.
Use sequence identifiers in harmless test datagrams so the receiver can distinguish loss, duplication and reordering. The sender’s successful send only establishes its local API outcome; it does not confirm delivery to the receiver’s application. Track sent, received, rejected and still-pending work under clearly scoped observations.
Vary one factor at a time. Compare buffer settings with the same payload distribution and consumer schedule. Then vary the service pause while keeping the applied setting fixed. Changing packet size, CPU affinity and buffer size together makes it difficult to explain which boundary changed.
Separate socket drops from other missing packets
SO_RXQ_OVFL, available since Linux 2.6.33, can attach an unsigned 32-bit ancillary counter describing packets dropped by that socket since creation. Receive it through a correctly sized recvmsg control buffer and validate the control-message type and length. A wrapper that ignores ancillary data discards this evidence.
Use differences within one socket lifetime and account for counter wrap. A value arrives with a received packet, so silence alone does not provide a fresh counter sample. If all later packets fail to arrive, the absence of new ancillary information cannot prove the drop count stayed unchanged.
This counter is not an end-to-end loss oracle. Packets may be lost before reaching that socket, and the application can also discard data after receipt. Pair it with the sequence record and appropriately scoped interface/protocol counters. Keep each counter’s reset epoch and collection interval explicit.
| Observation | Next interpretation step |
|---|---|
| Readback below the intended accounting value | Inspect the applied cap, minimum and exact option path. |
| Socket drop delta with reader pauses | Qualify buffering and consumer service envelope together. |
| Sequence gaps without socket-drop evidence | Investigate earlier path loss, missing counter samples and application discard. |
| No drops but excessive age | Check whether added buffering violates the latency contract. |
A larger buffer changes failure behavior
More capacity can absorb a bounded interruption but cannot fix sustained arrival above service rate. It may also let stale data accumulate longer before rejection becomes visible. Choose the smallest justified budget that meets the supported burst and age constraints, with a deliberate overload policy.
For current-state telemetry, retaining every old sample may be less useful than a documented coalescing policy after receipt. For events that must be preserved, the protocol needs admission, retry or durable handling beyond a socket option. Do not advertise reliable delivery merely because a larger receive buffer made a short test pass.
Executed readback and proposed traffic tests
On an x86-64 Linux 6.18.44 host, one unbound AF_INET/SOCK_DGRAM socket requested 65,536 bytes for each normal buffer option. getsockopt returned 131,072 for SO_RCVBUF and 131,072 for SO_SNDBUF. The socket was then closed. These are actual configuration readbacks, not queue occupancy, drop or throughput measurements.
No packet traffic, privileged force option or system-wide setting change was used. Proposed target tests should cover the maximum supported burst, bounded consumer pauses, smallest and largest payloads, reconnect/socket recreation and overload beyond the supported envelope. Save the requested/applied pair with every run.
The release artifact is a setting-to-observation table tied to workload and socket lifetime. It should show where the applied budget came from, how drops were attributed and whether latency stayed acceptable. That is the evidence needed before claiming the buffer setting solves the product’s receive problem.
