Unix-Socket File Descriptor Passing: Define Who Owns the Received Device Handle

Receiving an SCM_RIGHTS message creates descriptor references that your process must either accept into a defined owner or close on every rejection path. It does not transfer the sender’s descriptor number, revoke the sender’s access or guarantee that the application accepted the handle. Treat ancillary parsing and ownership admission as one carefully bounded transaction.

This guide follows Linux man-pages 6.12 for AF_UNIX descriptor passing. The proposed device-service protocol uses a record-oriented local channel with an explicit handle count and request identity; stream sockets require additional framing. An executed host fixture used only an owned temporary file and socket pair, never a device handle.

Model the shared open file description

The Unix-socket documentation explains that SCM_RIGHTS effectively duplicates a reference into the receiver’s descriptor table. The numeric descriptor usually differs between processes. The shared open file description can carry a common file offset and status flags; descriptor flags such as close-on-exec are separate.

This matters when a broker passes a serial port or regular file. Changing a shared status flag can affect another holder, and reading through one regular-file reference advances the shared offset. Passing a descriptor is not equivalent to independently opening the pathname again. Document whether the sender retains an active reference, keeps it only until acknowledgment, or closes it after a defined handoff.

Closing the sender’s reference after a successful send does not mean the receiver accepted the application request. The receiving kernel or queue may hold the passed reference until delivery or cleanup. If the sender needs acceptance evidence, require a protocol acknowledgment tied to the transfer identity. Do not blindly resend uncertain transfers and create duplicate authorities.

Bound the wire and ancillary contract

Define the permitted message type, payload size, descriptor count and handle roles. For example, a hypothetical request may carry exactly one read-only data handle plus a request identifier. A message with two descriptors is a protocol error even if both are valid kernel references. A message with no descriptor must not fall back to interpreting an integer in the payload as a local handle.

The ancillary-data macros distinguish CMSG_SPACE, used to reserve padded buffer space, from CMSG_LEN, used for a control message’s declared length. Initialize control storage, traverse with CMSG_FIRSTHDR and CMSG_NXTHDR, and copy payload values safely rather than assuming CMSG_DATA has arbitrary-type alignment.

Check level, type, length, integer-array size and total received count. Treat unexpected rights messages as resources requiring cleanup even if the surrounding payload is rejected. Do not stop parsing at the first error and leak descriptors already installed by the receive operation.

Make truncation a terminal admission failure

The recvmsg result flags include MSG_CTRUNC when ancillary data were discarded because the buffer was too small. Reject a transfer whose required control information is incomplete. Close every descriptor that was actually delivered before reporting failure; do not assume truncation means the receiver obtained none.

Linux automatically closes excess passed descriptors that do not fit the ancillary buffer or would exceed the receiver’s descriptor limit. That kernel cleanup does not close the subset successfully returned to the application. This distinction is essential for a negative test: the message can be invalid while one or more live references still need explicit release.

For a record-oriented channel, also reject payload truncation according to its framing contract. For stream channels, send accompanying data as required and use recvmsg consistently at the protocol’s ancillary boundaries. Stream reads do not generally preserve application message boundaries, even though ancillary data impose documented barriers.

Set close-on-exec at receipt

Use MSG_CMSG_CLOEXEC with recvmsg when received references must not survive helper execution. Linux has supported this flag since 2.6.23. Setting FD_CLOEXEC afterward leaves a window in a multithreaded process where another thread can fork and exec with the new descriptor still inheritable.

The sender’s own close-on-exec setting is not a sufficient receiver policy. Record inheritance intent independently for every newly received descriptor. If a handle is deliberately passed onward, that is another explicit ownership and permission boundary, not an incidental consequence of launching a helper.

Validate capability before assigning a device role

A valid descriptor can refer to the wrong kind of object. Apply the service’s supported, side-effect-free checks for object type, access mode and expected interface before performing operations. A pathname in the payload is only a claim about the descriptor, not proof of its origin or capabilities.

Authenticate the peer under the local-service policy before accepting authority-bearing handles. Even an authenticated peer can send malformed or inappropriate input, so keep parsing limits and cleanup independent from authorization. Do not probe a received device with arbitrary ioctls merely to discover what it is.

Use a small admission state machine: received/unowned, validated, accepted by named subsystem, and released. Exactly one component owns cleanup at every transition. If validation or allocation fails, the unowned-receive cleanup closes all delivered references. Once ownership transfers, the receiver wrapper must not close them again.

Executed fixture and proposed fault cases

A host fixture on x86-64 Linux 6.18.44 sent one temporary-file descriptor through an owned socket pair and received it with MSG_CMSG_CLOEXEC. The received descriptor was noninheritable. Reading two bytes through it advanced the sender’s file offset to two, demonstrating the shared open description.

The fixture then sent two references with control space sized to return only one descriptor. MSG_CTRUNC was observed, one delivered descriptor was identified and closed, and the temporary resources were cleaned up. This was a same-process descriptor-table exercise; it did not test a cross-process acknowledgment protocol, resource exhaustion or device access.

Proposed qualification adds missing, extra and unexpected descriptors; malformed metadata; receiver rejection after successful receipt; peer exit before acknowledgment; and helper execution concurrent with receipt. Require descriptor counts to return to the baseline after each rejected transfer and accepted handles to have one documented final owner.

The final artifact is an ownership ledger linked to the ancillary decoder. It should make uncertain delivery visible, keep duplicated references intentional and prevent a rejected message from leaving a port or file unexpectedly alive.