MQTT Packet Identifiers: Prove Safe Reuse Across QoS Handshakes
Allocate MQTT Packet Identifiers from one outbound ownership pool shared by QoS-positive PUBLISH, SUBSCRIBE and UNSUBSCRIBE. Release an identifier only after the corresponding terminal acknowledgement has been validated and processed. A successful PUBREC advances QoS 2 state but does not release its identifier; a failed PUBREC ends that exchange.
This guide pins the MQTT 5.0 Standard, 7 March 2019, sections 2.2.1, 4.3 and 4.4. The deliverable is an allocator and state-transition qualification procedure. Business-message deduplication, request/response Correlation Data and broker Session takeover are separate contracts. An original serialized ownership model was executed, not a client library or broker handshake.
Define the pool’s real scope
A Packet Identifier is a nonzero two-byte value. The client uses a unified set for its outstanding PUBLISH, SUBSCRIBE and UNSUBSCRIBE commands. Separate counters for those packet types are unsafe if they can allocate the same active value.
The server and client allocate independently. An outbound client PUBLISH with identifier seventeen can coexist with an unrelated inbound server PUBLISH using seventeen. Keep direction and Session context in the lookup key instead of placing both exchanges in one undifferentiated table.
QoS 0 PUBLISH has no Packet Identifier. Do not insert a dummy identifier into that packet or reserve an outbound identifier merely because the application publication has its own tracking number.
Make the terminal event explicit
| Outgoing command | Intermediate state | Identifier becomes reusable after processing |
|---|---|---|
| QoS 1 PUBLISH | Waiting for PUBACK | Corresponding legal PUBACK |
| QoS 2 PUBLISH | Waiting for PUBREC | Failed PUBREC with legal reason at least 0x80, or later PUBCOMP |
| Successful QoS 2 PUBREC | Send PUBREL with the same identifier; wait for PUBCOMP | Corresponding PUBCOMP |
| SUBSCRIBE | Waiting for SUBACK | Corresponding SUBACK |
| UNSUBSCRIBE | Waiting for UNSUBACK | Corresponding UNSUBACK |
Release concerns protocol ownership, not whether the application liked the result. A negative PUBACK still acknowledges the publication. A SUBACK containing rejected subscription results still completes that SUBSCRIBE exchange; the individual filter failures remain available to the application after the identifier is freed.
Validate acknowledgement structure, permitted reason codes, identifier and expected phase before changing the table. An arbitrary byte below 0x80 is not automatically legal. The local model assumes this decoding and legality gate has already succeeded.
Do not confuse payload release with identifier release
After successful PUBREC, a sender may no longer need the original PUBLISH payload for retransmission, but it still owns the exchange’s Packet Identifier while PUBREL is outstanding. A memory optimization that frees the payload must leave enough protocol state to complete that phase.
Keep separate fields for application record, retransmittable payload and protocol phase. One generic “publish buffer released” callback cannot safely stand in for all three ownership transitions.
Once PUBREL has been sent, the MQTT 5 sender must not resend the original PUBLISH. Recovery must follow the remaining handshake phase. This is another reason to store the phase explicitly instead of retaining only a boolean indicating that some publication remains pending.
A hypothetical three-slot allocator
The reduced fixture allocates identifier one to QoS 2, two to SUBSCRIBE and three to UNSUBSCRIBE. A fourth allocation fails with bounded exhaustion. A successful PUBREC for one changes its phase to waiting for PUBCOMP, but the allocator remains full.
After SUBACK for two, the next QoS 1 publication receives two. UNSUBACK releases three; PUBCOMP releases one. The model also verifies that a PUBACK mistakenly associated with the QoS 2 PUBREC phase does not free that entry.
This small pool makes a resource defect easy to reproduce. It is a local capacity limit, not a reduced MQTT wire identifier range. The production pool may deliberately support fewer outstanding commands than the protocol’s 65,535 usable values.
Test exhaustion without overwriting an owner
The executed full-range case allocated all 65,535 nonzero values uniquely, rejected another allocation, released identifier 32,768 and allocated that exact available value for SUBSCRIBE. Zero was never returned.
Use a bounded availability structure and an explicit admission result. Do not loop indefinitely waiting for a free identifier inside the network-processing task, because that same task may need to process the acknowledgements that make progress possible.
Identifier capacity and Receive Maximum are different restrictions. The former includes subscription commands; the latter controls QoS-positive publication flow in a particular connection direction. Satisfying one does not prove the other is satisfied.
Preserve state across the right recovery boundary
An application timeout or a lost socket does not by itself complete an MQTT exchange. Do not release its identifier just because a caller stopped waiting. Resolve the Session’s retained protocol state through the supported client implementation.
Section 4.4 requires unacknowledged QoS-positive PUBLISH and PUBREL packets to retain their original identifiers when reconnecting with Clean Start zero and an existing Session. The reconnect decision and Session Present result therefore matter; replacing a socket is not sufficient evidence that every identifier is free.
The fixture retained a pending phase and original identifier in a simple snapshot. It did not implement Session negotiation, persistence, retransmission ordering or old callback rejection. Those require tests of the selected library and its actual recovery API.
Audit the ownership invariants
- Record one sender-owned identifier pool across all supported initiating packet types.
- Keep inbound peer identifiers separate from outbound identifiers.
- Assert that every active identifier has one command and one expected protocol phase.
- Exercise successful and failed QoS 2 paths without early reuse.
- Exhaust both the configured pool and the full wire range in suitable host tests.
- Check that valid negative outcomes release protocol resources while preserving application failure evidence.
- Qualify resumed-Session behavior with the actual client rather than clearing the table on socket close.
The executed model established serialized allocation and transition invariants. It did not establish thread safety, broker interoperability or application exactly-once execution. A reusable Packet Identifier is permission to begin a new protocol exchange, not a durable identity for the business event carried by the old one.
