Overlapping MQTT Subscriptions: Use Subscription Identifiers to Explain Duplicate Deliveries

When one publication matches several subscriptions in the same client session, multiple application deliveries can be legitimate even without a retransmission. Record the matching filters and MQTT 5 Subscription Identifiers before blaming QoS retries. Preserve identifier multiplicity: repeated identifiers can describe multiple matching subscriptions rather than a malformed packet.

This guide pins the OASIS MQTT 5.0 Standard, 7 March 2019, focusing on nonshared overlapping subscriptions. An original filter/identifier model was executed for the stated examples. No broker, client callback behavior, network trace or full wildcard parser was tested.

Build a filter-to-identifier ledger

For each successful subscription, record the exact topic filter, Subscription Identifier if supplied, requested and granted QoS, relevant options and subscription generation. Keep that ledger separate from the Packet Identifier used to correlate a SUBSCRIBE with its SUBACK.

A Subscription Identifier labels why a server-to-client PUBLISH matched this client’s subscription state. A Packet Identifier belongs to a particular protocol exchange. An application event identifier belongs to the publisher’s data contract. None is a general replacement for either of the others.

Start with one publication and two filters

Consider the following hypothetical subscription state:

Topic filter	Identifier	Matches factory/device07/temperature?
factory/+/temperature	11	Yes
factory/device07/#	22	Yes
factory/device08/#	33	No

Sections 3.3.2.3.8 and 3.3.4 explain the relevant delivery behavior. A single delivered copy includes the identifiers of all matching subscriptions that have them, in unspecified order. The server may also deliver additional copies for additional matching subscriptions, carrying the applicable identifiers.

Therefore a PUBLISH with identifiers eleven and twenty-two is not two payloads hidden in one packet. It is one delivery with two matching-subscription explanations. Two deliveries associated with those separate matching subscriptions can also be a permitted result. Do not write a test oracle that requires exactly one callback unless the broker/client combination supplies a narrower, documented contract.

Preserve repeated identifiers

A client can choose the same Subscription Identifier for more than one subscription. When both match, a delivered packet can contain that value more than once. Converting the parsed identifier list to a set destroys the distinction between one match and several matches using the same label.

Also remember that the identifier is a property of a SUBSCRIBE packet, not an independent field beside each filter in that packet. If one request supplies several filters under one identifier, they share that label. Use separate requests and distinct identifiers when the diagnostic requirement is to identify each filter unambiguously.

The executed model matched the example topic to identifiers eleven and twenty-two, accepted either order in a combined vector, and compared that explanation with separate single-identifier vectors. Reusing identifier seven for both matching filters produced the list seven, seven. These are model assertions, not observed packets from a broker.

Do not use DUP as the only duplicate classifier

Subscription overlap is a different cause from retransmission of one delivery exchange. Preserve the application event identity, connection generation, Packet Identifier where present, DUP flag and complete Subscription Identifier list. Interpret them together rather than treating one bit as a universal duplicate detector.

For a hypothetical dashboard client, a broad telemetry subscription and a later device-specific subscription might both feed the same update handler. The fix could be to remove unnecessary overlap, route by explicit subscription purpose, or deduplicate effects by application identity. Choosing among those depends on whether two handlers were intentionally meant to receive the same event.

Do not deduplicate distinct sensor events merely because their payload values match. Conversely, two packets with different protocol identifiers can still represent the same published application event. Keep the business identity rule separate from the subscription diagnostic.

Handle subscription changes without inventing stale-map errors

Check the server’s advertised Subscription Identifier support before relying on the feature. Preserve per-filter SUBACK outcomes; a requested filter that was rejected must not be treated as active merely because another filter in the same request succeeded.

The server may send matching PUBLISH packets before the SUBACK. Arrange client-side routing so an identifier requested in a pending subscription can be recognized without prematurely claiming that every requested filter has succeeded. This matters when the callback layer otherwise looks up only a post-SUBACK table.

On replacement, retain enough generation information to interpret in-flight deliveries. Section 3.8.4 permits old or new identifiers in specified retransmission circumstances after a subscription is remade. Once the server has sent a PUBLISH containing the new identifier, it must not revert to the old one. A changed identifier alone does not prove broker corruption. Avoid immediately reusing an old label for an unrelated handler without accounting for outstanding traffic.

An isolated overlap qualification

1. Use a test publisher that adds a unique application event identity to each publication.
2. Subscribe to one filter and establish the baseline delivery and identifier evidence.
3. Add the overlapping filter with a distinct identifier and publish one new event.
4. Record every packet and callback, including the identifier list with repeated values preserved.
5. Repeat with the same identifier assigned to both filters, then with one filter removed.
6. Exercise reconnect and filter replacement while retaining the event and subscription-generation ledger.

Keep the first tests free of injected transport loss so overlap is the controlled variable. Add retransmission scenarios only after the matching-filter explanation is established. Acceptance requires every observed delivery to fit the active or permitted in-flight subscription state, with application effects handled according to their own identity contract. The goal is an explained delivery count, not an assumption that any repeated payload proves a network retry.