Deduplicate notifications by the caller's key, not the message text

20 September 2026 · 7 min read

A retried dispatch and a double-fired event look the same from inside a notification service: same recipient, same template, same rendered body. Deduplicating on that body suppresses the wrong things. It collapses two genuinely different events that happen to be worded identically, and it stops matching a retry as soon as the text shifts by one variant. The only thing that separates "send this once" from "send this twice" is a fact the caller holds: which event this is.

The three cases a sender has to tell apart#

The first two rows want suppression, the third wants delivery, and the rendered string is identical in all three. No function of the message body can split that table, because the information that splits it is not in the body. Text-based deduplication answers a different question than the one you are asking.

Text equality hashes the wrong thing#

Rendered text is a function of the template, the context and the locale, and both ends of that function move under you.

In one direction, a template compresses an event into a sentence, so distinct events collapse onto the same output. In notify-dispatch a shipment notification is one template with a variant per language:

from notify_dispatch import LocalizedTemplate, Variable

shipped = LocalizedTemplate.from_texts(
    "order_shipped",
    {
        "en": "Order {order_id} is on its way.",
        "de": "Bestellung {order_id} ist unterwegs.",
        "pt-BR": "O pedido {order_id} está a caminho.",
    },
    variables=(Variable("order_id", str),),
)

shipped.render({"order_id": "A-1042"}, locale="pt_br")
# 'O pedido A-1042 está a caminho.'

The original parcel and the replacement parcel for the same order render the same string. A text-keyed store calls the second one a duplicate, and the customer never learns the replacement is moving.

In the other direction, the same event renders differently over time. Locale resolution drops one subtag at a time and ends at the default locale, and the receipt records which variant was actually used:

receipt = dispatcher.send(
    Recipient(phone="+49151000000", locale="de-AT"), shipped, {"order_id": "A-1042"}
)
receipt.message.locale  # 'de' (there is no de-AT variant, so the language matched)

Add a de-AT variant between the first attempt and its retry and the text changes while the event does not. The same happens when a copy fix lands mid-incident, or when a fallback to a different channel rewrites the wording. Text-keyed deduplication stops matching at exactly the moment a translation or an edit ships, so it fails quietly and on a schedule nobody controls.

A key is a fact, not a digest#

The caller already knows what happened. It has an order, a parcel, a payment, and it knows which event it is handling. That pair is the key:

key = f"{order.id}:order_confirmed"

Two properties make it work. First, it comes only from state that already exists and does not change under retry: no uuid4(), no timestamp, no hash of the body. A key minted at the call site is not a key, it is a new event every time, and the deduplication layer turns into an expensive no-op that everyone believes in.

Second, its granularity matches the real-world thing that may only happen once. If an order can ship in two parcels, order.id plus order_shipped is too coarse and the second parcel gets swallowed. The key is then the parcel identifier plus the event. If a payment can be retried by the customer, the attempt identifier is the unit, not the order.

A library cannot make that granularity decision for you. Nothing inside a notification package knows whether "order shipped" means once per order or once per parcel in your business, and guessing produces the worst failure mode available: a message that was never sent and never logged as skipped. So the key is supplied per notification by the caller, and it is checked before a provider is ever contacted. The only correct default is no default.

The check belongs before the providers#

notify-dispatch already has this ordering rule for template contracts. A missing variable is a programming error, and it surfaces without anyone's phone ringing:

def test_template_errors_surface_before_any_provider_is_called():
    sms = Provider()
    dispatcher = Dispatcher([SmsAdapter(sms)])

    with pytest.raises(MissingVariableError):
        dispatcher.send(Recipient(phone="+491"), ORDER_CONFIRMED, {})
    assert not sms.calls

The Provider in those tests is a recording double: it appends every call it receives, so assert not sms.calls is a statement about the outside world, not about a return value. Deduplication tests have the same shape and the same assertion. Send twice with one key, and the proof is that the provider heard about it once.

This matters beyond tidiness. The dispatcher picks the first channel the recipient can be reached on, and falls through when a channel has no address. One send can therefore touch more than one candidate channel while remaining one notification. If the deduplication record is written per provider attempt rather than per send, internal fallback starts looking like a duplicate to your own system.

"Not delivered" is not "already sent"#

The sharp edge is what happens when a send does not complete. Three outcomes in this API are not deliveries:

• UnroutableError: no candidate channel had both an address and a registered adapter.
• QuietHoursError: every channel the recipient allows is inside a do-not-disturb window and the send was not marked Urgency.URGENT.
• DeliveryError: the provider itself failed, wrapped with the channel and address it failed on.

If the key is consumed on the way in and never released, all three of these turn a recoverable failure into a message the customer will never receive, because the retry that was supposed to fix it now looks like a duplicate. If the key is only recorded after a confirmed send, two workers racing on the same event can both pass the check.

No configuration removes this trade-off. There is only a decision about which failure you prefer, made once and written down. For notifications about money moving, we reserve the key first and release it on a failure that is known not to have reached the provider, keeping a provider timeout (where we genuinely do not know) on the at-most-once side. A duplicate shipping notice is an annoyance, while a duplicate payout alert costs a support conversation and some trust.

What it costs to run#

A key store is a growing set of small rows, sized by events rather than by messages, and it needs exactly one tuning decision: how long a key is remembered. The retention window has to be longer than the longest path a retry can take, and that is not the queue's default backoff, it is the manual replay a week later when someone reprocesses a day of events. Shorter than that, and duplicates come back precisely during an incident, when nobody has the attention to spot them.

Two details save more grief than they cost. Namespace the keys per environment, so a staging replay cannot suppress a production send. And keep the key readable: A-1042:order_confirmed tells an on-call engineer what was suppressed and why, while a digest tells them nothing at three in the morning.

The rule fits in a sentence. The body answers "what does this say", and deduplication needs the answer to "which event is this". Only one of those is available to the sender, and it is not the one on the screen.