Idempotency: How to Prevent Duplicate Automation Runs

Idempotency For Automation

Idempotency means an automation run can be repeated without changing the final outcome after the first successful effect. In practice, you design each action so a second attempt with the same intent does not create a second patient message, a second invoice line, or a second database row. This matters when retries happen due to timeouts, webhook redelivery, queue reprocessing, or operator re-runs after a partial failure.

Consider a workflow that sends a “lab results ready” notification. If the webhook arrives twice, a non-idempotent system sends two notifications. If the workflow stores a record keyed by the notification intent, the second run detects the prior completion and stops. The same pattern applies to updating a medication list, generating a prior authorization draft, or syncing appointment status between systems.

Idempotency is not the same as “at most once delivery” from a messaging system. Many systems deliver at least once, and duplicates are expected. Your job is to make the downstream effect safe under duplicates, even when the upstream system retries.

Why Duplicate Runs Happen

Duplicate automation runs usually come from retries and replays, not from malicious behavior. A common trigger is a timeout: the caller does not receive a response, retries the request, and the original action already completed. Another trigger is webhook redelivery: some platforms resend events when they do not get a 2xx response quickly enough.

People also get duplicates from “manual re-run” habits. An operator sees a partial failure, clicks rerun, and the automation repeats steps that already succeeded. When the workflow mixes read steps and write steps without a clear boundary, it becomes hard to tell which writes already happened. I’ve seen teams treat “step succeeded” as a guarantee, then later discover the step succeeded but the final commit failed, leaving the system in a half-finished state.

Supporting technologies shape the failure modes. Message queues, webhook handlers, and job schedulers each have their own retry logic and delivery guarantees. Database transactions help, but they do not automatically prevent duplicates across multiple services. If your automation spans an API call, a database write, and a third-party email send, you need idempotency at the level of the overall intent, not only inside one component.

Solutions And Practical Advice

Use Idempotency Keys Per Intent

Assign an idempotency key that represents the business intent of the automation. For example, “send lab results notification for patient X at timestamp T for report R” can map to a key like patientId + reportId + notificationType. Store the key and the outcome in a durable place, then check it before performing side effects.

In an API design, the idempotency key travels with the request. In a workflow engine, the key can be computed at the start of the run and attached to every downstream write. If you use a database, a unique constraint on a table column holding the key prevents two workers from both creating the same record. When I audit systems, I look for a unique index rather than only an application-level “check then insert,” because concurrent workers can race.

Realistic outcome: with a unique constraint and a stored result, duplicate webhook deliveries typically collapse into one side effect. The remaining risk becomes “partial completion,” which you handle with careful transaction boundaries and status tracking.

Track State With A Dedup Table

Create a deduplication table that records each idempotency key, the current status, timestamps, and a pointer to the created resource. Status values like “processing,” “succeeded,” and “failed” help you decide what to do on retries. If a retry arrives while the first run is still “processing,” you can either wait, return the prior result, or mark the retry as a no-op depending on your tolerance.

Keep the dedup record in the same database as the side-effect record when possible. If you must call external services, store the dedup record first, then perform the external call, then update the dedup status in a second step. That sequence reduces the chance that a retry starts from scratch after the external call already happened.

Small detail that matters: choose a retention window for dedup records. Many teams keep keys for 30–180 days to cover retry delays and operator re-runs; longer retention increases storage and index size. If you use PostgreSQL, a partial index on “succeeded” rows can keep lookups fast when the table grows.

Make Writes Atomic And Side Effects Guarded

Use database transactions for the writes that must be consistent. For example, inserting an “appointment confirmed” row and updating the patient’s appointment status should happen in one transaction. Then guard side effects like email or SMS with the dedup status so they only fire once.

When you cannot make the entire workflow atomic, separate “state changes” from “external side effects.” A common pattern is: (1) write the dedup record and the internal state in a transaction, (2) enqueue or call external side effects based on that committed state, (3) mark the dedup record as “side effect sent.” If the external call times out, the next run sees “side effect not sent” and retries only that part.

Realistic outcome: you reduce duplicates from “send twice” to “send once, retry until success.” The remaining risk is if the external provider itself duplicates sends; in that case, you also need provider-level idempotency or message deduplication, when available.

Handle Retries With Clear Policies

Define retry behavior for each failure class. Network timeouts often warrant retry, while validation errors should not. If you treat all errors as retryable, you create duplicate side effects when the first attempt actually succeeded but the response failed.

Use correlation IDs and structured logs so you can trace a key across runs. A correlation ID helps you confirm whether a duplicate is truly a duplicate or a “same intent, different key” bug. I’ve debugged cases where the idempotency key included a field that changed between attempts, like “request createdAt,” which made every retry look unique.

Set a maximum retry count and a backoff strategy. Many systems use exponential backoff with jitter; the exact numbers depend on your queue and provider limits. If you see repeated failures, stop retrying and route to a human review queue rather than hammering the same external endpoint.

Case Examples

Webhook Redelivery For Patient Updates

An integration receives appointment status webhooks from a scheduling system. The webhook handler triggers an automation that updates the appointment record and sends a confirmation message. The scheduling system redelivers the same event when the handler returns a 504 timeout after 30 seconds.

The team adds an idempotency key built from eventId + patientId + statusType. They create a dedup table with a unique index on the key. On the first run, the automation writes the internal appointment status and marks the dedup record as “succeeded,” then sends the confirmation message and updates “message sent.” On the redelivery, the handler sees the dedup key already succeeded and skips both the internal update and the message send.

Result: the appointment status remains correct, and the confirmation message count stays at one even when the webhook repeats.

Manual Re-Run After Partial Failure

A billing workflow generates an invoice draft, then calls an external payment system to create a payment intent. The invoice draft creation succeeds, but the payment system call times out. An operator reruns the automation from the dashboard, which repeats the invoice draft creation and creates a duplicate draft.

The fix uses an idempotency key derived from patientId + billingPeriod + invoiceType. The automation first creates or reuses the invoice draft record keyed by that idempotency key. The payment intent step checks the dedup status “payment intent created” before calling the external system again. When the operator reruns, the workflow reuses the existing invoice draft and only retries the payment intent creation.

Result: reruns become safe, and the team can recover from timeouts without manual cleanup.

Checklist And Comparison

Use this decision support checklist to choose where idempotency belongs in your automation. The goal is to prevent duplicate side effects, not to hide failures.

Step-by-step checklist for a new automation:

1. List every side effect that must happen once (messages, invoice creation, record transitions).
2. Define an idempotency key from stable identifiers that do not change between retries.
3. Create a dedup record with a unique constraint on the key.
4. Write internal state in a transaction before calling external services.
5. Guard each external side effect with dedup status so retries only repeat the missing part.
6. Log the idempotency key and correlation ID in every step; version your workflow logic (for example, “workflow v1.3”) so you can compare behavior across deployments.
7. Set retry limits and backoff; route persistent failures to manual review.

Common Mistakes

One frequent mistake is generating the idempotency key from data that changes across attempts. If the key includes “current time” or a random request ID, every retry becomes a new intent. Another mistake is relying on “check then insert” without a unique constraint, which fails under concurrency when two workers run at the same time.

Teams also forget that idempotency must cover the full intent, not only the first step. If your workflow writes a record and then sends a message, you need deduplication for the message send too. Otherwise, a retry after the message send timeout can create a second message even though the internal record already exists.

Another practical issue is mixing “processing” and “succeeded” states without clear rules. If you mark “processing” too early and never update it, retries may skip work forever. If you mark “succeeded” before the external side effect completes, retries may skip the missing external action and leave the system inconsistent.

I’ve also seen teams store dedup keys in memory caches. Cache eviction turns idempotency into a best-effort feature, and duplicates return after restarts. Durable storage matters because retries can occur after deployments, autoscaling events, or queue replays.

FAQ

What Is An Idempotency Key?

An idempotency key is a stable identifier that represents the intent of an automation run, such as “send notification for reportId.” The system stores the key and the outcome so repeated runs with the same key do not repeat side effects.

How Do I Choose A Stable Key?

Use identifiers that do not change across retries, like patientId, eventId, reportId, and action type. Avoid including fields that vary per attempt, such as timestamps, random request IDs, or “lastUpdated” values.

Does Idempotency Replace Retries?

No. Idempotency makes retries safe when the first attempt may have partially completed. Retry policies still control how long you keep trying and which errors trigger retries.

What If The External Provider Has No Idempotency?

Use your own deduplication to prevent repeated calls, and track “side effect sent” status. If the provider can still duplicate internally, you may need provider-specific deduplication features or reconciliation steps based on provider message IDs.

How Long Should I Keep Dedup Records?

Keep them long enough to cover your maximum retry and redelivery windows plus operator re-runs. Many teams use 30–180 days, but the right number depends on queue settings and webhook retry behavior.

Author's Insight

Idempotency design is mostly about failure modeling: timeouts, redeliveries, concurrency, and partial completion. A durable dedup record with a unique constraint on the idempotency key prevents the most common duplicate outcomes. Status tracking turns “retry” from a blind repeat into a targeted continuation of the missing step.

When I review automation specs, I look for one detail that often gets missed: the idempotency key must remain stable across retries and across workflow versions. If a workflow changes its key format, old retries can bypass deduplication and create duplicates.

For health-adjacent workflows, treat idempotency as a data integrity control, not a logging feature. The system should behave safely even when the same event arrives twice or when an operator reruns a job after a timeout.

Key Takeaways

• Idempotency prevents duplicate side effects by tying each automation intent to a stable idempotency key.
• Use durable deduplication with a unique constraint, then track status so retries repeat only the missing work.
• Make internal writes atomic and guard external calls with dedup status to handle partial failures.
• Define retry policies by error class and log the idempotency key for traceability.