Prevent Duplicate Posts with Idempotent Publishing

Expect the same job to return

Cloudflare Queues documents at-least-once delivery, so a consumer must account for repeated messages. Even without a queue, an HTTP timeout can leave the sender unsure whether a remote CMS created a post. Idempotency means repeating the same intended operation does not multiply its effect. It is a recommended property of the publishing system, not a promise that every CMS provides a native idempotency feature. For a hypothetical gardening store, retrying an article about indoor herbs should update or confirm the existing result rather than create another article with a numbered slug.

Give the operation a stable identity

Create an operation identifier before the first attempt and reuse it for retries of that same operation. Include the destination and intended revision in the identity model so two sites or distinct edits do not collide. Keep article identity separate from operation identity: one article can have many legitimate update operations. A database uniqueness rule can prevent competing workers from independently claiming the same operation, but design the state transitions carefully. A row saying started is not proof that the remote publication completed, and a crash can leave that intermediate state behind.

Persist remote identifiers immediately

When a CMS returns a post identity, store it with the local article and operation. Subsequent updates should target that record. WordPress exposes individual post operations, providing a concrete destination identity to track. If the first request times out before the identifier is recorded, use your defined reconciliation method before creating again. That might involve a supported external reference, a controlled lookup, or an operator review. Do not assume a slug search is always an unambiguous identifier: editors can change slugs and unrelated articles can use similar names.

Make state transitions recoverable

Use explicit states for pending, processing, destination created, verification pending, and completed, adapting the names to your product. Save enough evidence at each boundary to resume safely. A worker that restarts after creating the remote draft should continue from that draft, not restart the entire process. Prevent concurrent retries from racing to publish conflicting revisions. For the gardening store, two workers receiving the same herb guide must converge on one remote article. Design the recovery path alongside the happy path rather than adding a duplicate-cleanup script after launch.

Test ambiguity instead of only rejection

The most revealing test is a successful remote create followed by a lost response. Simulate that condition and confirm the retry does not produce another page. Also test duplicate queue messages, overlapping worker attempts, and a delayed old update arriving after a newer one. Record the operation identity in safe logs so the sequence is reconstructable. A clean status dashboard should distinguish completed, definitely failed, and uncertain outcomes. When uncertainty remains, pause for reconciliation rather than multiplying public content. Reliability comes from preserving identity across failures, not from increasing the number of retries.

Make the duplicate test part of integration acceptance, not just queue testing. The same operation can be repeated by a browser action, operator recovery, or network library even when the queue itself behaves as expected.