The first launch is fine. You're watching it, you catch the odd cases by hand, you're there.
The second launch is mostly fine. You remember what went wrong last time and add a fix or two.
The third launch is when it breaks, and you're not watching, because by now you assume it works.
Serious automation failures rarely happen on the first run. They happen on the third or fourth, once a workflow has earned enough trust to run unsupervised but hasn't been hardened enough to deserve it.
Idempotency: what makes a pipeline survivable
An idempotent operation gives the same result whether it runs once or ten times. For a pipeline, that means a crash halfway through followed by a restart doesn't duplicate data, charge a customer twice or send the same email again.
Most pipelines aren't idempotent by default. They assume a clean start. When they restart mid-run (a network blip, a container restart, a dependent service timing out), they redo work that was already done and create duplicates or half-written records.
The fix is to create idempotency keys early and store them. Before any write (a CRM update, a database insert, an email send), check whether this item has already been processed. A simple processed flag in a lookup table, checked at the start of each loop iteration, removes the whole class of errors caused by re-running.
This matters most for payments, lead creation, and anything where a duplicate is visible to a customer.
Retries: when to wait and when to fail fast
Not every error deserves a retry, and the difference matters.
Retryable errors are temporary: the API was briefly down, the network timed out, the rate limit resets in 60 seconds. Retry these with exponential backoff (wait 1 second, then 2, then 4, then 8) with around 20% random jitter, so all your retries don't hit the same server at the same instant. Three to five attempts is usually enough before escalating.
Structural errors aren't temporary. A 401 means the credentials are wrong, and retrying won't fix them. A 422 means the data is malformed, and the same data will fail every time. Fail fast on these and send them to a manual review queue instead of burning retries.
In n8n, we write this as a mapping in a Code node: 429 waits and retries, 5xx retries with backoff, other 4xx go to the error queue, 200 carries on. Make the mapping explicit, not implied.
Checkpoints with a person in them
Not every step should run to completion on its own. Some should pause and wait for a decision.
The usual cases: the first run of a big batch (let someone confirm the first 10 records before you process 50,000), creative approval before ads go out, and writes that are hard to undo, like deleting records, messaging customers or triggering payments.
In n8n, a checkpoint is a Wait node. The workflow pauses until a webhook calls it back or a timer runs out. Pair it with a Slack or email message to the right person: "Pipeline paused at approval. Click here to continue."
That isn't automation failing. It's deliberate design. The aim isn't to take people out of every step. It's to take them out of the steps where they add nothing and keep them where their judgement counts.
Test the failures before launch
What prevents third-launch failures is testing how the pipeline fails before it goes live. Ask these of every pipeline:
- What happens if an API returns a 503 at step 4? Does the whole workflow crash, or does an error handler catch it?
- What happens if a field the next node needs is empty? Does it fail with a clear error, or pass bad data along quietly?
- What happens if the same record is processed twice? Does it duplicate, or does the idempotency check stop it?
- What happens if the error workflow itself fails?
Run each scenario with test data before launch. The execution log shows exactly what each node received and produced, so use it while testing, not just after something breaks in production.
Monitoring
A pipeline that fails silently isn't a production pipeline. It's a hope.
The minimum: a central error workflow that fires on any failed execution, posts to Slack with the workflow name, the failing node, the error message and a link to the execution log, and records the failure in a table. In n8n, that's an error workflow built on the Error Trigger node.
For critical pipelines, add outside monitoring too: an uptime checker on your webhook endpoints, and an on-call tool such as PagerDuty for failures that need someone awake.
The test is simple. If the pipeline fails at 3am on a Sunday, how long until someone knows? If the answer is "when the client asks on Monday", the monitoring isn't finished.
Pipelines that still work on the third launch, and the thirtieth, aren't magic. They have idempotency, a clear retry policy, checkpoints where judgement matters, failure cases tested in advance, and monitoring that tells you straight away when something breaks. Every automation we deliver through our AI automation work is built this way.
