Error Recovery — Errors Are Not the End, They're the Start of a Retry

"Errors are not the endpoint, they're the starting point for a retry" — upgrade tokens, compress context, switch models.

The Problem

In previous articles, we mentioned a very important point: anything involving AI has a probability of error, so you need to prepare a mechanism for how to respond after errors.

Especially since previously it was just some peripheral modules erroring—but what if the entire product errors out? If an agent errors out or crashes, and it doesn't retry, or switch models, or even reduce and compress the context, but just errors out directly—then the user experience is absolutely catastrophic!

Generally speaking, API errors are the most common, mainly including three types:

• Output truncated: it got cut off halfway because the tokens ran out;

• Context over limit: still too long after compression, and it blows up anyway;

• Temporary failure: for example, the server suddenly gets overloaded, etc.;

Just as the original text says: an Agent that doesn't handle errors is like a car that stalls at the slightest touch.

The Solution

The solution is actually the same as when it's not AI: think through a corresponding strategy for every place where a failure can occur, then require the AI to execute according to the strategy and rerun the loop.

The three most common recovery modes—of course, there are actually far more than what's listed here, but we're not professional engineers, so we just need to understand roughly what's going on:

How It Works

Output Truncated

We should all know that model conversation generation has a maximum token limit, representing the maximum consumption of a single conversation. But if the task is too complex and has already reached the limit, then the output will be truncated.

⚠️ Please distinguish between 2 kinds of output truncation. Here it's not that the user has run out of credit, but that the set maximum token consumption limit for a single conversation has been reached. If it's out of credit, generally you stop generating and tell the user they're out of credit, like what CC does.

Generally, a multi-step strategy is adopted:

• The first time it happens, use an 8x space expansion, meaning expand the maximum token limit to 8 times the original, then retry the same request. If it's still not enough in this case, then save the content of the truncated output and start continuing the writing.

• Continuation isn't infinite either, because as we said before, models have context limits. If you continue too much, there's almost no attention left, so generally it won't exceed 3 times.

• If even this can't be resolved, then just give up, exit the loop, and tell the user it's dead.

Context Over Limit

As mentioned before, it's possible that L4 execution fails, and if it still fails after 3 polls, it ends; or even after L4 has fully run, it's still over. This must trigger the "fallback", which is the emergency compression mechanism.

This scheme was introduced before, so I won't go into too much detail again.

Temporary Failure

There are so many of these that it's hard to enumerate them all. But generally there are jitter strategies to prevent network fluctuations, as well as switching to backup models and degradation strategies, etc. I won't go into too much detail here either.