What happens when an agent worker crashes?
Exploring checkpoints, retries, and the gap between executing a tool and recording its result.
The question
An agent makes a tool call. The tool succeeds, but the worker crashes before saving the result. When the task is retried, how does the system know whether it is safe to call that tool again?
This example entry outlines an experiment. It does not describe a completed project.
The experiment
Start with a small task runner and introduce a deliberate failure at three points:
- Before the tool call.
- After the tool call, but before the checkpoint.
- After the checkpoint, but before the task is acknowledged.
Record which actions run again after the worker restarts.
The important boundary
A checkpoint records what the runner knows. It cannot make a separate external side effect atomic. A stable idempotency key helps when the tool supports one; otherwise the runner needs reconciliation or explicit human review for an ambiguous result.
“Retry the task” is a policy. “Is this action safe to repeat?” is a property of the action.
What a useful follow-up would contain
Include the failure trace, the resulting state, and the change that prevents an unintended duplicate. Link to the implementation and show the limits that remain.