Practice: Agent orchestration and durable execution
Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.
Main tool: Temporal · Temporal makes every agent step a recorded event that is replayed after a crash, so orchestration patterns, retries, checkpoints and human approval can all be practised on one laptop with a single engine. · Alternatives: LangGraph checkpoints graph state inside the agent framework itself, AWS Step Functions runs declarative state machines as a managed cloud service, the OpenAI Agents SDK and the Claude Agent SDK coordinate agent handoffs and tools but leave durability to you, and plain Postgres can hold hand-rolled checkpoints with no replay engine.
Every step links into the tool's own documentation; links were checked on 2026-10-11 · version 1
Setup (one time)
1. Read what durable execution means and which failures it is meant to absorb before installing anything.
Every later step tests one claim: a run survives crashes without losing or repeating work. Knowing the claim first tells you what to look for.
Explained in: What is Temporal? | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: You can state in one sentence what happens to a running workflow when its worker process dies.
2. Pick the SDK language you know best and install that SDK and the Temporal command line tool by following its quickstart.
The quickstart installs a local development service, the web UI and the SDK together, with no paid cloud and no GPU.
Explained in: Quickstarts | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The Temporal command line tool prints its version and the SDK imports cleanly in an empty project.
3. Learn the parts of a Temporal application: workflows, activities, workers and the client that starts runs.
Agent code splits along these lines: deterministic orchestration goes in the workflow, model and tool calls go in activities.
Explained in: What is Temporal? | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: You can sketch your planned agent on paper with each box labelled workflow, activity, worker or client.
4. Start the local development service and open its web UI in a browser.
The server holds all run state, so nothing executes until it is up, and the UI is where you will read event histories.
Explained in: Temporal architecture | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The web UI loads and shows an empty list of workflow executions in the default namespace.
Steps
1. Run the quickstart workflow for your SDK from start to finish against the local service.
A first complete run proves the worker, the client and the server are talking before you add any agent logic.
Explained in: Quickstarts | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The web UI shows one workflow execution with status Completed and a result value.
2. Write a workflow that models one agent loop: an activity that calls a stubbed model, then an activity that calls a stubbed tool, repeated until the stub says done.
This is the core object model for agents: the workflow decides which step runs next and what context it gets, while activities do the unreliable work.
Explained in: Temporal Workflow | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: A run completes and its event history lists one scheduled and one completed event per model call and tool call.
3. Run the agent workflow with a slow tool activity, kill the worker process halfway through, then start the worker again.
This is the durability promise tested directly: a long run must continue from where it stopped instead of restarting from the top.
Explained in: Temporal Workflow | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The run finishes after the restart and the already completed model calls appear only once in the history.
4. Compare the event history of the resumed run with your workflow code, line by line, to trace how replay rebuilt its state.
Replay is how a workflow engine restores state without a snapshot. Understanding it explains every rule about what workflow code may do.
Explained in: Temporal Workflow | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: You can point at each history event and name the line of workflow code that produced it.
5. Create a deliberate non-deterministic change by reordering two activity calls in the workflow while a run is still open, then restart the worker.
Deploys happen while agent runs are in flight, and a changed workflow that no longer matches its history fails on replay.
Explained in: Temporal Workflow Definition | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The open run reports a non-determinism error in the worker log or web UI, and it recovers when you revert the change.
6. Write an orchestrator workflow that fans work out to three specialist agent workflows in parallel and merges their results.
Fan-out and fan-in with a supervisor is the most common multi-agent coordination pattern, and the task orchestration patterns show how to express it durably.
Explained in: Temporal Design Patterns | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The web UI shows one parent execution linked to three child executions, and the parent result contains all three outputs.
7. Create a second coordination variant from the AI agent design patterns, such as a router that hands off to one specialist agent chosen by the model output.
Comparing a handoff pattern with the fan-out pattern on the same engine shows which context each agent receives and what each pattern costs.
Explained in: Durable AI | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: Two runs with different inputs each start a different specialist, visible as different child executions in the UI.
8. Configure a retry policy with backoff and a maximum attempt count on the model activity, make the stub fail with a rate limit error twice, and pass an idempotency key to the tool activity.
Retries make rate limits survivable, but a retried tool call that is not idempotent repeats side effects such as sending a message twice.
Explained in: Temporal Design Patterns | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The history shows the model activity succeeding on its third attempt and the tool stub logs exactly one side effect per key.
9. Enable a budget guard that tracks steps and estimated token spend in workflow state and requests cancellation when the cap is exceeded, with cleanup in the activities.
An agent loop without a ceiling can burn money indefinitely. Cancellation stops it cleanly while still letting in-flight work clean up.
Explained in: Cancellation and Termination | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: A run with a low cap ends with status Canceled, and the cleanup step appears in its history.
10. Write a human approval gate: the workflow waits before a risky tool call until an approve or reject message arrives, and a read-only message returns the pending action.
The wait is a durable checkpoint that can last days at no compute cost. Choosing the right message type decides whether the approver gets a confirmed result.
Explained in: Temporal Workflow message passing - Signals, Queries, & Updates | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: A run sits in Running until you send approval from the command line tool, then completes, and a rejection takes the other branch.
11. Compare the persisted event histories of a clean run and a crashed and resumed run of the same agent, then restart the local service with a database file and check that both are still there.
The persistence layer is where checkpoints actually live. Comparing two histories is how you prove no step was lost or repeated.
Explained in: Temporal architecture | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: Both histories show the same sequence of completed activities, and both runs are still listed after a full service restart.
12. Deploy a self-hosted Temporal Service in containers with Postgres as the persistence store, point your workers at it, and rerun the approval and crash tests; this is the local substitute for the paid managed cloud.
Production separates the server, the database and the workers, so repeating the tests against that shape shows what you would operate for real.
Explained in: Temporal Platform production deployments | Temporal Documentation (docs.temporal.io), checked 2026-10-11
Done when: The agent workflow survives a restart of the worker container and of the server container, and its history is readable from the Postgres-backed service.