Halyard 1.4 closes the loop between what goes wrong in production and what your test suite knows about. Two features do most of the work.

Trace replay

Every Halyard Cloud trace now has a Replay as eval button. It captures the user input, the tool calls the agent made and the responses it got, and writes them out as a new eval case in your repo. Edit the expected outcome, commit, and that failure can never ship silently again.

Replay uses the same sandbox as your local evals, so writes are recorded instead of executed. Replaying a refund that went wrong won’t refund anyone twice.

Runtime guardrails

The checks you write for evals can now run in production too. Mark an assertion as a guard, such as “refund amount never exceeds the order total”, and Halyard enforces it on live traffic. A guard can log, block the tool call, or hand the conversation to a human.

The best guardrail is a test you already trust.

Also in this release

• Parallel case execution is now the default, cutting median suite time by about 40%.

• Pass-rate history per eval, so a slow decline is visible before it becomes a regression.

• Slack alerts now link straight to the failing trace.

Upgrade with npm install halyard@1.4. Guardrails are available on every Cloud plan, including the free tier.