ConstolveAgent reliability evidence
Sign inSee the evidence
How it works

Three ways in, one workflow

Whichever mode you start from, the work converges: intake, audit, contract, test, run, analyze, lock regression, patch, replay proof.

  1. Step 1
    Intake

    Declare the agent and its environment.

  2. Step 2
    Audit

    Present, partial, or missing — stated per input.

  3. Step 3
    Contract

    Expert-approved obligations and limits.

  4. Step 4
    Test and run

    Cases executed repeatedly against declared tools.

  5. Step 5
    Analyze and lock

    Failures grouped, confirmed ones locked as regressions.

  6. Step 6
    Patch and prove

    Baseline versus patched replay, decision recorded.

Mode 1 · Pre-deployment

Evaluate an existing agent

You have an agent that runs. You want to know how it behaves under repetition and pressure before release.

Required inputs
  • Agent policy or system instructionsRequired
  • Tool schemas with inputs and outputsRequired
  • A runnable interface to the agentRequired
  • Sandbox environment or recorded tool responsesRequired
  • Known business rules and escalation policyHelpful
Readiness audit
  • Each input is marked present, partial, or missing.
  • Partial tool schemas are named individually, not averaged into a score.
  • The audit result decides which runs are even possible.
Contract review
  • Clauses are drafted from the policy and cited back to the source line.
  • Anything not supported by a citation is marked unresolved, not inferred silently.
  • A domain expert approves, requests changes, or marks a clause unknown.
Output
  • Test cases tied to specific clauses.
  • Repeated runs with structural checks and an outcome distribution.
  • Failure clusters explained in plain language, with traces attached.
If required evidence is missing
Tool schemas incomplete

Readiness: needs tool schemas. Cases touching those tools are blocked.

No sandbox and no recorded responses

Readiness: needs sandbox data. Nothing is executed.

Policy present but ambiguous

Contract clauses marked unresolved and routed to expert review.

Blocked inputs stay blocked. Nothing is invented to keep a run moving.

Mode 2 · Post-deployment

Replay a production incident

Something went wrong in production. You want to understand it, and to stop it coming back.

Required inputs
  • Production trace with steps and tool callsRequired
  • Tool inputs and outputs inside the traceRequired
  • Agent policy version used at the timeHelpful
  • Tool schemasHelpful
Readiness audit
  • A trace-readiness scorecard checks event completeness, tool I/O, and version attribution.
  • The outcome is one of: replay ready, recorded replay only, observability only, or blocked.
  • Constolve never fabricates a tool response to make a replay run.
Contract review
  • The policy in force at incident time is reconstructed where possible.
  • Clauses the incident touched are flagged for review before any fix is proposed.
Output
  • Baseline replay timeline of what actually happened.
  • Counterfactual replays under safe controls: tool timeout, missing response field, permission denied, stale response, declared state change.
  • A side-by-side diff, and a locked regression case if the failure is confirmed.
If required evidence is missing
Tool responses absent from the trace

Blocked: missing tool responses. Observability only.

Tool schema unknown

Blocked: missing tool schema. Counterfactuals unavailable.

Policy version not recorded

Needs policy review before the failure is attributed.

Blocked inputs stay blocked. Nothing is invented to keep a run moving.

Mode 3 · Contract-first

Build from an expert contract

The agent does not exist yet. A domain expert defines what it must and must not do, before code.

Required inputs
  • Domain expert availabilityRequired
  • Obligations and forbidden actionsRequired
  • Tool inventory and rulesRequired
  • Escalation conditionsRequired
  • Declared world assumptionsRequired
Readiness audit
  • Open questions are listed as gaps with a named decision owner.
  • No gap is closed by assumption; it stays open until a person decides.
Contract review
  • The contract is the primary artifact, written before the agent.
  • Each clause carries its own review state and reviewer.
Output
  • An approved contract with obligations, forbidden actions, ordering, and escalation.
  • A verification plan mapping each clause to how it will be checked.
  • A release-readiness view that stays blocked while gaps are open.
If required evidence is missing
Open gap with no decision owner

Release readiness stays blocked.

World assumption undeclared

Clause marked unknown; verification for it is inconclusive.

Blocked inputs stay blocked. Nothing is invented to keep a run moving.

Not sure which mode fits? Tell us what evidence you already have and we will map it for you.

Request a pilotExplore the demo