Human review & kill switch

Match review to action risk, preserve meaningful reviewer choices, sample bounded autonomy, and suppress new or in-flight AI work instantly by scope.

Human oversight is useful only when the reviewer has the authority, context, time, and genuine ability to change the outcome. LendEasy does not turn “click approve” into a universal safety claim.

Review modes

Mode	When used	Human decision
Pre-action approval	High-impact or A2 prepared actions.	Accept, edit, reject, escalate, or request facts before effect.
Concurrent supervision	Live voice or complex interaction where configured.	Take over, pause tool access, or terminate session.
Post-action sampling	Qualified A3 low-risk actions.	Confirm, coach, correct downstream state, or open incident.
Investigation review	Quality, complaint, drift, or policy trigger.	Determine root cause and containment.

The action policy—not merely “AI touched it”—selects review mode. Some summaries can be sampled; a loan modification may require distinct approval regardless of author.

Review record

The implemented POST /v1/servicing/ai/outputs/{outputId}/review accepts, edits, rejects, ignores, or escalates a grounded output and retains the review evidence. The original output remains immutable beside the edited or accepted result.

{
  "reviewOutcome": "EDITED",
  "finalContent": "Customer missed the June payment and has no active bankruptcy restriction.",
  "reason": "Removed unsupported speculation.",
  "evidenceIds": [8801]
}

Reviewers see cited facts and freshness, not only polished prose. A missing citation or expired fact blocks acceptance where policy requires grounding.

Sampling

Sampling combines random coverage with risk strata:

new model, prompt, tool, or policy versions;
regulated or sensitive case types;
low confidence or missing-fact recovery;
reviewer edits and reversals;
customer complaints and opt-outs;
subgroup and language cohorts;
rare actions with high severity.

Metrics remain segmented by action, model, prompt, policy, cohort, and reviewer. Acceptance rate alone is not a quality metric.

Reviewers read the output through GET /v1/servicing/ai/outputs/{outputId}, which returns the proposed content, confidence, groundingStatus, and — once reviewed — the review evidence and reviewer identity. It deliberately excludes prompts, model identifiers, secrets, and deployment configuration: a reviewer judges the grounded claim, not the machinery that produced it. Registered session provenance travels with the output so the judgement can still be attributed to a runtime and policy version.

Suppression scopes

A kill switch can stop all AI work or a narrow slice by tenant, agent profile, model version, queue, case type, task type, channel, tool, or action. Dispatch checks suppression before starting; active sessions check it before every tool call.

The implemented control is POST /v1/servicing/admin/ai-suppressions, and the same route both suppresses and restores:

curl -X POST "$BASE/v1/servicing/admin/ai-suppressions" \
  -H "Authorization: Bearer $TOKEN" \
  -H "LendEasy-Tenant: demo-lender" \
  -H "Content-Type: application/json" \
  -d '{
    "scopeType": "QUEUE",
    "scopeRef": "COLLECTIONS",
    "state": "SUPPRESSED",
    "reason": "Pause AI dispatch during incident review."
  }'

scopeType comes from a closed set, so a caller cannot invent a scope the runtime does not evaluate. reason is required in both directions — restoring is as much an operational decision as stopping. New dispatch and in-flight work both recheck current suppression, which is why the switch takes effect without waiting for sessions to drain.

ADMIN_SERVICING_AI authorizes this route. It is distinct from REVIEW_SERVICING_AI_OUTPUT: a reviewer judging output does not thereby hold the kill switch, and an administrator holding the switch does not thereby approve work.

suppression activated
  ├─ queued sessions → declined
  ├─ active generation → stopped at boundary
  ├─ future tool call → rejected
  └─ already submitted external effect → monitored/reconciled, never duplicated

Stopping an AI session cannot recall an ACH file or message already accepted by a provider. Those effects follow their owning cancellation and reconciliation state machines.

Restore safely

Restoration records scope, incident/cause, remediation, validation evidence, approver, time, and monitoring plan. Restoring one profile does not implicitly restore a globally suppressed model or tool.

A reviewer and the AI session are always distinct actors. Where maker-checker applies, the reviewer must also satisfy the action’s role and separation-of-duties policy.