← Home

LLM · RETRIEVAL · HUMAN REVIEW

AI Automation & Evaluation Lab

Turn a business request into validated requirements, inspect supporting evidence and approve a simulated action.

Independent working demonstration using synthetic data. Not a client deployment.

Request → real LLM → evidence → validation → your decision → simulated action

Try the clear request, inspect its citations, approve the proposed task, then repeat it. The action key stays the same. Compare with the injection or missing-requirements scenario.

I have a similar task →
Runtime and versions
{
  "provider": "Cloudflare Workers AI",
  "model": "@cf/meta/llama-3.2-3b-instruct",
  "prompt": "ai-request/1",
  "schema": "ai-output/1",
  "retrieval": "lexical-idf/1",
  "suite": "portfolio-ai/1"
}

Preset inputs only. No uploads, real customer data or external mutations. Up to 40 provider attempts per day across this demo; at most two attempts per request. Evidence expires after 24 hours. Repeated presets reuse the session result.

Synthetic source requests are in English; the knowledge base is localized. The interface and requested model prose use your selected language. Source quotes and API field names retain their original text.

Input

Business request / retrieval query

Synthetic request from Demo Northstar, contact Alex Demo: synchronize HubSpot customer updates to Demo Billing API. Data objects: customers and invoices. Compare invoice identifiers, currency and net amounts. Use a stable request ID, route failed updates to review. Normal urgency. Propose create_task only. Task scope: prepare a review task for a one-way daily customer sync, keyed by customer ID. The owner is Alex Demo. Acceptance: compare invoice identifiers, currency and net amounts; flag differences for review. The target is a synthetic test environment. All requirements needed to prepare this task are supplied.

Synthetic source requests are in English; the knowledge base is localized. The interface and requested model prose use your selected language. Source quotes and API field names retain their original text.

Inspect the synthetic knowledge base

API-01 · Integration scope

Agree source and target API contracts before implementing a customer synchronization.

BILL-01 · Billing controls

Compare invoice identifiers, currency and net amounts; route differences to human review.

RETRY-01 · Retry policy

Retry explicit transient failures at most once. An ambiguous timeout remains UNKNOWN until reviewed.

IDEM-01 · Duplicate prevention

Use a stable request identifier and a unique approved-action key to prevent duplicate actions.

SEC-01 · Access policy

Do not put credentials in requests. All downstream actions in this lab are simulations requiring human approval.

MIG-01 · Migration checklist

Validate identifiers, required fields and control totals before moving records; retain rejected-record evidence.

ACC-01 · Acceptance criteria

Define expected outcomes and test both successful and rejected inputs before acceptance.

SUP-01 · Escalation policy

Priority requests and unresolved requirements must be reviewed by a person before an action is approved.

REV-01 · Review policy

A task needs a named review contact. Missing scope, ambiguous system identity or conflicting policies requires clarification, not automatic approval.

DATA-01 · Synthetic-data policy

Only synthetic records may be used in this demonstration. No real CRM or email system is connected.