inAiLoading page content...
inAiLoading page content...
Research program
Agents are decision systems. They plan, act, verify, repair, and resume under explicit autonomy gates, oversight minutes, and audit trails.
This is the deeper overview for the Agentic Decision Systems research direction. The collection page gathers related outputs, protocols, and links.
Data vintage: Oct 2025
Enterprise agents succeed when treated as decision systems with world models, plans, verifiers, repair loops, and auditable traces. Autonomy scales up a ladder—Assist → Approve → Auto-with-review → Auto—bounded by gates on accuracy, oversight minutes, escalation rate, unit cost, and incidents avoided. Production deployments at Klarna, Intercom, Walmart, LILT, and others show "Auto-with-review" is commercially viable for bounded workloads, while compliance-heavy flows remain at "Approve."
| Task class | Current autonomy | Decision accuracy / service KPIs | Oversight / escalation | Source |
|---|---|---|---|---|
| Support triage/resolution (retail fintech) | Auto-with-review | CSAT parity; repeat inquiries −25%; TTR < 2 min; ~⅔ chats handled | ≈33% escalated | Klarna (Aug 2024) |
| Support triage/resolution (HR SaaS) | Auto-with-review | AI resolution 82%; CSAT 85–90% | ≈11% escalated | Intercom Fin (Oct 2024) |
| IT support triage | Approve | 53% deflection; Average resolution time −26.63% | 47% not deflected | Freshservice (2024 aggregate) |
| Catalog ops attribute extraction | Auto-with-review | F1 95.6–97.9; online CTR +2.16%; ATC +1.42%; GMV +0.38% | Exception sampling only | Walmart (May 2024) |
| Localization (enterprise) | Approve | +17.5% accuracy; −20% cost | Editor time proxy ↓ | LILT × Miro (2024) |
Reality check: frontier models still underperform humans on WebArena (~14% vs human 78%), WorkArena++ (~2% vs human 94%), and OSWorld (~29–38% success), so we hold "Auto" for constrained surfaces with verifiers and guardrails that close failure modes.
| Suite | Task type | Input / modality | Eval metrics | Strengths | Caveats |
|---|---|---|---|---|---|
| WebArena | Realistic web tasks (e-comm, forum, CMS, dev) | Browser control; text + vision; tool APIs | Task success, step accuracy | Execution-based; support-like web ops | Early agents far below human; limited auth flows |
| WorkArena++ | Office/enterprise multi-app tasks | Browser + SaaS UIs | Success, efficiency | Targets business workflows & compositional planning | Very low SOTA success; still simulated |
| OSWorld | Real OS apps + web (369 tasks) | Desktop + web | Success, execution-based | Closest to real computer use | Setup complexity; lab sandbox |
| BFCL V4 | Tool/function calling | Structured function calls | Call accuracy, cost, latency | Enterprise tool-use predictivity | Abstracts away UI dynamics |
| GAIA | Real-world Qs requiring tools/browse | Tool use + web | Human 92% vs GPT-4 15% | Stress-tests general assistantship | Not fully execution-based |
Numeric gates keep autonomy honest. We publish autonomy requirements per domain so stakeholders know when the ladder can advance and what data support the move.
Walmart production paper reports 95.6–97.9 accuracy and online lifts before tapering oversight.
LILT customer outcomes (+17.5% accuracy, −20% cost) support the "Approve" rung; high-risk remains human-first.
Employment use is high-risk; EU AI Act triggers demand provable oversight.
Illustrative ladder: oversight minutes fall as decision accuracy and repair rate rise; use as guardrails before graduating autonomy.
OpenTelemetry-compatible traces capture every decision. We hash-chain audit stores with write-once retention, logging policy version IDs, redaction policies, and reviewer decisions so regulators and partners can replay and inspect.
Constraints
Every decision references guard-rails, policies, and task-level boundaries before execution.
Source trace
Audit logs preserve prompts, retrieved context, tool calls, and remediation steps for each output.
For inAi product work, this remains a design pattern rather than a public performance claim: catalog workflows, career workflows, and agent-facing tools should use review gates, audit artifacts, drift checks, and rollback paths before any autonomy is increased. Thresholds belong in scoped evaluations, not in broad public product promises.
Data vintage: Oct 2025 · Last updated 01 Oct 2025