LM
Last Mile Report
AI agent task-completion benchmark · Where agent journeys actually end.
July 21, 2026

Executive thesis

The scan said ready. The agent still failed.

Static scans look for signals. Agents reveal what actually works.

Cross-model lab report: every run on this page was driven by kimi-k2.5-cloud under the same deterministic rules as the published index — 34/63 measurable tasks completed (54%); 1 infrastructure-errored run(s) excluded, never scored as failures. Each dossier carries the probe evidence (llms.txt discovery, WebMCP tool calls, consent walls) and, for cloud runs, the full session replay.

Run summary
3Sites tested
150Browser tasks
0 / 3Sites completed
100%Verified outcomes
Evaluation flow
01Static scan

Baseline readiness signal.

02Agent run

Real browser task attempt.

03Verified outcome

Binary pass/fail with evidence.

3Sites
150Tasks
0 / 3Sites completed
100%Verified outcomes
DeterministicSame inputs. Repeatable results.
Evidence-firstScreenshots, traces, console signals.
No averagesBinary task completion only.
ActionableEach failure maps to a fix.

Last Mile Index

Binary results. Verified evidence. No averages.

0 scan/run inversions detected

Sites are ranked by tasks completed by a real browser agent. The static scan column stays visible so the report can show where readiness signals disagree with task reality.

Rank Scenario Tasks Passed Agent Run Layer-1 Static Scan Key Failure Evidence Page
1 Friction Airways (ready) 43 / 50 Fail — Task Completion Evidence 1
2 Friction Airways (median) 25 / 40 +10 gated Fail — Task Completion Evidence 2
3 Friction Airways (hostile) 0 / 34 +16 gated Fail — Discoverability Evidence 3

AI Opportunity Tree

Where agents fall off the customer journey — and the fix at each stage.

68/150 agent-ready · 26 gated at an access wall

Each enhancement is mapped to the customer-journey funnel. A leaf is Pass (agent completed it), Blocked (an honest access wall — login / booking reference / payment), or Fail (the site broke the agent before any wall). Failing leaves carry a concrete Tier 0–2 fix.

Inspiration & Planning

66
6/12 agent-ready

Compare & Select

1311
13/24 agent-ready

Book Travel

1329
13/24 agent-ready

Plan & Prepare

558
5/18 agent-ready

Get to the airport & await flight

972
9/18 agent-ready

Take flight

12
1/3 agent-ready

Connect

324
3/9 agent-ready

Arrive at the destination

651
6/12 agent-ready

Stay involved

717
7/15 agent-ready

Help

528
5/15 agent-ready

Evidence dossier · Rank 1

Friction Airways (ready)

Task: Fa airport journey assistant

Fail

Evidence captured

Screenshot exhibit unavailable for this fixture. The production run record can attach a browser screenshot here.

Session replay — full browser recording of this run (Browserbase).

Short diagnosis

Task Completion: The agent could not recover because the interface did not expose actionable error feedback.

The scan could not see the journey-level blocker: Task Completion.

What the agent experienced

The agent could not recover because the interface did not expose actionable error feedback.

Friction signals

✓ Structured path✓ Consent wall✓ Input affordances✓ Failure transparency✓ Discoverability

Discovered via /llms.txt (20 links indexed at /llms.txt).

Primary failure: Task Completion

Signal evidence

Structured path
WebMCP · present · document.modelContext · tools: flights · invoked: getFare, searchFlights
Discoverability
entry point found · channel: dom+llms.txt · /llms.txt: 20 links
/llms.txt links
/fares · /total-cost · /total-cost.md · /baggage · /tool-fare · /tool-fare.md · /manage · /budget-destinations · /budget-destinations.md · /inspiration · /inspiration.md · /why-this-flight
Discovery candidates
Flight status (markdown) → /flight-status.md · Flight status → /flight-status

Discovered via /llms.txt (20 links indexed at /llms.txt).

What the site does right

On the same site, Fa ancillary timing completed through structured agent surfaces:

Structured path
WebMCP · present · document.modelContext · tools: flights, bagTiming · invoked: getFare, searchFlights, getBagTimingPrice · verified ✓ → SAR 2050
Discoverability
no entry point found · channel: dom

Agent run trace

  1. 01Go to frictionair.me-892.workers.dev (200)
  2. 02Read the page
  3. 03Type "FA7Q2X" into input[placeholder="6-character code from your confirmation email."] — timed out
  4. 04Read page region "form"
  5. 05Read page region "input"
  6. 06Click input
  7. 07call_webmcp_tool
  8. 08Read page region "form input"
  9. 09Read page region "body form"
  10. 10Click form input:nth-of-type(1)
  11. 11Type "FA7Q2X" into form input:nth-of-type(1)
  12. 12Click form input:nth-of-type(2)
  13. 13Type "Test" into form input:nth-of-type(2)
  14. 14Click button
14 stepsmax_steps

Recommended fix

Tier 0

Publish core info as agent-readable content

Ensure the answer to the task is available as readable static content (e.g., a text table, a paragraph) so an agent can extract it without navigating a transactional flow. Transactional paths can come later; readable content is the wedge.

Source: Chrome modern-web-guidance: agentic-forms — github.com/GoogleChrome/modern-web-guidance

Evidence dossier · Rank 2

Friction Airways (median)

Task: Fa airport journey assistant

Fail

Evidence captured

Screenshot exhibit unavailable for this fixture. The production run record can attach a browser screenshot here.

Session replay — full browser recording of this run (Browserbase).

Short diagnosis

The journey completed and the extracted answer passed verification.

The static scan did not prove task success. The agent completed the journey and the answer was verified.

What the agent experienced

The agent found a readable answer: Gate A12, Terminal 3. Flight FA482 · RUH → LHR is on time, departing at 14:30 on 2026-07-15, arriving at 18:05. Boarding at 13:50 (40 min before departure). Arrive at Terminal 3 at least 3 hours before an international departure..

Friction signals

✓ Structured path✓ Consent wall✓ Input affordances✓ Failure transparency× Discoverability

Signal evidence

Structured path
present · document.modelContext
Discoverability
no entry point found · channel: dom

Agent run trace

  1. 01Go to frictionair.me-892.workers.dev (200)
  2. 02Read the page
  3. 03Type "FA7Q2X" into input[name='booking_reference'], input[placeholder*='booking'], input[id='booking-reference'] — timed out
  4. 04Read page region "form"
  5. 05Click input
  6. 06Type "FA7Q2X" into input
  7. 07Click input[type='text']:nth-of-type(2), input:nth-of-type(2)
  8. 08Type "Test" into input:nth-of-type(2)
  9. 09Click button
  10. 10Wait 2s
  11. 11Read the page
  12. 12Finish — answer found: "Gate A12, Terminal 3. Flight FA482 · RUH → LHR is on time, departing at 14:30 on 2026-07-15, arriving at 18:05. Boarding at 13:50 (40 min before departure). Arrive at Terminal 3 at least 3 hours before an international departure."
12 stepscompleted

Evidence dossier · Rank 3

Friction Airways (hostile)

Task: Fa airport journey assistant

Fail

Evidence captured

Screenshot exhibit unavailable for this fixture. The production run record can attach a browser screenshot here.

Session replay — full browser recording of this run (Browserbase).

Short diagnosis

Task completion: The agent could not recover because the interface did not expose actionable error feedback.

The scan could not see that the journey failed under real browser operation.

What the agent experienced

The agent could not recover because the interface did not expose actionable error feedback.

Friction signals

✓ Structured path✓ Consent wall✓ Input affordances✓ Failure transparency× Discoverability

Signal evidence

Structured path
present · document.modelContext
Discoverability
no entry point found · channel: dom

Agent run trace

  1. 01Go to frictionair.me-892.workers.dev (200)
  2. 02Read the page
  3. 03call_webmcp_tool
  4. 04Read page region "form"
  5. 05Read page region "input"
  6. 06Read the page
  7. 07Finish — gave up: The booking reference wall: The page explicitly states "Airport journey assistant is available by phone only for now. Please call our service centre." There is no booking reference/last name lookup form available on the page. The task cannot proceed because the airport journey assistant is phone-only, blocking machine-readable access.
7 stepsgave_up