Live benchmarks
GET /api/v1/rankings returns the live runner benchmark — one entry per runner (browserbase, steel, skyvern, …), aggregated across every domain and task from real run outcomes. Scores update as runs finish. Use it to see which runners are performing before you pin one; the same data renders at /benchmarks.
Response
data is ordered by rank_score— the Wilson lower bound (95%) on lifetime success rate, so small samples don't outrank proven runners. Per-domain intelligence (which runner wins on a specific site) stays internal to routing: smart — the public benchmark is runner-level aggregates only. Benchmarks and routing posteriors use success/fail/cost/latency counts — not screenshots, downloads, or vault secrets. See Data handling & retention.
The contract is relative-only: entries expose rates and shares — success_rate / failed_rate / blocked_rate, rank_score, traffic_share (share of platform runs, 0–1), traffic_delta_pct (week-over-week change), and trend buckets of { day, share, success_rate }. Absolute run and volume counts are not part of the public API. The same rule applies platform-wide — even the /api/v2/* playground endpoints (not the product API) expose only traffic_share and traffic_share_series in place of the removed absolute-count fields. They do not list target domains.
Field reference
GET/api/v1/rankings (liveRankings)
Live runner benchmark aggregated across all domains from real outcomes
Response
| Field | Type | Req | Description |
|---|---|---|---|
| object | "list" | — | — |
| live | boolean | — | — |
| group_by | "runner" | — | — |
| updated_at | string (date-time) | — | — |
| trend_days | integer | — | — |
| data | any[] | — | Ordered by rank_score descending. |
| data[] | object | — | Runner-level aggregate. Volume fields are relative-only (traffic_share, traffic_delta_pct); the absolute-count fields (successes/failures/blocks/total/window_runs) were removed from the contract. |
| data[].runner | string | — | — |
| data[].label | string | — | — |
| data[].mode | string infra | agent | oss | — | — |
| data[].availability | object | — | Live availability from the runner-health circuit breaker. `suspended` runners are skipped by smart routing, fallback chains, fan-outs, and races until `until`; `probation` is half-open — the cooldown expired, the runner is eligible again, and the next routed attempt is the re-probe (reason/detail/until describe the suspension it is recovering from). Pure surfacing — routing already excludes suspended runners, so you never need to filter yourself; use this to explain pool coverage or to avoid pinning a suspended runner. |
| data[].success_rate | number | — | — |
| data[].failed_rate | number | — | — |
| data[].blocked_rate | number | — | — |
| data[].rank_score | number | — | Wilson lower bound (95%) on lifetime success rate — default sort key. |
| data[].domains | integer | — | Distinct domains with recorded outcomes. |
| data[].traffic_share | number | — | Share of recent-window routed attempts (0–1). |
| data[].traffic_delta_pct | number | — | Traffic change, last 7 trend days vs the 7 before (%); null without a base. |
| data[].is_new | boolean | — | True when the runner has current-week traffic but no prior-week base. |
| data[].low_sample | boolean | — | Qualitative small-sample hint (never the count itself). |
| data[].avg_latency_ms | number | — | — |
| data[].p50_latency_ms | number | — | — |
| data[].p95_latency_ms | number | — | — |
| data[].avg_cost_usd | number | — | — |
| data[].last_seen_at | string (date-time) | — | — |
| data[].trend | any[] | — | — |
| tasks | any[] | — | — |
| tasks[] | object | — | — |
| tasks[].task | string | — | Task intent (extract, search, login_mfa, …) |
| tasks[].share | number | — | Share of recorded task outcomes. |
| tasks[].top_runners | object[] | — | — |
| note | string | — | — |
HTTP statuses
200— Live runner benchmarks — one entry per runner, aggregated across all domains
Generated from OpenAPI. Do not hand-maintain this table.