Benchmark

Runner benchmark

Live comparison of 250+ runner combinations from real Banana Peel traffic. Every routed attempt — success, failure, or block — updates success rate, cost, and latency. Volumes stay relative: traffic share, never weekly or monthly run counts.

Last benchmark snapshot Aug 27, 2026, 07:15 PM UTC

Methodology Docs API

Runner comparison

Highest success, best value, and fastest from live routed attempts — then the same runners on three charts. Filter who appears with the selector on each card.

Highest success

Playwright MCP

70.8%

Best value

Stagehand

$0.011 / task

Fastest

ChatGPT Agent

7.9s p50

Success rate

Best first · lifetime success across every domain

Cost per task

Cheapest first · average USD per executed attempt

Time per task

Fastest first · median (p50) wall time per attempt

Cost efficiency

Success vs average cost. The dashed line is the Pareto frontier — cheaper or more accurate than the runners below it.

Hover a point for exact values · colors match the comparison charts

Smart routing — internal calibration

The router picks a winner per job. Quality is jobs completed — smart-routed runs that found a genuine success, over every smart run — versus the best fixed pin you'd pick in advancefor the whole basket, and versus a perfect-hindsight oracle (best runner per site, chosen with a time machine). Pin scores use non-learning attempts only, cells with fewer than 5 outcomes are unscored, and a thin cell cannot inherit the pool's weight.

These figures come from internal QA and calibration runs, not customer traffic. Learning probes are excluded. The label lifts once enough customer jobs land.

Of perfect-hindsight

93%

Zero config — fraction of the oracle

Vs best advance pin

+37.4 pp

Jobs completed vs pinning Playwright MCP for everything

Different winner

4 of 4

Tiers where the best pin is a different runner

Fewer failed jobs

37.4 pp fewer

Failed-job share vs that best advance pin

On easy pages, Deck Agent hits 75.0% and Smart 99.4%— a cheap pin ties. On hardened sites, Smart completes 61.2% of jobs; that same pin drops to 2.8%. The residual spend is the premium for covering tasks that go to 0% on a fixed runner, and for not maintaining a per-site runner map when defenses change.

Full-basket frontier

Y = success on the whole domain basket. X = effective $ per success (log), failures kept in the denominator. The Smart → oracle gap is routing regret.

Leaderboard

Every listed runner. Click a column to sort. Traffic is share of recent routed attempts — never weekly or monthly run counts.

#
1	Playwright MCPoss	70.8%	$0.12	28.9s	2.6%	0.65
2	Deck Agentagent	66.6%	$0.26	1.2m	6.9%	0.64
3	Browser Useagent	66.2%	$0.053	1.5m	6.7%	0.63
4	Gemini Agentagent	69.2%	$0.012	11.5s	1.0%	0.60
5	Claude Computer Useagent	61.9%	$0.055	12.6s	0.7%	0.52
6	Lightpandainfra	57.9%	$0.017	19.5s	1.0%	0.50
7	Stagehandoss	56.5%	$0.011	1.4m	1.5%	0.50
8	Playwrightoss	49.0%	$0.020	23.9s	2.8%	0.45
9	ChatGPT Agentagent	53.3%	$0.070	7.9s	0.6%	0.42
10	Seleniumoss	49.5%	$0.030	20.9s	0.7%	0.40
11	Manusagent	51.1%	$0.56	1.1m	0.5%	0.37
12	Kernelinfra	45.3%	$0.014	26.4s	0.8%	0.37
13	Browserbase Stealthinfra	45.2%	$0.078	29.6s	0.7%	0.35
14	HyperAgentoss	44.2%	$0.006	23.9s	0.6%	0.34
15	Chrome DevTools MCPoss	41.8%	$0.018	5.6s	0.6%	0.33
16	Skyvernagent	39.4%	$0.10	5.0m	2.7%	0.33
17	Steelinfra	36.5%	$0.027	52.1s	3.6%	0.32
18	TinyFishagent	41.3%	$0.52	58.8s	0.5%	0.28
19	Browserbaseinfra	32.5%	$0.085	38.2s	1.9%	0.27
20	Hyperbrowserinfra	29.7%	$0.022	23.0s	2.8%	0.25
21	Anchor Browserinfra	29.5%	$0.028	33.2s	2.4%	0.25
22	Steel + GPT-4.1infra	35.9%	$0.063	47.7s	0.7%	0.24
23	Browserlessinfra	32.9%	$0.005	215ms	0.5%	0.24
24	Airtopinfra	30.1%	$0.23	1.2m	0.6%	0.22
25	Steel + o4-miniinfra	30.6%	$0.038	1.0m	0.8%	0.21
26	Steel + GPT-4.1 miniinfra	29.0%	$0.018	45.4s	0.8%	0.19
27	Crawleeoss	26.3%	$0.001	1.4s	0.6%	0.17
28	Browserbase + Claude Haiku 4.5infra	28.6%	$0.050	8.9s	0.3%	0.15
29	Anchor + Claude Haiku 4.5infra	27.6%	$0.020	9.1s	0.4%	0.15
30	Browserbase Stealth + Gemini 2.5 Proinfra	27.6%	$0.059	15.3s	0.4%	0.15
31	Asteroidagent	22.2%	$0.12	36.6s	0.8%	0.15
32	Jina Readerinfra	24.4%	$0	1.2s	0.3%	0.14
33	Puppeteeross	24.4%	$0.032	13.8s	0.4%	0.14
34	Crawlee + Claude Sonnet 5oss	25.8%	$0.001	3ms	0.4%	0.14
35	Crawlee + Gemini 2.5 Prooss	25.8%	$0.003	4ms	0.4%	0.14
36	Steel + GPT-4o miniinfra	23.4%	$0.013	41.7s	0.6%	0.14
37	Crawlee + Gemini 2.5 Flash-Liteoss	25.0%	$0	6ms	0.4%	0.13
38	Crawlee + Gemini 2.5 Flashoss	25.0%	$0.001	3ms	0.4%	0.13
39	Browserbase Stealth + GPT-4.1 miniinfra	25.9%	$0.054	7.7s	0.3%	0.13
40	Browserbase Stealth + Claude Opus 5infra	25.9%	$0.095	12.2s	0.3%	0.13
41	Anchor + Claude Opus 5infra	25.9%	$0.21	13.9s	0.4%	0.13
42	Anchor + GPT-4o miniinfra	22.7%	$0.019	22.7s	0.6%	0.13
43	Anchor + Gemini 2.5 Flash-Liteinfra	25.0%	$0.013	8.2s	0.4%	0.13
44	Anchor + Claude Sonnet 5infra	24.1%	$0.056	14.5s	0.4%	0.12
45	Browserbase Stealth + o4-miniinfra	24.1%	$0.062	10.7s	0.4%	0.12
46	Anchor + Gemini 2.5 Flashinfra	20.8%	$0.016	27.1s	0.8%	0.12
47	LLM Scrapeross	21.9%	$0	1.2s	0.3%	0.12
48	Anchor + Gemini 2.5 Proinfra	22.2%	$0.025	21.8s	0.5%	0.12
49	Steel + Claude Haiku 4.5infra	20.4%	$0.031	27.0s	0.6%	0.11
50	Crawlee + GPT-4.1 minioss	22.6%	$0.001	2ms	0.4%	0.11
51	Steel + Gemini 2.5 Proinfra	21.1%	$0.028	16.9s	0.6%	0.11
52	Crawlee + Claude Haiku 4.5oss	21.9%	$0.001	6ms	0.4%	0.11
53	Steel + Gemini 2.5 Flash-Liteinfra	22.2%	$0.009	6.8s	0.4%	0.11
54	Steel + Claude Opus 5infra	22.2%	$0.12	10.3s	0.4%	0.11
55	Kernel + Claude Sonnet 5infra	21.4%	$0.079	181ms	0.4%	0.10
56	Anchor + o4-miniinfra	20.7%	$0.022	14.0s	0.4%	0.10
57	Browserbase Stealth + Gemini 2.5 Flash-Liteinfra	20.7%	$0.041	7.0s	0.4%	0.10
58	Anchor + GPT-4.1infra	20.7%	$0.057	8.8s	0.4%	0.10
59	Steel + Claude Sonnet 5infra	19.4%	$0.053	37.2s	0.5%	0.10
60	Midscene.jsoss	18.6%	$0.004	5.9s	0.3%	0.10
61	Anchor + GPT-4.1 miniinfra	18.6%	$0.023	21.5s	0.6%	0.10
62	Steel + Gemini 2.5 Flashinfra	18.9%	$0.009	22.3s	0.5%	0.09
63	Crawlee + GPT-4.1oss	19.4%	$0.002	4ms	0.4%	0.09
64	Crawlee + GPT-4ooss	19.4%	$0.002	1ms	0.4%	0.09
65	Crawlee + o4-minioss	19.4%	$0.002	7ms	0.4%	0.09
66	Crawlee + Claude Opus 5oss	19.4%	$0.003	2ms	0.4%	0.09
67	Puppeteer + o4-minioss	19.4%	$0.037	18.7s	0.4%	0.09
68	Crawlee + GPT-4o minioss	18.8%	$0.001	2ms	0.4%	0.09
69	Kernel + o4-miniinfra	17.9%	$0.025	203ms	0.4%	0.08
70	Puppeteer + GPT-4.1 minioss	17.2%	$0.029	6.1s	0.4%	0.08
71	Browserbase Stealth + GPT-4.1infra	17.2%	$0.064	7.0s	0.4%	0.08
72	Steel + GPT-4oinfra	17.2%	$0.064	8.8s	0.4%	0.08
73	Browserbase Stealth + GPT-4oinfra	17.2%	$0.081	7.8s	0.4%	0.08
74	Browserbase + Claude Sonnet 5infra	17.2%	$0.098	9.5s	0.4%	0.08
75	Exainfra	25.0%	—	—	0.0%	0.07
76	Puppeteer + Gemini 2.5 Flash-Liteoss	16.1%	$0.018	5.8s	0.4%	0.07
77	Puppeteer + Claude Sonnet 5oss	16.1%	$0.21	22.5s	0.4%	0.07
78	Puppeteer + Claude Opus 5oss	16.1%	$0.28	11.5s	0.4%	0.07
79	Kernel + GPT-4.1 miniinfra	15.4%	$0.013	1ms	0.3%	0.06
80	Kernel + Gemini 2.5 Proinfra	14.8%	$0.035	150ms	0.4%	0.06
81	Kernel + GPT-4oinfra	14.8%	$0.040	178ms	0.4%	0.06
82	Browserbase Stealth + Gemini 2.5 Flashinfra	13.8%	$0.040	9.8s	0.4%	0.06
83	Browserbase + o4-miniinfra	13.8%	$0.043	9.3s	0.4%	0.06
84	Browserbase Stealth + Claude Haiku 4.5infra	13.8%	$0.044	9.3s	0.4%	0.06
85	Browserbase + Gemini 2.5 Flashinfra	13.8%	$0.045	12.1s	0.4%	0.06
86	Browserbase + GPT-4.1 miniinfra	13.8%	$0.050	6.7s	0.4%	0.06
87	Browserbase + Gemini 2.5 Proinfra	13.8%	$0.060	13.0s	0.4%	0.06
88	Browserbase + Claude Opus 5infra	13.8%	$0.26	7.4s	0.4%	0.06
89	Puppeteer + GPT-4o minioss	13.3%	$0.007	3.1s	0.3%	0.05
90	Puppeteer + Gemini 2.5 Flashoss	13.3%	$0.021	10.3s	0.4%	0.05
91	Browserbase + GPT-4.1infra	13.3%	$0.079	6.1s	0.4%	0.05
92	Browserbase + GPT-4oinfra	13.3%	$0.12	6.3s	0.4%	0.05
93	Playwright + Gemini 2.5 Flashoss	12.9%	$0.022	14.1s	0.4%	0.05
94	Puppeteer + Gemini 2.5 Prooss	12.9%	$0.029	11.7s	0.4%	0.05
95	Puppeteer + GPT-4.1oss	12.9%	$0.15	4.0s	0.4%	0.05
96	Anchor + GPT-4oinfra	11.1%	$0.031	21.5s	0.6%	0.05
97	Playwright + GPT-4.1oss	12.1%	$0.071	7.2s	0.4%	0.05
98	Firecrawlinfra	10.5%	$0.001	1ms	0.3%	0.04
99	Kernel + Claude Haiku 4.5infra	12.0%	$0.005	1ms	0.3%	0.04
100	Kernel + Gemini 2.5 Flash-Liteinfra	12.0%	$0.008	1ms	0.3%	0.04
101	Airtop + Gemini 2.5 Flash-Liteinfra	12.0%	$0.086	1ms	0.3%	0.04
102	Airtop + Claude Haiku 4.5infra	11.5%	$0.092	1ms	0.3%	0.04
103	Airtop + GPT-4.1 miniinfra	11.5%	$0.092	1ms	0.3%	0.04
104	Airtop + Claude Sonnet 5infra	11.5%	$0.093	1ms	0.3%	0.04
105	Airtop + Claude Opus 5infra	11.5%	$0.14	1ms	0.3%	0.04
106	Kernel + Gemini 2.5 Flashinfra	11.1%	$0.011	148ms	0.4%	0.04
107	Hyperbrowser + GPT-4.1 miniinfra	11.1%	$0.019	1ms	0.3%	0.04
108	Browserbase Stealth + GPT-4o miniinfra	11.1%	$0.031	6.9s	0.3%	0.04
109	Airtop + Gemini 2.5 Proinfra	11.1%	$0.12	1ms	0.3%	0.04
110	Playwright + Claude Haiku 4.5oss	10.3%	$0.014	7.3s	0.4%	0.04
111	Playwright + GPT-4.1 minioss	10.3%	$0.015	8.1s	0.4%	0.04
112	Hyperbrowser + Claude Haiku 4.5infra	10.3%	$0.019	80ms	0.4%	0.04
113	Playwright + Claude Opus 5oss	10.3%	$0.11	8.2s	0.4%	0.04
114	Browserbase Stealth + Claude Sonnet 5infra	10.3%	$0.21	402ms	0.4%	0.04
115	Playwright + Claude Sonnet 5oss	10.3%	$0.23	14.4s	0.4%	0.04
116	Playwright + Gemini 2.5 Flash-Liteoss	10.0%	$0.017	3.3s	0.4%	0.03
117	Browserbase + Gemini 2.5 Flash-Liteinfra	10.0%	$0.042	5.1s	0.4%	0.03
118	Playwright + Gemini 2.5 Prooss	10.0%	$0.042	17.0s	0.4%	0.03
119	Puppeteer + Claude Haiku 4.5oss	9.7%	$0.024	14.4s	0.4%	0.03
120	Parallel Task APIagent	9.1%	$0.010	7ms	0.3%	0.03
121	Puppeteer + GPT-4ooss	9.1%	$0.11	9.7s	0.8%	0.03
122	Scrapflyinfra	8.1%	$0.002	1ms	0.3%	0.03
123	Parallelinfra	14.3%	—	—	0.0%	0.03
124	SerpAPIinfra	14.3%	—	—	0.0%	0.03
125	Tavilyinfra	14.3%	—	—	0.0%	0.03
126	Kernel + GPT-4o miniinfra	8.0%	$0.009	1ms	0.3%	0.02
127	Kernel + Claude Opus 5infra	8.0%	$0.078	1ms	0.3%	0.02
128	Airtop + Gemini 2.5 Flashinfra	8.0%	$0.084	1ms	0.3%	0.02
129	Airtop + GPT-4.1infra	7.7%	$0.075	1ms	0.3%	0.02
130	Airtop + o4-miniinfra	7.7%	$0.083	1ms	0.3%	0.02
131	Hyperbrowser + Gemini 2.5 Flashinfra	7.4%	$0.003	1ms	0.3%	0.02
132	Kernel + GPT-4.1infra	7.4%	$0.042	6ms	0.4%	0.02
133	Hyperbrowser + GPT-4oinfra	7.1%	$0.014	69ms	0.3%	0.02
134	Hyperbrowser + GPT-4.1infra	7.1%	$0.016	100ms	0.3%	0.02
135	Playwright + GPT-4o minioss	6.9%	$0.012	3.8s	0.4%	0.02
136	Playwright + o4-minioss	6.9%	$0.017	8.6s	0.4%	0.02
137	Browserbase + GPT-4o miniinfra	6.9%	$0.038	423ms	0.4%	0.02
138	Playwright + GPT-4ooss	6.7%	$0.081	4.5s	0.4%	0.02
139	Decodoinfra	5.9%	$0.002	1ms	0.3%	0.02
140	Hyperbrowser + Gemini 2.5 Proinfra	3.9%	$0.002	1ms	0.3%	0.01
141	Browserless + Claude Haiku 4.5infra	3.9%	$0.004	1ms	0.3%	0.01
142	Browserless + Claude Opus 5infra	3.9%	$0.037	1ms	0.3%	0.01
143	Airtop + GPT-4o miniinfra	3.9%	$0.065	1ms	0.3%	0.01
144	Hyperbrowser + Claude Opus 5infra	3.9%	$0.20	4ms	0.3%	0.01
145	Browserless + GPT-4.1infra	3.7%	$0.008	129ms	0.3%	0.01
146	Browserless + o4-miniinfra	3.7%	$0.008	1ms	0.3%	0.01
147	Browserless + Claude Sonnet 5infra	3.7%	$0.019	1ms	0.3%	0.01
148	Browserless + Gemini 2.5 Flashinfra	3.6%	$0.001	127ms	0.3%	0.01
149	Hyperbrowser + o4-miniinfra	3.6%	$0.005	80ms	0.3%	0.01
150	Hyperbrowser + Gemini 2.5 Flash-Liteinfra	3.6%	$0.009	68ms	0.3%	0.01
151	Hyperbrowser + GPT-4o miniinfra	3.6%	$0.009	95ms	0.3%	0.01
152	Exa Researchagent	3.2%	$0.024	1ms	0.3%	0.01
153	ScraperAPIinfra	3.0%	$0.001	1ms	0.3%	0.01
154	ZenRowsinfra	3.0%	$0.001	1ms	0.3%	0.01
155	Serperinfra	3.0%	$0.001	1ms	0.3%	0.01
156	Diffbotinfra	3.0%	$0.002	1ms	0.3%	0.01
157	ScrapingBeeinfra	3.0%	$0.005	1ms	0.3%	0.01
158	Apifyinfra	2.9%	$0.002	1ms	0.3%	0.01
159	AgentQLinfra	2.9%	$0.002	1ms	0.3%	0.01
160	Notteinfra	2.8%	$0.001	1ms	0.3%	0.00
161	SeleniumBaseoss	0.0%	$0	1ms	0.3%	0.00
162	Browser Useoss	0.0%	$0	1ms	0.3%	0.00
163	Browserless + GPT-4.1 miniinfra	0.0%	$0	1ms	0.3%	0.00
164	Browserless + GPT-4oinfra	0.0%	$0	1ms	0.3%	0.00
165	Browserless + GPT-4o miniinfra	0.0%	$0	1ms	0.3%	0.00
166	Camoufoxoss	0.0%	$0	1ms	0.3%	0.00
167	Crawl4AIoss	0.0%	$0	1ms	0.3%	0.00
168	Magnitudeoss	0.0%	$0	4ms	0.3%	0.00
169	nodriveross	0.0%	$0	1ms	0.3%	0.00
170	ScrapeGraphAIoss	0.0%	$0	1ms	0.3%	0.00
171	Browserless + Gemini 2.5 Flash-Liteinfra	0.0%	$0.001	1ms	0.3%	0.00
172	Cloudflare Browser Renderinginfra	0.0%	$0.001	1ms	0.3%	0.00
173	Browserless + Gemini 2.5 Proinfra	0.0%	$0.008	1ms	0.3%	0.00
174	Hyperbrowser + Claude Sonnet 5infra	0.0%	$0.011	1ms	0.3%	0.00
175	Airtop + GPT-4oinfra	0.0%	$0.11	1ms	0.3%	0.00

Why we run it

Banana Peel routes browser tasks across 250+ runner combinations. This page is the public scoreboard that smart routing already learns from: every terminal attempt — success, failure, or block — updates success rate, cost, and latency. There is no separate synthetic suite and no paper to cite. The numbers are live production outcomes, aggregated across every domain.

Rank order on the API uses the Wilson lower bound (95%) on lifetime success so a thin sample does not outrank a proven runner. Cost is average USD per executed attempt in the recent window. Time is median (p50) wall-clock. Traffic is that runner's share of recent routed attempts. We never publish weekly or monthly run counts. Smart routing efficiency scores the router as a selector: jobs that found a winner over all smart jobs, versus the best fixed pin on the full domain basket and versus a hindsight oracle. Pin headlines use non-learning attempts, drop cells with fewer than 5 outcomes, and cap inherited pool weight. When customer volume is still thin, that block is labeled internal calibration — not observed product performance.

Per-domain intelligence — which runner wins on a specific site — stays internal to routing: smart. The public benchmark is runner-level only. Aggregates are success, fail, cost, and latency — not screenshots, downloads, or vault secrets.

Benchmark API Smart routing Capabilities Retention

GET /api/v1/rankings · GET /api/v1/capabilities · snapshot Aug 27, 2026, 07:15 PM UTC

loading live benchmark…

Runner benchmark