IBM Research: model routing is a systems optimization problem, not classification
IBM Research reports that on 417 AppWorld tasks, Claude Sonnet 4.6 cost $79 versus GPT-4.1's $155 despite lower sticker pricing — caching on reused agent context flipped the math. Its optimization-based router hits 84% accuracy at $93 and 83s, a 21% cost and 9% latency cut versus Opus alone for a 4% accuracy drop, at roughly 6ms and 2kB overhead per task — letting a developer building agent routing treat cost, latency, and compliance as one solved frontier rather than a per-task model guess.
Source: huggingface.co ↗
Caching — something most routing discussions ignore entirely.
IBM Research
Why this matters
- → Caching flips model economics — Sonnet costs half of GPT-4.1 despite higher sticker pricing
- → Systems optimization, not classification, unlocks 21% cost and 9% latency gains
- → Routers must balance cost, latency, compliance, and reliability simultaneously
Routing as optimization