Routing Between Models

Routing

Part of: Choosing the Right Model

Most support emails are easy: "What are your hours?" A cheap, fast model nails them. Every so often a gnarly multi-part complaint arrives that the cheap model fumbles. Instead of sending every email to an expensive model just in case, smart systems try the cheap one first and escalate only the hard cases to a stronger model. That's a cascade, and it can cut costs dramatically. What it is A cascade (or escalation routing) sends a request to a cheap-fast model first . If that model handles it confidently, you're done, cheap and quick. If the input looks too hard, or the cheap model's answer fails a confidence check, you escalate : re-run the request on a stronger, pricier model . This builds on lesson 3's task routing, but with a twist. Static routing decides the tier upfront by task type. A cascade decides dynamically , often after the cheap model has already taken a swing. How it works The pattern is "try cheap, escalate on hard": Notice the cost of an escalated request is the cheap call plus the strong call, you pay twice when you escalate. The math only works because most requests stay cheap, so the rare double-charge averages out tiny. Why it matters Cascades shine when difficul

Challenge: Cascade Router