Bank Of America Routes AI Queries By Complexity To Reduce Model Costs

Bank Of America Routes AI Queries By Complexity To Reduce Model Costs

The Common Question

How can banks cut AI infrastructure costs without sacrificing model performance?

The Lesson in This Case

Bank of America built a routing layer called Orchestra that directs each query to the least expensive model capable of handling it, cutting AI costs across its deployed systems. A small classifier model makes the routing decision in real time, keeping high-cost models reserved for genuinely complex requests. Leaders evaluating AI spending should recognize that cost discipline at scale often comes from architecture choices made before queries ever reach a model.

Originally reported by Banking Dive. Read the full story here.