Principled Model Routing for Unknown Mixtures of Source Domains
Christoph Dann, Yishay Mansour, Teodor Vanislavov Marinov, Mehryar Mohri
The rapid proliferation of domain-specialized machine learning modelspresents a challenge: while individual models excel in specificdomains, their performance varies significantly across diverseapplications. This makes selecting the optimal model when faced withan unknown mixture of tasks, especially with limited or no datato estimate the mixture, a difficult problem.  We address thischallenge by formulating it as a multiple-source domain adaptation(MSA) problem. We introduce a novel, scalable algorithm thateffectively routes each input to the best-suited model from a pool ofavailable models.  Our approach provides a strong performanceguarantee: remarkably, for any mixture domain, the accuracy achieved by the bestsource model is maintained.  This guarantee is established through atheoretical bound on the regret for new domains, expressed as a convexcombination of the best regrets in the source domains, plus aconcentration term that diminishes as the amount of source dataincreases.  While our primary contributions are theoretical andalgorithmic, we also present empirical results demonstrating theeffectiveness of our approach.