Self-learning threshold-based load balancing

Diego Goldsztajn, Sem C. Borst, Johan S. H. Van Leeuwaarden, Debankur Mukherjee, Philip A. Whiting

Research output: Contribution to journalArticleScientificpeer-review


We consider a large-scale service system where incoming tasks have to be instantaneously dispatched to one out of many parallel server pools. The user-perceived performance degrades with the number of concurrent tasks and the dispatcher aims at maximizing the overall quality of service by balancing the load through a simple threshold policy. We demonstrate that such a policy is optimal on the fluid and diffusion scales, while only involving a small communication overhead, which is crucial for large-scale deployments. In order to set the threshold optimally, it is important, however, to learn the load of the system, which may be unknown. For that purpose, we design a control rule for tuning the threshold in an online manner. We derive conditions that guarantee that this adaptive threshold settles at the optimal value, along with estimates for the time until this happens. In addition, we provide numerical experiments that support the theoretical results and further indicate that our policy copes effectively with time-varying demand patterns.
Original languageEnglish
Pages (from-to)39-54
JournalINFORMS Journal on Computing
Issue number1
Publication statusPublished - Jan 2022


  • adaptive load balancing
  • many-server asymptotics
  • fluid and diffusion limits


Dive into the research topics of 'Self-learning threshold-based load balancing'. Together they form a unique fingerprint.

Cite this