Hamed Khosravi
← All publications

Rank Confidence Sequences: Anytime-Valid Leaderboards

Hamed Khosravi, Xiaoming Huo

arXiv preprint arXiv:2609.32211, 2026

Paper Code

0 A ahead B ahead look 8: a winner is declared looks at the leaderboard → anytime-valid sequence — contains the truth at every look fixed-sample interval, re-read each time
Two models that are genuinely tied, compared again and again as votes arrive. A fixed-sample interval is valid if you look once; re-read at every look it eventually excludes zero by chance and names a winner that does not exist. The confidence sequence is wider, and does not.

The problem

A public leaderboard is read continuously and updated as votes arrive, so a ranking that looks settled at one glance can be wrong at the next. The intervals behind those standings are valid for a single look, and a leaderboard is never looked at once.

What we do

Rank confidence sequences keep every pairwise claim valid at all times, however often the leaderboard is read, and however the decision to keep reading depends on what has been read so far.

What it buys you

Every pairwise claim on the board can be trusted at the moment it is read, with no pre-registered stopping rule and no correction for how many times anyone has looked.

Cite

@article{khosravi2026rank,
  title   = {Rank Confidence Sequences: Anytime-Valid Leaderboards},
  author  = {Khosravi, Hamed and Huo, Xiaoming},
  journal = {arXiv preprint arXiv:2609.32211},
  year    = {2026}
}