Hamed Khosravi
← All publications

Conformal Selective Acting: Anytime-Valid Risk Control for RLVR-Trained LLMs

Hamed Khosravi, Xiaoming Huo

NeurIPS 2026 (main track)

Paper Code

target 10% 0 target quietly exceeded inputs shift rounds of verifier feedback → certified threshold — risk held under target throughout one threshold, fixed up front
Risk on the answers the model chose to act on. One threshold fixed up front looks fine until the inputs shift, after which it quietly carries more risk than the target allows. The certified threshold is re-derived from verifier feedback as it arrives, and the guarantee holds at every round rather than at one planned sample size.

The problem

A language model trained with verifiable rewards keeps producing answers of uneven quality, and the inputs it meets in deployment are not the ones it was tuned on. A single confidence threshold picked in advance stops being the right threshold, and nothing announces when that happened.

What we do

Verifier feedback on each output updates an anytime-valid test that certifies a confidence threshold. The model acts when its score clears the threshold and abstains otherwise.

What it buys you

Risk stays bounded at every time step, not only at a sample size fixed in advance — so the system can be watched continuously, and stopped or shipped at any moment, without the guarantee quietly expiring.

Cite

@inproceedings{khosravi2026conformal,
  title     = {Conformal Selective Acting: Anytime-Valid Risk Control for {RLVR}-Trained {LLM}s},
  author    = {Khosravi, Hamed and Huo, Xiaoming},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026}
}