Hamed Khosravi
← All publications

PRIME: Towards Verifiable Evidence-Grounded Reasoning in LLM Agents

Hamed Khosravi, Xiaoming Huo

arXiv preprint arXiv:2610.22426, 2026

Paper Code Benchmark

Evidence routes — fixed before the agent sees the case Route A severity rating + curator-reviewed advisory Route B published advisory link → ready for investigation → ready for investigation Checks, as they resolve ✓severity rating ✗curator advisory ✓advisory link Agent alone “ready for investigation” Route A failed — not justified PRIME refuses — try the advisory link “ready for investigation” Route B complete — authorized
The routes are fixed before the agent sees the case. As checks resolve, an agent left to itself reports the conclusion even though the route it was on has failed. PRIME refuses it, points at the route that can still succeed, and authorizes the conclusion only once that route completes.

The problem

LLM agents are used to check records against an organization’s requirements — whether a vulnerability report is ready for investigation, say. Requirements usually allow several ways to be met, and agents fail in two directions: they accept a record after a required check has failed, or they settle for a weaker conclusion without trying another route that would still have succeeded.

What we do

PRIME (Precommitted Rules for Inference from Minimal Evidence) fixes the evidence routes before the agent looks at a case, specifying how each conclusion may be justified. A separate program records each check result and refuses any conclusion those results do not justify. It also points the agent at the checks that would support a stronger conclusion, or runs them itself, so that refusal is not the only correction available.

What it buys you

On 480 public-record tasks at a twenty-call budget, an agent shown the requirements reported unjustified conclusions on 26.7% of them; under PRIME that falls to none. Refusal alone is not the whole story — it buys the guarantee by giving up value. The directions and the extra checks are what win that value back, lifting the score for justified conclusions from 30.1 to 45.5 out of 100. The tasks come from SUFFICE, a benchmark of evidence-based review tasks built from real public records with a deterministic scorer.

Two panels over the information ladder: false authorization falls to zero under PRIME while supported utility rises
The information ladder on 480 public-record tasks at a twenty-call budget. Top: false authorization — conclusions an agent reported that its own check results did not justify. Bottom: supported utility — the value of the conclusions its evidence actually established.

Cite

@article{khosravi2026prime,
  title   = {PRIME: Towards Verifiable Evidence-Grounded Reasoning in {LLM} Agents},
  author  = {Khosravi, Hamed and Huo, Xiaoming},
  journal = {arXiv preprint arXiv:2610.22426},
  year    = {2026}
}