Partially Correlated Verifier Cascades in LLM Harnesses: Concave Log-Odds, Polynomial Reliability, and Blind-Spot Ceilings
This paper develops a mathematical theory for reliability in large language models (LLMs) by analyzing the behavior of verification gates in cascades. It explains why adding more gates can sometimes help, sometimes hurt, or plateau the reliability of LLMs, and provides a way to estimate this reliability using repeated verdicts or other methods. Practitioners might care about this because it can help them design more reliable and robust LLMs.