Is the confidence honest?
Building reliability diagrams and computing ECE from scratch to check an LLM's confidence
Latest articles
Where confidence comes from
Reading how sure an LLM was of its answer, from the probabilities it already computed
Forcing a valid answer
Grammars, one-letter choices, and an LLM that can only answer with a label from your list
Why typed decisions?
Getting an LLM to return a value your code can branch on with a confidence you can act on
