Someone pasted two sentences into an early build of our text checker and got a verdict back. They acted on it.
The verdict was not wrong because the math was wrong. It was wrong because there was nothing to measure. Most of our features need a minimum number of words or sentences before their statistics mean anything, and on two sentences almost none of them qualified. The build reduced that to a label anyway.
So we deleted the label. In proseprobe, our open-source project, the calibrated scale now has exactly one exit, and it carries no boolean field anywhere: a point value, a low and a high bound, the confidence that produced them, the band the value falls in, every band the interval touches, the per-feature contributions, and notes.
The gates are explicit. Any feature whose word or sentence minimum the text does not reach is skipped, and the notes name each skipped feature. If nothing survives the gates, the function returns the prior: value 0.5 across an interval of 0.0 to 1.0, with every band marked as spanned and no contributions at all. You cannot round that shape into anything. Nobody looks at it and sees an answer.
Two more things we would not ship without now.
The interval is not decoration. It comes from the requested confidence, defaults to 0.95, folds in an irreducible spread we refuse to go below, and widens further under 250 words. When the interval touches more than two bands, the result says in writing that the point value is indicative only.
And every surviving feature reports its own signed shift, sorted largest first, with the shifts summing to the log-odds behind the value. Each feature's evidence is clamped to a ceiling, so one strange outlier cannot carry the whole score. The practical effect is that an argument about a result becomes an argument about one feature, and that is a conversation you can actually finish.
The product lesson came cheaper than the incident: if the data cannot support a yes or no, the API should not offer a field where a yes or no could live. Developers will consume whatever is cheapest to consume.
We would like to hear where you have taken a yes/no out of an API and found the honest version was the more useful one.