Figure gallery

Every figure in the book, with its chart type and a link to the chapter that shows it.

25 figures in 6 chapters. dot 17, dumbbell 3, bar 3, line 2.

From evidence to typed questions

Chapter 1.2 in Part I

Figure 1. Jev, Luna and a general language model all rank dissatisfied reviews a little better than a classifier trained on other coffee reviewsdotAUC for dissatisfied (one or two stars), log loss and the gain in AUC over the classifier, sampled coffee reviews, weighted
Figure 2. Jev is the cheapest of the three models per reviewdotMean billed cost per thousand reviews and median latency per call, by engine, October 2026
Figure 3. The three models' star scores miss the reviewer's rating by similar amountsdotMean absolute error of the star score and the share of reviews whose most likely level equals the rating, sampled coffee reviews, weighted
Figure 4. The review classifier needs thousands of labels to approach what Jev earns with nonedotNet value of the contact policy per thousand reviews against labeled reviews used to train the classifier, weighted
Figure 5. Removing the no-problem option changes most issue answers, while rewording or reordering changes only a fewdumbbellPercent of reviews whose answer differs from the original question, after one change and after asking the same question again, Jev and Luna, weighted

Chapter 1.3 in Part I

Figure 1. Model-based contact policies earn far more than a keyword rule or contacting everyone, and Jev's raw probabilities earn no clear premium over the classifierdotNet value of contacts per thousand reviews against contacting nobody, at the cost cutoff p*, by policy, weighted
Figure 2. The engines' probabilities are off by more than the classifier's until they are recalibrateddotExpected calibration error of the probability of dissatisfaction, as returned and after recalibration, by engine, weighted
Figure 3. Letting the data choose the cutoff moves it above p* for the enginesdotCutoff that maximizes net value, chosen on four folds of the sample and applied to the fifth, median of five folds, by model
Figure 4. A data-chosen cutoff adds the most net value for Jev, the least calibrated engine, and nothing for the classifierdumbbellNet value per thousand reviews at the cost cutoff p* and at the data-chosen cutoff, by model, weighted
Figure 5. For the same labels, recalibrating Jev earns more than fitting a classifier at every size trieddotNet value of the contact policy per thousand reviews against local labels used, weighted; whiskers run from the lowest to the highest draw

Chapter 2.1 in Part II

Figure 1. A classifier trained on earlier complaints agrees with consumers' product choices about as often as JevdotComplaints whose predicted product matches the one the consumer selected, percent, by model, with 95% intervals
Figure 2. At the same cutoff, Jev sends fewer complaints to people and misroutes more, because its stated probabilities run highbarWhere each routing policy sends complaints, percent of complaints, 2025, weighted
Figure 3. For the same labels, recalibrating Jev costs less than fitting a classifier at every size trieddotExpected routing cost per thousand complaints against local labels used, weighted; whiskers run from the lowest to the highest draw
Figure 4. A classifier trained on past outcomes finds money-back complaints better than the models that only read the complaintdotShare of complaints closed with monetary relief that fall in each model's top tenth by P(money back), percent
Figure 5. Describing past outcomes in the question helps the engines a little, and examples do not close the gap to the classifierdotAUC for complaints closed with monetary relief, by model and how the question was posed, with 95% intervals; 2,000 sampled 2025 complaints
Figure 6. For money back, the classifier matches Jev with about a thousand past complaintsdotMoney-back complaints caught in the top tenth, percent, against past complaints used to train the classifier, weighted

Chapter 2.2 in Part II

Figure 1. Ranking by the models finds the defects much sooner than ranking by diff sizelineDefective patches among the first reviews in each order, constructed patches, if each review reveals the defect
Figure 2. One question does most of the work, and equal weights dilute itdotAUC of each typed question for finding defective patches, constructed patches, by engine
Figure 3. On real agent patches, neither the models nor the rules clearly beat sorting by sizedumbbellAUC for the benchmark's unresolved label, across all patches and within the same issue, with 95% intervals

Chapter 3.1 in Part III

Chapter 4.1 in Part IV

Figure 1. Whether Jev's policy beats the simple alternative turns on calibrating its probabilitiesdotJev's policy against the simple alternative, percent of that alternative's value or cost, with 95% intervals
Figure 2. Jev's ranking score differs widely from one question to the nextdotAUC by question, Jev and the trained classifier, with 95% intervals; no classifier was trained for the patch and agent cases