# Decision AI > Case studies in turning model judgments into actions. Vishal Singh, NYU Stern. https://vishalsingh.org/decisionai Cite as: Singh, Vishal. Decision AI: Case studies in turning model judgments into actions. 2026. https://vishalsingh.org/decisionai Methods: https://vishalsingh.org/decisionai/reference/methods Every chapter has a report at /report, the same report as Markdown at /report.md (listed below), its facts at https://vishalsingh.org/decisionai/report//facts.json and each figure's rows at https://vishalsingh.org/decisionai/report//figures/.json. All reports in one file: https://vishalsingh.org/decisionai/llms-full.txt ## Chapters - [Introduction. About this booklet](https://vishalsingh.org/decisionai/about-this-booklet/report.md): This booklet explores the decision endpoints that major AI labs have recently released, an interface popularized by TypeSafe's Jev, by posing business and workflow decisions as typed questions and checking whether acting on the answers beats the simplest credible alternative. - [Chapter 1.1. Decision AI in applications and agents](https://vishalsingh.org/decisionai/decision-ai-in-applications-and-agents/report.md): An application can act on a model's judgment only through a stated policy, and in the opening bug-report example that policy routes a report to an engineer whenever the chance it can be reproduced exceeds 0.556. - [Chapter 1.2. From evidence to typed questions](https://vishalsingh.org/decisionai/from-evidence-to-typed-questions/report.md): On 5,000 sampled coffee reviews, two decision engines and a general language model rank dissatisfied reviewers with an AUC between 0.979 and 0.983, against 0.968 for a classifier trained on other reviews, and rewording the question changes some answers more than repeating it does. - [Chapter 1.3. From probabilities to actions](https://vishalsingh.org/decisionai/from-probabilities-to-actions/report.md): At an assumed contact cost of $6 and recovery worth $48, Jev's raw probabilities earn $5,076 per thousand reviews against $5,015 for a trained classifier, with no clear gap, and recalibrating Jev on labeled reviews raises it to $5,335, because its probabilities overstate dissatisfaction near the cutoff. - [Chapter 2.1. Resolving and routing customer complaints](https://vishalsingh.org/decisionai/resolving-and-routing-customer-complaints/report.md): A classifier trained on earlier complaints matches consumers' product choices about as often as Jev once it is fit to the actual mix, and it finds more of the complaints that end in money back, where it passes Jev with far fewer labels than the review task needs. - [Chapter 2.2. Allocating review attention to code changes](https://vishalsingh.org/decisionai/allocating-review-attention-to-code-changes/report.md): On patches written so that every one passes the required tests, the models rank defective changes well above acceptable ones while diff size does no better than chance, but on real agent patches the same models do only modestly better than chance. - [Chapter 3.1. Deciding when an agent is finished](https://vishalsingh.org/decisionai/deciding-when-an-agent-is-finished/report.md): On tasks written for this booklet, an agent's first finish met every checked requirement in 87.5% of runs, sending it back on a model's judgment raised that to 90.0%, about what sending every finish back achieves, and telling it which requirement failed raised that to 94.0%. - [Chapter 4.1. Comparing decision systems](https://vishalsingh.org/decisionai/comparing-and-auditing-decision-systems/report.md): Across reviews, complaints and agent work, the choice between an endpoint and a trained classifier turns on how many labels a task needs and whether the probabilities are calibrated, and an agent's work improved most when a check it could not influence named what had failed.