Part II
Scarce attention
Who should get a person's limited time, on a complaint desk and in a queue of code changes, and how far a tidy test set carries to real work.
Chapters
- 2.1
Resolving and routing customer complaints
A classifier trained on earlier complaints matches consumers' product choices about as often as Jev once it is fit to the actual mix, and it finds more of the complaints that end in money back, where it passes Jev with far fewer labels than the review task needs.
DraftRecordscfpb-complaints
- 2.2
Allocating review attention to code changes
On patches written so that every one passes the required tests, the models rank defective changes well above acceptable ones while diff size does no better than chance, but on real agent patches the same models do only modestly better than chance.
DraftRecordsconstructed-patchesswe-rebenchnebius-openhands-trajectories