How much consumer complaint text is written with AI?

Among complaints to the Consumer Financial Protection Bureau about products other than credit reporting, the estimated share of narratives written or polished with a chatbot was 0.2% in January 2023, 12.9% in August 2024 and 58.8% on average from January to June 2026. These estimates cover narratives that are not copies of another complaint. In credit reporting the rise was steeper: 16.4% by December 2023 and 67.9% in July 2025, where the credit-reporting series ends because few of those complaints were published afterward. This case shows how those numbers were produced from 3.6 million published complaints, why the first answer a detector gives is wrong, and what the data say about who uses AI to complain and whether it changes what companies do.

The question matters to anyone who reads complaints for a living. Companies, regulators and researchers treat the length, detail and tone of a complaint as signals of how serious it is and how much effort the consumer put in. When a chatbot can turn three angry sentences into a formal letter that cites the Fair Credit Reporting Act, those signals change meaning.Related work. Liang et al. estimated the AI share of CFPB complaints through August 2024 with a word-frequency method.¹ Pew Research Center measured AI text on the web with a document-level detector.² This case uses both approaches and extends the series to mid-2026.

The data: 3.6 million narratives, frozen in August 2026

When consumers file a complaint with the CFPB they can describe what happened in their own words and agree to have that description published. On August 14, 2026 the CFPB stopped publishing these narratives and moved them to a public archive.³ The archive used here holds 3,559,841 narratives received from May 2018 to August 2026, with the structured fields the CFPB publishes for every complaint: product, issue, company, state, ZIP code, tags, the company's response and whether it responded on time.⁴

Three features of the archive shape every result below. First, the agency edits the text before publishing it. Names, dates and account numbers become XXXX, and from 2020 on almost every non-ASCII character is removed: 564 of 199,462 narratives from 2018 and 2019 contain one, against 22 of 3,360,379 from 2020 on. The em dash, a character often cited as a sign of chatbot writing, therefore cannot be observed at all. Second, 62.8% of narratives are near-duplicates of at least one other narrative, many of them form letters filed by credit repair services. In credit reporting the clustered share rose from 57.8% of narratives in 2018 to 87.8% in 2024. Third, fewer consumers chose to publish: 42.1% of complaints came with a narrative in 2022 and 22.5% in 2025, and credit-reporting narratives fall sharply after July 2025 and almost stop in January 2026.

Counting chatbot vocabulary is the obvious first step, and it finds a large change.⁵ The word "formally" appeared in 0.3% of narratives a month in 2019 to 2022 and in 9.6% from August 2024 to July 2025, 35 times its projected trend. Vocabulary alone cannot separate AI drafting from other changes in how people write to the CFPB, though. Fitting the same trend to 2019 and 2020 and projecting to 2022, before any chatbot was public, produces ratios up to 25 from the spread of credit-repair letters alone. Measuring the share of AI text needs a method whose errors can be measured.

A detector, checked against 7,000 chatbot rewrites

The detector is a decision model reached through OpenRouter's Decisions API.⁶ It reads a narrative and answers one question, "Was this complaint written, rewritten or polished with an AI chatbot such as ChatGPT?", with a probability between 0 and 1. A narrative is flagged when the probability is 0.18 or higher. Nothing is generated, so scoring costs only the input text: 493,976 narratives cost $8.38.

A detector's score means little until its errors are measured on texts whose origin is known. The validation set starts from 500 narratives received in 2019 to 2021, before any chatbot was public, with no near-duplicate and 40 to 600 words. Seven chatbots rewrote each of them twice, once with a request to "polish" the grammar and tone and once with a request to rewrite it as a formal complaint, giving 6,997 usable rewrites. The rewrites were put through the CFPB's own edits (ASCII only, spaces around colons and parentheses) so they look like published text.Choosing the detector. Seven decision models were scored on the same set. Liquid d1-ultra and Nace.AI Drex Max both reached an AUC of 0.987; d1-ultra was chosen for its lower false-positive rate (1.2% against 4.8%). AUC is the probability that a random rewrite scores above a random original.

At the 0.18 cutoff, d1-ultra flags 1.2% of the 500 originals (its false-positive rate) and 91.9% of the 6,997 rewrites (its true-positive rate). Figure 1 shows where it misses. Full rewrites are flagged 93% to 100% of the time for every chatbot. Light polishing is harder: the detector flags 65.3% of GPT-5's polished versions and 76.4% of Claude Sonnet 4.5's.

Correcting the flag rate for the detector's errors

The share of narratives a detector flags is not the share written with AI. It counts false positives among human texts and misses some AI texts. If p is the true AI share, the flag rate is p·TPR + (1 − p)·FPR, which can be solved for p:

TPR is the 91.9% measured on the rewrites. The false-positive rate is better measured on the complaints themselves: every narrative received in 2019 to 2021 was written without a chatbot, so whatever the detector flags in those years is a false positive. That turns out to depend on the kind of narrative. Figure 2 compares one-off narratives with members of near-duplicate clusters. From 2019 to 2022 the detector flagged 1.3% to 3.3% of one-off narratives a year and 4.5% to 14.1% of cluster members. Form letters from credit repair services read like a chatbot to the detector even when they were written years before ChatGPT.

The estimates in the rest of this case therefore use narratives with no near-duplicate, with a false-positive rate measured on the same kind of narrative in 2019 to 2021: 1.9% for credit reporting and 2.0% for other products in the stratified sample. A template written once with a chatbot and filed a thousand times would count a thousand times in an all-narratives series; restricting to one-off narratives counts writers more closely.

Adoption began in spring 2023 and had not leveled off by mid-2026

Figure 3 shows the monthly estimate for every narrative with no near-duplicate and 40 to 600 words, 1,005,680 narratives from January 2019 to June 2026. It uses a second method that does not depend on the detector: the word-frequency estimator of Liang et al.¹ Each narrative is reduced to the set of words it contains. The estimator compares how often 100 informative words appear in human complaints and in the chatbot rewrites, and finds the mixture of the two that best explains each month's narratives. No individual narrative is classified.Calibration. On test mixtures with a known AI share the raw estimate follows the line 0.022 + 0.777×share, so the reported estimates are corrected by that line, the same role FPR and TPR play for the detector.

Before November 2022 the monthly estimates lie between −1.1% and 0.7% for other products, which is the method's noise around zero. The first month clearly above that range is March 2023 for other products (2.1%) and May 2023 for credit reporting, three to five months after ChatGPT's release on November 30, 2022. Liang et al. found a rise starting three to four months after release.¹ For other products the estimate was 7.7% in December 2023, 12.9% in August 2024, 19.1% in December 2024, 40.2% in June 2025 and 51.4% in December 2025. It rose by 2.7 percentage points a month from July 2024 to June 2025 and by 2.2 points a month in the following year, so the series had not leveled off when the archive ends.

One month stands out. In January 2025 the estimate for other products jumps to 45.0%, from 19.1% in December 2024, and falls back to 27.7% in February. That month Block, Inc. and Early Warning Services received a wave of complaints, many of them distinct texts that the near-duplicate filter does not catch; they make up 28% of that month's narratives about other products in the estimate. Without those two companies the January 2025 estimate is 28.9%.

The detector agrees with the word-frequency method. Figure 4 puts the two side by side by quarter for narratives with no near-duplicate. For other products the detector gives 13.3% in the second quarter of 2024 and 62.5% in the second quarter of 2026; the word-frequency method gives 9.6% and 64.1%. In 13 of the 15 quarters from late 2022 to mid-2026 the two are within 3.7 percentage points of each other. They differ by 9.2 points in the first quarter of 2025, the quarter of the Block wave, and by 5.7 points in the third. Credit-reporting estimates differ more because the sample holds few credit-reporting narratives with no near-duplicate. Agreement does not prove both are right, since both learned what AI text looks like from the same 6,997 rewrites, but they reach it through different features: the detector reads each whole narrative, and the word-frequency method counts 100 words across thousands.

Who uses a chatbot to complain

To compare groups, d1-ultra scored every narrative with no near-duplicate received from January 2024 to June 2025, 393,976 in all, plus 100,000 from 2019 to 2021 to measure the false-positive rate. Over this window 28.5% of these narratives are estimated to be AI-assisted (95% interval 28.1% to 28.8%): 39.2% in credit reporting and 22.6% in other products.Why this window. It is the last period in which both credit reporting and other products have enough published narratives. Every number in this section is an average over January 2024 to June 2025, a period in which the share roughly tripled.

The differences across products are large (Figure 5). Among other products the share runs from 36.0% for money transfers and virtual currency to 8.8% for prepaid cards. The largest gap is by age. Consumers can tag a complaint as coming from an older American (62 or over) or a servicemember. The estimate is 6.1% for complaints tagged Older American, against 30.8% for untagged complaints, a gap of 18.9 percentage points when product and month are held fixed. Servicemember complaints are in between, at 22.4%.

Places differ too. The narratives carry the consumer's ZIP code, which links each complaint to census data on the surrounding three-digit ZIP area. Figure 6 sorts ZIP3 areas into fifths with equal numbers of narratives. The estimated share rises from 21.3% in the least urban fifth to 34.0% in the most urban, and from 21.4% in areas where the fewest residents were born abroad to 33.6% where the most were. It falls from 33.5% to 22.5% across fifths ordered by the share of residents 65 and older, which matches the Older American tag. With product and month held fixed, the most and least urban fifths differ by 11.8 percentage points and the most and least foreign-born by 12.3.

The education pattern runs opposite to Liang et al., who found higher adoption in areas with lower educational attainment.¹ Here the share is 25.6% in the fifth of ZIP3 areas with the fewest college graduates and 29.1% in the fifth with the most, a gap of 5.6 points (3.8 to 7.4) with product and month fixed. The census measures move together across areas, and this comparison does not separate them. Area characteristics also describe neighborhoods, not complainants: a higher share in areas with many immigrants does not show that immigrants are the ones using chatbots.

By state, the estimate for other products ranges from 9.6% in West Virginia to 28.6% in the District of Columbia (Figure 7). Liang et al. found Arkansas and Missouri highest from January to August 2024. Over the same months these data rank them 14th and 21st of 51 for all products. The two studies differ in method and in coverage: theirs covers all complaints with narratives, and this one leaves out near-duplicates, which vary a great deal by state.

Reworded templates pass the duplicate filter

Credit reporting is different. Pennsylvania ranks 2nd of 51 for credit reporting but 14th for other products, and one area stands out. In ZIP3 191 (Philadelphia), consumers filed 5,854 narratives with no near-duplicate in the window, 37% of Pennsylvania's. The detector flags 42.8% of them, against 27.8% nationally. Reading the flagged narratives shows why. Many begin with a version of the same sentence:

Section 605B of the Fair Credit Reporting Act lets identity-theft victims block accounts from their credit reports.⁷ Of Philadelphia's narratives that cite it, 44% open with a reworded version of this sentence, against 4% elsewhere. Across the country, narratives citing 605B were 0.6% of credit-reporting narratives with no near-duplicate in 2019 to 2021 and 4.4% in January 2024 to June 2025, and the detector flags 49.3% of them against 36.9% of other credit-reporting narratives.

The near-duplicate filter compares sequences of five words, so it catches copies but not rewordings. A filer who has a chatbot reword one letter for each client produces exactly what this case counts as AI-assisted complaints, but the decision to use AI was made once, by the filer. The credit-reporting estimates by place therefore partly measure where such filers' clients live. The estimates for other products, where 605B letters are 0.3% of narratives, are the better guide to where consumers themselves use AI.

Flagged complaints do not get more relief

Each published complaint records how the company closed it. For products other than credit reporting, 8.7% of unflagged narratives from January 2024 to June 2025 were closed with monetary relief, against 5.7% of flagged ones, a gap of 3.0 percentage points. Most of that raw gap reflects which companies and products receive AI-assisted complaints. Comparing complaints to the same company, about the same product, in the same month, the gap shrinks to 0.8 points (95% interval 0.3 to 1.2 points). Adding the issue and the narrative's length widens it to 1.8 points, because flagged narratives are longer (a median of 210 words against 124) and longer narratives get monetary relief more often.

The comparison that matters is the gray one in Figure 8. Before ChatGPT, every flagged narrative was written by a person whose style the detector mistook for a chatbot's. Among those complaints the same within-company gap is 0.7 points, with an interval from 2.3 points lower to 0.9 points higher. The 2024 to 2025 gap lies inside that range, so these data cannot separate an effect of using AI from an effect of writing in a formal, chatbot-like style. For any relief, monetary or not, the within-company gap is 0.3 points lower for other products and 3.8 points in favor of flagged complaints in credit reporting, with an interval from 0.3 points lower to 7.9 higher. Timely responses are 97% to 99% for every group, flagged or not.

These gaps are associations. Consumers who use a chatbot may differ from those who do not in persistence, knowledge of their rights or the strength of their case, and any of these could move the chance of relief in either direction. Firms may also learn to discount polished text. A causal estimate would need variation in AI use unrelated to the complaint's merits, such as a randomized offer of a writing assistant on a complaint portal.

What these data cannot show

Both methods learned what AI text looks like from 6,997 rewrites of 2019 to 2021 complaints by seven chatbots under two requests. Consumers who drafted from scratch, used an editing tool or edited the chatbot's output are covered only as far as these rewrites resemble their text, and newer chatbots in 2026 may be harder to detect. If so, the later estimates are too low. The correction also holds the false-positive rate at its 2019 to 2021 level. If people's own writing moved toward the style the detector flags, the later estimates are too high.

The group comparisons assume the detector's true-positive rate is the same across groups, which the validation set cannot check by state or ZIP area. Every share is a share of published narratives, and the decision to publish may itself depend on how the complaint was written. Finally, the estimates count narratives, and paraphrased templates show that one decision to use AI can produce thousands of them.

Questions for discussion

1. If chatbots make complaints longer, more formal and better organized, who gains? Older consumers and rural areas use them least. Does AI make it easier for some consumers to be heard, or will firms learn to discount polished text?
2. Firms and regulators use complaint text to triage. What happens to that signal when a formal, detailed letter costs nothing to produce? Compare online reviews, where length and detail have been used as signals of quality.
3. The duplicate filter missed reworded templates. How would you redesign it, and what would you lose by grouping complaints by meaning rather than by wording?
4. The CFPB stopped publishing narratives in August 2026. What research and teaching are lost when an administrative text source goes dark, and what does a frozen archive still allow?

@misc{singh2026cfpb,
  author = {Singh, Vishal},
  title  = {How much consumer complaint text is written with AI?},
  year   = {2026},
  note   = {Teaching case, NYU Stern School of Business},
  url    = {https://vishalsingh.org}
}