A field booklet · Data-Driven Decision Making
The Measure of Words
Text as data, from word counts to language models.
Six data essays, two live studios, and a small data shelf on turning language into evidence — how analysts count, discover, and measure meaning in text, from dictionary word counts to LLM annotation at scale.
by Vishal Singh · Professor of Marketing, NYU Stern
- chapters
- 06 chapters
- live studios
- 02 live studios
- datasets
- 04 datasets
- documents measured
- 300k+ documents measured
How to read this booklet
Three moves, in order
Every essay that follows uses one or more of these moves. Together they are the working grammar of text as data.
I
Counting
Dictionaries, word frequencies, readability. Transparent, fast, and auditable — the measures you can defend line by line.
II
Discovering
Topic models, clustering, embeddings. Let the corpus reveal its own structure before you impose one on it.
III
Measuring
LLMs as annotators: context-sensitive constructs — framing, stance, moral language — scored at scale and validated like any instrument.
Contents
The chapters
Read in order — the field guide first, then four corpora of increasing scale, ending where classification hands off to LLM measurement. Each opens in a new tab.
- 01
Field guide · Three generations of text-as-data
01Counting, Discovering, Measuring — text analysis with and without LLMs
Dictionaries count, topic models discover, and language models measure. The opening essay maps the whole toolkit — what each generation of text analysis can and cannot see, and when a word count still beats a transformer.
DictionariesTopic modelsLLM measurementRead - 02
Case study · The @realDonaldTrump archive · 2013–2018
02Two Thumbs, One Account
During the 2016 cycle one account posted from two devices — staff on an iPhone, the candidate on an Android. Word frequencies, time-of-day signatures, and sentiment indices are enough to tell the two thumbs apart.
Authorship classificationSentiment indicesRead - 03
Case study · ≈20,000 tweets around an acquisition
03Did Goose Island Sell Out?
When Anheuser-Busch InBev bought Chicago’s Goose Island in 2011, Twitter supplied a natural experiment: split the mentions into the weeks before and after the deal and measure what “selling out” actually does to a craft brand’s voice.
SentimentEvent windowRegressionRead - 04
Corpus study · 91,493 annual reports · fifteen years
04The Risk Section That Ate the 10-K
Corporate risk disclosure tripled in length, got harder to read, and learned new words on a schedule you can date — cyber, pandemic, climate. Boilerplate, measured at industrial scale.
ReadabilityVocabulary datingN-gramsRead - 05
Corpus study · 209,527 HuffPost headlines · 2012–2022
05When Politics Ate the Newsroom
Politics triples its share of the newsroom, the listicle dies, and the arrival of every word can be dated. Ten years of headlines read as a time series of editorial attention.
Category sharesTerm timelinesRead - 06
LLM measurement · Broadcast news, classified at scale
06AI’s Split-Screen Politics
Left-leaning channels cast AI as a classroom, a risk, and a governance problem; right-leaning channels cast it as a business engine and a national race. LLM classification turns framing itself into a measurable variable.
LLM classificationFraming analysisRead
Interlude
Live studios
The essays are fixed arguments; these are open instruments. Both run on live text feeds — bring your own question.
Live studioGlobal news & television · live GDELT APIs
GDELT Media Agenda Lab
Search global news and TV coverage as an agenda-setting lab: compare attention, tone, source geography, and station airtime, with evidence cards built from live queries.
Open studio
Live studioMillions of CFPB consumer complaints
CFPB Crisis Monitor
Public complaint narratives as a crisis early-warning system: pin incident spikes, read consented narratives, and separate product-mix shifts from real operational improvement.
Open studioAppendix
The data shelf
The corpora behind the chapters, packaged for the classroom — download, replicate, disagree.
Trump tweet device corpus
Tweets with device metadata — the raw material behind Chapter 02.
DatasetBeer acquisition tweet sentiment corpus
Goose Island mentions, before and after the deal — Chapter 03’s corpus.
DatasetPolitical books review corpus
Review text with ratings and metadata, ready for classification exercises.
DatasetRentHop apartment listings
Listing descriptions as features — text meets structured prediction.
Dataset
Companion reading
Where the theory lives
Every method used above has a chapter in the D3M book — Part V walks from bag-of-words to embeddings, GPT-as-measurement, and structured LLM outputs.
Next in this booklet
A chapter in production
The next essays extend the same measurement discipline a century back in time.
In production · arriving soon
Moral Language and Political Orientation in the Historical American Press
The digitized American press · 1890–1935
How did newspapers use moral language to make policies and social change look legitimate — or illegitimate? Moral-foundations dictionaries and context-sensitive LLM annotation over millions of historical newspaper articles, anchored on two first cases: Prohibition (1918–1933) and the influenza restrictions of 1918–19.