History · Level 3 · Advanced
Characterizing the Google Books Corpus: Strong Limits to Inferences of Socio-Cultural and Linguistic Evolution
Pechenick EA, Danforth CM, Dodds PS (2015). PLOS ONE. doi:10.1371/journal.pone.0137041 · CC BY 4.0
Step 1 of 2Read the paper
Many studies use Google Books word counts to make claims about culture and history. This paper asks whether the corpus is a fair sample of what people wrote and read. Using an information-theory measure (Jensen–Shannon divergence), the authors compare word use between years and decades from 1800 to 2000 in four English data sets and identify which words drive the changes.
How to read it
- Introduction: the "library" argument (one copy per book), the Frodo example and the figure/Figure test in Fig 2.
- Methods: you do not need the formulas. Know why JSD is used instead of KL divergence and what makes a word contribute.
- Results: compare the word shifts for English vs English Fiction v1 vs v2 (Figs 7–12) and the Lanny Budd example (Fig 13).
- Fig 16 and the Concluding Remarks: why year mentions undermine an earlier "we forget faster" claim.
Step 2 of 2Take the test
15 questions · about 50 minutes including reading · pick an answer to see straight away if it’s right. Nothing is saved.
Loading progress…
Paper: Pechenick EA, Danforth CM, Dodds PS, PLOS ONE (2015), licensed CC BY 4.0. Summary and questions are Research Nest’s own; read the original for the full study.