A corpus of online discussions for research into linguistic memes

Abstract

We describe a 460-million word corpus of online discussions. The data are collected from public news websites and community-of-interest Internet forums, and are designed to support research on the propagation of socially relevant ideas, a.k.a., “memes.” A structural and statistical description of the corpus is given, and the employed methods of website monitoring, collection, and extraction are described. We also present preliminary linguistic research on the corpus. We show that the corpus represents language from a wide variety of social and psychological communities, that discussion structure and popularity can be predicted in large part from lexical analysis, and that standard epidemiological models provide good fit for diachronic patterns of population-level lexical adoption.