Representing the Twittersphere: Archiving a representative sample of Twitter data under resource constraints

Airo Hino*, Robert A. Fahey

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

27 Citations (Scopus)


The rising popularity of social media posts, most notably Twitter posts, as a data source for social science research poses significant problems with regard to access to representative, high-quality data for analysis. Cheap, publicly available data such as that obtained from Twitter's public application programming interfaces is often of low quality, while high-quality data is expensive both financially and computationally. Moreover, data is often available only in real-time, making post-hoc analysis difficult or impossible. We propose and test a methodology for inexpensively creating an archive of Twitter data through population sampling, yielding a database that is highly representative of the targeted user population (in this test case, the entire population of Japanese-language Twitter users). Comparing the tweet volume, keywords, and topics found in our sample data set with the ground truth of Twitter's full data feed confirmed a very high degree of representativeness in the sample. We conclude that this approach yields a data set that is suitable for a wide range of post-hoc analyses, while remaining cost effective and accessible to a wide range of researchers.

Original languageEnglish
Pages (from-to)175-184
Number of pages10
JournalInternational Journal of Information Management
Publication statusPublished - 2019 Oct


  • Data collection
  • Representativeness
  • Sampling
  • Social media
  • Twitter

ASJC Scopus subject areas

  • Information Systems
  • Computer Networks and Communications
  • Library and Information Sciences


Dive into the research topics of 'Representing the Twittersphere: Archiving a representative sample of Twitter data under resource constraints'. Together they form a unique fingerprint.

Cite this