Variation across Scales: Measurement Fidelity under Twitter Data Sampling

Siqi Wu, Marian-Andrei Rizoiu, Lexing Xie

Paper type: Full

Keywords: attention, bias, cascades, changes, collection, graphs, influences, measures, networks, rates, structure, terms, tweets, twitter

2020-06-11 P12 (22:00-23:00 GMT) [Zoom] [Cal]

Abstract: A comprehensive understanding of data bias is the cornerstone of mitigating biases in social media research. This paper presents in-depth measurements of the effects of Twitter data sampling across different timescales and different subjects (entities, networks, and cascades). By constructing two complete tweet streams, we show that Twitter rate limit message is an accurate measure for the volume of missing tweets. Despite sampling rates having clear temporal variations, we find that the Bernoulli process with a uniform rate well approximates Twitter data sampling, and it allows to estimate the ground-truth entity frequency and ranking with the observed sample data. In terms of network analysis, we observe significant structure changes in both the user-hashtag bipartite graph and the retweet network. Finally, we measure the retweet cascades. We identify risks for information diffusion models that rely on tweet inter-arrival times and user influence. This work calls attention to the social data bias caused by data collection, and proposes methods to measure the systematic biases introduced by sampling.

Similar Papers

Influence Maximization Using Influence and Susceptibility Embeddings
George Panagopoulos , Fragkiskos D. Malliaros , Michalis Vazirgiannis
Generating Realistic Interest-Driven Information Cascades
Federico Cinus , Francesco Bonchi , Corrado Monti , André Panisson
When Your Friends Become Sellers: An Empirical Study of Social Commerce Site Beidian
Hancheng Cao , Zhilong Chen , Fengli Xu , Tao Wang , Yujian Xu , Lianglun Zhang , Yong Li