Modeling and Measuring Expressed (Dis)belief in (Mis)information

Shan Jiang, Miriam Metzger, Andrew Flanagin, Christo Wilson

Paper type: Full

Keywords: claims, classifiers, large_scale, measures, misinformation, tweets

2020-06-09 P3 (22:00-23:00 GMT) [Zoom] [Cal]

Abstract: The proliferation of online misinformation has been raising increasing societal concerns about its potential consequences, e.g., polarizing the public, eroding trust in institutions. These consequences are framed under the public's susceptibility to such misinformation -- a narrative that needs further investigation and quantification. To this end, our paper proposes an observational approach to model and measure expressed (dis)beliefs in (mis)information by leveraging social media comments as a proxy. We collect a sample of tweets in response to misinformation and annotate them with (dis)belief labels, explore the dataset using lexicon-based methods, and finally build classifiers based on the state-of-the-art neural transfer-learning models. Under a domain-specific thresholding strategy, the best-performing unbiased classifier archives macro-F1 scores around 0.86 for disbelief and 0.80 for belief. Applying the classifier, we conduct a large-scale measurement study and show that, overall, 12%--15% social media comments express disbelief and 26%--20% express belief, with the left-bounds representing comments in response to true claims and the right-bounds to false ones. Our results also suggest an extremely slight time effect of falsehood awareness, a positive effect of fact-checks to false claims, and a difference in (dis)belief across social media platforms.

Similar Papers

Analysing the Extent of Misinformation in Cancer Related Tweets
Rakesh Bal , Sayan Sinha , Swastika Dutta , Risabh Joshi , Sayan Ghosh , Ritam Dutt
A Benchmark Dataset of Check-Worthy Factual Claims
Fatma Arslan , Naeemul Hassan , Chengkai Li , Mark Tremayne
When Does Trust in Online Social Groups Grow?
Shankar Iyer , Justin Cheng , Nick Brown , Xiuhua Wang
The Effects of an Informational Intervention on Attention to Anti-Vaccination Content on YouTube
Sangyeon Kim , Omer F. Yalcin , Samuel E. Bestvater , Kevin Munger , Burt L. Monroe , Bruce A. Desmarais