Appeal to values is fundamental in mass persuasive communication. Appropriate framing may enable effective communication campaigns designed to promote vaccination uptake or pro-environmental attitudes, or can be employed to detect propaganda attempts. In this tutorial, we give an overview of how the basic human and moral values are interpreted and quantified according to the psychological literature, how they can be assessed from user generated data, and how they may be employed in persuasion and propaganda identification.

  • When: July 17, 2023
  • Where: @IC2S2 2023 in Mærsk Tower of the University of Copenhagen, Copenhagen, Denmark
  • Tutorial Slides: are available here.


In the first part of the tutorial, we provide an overview of traditional survey methods [24, 10], and discuss their applicability to the new forms of discourse, the validity of recruitment using the Internet [26, 6] and social media [15, 23, 14]. We briefly cover the entailed biases of each source as well as the entire pipeline [22, 5]. Finally, we showcase studies where applying computational methods to large amounts of social media data helped understanding the underlying values associated with specific domains, such as politics [25, 16, 21], health [20, 18, 19], charitable giving [12], and privacy [1].

Hands-on demonstration

In the second part of this tutorial, we will lead a hands-on demonstration of tools for (1) moral value extraction from text [3, 4], (2) network analysis for opinion clustering [7], and (3) persuasion techniques identification [8, 28] in two scenarios: the COVID-19 vaccination debate and the recent Russian invasion of Ukraine. Thus, the tools will include NLP, network analysis, and persuasion detection techniques giving the attendees tools for incorporating human value analysis in their social media communications research.

No technical prior knowledge of natural language processing, network analysis and machine learning is assumed. Familiarity with Python is helpful for getting the most out of the hands-on session.

Tutorial Outline

  1. Human and moral values
  2. Propaganda, persuasion & coordinated behaviour
  3. Controversy detection

Hands-on Session

As part of the hands-on tutorial, you are invited to execute code in-site. To do so, it is recommended to previously install the following pip packages: numpy, pandas, searborn, matplotlib, scipy, scikit-learn, tqdm, moralstrength. It is possible to install them with the following command:

pip install numpy pandas searborn matplotlib scipy scikit-learn tqdm moralstrength

Tutorial Speakers

Yelena Mejova

ISI Foundation, Italy

Yelena Mejova is a Senior Research Scientist at the ISI Foundation, Turin, Italy, working in the area of Data Science for Social Impact and Sustainability. Her research concerns the use of social media in health informatics, especially in lifestyle diseases, as well as for tracking political speech and other cultural phenomena. Previously as a scientist at the Qatar Computing Research Institute, Yelena was a part of the Social Computing Group working on computational social science, especially as applied to tracking real-life health signals. She was the general co-chair of ICWSM’22 and currently, she is the co-Editor-in-Chief of EPJ Data Science.

Kyriaki Kalimeri

ISI Foundation, Italy

Kyriaki Kalimeri is a researcher at the ISI Foundation, Turin, Italy. She received her PhD in Brain and Cognitive Sciences from the University of Trento and her Diploma in Electrical and Computer Engineering from the Technical University of Crete. Her research lies at the intersection of computational social science, social media analysis, and machine learning. The focus is on the automatic prediction of psychological characteristics and moral worldviews from digital data, employing machine learning techniques, translating data into insights for the design of effective communication strategies. She co-organized (with Yelena Mejova) the Social Media and Health workshop in ICWSM’18.

Giovanni Da San Martino

Department of Mathematics, University of Padova, Padova, Italy.

Giovanni Da San Martino is Assiociate Professor at the University of Padova. His research interests are at the intersection of machine learning and natural language processing. He has co-authored more than 90 papers on the subject in journals and top tier conferences. He received his Ph.D from the University of Bologna in 2009. Prior to joining the University of Padova, he was a Scientist at Qatar Computing Research Institute. He served as general chair for CLEF 2022 and he has been organiser of several events around the topic of propaganda detection and disinformation: workshops (SemEval 2023, CLEF'19--CLEF'22, SocInfo'19, NLP4IF'19--NLP4IF'22), shared tasks (EMNLP'19, SemEval2020, SemEval2021)), tutorials (IJCAI'20, EMNLP'20, WSDM'22, WWW'22). He is member of the Editorial Board of the journals Neural Networks and Information Processing & Management.

Oscar Araque

Universidad Politécnica de Madrid (Technical University of Madrid, UPM)

Oscar Araque is currently Assistant Professor at Universidad Politécnica de Madrid (UPM). His research interest includes the application of machine-learning techniques for natural language processing. In addition, his research interests lie in introducing specific domain knowledge into machine learning systems to enhance sentiment and emotion analysis techniques and their applications to new domains, such as radicalization narratives. His work has received four distinguished prizes: Most Cited Scientific Paper Award 2020 by the Universidad Politécnica de Madrid, Prize for the Most Cited Scientific Article Originating from a UPM Doctoral Thesis 2021, ISDEFE Award for the Best Doctoral Thesis in Security and Defense 2020, and Extraordinary Doctoral Thesis Award - ETSIT UPM 2022.


[1]Norah Abokhodair, Sofiane Abbar, Sarah Vieweg, and Yelena Mejova. Privacy and social media use in the Arabian Gulf: Saudi Arabian & Qatari traditional values in the digital world. The Journal of Web Science, 3, 2017.
[2] Avnika B Amin, Robert A Bednarczyk, Cara E Ray, Kala J Melchiori, Jesse Graham, Jeffrey R Huntsinger, and Saad B Omer. Association of moral values with vaccine hesitancy. Nature Human Behaviour, 1(12):873–880, 2017.
[3]Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. Moralstrength: Exploiting a moral lexicon and embedding similarity for moral foundations prediction. Knowledge-based systems, 191:105184, 2020.
[4]Oscar Araque, Lorenzo Gatti, and Kyriaki Kalimeri. Libertymfd: A lexicon to assess the moral foundation of liberty. In Proceedings of the 2022 ACM Conference on Information Technology for Social Good, pages 154–160, 2022.
[5]Mariano G Beiro and Kyriaki Kalimeri. Fairness in vulnerable attribute prediction on social media. Data Mining and Knowledge Discovery, 36(6):2194–2213, 2022.
[6]Matthew JC Crump, John V McDonnell, and Todd M Gureckis. Evaluating Amazon’s mechanical turk as a tool for experimental behavioral research. PloS one, 8(3):e57410, 2013.
[7]Giuseppe Crupi, Yelena Mejova, Michele Tizzani, Daniela Paolotti, and Andre Panisson. Echoes through time: Evolution of the Italian covid-19 vaccination debate. In Proceedings of the International AAAI Conference on Web and Social Media, volume 16, pages 102– 113, 2022.
[8]Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barron-Cedeno, and Preslav Nakov. Prta: A system to support the analysis of propaganda techniques in the news. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 287–293, 2020.
[9]Giovanni Da San Martino, Seunghak Yu, Alberto Barron-Cedeno, Rostislav Petrov, and Preslav Nakov. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 5636–5646, 2019.
[10]Jesse Graham, Brian A Nosek, Jonathan Haidt, Ravi Iyer, Spassena Koleva, and Peter H Ditto. Mapping the moral domain. Journal of personality and social psychology, 101(2):366, 2011.
[11]Jonathan Haidt and Craig Joseph. Intuitive ethics: How innately prepared intuitions generate culturally variable virtues. Daedalus, 133(4):55–66, 2004.
[12]Joe Hoover, Kate Johnson, Reihane Boghrati, Jesse Graham, and Morteza Dehghani. Moral framing and charitable donation: Integrating exploratory social media analyses and confirmatory experimentation. Collabra: Psychology, 4(1), 2018.
[13]Garth S Jowett and Victoria O’donnell. Propaganda & persuasion. Sage publications, 2018.
[14]Kyriaki Kalimeri, Mariano G Beiro, Andrea Bonanomi, Alessandro Rosina, and Ciro Cattuto. Traditional versus Facebook-based surveys. Demographic research, 42:133–148, 2020.
[15]Kyriaki Kalimeri, Mariano G Beiro, Matteo Delfino, Robert Raleigh, and Ciro Cattuto. Predicting demographics, moral foundations, and human values from digital behaviours. Computers in Human Behavior, 92:428–445, 2019.
[16]Huyen T Le, GR Boynton, Yelena Mejova, Zubair Shafiq, and Padmini Srinivasan. Revisiting the american voter on twitter. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 4507–4519, 2017.
[17]Michelle Low, Ma Wui, and Glenda Lopez. Moral foundations and attitudes towards the poor. Current Psychology, 35(4):650–656, 2016.
[18]Yelena Mejova. Information sources and needs in the obesity and diabetes twitter discourse. In Proceedings of the 2018 international conference on digital health, pages 21–29, 2018.
[19]Yelena Mejova, Youcef Benkhedda, and Khairani Khairani. Halal culture on instagram. frontiers in digital humanities 4 (2017), 21, 2017.
[20]Yelena Mejova and Kyriaki Kalimeri. Effect of values and technology use on exercise: implications for personalized behavior change interventions. In Proceedings of the 27th ACM Conference on User Modeling, Adaptation and Personalization, pages 36–45, 2019.
[21]Marlon Mooijman, Joe Hoover, Ying Lin, Heng Ji, and Morteza Dehghani. Moralization in social networks and the emergence of violence during protests. Nature human behaviour, 2(6):389–396, 2018.
[22]Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman. Social data: Biases, methodological pitfalls, and ethical boundaries. Frontiers in Big Data, 2:13, 2019.
[23]Michael F Schober, Josh Pasek, Lauren Guggenheim, Cliff Lampe, and Frederick G Conrad. Social media analyses for social measurement. Public opinion quarterly, 80(1):180– 211, 2016.
[24]Shalom H Schwartz. An overview of the schwartz theory of basic values. Online readings in Psychology and Culture, 2(1):2307–0919, 2012.
[25]Nathaniel Swigger. The online citizen: Is social media changing citizens’ beliefs about democratic values? Political behavior, 35(3):589–603, 2013.
[26]Zeynep Tufekci. Big questions for social media big data: Representativeness, validity and other methodological pitfalls. In Eighth international AAAI conference on weblogs and social media, 2014.
[27]Christopher Wolsko, Hector Ariceaga, and Jesse Seiden. Red, white, and blue enough to be green: Effects of moral framing on climate change attitudes and conservation behaviors. Journal of Experimental Social Psychology, 65:7–19, 2016.
[28]Yifan Zhang, Giovanni Da San Martino, Alberto Barron-Cedeno, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, et al. Tanbih: Get to know what you are reading. arXiv preprint arXiv:1910.02028, 2019.
Template by pFind Goodies