Digital Communication and Multimodal Features: Functioning of Emoji in Interpersonal Communication

Cover Page

Cite item


Technical advances and digital means of communication have led to the development of digital semiotics which is characterised by its multimodality and abounds in paralinguistic elements such as emojis, emoticons, memes, etc. These extralinguistic elements serve as a compensatory mechanism in the new communication means. The increasing interest of users in various iconic signs and symbols generates the research interest in different fields of knowledge. The study aims to consider cognitive, semiotic and psycholinguistic features of emojis in interpersonal communication through analysing their functions in text messages and in social network messages. An attempt to reveal their persuasive mechanism is made. The research is based on a large scale dataset comprised of the private text messages as well as public posts on social networks which include verbal and nonverbal / iconic elements. The research data presents a multilingual bank of English, Russian and French sources. The research methods include context analysis, linguistic and pragmatic analysis and content analysis. The findings show that emojis in private interpersonal communication perform a number of functions, namely nonverbal, emotive, pragmatic, punctuation, substitutional, decorative and rhetorical functions. These iconic symbols incorporated in the interpersonal digital communication present a compensatory mechanism and the means of persuasion of a message addressee / recipient. The combination of verbal and iconic elements triggers a double focusing mechanism, and the perception is shaped by all cognitive mechanisms including rational and emotional, unconscious components.

About the authors

Elena A. Koltsova

Saint Petersburg Mining University

Author for correspondence.
ORCID iD: 0000-0001-8724-2181

Candidate of Philological Sciences, Associate Professor, Associate Professor of the Department of Department of Foreign Languages

2, 21st Line, St Petersburg, Russian Federation, 199106

Faina I. Kartashkova

Ivanovo State University

ORCID iD: 0000-0001-7095-9143

Dr. Sc. (Philology), Professor, Professor of the Department of Foreign Philology, Honorary Worker of Higher Professional Education of the Russian Federation, Head of NOC «Laboratory of Communicative Human Behavior»

39, Ermaka st., Ivanovo, Russian Federation, 153025


  1. Dresner, E., & Herring, S.C. (2010). Functions of the Nonverbal in CMC: Emoticons and Illocutionary Force. Communication Theory, 20(3), 249-268. j.1468-2885.2010.01362
  2. Aull, B. (2019). A study of phatic emoji use in WhatsApp communication. Internet Pragmatics, 2(2), 206-232.
  3. Bai, Q., Dan, Q., Mu, Z., & Yang, M. (2019). A Systematic Review of Emoji: Current Research and Future Perspectives. Frontiers in Psychology, 10. fpsyg.2019.02221
  4. Evans, V. (2017). The Emoji Code: The Linguistics Behind Smiley Faces and Scaredy Cats. UK: Michael O’Mara Publ.
  5. Prada, M., Rodrigues, D.L., Garrido, M.V., Lopes, D., Cavalheiro, B., & Gaspar, R. (2018). Motives, frequency and attitudes toward emoji and emoticon use. Telematics and Informatics, 35(7), 1925-1934.
  6. Rodrigues, D., Prada, M., Gaspar, R., Garrido, M.V., & Lopes, D. (2017). Lisbon Emoji and Emoticon Database (LEED): Norms for emoji and emoticons in seven evaluative dimensions. Behavior Research Methods, 50(1), 392-405.
  7. Danesi, M. (2016). The semiotics of emoji: The rise of visual language in the age of the internet. Bloomsbury Publ.
  8. McLuhan, M. (1962). The Gutenberg Galaxy: The Making of Typographic Man. London & New York.
  9. Emoji Report 2015. Emoji Research Team 2015. URL: Report_2015.pdf. (accessed: 20.12.2021).
  10. Kaye, L.K., Wall, H.J., & Malone, S.A. (2016). “Turn that frown upside-down”: A contextual account of emoticon usage on different virtual platforms. Computers in Human Behavior, 60, 463-467.
  11. Ge, J. (2019). Emoji Sequence Use in Enacting Personal Identity. In: Companion Proceedings of The 2019 World Wide Web Conference.
  12. Lu, X., Ai, W., Liu, X., Li, Q., Wang, N., Huang, G., & Mei, Q. (2016). Learning from the ubiquitous language. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing.
  13. Ljubešić, N., & Fišer, D. (2016). A Global Analysis of Emoji Usage. In: Proceedings of the 10th Web as Corpus Workshop.
  14. Cheng, L. (2017). Do I mean what I say and say what I mean? A cross cultural approach to the use of emoticons & emojis in CMC messages. Fonseca Journal of Communication, 15, 199-217.
  15. Chik, A., & Vásquez, C. (2017). A comparative multimodal analysis of restaurant reviews from two geographical contexts. Visual Communication, 16(1), 3-26.
  16. Lin, T.J., & Chen, C.H. (2018). A preliminary study of the form and status of passionate affection emoticons. International Journal of Design, 12(2), 75-90.
  17. Koltsova, Е.А. (2020). New mechanisms of online communication in the 21st century. In: A Person in Modern Communication. Ivanovo. pp. 23-30. (In Russ.).
  18. Herring, S.C., & Dainas, A.R. (2020). Gender and Age Influences on Interpretation of Emoji Functions. ACM Transactions on Social Computing, 3(2), 1-26.
  19. Chen, Z., Lu, X., Ai, W., Li, H., Mei, Q., and Liu, X. (2018). Through a gender lens: learning usage patterns of emojis from large-scale android users. In: Paper Presented at the 2018 World Wide Web Conference on World Wide Web (Lyon).
  20. Butterworth, S.E., Giuliano, T.A., White, J., Cantu, L., & Fraser, K.C. (2019). Sender Gender Influences Emoji Interpretation in Text Messages. Frontiers in Psychology, 10.
  21. Cappallo, S., Svetlichnaya, S., Garrigues, P., Mensink, T., & Snoek, C.G.M. (2019). New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval. IEEE Transactions on Multimedia, 21(2), 402-415.
  22. Gawne, L., & McCulloch, G. (2019). Emoji as digital gestures. Language@ Internet, 17(2).
  23. Na’aman, N., Provenza, H., & Montoya, O. (2017). Varying Linguistic Purposes of Emoji in (Twitter) Context. In: Proceedings of ACL 2017, Student Research Workshop.
  24. Das, G., Wiener, H.J.D., & Kareklas, I. (2019). To emoji or not to emoji? Examining the influence of emoji on consumer reactions to advertising. Journal of Business Research, 96, 147-156.
  25. Ge, J., & Gretzel, U. (2018). Emoji rhetoric: a social media influencer perspective. Journal of Marketing Management, 34(15-16), 1272-1295. 7x.2018.1483960
  26. Voinov, D.A. (2016). Emoji as a technology for political communication on the Internet. Electronic Periodical Scientific Journal the Age of Quality, 4, 103-112. (In Russ.).
  27. Esposito, G., Hernández, P., van Bavel, R., & Vila, J. (2017). Nudging to prevent the purchase of incompatible digital products online: An experimental study. PloS one, 12(3), e0173333.
  28. Moreno-Sandoval, L.G., Sánchez-Barriga, C., Buitrago, K.E., Pomares-Quimbaya, A., & Garcia, J.C. (2018). Spanish Twitter Data Used as a Source of Information About Consumer Food Choice. Machine Learning and Knowledge Extraction, 134-146.
  29. McShane, L., Pancer, E., Poole, M., & Deng, Q. (2021). Emoji, Playfulness, and Brand Engagement on Twitter. Journal of Interactive Marketing, 53, 96-110. intmar.2020.06.002
  30. Chairunnisa, S., & A.S., B. (2017). Analysis of Emoji and Emoticon Usage in Interpersonal Communication of Blackberry Messenger and WhatsApp Application User. International Journal of Social Sciences and Management, 4(2), 120-126 v4i2.17173
  31. Gibson, W., Huang, P., & Yu, Q. (2018). Emoji and communicative action: The semiotics, sequence and gestural actions of “face covering hand.” Discourse, Context & Media, 26, 91-99
  32. Phan, W.M. J., Amrhein, R., Rounds, J., & Lewis, P. (2017). Contextualizing Interest Scales with Emojis: Implications for Measurement and Validity. Journal of Career Assessment, 27(1), 114-133.
  33. Brody, N., & Caldwell, L. (2017). Cues filtered in, cues filtered out, cues cute, and cues grotesque: Teaching mediated communication with emoji Pictionary. Communication Teacher, 33(2), 127-131.
  34. Kelly, R., & Watts, L. (2015). Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of technology appropriation: Unanticipated users, usage, circumstances, and design, 2.
  35. Koltsova, Е.А. (2018). Emoji as a new form of communication: functional and cognitive aspects. Cognitive Studies of Language, 35, 121-128. (In Russ.).
  36. Goren, C.C., Sarty, M., & Wu, P.Y. (1975). Visual following and pattern discrimination of facelike stimuli by newborn infants. Pediatrics, 56(4), 544-549.
  37. Johnson, M.H. (1991). Information processing and storage during filial imprinting. Cambridge: Cambridge University Press.
  38. Ekman, P., Friesen, W.V., & Ellsworth, P. (1972). Introduction to Research Findings. Emotion in the Human Face.
  39. Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223-233. s1364-6613(00)01482-0
  40. Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2002). Human neural systems for face recognition and social communication. Biological Psychiatry, 51(1), 59-67. s0006-3223(01)01330-0
  41. Yuasa, M., Saito, K., & Mukawa, N. (2011). Brain activity when reading sentences and emoticons: an fMRI study of verbal and nonverbal communication. Electronics and Communications in Japan, 94(5), 17-24.

Supplementary files

Supplementary Files
1. Fig. 1. Emoji as a nonverbal component of communication

Download (10KB)
2. Fig. 2. Emotive function of emojis

Download (2KB)
3. Fig. 3. Emoji chain

Download (1KB)
4. Fig. 4. Emojis in phatic communication

Download (3KB)
5. Fig. 5. Punctuation function of emojis

Download (5KB)
6. Fig. 6. Substitutional function of emojis

Download (5KB)
7. Рис. 7. Декоративная функция эмодзи

Download (3KB)

Copyright (c) 2022 Koltsova E.A., Kartashkova F.I.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies