Digital Communication and Multimodal Features: Functioning of Emoji in Interpersonal Communication

Cover Page

Cite item

Full Text

Abstract

Technical advances and digital means of communication have led to the development of digital semiotics which is characterised by its multimodality and abounds in paralinguistic elements such as emojis, emoticons, memes, etc. These extralinguistic elements serve as a compensatory mechanism in the new communication means. The increasing interest of users in various iconic signs and symbols generates the research interest in different fields of knowledge. The study aims to consider cognitive, semiotic and psycholinguistic features of emojis in interpersonal communication through analysing their functions in text messages and in social network messages. An attempt to reveal their persuasive mechanism is made. The research is based on a large scale dataset comprised of the private text messages as well as public posts on social networks which include verbal and nonverbal / iconic elements. The research data presents a multilingual bank of English, Russian and French sources. The research methods include context analysis, linguistic and pragmatic analysis and content analysis. The findings show that emojis in private interpersonal communication perform a number of functions, namely nonverbal, emotive, pragmatic, punctuation, substitutional, decorative and rhetorical functions. These iconic symbols incorporated in the interpersonal digital communication present a compensatory mechanism and the means of persuasion of a message addressee / recipient. The combination of verbal and iconic elements triggers a double focusing mechanism, and the perception is shaped by all cognitive mechanisms including rational and emotional, unconscious components.

About the authors

Elena A. Koltsova

Saint Petersburg Mining University

Author for correspondence.
Email: elena.a.koltsova@gmail.com
ORCID iD: 0000-0001-8724-2181

Candidate of Philological Sciences, Associate Professor, Associate Professor of the Department of Department of Foreign Languages

2, 21st Line, St Petersburg, Russian Federation, 199106

Faina I. Kartashkova

Ivanovo State University

Email: kartashkova@rambler.ru
ORCID iD: 0000-0001-7095-9143

Dr. Sc. (Philology), Professor, Professor of the Department of Foreign Philology, Honorary Worker of Higher Professional Education of the Russian Federation, Head of NOC «Laboratory of Communicative Human Behavior»

39, Ermaka st., Ivanovo, Russian Federation, 153025

References

  1. Dresner, E., & Herring, S.C. (2010). Functions of the Nonverbal in CMC: Emoticons and Illocutionary Force. Communication Theory, 20(3), 249-268. https://doi.org/10.1111/ j.1468-2885.2010.01362
  2. Aull, B. (2019). A study of phatic emoji use in WhatsApp communication. Internet Pragmatics, 2(2), 206-232. https://doi.org/10.1075/ip.00029.aul
  3. Bai, Q., Dan, Q., Mu, Z., & Yang, M. (2019). A Systematic Review of Emoji: Current Research and Future Perspectives. Frontiers in Psychology, 10. https://doi.org/10.3389/ fpsyg.2019.02221
  4. Evans, V. (2017). The Emoji Code: The Linguistics Behind Smiley Faces and Scaredy Cats. UK: Michael O’Mara Publ.
  5. Prada, M., Rodrigues, D.L., Garrido, M.V., Lopes, D., Cavalheiro, B., & Gaspar, R. (2018). Motives, frequency and attitudes toward emoji and emoticon use. Telematics and Informatics, 35(7), 1925-1934. https://doi.org/10.1016/j.tele.2018.06.005
  6. Rodrigues, D., Prada, M., Gaspar, R., Garrido, M.V., & Lopes, D. (2017). Lisbon Emoji and Emoticon Database (LEED): Norms for emoji and emoticons in seven evaluative dimensions. Behavior Research Methods, 50(1), 392-405. https://doi.org/10.3758/s13428-017-0878-6
  7. Danesi, M. (2016). The semiotics of emoji: The rise of visual language in the age of the internet. Bloomsbury Publ.
  8. McLuhan, M. (1962). The Gutenberg Galaxy: The Making of Typographic Man. London & New York.
  9. Emoji Report 2015. Emoji Research Team 2015. URL: http://emogi.com/documents/Emoji_ Report_2015.pdf. (accessed: 20.12.2021).
  10. Kaye, L.K., Wall, H.J., & Malone, S.A. (2016). “Turn that frown upside-down”: A contextual account of emoticon usage on different virtual platforms. Computers in Human Behavior, 60, 463-467. https://doi.org/10.1016/j.chb.2016.02.088
  11. Ge, J. (2019). Emoji Sequence Use in Enacting Personal Identity. In: Companion Proceedings of The 2019 World Wide Web Conference. https://doi.org/1145/3308560.3316545
  12. Lu, X., Ai, W., Liu, X., Li, Q., Wang, N., Huang, G., & Mei, Q. (2016). Learning from the ubiquitous language. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. https://doi.org/10.1145/2971648.2971724
  13. Ljubešić, N., & Fišer, D. (2016). A Global Analysis of Emoji Usage. In: Proceedings of the 10th Web as Corpus Workshop. https://doi.org/10.18653/v1/w16-2610
  14. Cheng, L. (2017). Do I mean what I say and say what I mean? A cross cultural approach to the use of emoticons & emojis in CMC messages. Fonseca Journal of Communication, 15, 199-217. https://doi.org/10.14201/fjc201715199217
  15. Chik, A., & Vásquez, C. (2017). A comparative multimodal analysis of restaurant reviews from two geographical contexts. Visual Communication, 16(1), 3-26. https://doi.org/10.1177/1470357216634005
  16. Lin, T.J., & Chen, C.H. (2018). A preliminary study of the form and status of passionate affection emoticons. International Journal of Design, 12(2), 75-90.
  17. Koltsova, Е.А. (2020). New mechanisms of online communication in the 21st century. In: A Person in Modern Communication. Ivanovo. pp. 23-30. (In Russ.).
  18. Herring, S.C., & Dainas, A.R. (2020). Gender and Age Influences on Interpretation of Emoji Functions. ACM Transactions on Social Computing, 3(2), 1-26. https://doi.org/10.1145/3375629
  19. Chen, Z., Lu, X., Ai, W., Li, H., Mei, Q., and Liu, X. (2018). Through a gender lens: learning usage patterns of emojis from large-scale android users. In: Paper Presented at the 2018 World Wide Web Conference on World Wide Web (Lyon). https://doi.org/10.1145/3178876.3186157
  20. Butterworth, S.E., Giuliano, T.A., White, J., Cantu, L., & Fraser, K.C. (2019). Sender Gender Influences Emoji Interpretation in Text Messages. Frontiers in Psychology, 10. https://doi.org/10.3389/fpsyg.2019.00784
  21. Cappallo, S., Svetlichnaya, S., Garrigues, P., Mensink, T., & Snoek, C.G.M. (2019). New Modality: Emoji Challenges in Prediction, Anticipation, and Retrieval. IEEE Transactions on Multimedia, 21(2), 402-415. https://doi.org/10.1109/tmm.2018.2862363
  22. Gawne, L., & McCulloch, G. (2019). Emoji as digital gestures. Language@ Internet, 17(2).
  23. Na’aman, N., Provenza, H., & Montoya, O. (2017). Varying Linguistic Purposes of Emoji in (Twitter) Context. In: Proceedings of ACL 2017, Student Research Workshop. https://doi.org/10.18653/v1/p17-3022
  24. Das, G., Wiener, H.J.D., & Kareklas, I. (2019). To emoji or not to emoji? Examining the influence of emoji on consumer reactions to advertising. Journal of Business Research, 96, 147-156. https://doi.org/10.1016/j.jbusres.2018.11.007
  25. Ge, J., & Gretzel, U. (2018). Emoji rhetoric: a social media influencer perspective. Journal of Marketing Management, 34(15-16), 1272-1295. https://doi.org/10.1080/026725 7x.2018.1483960
  26. Voinov, D.A. (2016). Emoji as a technology for political communication on the Internet. Electronic Periodical Scientific Journal the Age of Quality, 4, 103-112. (In Russ.).
  27. Esposito, G., Hernández, P., van Bavel, R., & Vila, J. (2017). Nudging to prevent the purchase of incompatible digital products online: An experimental study. PloS one, 12(3), e0173333.
  28. Moreno-Sandoval, L.G., Sánchez-Barriga, C., Buitrago, K.E., Pomares-Quimbaya, A., & Garcia, J.C. (2018). Spanish Twitter Data Used as a Source of Information About Consumer Food Choice. Machine Learning and Knowledge Extraction, 134-146. https://doi.org/10.1007/978-3-319-99740-7_9
  29. McShane, L., Pancer, E., Poole, M., & Deng, Q. (2021). Emoji, Playfulness, and Brand Engagement on Twitter. Journal of Interactive Marketing, 53, 96-110. https://doi.org/10.1016/j. intmar.2020.06.002
  30. Chairunnisa, S., & A.S., B. (2017). Analysis of Emoji and Emoticon Usage in Interpersonal Communication of Blackberry Messenger and WhatsApp Application User. International Journal of Social Sciences and Management, 4(2), 120-126 https://doi.org/10.3126/ijssm. v4i2.17173
  31. Gibson, W., Huang, P., & Yu, Q. (2018). Emoji and communicative action: The semiotics, sequence and gestural actions of “face covering hand.” Discourse, Context & Media, 26, 91-99 https://doi.org/10.1016/j.dcm.2018.05.005
  32. Phan, W.M. J., Amrhein, R., Rounds, J., & Lewis, P. (2017). Contextualizing Interest Scales with Emojis: Implications for Measurement and Validity. Journal of Career Assessment, 27(1), 114-133. https://doi.org/10.1177/1069072717748647
  33. Brody, N., & Caldwell, L. (2017). Cues filtered in, cues filtered out, cues cute, and cues grotesque: Teaching mediated communication with emoji Pictionary. Communication Teacher, 33(2), 127-131. https://doi.org/10.1080/17404622.2017.1401730
  34. Kelly, R., & Watts, L. (2015). Characterising the inventive appropriation of emoji as relationally meaningful in mediated close personal relationships. Experiences of technology appropriation: Unanticipated users, usage, circumstances, and design, 2.
  35. Koltsova, Е.А. (2018). Emoji as a new form of communication: functional and cognitive aspects. Cognitive Studies of Language, 35, 121-128. (In Russ.).
  36. Goren, C.C., Sarty, M., & Wu, P.Y. (1975). Visual following and pattern discrimination of facelike stimuli by newborn infants. Pediatrics, 56(4), 544-549.
  37. Johnson, M.H. (1991). Information processing and storage during filial imprinting. Cambridge: Cambridge University Press.
  38. Ekman, P., Friesen, W.V., & Ellsworth, P. (1972). Introduction to Research Findings. Emotion in the Human Face. https://doi.org/10.1016/b978-0-08-016643-8.50023-9
  39. Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4(6), 223-233. https://doi.org/10.1016/ s1364-6613(00)01482-0
  40. Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2002). Human neural systems for face recognition and social communication. Biological Psychiatry, 51(1), 59-67. https://doi.org/10.1016/ s0006-3223(01)01330-0
  41. Yuasa, M., Saito, K., & Mukawa, N. (2011). Brain activity when reading sentences and emoticons: an fMRI study of verbal and nonverbal communication. Electronics and Communications in Japan, 94(5), 17-24. https://doi.org/10.1002/ecj.10311

Supplementary files

Supplementary Files
Action
1. Fig. 1. Emoji as a nonverbal component of communication

Download (10KB)
Indexing metadata
2. Fig. 2. Emotive function of emojis

Download (2KB)
Indexing metadata
3. Fig. 3. Emoji chain

Download (1KB)
Indexing metadata
4. Fig. 4. Emojis in phatic communication

Download (3KB)
Indexing metadata
5. Fig. 5. Punctuation function of emojis

Download (5KB)
Indexing metadata
6. Fig. 6. Substitutional function of emojis

Download (5KB)
Indexing metadata
7. Рис. 7. Декоративная функция эмодзи

Download (3KB)
Indexing metadata

Copyright (c) 2022 Koltsova E.A., Kartashkova F.I.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies