Augmented virtuality technology in education: taxonomy of augmented virtuality types

Cover Page

Cite item

Full Text

Abstract

Problem statement. The authors discuss the interim results, which analyzed the least of all the studied immersive technologies - augmented virtuality technology and its educational potential. Various approaches to the augmented virtuality implementation by its addition with real objects are identified. Methodology. The most effective application spheres of augmented virtuality: transfer, copy, modified transfer, modified copy, are analysed. The augmented virtuality usage in education gives an opportunity to place a student into a virtual environment and increase the realism of the educational simulation significantly (in comparison with virtual reality technology) by maintaining the interaction with the real world. The transfer is the realization of a real object in virtual space in the line of sight. This technology allows to convey the object's appearance in the most efficient way, however, it reduces the immersion effect. The most productive usage can occur during work with people and with the presence of real objects. The copy is a virtual copy display without functional changes, presented as a 3D model instead of a real object. It allows to achieve maximum immersion and to implement relatively accurate interaction. The modified transfer is the real object implementation to the line of sight with functional and visual changes in the virtual space. This technology is most effective in conducting virtual excursions with a real guide and maximum immersion with the partial presence of real objects. It can be used while demonstrating physical phenomena with real objects. The modified copy is a display of a virtual copy with functional or visual changes in relation to the original object. It is advisable to use it in the absence of real objects or during developing practical skills. Results. Each adding virtuality method has its own advantages, disadvantages, implementation technologies and application areas. The choice depends on the educational task and learning conditions. Conclusion. The augmented virtuality usage in education gives an opportunity to place a student into a virtual environment and increase the realism of the educational simulation significantly (in comparison with virtual reality technology) by maintaining the interaction with the real world. The choice of one of the four main ways of reality adding should be determined by the goals, content and methods of teaching. Methods that are used in education may be combined in various combinations. This allows to apply augmented virtuality to various educational purposes.

Full Text

Problem statement. Currently, the technology of augmented virtuality is rarely used during the educational process despite its great educational potential [1]. Unfortunately, the technology of augmented virtuality sometimes is applied without taking into account its specifics, so the educational effectiveness may even decrease. “Transferring” the user's hands into a virtual space (where a small assortment of virtual instruments is often used) can be an example of improper application of augmented virtuality that may lead to a drop in learning efficiency. In this case, the real image of the user's hands will be significantly different from the virtual environment. Consequently, the learner's level of immersion can reduce drastically [2]. Furthermore, the convenience and efficiency of interaction with virtual space can be significantly decreased due to the inability to transmit (with the current level of technology) some tactile feedback (touches, vibrations, weight, features of controls, etc.). In this instance the usage of specialized virtual reality systems controllers is preferable. As immersive technologies, particularly augmented virtuality, play a significant role in the development of educational systems, it is useful to analyze the research, that exists in this area. It should be noted that few research works and papers reveal this technology in an educational field [3-10]. However, these studies offer broad perspectives on the utilization and benefits of augmented virtuality technology in educational field. Moreover, certain scholars for example, focus on specific elements, such as perception of material in virtual reality and how to use it in educational purposes [8]. Russian researchers from Moscow City University have uncovered that integration of augmented virtuality can be useful and helpful in educational activities with primary school students. It can successfully influence their cognitive processes [9]. What is more, some researchers found out that the use of augmented virtuality technology can cultivate specialized competencies for educators utilizing such immersive tools [11]. Also it has been observed that augmented virtuality facilitates the engagement of multiple sensory modalities, enhances precision in interaction with objects and systems, enables feedback mechanisms, improves the quality and depth of mentor-student communication, and supports practical training with real devices within simulated virtual environments [7]. Some experts have noted the high technical demands associated with implementing learning through augmented virtuality [6]. Augmented virtuality underutilization in education lead to reduced educational effectiveness when applied incorrectly without considering specific factors. Improper applications, such as transferring the user's hands into a virtual space without accounting for differences between the real and virtual environments, can lower learner immersion levels and hinder interaction efficiency due to the inability to transmit necessary tactile feedback. The aim of the study is to demonstrate the augmented reality education potential and to identify various approaches to the augmented virtuality implementation by its addition with real objects. Methodology. To solve the problem of augmented virtuality underutilization and improper usage, it is possible to identify various types of virtuality additions and determine the areas of applicability in the educational field for each of them [12; 13]. This research proposes the classification of technologies for supplementing virtuality according to types of transferring real objects into virtual. Results and discussion. A transfer (transmission) - this is a “literal” representation of a real object in virtual space: body parts, humans, tools, devices, objects being studied, etc. At the same time, the user simultaneously sees both the virtual environment and certain real objects (Figure 1). Изображение выглядит как текст, снимок экрана Автоматически созданное описание Figure 1. An implementation example of augmented virtuality educational system (the transfer technology). All objects are real except the background environment Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://ldpr.ru/event/382913 (accessed: 01.12.2023); https://thumbs.dreamstime.com/b/research-institute-d-cg-rendering-95173764.jpg (accessed: 01.12.2023); https://abc.ru/catalog/mikroskop-levenhuk-7s-ng-monokularnyj-125834.html (accessed: 01.12.2023); https://avatars.mds.yandex.net/i?id=dfc1f9bb29683bd4113cb9e84f1ddb93_l-5329796-images-thumbs&n=13 (accessed: 01.12.2023). Such type of mapping can be implemented in various ways: - with the virtual reality helmets usage; - the video signal is broadcast through external cameras, transferring the object “cut out” from the real environment to the virtual space (Figure 2). http://www.iamwire.com/wp-content/uploads/2019/06/Augumented-reality-the-user-can-see-video-of-their-real-hands-in-a-VR-view.png Figure 2. The difference in the appearance of real shooting and computer graphics Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://topuch.com/11-cinip-2-tosan-tjb-oushini-ati-jni/956500_html_7874b54b81e00cb5.jpg (accessed: 01.12.2023). “Cutting out” a real object from a completely virtual space (in a virtual reality helmet) is relatively easy to implement with available equipment, however, the quality of such transfer usually remains at a low level. The difference in the appearance of real shooting and computer graphics reduces the effect of immersion. The creation of a high-precision virtual copy of the object is carried out on a 3D scanning technology basis. The real object appearance, its functionality, and the virtual copy should match as much as possible. The virtual copy must repeat the position and state of the original in real-time (Figure 3). Creating a high-precision virtual 3D model of a real object or person requires a 3D scanner. The final image may be more uniform and the real objects will not differ from the virtual ones so much, however, some very small details may be lost with the augmented reality glasses usage [14; 15] (Figure 4). Adding a virtual environment using augmented reality is functionally similar to “cutting out” a real object from a completely virtual space in a virtual reality helmet [16]. Due to the peculiarities of the current level of development of augmented reality technology, the virtual space will be translucent and through it will be possible to observe the real environment, which increases safety, but reduces the effect of immersion with the Augmented Virtuality room usage the user sees the virtual environment around using projection technology or screens and simultaneously interacts with real objects and people nearby (Figure 5). Изображение выглядит как небо Автоматически созданное описание Figure 3. Image quality in virtual reality glasses Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Изображение выглядит как текст, мужской Автоматически созданное описание Figure 4. The augmented reality system builds a virtual world on a transparent display, leaving transparent areas occupied by real objects Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Picture is taken from: https://www.zdnet.com/article/hololens-chief-kipman-is-out-so-whats-next-for-microsofts-metaverse-strategy/ (accessed: 01.12.2023). Изображение выглядит как внутренний, человек, мужчина, стоит Автоматически созданное описание Figure 5. CAVE Automatic Virtual Environment Source: screenshot by Alexandr V. Grinshkun, Anastasia S. Zakova, taken from: https://www.youtube.com/watch?v=Gb9ayYGM-4c (accessed: 01.12.2023). Augmented virtuality rooms, also known as CAVE Automatic Virtual Environment [17], have become widespread due to the relatively simple technical implementation and to the possibility of using one room by several people. As noted earlier, the immersion effect into a virtual environment during the transfer can be drastically decreased due to the significant visual difference between real objects and virtual ones. However, the transfer can show (present) valuable results if it is necessary to demonstrate something very accurately from the real world in a virtual space [18; 19]. More accurate facial expressions and human movements transfer can be an example of appropriate usage of the transfer technology. Transfer, besides, is useful when a virtual image of a real object, the real object itself, or its textured 3D model is absent. This technology is expedient when a majority of objects for user’s interaction are available in real circumstances, however, it is necessary to demonstrate inaccessible environments such as a laboratory, a museum, excavations, other countries, etc. A copy - is a real object replacement with its virtual copy without functional changes, presented in 3D model form. The user observes only virtual space, while some objects in the virtual world are closely related to real ones. It means that virtual objects repeat the real object’s attributes including position in space, shape, dimensions, etc. This type of augmented virtuality in the comparison with the technology of virtual reality allows providing more realistic tactile feedback because the user interacts with real objects and virtual space at the same time. The copy in contrast with the transfer does not destroy the immersion effect, since all objects look uniform. It is advisable to use real objects (such as tools, devices, etc.) or precisely made substitute models, which have all the necessary features for the full-fledged tactile feedback implementation as real objects. As an effective way of copy usage, the student’s placement in virtual space can be mentioned. In these circumstances, the learner has a possibility of interaction with various objects that are essentially important in receiving the tactile feedback, since the accuracy reduction of its visualization is possible. It should be noted that an essential condition during copy usage is the presence of real objects (Figure 6). Изображение выглядит как текст, снимок экрана, диаграмма Автоматически созданное описание Figure 6. An implementation example of an augmented virtuality educational system (the copy technology) Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://ldpr.ru/event/382913 (accessed: 01.12.2023); https://thumbs.dreamstime.com/b/research-institute-d-cg-rendering-95173764.jpg (accessed: 01.12.2023); https://abc.ru/catalog/mikroskop-levenhuk-7s-ng-monokularnyj-125834.html (accessed: 01.12.2023); https://tr.pinterest.com/pin/play-the-games--501447739742847562/ (accessed: 01.12.2023). Изображение выглядит как текст, снимок экрана Автоматически созданное описание Figure 7. An implementation example of an augmented virtuality educational system (the modified transfer technology) Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://ldpr.ru/event/382913 (accessed: 01.12.2023); https://thumbs.dreamstime.com/b/research-institute-d-cg-rendering-95173764.jpg (accessed: 01.12.2023); https://abc.ru/catalog/mikroskop-levenhuk-7s-ng-monokularnyj-125834.html (accessed: 01.12.2023); https://tr.pinterest.com/pin/play-the-games--501447739742847562/ (accessed: 01.12.2023). All objects are virtual. The virtual 3D model of the microscope exactly repeats the actual object. A modified transfer is a kind of virtuality augmenting, where the user, besides the virtual environment, can observe and interact with real objects (Figure 7). Unlike in transfer, the system adds new properties to real objects such as instrument readings, dynamic instructions, malfunction emulation, various effects, phenomena, clothing changes, etc. The system does not only add a virtual background, but also adds an inaccessible “property” to a real object - a virtual model of a living cell. If it is necessary to demonstrate an inaccessible environment and a high-precision integrated object demonstration that is available but does not have all essential properties the modified transfer usage is expedient. In this case, the required trait can be added in the virtual environment with the help of modification. Изображение выглядит как текст, снимок экрана Автоматически созданное описание Figure 8. An implementation example of an augmented virtuality educational system (the modified copy technology) Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://ru.pinterest.com/pin/lmpara-steampunk--762445411908051705/ (accessed: 01.12.2023); https://ldpr.ru/event/382913 (accessed: 01.12.2023); https://thumbs.dreamstime.com/b/research-institute-d-cg-rendering-95173764.jpg (accessed: 01.12.2023); https://tr.pinterest.com/pin/play-the-games--501447739742847562/ (accessed: 01.12.2023). The 3D model from the Microsoft Office stock model database. A modified copy is a virtuality’s addition type, that allows the user to observe only virtual space. At the same time, some objects in the virtual world are closely related to real objects (as in copy technology). Virtual modified copy has an important change of the transferred object (Figure 8). Unlike in copy, the system adds new functions or visual traits to real objects. What is more, a major “related” property is the position in space while other traits can be changed by modification. All objects are virtual. The wooden microscope model sample is replaced by a functional 3D model of a working professional microscope by the system. Thus, it is possible to increase the tactile feedback level in relation to ordinary virtual reality, even if there are no full-fledged real objects, whereas there are real substitute models resembling a real object in shape, weight, texture, controls, etc. Objects that are printed on a 3D printer, assembled with the help of various constructors, wood, paper, or papier-mache can become such substitute models. The modified copy allows using real substitute models in the object of study absence. The augmented virtuality system will replace the appearance of the “blank” with the required one and simulate the missing functionality. The modified copy application is expedient when the classroom environment is unsuitable to demonstrate or to conduct an experiment, in the event, the educational task is to develop a basic practical skill for a particular job (Figure 9). Изображение выглядит как микроскоп Автоматически созданное описание Figure 9. Models of microscopes Source: created by Alexandr V. Grinshkun, Anastasia S. Zakova. Pictures are taken from: https://ru.pinterest.com/pin/lmpara-steampunk--762445411908051705/ (accessed: 01.12.2023); https://ru.pinterest.com/pin/755197431242074246/ (accessed: 01.12.2023); https://ru.pinterest.com/pin/399272323184873644/ (accessed: 01.12.2023); https://rc74.ru/raznoe/kak-sdelat-mikroskop-iz-bumagi-bumazhnyj-mikroskop.html (accessed: 01.12.2023); https://main-cdn.sbermegamarket.ru/big2/hlr-system/-22/944/508/374/915/100066718875b0.jpeg (accessed: 01.12.2023). The 3D model from the Microsoft Office stock model database. This allows the augmented virtuality systems to work similarly to classical virtual reality, yet the tactile feedback will be improved due to the presence of physical substitute models. It should be noted that this method is hardly applicable when the interaction between users is necessary. The difficulty appears because of poor emotion and gesture conveyance by virtual copies or due to the low detail relativity of 3D-models. Thus, the most suitable type of reality adding for educational purposes may be found: - the transfer is useful for work with people and with the actual objects presence; - the copy allows the maximum immersion and precise interactions; - the modified transfer may be used for virtual excursions with a real guide and maximum immersion into environment with the partial presence of real objects and the need to demonstrate physical phenomena with real objects; - the modified copy is useful in the absence of actual objects and helps to develop practical skills. This research highlights the benefits and drawbacks of augmented virtuality and its usage areas. It demonstrates the possibility of developing and applying virtual models-samples for teaching schoolchildren with augmented virtuality usage in the most effective way. Conclusion. The article demonstrates that little number of researchers study the augmented virtuality usage in education. It is important to notice that augmented virtuality has huge educational potential and some researchers are trying to investigate the pedagogical tools and approaches that will identify how to use this technology properly in school education. The augmented virtuality usage in education gives an opportunity to place a student into a virtual environment and increase the realism of the educational simulation significantly (in comparison with virtual reality technology) by maintaining the interaction with the real world. The choice of one of the four main ways of reality adding should be determined by the goals, content and methods of teaching. Methods that are used in education may be combined in various combinations. This allows to apply augmented virtuality to various educational purposes.
×

About the authors

Alexandr V. Grinshkun

Moscow City University

Author for correspondence.
Email: grinshkunav@gmail.com
ORCID iD: 0000-0003-3882-2010

Candidate of Pedagogical Sciences, Associate Professor of the Institute of Digital Education

4 Vtoroy Selskohoziajstvenny Proezd, Moscow, 129226, Russian Federation

Anastasia S. Zakova

Russian State University of Justice

Email: zakovaas@mgpu.ru
ORCID iD: 0000-0002-9262-0180

teacher of foreign language

69 Novocheremushkinskaya St, Moscow, 117418, Russian Federation

References

  1. Shunina LA. Conditions for the formation of professional competencies for future teachers to work with digital technologies within the digital economy. Actual Problems of Theory and Practice of Teaching Mathematics, Computer Science and Physics in the Modern Educational: Conference Proceedings (Kursk, 10-11 December 2019). Kursk: Kursk State University; 2019. (In Russ.)
  2. Milgram P, Kishino AF. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems. 1994;E77-D(12):1321-1329.
  3. Grinshkun AV. The use of augmented virtuality as an immersive educational technology in the framework of specialized education of schoolchildren. Profile School. 2020;8(4):27-31. (In Russ.)
  4. Azevich AI. Augmented reality and augmented virtuality as types of immersion learning technologies. Fundamental Problems of Teaching Mathematics, Computer Science and Informatization of Education: A Collection of International Scientific Conference Abstracts Dedicated to the 180th Anniversary of Pedagogical Education in Yelets (Yelets, 25-27 September 2020). Yelets; 2020. (In Russ.)
  5. Steinicke F, Bruder G, Rothaus K, Hinrichs K. Poster: a virtual body for augmented virtuality by chroma-keying of egocentric. IEEE Symposium on 3D User Interfaces. Lafayette; 2009.
  6. Azevich AI. Didactic potential of virtual reality and augmented virtuality technologies. MCU Journal of Informatics and Informatization of Education. 2022;(2):7-17. (In Russ.) https://doi.org/10.25688/2072-9014.2022.60.2.01
  7. Levitsky ML, Grinshkun AV. Immersive technologies: ways to augment virtuality and how to use them in education. MCU Journal of Informatics and Informatization of Education. 2020;(3):21-25. (In Russ.) https://doi.org/10.25688/2072-9014.2020.53.3.03
  8. Topliss J, Lukosch S, Coutts E, Piumsomboon T. Manipulating underfoot tactile perceptions of flooring materials in augmented virtuality. Applied Science. 2023;13(24). https://doi.org/ 10.3390/app132413106
  9. Zaslavskaya OY, Buerakova SN. Approaches to building a system for evaluating learning outcomes based on the use of augmented virtuality technology. MCU Journal of Informatics and Informatization of Education. 2022;(3):7-21. (In Russ.) https://www.doi.org/10.25688/2072-9014.2022.61.3.01
  10. Shin J, Lee K. Incorporating real-world object into virtual reality: using mobile device input with augmented virtuality. Multimed Tools Appl. 2024;83:46625-46652. https://doi.org/10.1007/s11042-022-13637-x
  11. Levitsky ML, Zaslavskaya OY. The concept of the implementation of fundamental approaches to the introduction of augmented virtuality in the system of general education. MCU Journal of Informatics and Informatization of Education. 2022;(4):7-21. (In Russ.) https://doi.org/10.25688/2072-9014.2022.62.4.01
  12. Levitsky ML, Zaslavskaya OY, Grinshkun AV, Azevich AI, Bazhenova SA, Andreikina EK, Puchkova ES. Fundamental foundations of the use of augmented virtuality technology in general education. Voronezh; 2020. (In Russ.)
  13. Utegenov N. Virtual and augmented reality (VR and AR). Universum: Technical Sciences. 2022;(7):23-26. (In Russ.) Available from: https://7universum.com/ru/tech/archive/item/14088 (accessed: 07.02.2024).
  14. Godoy CH Jr. Augmented reality for education: a review. International Journal of Innovative Science and Research Technology. 2020;5(6):39-45.
  15. Aroba OJ. The implementation of augmented reality on the Internet of Things for virtual learning in higher education. International Journal of Computing Sciences Research. 2024;8:2536-2549. https://dx.doi.org/10.25147/ijcsr.2017.001.1.174
  16. Polathan K. Augmented reality and virtual reality applications in education. VI International Kaoru Ishikawa Business Administration and Economy Congress (Mexico, 24-25 November 2022). Mexico: Universidad Juarez Autonoma de Tabasco; 2022.
  17. Cruz-Neira C, Sandin DJ, DeFanti TA, Kenyon RV, Hart JC. The CAVE: audio visual experience automatic virtual environment. Communications of the ACM. 1992;35(6):64-72. https://doi.org/10.1145/129888.129892
  18. Jaybhaye SM, Natekar D, Nayakodi P, Raut N, Jahagirdar O. TeachAR - augmented reality-based education application. Journal of Physics: Conference Series. 2023;2601:1-7. https://doi.org/10.1088/1742-6596/2601/1/012012
  19. Abdoli-Sejzi A. Augmented reality and virtual learning environment. Journal of Applied Sciences Research. 2015;11(8):1-5.

Supplementary files

Supplementary Files
Action
1. JATS XML
Download

Copyright (c) 2024 Grinshkun A.V., Zakova A.S.

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.