Formation of students’ communication skills using the “Dailo” interactive dialogue system based on artificial intelligence technologies

Full Text

Problem statement. Communication plays a key role in the professional activity of any person, regardless of his field of work. This is not only the exchange of information, but also building relationships, managing processes, coordinating actions, resolving conflicts and making decisions. The importance of communication is especially relevant in the modern world, where the ability to work in a team, the ability to listen and be heard, the ability to persuade and find a common language with people are increasingly valued. Also, communications in professional activities include not only the ability to speak, but also the ability to listen, understand and interpret information. This helps create effective working relationships, improve productivity and achieve common goals. Without effective communications, it is difficult to imagine the successful completion of any project or task. Therefore, the development of communication skills is one of the priorities for a professional. Communication is understood as the process of exchanging information between subjects, where each of them carries out a purposeful action and strives to solve a problem[10] [1]. O.A. Svetlyakov and A.A. Isakhanova note that a large amount of time is devoted to the formation of professional communication skills in the practice of higher and secondary specialized education [2; 3], and when adapting to the workplace, in each of the educational practices the process of developing professional communication skills has its own characteristics. At university or college, students mostly practice public speaking, debating, teamwork, and negotiation skills. A.G. Mikhailova notes that role-playing games and case studies play an important role, which help students learn to solve specific communicative problems in conditions as close as possible to real ones [4]. Teachers demonstrate models of effective communication to students and analyze their communicative behavior, making recommendations for its improvement. In the workplace, the formation of communication skills occurs in the process of direct communication with colleagues, clients and partners. It is important to note that the formation of professional communication skills directly affects the company’s profit, therefore many companies conduct special communication trainings, games and events for their employees, create educational programs, invite experienced colleagues and coaches to share their experience and sort out any communication problems to improve the communication competencies of employees [5; 19]. Thus, the formation of professional skills is one of the important tasks of companies’ personnel policies, and specialists are constantly looking for new and effective tools to solve this problem. Pedagogical technologies for the formation of communication skills are constantly being improved, new methods and practices are emerging to solve this problem. For example, A.G. Mikhailova and Jaye Selvam describe such practices as developing projects in the process of teamwork that were presented at scientific conferences to develop communicative competence [6; 17], A.A. Tolsteneva and E.N. Galkin discuss the practices of developing students’ communication skills through participation in professional trainings; master classes [7; 18], E.A. Demina describes the use of storytelling techniques [8]. L.V. Abdrakhmanova describes game-based learning technologies for developing communication skills [9-12]. It is assumed that one of the tools for developing communication skills can be a system of interactive dialogues, implemented using artificial intelligence systems. An interactive dialogue system based on artificial intelligence is a technology that allows you to create smart dialogue systems that can conduct a natural conversation with the user, answer his questions, offer recommendations and execute commands. This is achieved through the use of artificial intelligence algorithms in the field of semantic analysis of natural language, which establish the semantic similarity of phrases and, based on this, advance the user through the plot in one direction or another. The system can create various communication situations, adapt to the user’s level of knowledge and offer him optimal response options. The system can also offer various story situations in which the user can train some type of communication. The formative effect when interacting with such a system is achieved through the following mechanisms: - the user learns the structure of the dialogue as it is specified in the simulator; - the user moves through the story and consults the client, following the prompts of the simulator; - after each phrase, the user receives feedback from the character on how successful his answer was; - at the end of the training, the user receives a final grade with a detailed explanation of where the score was reduced and for what; - after training, the user can listen back to each of his phrases and see how the phrase was rated by the system. Thus, the simulator participant can enter into a dialogue that is as close to the real thing as possible, and receives detailed feedback on the stages of the dialogue and on each phrase he said. Within the framework of the interactive dialogue system, it is possible to implement various teaching methods: - linear training dialogues, during which the user follows prompts and dialogue structure to complete a story task; - dialogue simulators, during the implementation of which the user tries various options for action and receives feedback on which one was correct and why; - text quests, during which the user sort of goes through the game, earns points and independently guesses which course of action is the best. In this dialogue, the method of linear dialogues was used, namely the structure “linear dialogue with variable plots”. This technique will be effective for developing communication skills, as it will allow users who have not previously been involved in professional communications with clients to quickly master this type of communication, detect and get rid of typical mistakes in dialogue and adapt to a new professional situation. Thus, the issue of developing professional communication skills is certainly important both at the university and within the corporate university. Moreover, there are many techniques that are successfully used for this. However, there is not enough data on how effective these techniques are using information technology tools, in particular artificial intelligence technologies. Methodology. This study discusses the effectiveness of achieving educational results in the field of professional communications using interactive dialogue systems implemented using artificial intelligence. To solve this problem, an experiment was developed to develop communication skills using a system of interactive dialogues using artificial intelligence technologies. Communication skills were taught using dialogue simulators based on artificial intelligence systems called “Dailo”. This tool creates communication scenarios with different behaviour possibilities for the participant and the system (Figure). Artificial intelligence analyses each statement of the participant and associates it with a pre-established pattern of behaviour. In response to the participant's rated statement, the character on the screen demonstrates a variety of reactions. This creates a sense of real communication between the participant and the character, and also allows the participant to independently direct and develop the plot. Depending on the participant’s statements, the simulator can lead to the resolution of negotiations or to conflict. During the research, employee training was developed and conducted in two companies. After the training, control sections (check conversations) were made to test communication skills. Control sections were measured by experts according to the following criteria: - carrying out all the necessary stages of the dialogue (greeting, clarifying the request, consulting the client, and so on); - the essence of the consultation: solving the client’s request, providing all the necessary information; - handling objections and correctly resolving complaints. As a result, employees were given points for the entire control section (dialogue) from 0 to 100. A total of 3 sections were carried out over three months. Training and control sections were carried out based on different techniques in two companies, so, unfortunately, we cannot compare them, but we can state and interpret the data for each company separately. In both companies, the structure of the educational dialogue corresponded to the structure of “linear dialogue with variable plot scenarios.” Изображение выглядит как в помещении, одежда, мебель, человек Автоматически созданное описание Creating communicative situations and organizing dialogue using artificial intelligence tools Source: created by Sofya I. Dreytser. The obtained data were processed by the Mann - Whitney statistical test for the data of the first company, and the Pearson correlation was calculated for the data of the second company. Results and discussion. For company No. 1 the following data was obtained. A comparative measurement was carried out over several months between the experimental and control groups. The experimental group was trained in communication skills to interact with clients using the “Dailo” interactive dialogue system. The control group studied independently by memorising dialogue phrases and scripts. The number of employees in the experimental and control groups was 4 and 5 people. The verification criteria included following the script and performing communicative tasks of the dialogue: providing complete information upon request, answering questions and working with client objections. As a result of training using the interactive dialogue system, the data presented in Table 1 was obtained. As a result of training without the use of an interactive dialogue system, the data presented in Table 2 was obtained. Table 1 Results of an expert control cross-section of the communication skills of employees who were trained using the interactive dialogue system Participants Control section 1 (June), score Control section 2 (July), score Control section 3 (August), score The difference between the best and worst attempts Participant 1 92 85 72 20 Participant 2 - 91 69 22 Participant 3 86 100 98 14 Participant 4 88 100 76 24 Source: compiled by Sofya I. Dreytser. Table 2 Results of an expert control cross-section of the communication skills of employees who were trained without the help of an interactive dialogue system Participants Control section 1 (June), score Control section 2 (July), score Control section 3 (August), score The difference between the best and worst attempts Participant 1 89 90 92 3 Participant 2 87 88 - 1 Participant 3 89 84 81 8 Participant 4 - 86 97 11 Participant 5 - 97 98 1 Source: compiled by Sofya I. Dreytser. It can be seen that in both cases the assessments are not uniform, the quality assessment fluctuates, there are both positive and negative dynamics. The applied statistical tests also showed no significant difference between the data sets. Of interest, however, are the following differences between the two groups. We see that in the first group, where training was carried out using the interactive dialogue system, all participants showed a significant gap in scores between the best and worst attempt, while in the group trained without the use of the interactive dialogue system, there was practically no gap in scores points. It is necessary to mention that all measurements were carried out at least a month apart by one assessor, so the human factor in the measurement process can be eliminated. To confirm the statistical significance of the difference between the data of the first and second groups, the Mann - Whitney test was used, since this criterion can be used to compare small groups of three people, and it is also intended to compare data from two independent groups that do not obey the law of normal distribution. Comparison of data using a statistical test shows a value of 0 with an asymptotic significance of 0.009. The obtained empirical value Uem(0) is in the significance zone at p = 0.01, which means that the difference between the two data sets is statistically significant. This observation allows us to formulate a hypothesis about a system of interactive dialogues using artificial intelligence technologies. The hypothesis is that learning through memorisation of scripts and learning through communication in an interactive dialogue system leads to different types of educational results. Namely, the system of interactive dialogues allows you not only to remember the correct phrases of the dialogue, but also to improvise within the meaning of the communicative actions that need to be performed. This proposal is due to the fact that, firstly, all participants in the interactive dialogue system received scores above 80 points out of 100 as a result of training in the system itself, and secondly, the average value of the worst attempt at a control cut for these participants is 71 points out of 100. This combination of factors leads to the assumption that the overall dialogue was conducted correctly, but points were reduced for minor errors. Thus, the assumption about the main educational result of the “Dailo” interactive dialogue system is that the system provides the ability to flexibly reproduce and improvise in the dialogue process, whereas ordinary memorisation without training does not give such a result. To test this hypothesis, we need to turn to data from a second company. For company No. 2 the following data was obtained. A total of 26 employees were trained in two subject areas. As a result of the training, data on employee satisfaction and the results of the control section were obtained, as well as the average employee satisfaction score in the system itself and the average training score in the system. The verification criteria were not following the script, as in the previous company, but flexible dialogue and fulfilment of communication tasks: clarifying the client’s request, providing complete information and satisfying the client’s request. The data obtained as a result of training using the interactive dialogue system was processed as follows: it was calculated how many participants rated the dialogue simulator positively and how many negatively, and it was also calculated how many participants showed a positive result during the control section and how many negative. A positive review of the simulator is understood as a review in which the participant liked to learn using the simulator; there are minor insignificant comments. A negative review of a simulator is understood as a review in which the participant did not like learning using the simulator, and he sees serious shortcomings of such a tool. A positive result of the control section was understood as a situation where the dialogue conducted by the participant with the client met all evaluation criteria: the client’s request was completely clarified and satisfied, the information provided was complete and truthful, and minor errors may have been made. The negative result of the control section was understood as a situation when the dialogue conducted by the participant with the client contained gross errors and inconsistencies, for example, the participant did not provide some important information or did not fully clarify the client’s request. A result of data processing presented in Table 3. Table 3 Results of the expert control cross-section of the communication skills of employees who were trained using the interactive dialogue system Lesson 1 Lesson 2 Data Positive (above control value) Negative (below control value) Data Positive (above control value) Negative (below control value) Number of reviews about the lesson 16 6 Number of reviews about the lesson 19 3 Control measurement results 5 17 Control measurement results 20 2 Source: compiled by Sofya I. Dreytser. The opposite picture is observed for two lessons from one company: in the first case, those participants who responded rather negatively to the lesson showed good results, and those participants who responded positively to the lesson received poor results. While in lesson 2 there is a direct correlation: participants who responded positively to the lesson got a good result on the control section. These conclusions are confirmed using the Pearson statistical test, which can be used to calculate the correlation of variables with each other. The results according to the Pearson criterion for the first lesson are equal to -1, and for the second lesson they are equal to 1, which confirms the inverse and direct correlation in these lessons. The results give rise to new questions and hypotheses about what the findings are related to. As a result of the analysis of data from company No. 1, it was assumed that the main result of this tool, the “Dailo” dialogue simulator, is the ability to flexibly reproduce and improvise in the process of dialogue while maintaining its logic and essence. The data from company No. 2 partially confirm this hypothesis, however, the inverse correlation in the results of lesson 1 of company No. 2 does not allow us to fully verify this. The analysis of the structure of the dialogues themselves, proposed for training in company No. 2, suggests that the first and second dialogues are significantly different: - the second dialogue is provided with explanations and clarifications of what action the user should perform and why this particular method is proposed, but the first dialogue does not contain such explanations; - the second dialogue contains fewer dialogue scenario options, so the user can go through them all, while the first dialogue has more than 10 scenario options, two of which are unique and can be encountered by the user in less than 1 playthrough out of 10, so participants did not have the opportunity study the dialogues completely. Thus, an analysis of the methodology shows that in the first lesson the dialogue developer made several significant mistakes. It can be assumed that the learning results as a result of studying the first dialogue should be lower than as a result of studying the second dialogue, but the results are approximately comparable, but the feedback in the first lesson is actually more negative. A number of works have analyzed the influence of components of educational independence, including defining reflection, on educational results, as well as on students’ attitudes towards the educational process [13; 14]. This is also manifested in the ability to evaluate educational materials (a task) and determine what parameters are missing in the task itself so that it can be solved. Such students who have determining reflection are also able to complete for themselves the missing components of the educational task, and due to this achieve high educational results [15; 16]. There is an assumption that students with a high level of determining reflection, who can distinguish a complete task from an incomplete one, took part in this experiment. Therefore, the participants showed high results in both dialogues, and left comments about the first dialogue, indicating that the learning task in this dialogue was incomplete, or that the task could not be solved using these materials. This is evidenced by the reviews themselves left by the participants, in addition to the assessments: “It would be possible to have more cases and make small alternatives for phrases”, “In some places it is useful to speak the entire dialogue structurally, but there were difficulties with specific answers in the dialogue” - that is, the participants note that they would like to change the design of the communicative tasks they encountered in the dialogue. Conclusion. As a result of the experiments, interesting data were obtained that allow us to formulate a number of hypotheses. In the first experiment, the data indicate that the dialogue simulator teaches the ability to flexibly reproduce and improvise in the process of dialogue while maintaining its logic and essence. In the second experiment, the data obtained allow us to partially confirm this hypothesis and give rise to a new assumption that the tool can be used to diagnose defining reflection by providing participants with predetermined and underdetermined tasks, where it is necessary to complete the task itself or the method of solving it. It can be assumed that participants who can point out the different nature of the tasks have defining reflection. These assumptions are hypotheses and require further verification, which will determine the direction of research and development on the use of artificial intelligence systems in the educational process.

About the authors

Sofya I. Dreytser

Moscow City University; LLC “Cerevrum”

Author for correspondence.
Email: dreitsersi562@mgpu.ru
ORCID iD: 0000-0001-8549-1627

PhD student, Department of Education Informatization, Institute of Digital Education, Moscow City University ; instructional designer, LLC “Cerevrum”

4 Vtoroy Selskohoziajstvenny Proezd, Moscow, 129226, Russian Federation;42 Bol'shoi Bulvar, bldg 1, Moscow, 121205, Russian Federation

References