RUDN Journal of Psychology and PedagogicsRUDN Journal of Psychology and PedagogicsВестник Российского университета дружбы народов. Серия: Психология и педагогика2313-16832313-1705Peoples’ Friendship University of Russia named after Patrice Lumumba (RUDN University)4567910.22363/2313-1683-2024-21-4-1137-1166MEYQMOCURRENT TRENDS IN PERSONALITY RESEARCHСОВРЕМЕННЫЕ ТЕНДЕНЦИИ РАЗВИТИЯ ИССЛЕДОВАНИЙ ЛИЧНОСТИResearch ArticlePresentation and Analysis of Structural Knowledge in Learning Tasks Using the Example of a Complex Literary TextПредставление и анализ структурного знания в задачах обучения на примере сложного литературного текстаhttps://orcid.org/0000-0002-3082-2879DozortsevVictor M.ДозорцевВиктор Михайлович

Doctor of Engineering Sciences, Development Director

доктор технических наук, директор по развитию

vdozortsev@mail.ruhttps://orcid.org/0009-0004-4072-1600VishtalEvgeniya A.ВиштальЕвгения Андреевна

Bachelor of Applied Physics and Maths, Head of the Anti-Fraud Department

бакалавр прикладной физики и математики, руководитель отдела по борьбе с мошенничеством

vishtal.ea@phystech.eduhttps://orcid.org/0009-0000-3293-1350AshirovaEkaterina S.АшироваЕкатерина Сергеевна

Master of Pedagogy, Teacher of Russian language and literature

магистр педагогики, учитель русского языка и литературы

nezabutkka@mail.ruhttps://orcid.org/0009-0002-3110-5908MironovaAnastasia S.МироноваАнастасия Сергеевна

Master of Applied Physics and Maths, product manager

магистр прикладной физики и математики, продакт менеджер

an.mironova.gml@gmail.comDigital Technology Center LLCООО «Центр цифровых технологий»Untitled bankSt. Petersburg Gubernatorial Physics and Mathematics Lyceum 30Санкт-Петербургский губернаторский физико-математический лицей 30YandexКомпания Яндекс15122024214VOL 21, NO4 (2024)ТОМ 21, №4 (2024)1137116629082025Copyright ©; 2024, Dozortsev V.M., Vishtal E.A., Ashirova E.S., Mironova A.S.Copyright ©; 2024, Дозорцев В.М., Вишталь Е.А., Аширова Е.С., Миронова А.С.2024Dozortsev V.M., Vishtal E.A., Ashirova E.S., Mironova A.S.Дозорцев В.М., Вишталь Е.А., Аширова Е.С., Миронова А.С.https://creativecommons.org/licenses/by-nc/4.0https://journals.rudn.ru/psychology-pedagogics/article/view/45679

The lack of reliable instruments for automated assessment of learning outcomes leads to an overload of teachers who do not have sufficient resources to fully and objectively assess numerous students. A promising approach to solving this problem seems to be the use of markers of the formation of students’ structural knowledge, describing the interrelations of the components of the object being studied. The purpose of this study, based on the novel The Master and Margarita by M.A. Bulgakov, is to show that the structural knowledge of the subjects reflects the features and current level of understanding of the novel, as well as the change in this level as a result of learning. The study used PathFinder network scaling algorithm to extract and visualize significant connections between the novel’s characters based on subjective assessments of their pairwise connectivity. The experimental group consisted of high school seniors who studied the novel as part of the school curriculum under the guidance of their teacher. The comparison group included readers of the novel with different levels of experience in comprehending the text. The results of the study confirmed the possibility of assessing the structural knowledge of complex literary texts by the parameters of their network representation (coherence, expansion, and conciseness, balance of tree-like and coalition relationships). A significant difference was found in key indicators of the structural knowledge in the experimental and comparison group. According to the learning results in the experimental group, a statistically significant increase was revealed in the correlation of the students’ structural knowledge with that of their teacher. The analysis of the shortcomings of the extracted knowledge structures makes it possible to individualize teaching, reduce labor intensity and increase objectivity of the assessment of the learning results. The proposed approach requires verification on large samples and in other knowledge areas (in particular, in the training of operators of complex technical systems).

Дефицит надежных средств автоматизированного оценивания результатов обучения приводит к перегрузке преподавателей, не имеющих достаточного ресурса для полноценного и объективного оценивания множества обучаемых. Перспективный подход к решению проблемы - использование маркеров сформированности структурного знания обучаемых, описывающего взаимосвязи компонентов изучаемого объекта. Цель исследования - на основе романа М.А. Булгакова «Мастер и Маргарита» показать, что структурное знание испытуемых отражает особенности и текущий уровень понимания романа, а также изменение этого уровня в результате обучения. В исследовании используется метод сетевого шкалирования PathFinder, извлекающий и визуализирующий значимые связи между героями романа на основе субъективных оценок их попарной связности. Экспериментальная выборка представлена учениками выпускного класса гимназии, изучавшими роман в рамках школьной программы под руководством преподавателя. В сравнительную группу вошли читатели романа с разным опытом осмысления текста. Подтверждена возможность оценивания структурного знания сложных литературных текстов по параметрам их сетевого представления (когерентность, распространенность и лаконичность, сбалансированность древовидных и коалиционных связей). Обнаружено существенное различие по ключевым показателям структурного знания в экспериментальной и сравнительной группах. Выявлено статистически значимое повышение корреляции структурного знания школьников со структурным знанием преподавателя по результатам обучения. Анализ недостатков извлекаемых структур знаний позволяет индивидуализировать обучение, снизить трудоемкость и повысить объективность оценивания результатов обучения. Предложенный подход требует проверки на б о льших выборках испытуемых и в других предметных областях (в частности в тренинге операторов сложных технических систем).

structural knowledgenetwork scalinggraph theoryPathFinder algorithmcoherenceMikhail Bulgakov’s novel The Master and Margaritaструктурное знаниесубъективное сетевое шкалированиетеория графовметод PathFinderкогерентностьроман М.А. Булгакова «Мастер и Маргарита»Acton, W.H., Johnson, P.J., & Goldsmith, T.E. (1994). Structural knowledge assessment: Comparison of referent structures. Journal of Educational Psychology, 86(2), 303–311. https://doi.org/10.1037/0022-0663.86.2.303Дозорцев В.М., Миронова А.С. К использованию характеристик структурного знания для оценки переноса в тренинге операторов технологического процесса // Актуальные проблемы психологии труда, инженерной психологии и эргономики. Вып. 9 / под ред. А.А. Обознова, А.Л. Журавлева. Москва : Изд-во «Институт психологии РАН», 2020. С. 117–162. https://doi.org/10.38098/ergo.2020.009Amith, M., Cunningham, R., Savas, L.S., Boom, J., Schvaneveldt, R., Tao, C., & Cohen, T. (2017). Using pathfinder networks to discover alignment between expert and consumer conceptual knowledge from online vaccine content. Journal of Biomedical Informatics, 74, 33–45. https://doi.org/10.1016/j.jbi.2017.08.007Дозорцев В.М., Обознов А.А., Назин В.А., Гуцыкова С.В., Миронова А.С. Формирование у операторов концептуального понимания технологического объекта: актуальная задача и объективный результат компьютерного тренинга // Автоматизация в промышленности. 2014. № 12. С. 13-19.Cañas, A. J., Hill, G., Carff, R., Suri, N., Lott, J., Gómez, G., Eskridge, T.C., Arroyo, M., & Carvajal, R. (2004). CmapTools: A knowledge modeling and sharing environment. Concept maps: Theory, methodology, technology. Proceedings of the First International Conference on Concept Mapping (vol. 1, pp. 125–133). Navarra: Universidad Publica de Navarra.Обознов А.А., Чернецкая Е.Д., Бессонова Ю.В. Концептуальные модели атомной станции у операторов с разным профессиональным стажем // Психологический журнал. 2013. Т. 34. № 4. С. 47–57.Dozortsev, V.M., & Mironova, A.S. (2020). On the use of structural knowledge characteristics for transfer assessment in the training of industrial operators. Actual problems of psychology of labor, engineering psychology and ergonomics. Proceedings of the Scientific and Practical Seminar, (iss. 9, pp. 117–162). Moscow: Institute of psychology Russian academy of sciences. (In Russ.) https://doi.org/10.38098/ergo.2020.009Acton W.H., Johnson P.J., Goldsmith T.E. Structural knowledge assessment: Comparison of referent structures // Journal of Educational Psychology. 1994. Vol. 86. No. 2. P. 303–311. https://doi.org/10.1037/0022-0663.86.2.303Dozortsev, V.M., Oboznov, A.A., Nazin, V.A., Gutsikova, S.V., & Mironova, A.S. (2014). Forming operator’s conceptual understanding of process plant: an actual task and an objective result of computerized training. Automation in Industry, (12), 13-19. (In Russ.).Amith M., Cunningham R., Savas L.S., Boom J., Schvaneveldt R., Tao C., Cohen T. Using pathfinder networks to discover alignment between expert and consumer conceptual knowledge from online vaccine content // Journal of Biomedical Informatics. 2017. Vol. 74. P. 33–45. https://doi.org/10.1016/j.jbi.2017.08.007Ericsson, K.A., & Simon, H.A. (1998). How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking. Mind, Culture, and Activity, 5(3), 178–186. https://doi.org/10.1207/s15327884mca0503_3Cañas A.J., Hil, G., Carff R., Suri N., Lot, J., Gómez G., Eskridge T.C., Arroyo M., Carvajal R. CmapTools: A knowledge modeling and sharing environment // Concept maps: Theory, methodology, technology. Proceedings of the First International Conference on Concept Mapping / ed. by A.J. Cañas, J.D. Novak, F.M. González. Navarra : Universidad Publica de Navarra, 2004. Vol. 1. P. 125–133.Furlough, C., & Gillan, D.J. (2020). Assessing the ability of multidimensional scaling and pathfinder networks to measure spatial knowledge. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 64(1), 303–307. https://doi.org/10.1177/ 1071181320641070Ericsson K.A., Simon H.A. How to study thinking in everyday life: Contrasting think-aloud protocols with descriptions and explanations of thinking // Mind, Culture, and Activity. 1998. Vol. 5. No 3. P. 178–186. https://doi.org/10.1207/s15327884mca0503_3Gardner, Н. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books.Furlough C., Gillan D.J. Assessing the ability of multidimensional scaling and pathfinder networks to measure spatial knowledge // Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2020. Vol. 64. No 1. P. 303–307. https://doi.org/ 10.1177/1071181320641070Goldsmith, T.E., Johnson, P.J., & Acton, W.H. (1991). Assessing structural knowledge. Journal of Educational Psychology, 83(1), 88–96. https://doi.org/10.1037/0022-0663.83.1.88Gardner Н. Frames of mind: The theory of multiple intelligences. New York : Basic Books, 1983. 484 p.Gunstone, R.F. (1980). Word association and the description of cognitive structure. Research in Science Education, 10(1), 45–53. https://doi.org/10.1007/bf02356308Goldsmith T.E., Johnson P.J., Acton W.H. Assessing structural knowledge // Journal of Educational Psychology. 1991. Vol. 83. No 1. P. 88–96. https://doi.org/10.1037/0022-0663.83.1.88Hagberg, A.A., Schult, D.A., & Swart, P.J. (2008). Exploring network structure, dynamics, and function using NetworkX. SciPy 2008. Proceedings of the 7th Python in Science Conference (pp. 11–15). Pasadena, CA: SciPy. https://doi.org/10.25080/tcwv9851Gunstone R.F. Word association and the description of cognitive structure // Research in Science Education. 1980. Vol. 10. No 1. P. 45–53. https://doi.org/10.1007/bf02356308Ifenthaler, D., Masduki, I., & Seel, N.M. (2011). The mystery of cognitive structure and how we can detect it: tracking the development of cognitive structures over time. Instructi­onal Science, 39(1), 41–61. https://doi.org/10.1007/s11251-009-9097-6Hagberg A.A., Schult D.A., Swart P.J. Exploring network structure, dynamics, and function using NetworkX // SciPy 2008. Proceedings of the 7th Python in Science conference. Pasadena, CA: SciPy, 2008. P. 11–15. https://doi.org/10.25080/tcwv9851Ifenthaler, D., Pirnay-Dummer, P., & Spector, J.M. (Eds.). (2008). Understanding models for learning and instruction: Essays in Honor of Norbert M. Seel. Springer New York.Ifenthaler D., Masduki I., Seel N.M. The mystery of cognitive structure and how we can detect it: tracking the development of cognitive structures over time // Instructional Science. 2011. Vol. 39. No 1. P. 41–61. https://doi.org/10.1007/s11251-009-9097-6Johnson, P.J., Goldsmith, T.E., & Teague, K.W. (1995). Similarity, structure, and knowledge: A representational approach to assessment. In P.D. Nichols, S.F. Chipman, R.F. Brennan (Eds.). Cognitively Diagnostic Assessment (pp. 221–249). Mahwah, N.J: Lawrence Erlbaum Associates, Inc. https://doi.org/10.4324/9780203052969-10Johnson P.J., Goldsmith T.E., Teague K.W. Similarity, structure, and knowledge: A representational approach to assessment // Cognitively Diagnostic Assessment / ed. by P.D. Nichols, S.F. Chipman, R.F. Brennan. Mahwah, N.J : Lawrence Erlbaum Associates, Inc, 1995. P. 221–249. https://doi.org/10.4324/9780203052969-10Jonassen, D.H. (1987). Assessing cognitive structure: Verifying a method using pattern notes. Journal of Research & Development in Education, 20(3), 1–14.Jonassen D.H. Assessing cognitive structure: Verifying a method using pattern notes // Journal of Research & Development in Education. 1987. Vol. 20. No 3. P. 1–14.Meyer, B. (2008). The effects of computer-elicited structural and group knowledge on complex problem solving performance. Ph.D. in Psychology Thesis. Berlin: Humboldt University Berlin. https://doi.org/10.5167/uzh-9204Meyer B. The effects of computer-elicited structural and group knowledge on complex problem solving performance : Ph.D. in Psychology Thesis. Berlin : Humboldt University Berlin, 2008. 238 p. https://doi.org/10.5167/uzh-9204Nisbett, R.E., & Wilson, T.D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231Nisbett R.E., Wilson T.D. Telling more than we can know: Verbal reports on mental processes // Psychological Review. 1977. Vol. 84. No 3. P. 231–259. https://doi.org/10.1037/0033-295X.84.3.231Oboznov, A.A., Chernetskaya, E.D., & Bessonova, Y.V. (2013). Conceptual models of nuclear station in operators with different occupational length of service. Psikhologicheskii Zhournal, 34(4), 47–57. (In Russ.)Pirnay-Dummer P., Ifenthaler D., Spector J.M. Highly integrated model assessment technology and tools // Educational Technology Research and Development. 2010. Vol. 58. No 1. P. 3–18. https://doi.org/10.1007/s11423-009-9119-8Pirnay-Dummer, P., Ifenthaler, D., & Spector, J.M. (2010). Highly integrated model assessment technology and tools. Educational Technology Research and Development, 58(1), 3–18. https://doi.org/10.1007/s11423-009-9119-8Politowicz M.S., Sato T., Chancey E.T., Yamani Y. Pathfinder networks for measuring operator mental model structure with a simple autopilot system // Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2022. Vol. 66. No 1. P. 883–887. https://doi.org/10.1177/1071181322661510Politowicz, M.S., Sato, T., Chancey, E.T., & Yamani, Y. (2022). Pathfinder networks for measuring operator mental model structure with a simple autopilot system. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 883–887. https://doi.org/10.1177/1071181322661510Sanchez S.M. Concept Sourcing for Mental Models: A Study of Mental Models & Expertise Using a Video Game Context. MS in Psychology Thesis. Raleigh, NC : North Carolina State University, 2021. 41 p.Sanchez, S.M. (2021). Concept Sourcing for Mental Models: A Study of Mental Models & Expertise Using a Video Game Context. MS in Psychology Thesis. Raleigh, NC: North Carolina State University.Schvaneveldt R.W., Durso F.T., Dearhold D.W. Network structures in proximity data // The psychology of learning and motivation. Vol. 24 / ed. by G.H. Bower. New York : Academic Press, Inc., 1989. P. 249–284. https://doi.org/10.1016/s0079-7421(08)60539-3Schvaneveldt, R.W., Durso, F.T., & Dearholt, D.W. (1989). Network structures in proximity data. In G.H. Bower (Eds.), Psychology of Learning and Motivation (vol. 24, pp. 249–284). New York: Academic Press, Inc. https://doi.org/10.1016/s0079-7421(08)60539-3Seel N.M. Educational diagnosis of mental models: Assessment problems and technology-based solutions // Journal of Structural Learning & Intelligent Systems. 1999. Vol. 14. No 2. P. 153–185.Seel, N.M. (1999). Educational diagnosis of mental models: Assessment problems and tech­nology-based solutions. Journal of Structural Learning & Intelligent Systems, 14(2), 153–185.Shavelson R.J. Some aspects of the correspondence between content structure and cognitive structure in physics education // Journal of Educational Psychology. 1972. Vol. 63. No 3. P. 225–234. https://doi.org/10.1037/h0032652Shavelson, R.J. (1972). Some aspects of the correspondence between content structure and cognitive structure in physics instruction. Journal of Educational Psychology, 63(3), 225–234. https://doi.org/10.1037/h0032652Snow R.E. Toward assessment of cognitive and conative structures in learning // Educational Researcher. 1989. Vol. 18. No 9. P. 8–14. https://doi.org/10.3102/0013189x018009008Snow, R.E. (1989). Toward assessment of cognitive and conative structures in learning. Educational Researcher, 18(9), 8–14. https://doi.org/10.3102/0013189x018009008Trumpower D.L., Sarwar G.S. Effectiveness of structural feedback provided by pathfinder networks // Journal of Educational Computing Research. 2010. Vol. 43. No 1. P. 7–24. https://doi.org/10.2190/ec.43.1.bTrumpower, D.L., & Sarwar, G.S. (2010). Effectiveness of structural feedback provided by pathfinder networks. Journal of Educational Computing Research, 43(1), 7–24. https://doi.org/10.2190/ec.43.1.bTrumpower D.L., Sharara H., Goldsmith T.E. Specificity of structural assessment of knowledge // The Journal of Technology, Learning and Assessment. 2010. Vol. 8. No 5. P. 1–32.Trumpower, D.L., & Vanapalli, A.S. (2016). Structural assessment of knowledge as, of, and for learning. In M. Spector, B. Lockee, M. Childress (Eds.), Learning, Design, and Techno­logy (pp. 1–22). Cham: Springer. https://doi.org/10.1007/978-3-319-17727-4_23-1Trumpower D.L., Vanapalli A.S. Structural assessment of knowledge as, of, and for learning // Learning, Design, and Technology / ed. by M. Spector, B. Lockee, M. Childress. Cham : Springer, 2016. P. 1–22. https://doi.org/10.1007/978-3-319-17727-4_23-1Trumpower, D.L., Sharara, H., & Goldsmith, T.E. (2010). Specificity of structural assessment of knowledge. The Journal of Technology, Learning and Assessment, 8(5), 1–32.Understanding models for learning and instruction: Essays in Honor of Norbert M. Seel / ed. by D. Ifenthaler, P. Pirnay-Dummer, J. M. Spector. NY : Springer New York, 2008. 330 p.