DEVELOPING AN APPROACH TO MULTIMODAL QUANTITATIVE ASSESSMENT OF INTERVIEWERS’ COGNITIVE LOAD: FIRST RESULTS OF A FIELD QUASI EXPERIMENT

Cover Page

Abstract


The belief that survey research instruments mediating communication between interviewer and respondent influence the quality of data obtained in the interview has become a conventional methodological wisdom long ago. However, the impact of the cognitive load experienced by the interviewer has not been systematically examined. When a questionnaire is filled up by an interviewer it is the latter who has to allocate limited individual resources of attention, memory, visual and motor control, active listening and interpretation in order to minimize the respondent’s misunderstanding of the questions and one’s own errors of the answers’ fixation. However, among various methods of pre-testing or evaluating survey mode effects and assessing instruments’ quality, the methods of multimodal quantitative estimation of instrument-related cognitive load experienced by interviewers during the interview are still rare or lacking. Thus, the article presents a brief review of subjective, behavioral and physiological measures of the cognitive load, which are used in such disciplinary fields as cognitive science, ergonomics, etc., and a discussion of preliminary findings of the field quasi-experiment conducted among the interviewers of the Russian Longitudinal Monitoring Survey on the first stage of transition to the CAPI mode. The quasi-experiment findings prove some possibilities and limitations of the parallel use of a version of the cognitive load rating scale developed by F. Paas and a simple physiological measure (heart rate) supplemented with a background screen video capture from Android-based tablets used for CAPI interviews for the multimodal quantitative evaluation and optimization of the interviewer’s cognitive load.


About the authors

I F Deviatko

National Research University Higher School of Economics; Institute of Sociology of FCTAS RAS

Author for correspondence.
Email: deviatko@gmail.com
Myasnitskaya St., 11, Moscow, 101000, Russia; Krzhizhanovskogo St., 24 / 35-5, Moscow, 117218, Russia

-

References

  1. Bogdanov M.B., Lebedev D.V. “Glavnoe — ne boyatsya” — vozdejstvie treningov na ozhidaniya intervyuerov pri perekhode ot PAPI k CAPI [“Do not worry!” — training’s impact on interviewers’ expectations towards the change from PAPI to CAPI]. Sociologiya: Metodologiya, Metody, Matematicheskoe Modelirovanie. 2017; 45: 102—132 (In Russ.).
  2. Deviatko I.F., Lebedev D.V. Glazami intervyuera, glazami respondenta: kontury novogo podkhoda k otsenke kognitivnoj nagruzki pri provedenii oprosa [Through the eyes of the interviewer, through the eyes of the respondent: A new approach to the assessment of cognitive load during the interview. Monitoring Obschestvennogo Mneniya: Ekonomicheskie i Socialnye Peremeny. 2017; 5: 1—19 (In Russ.).
  3. Terentev E.A., Mavletova A.M., Kosolapov M.S. Intervyuirovanie s pomoschyu kompyuternyh tekhnologij v longityudnyh obsledovaniyah domohozyajstv [Computer-assisted personal interviewing for longitudinal household studies]. Monitoring Obschestvennogo Mneniya: Ekonomicheskie i Socialnye Peremeny. 2018; 3: 47—64 (In Russ.).
  4. Bassili J.N., Scott B.S. Response latency as a signal to question problems in survey research. Public Opinion Quarterly. 1996; 60 (3): 390—399.
  5. Bratfisch O., Borg G., Dornic S. Perceived Item Difficulty in Three Tests of Intellectual Performance Capacity. Stockholm: Institute of Applied Psychology; 1972. Report No. 29.
  6. Chen F. et al. Robust Multimodal Cognitive Load Measurement. Springer International Publishing; 2016.
  7. Couper M.P., Burt G. Interviewer attitudes toward computer-assisted personal interviewing (CAPI). Social Science Computer Review. 1994; 12 (1): 38—54.
  8. Critchley H.D. Electrodermal responses: What happens in the brain. The Neuroscientists. 2002; 8 (2): 132—142.
  9. De Leeuw E.D., Hox J.J., Snijkers G. The effect of computer-assisted interviewing on data quality. International Journal of Market Research. 1995; 37 (4): 325—344.
  10. Hoogerheide V., Renkl A., Fiorella L., Paas F., van Gog T. Enhancing example-based learning: Teaching on video increases arousal and improves problem-solving performance. Journal of Educational Psychology. 2018. http://dx.doi.org/10.1037/edu0000272.
  11. Hӧhne J.C., Schlosser S., Krebs D. Investigating cognitive effort and response quality of question formats in web surveys using paradata. Field Methods. 2017; 29 (4): 365—382.
  12. Jbara A., Feitelson D.G. How programmers read regular code: A controlled experiment using eye tracking. Empirical Software Engineering. 2017; 22 (3): 1440—1477.
  13. Kaminska O., Foulsham T. Real-world eye-tracking in face-to-face and web modes. Journal of Survey Statistics and Methodology. 2014; 2 (3): 343—359.
  14. Paas F.G. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology. 1992; 84 (4): 429—434.
  15. Paas F., Tuovinen J.E., Tabbers H., Van Gerven P.W.M. Cognitive load measurement as a means to advance cognitive load theory. Educational psychologist. 2003; 38 (1): 63—71.
  16. Sharot T., Phelps E.A. How arousal modulates memory: Disentangling the effects of attention and retention. Cognitive, Affective, & Behavioral Neuroscience. 2004; 4 (3): 294—306.
  17. Stodel M. But what will people think? Getting beyond social desirability bias by increasing cognitive load. International Journal of Market Research. 2015; 57 (2): 313—321.

Statistics

Views

Abstract - 643

PDF (Russian) - 318

Cited-By


PlumX

Dimensions


Copyright (c) 2018 Deviatko I.F.

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

This website uses cookies

You consent to our cookies if you continue to use our website.

About Cookies