16/06/2021

Cognitive interviewing for testing survey questions

Presenter(s): Pamela Campanelli, The Survey Coach

e-mail(s): dr.pamela.campanelli@thesurveycoach.com

Date: 27 April 2020


Webinar aims

This webinar is designed to introduce participants to a powerful and efficient method of testing survey questions called cognitive interviewing. Cognitive interviewing itself encompasses a range of methods. The most popular ones include think-alouds, probing, and observation. This webinar gives an overview of what cognitive interviewing is and how it is used. It also covers some important tips on questionnaire design and how to improve your survey questions by using cognitive interview findings.


Webinar learning outcomes
By the end of the course participants will

  • Have a better understanding of what cognitive interviewing is.
  • Have knowledge of some key principles of questionnaire design.
  • Have knowledge to improve survey questions with cognitive interviewing results.

Webinar content

  • What is cognitive interviewing?

  • The four cognitive steps in answering a survey question

  • Cognitive interview tools with a focus on thinking aloud and probing

  • Some important tips for questionnaire design

  • Simple cognitive analysis

  • Example of using the cognitive results to improve a survey question

  • Appendix on peripheral cognitive interviewing tools, doing a proper introduction to the cognitive interview, tips for solving question comprehension, memory and sensitivity issues, question formats to be beware of or avoid, more complicated cognitive interviewing analysis and reporting, list of cognitive interviewing resources


Difficulty level

Introductory


Prerequisites for the webinar

No prerequisite


Further reading and resources

The two best readings

  • Willis, G.B. (2005), Cognitive Interviewing: A Tool for Improving Questionnaire Design, Thousand Oaks, CA: Sage. THIS IS THE KEY BOOK – HIGHLY RECOMMENDED.

  • Fowler, F.J. Jr., (1995), Improving Survey Questions: Design and Evaluation, Applied Social Research Methods Series Volume 38, Thousand Oaks, CA: SAGE Publications (Chapters 1-4). (Dr. Campanelli loves this book. Only one part is not accurate due to time. This part is explained in the course appendix.) You can access it free as part of a free 30 trial here: https://www.scribd.com/doc/188087201/FOWLER-F-1995-Improving-survey-questions-pdf.

Other readings if you can’t access the books

More advanced issues in cognitive interviewing

  • Willis, G.B. (2015), Analysis of the Cognitive Interview in Questionnaire Design, Oxford: Oxford University Press. THIS SHOWS SOME OF THE DIFFERENT WAYS IN WHICH COGNITIVE INTERVIEW DATA CAN BE ANALYSED

  • Collins, D. (ed.) (2015), Cognitive Interviewing Practice, Sage. EXTREMELY THOROUGH VERSION OF COGNITIVE INTERVIEWING AS A “QUALITATIVE” METHOD.

  • Boeije, H.R. and Willis, G.B. (2013), The Cognitive Interviewing Reporting Framework (CIRF), Methodology,9(3): 87-95. https://www.researchgate.net/publication/273202311_The_Cognitive_Interviewing_Reporting_Framework_CIRF

  • Miller, K. (2011), Cognitive Interviewing, Chapter 5 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.

  • Willis, G. (2011), Response 1 to Miller’s Chapter: Cognitive Interviewing, Chapter 6 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.

  • Conrad, F. (2011), Response 2 to Miller’s Chapter: Cognitive Interviewing, Chapter 7 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.

  • Blair, J. and Conrad, F. (2011), Sample Size for Cognitive Interview Pretesting, Public Opinion Quarterly, 75(4) 636–658.


Presentation material

Slides_main presentation

Slides_appendix

Additional material


Recording of the webinar