Presenter(s): Pamela Campanelli, The Survey Coach
Date: 27 April 2020
This webinar is designed to introduce participants to a powerful and efficient method of testing survey questions called cognitive interviewing. Cognitive interviewing itself encompasses a range of methods. The most popular ones include think-alouds, probing, and observation. This webinar gives an overview of what cognitive interviewing is and how it is used. It also covers some important tips on questionnaire design and how to improve your survey questions by using cognitive interview findings.
Webinar learning outcomes
By the end of the course participants will
- Have a better understanding of what cognitive interviewing is.
- Have knowledge of some key principles of questionnaire design.
- Have knowledge to improve survey questions with cognitive interviewing results.
What is cognitive interviewing?
The four cognitive steps in answering a survey question
Cognitive interview tools with a focus on thinking aloud and probing
Some important tips for questionnaire design
Simple cognitive analysis
Example of using the cognitive results to improve a survey question
Appendix on peripheral cognitive interviewing tools, doing a proper introduction to the cognitive interview, tips for solving question comprehension, memory and sensitivity issues, question formats to be beware of or avoid, more complicated cognitive interviewing analysis and reporting, list of cognitive interviewing resources
Prerequisites for the webinar
Further reading and resources
The two best readings
Willis, G.B. (2005), Cognitive Interviewing: A Tool for Improving Questionnaire Design, Thousand Oaks, CA: Sage. THIS IS THE KEY BOOK – HIGHLY RECOMMENDED.
Fowler, F.J. Jr., (1995), Improving Survey Questions: Design and Evaluation, Applied Social Research Methods Series Volume 38, Thousand Oaks, CA: SAGE Publications (Chapters 1-4). (Dr. Campanelli loves this book. Only one part is not accurate due to time. This part is explained in the course appendix.) You can access it free as part of a free 30 trial here: https://www.scribd.com/doc/188087201/FOWLER-F-1995-Improving-survey-questions-pdf.
Other readings if you can’t access the books
Krosnick, J. A. and Presser, S. (2010), Questionnaire Design in J. D. Wright and P.V. Marsden (eds), Handbook of Survey Research (Second Edition), San Diego, CA: Elsevier. 2009 pre-publication version available at
Willis GB. (1994), Cognitive Interviewing and Questionnaire Design: A Training Manual. A “how to” guide. Cognitive Methods Staff, Working Paper Series, No. 7, Office of Research and Methodology, National Centre for Health Statistics and Center for Disease Control publication. http://www.srl.uic.edu/links/CMS_WP07_Willis_1994_CogIntTraining.pdf
More advanced issues in cognitive interviewing
Willis, G.B. (2015), Analysis of the Cognitive Interview in Questionnaire Design, Oxford: Oxford University Press. THIS SHOWS SOME OF THE DIFFERENT WAYS IN WHICH COGNITIVE INTERVIEW DATA CAN BE ANALYSED
Collins, D. (ed.) (2015), Cognitive Interviewing Practice, Sage. EXTREMELY THOROUGH VERSION OF COGNITIVE INTERVIEWING AS A “QUALITATIVE” METHOD.
Boeije, H.R. and Willis, G.B. (2013), The Cognitive Interviewing Reporting Framework (CIRF), Methodology,9(3): 87-95. https://www.researchgate.net/publication/273202311_The_Cognitive_Interviewing_Reporting_Framework_CIRF
Miller, K. (2011), Cognitive Interviewing, Chapter 5 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
Willis, G. (2011), Response 1 to Miller’s Chapter: Cognitive Interviewing, Chapter 6 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
Conrad, F. (2011), Response 2 to Miller’s Chapter: Cognitive Interviewing, Chapter 7 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
Blair, J. and Conrad, F. (2011), Sample Size for Cognitive Interview Pretesting, Public Opinion Quarterly, 75(4) 636–658.
Recording of the webinar