Presenter(s): Pamela Campanelli, The Survey Coach
e-mail(s): dr.pamela.campanelli@thesurveycoach.com
Date: 27 April 2020
Webinar aims
This webinar is designed to introduce participants to a powerful and efficient method of testing survey questions called cognitive interviewing. Cognitive interviewing itself encompasses a range of methods. The most popular ones include think-alouds, probing, and observation. This webinar gives an overview of what cognitive interviewing is and how it is used. It also covers some important tips on questionnaire design and how to improve your survey questions by using cognitive interview findings.
Webinar learning outcomes
By the end of the course participants will
- Have a better understanding of what cognitive interviewing is.
- Have knowledge of some key principles of questionnaire design.
- Have knowledge to improve survey questions with cognitive interviewing results.
Webinar content
-
What is cognitive interviewing?
-
The four cognitive steps in answering a survey question
-
Cognitive interview tools with a focus on thinking aloud and probing
-
Some important tips for questionnaire design
-
Simple cognitive analysis
-
Example of using the cognitive results to improve a survey question
-
Appendix on peripheral cognitive interviewing tools, doing a proper introduction to the cognitive interview, tips for solving question comprehension, memory and sensitivity issues, question formats to be beware of or avoid, more complicated cognitive interviewing analysis and reporting, list of cognitive interviewing resources
Difficulty level
Introductory
Prerequisites for the webinar
No prerequisite
Further reading and resources
The two best readings
-
Willis, G.B. (2005), Cognitive Interviewing: A Tool for Improving Questionnaire Design, Thousand Oaks, CA: Sage. THIS IS THE KEY BOOK – HIGHLY RECOMMENDED.
-
Fowler, F.J. Jr., (1995), Improving Survey Questions: Design and Evaluation, Applied Social Research Methods Series Volume 38, Thousand Oaks, CA: SAGE Publications (Chapters 1-4). (Dr. Campanelli loves this book. Only one part is not accurate due to time. This part is explained in the course appendix.) You can access it free as part of a free 30 trial here: https://www.scribd.com/doc/188087201/FOWLER-F-1995-Improving-survey-questions-pdf.
Other readings if you can’t access the books
-
Krosnick, J. A. and Presser, S. (2010), Questionnaire Design in J. D. Wright and P.V. Marsden (eds), Handbook of Survey Research (Second Edition), San Diego, CA: Elsevier. 2009 pre-publication version available at
https://web.stanford.edu/dept/communication/faculty/krosnick/docs/2009/2009_handbook_krosnick.pdf -
Willis GB. (1994), Cognitive Interviewing and Questionnaire Design: A Training Manual. A “how to” guide. Cognitive Methods Staff, Working Paper Series, No. 7, Office of Research and Methodology, National Centre for Health Statistics and Center for Disease Control publication. http://www.srl.uic.edu/links/CMS_WP07_Willis_1994_CogIntTraining.pdf
More advanced issues in cognitive interviewing
-
Willis, G.B. (2015), Analysis of the Cognitive Interview in Questionnaire Design, Oxford: Oxford University Press. THIS SHOWS SOME OF THE DIFFERENT WAYS IN WHICH COGNITIVE INTERVIEW DATA CAN BE ANALYSED
-
Collins, D. (ed.) (2015), Cognitive Interviewing Practice, Sage. EXTREMELY THOROUGH VERSION OF COGNITIVE INTERVIEWING AS A “QUALITATIVE” METHOD.
-
Boeije, H.R. and Willis, G.B. (2013), The Cognitive Interviewing Reporting Framework (CIRF), Methodology,9(3): 87-95. https://www.researchgate.net/publication/273202311_The_Cognitive_Interviewing_Reporting_Framework_CIRF
-
Miller, K. (2011), Cognitive Interviewing, Chapter 5 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
-
Willis, G. (2011), Response 1 to Miller’s Chapter: Cognitive Interviewing, Chapter 6 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
-
Conrad, F. (2011), Response 2 to Miller’s Chapter: Cognitive Interviewing, Chapter 7 in Madans et al (eds), Question Evaluation Methods, Hoboken, NJ: Wiley.
-
Blair, J. and Conrad, F. (2011), Sample Size for Cognitive Interview Pretesting, Public Opinion Quarterly, 75(4) 636–658.
Presentation material
Recording of the webinar