The mode effect in mixed-mode surveys

Presenter(s): Claudia De Vitiis and Francesca Inglese, Italian National Statistical Institute (ISTAT)

Date: 26 May 2020

Webinar aims

The mixed mode (MM), i.e. the use of different collection techniques in the same survey, is a relatively new approach that NSI are adopting especially for social surveys. Its use is spreading both to contrast declining response rates and to reduce the total cost of the surveys. However, MM introduces a bias, named mode effect, that must be addressed at different levels: in the design phase by defining the best collection instruments to contain the measurement error; in the estimation phase by assessing and treating the bias effects due to the introduction of MM, in order to ensure the accuracy of the estimates.

The aim of this course is to provide the participants with a review of methods for the assessment and treatment of mode effect, some of them applied in a specific survey experimental context.

Webinar learning outcomes

By the end of the course participants will

Be aware of the type of bias effects introduced by mixed-mode
Have a general knowledge of the theoretical and inferential framework for the evaluation of mode effects
Have knowledge of some of the methods proposed in the literature for the assessment of mode effects
Have received illustration of the steps followed for the analysis of mode effects in a specific survey context

Webinar content

The webinar discusses

Theoretical framework and formalization of the two effects
The impact of total non-response in mixed-mode surveys
Methods for the treatment of selection and measurement effects
An example of analysis of mode effect in a specific survey context

Difficulty level

Advanced

Prerequisites for the webinar

Basic knowledge of survey methodology
Basic knowledge of multivariate statistics

Further readings and resources

Buelens, B. and Van den Brakel, J. A. (2015). Measurement error calibration in mixed-mode, Sociological methods & Research, 4483, pp 391-426.

de Leeuw, Edith D. To mix or not to mix data collection modes in surveys. Journal of Official Statistics, (2005) 21, 2, 233-255. Available at http://www.jos.nu/Articles/abstract.asp?article=212233

Hox, J., de Leeuw, E. D. and Klausch, T. (2015). “Mixed Mode Research: Issues in Design and Analysis.” Invited paper presented at the International Conference on Total survey error: improving quality in the era of big data. Baltimore, 19-22 September.

Martin P. and Lynn, P. (2011). The effects of mixed mode survey designs on simple and complex analyses. Centre for Comparative Social Surveys. Working Paper Series. Paper n.04.

Rosenbaum, P. R. and Rubin, D. B. (1983) The Central Role of the Propensity Score in Observational Studies for Causal Effects, Biometrika, 70, pp. 41-55.

Schouten B., Shlomo, N. and Skinner, C. (2011). Indicators for Monitoring and Improving Representativity of Response. Journal of Official Statistics 27, pp. 231–253.

Vandenplas, C., Loosveldt, G. and Vannieuwenhuyze, J. T. A. (2016). Assessing the use of mode preference as a covariate for the estimation of measurement effects between modes. A sequential mixed mode experiment. Method, data, Analyses. Vol. 10(2), 2016, pp. 119-142.

Vannieuwenhuyze, J. T. A., Loosveldt, G. and Molenberghs, G. (2010). A Method for Evaluating Mode Effects in Mixed-mode Surveys. Public Opinion Quarterly, Volume 74, Issue 5, 1 January 2010, Pages 1027–1045,https://doi.org/10.1093/poq/nfq059.

Presentation material

Presentation Material Webinar The Mode Effect Surveys

Recording of the webinar

Recording Webinar The Mode Effect Surveys