Comparability of Formative Measures in Cross-national Surveys

This seminar forms part of the Research Training Centre's Talking Methods seminar series delivered in association with the Scottish Graduate School of Social Science (SGSSS).

Abstract

Political and social scientists, employing surveys in their research, often utilize single questions or multi-item scales to measure complex theoretical constructs, such as values, attitudes, personality traits, or beliefs. In comparative contexts, it is of essential importance to ensure that a specific measurement instrument functions equivalently in different countries (or other culturally dissimilar units). “Functions equivalently” typically means that the measurement instrument provides equivalent numerical scores for individuals living in different environments but sharing identical positions on the measured construct. This property, generally known as measurement invariance/equivalence, is the key prerequisite of any meaningful cross-country comparison.

Measurement invariance constitutes an important research field in the modern survey methodology, but it is typically discussed from the perspective of so called reflective measurement models, which tie measurement validity (including comparability) of complex multi-item constructs to the strength (and also cross-national similarity) of correlations between their observed indicators. Factor analysis and its extension, Multiple Group Confirmatory Factor Analysis (MGCFA), specifically devised for comparative analysis, are perhaps the most widely known (and used) members of this family of measurement methods. Yet, there exist another broad group of powerful measurement approaches – so called formative measures, which do not require strong correlations between different indicators of the same construct and still allow for relatively accurate measurement of important attitudinal constructs.

The key goal of this seminar is to introduce the latter measurement approach, which remains largely unknown to the political science community, with a specific focus on its advantages and disadvantages for the purposes of cross-national research. Professor Sokolov will begin by briefly explaining what the key defining features of formative measures are and how these measures differ from reflective measures, more popular in modern cross-national survey research. He will then introduce a general taxonomy of formative measurement models, proposed by Bollen and Diamantopoulos (2017). Next, the speaker will discuss various approaches to comparability testing available for different classes of formative measures and outline their relative strengths and weaknesses. He will finish by discussing the potential effects of measurement error (which is much more difficult to address at either item- or construct level in formative models, compared to reflective models) on the cross-cultural comparability of formative measures, and show that, while potentially devastating for the validity of comparisons between specific individuals from different countries, they are often not as critical when what is compared are country-level summaries (e.g., national mean scores). Moreover, researchers can sometimes control the impact of systematic measurement error on the results of the aggregate-level inferential analyses (e.g. regressions using formative mean scores as inputs) by directly integrating simple measures of survey data quality in the modelling process. All technical concepts discussed in the presentations, are illustrated empirically using two novel constructs, measuring mass-level liberal and authoritarian notions of democracy in the World Values Surveys, recently introduced by Kirsch and Welzel (2019).

References:

1. Bollen, K. A., & Diamantopoulos, A. (2017). In defense of causal-formative indicators: A minority report. Psychological methods, 22(3), 581.
2. Kirsch, H., & Welzel, C. (2019). Democracy misunderstood: authoritarian notions of democracy around the globe. Social Forces, 98(1), 59-92.

Biography

Associate Professor Boris Sokolov is a Research Fellow at the Ronald F. Inglehart Laboratory for Comparative Social Research at the Higher School of Economics, St. Petersburg, Russia.

University Profile: https://www.hse.ru/en/org/persons/50169851