Getting robust and valid data on decision policies: The factorial survey

The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.