Getting robust and valid data on decision policies: The factorial survey

Brian Taylor, Richard Zeller

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.
Original languageEnglish
Pages (from-to)27-42
Number of pages16
JournalIrish journal of Psychology
Issue number1-2
Publication statusPublished (in print/issue) - 30 Jun 2007

Bibliographical note

Prof Rich Zeller died on 16 April 2009, leaving a lasting legacy of inspiration in the understanding of statistics in real-world applications.


  • Decision making; experiment; professional judgement; research method; survey; vignette.


Dive into the research topics of 'Getting robust and valid data on decision policies: The factorial survey'. Together they form a unique fingerprint.

Cite this