Getting robust and valid data on decision policies: The factorial survey

Brian Taylor, Richard Zeller

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.
LanguageEnglish
Pages27-42
Number of pages16
JournalIrish journal of Psychology
Volume28
Issue number1-2
Publication statusPublished - 30 Jun 2007

Fingerprint

risk society
experiment
research planning
criticism
health

Keywords

  • Decision making; experiment; professional judgement; research method; survey; vignette.

Cite this

@article{f353b800409540dab8ab070a2ca613ef,
title = "Getting robust and valid data on decision policies: The factorial survey",
abstract = "The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.",
keywords = "Decision making; experiment; professional judgement; research method; survey; vignette.",
author = "Brian Taylor and Richard Zeller",
note = "Prof Rich Zeller died on 16 April 2009, leaving a lasting legacy of inspiration in the understanding of statistics in real-world applications.",
year = "2007",
month = "6",
day = "30",
language = "English",
volume = "28",
pages = "27--42",
journal = "The Irish Journal of Psychology",
issn = "0303-3910",
number = "1-2",

}

Getting robust and valid data on decision policies: The factorial survey. / Taylor, Brian; Zeller, Richard.

In: Irish journal of Psychology, Vol. 28, No. 1-2, 30.06.2007, p. 27-42.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Getting robust and valid data on decision policies: The factorial survey

AU - Taylor, Brian

AU - Zeller, Richard

N1 - Prof Rich Zeller died on 16 April 2009, leaving a lasting legacy of inspiration in the understanding of statistics in real-world applications.

PY - 2007/6/30

Y1 - 2007/6/30

N2 - The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.

AB - The factorial survey is a research design using true-to-life vignettes for the study of decision policies and normative judgments, capitalising on the possibilities of advances in computing. The factorial survey combines the internal validity strengths of the classical random assignment experiment with the external validity strengths of the sample survey. Compared to the better known factorial experiment, the factorial survey uses more independent variables (IV) and more levels of IVs, and thus has greater fidelity to the complexity of real world decisions. The factorial survey handles the question of spuriousness through random assignment of IVs to vignettes and of vignettes to respondent. Respondents may be selected randomly from the population of interest. Practical issues in use of the method are explained, and an example is given of the use of a factorial survey to study professional judgment in health and social care. Criticisms of the factorial survey, such as whether sample sizes are unduly inflated by treating decisions rather than respondents as the unit of analysis, are addressed. Strengths of the method and prospects for the future, particularly in the context of a ‘risk society’, are discussed.

KW - Decision making; experiment; professional judgement; research method; survey; vignette.

M3 - Article

VL - 28

SP - 27

EP - 42

JO - The Irish Journal of Psychology

T2 - The Irish Journal of Psychology

JF - The Irish Journal of Psychology

SN - 0303-3910

IS - 1-2

ER -