The reliability and validity of Northern Ireland multiple deprivation measurement: An objective, data-driven interrogation of a subjective, consensus-derived instrument.

Background: Multiple deprivation measures (MDM) have been used to inform policy and resource allocation in Northern Ireland (NI) for many years. The current measure (NIMDM 2017) attempts to capture deprivation across seven domains (Income, Employment, Health & Disability, Education, Skills & Training, Access to Services, Living Environment and Crime & Disorder). The indicators that inform these domains have been derived using subjective, consensus-driven approaches, and the weightings of these indicators and domains have, in turn and in part, been determined using factor analysis. The reliability and validity of this measure, however, has, to our knowledge, never been empirically tested.
Methods: Accessing NIMDM 2017 domain-indicator data via the NI Neighbourhood Information Service and informed by the statistical methods used by those who designed the measure, we employed correlational, internal consistency, and factor analyses to test eight separate hypotheses regarding the reliability and inferred dimensional structure of the NIMDM. To assess the measure’s validity/predictive utility, we used NIMDM data linked to hospital admissions data to test three separate hypotheses relating to health outcomes for a range of common health conditions, each of which have been evidenced to be associated with deprivation.
Results: Regarding NIMDM reliability, correlational analyses indicated that several within-domain indicators were not significantly correlated, that some between-domain correlations exceeded within-domain associations, and that indicators within the Living domain demonstrated limited coherence. While internal consistency testing indicated that some domains were reliably identified by indicators, the Income and Living domains were not. CFAs failed to support the inferred hierarchical dimensional structure of the measure. Regarding NIMDM validity, correlational analyses indicated that hospital admissions were indeed correlated with deprivation status but ranged from an extremely weak -0.06 to 0.01. Regarding health risk, NIMDM seemed to meaningfully predict risk and hospitalization for chronic obstructive pulmonary disease (COPD), mental and behavioural disorders due to psychoactive substance use, and depression only.
Conclusion: While factor analysis may have been used in the design of the NIMDM 2017, treating the NIMDM indicators as being reflective of the deprivation latent constructs did not result in acceptable model fit. Given that much of the literature concerning the conceptualisation and measurement of deprivation advocated a formative modelling approach, where the indicators are specified to be predictors of the deprivation latent constructs, there should be greater attention paid to harmonising the theoretical and statistical representations of deprivation.