The Effects of Monte Carlo Sampling on Automated Valuation Model Performance in Real Estate Assessment

Purpose: The aim of this research is to examine the effects of Monte Carlo sampling methods on
the accuracy, precision, and consistency of automated valuation models (AVMs). Currently,
compliance to international ratio standards has been measured against one single configuration of an
AVM. However, the application of Monte Carlo sampling within AVMs has the potential to improve
on model performance indicators concerning accuracy, precision, and consistency and provide more
flexibility for the assessment community.
Methodology: This paper examines the process of both in-sample and hold-out sample selection.
It applies 750 Monte Carlo simulations and calibrates three AVMs using multi-linear regression,
gradient boosting machine regression, and geographically weighted regression approaches. Each of
the model iterations are evaluated for accuracy, precision and consistency based on the IAAO (2013)
Standard on Ratio Studies.
Findings: The findings show that applying Monte Carlo techniques to the sampling process in
automated valuation model calibration allows for the emergence of normal distributions in the
frequency distributions of the model performance indicator results. These normal distributions can
subsequently be used to calculate inferential statistics to estimate the probability of non-compliance
of an AVM.
Originality: While the Monte Carlo method has been applied to real estate studies previously, this
technique has not been used to estimate non-compliance against international standards. This paper
solely addresses the outcomes of the model performance indicators rather than the coefficients in the
AVMs. For scenarios where there is limited sales evidence, the analysis indicates that the application
of different sample selection configurations can provide support for performance analysis.