Sampling behaviour of kappa and weighted kappa in the null case

Kappa and weighted kappa are widely used statistics for measuring inter-rater agreement. New formulas for their asymptotic standard errors are proposed under the condition that their population values are zero, and their behaviours compared to that of previous formulas used to test the null hypothesis that these population measures are zero. The results indicate that critical ratios based on the new formulas provide better control of Type I errors. The implications of these findings for the prediction analysis of cross-classification are also considered.