Assessing inter- and intra-agreement for dependent binary data

A Bayesian hierarchical correlation approach

Research output: Contribution to journalArticle

3 Citations (Scopus)

Abstract

Agreement measures are designed to assess consistency between different instruments rating measurements of interest. When the individual responses are correlated with multilevel structure of nestings and clusters, traditional approaches are not readily available to estimate the inter- and intra-agreement for such complex multilevel settings. Our research stems from conformity evaluation between optometric devices with measurements on both eyes, equality tests of agreement in high myopic status between monozygous twins and dizygous twins, and assessment of reliability for different pathologists in dysplasia. In this paper, we focus on applying a Bayesian hierarchical correlation model incorporating adjustment for explanatory variables and nesting correlation structures to assess the inter- and intra-agreement through correlations of random effects for various sources. This Bayesian generalized linear mixed-effects model (GLMM) is further compared with the approximate intra-class correlation coefficients and kappa measures by the traditional Cohen's kappa statistic and the generalized estimating equations (GEE) approach. The results of comparison studies reveal that the Bayesian GLMM provides a reliable and stable procedure in estimating inter- and intra-agreement simultaneously after adjusting for covariates and correlation structures, in marked contrast to Cohen's kappa and the GEE approach.

Original languageEnglish
Pages (from-to)173-187
Number of pages15
JournalJournal of Applied Statistics
Volume39
Issue number1
DOIs
Publication statusPublished - 2012 Jan 1

Fingerprint

Binary Data
Dependent Data
Cohen's kappa
Linear Mixed Effects Model
Generalized Estimating Equations
Correlation Structure
Intraclass Correlation Coefficient
Random Effects
Statistic
Covariates
Adjustment
Equality
Evaluation
Estimate
Generalized estimating equations
Correlation structure
Model

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

@article{151e4df42a8945c7b29e18438d585d22,
title = "Assessing inter- and intra-agreement for dependent binary data: A Bayesian hierarchical correlation approach",
abstract = "Agreement measures are designed to assess consistency between different instruments rating measurements of interest. When the individual responses are correlated with multilevel structure of nestings and clusters, traditional approaches are not readily available to estimate the inter- and intra-agreement for such complex multilevel settings. Our research stems from conformity evaluation between optometric devices with measurements on both eyes, equality tests of agreement in high myopic status between monozygous twins and dizygous twins, and assessment of reliability for different pathologists in dysplasia. In this paper, we focus on applying a Bayesian hierarchical correlation model incorporating adjustment for explanatory variables and nesting correlation structures to assess the inter- and intra-agreement through correlations of random effects for various sources. This Bayesian generalized linear mixed-effects model (GLMM) is further compared with the approximate intra-class correlation coefficients and kappa measures by the traditional Cohen's kappa statistic and the generalized estimating equations (GEE) approach. The results of comparison studies reveal that the Bayesian GLMM provides a reliable and stable procedure in estimating inter- and intra-agreement simultaneously after adjusting for covariates and correlation structures, in marked contrast to Cohen's kappa and the GEE approach.",
author = "Miao-Yu Tsai",
year = "2012",
month = "1",
day = "1",
doi = "10.1080/02664763.2011.578623",
language = "English",
volume = "39",
pages = "173--187",
journal = "Journal of Applied Statistics",
issn = "0266-4763",
publisher = "Routledge",
number = "1",

}

Assessing inter- and intra-agreement for dependent binary data : A Bayesian hierarchical correlation approach. / Tsai, Miao-Yu.

In: Journal of Applied Statistics, Vol. 39, No. 1, 01.01.2012, p. 173-187.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Assessing inter- and intra-agreement for dependent binary data

T2 - A Bayesian hierarchical correlation approach

AU - Tsai, Miao-Yu

PY - 2012/1/1

Y1 - 2012/1/1

N2 - Agreement measures are designed to assess consistency between different instruments rating measurements of interest. When the individual responses are correlated with multilevel structure of nestings and clusters, traditional approaches are not readily available to estimate the inter- and intra-agreement for such complex multilevel settings. Our research stems from conformity evaluation between optometric devices with measurements on both eyes, equality tests of agreement in high myopic status between monozygous twins and dizygous twins, and assessment of reliability for different pathologists in dysplasia. In this paper, we focus on applying a Bayesian hierarchical correlation model incorporating adjustment for explanatory variables and nesting correlation structures to assess the inter- and intra-agreement through correlations of random effects for various sources. This Bayesian generalized linear mixed-effects model (GLMM) is further compared with the approximate intra-class correlation coefficients and kappa measures by the traditional Cohen's kappa statistic and the generalized estimating equations (GEE) approach. The results of comparison studies reveal that the Bayesian GLMM provides a reliable and stable procedure in estimating inter- and intra-agreement simultaneously after adjusting for covariates and correlation structures, in marked contrast to Cohen's kappa and the GEE approach.

AB - Agreement measures are designed to assess consistency between different instruments rating measurements of interest. When the individual responses are correlated with multilevel structure of nestings and clusters, traditional approaches are not readily available to estimate the inter- and intra-agreement for such complex multilevel settings. Our research stems from conformity evaluation between optometric devices with measurements on both eyes, equality tests of agreement in high myopic status between monozygous twins and dizygous twins, and assessment of reliability for different pathologists in dysplasia. In this paper, we focus on applying a Bayesian hierarchical correlation model incorporating adjustment for explanatory variables and nesting correlation structures to assess the inter- and intra-agreement through correlations of random effects for various sources. This Bayesian generalized linear mixed-effects model (GLMM) is further compared with the approximate intra-class correlation coefficients and kappa measures by the traditional Cohen's kappa statistic and the generalized estimating equations (GEE) approach. The results of comparison studies reveal that the Bayesian GLMM provides a reliable and stable procedure in estimating inter- and intra-agreement simultaneously after adjusting for covariates and correlation structures, in marked contrast to Cohen's kappa and the GEE approach.

UR - http://www.scopus.com/inward/record.url?scp=84858200415&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858200415&partnerID=8YFLogxK

U2 - 10.1080/02664763.2011.578623

DO - 10.1080/02664763.2011.578623

M3 - Article

VL - 39

SP - 173

EP - 187

JO - Journal of Applied Statistics

JF - Journal of Applied Statistics

SN - 0266-4763

IS - 1

ER -