The Reliability of Clinical Adjudication Panels in the Longitudinal Study of Implantable Cardioverter Defibrillators
University of Michigan
Charles E. Gaber, Robert Greenlee
Center for Clinical Epidemiology and Population Health
Research area: Epidemiology
Background: Clinical trials have demonstrated the benefits of implantable cardioverter-defibrillators (ICDs) in the primary prevention of sudden cardiac death by delivering therapy for malignant ventricular arrhythmias. The Longitudinal Study of Implantable Cardioverter Defibrillators (LSICD) utilized clinical adjudication panels to determine the occurrence and appropriateness of ICD therapy outcomes in routine clinical practice settings. We characterized the reliability of these adjudication panels.
Methods: Study-eligible patients had to receive an ICD for the first time for the primary prevention of sudden cardiac death, have left ventricular systolic dysfunction, and be a patient in the seven participating health systems. All treated episodes for each patient were reviewed by a central physician panel, with a sample additionally reviewed by an external panel of experts. Kappa statistics and percent agreement were calculated to characterize the level of agreement between the local provider’s interpretation of therapy appropriateness, as documented in the medical record, and the final adjudicated determination of device therapy appropriateness. Similar calculations were made to assess inter-reviewer reliability in both panels and agreement between the final central panel and final external panel determinations.
Results: Kappa for agreement between the local provider determined appropriateness and the final adjudicated appropriateness was 0.79 (95% CI: 0.77, 0.81). Central panel inter-reviewer kappa statistics for the adjudicated variables ranged from 0.45 to 0.92. Specifically, kappa for central adjudicated appropriateness was 0.72 (95% CI: 0.69, 0.75). External panel inter-reviewer kappa statistics for the adjudicated variables ranged from 0.43 to 0.85. Kappa for external adjudicated appropriateness was 0.60 (95% CI: 0.54, 0.67). Kappa statistics for agreement between the final central adjudication and final external adjudication ranged from 0.19 to 0.95.
Conclusions: Agreement between and amongst panels is favorable. Adjudication panels are a reliable method of determining outcomes in the LSICD. Further studies can validate this finding in other study settings.