Preliminary Psychometric Comparisons of Remote-Televideo and Face-to-Face Administration of a Commonly Applied Neurocognitive Assessment Battery

Aaron S. Kemp, BA; Kirstan Gooch, BSc; Joisabel Goldberg, BA; Sitha Bun, BSc; Dimitrios Gripeos, BA; Christopher Reist, MD; Barton W. Palmer, PhD; Mark Bondi, PhD; and James P. O'Halloran, PhD
University of California Irvine School of Medicine, Irvine (Dr Reist, Mr Gripeos, and Mr Kemp); NeuroComp Systems, Inc, Irvine, California (Dr O'Halloran, Mr Kemp, Ms Bun, Ms Goldberg, and Ms Gooch); Veterans Administration Healthcare System, Long Beach, California (Dr Reist and Mr Gripeos); University of California, San Diego, and Veterans Affairs Medical Research Foundation, San Diego, California (Drs Bondi and Palmer)

This poster presentation was supported by the NIH Small Business Innovation Research (SBIR) Program.

Background: Neurocognitive assessment is an essential component of controlled clinical trials of candidate “cognitive-enhancing” treatments for a range of disorders. However, manual administration of large, paper-based, neurocognitive batteries is often inefficient, error-prone, and inconsistent across multiple sites. Existing computerized testing systems are also limited in both the assessment instruments available and the range of impairments that can be accommodated with the subject sitting alone at a single display. Therefore, a unique, dual-display, computerized testing system was developed, with funding from the National Institute of Mental Health (NIMH), that integrates (rather than replaces) the examiner for computerized administration of standard neurocognitive assessment batteries.^1,2 The dual-display configuration has now been extended to support two-way, wireless televideo communications for remote administration by an examiner at a distal location. The purpose of the current study was to evaluate the feasibility of this novel configuration by comparing the concurrent validity of remote, televideo-based (RT) administration of a representative battery of 12 widely applied neurocognitive assessment instruments with face-to-face (FF) administration of the same battery.

Methods: The neurocognitive battery was administered to 25 healthy subjects with no history of psychiatric diagnoses via both methods ~30 days apart with the order counterbalanced across participants. Additional data are currently being collected among patients diagnosed with mild cognitive impairment, schizophrenia, or traumatic brain injury (~15 each), and these data will also be available to further evaluate concurrent validity and test-retest reliability within these impaired populations.

Results: Intraclass Correlation Coefficient (ICC) comparisons of concurrent validity between RT and FF batteries yielded highly significant measures of consistency and absolute agreement for all 12 tests and no significant mean differences were found between the methods using paired-samples T-test comparisons.

Conclusions: While still requiring further validation among patients with known impairments, these preliminary results support the feasibility of administering a computerized battery of commonly applied neurocognitive assessment instruments via remote, televideo interactions with an expert examiner at a distal location.

References

1. O'Halloran JP, Kemp AS, Gooch KN, et al. Psychometric comparison of computerized and standard administration of the neurocognitive assessment instruments selected by the CATIE and MATRICS consortia among patients with schizophrenia. Schizophr Res. 2008;106(1):33–41.
2. O'Halloran JP, et al. Concurrent validity of a computerized neurocognitive assessment system encompassing CATIE and MATRICS batteries. Schizophr Bull. 2007;33(2):571.

Back      Poster