Validation of a computerized cognitive assessment tool: Assessing the locked-in patient.

Jen Ayala, Angela Jefferson, Stephanie Cosentino, Rachel Kelley, Carol Lippa, Sandra Koffler, Lauren Littlefield, Philip Schatz, Douglas Chute


Abstract:

Objective: We developed a computerized assessment tool for validation in screening cognitive competency for locked-in patients. The clinical problem is that some locked-in patients (eg. people with ALS) may have difficult to measure cognitive deficits when their response capabilities are severely language or motor impaired.

Method: Our program assesses twenty-four areas of cognitive functioning in attention, language, memory, visual spatial and executive functioning. A technique is used where answers to questions are highlighted by a strobe and a selection is made by a simple motor response (eg. a tap on the computer mouse, or in the future an eye-blinking response). We tested a dementia group, an age matched control group and a student control group.

Results: Using a multivariate anova comparing the three groups, significant differences were found in almost all categories. On one category there were no differences between the three groups, but the test will be kept as a measure of effort. Another category showed no difference between the groups because there was only one question. In Bonferroni post-hoc analyses comparing each group, the demented group performed significantly worse than either control group.

Conclusion: These results confirm that the assessment tool is valid in differentiating between normal and impaired cognitive functioning. The plan is to reduce the length of the test, create cutoff scores for normal cognitive functioning, and create a profile for use with locked-in patients.


Introduction:

  • ALS is a motor neuron disease that leads to the death of the motor neurons responsible for voluntary movements and muscle power, which results in locked-in syndrome.
  • Those with ALS are faced with many important questions concerning normal everyday activities, as well as whether to use ventilators or to end life support.
  • Because of physical impairments, those with ALS canŐt move their hands to communicate and canŐt speak, so it has been difficult to cognitively assess them.
  • Many neuropsychological tests do not account for motor and speech impairments in the severely impaired.
  • Computers can help assist in making tests more valid and reliable, and account for motor and speech impairments.
  • The present pilot study uses a special technique to compensate for speech and motor impairment.
  • The pilot study uses controls and dementia patients to validate the software in differentiating between normal and impaired cognitive functioning.
  • After the software is validated, ALS patients will be tested to understand their cognitive functioning.

Rationale:

The cognitive impairments seen in ALS could cognitively match Dementia. If we use dementia patients as the impaired group, we can first see if the test can differentiate between normal and impaired cognitive functioning.

If the test proves valid, it can be administered to ALS patients, to see which pattern they follow; normal, impaired, or their own.

Hypothesis:

The SLIP test will differentiate between normal and impaired cognitive functioning, by the demented group having more incorrect responses than the normal group.


Methodology

Participants:
Nine (7 male, 2 female) dementia patients ranging in age from 38 to 82.

Twenty-eight (13 male, 15 female) age-matched normal cognitive functioning individuals ranging in age from 28 to 79.

Sixty-one normal cognitive functioning college-aged students were also tested.

Materials:
The SLIP test looks at quality of life and the domains of language, attention, memory, visual spatial and executive functioning in 24 areas:

Quality of life: orientation, general health and severity of ALS

Language: reading-word identification, reading-sentence completion, naming, matching pictures to phrases, matching shapes to phrases, and complex comprehension

Attention: attending to a short story, serial computations, attention to numbers, and visual attention

Memory: video memory, list learning, remote memory, object learning, memory for faces, video delayed recall, list delayed recall, and object delayed recall

Visual-spatial: mental rotation, progressive matrices, and hidden blocks.

Executive functioning: pattern recognition, categorization, and conceptual reasoning

Procedures:
The software was run on a G3 laptop computer. The program runs through each of the testing categories one after the other.

To compensate for motor and speech impairment, a strobing technique highlights each answer choice and the subject uses a mouse click to respond.

The software is self-narrated for easier use and better reliability, although an experimenter is present for assistance.

Analyses:
The computer collected each response and the experimenter calculated the number correct and incorrect. The percentage correct was calculated for each category. A multivariate ANOVA statistic with post-hoc Bonforonni corrections compared performances of the three groups.


Results:
The Dementia group had lower percentages correct for all 24 areas tested. The following are graphs of the percentages of all the 24 tests, separated by cognitive domain.

Multivariate ANOVA comparing the three groups:
Significant differences were found on the following tests: p< .001

Language: reading-word identification, reading-sentence completion,naming, matching pictures to phrases, complex comprehension
Attention: serial computation, attention to numbers, visual attention
Memory: video memory, list recognition, remote memory, object recognition, memory for faces, video delayed recall, object delayed recall
Visual-Spatial: mental rotation, progressive matrices, hidden blocks
Executive Functioning: pattern recognition, categorization, and conceptual reasoning

No significant differences were found on the following tests:

Language: matching shapes to phrases (p=.767)
Attention: attending to a story (p=.101)

Post-Hoc Analyses (With Bonferonni Correction):

The demented group performed significantly worse than the age-matched and student controls in the following categories: p<.05

Language: reading-word identification, naming, matching-pictures to phrases, complex comprehension
Attention: serial computation, attention to numbers
Memory: video memory, list recognition, object recognition, memory for faces, video delayed recall, list delayed recall, object delayed recall
Visual-Spatial: mental rotation, progressive matrices, hidden blocks
Executive Functioning: pattern recognition, and categorization

The demented group performed significantly worse than the students, but not the age-matched control group:

Language: reading-sentence completion (p=.821)
Visual-Spatial: visual attention (p=.083)

The demented group performed significantly worse than the age-matched control group, but not the student control group:

Memory: remote memory (p=1)
Executive Functioning: conceptual reasoning (p=1)

Discussion/Conclusions:
The dementia group performed worse than the control groups on all 24 tests measuring language, attention, memory, visual-spatial, and executive functioning.

These results mean that the SLIP can differentiate between normal and impaired cognitive functioning.

Tests that did not show a significant difference between the impaired and non-impaired groups were due to either not enough questions within the test or the dementia group performing well. These tests will be revised or excluded from the program.

In order to obtain more reliable and valid results, approximately twenty more dementia patients will be tested.

Cut-off scores will be established for normal and impaired cognitive functioning.

If the test is still valid, an ALS group will be tested to understand their cognitive functioning.