The Journals of Gerontology Series B: Psychological Sciences and Social Sciences 56:P187-P191 (2001)
© 2001 The Gerontological Society of America
Measuring Disability in Nursing Home Residents
Validity and Reliability of a Newly Developed Instrument
Mirjam Valka,
Marcel W. M. Posta,
Herman J. M. Coolsb and
Guus A. J. P. Schrijversa
a Julius Center for General Practice and Patient Oriented Research, Utrecht University Medical Center, The Netherlands
b Department of Nursing Home Medicine, Leiden University Medical Center, The Netherlands
Mirjam Valk, Julius Center for General Practice and Patient Oriented Research, Utrecht University Medical Center, P.O. Box 80035, 3508 GA Utrecht, The Netherlands E-mail: mvalk{at}trimbos.nl.
Decision Editor: Toni C. Antonucci, PhD
 |
Abstract
|
|---|
A 24-item multidimensional nurse-administered Nursing Home Disabilities Instrument (NHDI) was developed to measure disabilities in nursing home residents. We present the psychometric features and value of this instrument, with the following domains assessed: Mobility, Activities of Daily Living (ADLs), Alertness, Resistance to Nursing Assistance, Incontinence, Cognition, and Perception. Test–retest and interrater reliability was assessed using the Spearman correlation coefficient. Internal consistency was examined by Cronbach's alpha. Criterion validity tests were performed by comparing the scales with scales of the Elderly Residents Rating Scale (BOP). Test–retest reliability correlation coefficients ranged from 0.63 to 0.94. Interrater reliability was high for the scales Cognition, Mobility, ADL, and Incontinence (0.79 to 0.93), moderate for Resistance (0.51), and low for Perception (0.33). Cronbach's alpha of the scales was high, ranging from 0.78 (Alertness) to 0.93 (Mobility); only Perception showed a low alpha: 0.54. Criterion validity was high for Cognition, ADL, and Mobility (0.75 to 0.78), and moderate for Alertness (0.59). The NHDI appears to be a valid and efficient multidimensional instrument for measuring disabilities in nursing home residents. These findings imply that the NHDI is a useful instrument for nursing homes to achieve a reliable assessment of cognitively impaired elders.
THE population of elderly residents in nursing homes is expected to double over the next several decades. Nursing home residents are characterized by frequent comorbidity and disability, and disability in these residents often has a multidimensional nature, occurring in physical, psychosocial and other domains (
Bergner, Bobbit, and Carter 1987
;
Boyle and Torrance 1984;
Gerety et al. 1994;
Katz et al. 1963;
Ware 1987). The provision of appropriate care in nursing facilities requires comprehensive knowledge of the residents' capabilities, limitations, and problems. Accordingly, evaluation of functional status is particularly important for institutionalized elders. Evaluation of functional status and measurement of the level of care required are commonly used in clinical and research contexts. Measurement tools can assess functional deficits and service needs of disabled individuals. Costs of health care can also be predicted. Comprehensive measures, covering a broad range of functional domains of nursing home residents, are scarce. Various domain-specific instruments are available (
Dautzenberg, Schmand, Vriens, Deelman, and Hooijer 1991;
De Graaf and Deelman 1992;
Feher et al. 1992;
Folstein, Folstein, and McHugh 1975;
Katz et al. 1963;
Mahoney and Barthel 1965;
Reisberg 1988;
Reisberg, Ferris, de Leon, and Crook 1988;
Tombaugh and McIntyre 1992). However, obtaining an overall assessment of the status of nursing home residents is still difficult. Combining instruments to obtain comprehensiveness may cause redundancy, become too long, or require skilled personnel to complete.
The Minimum Data Set (MDS) (Casten, Lawton, Parmalee, & Kleban, 1998;
Hawes et al. 1995;
Hawes, Phillips, Mor, Fries, and Morris 1992;
Lawton et al. 1998) measures a broad range of health-related outcomes in nursing home residents. The MDS contains 284 items designed to assess the medical, psychological, and social characteristics of nursing home residents. Although the instrument can be used to obtain data for research, a major disadvantage is that it is very time consuming to complete. Moreover, only the domains Cognition and ADL proved to yield valid results (
Hartmaier, Sloane, Guess, and Koch 1994;
Hartmaier, Sloane, Guess, Koch, Michel, and Phillips 1995;
Lawton et al. 1998;
Teresi and Holmes 1992). Accordingly, it is of limited value for clinical research and population-based needs assessment. In Dutch nursing homes the Elderly Residents Rating Scale (BOP) and the Clinical Behavior Assessment Scale (GIP) are widely used instruments to monitor the status of nursing home residents (
Diesveldt 1981;
Melchior and Frederiks 1992;
Van der Kam, Mol, and Wimmers 1971; Verstraten & Eekelen, 1987). The BOP is a 35-item instrument used for somatic as well as psychogeriatric residents; it has been proven to yield valid results (
Diesveldt 1981;
Van der Kam et al. 1971). The GIP is a rating scale that describes aspects of apathy, and cognitive and affective symptoms in elderly residents. This rating scale was especially developed for cognitively impaired residents. However, validity has not yet been evaluated (
De Jonghe, de Kat, and de Reus 1994;
De Jonghe, Calis, and Boom-Poels 1996).
Despite the obvious need for valid and efficient instruments that measure multiple functional domains, these are currently lacking. Both research and practice in nursing homes could benefit if a valid instrument were available. Such an instrument should provide an accurate assessment of major impairment and disability. For practical purposes, the instrument should be concise and not require specialized personnel (i.e., psychologists or social workers and suchlike) to administer. We developed a short and simple nurse-administered observational instrument to establish the functional status and cognitive status of cognitively impaired nursing home residents. In this article, the psychometric properties of the Nursing Home Disabilities Instrument (NHDI) are presented.
|
Methods
|
|---|
Residents
For this study, psychogeriatric residents (demented, frail elders) on four wards of Albert van Koningsbruggen Care Center, a nursing home center in Utrecht, were included. Data collection was carried out in June and July 1998. Nursing homes in The Netherlands are organized in somatic and psychogeriatric wards. All residents admitted on psychogeriatric wards of the nursing home were assessed using the NHDI. There were no exclusion criteria. The Care Center residents had proportions and distribution of disabilities similar to residents in other Dutch nursing homes (Institute of Informatics in Health and Welfare [SIG], 1995).
Measurement
To develop the NHDI, the SIG registration for nursing homes, the GIP questionnaire, and the MDS Cognition scale were used. SIG collects patient data from almost all nursing homes quarterly. Two items measuring vision and hearing, four items measuring continence, and three items measuring mobility were taken from the SIG inquiry. The GIP, a Dutch observation scale for psychogeriatric nursing home residents, consists of 14 subscales measuring different types of behavior. Six individual items from two subscales that measure alertness were taken from the GIP, as was the item measuring resistance to nursing assistance. The MDS-Cogs taken from the MDS is a scale that consists of eight items to assess the cognitive status of nursing home residents. We omitted the ADL question from this scale and attached this item to two other ADL self-performance questions related to skills needed to toilet independently. The following domains were assessed: Mobility, Activities of Daily Living (ADLs), Alertness, Resistance to Nursing Assistance, Incontinence, Cognition, and Perception. All items are measured on an ordinal level, except three items of the Cognition scale, which are measured on a nominal level. All scale scores range from 0 (no disabilities) to 100 (severely disabled). When combined, these items yield a 24-item nurse-administered observational instrument that measures the functional and cognitive status of demented patients. For a full description of the domains, items, and pertaining scores, see Table Aa .
Validity and reliability were evaluated. Internal consistency (construct validity) of the scales and test–retest and interrater reliability were measured. Finally, agreement between existing validated instruments was established (criterion validity). Regular nursing personnel conducted all ratings. All nurses were informed and instructed about the instrument. The nurse who knew the patient best or was the supervisor of the resident performed the ratings.
Data Analysis
First, minimum and maximum values, means, standard deviations, and the distribution of the values were established. Exploratory factor analysis was used to confirm the dimensions hypothesized, and the different dimensions were scaled. Scale scores for each domain were computed as the sum-scores of the items in each domain.
Reliability for each domain was examined using Cronbach's alpha. This is a measure for internal consistency, or the degree to which items within a domain are related to each other. Scores greater than 0.60 are considered to be satisfactory, scores >0.80 indicate good, and a Cronbach's alpha >0.90 indicates excellent internal consistency. For assessment of reproducibility (test–retest) and interrater reliability, Spearman-Brown correlation coefficients were computed. The nonparametric Spearman correlation coefficient is based on ranks of the data instead of the actual values. Criterion validity tests were performed by relating the scales of the instrument to different scales of the Dutch BOP rating scale. This is a well known and widely used instrument for nursing home residents with proven validity. The nonparametric Spearman-Brown correlation also was used for this test. Mobility was related to physical invalidity, Cognition to mental invalidity, ADL to helplessness, and Alertness to inactivity. For the item Resistance to Nursing Assistance there were no appropriate comparisons with an already validated scale. Likewise, Perception and Incontinence could not be compared to any corresponding scale of the BOP.
|
Results
|
|---|
The total population consisted of 115 residents admitted on four psychogeriatric wards. The majority of residents were women (78%) and the mean age was 81.5 years. All were included in the study. For 112 residents a first assessment was available; 3 were lost to follow-up. Test–retest reliability data on 101 residents were available. The second measurement (1 to 2 weeks later) was performed by the same nurse. For interrater reliability, data on 111 residents were available.
Reliability
The associations of items within a domain, as measured by means of Cronbach's alpha, were obtained on each of the scales (lowest: Perception 0.54; highest: Mobility 0.93). Table 1 presents the mean scores, SD, median, and Cronbach's alpha of the domains. The test–retest reliability was good to excellent; see Table 2 . Correlation coefficients ranged from 0.63 to 0.94. Interrater reliability for the scales Cognition, Incontinence, Mobility, and ADL was high (0.79 to 0.93), and moderate for Resistance to Nursing Assistance (0.51). Perception showed very low interrater reliability 0.33 (see Table 3 ).
Validity
Criterion validity
Three subscales showed high correlation coefficients (Cognition, ADL, and Mobility: 0.78, 0.76 and 0.75, respectively). Alertness correlated moderately with the inactivity subscale of the BOP with r = 0.59. The Incontinence and Perception scales could not be compared with a subscale as measured in the BOP, as presented in Table 4 .
Construct validity
In Table 5 , correlations on the scales are presented. The intercorrelations ranged from 0.02 to 0.72. Perception was not correlated with any other dimension. ADL and Cognition correlated very strongly with each other and with the Incontinence scale.
|
Discussion
|
|---|
The NHDI is a short but multidimensional measure for physical and cognitive functioning of psychogeriatric nursing home residents. In this study, the NHDI showed good internal consistency and test–retest and interrater reliability. The final version of the instrument comprises 6 dimensions and 24 items: Mobility, ADL, Cognition, Incontinence, Alertness, and one question on Resistance to Nursing Assistance. The NHDI takes only 5 minutes to complete if used by a nurse familiar with the patient. Compared to the BOP, an often used instrument in The Netherlands, the NHDI showed good convergent validity while being more efficient. The intercorrelation between the psychosocial and physical scales was high. This finding is substantiated by work in other disabled populations (
Brooks, Jordan, Divine, Smith, and Neelon 1990;
Ekkerink 1994). The relationship found between psychosocial and physical disabilities may be of particular relevance to those caring for nursing home residents, as both domains are important in this population.
The Perception scale did not work well. The intercorrelations of this scale with other scales were low. Accordingly, we decided to omit this scale from the final version of the NHDI. We believe, however, that in psychogeriatric residents it is important to know if the patient can see and hear enough to communicate. Sight and hearing appeared difficult for nurses to observe and interpret, and specific medical examination may be required. Further research in this area is needed.
Achieving a reliable assessment of functional status can be important in a nursing home setting for research and needs assessment. However, at times this is quite difficult. Nursing home residents with particular health problems present measurement challenges. Health status and residents' abilities to communicate may vary considerably over the assessment period. Accordingly, the level of care required will vary considerably over time. Moreover, some conditions in elders are inherently difficult to assess, particularly among persons with cognitive impairment and communication difficulties. These characteristics of nursing home residents have considerable implications for reliable measurement. The domains of the NHDI that measure key areas of functional status meet the standard for good to excellent reliability. These include: ADL Self-performance, Mobility, Cognition, and (In)continence. Disabilities in these areas cause persons to enter and remain in nursing homes. They are among the most common conditions encountered in nursing home residents and therefore may capture the key outcomes of interest (
Hawes et al. 1995).
A mixture of nominal and ordinal items was used in our instrument. This results in differential weighing of the items, which is not supported by additional analysis. This is a potential problem of instruments like ours. We believe, however, that the applicability and interpretation of the instrument still holds. For individuals, change over time can be monitored. Also, between-resident comparison is possible, especially per domain. The significance and meaning of the overall score attained using the instrument is less clear. Despite the obvious flaws, the instrument meets the original goal. It is short, comprehensive, and simple to use and can be used to classify residents according to their functional as well as cognitive status.
The NHDI may be useful for needs assessment during the intake of new nursing home residents and for periodical monitoring in the nursing home. We had anticipated that the NHDI would give information comparable to the MDS despite using only a fraction of the items and time needed. Additional research will be necessary to confirm this expectation.
Nevertheless, the NHDI appears to yield valuable information about the population of nursing homes. As workload is increasing in nursing homes, the NHDI could be useful to match workload and personnel resources. In addition, these data may have implications for reimbursement.
|
Acknowledgments
|
|---|
We thank Dr. Dennis Kodner and Dr. Corinne Kay for their helpful comments on the manuscript and the nursing staff of Care Center Albert van Koningsbruggen for their assistance and cooperation in collecting the data.
Received for publication May 1, 1999.
Accepted for publication December 7, 1999.
|
Appendix
|
|---|
|
References
|
|---|
- Bergner M., Bobbit R., Carter W., 1987. Health status measures: An overview and guide for selection. Annual Review of Public Health 8:191-210. [Medline]
- Brooks W. B., Jordan J. S., Divine G. W., Smith K. S., Neelon F. A., 1990. The impact of psychologic factors on measurement of functional status. Medical Care 28:793-804. [Medline]
- Boyle M. H., Torrance G. W., 1984. Developing multiattribute health indexes. Medical Care 22:1045-1057. [Medline]
- Casten R., Lawton M. P., Parmelee P. A., Kleban M. H., 1998. Psychometric characteristics of the Minimum Data Set I: Confirmatory factor analysis, under review. Journal of the American Geriatrics Society 46:726-735. [Medline]
- Dautzenberg P. L. J., Schmand B., Vriens M. T. S., Deelman B. G., Hooijer C., 1991[Validation study of the Cognitive Screening test and the Mini Mental State in an elderly hospitalized population]. Validiteitsonderzoek van de Cognitieve Screening test en de Mini-Mental State Examination bij een oudere ziekenhuispopulatie. Nedederlands Tijdschrift voor Geneeskunde 135:850-855.
- De Graaf A., Deelman B. G., 1992. Cognitive Screening Test Swets en Zeitlinger, Handleiding, Lisse.
- De Jonghe J. F. M., de Kat M. G., de Reus R., 1994[The validity of the Clinical Behavioral Assessment Scale]. De validiteit van de Gedragsobservatieschaal voor de Intramurale Psychogeriatrie (GIP): een vergelijking met de BOP en NOSIE. Tijdschrift voor Gerontologie en Geriatrie 25:110-116. [Medline]
- De Jonghe J. F. M., Calis P. J. A., Boom-Poels P. G. M., 1996[Behavioral dimensions in elderly patients: Factoranalysis of the GIP]. Gedragsdimensies van oudere patiënten: Factorstructuur van de Gedrags- observatieschaal voor de Intramurale Psychogeriatrie (GIP). Tijdschrift voor Gerontologie en Geriatrie 27:159-164. [Medline]
- Diesveldt H. F. A., 1981[The BOP 10 years]. De BOP 10 jaar. Gerontologie 12:139-147. [Medline]
- Ekkerink J. L. P. E., 1994[The course of dementia after admission to a Dutch nursing home, dissertation]. Het beloop van dementie bij ouderen in het verpleeghuis Katholieke Universiteit Nijmegen, The Netherlands.
- Feher E. P., Mahurin R. K., Doody R. S., Cooke N., Sims J., Pirozzolo F. J., 1992. Establishing the limits of the Mini-Mental State. Archives of Neurology 49:87-92. [Abstract/Free Full Text]
- Folstein M. F., Folstein S. E., McHugh P. R., 1975. Mini Mental State: A practical method for grading the cognitive status of residents for the clinician. Journal of Psychiatry Research 12:189-198.
- Gerety M. B., Cornell J. E., Mulrow C. D., Tuley M., Hazuda H. P., Lichtenstein M., Kanten D. N., Aguilar C., Kadri A. A., Rosenberg J., 1994. The Sickness Impact Profile for nursing homes (SIP-NH). Journal of Gerontology: Medical Sciences 49:M2-M8.
- Hartmaier S. L., Sloane P. D., Guess H. A., Koch G. G., 1994. The MDS Cognition Scale: A valid instrument for identifying and staging nursing home residents with dementia using the Minimum Data Set. Journal of American Geriatrics Society 42:1173-1179.
- Hartmaier S. L., Sloane P. D., Guess H. A., Koch G. G., Michel C. M., Phillips C. D., 1995. Validation of the Minimum Data Set Cognitive Performance Scale: Agreement with the Mini-Mental State Examination. Journal of Gerontology: Medical Sciences 50A:M128-M133. [Abstract]
- Hawes C., Morris N. J., Phillips C. D., Mor V., Fries B. E., Nonemaker S., 1995. Reliability estimates for the Minimum Data Set for nursing home assessment and care screening (MDS). The Gerontologist 35:172-178. [Abstract]
- Hawes C., Phillips C. D., Mor V., Fries B. E., Morris J. N., 1992. MDS-data should be used for research. The Gerontologist 32:563-564.
- Institute of Informatics in Health and Welfare. (1995). SIG, jaarboek, 1995. Utrecht, The Netherlands: Author.
- Katz S., Ford A. B., Moskowitz R. W., Jackson B. A., Jaffe M. W., Cleveland M. A., 1963. Studies of illness in the aged. The index of ADL: A standardized measure of biological and psychosocial function. Journal of the American Medical Association 185:914-919.
- Lawton M. P., Casten R., Parmelee P. R., van Haitsma K., Corn J., Kleban M. H., 1998. Psychometric characteristics of the Minimum Data Set II: Validity. Journal of American Geriatrics Society 46:736-744.
- Mahoney F. I., Barthel D. W., 1965. Functional evaluation: The Barthel Index. A simple index of independence useful in scoring improvement in the rehabilitation of the chronically ill. Maryland Medical Journal 14:61-65.
- Melchior M. E. W., Frederiks C. M. A., 1992[The BOP and the GIP in psychogeriatric patients]. De BOP en de GIP in de psychogeriatrie. Verpleegkunde 2:97-105.
- Reisberg B., Ferris S. H., de Leon M. J., Crook T., 1988. Global Deterioration Scale (GDS). Psychopharmacology Bulletin 24:661-663. [Medline]
- Reisberg B., 1988. Functional assessment staging. Psychopharmacology Bulletin 24:653-659. [Medline]
- Teresi J. A., Holmes D., 1992. Should MDS-data be used for research?. The Gerontologist 32:148-151. [Medline]
- Tombaugh T. N., McIntyre N. J., 1992. The Mini-Mental State Examination: A comprehensive review. Journal of American Geriatrics Society 40:922-935.
- Van der Kam P., Mol F., Wimmers M. F. H. G., 1971[Elderly Residents Rating Scale]. Beoordelingsschaal voor Oudere Patiënten Van Loghum Slaterus, Deventer.
- Verstraten P. F. J., van Eekelen C. W. J. M., 1987[Clinical Behavioral Assessment Scale]. Handleiding voor de GIP: Gedragsobservatieschaal voor de Intramurale Psychogeriatrie Van Loghum Slaterus, Deventer.
- Ware J. E., 1987. Standards for validating health measures: Definition and content. Journal of Chronic Disease 40:473-480. [Medline]
This article has been cited by other articles:
|
|
|
|
 
M W. Post, J M. Visser-Meily, and L S. Gispen
Measuring nursing needs of stroke patients in clinical rehabilitation: a comparison of validity and sensitivity to change between the Northwick Park Dependency Score and the Barthel Index
Clinical Rehabilitation,
February 1, 2002;
16(2):
182 - 189.
[Abstract]
[PDF]
|
|
|
TOP
Abstract
Methods
