| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
RESEARCH ARTICLE |
1 University of Minnesota, Minneapolis.
2 University of Pittsburgh, Pennsylvania.
3 St. Cloud State University, Minnesota.
Address correspondence to Robert L. Kane, MD, Division of Health Services Research and Policy, University of Minnesota School of Public Health, 420 Delaware St. SE, D351 Mayo (MMC 197), Minneapolis, MN 55455. E-mail: kanex001{at}umn.edu
 |
Abstract |
|---|
 TOP Abstract BackgroundMethodsResultsDiscussionReferences |
|---|
Methods. We compared QOL domain scores for nursing home residents and 1,326 staff proxies and 989 family proxies at the individual and facility level using means, Pearson correlation statistics, and intraclass correlations. Regression models adjusted for residents' age, gender, length of stay, ability to perform activities of daily living, and cognition.
Results. For each domain in more than half the cases, proxy means were within 1 SD of the resident means. Resident and family proxy individual reports for selected domains were correlated at 0.14 to 0.46 (all p <.000). Resident and staff proxy individual reports were correlated at 0.13 to 0.37 (all p <.000). Correlation of mean levels by facility for staff proxies was 0.26 to 0.64 (generally p <.05) and for family proxies 0.13 to 0.61 (p <.01 except for one domain).
Discussion. Although staff and family proxy domain scores are significantly correlated with resident scores, the level of correlation suggests they cannot simply be substituted for resident reports of QOL. Determining how proxy reports can be used for residents who cannot be interviewed at all remains an unresolved challenge.
Quality of life (QOL) is an intensely personal and primarily subjective concept, rooted in the experience of the person living the life. In contrast, studies of health outcomes tend to use only brief and rather concrete outcomes (such as functionality) as obeisance to measuring QOL. However, when examining the adequacy of nursing facilities (NFs), which have been so long dreaded and shunned by prospective residents, attention to a broader range of QOL domains and examination of resident's subjective experience are required both for continuous quality improvement and for NF regulations (Noelker & Harel, 2001
; Wunderlich & Kohler, 2001). Arguably, QOL outcomes are as important as any clinical outcomes when NFs are held to account, yet using QOL measures in NF settings presents additional difficulties.
In the NF context, one important challenge is collecting resident-reported data on QOL. Some residents will be unable to report directly on their own QOL because of cognitive or physical impairments. Other residents can be interviewed about their lives, but doing so is time consuming and may require an expensive outside assessor because of possible intimidating effects or other biases when NF staff ask the questions. In long-term care, it has been a widespread practice to mingle data collected directly from older people getting care with data collected from family members and other proxies, using self-report and other-report somewhat interchangeably according to convenience (Albert et al., 1996; Rabins, Kasper, Kleinman, Black, & Patrick, 1999; Rubenstein, Schairer, Wieland, & Kane, 1984) or relying solely on staff proxy information as is done for the mandatory minimum data set (MDS) (Morris et al., 1990). In this article, we use a large data set with two sources of proxy data (families and direct care staff) for a large sample of residents who also reported on their own QOL to examine the congruence of resident self-report and other sources of feedback on resident QOL. The results shed light on what the implications of using proxy data may be, whether proxies are used exclusively (as is the case with the current MDS) or to assess those who cannot report on their own.
|
BACKGROUND |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
A systematic literature review of 24 clinical studies from 1990 to 1999 showed variation in agreement between proxies and older subjects according to the nature of the inquiry (Neumann, Araki, & Gutterman, 2000). The authors concluded that comparability tended to be good regarding levels of functioning (although proxies tended to identify more impairment than older people) and that agreement was also good with regard to assessments of overall health, chronic physical conditions, and physical symptoms. Agreement on what the older person preferred for health states, the older person's experience on depressive symptoms, and psychosocial well-being was low to moderate.
Numerous studies have compared measures of functioning, mood, preference, and QOL for older people with specific conditions and in specific circumstances to proxy assessors, including for cancer patients (Curtis & Fernsler, 1989; Moinpour, Lyons, Schmidt, Chansky, & Patchell, 2000; Sigurdardottir, Brandberg, & Sullivan, 1996; Sneeuw et al., 1997; Sneeuw, Aaronson, Sprangers, Detmar, Wever, & Schornagel, 1999), epilepsy (Hays et al., 1995), stroke (Dorman, Waddell, Slattery, Dennis, & Sandercock, 1997), coronary surgery (Page, Verhoef, & Emes, 1995), intensive care units (Capuzzo, Grasselli, Carrer, Gritti, & Alvisi, 2000), elderly outpatients in general (Epstein, Hall, Tognetti, Son, & Conant, 1989), persons with developmental disability (Stancliffe, 1999), and even persons with Alzheimer's disease (Hickey & Bourgeois, 2000; Novella et al., 2001). A meta-analysis comparing physician and patient judgments about QOL showed substantial discrepancies (Janse, Gemke, Uiterwaal, van der Tweel, Kimpen, & Sinnema, 2004). QOL outcome measures used include disease-specific measures and general health-related QOL measures. The studies varied greatly in their criteria for defining proxies. Some required a proxy to be "someone who knows the study subject well" in a nonprofessional capacity, which could be a family member, next of kin, friend, or informal caregiver; some asked the older person to nominate the proxy; a few accepted caregiving staff; and some used either a family member or a provider, depending on availability. Four studies included both a family member and a health care provider. From these studies, we find varying amounts of agreement between subject and proxy with agreement most pronounced when the items were concrete and verifiable. Even when general agreement was high for sample means, striking discordance between pairs was found in specific cases.
Although proxies are frequently used, their validity is rarely discussed in depth. For example, in the National Hospice Study, a substantial amount of the reports on pain control came from proxies because the patients were either too sick to respond or already dead; however, no evidence was presented to show that a third party could describe another person's pain (Greer, Mor, Morris, Sherwood, Kidder, & Birnbaum, 1986). Typically, most literature portrays the study subject as a gold standard, considering proxy responses that differed from the subject's to be wrong. But some commentators also discuss the possibility that the person's own self-reports could also be wrong and lament the lack of a gold standard or proposed that some other source be used as a criterion, such as a clinical assessment or administrative records. In general, there has been a tendency for more investigators in long-term care to attempt to gather direct perceptions from residents, including those with substantial cognitive impairment (Albert et al., 1999; Brod, Stewart, Sands, & Walton, 1999; Logsdon, Gibbons, McCurry, & Teri, 1999), and some investigators have experimented with different ways of framing questions to maximize the likelihood of getting responses directly in the residents' own voices (Simmons, Babineau, Garcia, & Schnelle, 2002; Simmons & Schnelle, 1999).
We embarked on this study at a time when interest in measuring QOL of nursing home residents and frail older people in other residential settings was high. States have been interested in report cards comparing the relative ability of various licensed care providers to deliver a good QOL to residents (Zimmerman & Bowers, 2000). The National Quality Forum identified QOL and resident satisfaction as important but presently omitted indicators of quality in nursing homes (National Quality Forum, 2004). Given how much easier data collection from family members would be, a debate is ongoing about whether family members could be solicited either instead of residents or when residents cannot easily be polled.
Quality-of-Life Measures in Nursing Homes
QOL is widely agreed to be a multidimensional concept, though theorists vary in the range of domains they attempt to tap (Abeles, Gift, & Ory, 1994; Birren, Lubben, Rowe, & Deutchman, 1991; Lawton, 1991; Stewart & King, 1994). Between 1998 and 2003, the Centers for Medicare and Medicaid Services (CMS) funded the iterative development of self-report measures of QOL tailored to relevant domains for NF residents. Because relocation to an NF is such a drastically life-altering change affecting all aspects of everyday life activities and relationships (Agich, 1993; Kane & Caplan, 1990; Tobin, 1999), QOL measures needed to be considered broadly and to embrace psychological and social outcomes. Based on previous work, literature review, and focus groups, QOL measures developed were designed to tap 11 QOL domains identified as important: comfort, autonomy, privacy, dignity, meaningful activity, relationships, food enjoyment, security, functional competence, spiritual well-being, and individuality (Kane, 2001, 2003). The developed scales, in their short form, contained three to six items per scale. As described elsewhere, these scales had good alpha reliabilities, except for the individuality domain, which was therefore dropped from the analyses (Kane et al., 2003). The remaining 10 domains were shown through confirmatory factor analysis to be independent constructs related to a common latent QOL construct. The three-item enjoyment scale in its final iteration related only to enjoyment of food, other items initially posited as "enjoyment" items having migrated to other domains or failed to load with any. The QOL measures also correlated well with a scale adapted from Brod and Stewart that was used to assess the resident's affective or mood state (Brod et al., 1999).
|
METHODS |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
Staff
At the same time as the field work with residents was done, we interviewed a NF direct care staff member for each resident. To select the staff interviewee, who was usually a certified nursing assistant, we identified staff members on the day or evening shift likely to know each resident well. If a primary nursing system was in place, we interviewed someone primarily assigned to the resident. To qualify to be interviewed, the staff member must have cared for that resident for at least 2 weeks. The same staff member was permitted to report on up to five residents. Staff were interviewed in person and given a $10 gift certificate for each interview.
Family
With the help of the NF social worker or at times the unit coordinators, we undertook a procedure to identify up to three family members currently involved with the resident in rank order based on the amount of contact with the resident. We mailed an explanatory letter and questionnaire to each identified family member. If no family member replied by mail, we followed up by phone. Only 13% of the family proxies were interviewed by phone. Responses from the family respondent with the highest contact rating were used in these analyses.
Variables
Resident data used for this study include the QOL subscales for which we had parallel information from staff and/or family and the emotion rating questions. As indicated above, domains were measured by three to six items. Almost all items used the response set "often, sometimes, rarely, never," which were in turn scored as 4, 3, 2, and 1, respectively; reverse coding was done so a positive response always meant a better QOL. Respondents who had difficulty with the complexity of that response set were offered the response options "mostly yes" or "mostly no." After empirically testing a variety of scoring methods, we interpolated the binary responses into the Likert responses, using a score of 3.8 for a binary positive response and 1.5 for a binary negative response. For comparability, all QOL scales were summed and divided by the number of items, so that all subscales ranged from 4 to 1, with 4 representing the better QOL.
Staff and family questionnaires were developed to parallel the questions posed to residents; in all cases, the staff or family respondents were instructed to answer the questions as they thought the residents would answer them, basing responses on their best guess as to the resident's appraisal rather than their own opinion. Because staff and family questionnaires were fielded along with the resident developmental questionnaire, the QOL domains had not yet been established. In the family and staff questionnaires, we omitted asking the respondents to judge the spiritual well-being items because we thought they would be unlikely to be known by a third party. We also omitted the dignity and relationship items from the staff interview because the items could call for substantial self-criticism from staff (e.g., Do staff treat Mrs. Jones roughly while giving her care? Do staff treat residents with respect?). Thus, we obtained nine matched domain scores for resident–family comparisons (comfort, functional competence, privacy, dignity, meaningful activity, enjoyment, security, relationships, and autonomy) and seven matched domains for resident–staff comparisons (omitting dignity relationships). We collected data on residents' current emotions from family and staff respondents.
Resident ADL functioning, age, gender, cognitive functioning, and admission date were abstracted from the MDS; admission date was used to create a length-of-stay variable.
Statistical Analysis
We created comparable scales based on matching items for each of the proxy respondent groups and the residents. The scales for both staff and family closely approximate the resident scales in length, being either identical, one question shorter, or (in two instances for staff and one for family) two questions shorter (but not falling below three items). We compared the mean values across groups using analysis of variance for independent samples and compared matched samples of each proxy and the corresponding residents with t tests. To test the correlations, we used Pearson correlation statistics, as well as calculating intraclass correlations (ICCs).
A regression model was used to adjust proxy responses to account for case mix differences. The independent variables describing residents used in the model were derived from MDS data. The ADL score was created by summing the number of dependent areas. The cognition score was modeled after the Cognitive Performance Score (CPS) but omitted the item on feeding (Morris et al., 1994). To specifically test the effect of cognition on agreement, we calculated the mean agreement for each of the six cognitive groups.
To test the feasibility of using facility-level as well as individual scores based on different respondents, we calculated scores in each domain for each group of respondents and compared them using the same measures of agreement described above.
|
RESULTS |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
.1. When the matched paired results were compared, the staff reports significantly (p <.01) differed from those of the residents on all domains, but the pattern of difference was not consistent. In six instances, the staff mean was higher. The staff mean was within 1 SD of the resident mean for 55% of all cases and within 60% of the cases for all but three items (meaningful activity, security, and autonomy). Matched family proxy means were significantly different from resident means for all domains except food enjoyment and relationships, but again the patterns differed. Family means were lower for four domains (comfort, functional capacity, meaningful activity, and security) but higher for three (privacy, dignity, and autonomy). The family mean was within 1 SD of the resident mean for 55% of all cases and within 60% of the cases for all but two items (comfort and relationships).
|
|
.3. Neither the Kendall nor the Spearman coefficients perform better (not shown). All but one of the ICC values is >.2, but only one is >.4. Another test of agreement used the measures of emotions. As shown in Table 3, the correlations between staff and residents were generally low; none of the Pearson coefficients or the ICC values reached.2. The correlations with family proxies were somewhat better. Three Pearson coefficients and two ICC values were >.2.
|
|
|
|
.4.
|
|
DISCUSSION |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
Confidence in proxies depends in large part on the nature of the data being sought and the way it will be used. Proxies can provide information about factual events, such as hospital admissions or even falls, where they have the opportunity to witness the event or are likely to know of its occurrence. However, relying on proxies to provide information about another person's emotions or subjective judgments on QOL raises much greater concerns.
Although the choice of correlation statistic should be based on the nature of the data being analyzed, in this study we found little difference in the results regardless of the statistic used. This low level of agreement urges caution in using either family or direct staff as proxies for resident reports when we have any possibility of getting feedback from residents themselves.
Our results were similar to those reported in other settings and with different populations. Interpretation of correlations is in itself subjective. The size of the correlation may be more important than statistical significance, which may reflect sample size. When one thinks of these results in terms of variance explained, a correlation of.3 means explaining < 10% of the variance in the resident's report; even a correlation of.4 explains only 16% of the variance. We thus conclude that one should use proxy reports of nursing home residents in areas such as QOL very cautiously. The generally poor performance of proxies who are customarily used as information sources should raise some concerns about limitations in the MDS mandated by CMS, which relies almost exclusively on observations and inferences from nursing home staff even for measures of pain (Fries, Simon, Morris, Flodstrom, & Bookstein, 2001).
This study does not address the fundamental question of how to definitively assess NF residents' QOL. We proceed from the belief that a resident self-report is the gold standard for evaluating the accuracy of the proxy informant's reports about QOL. We argue that if an individual is able to provide consistent and reliable responses to a structured query, the validity of the responses cannot be readily challenged. That is, concordance with what the individual reports regarding his or her own QOL is the criterion, rather than whether proxies can accurately report on the true mental state of the individual. By definition, that internal assessment is best known through the verbal expression of the individual. Clearly, there is room for debate about whether individuals with low levels of cognition are capable of assessing their own QOL. In our data, however, the congruence between residents and proxies was not much better when we looked only at residents with good cognitive functioning.
Some observers might argue that QOL should not be ascertained solely from NH residents who may have become so acculturated to their environment as to lose perspective. However, we hold that that this acculturation, which some might label as coping, is a critical element in determining QOL. Others support this need for a resident-centered approach and the incorporation of the resident voice or, more generally, the consumer voice in the appraisal of long-term care (Appelbaum, Straker, & Geron, 2000; Rubinstein, 2000; Schnelle, 2003).
Our findings have important implications for the immediate task of deciding how to assess QOL in nursing home residents and in the larger context of how much one can rely on proxies in general. With regard to the former, if we reject using proxies, we are left with the dilemma of how to address the QOL for those residents who cannot speak for themselves. Two approaches are possible: extrapolating from the views of residents who can respond, or developing a method to extrapolate QOL based on observations. Observations, even structured observations of residents' affect (Lawton, Van Haistma, & Klapper, 1966), are simply a variation on the proxy theme. Extrapolating from the reports of those residents who can respond entails other risks that the experiences of those who can communicate verbally are different from those of the severely cognitively impaired. In a regulatory environment, there is the risk that providers might focus on those who can report their QOL and neglect those who cannot. An alternative is to use a proxy approach, acknowledging and trying to minimize its flaws. The fact remains that when trying to get a sense of the residents' QOL for many possible uses, including reporting to potential consumers, the question of how to incorporate the experience of those unable to self-report, frequently owing to cognitive impairment, remains a perplexing problem.
For some purposes, the problem may be less critical. If one is simply trying to develop an overall score at the facility level for use in a quality-reporting system or benchmarking, for example, aggregated data might suffice. We have used the residents' reports of QOL to show differences among NFs (Kane et al., 2004). Aggregated reports, based on aggregated mean values, have much better concordance. Here the individual pairs need not agree, as long as one high score in one group offsets a similar score in the other.
Some limitations of this study must be acknowledged. Because of the overall design of the parent study, the questions used for the residents and the proxies differed slightly. The response metric for the residents was designed to maximize participation. A statistically determined cross-walk was developed (Kane et al., 2003) and used here.
Further research is warranted. We intend to do additional work with both the family and the staff reports to see if other techniques might be helpful for approximating QOL for residents unable to provide self-reports and to see if any family or staff characteristics predict better congruence with residents. Also of interest is whether certain items or domains garner greater proxy congruence than others, perhaps leading us to a theory of how an individual's reality can be understood by others. This general topic may also lend itself to qualitative work to help explain the discrepancies in quantitative results and to assess the characteristics of proxies or their experiences vis-à-vis the person in whose stead they reply that might make them more suited to the task.
Some clinical implications can also be cautiously drawn. The fact that staff, on whom residents are so dependent, respond to questions about QOL differently from the residents under their care suggests a difference in perception that underscores the need for resident-derived data on which to base efforts to improve resident QOL. Perhaps if QOL were included in the MDS as a basis for NF accountability, staff might be more motivated to discuss aspects of residents' lives and emotions and to listen carefully to the answers. Staff might then better anticipate the needs and preferences of residents and provide them more sensitive care.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Received for publication August 10, 2004. Accepted for publication April 19, 2005.
|
References |
|---|
|
TOPAbstractBackgroundMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  C. E. Bishop, D. B. Weinberg, W. Leutz, A. Dossa, S. G. Pfefferle, and R. M. Zincavage Nursing Assistants' Job Commitment: Effect of Nursing Home Organizational Factors and Impact on Resident Well-Being Gerontologist, July 1, 2008; 48(suppl_1): 36 - 45. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. N. Rahman and J. F. Schnelle The Nursing Home Culture-Change Movement: Recent Past, Present, and Future Directions for Research Gerontologist, April 1, 2008; 48(2): 142 - 148. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. E. Alkema and D. E. Alley Gerontology's Future: An Integrative Model for Disciplinary Advancement. Gerontologist, October 1, 2006; 46(5): 574 - 582. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
