| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||
RESEARCH ARTICLE |
1 Department of Medicine, 2 Center for Health Policy Research, University of California, Irvine.
3 Agency for Healthcare Research and Quality, Washington, D.C.
4 Fox School of Business and Management, Temple University, Philadelphia, Pennsylvania.
5 LaFollette School of Public Affairs, University of Wisconsin–Madison.
6 Department of Community and Preventive Medicine, University of Rochester, Rochester, New York.
Address correspondence to Dana Mukamel, PhD, Department of Medicine, Center for Health Policy Research, University of California, Irvine, 111 Academy, Suite 220, Irvine, CA 92697-5800. E-mail: dmukamel{at}uci.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Methods. We conducted a survey of a random national sample of 1,502 nursing home administrators; 724 responded. We analyzed frequency of responses to questions regarding views of the quality measures and actions taken.
Results. A model of nursing homes' behavior predicted that the report card would provide an incentive for facilities to improve quality. A majority of facilities (69%) reported reviewing their quality scores regularly, and many have taken specific actions to improve quality. Homes with poor quality scores were more likely to take actions following the publication of the report card.
Discussion. These findings suggest that the Nursing Home Compare report card has the potential to positively affect nursing home quality.
The quality of the services provided to nursing home residents has come under scrutiny, and more often than not has been judged to be inadequate (Mendelson, 1974
; Vladeck, 1980; Wheeler, 2003; Winzelberg, 2003). Reforms aimed at improving quality began in the mid-1980s with the Institute of Medicine (1986) report and the subsequent implementation of its recommendations in the Nursing Home Reform Amendment to the Omnibus Budget Reconciliation Act of 1987 (Harrington & Carrillo, 1999). Despite these efforts, which spanned most of the 1990s and which resulted in the implementation of a nationally mandated system for quality monitoring, quality problems in nursing homes persist. The U.S. General Accounting Office (1999) found significant deficiencies in one fourth of nursing homes and questioned the ability of state surveyors to detect serious care problems (U.S. General Accounting Office, 1998). Harrington and Carrillo reported that although the number of deficiency citations for nursing homes declined, many quality problems persisted over the period 1991 to 1997. The medical literature has continued to identify many clinical problems in nursing homes, including malnutrition (Abbasi & Rudman, 1994; Crogan, Shultz, Adams, & Massey, 2001), dehydration (Kayser-Jones, Schell, Porter, Barbaccia, & Shaw, 1999), medication errors (Gurwitz et al., 2000), and pressure ulcers (Spector & Fortinsky, 1998), and there is evidence that the quality of hotel services (room accommodations, meals, etc.) may be declining as well. A study of nursing homes in New York State showed that expenditures per day on hotel services, a likely indicator of quality, have been declining (Mukamel, Bajorska, & Spector, 2005).
Recognizing the continued concerns with poor quality in nursing homes, the Centers for Medicare and Medicaid Services (CMS) launched in November 2002 a quality initiative that included, for the first time, publication of a quality report card with measures of clinical quality for all nursing homes in the country (CMS, 2005). This report card follows a decade of quality report cards for hospitals, physicians, and managed care organizations. The evidence about the success of report cards in influencing referrals and their impact on quality is mixed (Mukamel & Mushlin, 2001; Mukamel, Weimer, Zwanziger, Huang-Gorthy, & Mushlin, 2004/2005; Mukamel, Weimer, Zwanziger, & Mushlin, 2002; Schauffler & Mordavsky, 2001; Werner & Asch, 2005). Moreover, the impact of report cards on nursing home care may be different than their impact in other sectors of the health care system (Mukamel & Spector, 2003). Nursing homes serve an older and more frail population (Harrington, O'Meara, Kitchener, Simon, & Schnelle, 2003) that has difficulty in assessing quality on its own, markets are often not competitive (Mukamel & Spector, 2002), and Medicaid is a dominant payer (Mukamel & Spector, 2003). In this article we examine how nursing homes have responded to the publication of the Nursing Home Compare report card.
A Model of Nursing Homes' Response to the Publication of Quality Report Cards
The rationale for public dissemination of quality report cards is based on the expectation that provision of such data to consumers, patients, and their agents (families, social workers, physicians) will create the appropriate demand incentives for high-quality care (Spector, Selden, & Cohen, 1998). Because nursing homes provide a complex service, which includes room and board, personal care, and medical care, the quality of care they provide is multifaceted. Therefore, choosing a nursing home requires assessing quality on many dimensions. Some of these, especially quality related to room and board services, are easier for consumers to evaluate through personal inspection or reports of the individual experiences of others. For ease of exposition, we refer to these dimensions of quality as observable quality. Clinical quality (e.g., a higher than appropriate rate of pressure sores), in contrast, is much more difficult for individuals to evaluate. The complexities of medical care and the dependence of health outcomes on factors other than clinical care itself (Iezzoni, 1993; Spector & Mukamel, 1998) limit people's ability to judge it. Extensive data, both in terms of sample sizes (i.e., the number of residents in a facility) and in terms of scope (i.e., information about individual risks that influence health outcomes), as well as sophisticated statistical analyses are required in order to infer quality of personal and medical care. Therefore, we refer to these dimensions as unobservable quality.
Because consumers are unable to differentiate facilities on unobservable dimensions of quality, their demand and willingness to pay is the same for nursing homes of high and low unobservable quality. The resulting failure of the market to reward high-quality personal and clinical care leads to disincentives for nursing homes to invest in quality in these areas. Quality report cards are designed to address this market failure by revealing unobservable quality to consumers.
One can model the response of nursing homes to the report card in a neoclassical economic framework by considering the objective that facilities maximize and the constraints they face. Consider first for-profit facilities (which account for more than 60% of nursing homes). Let us assume that these facilities choose how many patients to serve and at what quality levels (both observed and unobserved). Their choice will be such that they maximize their profits, given the following constraints: (a) the costs of producing each aspect of quality, which depend on the technology and the cost of inputs; (b) the demand they are facing, which reflects consumers' willingness to pay for each aspect of quality; and (c) state and federal regulations, which set a minimum standard for quality. The optimal choice for each quality dimension will be such that the marginal cost of producing it equals the marginal revenue that it can generate. Marginal revenues, in turn, will depend on both the private pay price and the number of Medicare and Medicaid patients that the facility can attract if it offers higher quality in that dimension. Because consumers do not respond to unobservable quality, facilities are not rewarded for providing higher quality in these dimensions, and have, therefore, no incentive to provide care at quality above the minimum set by the regulations. Since report cards have become available, however, and to the extent that the quality measures (QMs) reported in them capture quality of care, many dimensions of quality have become observable and, therefore, will influence consumer demand. Facilities now face a positive incentive, in the form of increasing revenue, to offer higher levels of quality.
One can use a similar model to describe the behavior of nonprofit facilities, with one modification: One can typically not assume that nonprofit nursing homes will maximize profits. (For a discussion of the objective function of nonprofits, see, for example, Weisbrod, 1988.) These facilities are, however, expected to meet a break-even constraint. Therefore, unlike the for-profit facilities that make choices based on marginal costs and revenues, the nonprofits choose the level of quality they offer such that average costs equal average revenues. Similar to the for-profits, these choices depend on demand and consumers' willingness to pay for quality. Thus, one can expect the incentives to offer higher quality levels provided by report cards that reveal unobservable quality to influence nonprofit nursing homes as well.
We should note one important caveat. The incentive to improve quality derives from the transformation of several dimensions of quality from unobserved to observed. Although adopting actions that improve quality is the desired impact of the publication of the report cards (we refer to such actions as functional strategies), nursing homes may adopt other actions that improve their scores on the report card, thus creating the desired demand response, but without actually improving quality (we refer to such actions as dysfunctional strategies). Dysfunctional strategies include "cream skimming" and "teaching to the test" (p. 137, Gormley & Weimer, 1999). The first refers to changing the mix of residents in the facility, possibly through admission of individuals who are less debilitated, such that the quality scores for the facility will improve without any actual increase in quality. Nursing homes are particularly likely to adopt this strategy if they are concerned that the reported QMs do not adequately adjust for resident risk. The second dysfunctional strategy arises because report cards are not comprehensive and do not include measures on all aspects of care. For example, the first five publications of the nursing home report card did not include measures related to urinary incontinence, even though it is a major aspect of quality in nursing homes. Thus, facilities can improve their publicly reported scores by shifting resources that contribute to quality in areas that are not included in the report card to areas that are included. This will lead to an increase in quality in reported areas but can possibly decrease quality in areas that are not reported on.
This behavioral model, which leads one to expect that nursing homes will improve quality but also raises the specter of potentially dysfunctional responses, motivated the study we present here, in which we examined the response of nursing homes to the initial publication of the Nursing Home Compare report card and the specific actions that they undertook. The empirical evidence to date is ambiguous. A survey of nursing home administrators in four states (Castle, 2005) found that 33% reported that they had used report cards in some fashion, and 51% said they were planning to use them in the future. Other studies have reported that some, but not all, QMs have improved (Dembner & Dedman, 2004; Zinn, Spector, Hsieh, & Mukamel, 2005). There is currently no information about adoption of dysfunctional strategies. In addition, researchers have not yet reported investigations of what specific actions, if any, nursing homes have undertaken in response to poor QMs.
Description of the Nursing Home Compare Report Card
CMS widely publicized the initial publication of the clinical QMs in the nursing home report card in November of 2002 through full-page advertisements in all of the major newspapers in the country. The CMS report card is Web based (http://www.medicare.gov/NHCompare/) and includes information about all facilities that are Medicare or Medicaid certified. It provides general information about the facility (e.g., ownership), number of deficiency citations, and ratio of staff hours to resident days. Initially the report card scored the performance of each nursing home on 10 clinical QMs. The set of measures was subsequently revised and expanded to 19. The QMs are updated every 3 months and published with a 6-month lag. The data are reviewed by state inspectors but are not audited.
The initial report card included six chronic care (long stay) measures, based on any resident with a full or quarterly Minimum Data Set assessment in the target quarter beginning April 2002. (Facilities with fewer than 30 eligible cases for a QM had a missing value for that QM.) These measures were the percentage of residents (a) whose ability to perform one of the following activities of daily living had deteriorated: feeding oneself, moving from one chair to another, changing positions while in bed, going to the bathroom alone; (b) with new infections; (c) with moderate pain daily, or excruciating pain at any frequency; (d) with pressure sores; (e) with pressure sores adjusted by the facility admission profile; (f) in physical restraints daily.
In addition, the report card included four postacute measures. These were based on any resident with a 14-day Prospective Payment System Minimum Data Set assessment in two consecutive target quarters. (Facilities with fewer than 20 eligible cases for a QM had a missing value for that QM.) These measures were the percentage of postacute residents (a) with delirium; (b) with delirium adjusted by the facility admission profile; (c) whose walking ability had improved or been maintained since admission; and (d) with moderate pain daily, or excruciating pain at any frequency.
|
METHODS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
To determine whether and how nursing homes responded to the publication of QMs, survey items addressed opinions regarding the perceived validity of the measures, importance and impact of public reporting on nursing home operations, performance benchmarking, and specific actions that the facility had undertaken in response to the publication. The survey specifically asked for actions taken in response to the first wave of QMs. The specific actions included in the survey were designed to capture the gamut of activities available to nursing homes. To ensure that we capture all possible actions, we asked administrators to write in additional actions if the survey did not explicitly include them. We pretested all survey questions with a small group of administrators in New York and California.
We sent the survey to chief administrators in a national 10% random sample (1,502 nursing homes) of all facilities that had appeared in the first publication of the Nursing Home Compare report card and that had at least one QM. Of the 1,502 nursing homes surveyed, 724 responded, yielding a response rate of 48%. This sample size means that the response frequencies in the sample estimate population frequencies with an accuracy of ± 1.8% or better. We conducted the survey in May and June 2004.
Analyses
To examine the potential for response bias we compared the characteristics of nursing homes that responded to those that did not. We tested differences using t tests for continuous variables and chi-square tests for dichotomous variables.
For each survey item, we examined the distribution of responses by calculating the percentage of facilities that chose each response category. For example, we report the percentage of facilities that reviewed the QMs and the percentage that did not.
The analysis of the specific actions taken reports the percentage of facilities that indicated that they undertook an action, as well as the average number of poor quality QMs for those that reported taking the action and those that reported not taking the action. We defined a poor quality QM as one at the bottom 20% of the state distribution. We also examined the data when poor quality outliers were defined by the 10th and the 40th percentiles. We found the strongest associations between actions and poor scores when we used the 20th percentile, which are the data we present here. Based on this definition, 77% of facilities had at least 1 of the 10 QMs designated as low quality.
In addition to examining each reported action separately, we also performed an analysis pooling data across all actions. We estimated a logistic regression model predicting whether a facility had undertaken any action as a function of the number of low QMs it had, and a Poisson regression predicting the number of actions undertaken as a function of the number of poor QMs.
To allow for generalizability to the national level, and because the response rate varied by region of the country and between for-profit and nonprofit facilities, we weighted the results presented here for the differential response rate. We note, however, that the weighted results were very similar to the unweighted results. The weights were probability weights calculated for each region/ownership stratum.
|
RESULTS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
|
|
Figure 1 provides insights with respect to these views. A large percentage of administrators recognized that the QMs were influenced by several factors other than quality, including variations in data coding (89%), unaccounted differences in case mix (81%), and unusual events (77%). Despite that, 60% believed that quality also influenced the QMs, and 17% believed that quality was the most important factor influencing the QMs. These responses suggest that although nursing home administrators were aware of the methodological complexities associated with these measures, they did not dismiss them as meaningless.
|
|
This information also suggests that at least in the short run, administrators did not expect the report card to influence consumers directly. However, because medical professionals—such as doctors and discharge planners in hospitals—seem to be very influential in individuals' choice of a facility, to the degree that they are aware of the QMs and incorporate this information into their referral recommendations, the QMs will have an indirect impact on demand.
How Do Facilities Benchmark Their Performance?
Despite their perception that consumers lack interest in (or at least awareness of) reported QMs, administrators' actions implied that they expected the report card to influence their standing in their market. The Nursing Home Compare Web site presents each facility's scores alongside state and national averages, making comparisons relative to the state and the nation very easy. However, making comparisons to local competitors in the same market requires a cumbersome search and manipulation of the information. Yet when asked to select the most important benchmark for comparison, 32% of administrators chose the state average and 28% indicated their local competitors. Only 2% of facilities chose the national comparison as the most important benchmark.
What Actions Have Nursing Homes Taken in Response to the Publication of the Nursing Home Compare Report Card?
Table 4 reports the responses of facilities to the following question: "Please indicate below if your facility adopted any of the following (actions) as a direct result of the publication of the QMs." It presents 22 specific actions that nursing homes could have taken following the publication of the report card. These include initiation of quality-improvement activities, changes in care protocols and work organization, changes in resources, changes in leadership, communications with consumers to explain the scores, and others. A review of responses to the question about whether a nursing home had taken any other action revealed that no actions other than the 22 explicitly included in the survey had been undertaken.
|
Among more specific actions, the most common were actions related to making changes in care protocols (36% had changed existing protocols, and 28% had developed new protocols) and training staff for the specific QM in which the facility had had a poor score (36%). In addition, 19% had changed the organization of work to empower workers, and 12% had revised job descriptions. These are actions that can, for the most part, be implemented with existing resources. In contrast, activities that required additional resources, such as purchasing new equipment or technologies (reported by 14%) or increasing staffing, either through direct hires, contract hires, or increasing wages and benefits (reported by fewer than 10% of homes) were much less likely to have occurred. This is a rational strategy in a revenue-constrained environment.
Changes in leadership might also be a response to poor scores, as they might be needed to enable more specific changes in care protocols and resource allocation. Responses indicated, however, that most nursing homes had not deemed those to be necessary. Fewer than 5% had changed the nursing director, and fewer than 1% had changed either the medical director or the owners of the facility. Changes in leadership may also be a signal to consumers that the nursing home is taking actions to improve. Nursing homes seem to have preferred to signal their consumers directly: 27% reported having engaged in direct communications with patients and their families to explain their scores.
The specific actions list also included two that could indicate the use of dysfunctional strategies: reallocating staff from other activities to care related to a poor QM (a teaching-to-the-test response) and changing the type of patient admitted (a cream-skimming response). Both were reported by a small percentage of facilities: 9% for the former and 4% for the latter. These low rates, coupled with the possibility that these changes had been undertaken for other reasons, such as a nursing home's recognition that they could provide better care if they specialized and admitted only some types of patients, suggest that dysfunctional responses were not widespread.
Are Facilities With Poor Quality Scores More Likely to Take Remedial Action?
Table 4 shows the average number of poor quality scores (below the 20th percentile in the state) for those facilities that had taken each specific action and those that had not. Although we chose to benchmark against the state distribution because this was the most important comparison identified by the administrators, future work should examine ranking of scores within each market.
We found a general tendency for those facilities that reported having taken remedial actions to have a larger number of poor QMs. For all actions except two, we found that the average number of poor QMs was higher for facilities that had taken the action. In only three cases, however, did t tests indicate significant differences at the.05 level. In a few instances, the differences in the average number of poor QMs were indeed small, but in most cases they were around 10% or more, suggesting that the lack of statistical significance may have been due to limited power. Therefore, we also performed an analysis pooling data across all actions. We estimated a logistic regression model predicting whether the facility had undertaken any action as a function of the number of low QMs it had. The estimated odds ratio was 1.187 (p value for the model and the coefficient less than.01, n = 699). We also estimated a Poisson regression predicting the number of actions as a function of the number of poor quality outliers and again found a positive and statistically significant association (marginal effect = 0.25, p value for the model and the coefficient less than.01, n = 699). These pooled analyses supported the expectation that, in general, facilities with poor scores were more likely to act on these scores compared with facilities with better scores.
Are Facilities Advertising High Quality Scores?
Given the marketing potential of good scores, facilities that receive them may choose to capitalize on them by advertising. In our sample, 79% of facilities had had at least one high quality score. Only 10% of facilities indicated that they had advertised their high quality scores. There was no significant difference between facilities that did and did not advertise in terms of the number of high quality scores.
|
DISCUSSION |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
The findings we present are based on a survey of nursing home administrators and, as all such surveys, may be subject to self-report bias. Administrators may have exaggerated their reporting of functional responses and minimized their reporting of dysfunctional responses. The survey, however, was anonymous, thus minimizing incentives to respond in a socially acceptable fashion. Furthermore, the face validity of the administrators' reports is enhanced by the finding that the propensity to undertake actions was associated with having poor quality scores, as one would expect. Nevertheless, readers should consider this inherent limitation of surveys when evaluating the findings, and future research should be aimed at verifying these findings from other data sources.
The comparison of the quality of care of responders and nonresponders at the time of initial publication also suggests some response bias. Responders had significantly better QMs for short-term patients. However, there were no significant differences in five out of the six long-term QMs, and the sixth was actually worse for responders. Furthermore, deficiency citations, which apply to both short- and long-term care, were not different for the two groups. Thus, if there is a response bias, it is limited to quality of short-term care.
Nursing homes seem to be adopting a variety of different actions to address poor quality scores. Further research is required to examine which poor QMs are associated with which actions. The specific actions may also shed light on why national statistics show an improvement in some of the measures and not in others. In general, facilities were more likely to report reorganizing their staff, retraining staff, and changing care protocols than increasing staff. Perhaps the QMs that exhibited improvements are more amenable to the type of actions that were more widely adopted, whereas those that did not show improvements may need more resource-intensive interventions, such as increased staffing.
It is important to understand how each specific action undertaken by nursing homes is related to quality improvement. Although the survey responses we present here suggest that nursing homes are acting on the reported QMs, the survey does not provide in-depth information about the degree of commitment of facilities or their knowledge and ability to be effective in bringing about actual improvement. As Greenhalgh, Robert, Macfarlane, Bate, and Kyriakidou (2004) argued, success in improving quality depends on the willingness to change day-to-day routines, especially those of nurses' aides. Understanding the relationships between specific actions and resident outcomes is important because it offers insights regarding the potential long-term impact of report cards. If report cards engender only certain types of responses, perhaps due to resource constraints, their impact may be limited.
Education and dissemination of evidence-based information about effective strategies and their costs may prove to be an important complementary policy in this context. Recent studies have shown that better quality in nursing homes is not necessarily associated with higher costs. Mukamel and Spector (2000) found that New York State facilities that had better outcomes for risk-adjusted mortality, functional status, and decline in pressure sores also had lower costs. To help nursing homes implement quality-improvement actions, further research is necessary on the cost implications of specific quality-improvement strategies to identify those that are not associated with higher costs or that perhaps even lead to lower costs. Evidence generated by such research will strengthen the current efforts of the QIOs to educate nursing homes about ways to improve quality.
In addition, we should note that in this survey we addressed only the initial response of nursing homes to the report card. These responses may change over time as facilities gain a better understanding of the measures and as the influence of the report cards on consumer choice becomes clearer. If consumers continue to ignore the published measures and if others who refer patients (e.g., physicians and social workers) do not rely on the report cards, then we may actually observe a reversal and find that fewer facilities in the future will act on the QMs. Therefore, readers should not interpret the somewhat promising response we found to mean that these report cards will provide long-term incentives for quality improvement. Researchers should monitor the behavior of both consumers and providers over the coming years.
In summary, initial evidence suggests that the Nursing Home Compare report card has motivated many nursing homes to improve care, and that it was a stronger motivating force for facilities with poorer scores. It thus suggests that these report cards may potentially be an effective addition to policy tools employed in the effort to improve the quality of nursing homes. Further research is required to understand which specific strategies nursing homes chose, what their impact on quality is, and the extent to which nursing homes engage in dysfunctional strategies such as cream skimming and teaching to the test.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Received for publication July 20, 2006. Accepted for publication January 31, 2007.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
| ||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
