| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
RESEARCH ARTICLE |
a Boston University School of Social Work, Sargent College of Health and Rehabilitation Sciences
b Sargent College of Health and Rehabilitation Sciences
c The New England Research Institutes, Watertown, Massachusetts
Scott Miyake Geron, Associate Professor of Social Welfare Policy and Research, 264 Bay State Road, Boston, MA 02215 E-mail: sgeron{at}bu.edu.
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
Methods. Focus groups with African American, Hispanic, and non-Hispanic White older adults were audiotaped. Tapes were transcribed and analyzed with grounded theory methods. Correlational and common factor analyses were conducted to select items, and the instrument was field tested with 228 frail, low-income, older home care recipients.
Results. Test-retest reliabilities ranged from .68 to .88, with high internal consistency reliabilities. Substantial concurrent validity was achieved for subscale and overall HCSM scores. Home care satisfaction was not related to gender, age or race but was negatively associated with physical disability. Significant social desirability effects were found.
Discussion. In the increasingly important area of home care, the HCSM is the first measure developed on the basis of the views of older consumers that also meets standard psychometric criteria. The HCSM provides a consumer-based indicator of quality and can be used to examine changes in satisfaction over time and differences among providers or within a single agency.
AS the home becomes an increasingly common venue of care for frail older adults, determining whether these consumers are satisfied with those services is now recognized as an important outcome and a basic responsibility of those who provide care to this population (Applebaum and Woodruff 1991
; Geigle and Jones 1990; Geron 1998; Institute of Medicine 1986; Joint Commission on Accreditation for Healthcare Organizations 1988; Riley, Fortinsky, and Coburn 1992). Assessing client opinions is always challenging, but it is especially problematic in the home setting. Unfortunately, prior efforts to measure care satisfaction have not taken into account the unique features of care in the home and have not used measures that are based on consumer perspectives of satisfaction with care. Although considerable attention has been given to the development of client satisfaction measures in acute care (Cleary and McNeil 1988; Cryns, Nichols, Katz, and Calkins 1989; Davies and Ware 1988; Ware, Snyder, and Wright 1976) and mental health (Larsen, Attkisson, Hargreaves, and Nguyen 1979; Lebow 1983; Love, Caid, and Davis 1979), existing measures of older consumers' satisfaction with home and community-based long-term care services have several shortcomings.
First, many of the satisfaction instruments used in home care have been adapted from measures developed for assessment of client satisfaction with medical care services, even though care provided in the home or nursing home is different in many respects from acute care or office-based health or mental health care (Kane, Illston, Eustis, & Kale, 1991). Compared to the circumscribed episode of a physician visit or acute medical episode, long-term care centers on a consumer's daily living situation and is frequently of long duration. Moreover, although long-term care does sometimes require the use of sophisticated technologies, much of the service is "low tech," often provided by personnel with limited training and without professionally derived standards of practice.
Second, most instruments have been developed for a particular research or programmatic purpose (Geron 1996; Linn 1985; Linn, Linn, Stein, and Stein 1989; Lucas, Morris, and Alexander 1988; Pablo 1975; Rinke and Wilson 1988; Woerner and Phillips 1989). Although they contribute to the understanding of home care client satisfaction, ad-hoc measures do not allow for comparison across studies. Single-item global rating satisfaction measures have also been used frequently; unfortunately, these do not adequately capture the complexity of services such as health, mental health (Andrews and Withey 1976; A. L. Stewart and Ware 1992), or long-term care.
Third, most available measures are based on researcher or provider perspectives of satisfaction, not the perceptions of older recipients of services. Equally important, even when consumer involvement has been used (Woerner and Phillips 1989), the perspectives of minorities have been absent. The involvement of consumers is now widely recognized as necessary to ensure that the measure fully represents the dimensions of quality considered important to service recipients (Applebaum 1989; Arnold 1989; Miller-Hohl 1992; Riley et al. 1992; Rinke and Wilson 1988). Because satisfaction may have different meanings to different consumers (Geron 1995; Gutek 1978), the need to explore the meanings consumers attach to terms such as satisfaction, satisfied, or like is a critical first step in developing a home care satisfaction instrument.
Finally, none of the existing measures that have been developed to evaluate the services received by frail older adults in their homes have been rigorously tested for reliability and validity. Adequate psychometric testing is a prerequisite to accurately evaluating program or treatment effects; establishing norms for satisfaction ratings across services, client populations, and ethnic groups; investigating correlates of satisfaction; or evaluating causal models of satisfaction.
In addition to the clear need for a psychometrically sound instrument of home care satisfaction, questions remain about possible biases to responses to home care satisfaction instruments. Prior research has established that satisfaction measures are susceptible to response bias (Hays and Ware 1986; C. K. Ross, Steward, and Sinacore 1995). C. E. Ross and Mirowsky 1984, for example, found that acquiescence response bias—the tendency of participants to answer yes to questions regardless of their content—and socially desirable response bias—the tendency of participants to respond to questions with what they assume to be the normatively right answer or with the answer that they assume the interviewer wants to hear—are greater in older persons with lower socioeconomic status. Obviously, the extent to which response bias operates in home care satisfaction assessments is an important area of study.
The correlates of home care satisfaction are another important area of gerontological research that remains largely unexplored. The literature on satisfaction with acute care generally shows a positive relationship between patient satisfaction assessments and age, although other studies have found weak to no relationship (Aharony and Strasser 1993; Davies and Ware 1988; Greenley, Young, and Schoenherr 1982; Linn 1985; Locker and Dunt 1978; Pascoe 1983). Race and ethnicity are also inconsistently associated with satisfaction. Several studies found Whites to be more satisfied with medical care than non-Whites (Pascoe and Attkisson 1983), but others have found the opposite relationship or no relationship between satisfaction and ethnicity (e.g., Greenley and Schoenherr 1981; Linn 1985; Pulliman 1991). Client health status has been found to have a complex relationship to satisfaction, with most studies showing health status positively related to satisfaction (Aharony and Strasser 1993; Cleary and McNeil 1988; Hall, Milburn, and Epstein 1993; Lochman 1983; Proctor, Morrow-Howell, Albaz, and Weir 1992). A number of studies have found that health status is related to satisfaction, with patients in poorer health reporting less satisfaction (Hall et al. 1993; Marshall, Hays, and Mazel 1996). Although little research has been done on the relationship of physical functioning and satisfaction, one study found that participants with disabilities tend to be less satisfied with their medical care than patients without disabilities (Patrick, Scrivens, & Charlton, 1983).
The Home Care Satisfaction Measure (HCSM) was designed to overcome shortcomings identified in the existing literature. The specific purpose of the measure is to provide a standardized, general scale of home care satisfaction that is relatively brief, is easy to administer, and meets standard psychometric criteria for validity and reliability. In this article we present findings from the development and field test of the instrument. The Methods section describes the procedures used to develop a multidimensional measure of client satisfaction for frail older consumers receiving long-term home and community-based services derived from the perspectives of an ethnically diverse sample of service recipients. Findings from the analysis of field-test results are then presented, and include: (a) development of scores for individual services as well as an overall score for home care satisfaction; (b) evaluation of the test-retest reliability and validity of the HCSM; and (c) examination of the effects of sociodemographic factors and response bias on home care satisfaction.
|
Methods |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
; Pascoe 1983). A consumer's determination of satisfaction (or dissatisfaction) with a service results from (a) a cognitive evaluation or judgment of the consumer's actual experience of the service compared to his or her expectations about the service and (b) an affective response to the evaluation (Rust and Oliver 1994; Yi 1990). We hypothesized that assessments of any particular home health service are multidimensional; that is, that clients consider several different dimensions when evaluating their satisfaction with a service. The relevant dimensions of satisfaction may differ from one service to another. Clients then integrate assessments of individual services to produce a rating of their overall satisfaction. Each service contributes to the overall rating, although some services may be more influential than others.
Prototype instrument construction.
The development of the HCSM was based primarily on the perspectives of an ethnically diverse sample of home care clients about what they liked and did not like about the services they received. We held eight separate focus groups with African American, Hispanic, and non-Hispanic White older people (Jewish, Italian American, and Irish American) to probe differences in the meaning of satisfaction across racial and ethnic groups. Group membership ranged from 4 to 8 consumers, with a total of 42 participating.
Grounded theory methods were used to identify satisfaction dimensions that guided the development of the instrument (Charmaz 1990; Orona 1990). Focus group sessions were audiotaped and transcribed. Transcriptions of the focus groups were analyzed separately by the principal investigator and research associate into content and theme categories, some of which were identified in the focus groups and others that emerged in the analysis of the transcripts. At the same time, a "quote bank"—consisting of individual quotes from focus group participants—was compiled, with individual statements sorted into the theme or content categories. These quotes, as well as the themes and content categories, were considered, reviewed, and analyzed until dimensions of satisfaction were identified. As each dimension was explored, discussed, and refined, we used the quotes of the study participants to define and explicate the meaning of the theme. Ultimately, eight dimensions of home care satisfaction were identified, although not all dimensions applied to all five services: competency, humaneness, dependability, service adequacy, continuity of care, choice, accessibility, and advocacy. Two independent raters repeated the analysis of the transcripts and identified seven of the eight dimensions.
An item pool was created from several sources: the identified focus group dimensions, existing satisfaction instruments, previous measures developed by the principal investigator, and the quote bank developed from the focus groups. A card sort exercise was conducted in which 14 graduate students sorted draft items into the predefined satisfaction dimensions. More than 70% of all items were correctly placed in their intended dimension. Pretest interviews were conducted with 7 frail older people, and on the basis of these results, items were refined and in some cases new ones were added. Further pretesting was done with 25 older people, and a final field-test version of the client satisfaction measures was developed.
The prototype version of the instrument included a section for each of five long-term home care services: homemaker, home health aide, home-delivered meal, grocery, and care management. Each service section contained 30–40 items that addressed specific dimensions of care received. These sections contained a balance of positively worded and negatively worded items, which has been found to reduce acquiescence response bias (Sherbourne, Hays, and Burton 1994). The response option used was a 5-point Likert-type response option with an agree-disagree scale adapted from the Client Satisfaction Questionnaire (CSQ; Larsen et al. 1979): yes, definitely; yes, I think so; maybe yes, maybe no; no, I don't think so; and no, definitely not, with the middle category added for better gradation in responses. The wording chosen was found in the pretest to work well with draft item stems that required agree-disagree type response choices and item stems that required frequency (always-never) type response choices.
To obtain a culturally equivalent Spanish-language version, we translated the prototype instrument and all supporting information (letters, phone scripts, etc.) in a process called back-translation. In this three-step process, the original English was translated into Spanish by a bilingual speaker, the Spanish version was translated back into English by a second bilingual speaker, and then the original and back-translated English versions were compared (Brislin 1980).
Field test of prototype.
The prototype instrument was field tested with cognitively intact, community-dwelling adults aged 60 years or older who had received one or more of the selected in-home services for at least 6 months. One quarter of the participants were drawn from a representative sample of older adults from Springfield, MA participating in another study (Tennstedt and Chang 1998). For the remaining participants, a stratified sample of approximately equal numbers of African American, Hispanic, and White elders was randomly selected from client lists at four Boston area home care agencies that provided care management and home care services under the state's 2176 Medicaid waiver program. For the Springfield sample, each eligible person was sent an introductory letter explaining the purposes of the study. Participants were then contacted by telephone to schedule a home visit. For the Boston area sample, randomly selected clients were first contacted by an agency worker, given a brief description of the study, and asked for their consent to be contacted by the project. The field interviewers then contacted those granting consent by telephone to arrange a convenient time for a home visit. Participants were excluded if their care manager indicated they had a known significant memory impairment or could not complete a structured interview. All interviews were conducted by trained field interviewers. Before answering specific item questions, all respondents were given a card with the response choices typed in large letters. For all Spanish-speaking participants, the interview was conducted in Spanish. Field-test participants were matched by ethnicity to field-test interviewers whenever possible. Written informed consent was obtained from all respondents.
Demographic characteristics were obtained using standard survey items from previous NIA-funded surveys (Tennstedt and Chang 1998; Tennstedt, Crawford, and McKinlay 1993). The disability status of respondents was determined on the basis of their responses to measures of basic activities of daily living (ADLs) adapted from the Katz ADL Index (Katz 1959) and instrumental ADLs (IADLs) adapted from the OARS Multidimensional Functional Assessment (Center for the Study of Aging and Human Development 1978). We assessed the degree of difficulty in these activities using a measure employed in the 1984 and 1986 Longitudinal Study of Aging, a large probability sample of noninstitutional elderly people. The satisfaction items for each service were prefaced by items that recorded information about the length of time the respondent had received the service and the characteristics of the worker providing the service.
All participants completed a global satisfaction rating scale for each service. The basic scale, developed for this study, was a 20-cm visual analogue scale in the shape of a vertical thermometer, with the end points clearly anchored. Participants were shown three thermometers sequentially for each service they received and asked to rate (a) their overall satisfaction with that service, (b) whether the services were "provided as they like;"and (c) the overall quality of the service. Respondents indicated their opinion by placing a mark on the thermometer. The distance to the mark from 0 was measured manually in centimeters to the nearest millimeter and converted to a 0–100 scale. In addition to the individual service thermometers, respondents completed three similar thermometer ratings for all services considered together. Respondents who were visually impaired or who otherwise had difficulty completing the rating were asked to state a number between 0 and 100 that best reflected their opinion for each scale.
To assess for the possibility of item order effects, respondents were randomly assigned to complete the thermometer ratings either before or after the five service sections. The order of the thermometers was also randomly determined. We randomly assigned respondents to receive the all-services thermometers (i.e., the ratings of all services combined) before or after the individual service thermometers. Participants also completed a number of other measures, which are described in the following sections.
At the end of each interview, we asked participants to identify a caregiver or relative, friend, or neighbor "who knows them well." Caregivers were contacted and asked to schedule a brief in-person interview to complete thermometers similar to those for the participants. Using the thermometer ratings, caregivers assessed the participants' opinions of the services received.
Analyses
Development of the HCSM.
Two methods were used to evaluate the initial pool of items, identify the dimensions of satisfaction, and select the final items for the measure. First, for each service, correlations were computed among all items in the pool. Each matrix was used to test convergent validity (large correlations among items hypothesized to measure the same dimension) and divergent validity (small correlations among items measuring different dimensions). Several items that were not related to their intended dimensions or that appeared to tap several dimensions were eliminated from the pool, as were others that were confusing to respondents. The remaining items were then subjected to a factor analysis using oblique (Promax) rotation estimated by maximum likelihood. Separate factor analyses were performed for each service. Oblique rotation was used because the dimensions of satisfaction were presumed to be correlated with one another. We selected items to measure each service dimension on the basis of the magnitude of the factor loadings and the unidimensionality of the items. We also selected items to balance the number of positive and negative items for each dimension.
Scoring.
Responses to selected individual HCSM items were recoded to a 0–100 scale, where 0 was the lowest and 100 was the highest possible satisfaction score. Negatively worded items were rescaled so that higher scores represented greater satisfaction. HCSM service scores were computed as the mean of all item scores for a given service. Similarly, dimension scores for a service were computed as the mean of all item scores in the dimension. This approach gave all items equal weight, with the effect that dimensions measured by more items influenced the service score more. When items were missing, service and dimension scores were computed as the mean of all completed items as long as at least half of the relevant items were answered.
Development of overall satisfaction score.
The task of developing a method for combining individual service satisfaction scores into an overall satisfaction score on a 0–100 scale was complicated by the fact that respondents received different combinations of services. To determine whether some services had a greater impact on overall satisfaction than others, we performed a series of regression analyses in which we regressed the respondent's overall satisfaction with home care services (as assessed by the respondent's thermometer rating of all services combined) on the HCSM scores for individual services. The regression coefficients in these equations estimated the relative importance of each home care service to overall home care satisfaction. We estimated regression models for each of the most common service configurations. We also estimated a model for the full set of services using the correlations for observed cases for each of the possible service pairings (pairwise deletion). To determine whether different weights were needed for individual services to derive overall satisfaction scores, we tested for statistically significant differences between service score regression coefficients in the regression models. In addition, for the model containing the full set of services, we compared optimally weighted satisfaction scores to satisfaction scores computed simply by giving equal weight to each service. Equally weighted scores were equivalent to the mean of a respondent's service scores.
Reliability and validity analyses.
To assess test-retest reliability, one half of the participants were randomly selected to participate in a second administration of the survey instrument within 5–9 days of the first. One-way intraclass correlations between individual service scores and dimension scores within services completed at both time points were calculated. To assess concurrent validity of the subcale measures, we computed Pearson correlations between individual HCSM subscale scores and the individual service thermometer ratings completed by respondents and caregivers. Pearson correlations were also computed comparing the overall HCSM score and two overall satisfaction ratings: the CSQ (Larsen et al. 1979) and all services thermometer ratings for overall satisfaction completed by respondents and caregivers.
Effects of sociodemographic characteristics and response bias.
Regression analyses were used to explore the relationship between home care satisfaction and participants' age, ethnicity, and disability status and to assess the extent of response bias in frail respondents' satisfaction assessments. We included the RD-16 (Schuessler, Hittle, and Cardascia 1978) to examine the extent of socially desirable responding with administration of the HCSM. The RD-16 is a 16-item measure containing eight pairs of agree-disagree statements covering dejection, social estrangement, social opportunism, trust, social contentment, anomie, expediency, and self-determination. Two other possible contributors to response style effects on the satisfaction assessments were examined. The Positive and Negative Affect Schedule (PANAS; Watson, Clark, and Tellegen 1988) consists of two 10-item scales in which participants are asked how they would rate, on a scale from 1 to 5, words describing emotions or moods they felt during the past week. The PANAS has been used successfully with older adults (Kercher 1992). Additionally, a 9-item locus of control measure (Lachman and Weaver 1998) was employed. The instrument is composed of two subscales, Mastery and Constraints, scores for which are based on participants' agreement or disagreement with a series of statements associated with each subscale.
|
Results |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
The sample was comprised largely of women (79.4%) with substantial disability (M = 6.58 ADL/IADLs, SD = 3.28, range = 0–13) and low educational attainment (M = 9.5 years of education, SD = 4.2, range = 0–22). Because receipt of the home care services studied required Medicaid eligibility, all sample participants were low income. The average age of the participants was 75.5 years (SD = 8, range = 61–95); 44.3% were African American, 16.7% were Hispanic, and 39% were White. Measures of functioning show that 39.5% were dependent in three or more ADLs. All participants received care management services. In addition, 92.1% also received homemaker service, 40.8% received home health aide service, 41.2% received home-delivered meal service, and 24.6% received grocery service. Participants received services in a variety of combinations: 34.6% of the sample received two of the five services, 40.8% of the sample received three services; 21% received four services, and only 3% received all five services. The most typical combination of services was care management and homemaker (29.4%). For those who received three services, the most typical combination was care management, homemaker and home-delivered meal (16.2%), with care management, homemaker, and home health aide occurring almost as often (14.9%).
Description of the HCSM
The HCSM consists of 60 items measuring dimensions of satisfaction with five major home health services (Table 1 ). All of the items have direct referents: That is, they involve ratings of services received personally by the client (Ware 1978). The Homemaker Service subscale (HCSM-HM13), Home Health Aide Service subscale (HCSM-HHA13), and Care Management Service subscale (HCSM-CM13) each contain 13 items. The Home-Delivered Meal Service subscale (HCSM-MS11) contains 11 items, and the Grocery Service subscale (HCSM-GS10) contains 10 items. All of the subscales (except the Grocery Service subscale) contain a balance of positively and negatively worded items. All items use the same 5-point Likert-type response option used in the field test, which worked well with respondents in both the pretest and the field test. We found that providing participants with a response card with the response choices written in large type and carefully training interviewers facilitated the cards' ease of use and acceptance.
|
|
Overall Home Care Satisfaction Score
As described earlier, we performed a series of regression analyses in which we regressed the respondent's overall satisfaction with home care services on the HCSM scores for individual services. As shown in Table 3 , the regression coefficients were similar. None of the individual coefficients were significantly different from the smallest coefficient in a model, suggesting that, for these data, each service had approximately the same impact on overall satisfaction. As an additional check, we compared overall scores based on equally weighted scores to optimally weighted scores computed from the four-service regression equation. The correlation between these two sets of scores was .92, indicating that differential service weighting had little effect on scoring. As a result, overall HCSM scores were computed as the mean of a client's applicable service scores. In this sample, overall scores ranged from 13 to 98 (M = 75.2, SD = 14.4).
|
).
|
). These correlations, which ranged from .26 to .76, were all significantly greater than 0. Four of the five HCSM service scores were correlated more highly with the participant's thermometer ratings than the other indicators of service satisfaction. The HCSM overall satisfaction score was correlated more strongly with the CSQ—also a measure of overall satisfaction—than either of the satisfaction ratings. Table 5 also shows the two-way (participant and rater) intraclass correlations for the thermometer ratings provided by both respondents and caregivers. These reliability coefficients, ranging from .32 to .52, indicate that much of the variability in thermometer values was attributable to raters.
|
|
; Sherbourne et al. 1994) and the fact that people give lower satisfaction scores after they are asked items about specific domains than they do if satisfaction ratings are requested first (Ware and Davies 1983; Ware and Hays 1988). We assessed administration order effects by randomly assigning respondents to complete the appropriate thermometer ratings before or after completion of HCSM measures. All correlations between service scores and thermometer ratings were significantly greater than 0 and of similar magnitude regardless of the order of administration (Table 7 ). There were no statistically significant differences in HCSM mean service scores or standard deviations between respondents who completed thermometers prior to HCSM items and those who did the HCSM items before the thermometers. Therefore, it does not appear that the order of thermometer administration affected either the HCSM score levels or the correlations with thermometer values.
|
|
Discussion |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
Like most satisfaction measures, HCSM overall and subscale scores were negatively skewed, but mean scores for each service subscale and for overall home care satisfaction were well below their ceiling scores. The HCSM service and dimension scores possess an acceptable to high degree of internal consistency. The HCSM service score test-retest reliabilities were all large, ranging from .68 (in the smallest service group) to .88. Evidence for the validity of HCSM scores is demonstrated by large positive correlations between HCSM subscale scores and separate global satisfaction ratings for each service completed by respondents and caregivers, as well as between the HCSM overall home satisfaction score and separate global satisfaction ratings of all services combined and the CSQ (Larsen et al. 1979).
Although other studies have found age-related differences in satisfaction, with elderly persons typically reporting more satisfaction than younger persons, our analyses show that HCSM results were not influenced by respondents' ages, possibly suggesting that any age effects diminish among older adults. We also found that a client's race did not affect satisfaction with home care. Physical disability was associated with HCSM satisfaction scores, with more disabled participants generally less satisfied than participants with less disability. Some recent reviewers have concluded that physical health and satisfaction are correlated (Aharony and Strasser 1993), but the evidence is by no means conclusive (e.g, Marshall et al. 1996).
Response bias has been found to be a source of bias in previous studies of older adults and in research on satisfaction, and those findings receive some support here. We found significant response bias using a measure of social desirability. These findings were significant for the Homemaker Service subscale and for the overall HCSM score and approached significance for the Care Management Service subscale. The primary effect seems to be that social desirability leads to overreporting of satisfaction.
Social desirability effects may be partially attributable to the type of item referent used in the HCSM. There has been considerable debate in the literature about the type of item questions to include in satisfaction instruments (Hays and Ware 1986; Pascoe, Attkisson and Roberts 1983; Roberts, Pascoe and Attkisson 1983; M. A. Stewart and Wanklin 1978). In direct referent questions, participants are asked to evaluate care or services they personally receive; in general referent questions, participants are asked to evaluate care or services in general. Pascoe and his colleagues argued that satisfaction assessments employing direct referent items are more acceptable to participants; produce fewer missing data and less inconsistency in recording; and are not related to global or multidimensional measures of subjective well-being (Pascoe et al. 1983; Roberts et al. 1983). In contrast, Ware and his colleagues argued that measures that use general referent items lead to less socially desirable responding and fewer skewed responses (Hays and Ware 1986; Stewart and Wanklin 1978).
The HCSM is an example of the direct approach; that is, it is composed of items that require respondents to address specific aspects of the care they receive from various unskilled or nonmedical home care service providers. The prototype instrument used in the field test was a mix of both types of item referents, although most were direct referent items. The general referent items were found to have, on average, more missing data than direct referent items and to be less strongly correlated with other items in the same hypothesized dimension. In the factor analyses, the general referent items loaded less strongly than similar direct referent items. The results also provide partial support to concerns about direct referent items raised by Hays and Ware 1986 and others. The general referent items were less skewed, and as we described earlier, significant response bias was associated with the HCSM.
The general referent items in the prototype instrument were not selected for the HCSM largely on the basis of the results of the item analyses. However, in choosing direct referent items for the HCSM, and the measurement risks associated with them, we were also influenced by the type of clients who received home care services in our study—low-income frail elderly persons with little formal education who would benefit from items that were easily understandable—and the type of services under review. Unlike medical care, in which visits with physicians may be infrequent and in which the technical complexity of services is high, home care services are frequent and ongoing and involve tasks that most adults have performed most of their lives. We believed it would be confusing to ask general referent items about services that were so much a part of the everyday lives of these respondents.
Although the initial psychometric properties of the HCSM are generally very positive, a number of issues warrant further consideration. The question of how analysts should weight home care services to create an overall home care satisfaction score is a difficult methodological problem, because clients receive many different combinations of services. One might hypothesize that care management and food service would be less influential because they involve less personal services and respondents may not be aware of what their care manager is doing behind the scenes. However, food service ratings were strongly independently associated with thermometer ratings in our analyses, and care management was also a statistically significant predictor of the ratings. We did not find that any particular service was significantly more or less important than another in predicting overall satisfaction, even when we used pairwise deletion to retain all of the cases. Detecting differences between services would require much larger samples of clients in a variety of service configurations. We therefore believe that it makes sense at this point to weight all services equally in the absence of strong evidence to the contrary.
Two of the subscales in the HCSM—Homemaker Service and Home Health Aide Service—use equivalent items, except for one item that is worded slightly differently. From the consumer's point of view, the important dimensions of these services are the same, so it is not surprising that the items would be similar. Items that more clearly distinguished between these two services (e.g., items that addressed bathing and dressing for the Home Health Aide Service subscale) did not perform as well as the selected items in the item and factor analyses that we conducted. However, the equivalence of the two subscales for services that have obvious similarities but also clear differences suggests at least the possibility that consumers may have trouble distinguishing between these services when asked about them. Although both services are common in Massachusetts—more than 40% of the field-test sample received home health aide services and more than 90% received homemaker service—it is not unusual for the same person to perform both services. Another possible source of confusion is that some of the clients receiving home health aide service were receiving only services from the care management program (i.e., help with personal assistance tasks, but not skilled nursing services), whereas others were also receiving Medicare home heath aide services. For these reasons, the extent to which these subscales accurately reflect consumer views of these services requires further study.
The generalizability of the HCSM needs to be established. The HCSM was developed with a single state's system whose home care portfolio may be unique. Grocery service is clearly not a typical home care program, for example. Not all states provide both home health aide (sometimes called personal care) services and homemaker services; in some states that do, such as Massachusetts, a single worker may perform both sets of services. The HCSM is now being used in a number of different statewide programs (Hawaii and Florida included) and several substate programs and agencies. To date, the adaptability of the HCSM has been encouraging (judging by reactions from users, administrators, and evaluators). However, researchers need to replicate these analyses in larger samples of older persons to develop more confidence in the generalizability and psychometric properties of the measure.
More work is also needed on the sensitivity and appropriateness of the HCSM in different ethnic groups. Although we found no effects of ethnicity on HCSM responses, we make no assumption of its applicability to ethnic groups other than those included in the development sample. Like any standardized measure, the cultural appropriateness of the HCSM needs to be established with each new ethnic group to which it is applied. Research also is needed on the effect of mode of administration on HCSM results. The HCSM was designed to be used in brief in-person interviews by trained interviewers, but a telephone version of the instrument has been developed. One potential source of measurement error has been tentatively ruled out: There were no statistically significant differences in HCSM mean service scores or standard deviations between respondents who completed thermometers prior to HCSM items and those who did the HCSM items before the thermometers. However, more research is needed on whether HCSM service scores are influenced by the order in which questions are asked.
One of the major unanswered research questions about the HCSM is the ability of the HCSM to distinguish between providers known to differ in performance or quality. In this study, we did not have access to program characteristics or other client outcome data, so no assessment of the predictive validity of the HCSM was conducted. Research is also needed to determine the sensitivity of the HCSM in detecting changes in service conditions, service providers, or changes in the client's health or other life circumstances. The high nonresponse rate in the current study is also a potential source of bias (Aharony and Strasser 1993). If nonresponders have different levels of satisfaction than responders, the results from responders will be biased. In this study, no information was available on nonrespondents. At the present time, the extent of bias in satisfaction assessments from nonrespondents is unknown (Sherbourne et al. 1994).
Research on the effects of cognitive impairment on satisfaction with home care services is also needed, because in the current study we did not employ a standardized measure of cognitive functioning. Finally, the influence of client expectations on HCSM responses needs to be more closely evaluated. Most satisfaction researchers believe that a client's expectations about a service influence his or her satisfaction response. Expectations for care can vary culturally (Scheer and Luborsky 1991) and may be affected by length of service, class, and other factors (Linder-Pelz 1982; Pascoe 1983; Rust and Oliver 1994).
In summary, the HCSM is the first measure developed that is based on the views of older consumers while also meeting standard psychometric criteria. In the important area of home care, the instrument addresses an issue increasingly facing federal policy makers, state administrators, planners of community-based programs providing home care services for frail older adults, and researchers: how to systematically measure client satisfaction with home care services. The HCSM is designed to be used by agencies and organizations that arrange or provide community-based home care services to frail older adults residing in the community as well as by researchers studying consumer evaluations of home care services (either interprogram or sample to sample). The HCSM provides a consumer-based indicator of quality, and can be used as an outcome or performance measure to examine changes in satisfaction over time and to study differences among providers or within a single state or agency. Each subscale of the HCSM can be completed in 3–5 min. The HCSM can be supplemented by open-ended questions or items of particular interest to a particular service program. The measure is currently available in English, Spanish, and Russian, and it can be administered in a face-to-face or telephone interview with a trained assessor.
With recent support from The Retirement Research Foundation, the HCSM has been adopted for use in three statewide programs and one large nonprofit agency, and based on those experiences, we are currently developing training manuals and guides for prospective users. The Administration on Aging, U.S. Department of Health and Human Services, has recently selected the HCSM for use in a demonstration of performance outcome measures, and the HCSM is being considered for use or pilot tested in several other states and programs. To date, more than 9,000 telephone administrations of the HCSM have been completed, and benchmarks are being developed that will permit agencies to compare their performance to national home care satisfaction standards. We are continuing to conduct research on the HCSM and to work with users to productively use the results to improve the quality of home care services.
|
Acknowledgments |
|---|
Received for publication April 20, 1999. Accepted for publication February 25, 2000.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  J. M. Wiener, W. L. Anderson, and G. Khatutsky Are Consumer-Directed Home Care Beneficiaries Satisfied? Evidence From Washington State Gerontologist, December 1, 2007; 47(6): 763 - 774. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Hannum Rose, K. F. Bowman, E. E. O'Toole, K. Abbott, T. E. Love, C. Thomas, and N. V. Dawson Caregiver Objective Burden and Assessments of Patient-Centered, Family-Focused Care for Frail Elderly Veterans Gerontologist, February 1, 2007; 47(1): 21 - 33. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Applebaum, S. Kunkel, and K. Wilson Transforming Data Into Practical Information: Using Consumer Input to Improve Home-Care Services Gerontologist, February 1, 2007; 47(1): 116 - 122. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. J Porter and L. H Ganong Older Widows' Speculations and Expectancies Concerning Professional Home-Care Providers Nursing Ethics, September 1, 2005; 12(5): 507 - 521. [Abstract] [PDF]
|
|
|
|
 |
|
 J. W. Min Preference for Long-Term Care Arrangement and its Correlates for Older Korean Americans J Aging Health, June 1, 2005; 17(3): 363 - 395. [Abstract] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
