| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||||||||||
RESEARCH ARTICLE |
1 Department of Industrial Engineering, University of Miami, Florida.
2 Department of Psychiatry and Behavioral Sciences, University of Miami School of Medicine, Florida.
Requests for reprints should be sent to Joseph Sharit, Department of Psychiatry and Behavioral Sciences, University of Miami School of Medicine, 1695 NW 9th Avenue, Suite 3208 (D-101), Miami, FL 33136. E-mail: jsharit{at}miami.edu
 |
Abstract |
|---|
 TOP Abstract MethodsResultsDiscussionReferences |
|---|
IN THE United States and other parts of the world, the demographics of the workforce are changing in the direction of an increasing proportion of older workers and a decreasing proportion of younger workers. The implications of this change in the age distribution of the workforce are numerous and of growing concern. An important concern relates to the fewer number of younger workers available to support an increasing number of people who have retired from work and are living longer (Griffiths, 1999
; Warr, 2000).
Two ways of dealing with this problem are to encourage retired individuals to reenter the workforce and to convince currently employed older workers to remain working. Either strategy requires redesigning jobs and workplaces to accommodate age-related changes in abilities, and training older people to meet the demands of today's jobs (Czaja, 2001). Unfortunately, many managers assume that older workers cannot learn new skills or meet the challenges of jobs that rely on interacting with information technology. Although these are commonly held beliefs, there are little data to support them in the literature. Furthermore, many managers are unaware of factors that could facilitate the ability of older people to reenter or remain in the workforce.
Rapid growth in a type of work known as telecommuting or telework offers the potential for accommodating many of the needs of older workers. Telecommuting represents an "anytime–anyplace" form of work, whose rapid growth can be attributed to sharp increases in network capability and declines in the cost of technology (Buessing, 2000). In 2001 there were an estimated 28 million teleworkers in the United States (Davis & Polonko, 2001), and, with projections of continued growth in telework, the current number of teleworkers is likely to be much higher. Although telework is most appropriate for tasks that require low degrees of interworker cooperation and face-to-face communication, it can be adapted to a wide range of tasks.
The attractiveness of telecommuting to companies stems from an array of cost reductions and benefits. Cost reductions result from a savings in space and lower operational costs. The benefits include increased efficiency through more flexible work schedules, increased opportunities for retaining employees faced with the demands of providing family care, and higher job satisfaction and well-being (Buessing, 2000). Telecommuting also provides opportunities for people who are handicapped or have limited mobility and can reduce the stress associated with commuting. Many of these benefits are particularly relevant to older adults. In addition, in view of economic realities and the well-documented benefits to health and well-being of maintaining a productive lifestyle, many older adults would welcome the opportunity for part-time work (American Association of Retired Persons [AARP], 2002), which telecommuting can easily accommodate.
One job that is becoming more widespread in a growing service and consumer-oriented market economy and that is very amenable to telework is customer service. This type of work now can be done easily through electronic means. Evidence suggests that older adults enjoy this type of work due to its social aspects, even in the absence of voice communication (Sharit, Czaja, Nair, Hoag, & Leonard, 1998).
The focus of the present study was on evaluating the ability of older adults to perform an e-mail-based information search and retrieval customer service task. The study is part of a larger research program that is focused on older adults and computer-based work (Czaja & Sharit, 1993, 1998; Czaja, Sharit, Ownby, Roth, & Nair, 2001). Our primary emphasis in this article is on age differences in performance of this task and degree of skill acquisition as a function of extended practice. In general, the skill-acquisition literature indicates that although the performance of older adults improves with practice, their level of skill acquisition for equivalent levels of practice will be lower than that of younger adults (Rogers, Fisk, & Hertzog, 1994). We also address task performance requirements that may be particularly problematic for older adults, as well as attitudes older adults have regarding the desirability for performing this type of work.
|
METHODS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
).
We collapsed data on education into three levels: high school or less, some college or a college degree, and some graduate school or a graduate degree. There were no significant differences between the two age groups in level of education, 
2(2, N = 52) = 2.93, p =.231. All but two participants in each age group had at least some college education. Sixty-three percent of the younger participants had some college or were college graduates as compared with 40% of the older participants, whereas 52% of the older participants had some graduate education or a graduate degree as compared with 29.6% of the younger participants.
There were also no significant differences between the two age groups in occupational status (retired, working full time, working part time, working as a homemaker, and other), 2(4, N = 52) = 3.31, p =.518. Eight of the 25 younger participants were retired as compared with 12 of the 27 older participants.
We assessed potential differences among the participants in breadth of computer experience by using three items from a technology and computer experience questionnaire that was administered prior to the experiment. The items addressed proficiency with input devices (e.g., mouse, voice input), basic computer operations (e.g., deleting and transferring files), and applications (e.g., computer graphics, programming, and electronic mail). The scores on this measure could range between 0 and 26. There were significant age differences in breadth of computer experience, t = 2.89, p =.006, with the younger group having a greater breadth of prior computer experience.
Materials
Task description
The task we used in this study was a simulation of a job performed by customer service representatives of a fictitious company called Media Products, an online store that sells computers, accessories, supplies, software, and books. Like many Internet-based companies, the customer service function is handled by means of e-mail. Our development of this simulation was based on an analysis of companies that sell similar products (e.g., Apple Computer, Amazon.com) and on interviews with individuals who worked as customer service telecommuters for a small Internet company that sold software products.
The information that the participants needed to access to respond to customer queries was contained in a database composed of three primary sections: company policies and procedures, products, and customer and order information. The polices and procedures section consisted primarily of rule-based information grouped into 10 categories; corresponding submenus were available for pursuing deeper searches within each of these categories. The product section contained information on 6 different categories of products: computers, computer peripherals, computer accessories, consumables, software, and books. Separate submenus were available for accessing information regarding the products within each of these categories. The customer and order information section contained an alphabetic listing of information on customers and their orders.
In each of eight "work sessions," the participant's task was to sequentially open e-mails from a listing of 40 e-mails contained in an e-mail inbox window and respond to each customer's queries. Double clicking anywhere on the line corresponding to an e-mail in the inbox window resulted in that e-mail's text being displayed on the full screen. Clicking on the "search the database" button resulted in a split-screen configuration that displayed the e-mail in a separate window located in a fixed location; the main menu items corresponding to the three primary sections of the database system were displayed to the right of the e-mail. We chose this split-screen format to minimize working memory demands associated with remembering the message.
Figure 1 illustrates this split-screen format for an e-mail in which the customer is requesting two distinct items of information: one concerns company policies and procedures regarding shipping charges when a computer system is purchased, and the other concerns information regarding the type of screen for a particular computer system. This figure also illustrates the submenus that were displayed following the selection of the Policies and Procedures section of the database. Located under these submenus is a set of icons that, from left to right, allow the user to return to the main menu, return to the previous step, display a "history" window that lists all selections of information made to that point, send the e-mail reply, or stop the search. For this example, the appropriate selection would be the shipping submenu, which would result in the display of five submenus related to shipping (e.g., Order Tracking Number, Shipping Methods and Costs, Shipping Problems). The required submenu would be Shipping Methods and Costs. Clicking on this submenu would result in a listing of the five submenus in this category (e.g., Standard Shipping, Next Day Air), and the correct selection would be second day air. The final screen in this sequence would then appear (Figure 2). Clicking on the checkbox adjacent to "Computers" signifies the selection of information (which, for this example, represents the appropriate selection) to be contained in the e-mail reply to the customer.
|
|
|
Characterization of the e-mails
Each of the eight experimental work sessions contained 40 e-mails, and the participants were instructed to process as many of these e-mails as possible. The 40 e-mails associated with any given session had similar versions in the other seven sessions. Thus there were eight versions of 40 different e-mails. The e-mails in these different versions were worded differently, sent from different customers, and differed in their relative locations within the e-mail inbox. For example, a different version of an e-mail may have inquired about a particular feature of a different product but one that was in the same product category. For each version, the type of information constituting a correct e-mail response was essentially the same.
From the standpoint of information processing demands, the 40 e-mails in any session could be partitioned on the basis of each of the following three characteristics: (1) the number of items of information the customer requested within an e-mail, with each e-mail characterized according to whether it required one answer (single-selection e-mails) or more than one item of information to be selected (multiple-selection e-mails); (2) the number of sections of the database that the customer's e-mail required the user to search, with each e-mail characterized according to whether it required the user to consult only one section of the database (single-section e-mails) or make selections from more than one section of the database (multiple-section e-mails); and (3) amount of text, with each e-mail characterized according to whether the amount of text in the customer's e-mail was considered succinct, moderate, or excessive. Realistically, queries composed in the form of e-mails can range in style from terse solicitations for information to rambling discourses that require more focused attention on the part of the reader to identify relevant information. On this basis, we classified the amount of text category into succinct, moderate, or excessive text levels through consensus of three of the investigators. The e-mail illustrated in Figure 1 would be classified as succinct, multiple selection, and multiple section.
Thus in addition to analyzing task performance over the entire set of e-mails, these three different ways of partitioning the set of e-mails enabled us to analyze task performance on the basis of characteristics of e-mails that may be capable of imposing different types of information-processing demands. Note that these different e-mail categories are not necessarily mutually exclusive. For example, a multiple-selection e-mail may be a single-section e-mail, but a multiple-section e-mail must be a multiple-selection e-mail.
Setting and equipment
Training as well as task performance across all eight sessions was performed at workstations consisting of a desk, an adjustable chair, and a personal computer containing a 1.2-GHz Intel Celeron Processor, 19-in. (48.26 cm) monitor set at 1024 x 768 pixel resolution, mouse, expanded keyboard, and Windows 2000 Professional operating system. Prior to the start of the study, the assistant helped each participant adjust his or her chair and computer to the participant's satisfaction. Each participant used the same workstation for the duration of the study. All the workstations were contained in a single room designed to represent an office area.
Task performance measures
Each customer e-mail required one or more selections of information (i.e., answers). For an e-mail to be considered "correct" (i.e., responded to correctly), the information selected by the participant had to exactly match the required selections for that e-mail, implying that no information was omitted and no additional information, even if it addressed the subject matter in the customer's e-mail, was provided.
For each session, we computed six measures of task performance. We considered two measures of overall output: the total number of e-mails responded to (i.e., processed) and the percentage of correct e-mails. We defined the latter measure as the number of correct e-mails processed correctly divided by the total number of e-mails responded to, which provided an indication of overall output quality adjusted for overall output level. We computed two error rate measures: the wrong-selection rate and the overselection rate. We computed each of these measures relative to the total number of e-mails that the participant responded to in that session. We defined the wrong-selection rate as the total number of required (i.e., prespecified) selections omitted (either because incorrect selections were made in place of the correct selections or because an insufficient number of selections were made), divided by the total number of required selections. Overselection of information occurred when more selections were made than were required, regardless of whether these additional selections pertained to the query. Thus if an e-mail called for two specific selections to be made and the participant made three, one overselection would be attributed to this e-mail. We defined the overselection rate as the total number of overselections made, divided by the total number of required selections. Finally, we considered two measures of performance efficiency: the mean search time (in minutes) per e-mail and navigational efficiency. We defined the measure of navigational efficiency as the minimum number of steps or moves necessary for correctly processing an e-mail, divided by the actual number of steps taken, and we computed this only for those e-mails processed correctly.
Procedures
Full participation in the study required 6 days. On Day 1, candidates who met the prescreening criteria arrived to the study site for group testing by certified assistants on a cognitive battery consisting of 20 tests (Czaja, Sharit, Charness, Fisk, & Rogers, 2001). The testing lasted between 4 and 5 hr. The participants were also given a set of questionnaires to be filled out at home that consisted of a demographic and health questionnaire, a self-efficacy questionnaire, a technology and computer experience questionnaire, a computer attitude questionnaire, and a computer anxiety questionnaire. On Day 2, assistants administered ability tests that required individual testing (e.g., tests for hearing and near and far vision); these tests lasted about 40–60 min. We also followed this protocol at two other sites, Georgia Institute of Technology and Florida State University, as part of a multisite investigation of issues in aging and interaction with technology (Czaja, Sharit, Charness, et al., 2001). In this study, we examine the following cognitive ability measures: working memory (Alphabet Span; La Pointe & Engle, 1990), attention–concentration (Trailmaking Test Form B; Reitan, 1958), verbal ability (Nelson-Denney Reading Comprehension; Brown, Fischo, & Hanna, 1993), perceptual speed (Digit Symbol; Wechsler, 1981), memory span (California Verbal Learning Test; Delis, Kramer, Kaplan, & Ober, 1987), and psychomotor speed (Simple Reaction Time Task; Wilkie, Eisdorfer, Morgan, Lowenstein, & Szapocnik, 1990). On Day 2, assistants also administered the Hogan Personality Inventory (Hogan Assessment Systems, 1995), a work involvement questionnaire (Warr, Cook, & Wall, 1979), and a work alienation questionnaire (Kanungo, 1982), which were followed by a 40-min lunch break.
Assistants then trained the participants on the task in groups of five or six, with each participant seated at his or her own computer workstation. The assignment of participants to training groups was random and the training groups were composed of both younger and older participants. Assistants first gave participants an orientation to the computer, which was followed by formal training on the customer service task. The training process took about 2 hr and was guided by a facilitator who was a senior research assistant and a member of the University's medical school staff. The facilitator had extensive experience interacting with older adults in research studies and had an integral role in task development, including the development of the detailed training script that was used to guide the training process. The facilitator used a part-task approach to training (Fisk, Rogers, Charness, Czaja & Sharit, 2004), in which the task was broken down into manageable units or modules. For example, the participants were first exposed to the overall structure of the database and then to each of the database sections. This material was followed by sample customer e-mails tailored to each of the individual database sections, and finally by e-mails that required navigating across different sections of the database. A training assistant moved between workstations to provide assistance on the basic task concepts, interface tools, and sample e-mails.
Following training, all the participants in the group performed six practice e-mails completely on their own. Computer-based performance feedback was provided on several performance measures, including the number of e-mails responded to without any errors, the percentage of required selections correctly matched, and the number of overselections made. In addition, the facilitator conducted a review session that worked through the solution to each of the practice e-mails and provided the opportunity for addressing both general and specific questions. The study participants were also informed that this job could be performed from any location where a computer with an Internet connection is available, and that if they actually held this job they would be able to work from their homes.
On each of the next 4 days the participants performed two sessions of the task, completing as many of the 40 e-mails as possible in each 2-hr session. A 30-min refresher training preceded Session 1, and on each "work day" sessions were separated by a lunch break. On each day, assistants administered the modified Stress Arousal Checklist (Cruickshank, 1984) prior to the first session and following completion of each of the two sessions, and they administered the NASA TLX Workload Scale (Hart & Staveland, 1988) following completion of the second session. On the last day the participants also completed a job motivation questionnaire (Warr et al., 1979), a modified version of the Minnesota Satisfaction Questionnaire (Weiss, Dawis, England, & Lofquist, 1967), and the Task, Job, and Role Characteristics module of the Michigan Organizational Assessment Questionnaire (Seashore, Lawler, Mirvis, & Cammann, 1982). Assistants then conducted exit interviews with each participant. We adopted these measures, as well as the measures of personality, work involvement, and work alienation, for this study because of their relevance to work-related issues. We plan to report these data elsewhere; the focus in this article is on the performance data.
Five of the participants who began training did not complete the study. Reasons for dropping out included medical problems and, in two instances, lack of interest in the study. The participants who dropped out did not differ in any significant way from those who completed the study, and analyses of the data include only those participants who completed the study.
|
RESULTS |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
For the within-subjects factor, we used Wilks's lambda criterion for tests of significance. We performed a further analysis of significant interaction effects by using independent t tests for evaluating differences between age groups and the two e-mail categories that each contained two levels; for the amount of text e-mail category, we used Scheffe's test for evaluating differences between the three levels. We used the Bonferroni t statistic for evaluating differences across days. We report the results of statistical tests in terms of p values and effect sizes.
For these analyses, each level of the Days factor consisted of the mean value of the two sessions performed on that day. We provide means (unadjusted) and standard deviations for all six of the performance measures on each of the 4 days for the overall set of e-mails (Table 1) and for three of the performance measures (representing overall output, error rate, and performance efficiency) for each of the three e-mail categories (Tables 2–4).
|
|
2 =.439), and age group, F(1, 48) = 6.13, p =.017, 2 =.113, with the younger participants performing better than the older people and improvement occurring across each of the first 3 days but not on the fourth day. However, for the percentage of correct e-mails (which adjusts for the amount of output), although we still observed significant improvement across days, F(3, 46) = 15.00, p <.001, 2 =.494, age group differences were absent. Second, although we found no significant effects for the overselection rate, we observed significant improvement over days, F(3, 46) = 14.57, p <.001, 2 =.487, for the wrong-selection rate. These findings, including the much greater variability observed for the overselection rate measure, suggest that these two error measures reflect different aspects of performance, and that the phenomenon of overselection may require additional training intervention strategies. Third, although search time was significantly less for the younger people, F(1,48) = 7.09, p =.011, 2 =.129, and significant improvement in search time occurred across each of the days, F(3, 46) = 19.17, p <.001, 2 =.556, there was also a significant Age x Days x Covariate interaction, F(3, 46) = 3.19, p =.032, 2 =.172. This finding suggests that the increased improvement with practice observed for the older participants relative to their younger counterparts was mediated by prior experience with computers, with the less experienced people exhibiting greater improvement. Given that the older participants had significantly less computer experience, this finding implies that the experience with computers gained through performance on this task had a greater impact on the older participants. Finally, although improvement in navigational efficiency occurred each day, F(3, 46) = 35.95, p <.001, 2 =.706, we found no significant age group differences. Note that we computed this measure only for problems responded to correctly, thus suggesting that age is not a factor in navigational efficiency when the user has an appropriate "model" for problem solution.
The E-Mail Categories
In addition to the analysis of the overall set of e-mails, we conducted separate analyses for each of the three e-mail categories. Given the strong performance improvement (i.e., days) effect found for the analyses performed on the overall set of e-mails, we did not find it surprising that the addition of the e-mail category factor resulted in a large number of significant days by e-mail category interaction effects. Therefore, we only summarize the reporting of any further analyses of these interaction effects.
The selections (single vs. multiple selections) e-mail category
We found significant effects for the percentage of correct e-mails for age group, F(1, 96) = 4.07, p =.046, 2 =.041, days, F(3, 94) = 26.66, p <.001, 2 =.460, selections, F(1, 96) = 5.17, p =.025, 2 =.051, and the Days x Selections, F(3, 94) = 5.73, p =.001, 2 =.155, and Days x Age Group x Selections, F(3, 94) = 3.20, p =.027, 2 =.093, interactions. E-mails that were responded to were more likely to be correct if they required only single as compared with multiple selections. In addition, although we observed significant improvement across days on the single-selection e-mails for both age groups, we found significant improvement on the multiple-selection e-mails only for the older participants. For the wrong-selection rate measure, we found significant effects for days, F(3, 94) = 23.84, p <.001, 2 =.432, selections, F(1, 96) = 12.80, p =.001, 2 =.118, and the Days x Selections interaction, F(3, 94) = 8.10, p <.001, 2 =.206. We found stronger improvement effects for single-selection as compared with multiple-selection e-mails. (Note that the significantly higher error rates found for single-selection relative to multiple-selection e-mails, Table 2, though seemingly counterintuitive, are in fact logical because multiple-selection e-mails provide the possibility for achieving higher rates of matching the correct selections if these e-mails are responded to correctly.) An analysis of search time indicated significant effects of days, F(3, 94) = 34.59, p <.001, 2 =.525, age group, F(1, 96) = 12.24, p =.001, 2 =.113, selections, F(1, 96) = 43.16, p <.001, 2 =.310, and the Days x Age, F(3, 94) = 4.02, p =.010, 2 =.114, and Days x Selections, F(3, 94) = 8.12, p <.001, 2 =.206, interactions. Whereas the younger participants demonstrated, as we expected, lower search times than the older participants, the older people exhibited greater improvement in search times across days. Furthermore, the improvement in search times across days was more pronounced for the single-selection e-mails as compared with the multiple selection e-mails. Finally, we found significant effects of days, F(3, 94) = 29.72, p <.001, 2 =.487, selections, F(1, 96) = 6.28, p =.014, 2 =.061, and the Days x Selections interaction, F(3, 94) = 3.06, p =.032, 2 =.089, for navigational efficiency, with more pronounced improvement on this measure for the single-selection e-mails.
The sections (single vs. multiple sections) e-mail category
For the percentage of correct e-mails, we found significant effects for days, F(3, 94) = 17.96, p <.001, 2 =.314, age group, F(1, 96) = 4.57, p =.035, 2 =.045, sections, F(1, 96) = 7.99, p =.006, 2 =.077, and the Days x Sections interaction, F(3, 94) = 10.02, p <.001, 2 =.242. E-mails that were responded to were more likely to be correct for the younger participants, and when the e-mails required only single as compared with multiple sections to be searched. In addition, improvement across days was more pronounced for the single-section e-mails. For the wrong-selection rate, we found significant effects for days, F(3, 94) = 20.26, p <.001, 2 =.393, age group, F(1, 96) = 4.35, p =.040, 2 =.043, and the Days x Age, F(3, 94) = 3.12, p =.030, 2 =.091, and Days x Sections, F(3, 94) = 10.96, p <.001, 2 =.259, interactions. We found the significantly lower error rates observed for the younger as compared with the older participants only on Days 1 and 2. Furthermore, improvement was more pronounced for the single-section e-mails. We found a significant effect of sections, F(1, 96) = 4.54, p =.036, 2 =.045, for the overselection rate measure, with multiple-section e-mails being associated with higher error rates. For the measure of search time, we found significant effects for days, F(3, 94) = 29.26, p <.001, 2 =.483, age group, F(1, 96) = 12.65, p =.001, 2 =.116, sections, F(1, 96) = 40.05, p <.001, 2 =.294, and the Days x Age, F(3, 94) = 3.01, p =.034, 2 =.088, and Days x Sections, F(3, 94) = 2.80, p =.044, 2 =.082, interactions. The pattern of results was similar to the selections e-mail category, with the improvement in search times across days now being more pronounced for the single-section e-mails as compared with the multiple section e-mails. However, there was also a significant Age x Days by covariate interaction, F(3, 94) = 3.33, p =.023, 2 =.096. Finally, for this category of e-mails, navigational efficiency was sensitive only to the effect of days, F(3, 94) = 28.24, p <.001, 2 =.474, with significant improvement observed across each of the days.
The text (succinct vs. moderate vs. excessive text) e-mail category
For the percentage of correct e-mails, we found significant effects for days, F(3, 142) = 20.12, p <.001, 2 =.298, age group, F(1, 144) = 7.75, p =.006, 2 =.051, and the Days x Text interaction, F(6, 284) = 11.98, p <.001, 2 =.202. E-mails that were responded to were more likely to be correct for the younger participants, and when the e-mails were characterized by succinct text as compared with moderate and excessive amounts of text. In addition, improvement was more pronounced for e-mails characterized by succinct and moderate amounts of text as compared with excessive text. For the wrong-selection rate, we found significant effects for days, F(3, 142) = 26.81, p <.001, 2 =.362, age group, F(1, 144) = 6.70, p =.001, 2 =.044, and the Days x Age, F(3, 142) = 3.94, p =.010, 2 =.077, and Days x Text, F(6, 284) = 10.96, p <.001, 2 =.115, interactions. Although the younger people had significantly lower wrong-selection rates, we found more pronounced improvement on this error measure for the older participants, especially between Days 1 and 4. For the overselection rate, we found a significant Days x Text interaction, F(6, 284) = 3.04, p =.007, 2 =.060, with improvement on this measure only between Days 1 and 4 for the e-mails with succinct text. An analysis of search time indicated significant effects for days, F(3, 142) = 47.73, p <.001, 2 =.502, age group, F(1, 144) = 21.76, p <.001, 2 =.131, text, F(1, 144) = 9.01, p <.001, 2 =.111, and the Days x Age, F(3, 142) = 3.53, p =.017, 2 =.069, and Days x Text, F(6, 284) = 11.15, p <.001, 2 =.191, interactions. Again, despite exhibiting significantly higher search times, the older participants showed more pronounced improvement on this measure with practice, and both age groups improved more significantly for succinct e-mails as compared with e-mails containing moderate or excessive amounts of text. We also found a significant Age x Days x Covariate interaction, F(3, 142) = 2.73, p =.046, 2 =.054. Finally, we found a significant effects of days, F(3, 142) = 35.54, p <.001, 2 =.429, and a significant Days x Text interaction, F(6, 284) = 2.40, p =.028, 2 =.048, for navigational efficiency, with greater efficiency with practice for the succinct e-mails.
Relationships Among Task Performance and Cognitive Ability Measures
We selected six cognitive ability test measures in order to determine the extent to which relationships existed between cognitive abilities and task performance. As we noted earlier, the cognitive ability domains of interest were working memory, attention–concentration, perceptual speed, verbal ability, memory span, and psychomotor speed (Table 5). We selected these abilities on the basis of an analysis of the cognitive requirements of the task. The only cognitive ability for which we found age group differences was memory span [t(50) = 3.28, p =.002], with the younger participants scoring higher on this measure than the older participants. Given the age ranges associated with the younger and older age groups in this study, we do not find the absence of significant age group differences on most of these ability measures surprising.
|
|
Almost all of the participants were very enthusiastic about the experience, citing as their reasons the equipment (they liked the new computer, the screen size, and the mouse), the software application (they liked the text size, the split screen, the e-mails, and how the application worked), the training experience, and the opportunity to do something novel, stay focused, be challenged, and experience performance improvements. Finally, although a total of 12 participants stated that there was nothing they disliked about the task experience, the two most cited complaints by the participants were the inability to determine exactly what they did wrong during the eight sessions and dissatisfaction with their workstation desks and chairs. Complaints concerning feedback were voiced about equally between the younger and older participants, whereas complaints concerning the workstation were voiced more frequently by the older as compared with the younger participants.
|
DISCUSSION |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
) argue for investigating types of work that can accommodate the needs of older workers. Overall, the results of this study indicated that the participants in both age groups were able to perform an e-mail-based customer service task and that age-group differences on many of the performance criteria were either absent or present only on Days 1 and 2. The older participants also demonstrated marked improvement between Days 1 and 4 on the percentage of correct e-mails, even for the multiple-selection and multiple-section e-mails. These findings provide compelling evidence that the older adults are able to learn to solve relatively difficult problems with increased task practice, even in the absence of immediate feedback. However, the presence in some cases of an Age x Days x Covariate (breadth of computer experience) interaction strongly suggests that the differential rates of improvement were mediated to some extent by the exposure to the computer technology that the older participants received while performing the task. These findings indicate the importance during the early stages of learning of ensuring that older adults are reasonable skilled and confident in using the computer-interactive tools as well as the task procedures. Otherwise, cognitive resources that could immediately be directed toward developing a mental model of the task may instead be allocated to more peripheral concerns related to technology use.
A notable exception to this pattern of results occurred for the measure of navigational efficiency. Significant improvement occurred on this measure between each of the days without any significant or noticeable age-group differences. The absence of age-group differences on this measure is consistent with the results from a previous study (Czaja, Sharit, Ownby, et al., 2001), and it suggests that, within the age range used in this study and considering how navigational efficiency was defined, age does not appear to be a factor in navigational efficiency. This finding is contrary to other studies (e.g., Mead, Spaulding, Sit, Meyer, & Walker, 1990) that found that older adults use less efficient search strategies when searching computer databases. The differences may be due to the fact that, in this task, navigational efficiency was only derived for correct e-mails, and that, for this task, the search was restricted to three sections of a database as compared with searching on more complex databases such as the World Wide Web.
Another important finding was the distinction between the results for the two error measures. Although there was little indication of improvement on the overselection rate measure, the participants demonstrated much greater improvements on the wrong-selection rate measure, and there were some indications that the older participants benefited from practice more than their younger counterparts in remediating this type of error. The general lack of improvement in overselection rate suggests the possibility that participants were deficient in their ability to refine their "situation models" (Zwann, Magliano, & Graesser, 1995). In comprehending text, people typically rely on mental or situation models that provide representations of the situation that go beyond the structure of the text. In this task, these representations allow them to draw inferences regarding the types and amount of information that have to be selected from the e-mails. The results regarding the overselection rate suggest that the use of "training sets" that are representative of a wide variety of e-mail types and styles in conjunction with aiding techniques could possibly help people develop situation models of e-mails that could improve their ability to discern the most relevant information when confronting real-world applications. For example, highlighting techniques during training can emphasize identification of "facets" of text, which may be keywords or phrases, as the basis for information search and retrieval. Computer-aiding techniques can also be designed to demonstrate how these facets of text should be linked to information in the database.
The findings for the overselection rate and the percentage of correct e-mails demonstrate the effects of problem complexity on task performance and strongly suggest that the simpler e-mails are more manageable. In fact, the participants in both age groups, but especially those in the older group, showed decreases in the overselection rate on each of the 4 days in all three of the less complex e-mail subcategories—single-selection, single-section, and succinct e-mails. One implication of these findings is that training strategies directed at older workers (50+) should ensure that competency is first achieved on less complex problems.
The findings also indicate that component cognitive abilities are important predictors of performance of this type of task. These results are consistent with theories that view everyday competence as a manifestation of underlying basic abilities (e.g., Marsiske & Willis, 1998; Willis & Schaie, 1986). They are also in agreement with our previous studies (e.g., Czaja & Sharit, 1998; Czaja, Sharit, Ownby, et al., 2001) that demonstrate the importance of cognitive abilities to computer-based task performance. The data regarding abilities and performance suggest the need for considering performance support devices that can help the user focus his or her attention on what has to be done and keep track of multiple items of information. These types of design interventions offer the possibility for benefiting not only older adults but all uesers.
Additional insights into strategies for improving the design of this task were obtained from the exit interviews. For example, a number of the older participants suggested that it would be helpful to have a glossary of terms that could provide them with more background knowledge. Most encouraging were the findings that the vast majority of the participants would welcome the opportunity to perform this type of work on a telecommuting basis.
Overall, the findings from this study indicated that the older participants were able to learn tasks that reflect the ability to adapt to the technically oriented work environment. The findings also point to the importance of providing people with adequate training on both the technical and the procedural aspects of a job, and suggest design interventions that may be beneficial to all workers performing this type of task. Finally, the findings provided valuable insights into how to translate this research to the person's home (i.e., real-world) environment, where the user would be dependent on an online training program for gaining expertise on this task and could access the system for any length of time.
In conclusion, this telecommuting task paradigm enabled the participants' performance characteristics to be evaluated under relatively controlled "situated learning" conditions whereby knowledge and skills were imparted in realistic task contexts (Glaser & Bassock, 1989). This strategy reflects an ecological perspective to experimental research (Czaja & Sharit, 2003), and it has as its ultimate goal the optimization of design outcomes by systematically combining experimental and field studies (Kantowitz, 1992).
|
|
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Received for publication August 15, 2003. Accepted for publication May 24, 2004.
|
References |
|---|
|
TOPAbstractMethodsResultsDiscussionReferences |
|---|
This article has been cited by other articles:
 |
|
 |
|
  S. N. Nair, S. J. Czaja, and J. Sharit A Multilevel Modeling Approach to Examining Individual Differences in Skill Acquisition for a Computer-Based Task J. Gerontol. B. Psychol. Sci. Soc. Sci., June 1, 2007; 62(suppl_Special_Issue_1): 85 - 96. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
