| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|
| ||||||||||||||||||||||||
RESEARCH ARTICLE |
a Healthcare Technology Systems, LLC, Madison, Wisconsin
b Texas Tech University Health Sciences Center, Lubbock
James C. Mundt, Healthcare Technology Systems, LLC, 7617 Mineral Point Road, Suite 300, Madison, WI 53717 E-mail: Mundj{at}healthtechsys.com.
Decision Editor: Toni C. Antonucci, PhD
| Abstract |
|---|
|
|
|---|
THE tragic personal and social costs of dementia are staggering, and they can be expected to continue to rise with the aging of the "baby boom" generation. About 4 million Americans presently suffer from Alzheimer's disease (AD), the most common form of dementia, and fewer than 1 in 4 receives treatment. It is estimated that more than $100 billion is spent annually on AD, making it the third most expensive disease in the United States, following only heart disease and cancer. The prevalence of dementia due to AD or other diseases among persons more than 65 years old is roughly 15%, and the average lifetime cost per patient is presently $174,000. The unfortunate reality, at present, is that the cognitive and functional losses resulting from AD are permanent and irreversible. Promising treatments are available, however, that appear to slow the progressive deterioration associated with the disease. Such treatments can prolong personal independence, quality of life, and reduce total costs by delaying the need for institutionalization.
The most critical element in maximizing the benefits of current treatment options is the initiation of treatment as early as possible after disease onset (Cummings and Jeste 1999
; Duncan and Siegal 1998
). Presently, clinicians fail to recognize nearly 75% of moderate to severe dementia patients and more than 95% of patients in early stages of dementia (Gifford and Cummings 1999
). Often, up to 4 years may pass between the onset of symptoms and the time a diagnosis is made and treatment initiated (Larson, Reifler, Featherstone, and English 1984
). Diagnosis of dementia and grading of severity is a complex and imprecise clinical process. Postmortem autopsies of cerebral tissue are needed for definitive confirmation of specific pathologies. Clinicians use a broad array of objective tests of neurological functions and subjective assessments of functional capacity, in conjunction with other clinical information and laboratory tests, to diagnose and grade dementia, including such instruments as the Mini-Mental State Exam (MMSE; Folstein, Folstein, and McHugh 1975
), the Blessed Dementia Rating Scale (Blessed, Tomlinson, and Roth 1968
), the Clinical Dementia Rating Scale (Hughes, Berg, Danziger, Coben, and Martin 1982
), and the Global Deterioration Scale (Reisberg, Ferris, de Leon, and Crook 1988
). Clinical instruments such as these are important for maintaining consistent measurement standards and scientific nomenclature used to assess treatment outcomes and stage progressive dementia. These instruments are difficult to implement for general population screening, however, because they require face-to-face administration by a well-trained clinician.
Some memory abilities decline as a natural function of aging. Subjective memory complaints, however, may predict the earliest stages of dementia onset, particularly when accompanied by objective indicators of memory deterioration (Morris et al. 1991
; Schmand, Jonker, Hooijer, and Lindeboom 1996
). Many individuals present to clinicians for evaluation because of difficulties conducting daily activities that they previously had little difficulty performing. Individuals may delay seeking medical intervention for themselves or others close to them because of fear, personal embarrassment about symptoms, restricted access to medical care, or lack of knowledge about the existence of effective treatments that could preserve life quality and functional independence for a significant period after symptom onset. It is possible that such delays in identification, diagnosis, and treatment could be decreased by 50% or more through greater public education and effective screening.
Over the past two decades it has been demonstrated that survey instruments assessing the capacity to perform routine daily activities, such as managing financial records, playing games, shopping or preparing meals, and remembering appointments and medications, correspond well with clinician-based assessments of functional capacity (Morris et al. 1991
; Pfeffer et al. 1981
; Pfeffer, Kurosaki, Harrah, Chance, and Filos 1982
). Thus, general questionnaires on functional ability to perform routine daily activities may provide sensitive and specific screening devices for early detection of dementia in the general population. Hershey, Jaffe, Greenough, and Yang 1987
found that informant-based assessment of functional activities was 92% sensitive and 87% specific for detecting patients with vascular dementia, and a study of Chinese adults in Shanghai (Hill et al. 1993
) demonstrated a similar questionnaire to be 84% sensitive and 85% specific for detecting dementia.
In the present research we developed and validated a short dementia screening questionnaire, empirically derived from neuropsychological research data, that could be widely administered in the general population. We required that the instrument manifest adequate sensitivity, specificity, and positive predictive validity to support its implementation. We also designed the instrument so that it could be implemented accurately and consistently by lay persons with minimal clinical training or expertise.
| Methods |
|---|
|
|
|---|
Research Diagnoses of AD
Detailed symptom and medical histories were obtained from an informant (typically a spouse) for patients evaluated at the dementia clinics. Patients presenting to the dementia clinics were interviewed further to obtain complete medical and personal histories. In addition, extensive neuropsychological testing, including the MMSE, immediate and delayed word recall (Morris et al. 1989
), constructional praxis, controlled oral word association (Spreen and Benton 1977
), and the Boston Naming Test (Kaplan, Goodglass, and Weintraub 1983
), was conducted. The resulting data allowed diagnostic classification of these individuals for use in research. In determining the research diagnosis, six criteria were examined: (a) insidious and gradual onset of symptoms; (b) progressive decline over a period of two years or more; (c) evidence of 2 or more cognitive deficits that include memory impairment; (d) no disturbance of consciousness; (e) no significant contribution of psychiatric conditions to impairment; and (f) no significant contribution of medical conditions to impairment. Individuals meeting five of these six criteria were classified as possible AD patients, and those meeting all six criteria were classified as probable AD. Two qualified medical professionals independently evaluated patient classifications for research diagnoses.
As a result, a substantial sample of patients had data available from both the informant-based telephone interview and the medical evaluation follow-up from the dementia clinics. Because our goal in the present research was to develop a screening instrument that was sensitive and specific to early dementia, we excluded 28 patients classified as suffering from dementia other than AD and 75 patients identified as having mild cognitive disorders not associated with AD from subsequent analyses. Using the classification criteria noted previously, we included 272 patients extracted from the dementia screening clinic database for instrument development. This sample included 88 nondemented, 73 possible AD, and 111 probable AD patients.
The mean age of the complete screener development sample was 74.0 years (34.594.6 years,
) with an average of 12.6 years of education (129 years,
). There were 172 (63.2%) women and 100 (36.8%) men. Of those classified as possible or probable AD patients, the average age at initial informant-reported symptom onset was 73.2 years (45.893.1 years,
), producing an average duration since informant concern of 4.3 years (0.325.3 years,
). The mean MMSE was 17.4 (330,
). In the nondemented patients, the average age at initial informant-reported symptom onset was 62.9 years (29.581.4 years,
), duration of symptoms since initial informant concern was also 4.3 years (0.432.5 years,
), and the mean MMSE was 28.6 (2330,
).
Symptom Extraction From Informant Reports
As indicated previously, a semi-structured interview with a close informant or caregiver had been conducted by telephone before the neuropsychological evaluation at the dementia clinics and assignment of a research diagnosis. During this interview, the informants were asked to report signs of problems that they were concerned about and the sequence and timing of their appearance. Informant reporting of symptoms was open ended (i.e., not cued by the interviewers). These symptoms included behavioral problems (such as agitation, insomnia, or pacing), disorientation in time or location, problems with language, memory difficulties (such as remembering people, objects, or recent events), impairment of instrumental daily activities (such as driving or maintaining financial records), and appearance of psychiatric symptoms (such as delusions, hallucinations, or personality changes). The interviewers consistently coded reported symptoms and entered them into a database. Thus, each patient in the database with a research diagnosis, derived from medical and symptom history and neuropsychological testing done at the clinic, was coded as experiencing 1 or more of 43 distinct symptoms spontaneously reported by an informant.
We then analyzed the resulting symptom data, along with patient age and education information, using a machine learning algorithm (Quick, Unbiased, Efficient Statistical Tree [QUEST]; AnswerTree software, SPSS, Inc., 1998) to extract the information that maximized discrimination between the nondemented patients and the patients with possible or probable AD. This algorithm, like classification and regression tree analysis, is a binary treegrowing process that recursively partitions data into increasingly homogeneous subsets. The result is a series of hierarchical binary decisions with a single variable at each decision point to maximize separation between groups on the basis of a given target variable (such as the derived research diagnosis in this example). The initial exploratory analysis created a complex decision tree with 32 independent partitionings producing 31 terminal nodes (final decisionclassification points), some of which required addressing up to eight conjoint binary decisions to resolve. This "optimal" classification scheme, which used as much information as possible from the development sample, indicated a potential sensitivity of 92.4% (of possible or probable AD patients classified as such) and specificity of 80.7% (of nondemented patients so classified). The derived decision tree also indicated an optimal positive predictive value of 92.9% (likelihood of having AD diagnosis given a positive screen) and a negative predictive value of 79.8% (likelihood of not having an AD diagnosis given a negative screen). Interestingly, of the more than 40 variables available to the machine learning algorithm for maximizing discriminability between patient groups, only 8 informant-reported symptoms were extracted from the analysis in deriving the binary decision tree.
Item Extraction for Screening Instrument
The complex decision tree resulting from this exploratory, data-fitting statistical procedure was not intended for direct derivation of the new dementia screening instrument. We anticipated that the machine learning algorithm would (a) exploit all of the available information, (b) capitalize on any chance relationships in the data, and (c) derive decision rules for patient categorization that were excessively complex for practical application. The valueand purposeof the analysis was to identify the reported symptoms of greatest informational value for discriminating the patient groups, to provide structure for guiding focused discussions with dementia experts, and to provide estimates of the upper bounds for potential screening instrument performance using this type of information.
We printed out the analysis and complete binary decision tree resulting from application of the machine learning algorithm and used the results as a focal point for discussion during a day-long meeting that involved a gerontologist, a neurologist, a psychiatrist, and a neuropsychologist. Each of these experts provided unique perspectives and contributions to the discussion. The objective of the meeting was to integrate their clinical expertise with the empirical analysis of the database to produce a dementia screening instrument applicable to the general public that did not require clinical expertise or training to administer. At the conclusion of the meeting, a series of 11 questions was derived in a "yes/no" format ("don't know" was also an available response if informants were not able to assess existence of a given symptom) to serve as a Symptoms of Dementia Screener (SDS) (Table 1 ). The a priori scoring of the SDS, arrived at by consensus among the dementia experts, was that three or more affirmative answers to the set of questions would signify a positive dementia screen.
|
The validation sample included 60 women and 43 men, ranging in age from 46.6 to 100 years old (
). Educational attainment ranged from 0 to 23 years of formal education (
). The mean MMSE score was 21.8 (130,
). The mean age of initial informant-reported symptom was 70.3 years old (16.394.8 years,
), producing an average time between initial informant concern and clinical evaluation for dementia of 5.9 years (1.135.2 years,
). Symptom onset and duration data were derived from the initial open-ended telephone screening conducted before evaluation at the dementia clinics and thus reflect both dementia-related and chronic medical conditions that skew these distributions. Across the complete sample, 25 patients were found not to have significant cognitive impairment, 17 were diagnosed with mild and restricted cognitive impairment (typically due to the presence of depressive symptoms), 20 were diagnosed as possible AD (5 of 6 criteria), 35 were diagnosed as probable AD (6 of 6 criteria), and 6 were diagnosed as experiencing dementia other than AD.
| Results |
|---|
|
|
|---|
). The average single item score to total score correlation was .52 (
). Internal consistency measures of scale reliability, both Cronbach's alpha and a two-way mixed model computation of the intraclass correlation (people effect random, measurement effect fixed), were .80.
For the analyses of SDS performance reported later in this section, we combined the patients without indications of cognitive impairment (
) with the patients with mild and restricted cognitive impairment (
) to create a control group of 42 patients for comparison with the other 61 patients. We combined the unimpaired and mildly impaired patient groups into a single control group because the mildly impaired patients were not suffering from dementia. Thus, we believed that these patients should be grouped with other control patients. In addition, this decision had a pragmatic component, namely to produce two samples of approximately equivalent size so that misclassification of any given patient would not unduly influence the derived sensitivity or specificity estimates. Subsequent comparisons of the data between patient groups supported the pragmatic division of the validation sample. Geriatric depression is an important medical concern (prevalence was 52.4% across the complete sample), but was neither the focus of this study nor a consideration during screener development. In the control sample, 85.7% of the patients were clinically depressed, compared with only 29.5% of the 61 dementing patients. In addition, there were not statistically significant differences in the SDS scores of the two patient groups that were combined to make the control group, but these groups did have significantly lower scores than the other three patient groups, which did not differ from one another (Tukey LSD post-hoc comparisons,
). The use of an aggregated control group that includes mildly impaired patients does have important implications for interpreting the data obtained and further validating this instrument in the future. These implications are presented in the discussion. This grouping would be expected to bias the study against producing significant results and to underestimate specificity estimates.
First we investigated the performance of the a priori scoring criterion of three or more positive responses as the criterion for a positive screen. The results indicated that 59 of the 61 patients with possible or probable AD or other dementia diagnoses had a score of 3 or more on the SDS, but so did 25 of the 42 control patients. Thus, more than half of the "control" patients screened positive using the a priori criterion. The poor specificity of the screen was not due to inclusion of the restricted cognitive impairment patients with the unimpaired patients, because the percentage of patients screening positive in these two groups was virtually identical (58.8% and 60.0%, respectively). Although the analysis of applying the a priori scoring criterion does indicate that benefits would accrue from its use, because 70.2% of positive screens were in the dementia group and 89.5% of those screening negative were in the control group (positive and negative predictive values, respectively), the large number of "false positive" screens strongly suggests that a stricter scoring criterion would produce more favorable results. The results of using the a priori scoring criterion are shown on the first line of Table 2 .
|
). Inspection of the ROC curve suggested that optimal sensitivity and specificity would be obtained from use of a cutoff score of 5 or more for a positive screen. The sensitivity, specificity, and positive and negative predictive values of applying minimum cutoff scores of 4, 5, and 6 positive responses as criteria for a positive screen in the validation sample are shown in the second, third, and fourth lines of Table 2 . Comparative values for the MMSE using a 23/24 cutoff value are also provided. Table 3 presents the correlations between SDS and MMSE scores with the sociodemographic characteristics and performance on the neuropsychological tests across the complete validation sample. The correlation between SDS and MMSE scores was -.68, p < .001.
|
The apparent importance of these three items led to further explorations of the data focusing on the informational value of these questions. Results indicated that a simple four-item summative scale composed of these items plus Item 2 (short-term memory problems) formed a coherent, potentially useful scale. Internal consistency measures of scale reliability, both Cronbach's alpha and the intraclass correlation, were .73, and the average single item to total score correlation was .61 (
). The ROC curve for discriminating the two patient groups had an AUC of 0.946 with a standard error of 0.024 (95% CI = .899, .993), and the optimal decision point for determining a positive screen was endorsement of two or more of the four items. Applied to the validation sample, this simple, short scale indicated potential sensitivity and specificity of 98.4% and 81.0% with positive and negative predictive values of 88.2% and 97.1%.
Because these data were derived from machine learning procedures designed to extract maximal informational value from data, the generalizability of the results to other data sets must be tested by replication. Although it is almost certain that these results capitalized to some degree on chance associations in these data and thus will not perform as well in an independent data set, the potential of such a short, simple four-item "yes/no" summative scale clearly warrants further research.
| Discussion |
|---|
|
|
|---|
Thus, the new SDS instrument developed and validated in this study is not novel or unique in its intent; other similar instruments exist. Several characteristics distinguish the SDS from prior instruments, however. The SDS was derived empirically in a data-driven process that analyzed spontaneous, unprompted informant reports of observed symptoms. The SDS is not intended to be applied as a face-to-face informant interview by a clinician. It is shorter than most of the previously validated instruments, but it includes items that address problems associated with memory, psychiatric symptoms, activities of daily living, and instrumental activities of daily living. The dichotomous "yes/no" response options do not require informant judgments for rating symptom frequency or severity, and although wording of several items implies a symptomatic change over time (e.g., "more forgetful," "started" having trouble or needing help, "become irritable"), no particular time frame or previous reference point is specified. The result is a simple, straightforward questionnaire that is readily understood by informants, requires little cognitive processing for rendering responses, and can easily be administered by mail or telephone.
Early initiation of effective treatment is crucial to preserving functional independence and maximizing the quality of life for the elderly suffering from dementia. Improved and effective screening instruments, such as the 7-min neurocognitive assessment battery (Solomon, Sullivan, and Pendlebury 1998
), are being developed for and used in primary care settings to improve early detection and diagnosis of elderly dementing patients. Such instruments are important for taking advantage of the limited contact physicians have with individual patients, but they may not be used unless a dementia-related problem is suspected. However, caregivers or family members may not communicate concerns about possible cognitive impairment for many reasons ranging from personal embarrassment to inadequate knowledge about "normal" age-related cognitive decline or the existence of effective interventions.
The 11 questions of the SDS (Table 1 ) were derived by incorporating clinical expertise with informant reports of symptom development in patients with mild to moderate dementia. The SDS was designed for use by nonclinical personnel, possibly even to be handed out as a self-explanatory "questionnaire." Strengths of the instrument include administration by nonclinical persons, simplicity of comprehension, and potential anonymity of responses. Evidence from this validation study clearly establishes the SDS's potential utility as a telephone screening instrument. The best methods for implementing the SDS for general population screening and the means for evaluating the impact of such implementation have yet to be explored.
The SDS could be used in a variety of formats. It is readily amenable to use as a paper-and-pencil questionnaire or as a live telephone interview. National and community organizations, such as the Alzheimer's Association, local churches, or service providers for elderly people like Meals on Wheels, could provide front-line support for widespread dementia screening. There are no apparent barriers to computerized implementation of the instrument through either web-based interfaces or touch-tone telephones using interactive voice response technology (Mundt 1997
). Such technologies could facilitate an organized effort for general population dementia screening, analogous to the current National Depression Screening Day (Baer et al. 1995
). The data in Table 2 indicate that simple scoring of the instrument using a cutoff score of 5 or greater could easily be applied and would perform adequately.
To test this assertion, evaluation studies of community-based general population screening programs are needed that incorporate data feedback loops. For example, a regional organization, such as a local chapter of the Alzheimer's Association, could target and disseminate the screeners by telephone or mail to the community and facilitate education and awareness of the project to local treatment providers. Base rate community exposure to the screener would be known, by design. Obtaining screener results would require either data entry of telephone responses or return of completed screeners by mail. Return rates could be maximized by offering compensation directly or indirectly (e.g., donation to facilitating organization). These results would permit estimation of screener score distributions in the community and their relationship to additional sociodemographic information obtained with the screener. To evaluate program effectiveness and screening throughput, cooperation and feedback from treatment providers in the catchment area would be needed. Clinical results from neuropsychological evaluation and other tests would have to be linked to screener results for researchers to fully assess the impact of a population screening program. This information feedback loop would be the most difficult to support logistically. This type of validation research could be conducted completely within a managed care environment. However, potential challenges to sample representativeness and financial disincentives to find patients needing expensive treatments for incurable diagnoses earlier in the progression of the diseases (and therefore for more prolonged periods) may keep many such organizations from initiating such a program. Economic incentives by the rapidly expanding long-term care insurance industry, in order to delay benefit usage, may prompt underwriting of such programs in the future.
Widespread use of the screening instrument, by itself, could produce personal and societal benefits through earlier identification of dementia. Such benefits would be greatly enhanced, however, through integration of the screening instrument with patient education about effective and available treatments for dementia and referral mechanisms to reputable treatment facilities for further evaluation. Organizations such as the Alzheimer's Association already provide such services. Access to such information could easily be built into computer applications with 24 hours/day, 7 days/week availability. A pilot research program with such an aim is currently in progress, funded by the National Institute on Aging.
The simplicity and ease of the SDS may also be helpful in clinical practices. Given the established effectiveness of informant questionnaires (Jorm 1997
), the SDS could easily be given to persons accompanying elderly patients to physician appointments. The results would provide physicians a snapshot of current and recent changes in daily functioning and could alert them to the possible need for further evaluation. Ongoing research is examining this potential in a local gerontology practice, and the evidence suggests that the SDS has greater sensitivity for detecting dementing patients than the MMSE, but with lower specificity, which may reflect the inclusion of mildly impaired patients in the control group of this validation study. We will publish more complete description and analyses of these data after the study is complete.
Implications of the previous data for further SDS validation warrant two comments. First, the processes for obtaining "gold standard" clinical diagnoses are themselves imperfect, particularly in establishing reliable boundaries among normal, mild cognitive impairment and possiblemild dementia. Such diagnostic ambiguities necessarily constrain sensitivity and specificity estimations (Waite et al. 1998
). Second, a false positive screening result for a specific condition, such as dementia, may not represent a false alarm for identifying patients in need of treatment. Early diagnosis and treatment of dementia patients may best be accomplished in a two-step process, in which extensive diagnostic tests follow use of a screener (Stahelin, Monsch, and Spiegel 1997
). In such a process, screener sensitivity may be more critical than specificity, which is more crucial for diagnostic procedures used to determine appropriate treatment. Obviously, "well" patients who are not screened out represent an undesirable burden on health care delivery systems. However, patients receiving needed treatment for medical conditions not specific to the screener design (e.g., depression or mild cognitive impairment) do not represent false case findings with respect to needing treatment.
The primary limitation of the present study is that we used a regionally contrained clinical sample to derive and validate the instrument. The true test of this instrument will require validation in a community setting. Plans for such validation are currently in progress. It is difficult to know how, or why, a sample drawn from West Texas would differ systematically from a broader national sample, but the fact that these patients were already receiving clinical attention raises the possibility that they could differ from the population of early dementia patients who are not seeking information or assistance. This potential source of sample bias reflects the study design, which required the existence of a definitive research diagnosis to evaluate instrument performance.
Analyses exploring further optimization of SDS scoring, and the potential utility of a four-item subscale, may yield benefits in the future. The more complex, conjoint decision rule of the full SDS, suggested by the exploratory analysis of the validation data, might perform better than the simple summation of total items. The scoring process, however, carries a burden of complexity that would likely require use of a computer to avoid errors. Informant responses obtained directly by computer (e.g., by interactive voice response or an internet web page) or recorded on computer-scorable forms could make the additional scoring complexity transparent; however, manual use of the scoring algorithm, for instance in a clinical setting, would be difficult. The potential of a very short four-item screener with sound psychometric and predictive validities holds much more exciting promise. Much like the four-item CAGE questionnaire, which has become a standard screener for identifying alcohol problems in primary care settings (Samet, Rollnick, and Barnes 1996
), the four identified items of the SDS (Items 1, 2, 9, and 10) may someday become the core of a standard primary care screener for early detection of dementia. Given the exploratory manner in which these items were extracted from the present data, however, further speculation along these lines must await replication. Prior studies of very brief informant-based measures, often just a single question, do warrant optimism (see Jorm 1996
).
An interesting issue not addressed in the present study concerns potential adaptation or extension of this instrument for use as a self-screener. A recent direct-to-consumer program conducted by Pfizer/Eisai (AriceptTM) found that 22% of 40,000 respondents were individuals concerned about themselves. Subjective complaints about memory may be a significant predictor of dementia onset, particularly when they are associated with objective measures of memory impairment (Schmand et al. 1996
). Thus, elderly patients concerned about cognitive decline may represent the earliest opportunity for identifying impending onset of dementia. The specificity of the SDS obtained in the present study may thus be a conservative estimate, because the control group was derived from individuals with concerns about their cognitive functioning. A portion of them may actually have represented the mildest detectable group of AD patients and reflect the type of elderly persons most likely to participate in coordinated community screening efforts. Informant reports become more informative as dementias progress, because anosognosia may develop, preventing self-awareness of cognitive deficits (Wagner, Spangenberg, Bachman, and O'Connell 1997
). The point at which self-insight is lost is not known. Self-reported difficulties, corroborated by family members or objective neuropsychological testing, may provide the early warning signals that could permit effective interventions to preserve cognitive abilities, functional independence, and enhanced quality of life.
Early identification, diagnosis, and initiation of treatment for AD and other dementias are important to preserve elderly patients' quality of life and provide time for legal, financial, and medical planning. Community-based screening programs can contribute to such goals, but denial and fear on the part of patients and their families must be addressed if such programs are to be effective. Beneficial treatment options and coping strategies currently exist and are developing rapidly. Unfortunately, public awareness of such advances is limited. Many prospective patients who might benefit from early detection are not brought to medical attention because of perceptions like "nothing can be done, so it's better not to know." In the absence of education programs that address and correct such misperceptions, the effectiveness of a general population screening program would be limited and might serve to heighten the understandable sensitivities and fears of those they are intended to benefit.
| Acknowledgments |
|---|
| Footnotes |
|---|
Dr. Freed is now Chief of Psychology at Oregon State Hospital, Salem.
Received for publication April 29, 1999. Accepted for publication December 7, 1999.
| References |
|---|
|
|
|---|
| ||||||||||||||||||||||||
| HOME | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|---|