Biddle A, Watson L, Hooper C, et al. Criteria for Determining Disability in Speech-Language Disorders. Evidence Report/Technology Assessment No. 52 (Prepared by the University of North Carolina Evidence-based Practice Center under Contract No 290-97-0011). AHRQ Publication No. 02-E010. Rockville, MD: Agency for Healthcare Research and Quality. January 2002.
Approximately 42 million Americans have some type of communication disorder, costing the nation $30 billion to $154 billion for lost productivity, special education, and medical care annually. The quality of the numerous evaluation procedures and instruments for clinical decisionmaking about language, speech, or voice disorders influences decisions about access to services and funding (e.g., special education services, Social Security disability income). The RTI-University of North Carolina at Chapel Hill Evidence-based Practice Center conducted a systematic review of the literature to address two key questions about evaluating and diagnosing speech and language disorders in adults and children of particular concern to the Social Security Administration in making disability eligibility determinations: (1) What instruments have demonstrated reliability, validity, and normative data? (2) Do these instruments have predictive validity for an individual’s communicative impairment, performance, or both?
We conducted detailed searches of the English-language literature from 1966 to October 2000 using the MEDLINE, CINAHL, PsycLIT®, ERIC, Health and Psychosocial Instruments, and Cochrane Collaboration databases.
We included all English-language research on 18 instruments for children and adults in which investigators evaluated the instrument’s reliability, validity, or ability to predict future communicative impairment or functioning. Excluded were articles reporting the efficacy or effectiveness of specific interventions that did not provide information on the key questions, articles providing normative data from non-US populations, and all gray literature (i.e., literature not from peer-reviewed sources) except instrument manuals. An independent expert panel knowledgeable in language, speech, or voice disorders had identified the instruments we reviewed.
Data Collection and Analysis
We selected studies from among 1,238 citations using a process of duplicate, independent review of titles, abstracts, and, where necessary, full papers. We abstracted data on 92 articles or manuals, using single abstraction with subsequent review by clinical and methodological experts; reviewers also completed quality rating forms. Criteria used to evaluate reliability, validity, and other data reflect widely accepted or known standards for the psychometric properties of such instruments.
Among language disorder instruments, one (of three) for adults and four (of eight) for children met or nearly met our evaluation criteria for reliability and validity; two child-specific instruments provided data for subpopulations. Although these five instruments had norms, only the child-specific instruments provided nationally representative data. Two (of three) instruments for voice disorders met evaluation criteria; speech disorder instruments did not. Only four studies gave information on prediction of future communicative functioning and impairment.
Reliability and validity data for the majority of instruments rarely came from peer-reviewed literature; instrument manuals yielded most such data. Some manuals provided comprehensive data from well-conducted standardization studies; most did not. Because normative data were usually not derived from nationally representative samples, generalizing results beyond the populations studied was difficult. Sample size and representativeness problems limited the predictive validity studies. Overall, evidence about diagnostic or predictive properties of instruments addressing language, speech, and voice disorders is weak and incomplete at this time. The sparse evidence base suggests a substantial methodologic, clinical, and policymaking research agenda.