This article is an annotation of the following published article:

J Speech Lang Hear Res. 2012 Dec 28. [Epub ahead of print]
A Motor Speech Assessment for Children with Severe Speech Disorders: Reliability and Validity Evidence.
Strand EA, McCauley RJ, Weigand SD, Stoeckel RE, Baas BS.
Source
The Mayo Clinic, Rochester

Introduction

The Dynamic Evaluation of Motor Speech Skill (DEMSS) was developed to aid in the differential diagnosis of both younger children and those with more severe motor planning and programming difficulties such as those demonstrated by children with CAS.  Such an assessment is needed because tests of articulation and/or phonology are not able to capture the speech performance of children with severe speech impairments that significantly limit speech production skills.  In fact, some such children may be considered nonverbal with no intelligible speech.

In establishing the assessment which will be discussed, it was reasoned that in adults, a motor speech exam is often used to establish the presence of speech motor planning deficits.  A motor speech exam enables an SLP to observe speech ability of the client across various speech attempts using stimuli that vary in length and/or complexity and which are systematically organized in a hierarchical fashion.  While used in the adult population, the motor skill exam is used much less in pediatric clients and most published tests are lacking evidence of reliability and validity.

Reliability and validity are critical psychometric properties of a test.  Tests must be reliable so that test scoring and scores are not vulnerable to variations:

–       Upon repeated administration of the test

–       Due to different evaluators

–       Attributed to inconsistency of the evaluators

Validity refers to whether or not the test measures what it claims to measure.  In this present study, the authors were concerned with whether or not the DEMSS is actually a measure of speech motor planning and programming.

What is DEMSS?

Because neither articulation nor phonology tests can differentially diagnosis CAS, the DEMSS was created to examine speech motor planning performance in very young and/or severely speech impaired children.  More specifically, the DEMSS allows the examiner to sample for features that are most associated with CAS in that it examines articulatory accuracy, inconsistency of errors across repeated trials (including vowels) and accuracy within prosodic elements such as lexical stress.

Some features of DEMSS:

• Includes nine subtests and 66 utterances
• Results in 171 judgments related to overall articulatory accuracy, vowel accuracy, prosodic accuracy and consistency
• Yields four subscores that sum together for the DEMSS score
• Scoring occurs during assessment or upon videotape analysis afterward

Because DEMSS uses dynamic assessment, the impact of various types of cueing, the evaluator can make judgments regarding severity and prognosis.  Cues that facilitate accuracy can help direct treatment planning.  This quality of DEMSS is unique among assessment tools currently available.

Administering DEMSS

The administration of the DEMSS requires three main features:

1. With child’s eyes directed to clinician’s face as much as possible, asking the child to imitate utterances.
2. Depending on the quality/accuracy of the child’s response, clinician elicits additional imitative attempts using various levels of cueing.
3. Cueing is used to elicit improved accuracy over repeated trials.  Cueing may be visual, tactile or temporal in nature.

The DEMSS includes multi-dimensional scoring to document the child’s response to cueing as well as rules for determining articulatory accuracy, vowel accuracy, prosody, and consistency amid various forms of cueing.

Basic Research Question

Is the DEMSS a reliable and valid test in the differential diagnosis of childhood apraxia of speech?

Research Methods

In order to establish validity and reliability for DEMSS, the authors studied intra and inter-examiner, test/retest reliability and construct validity (the ability of the test to identify children with speech motor planning difficulty).  They did so in a convenience sample of 81 children consecutively referred for evaluation for speech sound disorders at the Mayo Clinic.  Children were excluded from the study if they had structural issues such as cleft palate, hearing loss, autism, dysarthria, or children for whom English was not their first language.  In regards to cognition, children solely needed to be able to attend to the clinician during the administration of the DEMSS, attempt to imitate and tolerate cueing.  The DEMSS was administered along with the battery of tests typically given at the Mayo Clinic for evaluation of speech sound disorders.   Additional tests included expressive and receptive language, a language sample to obtain mean length of utterance, a phonetic and phonemic inventory and an estimate of syntax and morphology.  Subjects received an oral structural evaluation and an articulation test.  The evaluators made their diagnoses of the child prior to scoring the DEMSS in order to reduce the chance of bias.  Various reliability procedures were employed to analyze test/retest, intrajudge, and interjudge reliability.  Additionally, a number of statistical analyses were applied to the DEMSS scoring including cluster analysis and calculations of sensitivity and specificity to determine construct validity.

Basic Findings

Test-Retest Reliability – 89% mean percentage agreement

Intrajudge reliability – 89% mean percentage agreement

Interjudge reliability – 91% mean percentage agreement

Validity – Cluster analysis revealed three major clusters of children – those with characteristics most often associated with CAS, those with mild CAS, and those with other speech sound disorder.  The probability of correct classification (the ability to discriminate children with or without CAS), based only on their DEMSS, was above 90%.

The authors report data to justify acceptable reliability (test/retest, intrajudge and interjudge reliability) of the DEMSS.  Additionally, the data supports that the DEMSS is valid for discriminating severe speech impairment due to speech motor planning deficits (CAS).

Highlights of This Study

• Evaluators could consistently score with DEMSS in both live and taped samples
• Findings suggest that the test can be feasible to use by experienced SLPs with only “modest” training in DEMSS administration.
• DEMSS is able to distinguish children with CAS characteristics versus those without.
• Clusters of children identified with DEMSS scores resemble groups based on the clinical diagnosis of CAS, mild CAS or other speech sound disorder.
• The total DEMSS score appeared to be the best discriminator of classification.
• DEMSS does not appear to over diagnosis CAS.
• Some children with more mild CAS symptoms may be missed by DEMSS.
• DEMSS is best used as part of an assessment battery.
• DEMSS is best used on younger children and more severely affected children.
• DEMSS uses most agreed to characteristics supporting differential diagnosis of CAS.
• Administration time for DEMSS ranged from seven minutes to 25 minutes.
• An extra bonus for the use of DEMSS is that it can help clinicians estimate severity and also the types of cueing and stimuli likely to be helpful in treatment.
• The study demonstrates that many children, even those with little speech output, can complete the DEMSS by attempting the imitation of test items.
• This test may help identify children who can most benefit from motor learning and speech motor control therapy methods.

[Click here to go to the PubMed abstract of the original article]