Low-risk community sample; large to start with (10,479), but attrition was high for the reference standard evaluation phase; 184 at the end; 14% of high-risk sample.
Yes, population study; mailed to almost 6,000 and got a 25% return rate. 699 filled out developmental and ASD screening questionnaires after child’s third birthday.
How representative was the sample?
These parameters were not reported.
Although the sample was diverse, there were a disproportionate number of Caucasian and highly educated families responding to later phases of the screening study.
Were there exclusion criteria based on other disabilities?
They specified that no exclusion criteria were exercised for either the population sample or the follow-up.
It was specified that children born preterm were excluded.
Screening instrument
Was there anything about how the screener was administered that would be different from its intended use in a nonresearch, community setting?
No.
No.
Were there any issues regarding the way it is scored in the study?
Note that the ITC can be failed in four different ways—low score on either or both of two subscales, total score; there may be differences in true and false positives given the source of fail criterion.
The authors explored predictive validity based on several different ways of using subscales scores and total score.
Reference standard
Did all children receive a BED from in-person evaluations? How extensive was the information available to the clinician making the Best Estimate Diagnosis?
Cognitive, ADOS-T, and ADI-R; children seen every 6 months up to three years of age. They evaluated children every 6 months and gave “at-risk” dx’s of ASD from 12 to 18 months, “provisional” dx’s from 19 to 31 months, and established dx’s from 32 to 36 months with ADI-R. Five children with provisional dx’s no longer had dx at the last evaluation.
Mixed—some children brought in for Best Estimate Diagnosis including all information, ADOS, and occupational therapy evaluation (). Three others were determined to have ASD based on diagnostic evaluations submitted by parents. Those evaluations all used the ADOS.
Were the reference standard evaluators blind to the screener risk status of the children?
Not reported.
Yes.
What diagnostic outcome categories were used to test prediction from screener to reference standard?
ASD, LD, DD, and no diagnosis. LD and DD defined by Mullen Scores, “other” by parameters such as motor delay.
ASD, other DDs’ diagnosis, or treated through EI services, developmental concerns (no diagnosis but concerns), and no concerns.
Timing and flow
Was there excessive attrition through any phase of screening and evaluation?
Out of 10,479, there were 1316 fails. Out of those, only 346 were referred for testing by the researchers, with a list of practical reasons why the others might have been missed. Out of 346 they lost another 232 for a variety of reasons, so in the end they worked with 184 high-risk children plus 41 TD children referred as a comparison group.
No.
Were there conditions besides attrition that filtered the negative and positive screens from the original screening to the reference standard diagnostic testing phase?
No.
No issues—they were able to make some assessment of developmental status of all 699.
Evaluation
How were performance/predictive values calculated?
They combined ASD with other DDs to calculate PPV because they were considering the ITC a broadband screener.
They were not able to see the FYI negatives in person but did have parents report diagnoses, EI services, developmental concerns, and two parent-rated screening questionnaires for DD and ASD symptoms.
Was performance/prediction for younger versus older children explored?
Screened at 12–15 months. But their breakdown showed that diagnosis was less stable at 12–18 months and became more stable towards 24 months.
N/A—all screened at 12 months.
Performance
What were the performance/predictive values?
PPV = .75 for all disabilities. PPV = .20 ASD alone.
Total score PPV = .14. NPV = .99. Se = .44. Sp = .97.
What was the developmental level of children detected?
IQs ranged widely but did include higher functioning children: MSEL Composite (M = 100; SD = 15). M = 78.6. SD = 17.5. Range = 49–106.
Sample size includes higher-functioning children but difficult to characterize because 9 children had ASD and only 6 had Mullen Composite scores; four were average or higher and two were very low. M = 94.7. Range = 62–127.
Of the false positives for ASD, what proportion had other developmental or learning disabilities?
69.7%.
65% (who met total score cutoff). 85% (who met two-domain cutoff).
ITC = Infant-Toddler Checklist; ASD = autism spectrum disorder; FYI = First-Year Inventory; BED = Best Estimate Diagnosis; ADOS-T = Autism Diagnostic Observation Schedule-Toddler Module; ADI-R = Autism Diagnostic Interview-Revised; dx = diagnosis; LD = language disorder; DD = developmental disability; EI = early intervention; TD = typically developing; PPV = positive predictive value; NPV = negative predictive value; Se = sensitivity; Sp = specificity; MSEL = Mullen scales of early learning; M = mean; SD = standard deviation.
