Controlled clinical trial results vs. real world observations
BOSTON — Will the treatment work in the real world? That's the issue often raised by the favorable outcome of a formal clinical trial, reports the April 2007 issue of the Harvard Mental Health Letter. It's so important that special terminology has been developed for it: "the gap between efficacy and effectiveness"—efficacy meaning proof in a carefully controlled trial, and effectiveness meaning success in the circumstances of everyday life.
A new study of a program for delinquent youth shows how great the gap is. When the program was conducted in an ordinary school environment, without close monitoring and specially trained professionals, it not only failed to replicate earlier successes but even made things worse in some ways.
Reconnecting Youth is a program for potential dropouts and other troubled high schoolers aimed at improving school attendance, performance, self-esteem, and drug avoidance. In controlled trials, the program reduced truancy and drug abuse, raised grades, and enhanced self-esteem. However, a new study of youths who were assigned to Reconnecting Youth classes conducted by regular teachers showed no effect on drug use, delinquency, or self-control. Furthermore, six months later, program participants were making worse choices about companions and weekend activities.