- Intervention: A comprehensive youth development program for economically disadvantaged teens, a key component of which is reproductive health care.
- Evaluation Methods: A large, multi-site randomized controlled trial.
- Key Findings: For girls, 40-50% reductions in pregnancies and births three years after random assignment (at average age 17). For boys, no effect on causing a pregnancy. Evidence of effects on high school completion and college enrollment are promising but not yet Top Tier due to study limitations.
Download a printable version of this evidence summary (pdf, 4 pages)
The Top Tier Initiative’s Expert Panel has identified this intervention as Top Tier
The Carrera program meets the Top Tier Evidence Standard, defined per recent Congressional legislation to include: Interventions shown in well-designed and implemented randomized controlled trials, preferably conducted in typical community settings, to produce sizeable, sustained benefits to participants and/or society.
Description of the Intervention
Sponsored by the Children’s Aid Society, the Carrera Adolescent Pregnancy Prevention program is a comprehensive youth development program for economically disadvantaged teens who enter the program at ages 13-15 and usually participate for three years, sometimes longer. The program is provided after school at local community centers, and runs for about three hours each weekday.1 It includes five main activities, as follows:
- Daily academic assistance (e.g., tutoring, homework help, assistance with college applications);
- Job Club 1-2 times per week, including such activities as learning to complete a job application and interview for a job;
- Family life and sex education 1-2 times per week, led by a reproductive health counselor;
- Arts activities 1-2 times per week (e.g. music, dance, writing, or drama workshops); and
- Individual sports activities 1-2 times per week (e.g. tennis, swimming, martial arts).
The program also provides free mental health and medical care through alliances with local health care providers. A key component is reproductive health care, including physical exams, testing for sexually transmitted infections, a range of contraceptive options, and counseling. Carrera program staff schedule the teens’ appointments and accompany them on their visits.
The program costs approximately $4,750 per teen per year to implement (2009 dollars). Click here to go to the program’s website.
EVIDENCE OF EFFECTIVENESS
This summary of the evidence is based on a systematic search of the literature, and correspondence with leading researchers, to identify all well-designed and implemented randomized controlled trials of the Carrera program. Our search identified one such study – a large, multi-site trial – summarized as follows. This trial evaluated the program as implemented in well-managed community youth agencies, where teens chose to participate after a recruitment campaign, and had parental consent. Thus, the trial’s findings apply only to the program as implemented under such conditions and cannot be presumed to apply, for example, to a more reluctant teen/parent population.
Overview of the Study Design: Large, multi-site randomized controlled trial of the Carrera program at 12 urban community centers in 6 states, during 1997-2004.
This was a randomized controlled trial of 1,163 teens aged 13-15, who were not parenting or pregnant, and who had been recruited to participate in youth activities at 12 urban community youth agencies across the country. Six of these agencies were located in New York City. The trial selected these agencies based on a judgment that had the capacity and staff to faithfully implement the program.
The teens were randomly assigned to either (i) the Carrera program; or (ii) a control group offered an alternate, less intensive program (at most sites, the agency’s regular youth program). 45% of the teens were African American or Caribbean black, and 29% were Hispanic. 58% were from single or no-parent households, and 54% lived in households that had no employed adult and/or received entitlement benefits (e.g., public assistance, Medicaid).
On average, Carrera group teens attended program activities for 12 hours per month during the three years after random assignment. At the end of the third year, 70% of the Carrera group teens were still involved in the program.
Effects of the Carrera program 3 years after random assignment, at average age 17:
These are the effects on all main outcomes that the study measured at the three-year follow-up, including any such outcomes for which no or adverse effects were found. The effects shown are statistically significant at the 0.05 level unless stated otherwise.
Effects on pregnancy, child-bearing, and contraception:
- For Carrera group females(compared to control group females):
- 40% less likely to have ever been pregnant (15% of Carrera group females had been pregnant vs. 25% of control group females).
- 50% less likely to have ever given birth (5% vs. 10%).
- More than twice as likely to be using Depo-Provera — a hormonal contraceptive — at last intercourse (22% vs. 9%).
- For Carrera group males(compared to control group males):
- No effect on the likelihood of causing a pregnancy or fathering a child.
The study found no significant effect on condom use by either males or females. For the full sample (males + females), the study found a small (7%) reduction in likelihood of having had sex. This effect was statistically significant at the 0.10 level but not the 0.05 level.
Effects on other outcomes (for the Carrera group versus the control group):
- 16% more likely to have had some work experience (89% of the Carrera group had work experience vs. 77% of the control group).
- Positive effects on some educational outcomes (PSAT scores and college visits) but not others (e.g. grades).
- No significant effects on substance abuse or on any measures of delinquency, such as arrests or fighting.
Effects of the Carrera program 7 years after random assignment, at average age 21:
The 7-year follow-up was limited by funding constraints to (i) the six New York City sites (rather than the full 12-site sample), and (ii) measurement of key educational outcomes (rather than the broader set of outcomes, including teen pregnancy, in the 3-year follow-up). All effects shown are statistically significant at the 0.05 level.
Importantly, we believe these education effects may be valid, but need confirmation in additional studies. This is because of a limitation in this study at the 7-year follow-up – namely, the difference in sample attrition for the Carrera group versus control group at the 7-year follow-up (see “Discussion of Study Quality,” below).
Effects (for the Carrera group versus the control group):
- 30% more likely to have graduated high school or obtained a G.E.D. (86% of the Carrera group had graduated or obtained G.E.D. vs. 66% of the control group).
- 37% more likely to be enrolled in college (63% vs. 46%)
These effects were roughly similar for males and females.
Discussion of Study Quality
- This was a large, multi-site study that evaluated the Carrera program as it is typically implemented in well-organized community youth agencies, thus providing evidence that the program is effective under real-world conditions.
- The study was reasonably long-term: Outcomes were measured 3 and 7 years after random assignment.
- The study had low to moderate overall sample attrition: Outcome data were obtained for 81% of the original sample at 3 years, and for 75% of the New York City sample at 7 years. Also, there was no difference in attrition rates between the Carrera and control groups at 3 years (there was, however, at 7 years – see last bullet below).
- Carrera and control group members in the 3 and 7 year follow-up samples were highly similar in their observable pre-program characteristics (e.g., demographics, previous sexual experience).
- The study measured outcomes for all teens assigned to the Carrera group, regardless of whether or how long they actually participated in the program (i.e., the study used an “intention-to-treat” analysis).
- A study limitation is that outcomes were measured through teens’ self-reports (on a written survey), and for the most part were not corroborated by outside measures. One exception is that the self-reported pregnancy and birth outcomes for Carrera group members at the New York City sites were confirmed by their medical records.
- The findings at the 7-year follow-up were limited in two ways: (1) Education outcomes were measured but teen pregnancy (the main outcome of interest) was not. (2) Sample attrition was higher for the Carrera group (32%) than for the control group (18%). Although the follow-up samples were still largely equivalent in their observable pre-program characteristics, this difference in attrition conceivably could have caused unobservable differences between the two groups, possibly leading to inaccurate estimates of the program’s effects at 7 years. Thus, we believe the 7-year findings on educational outcomes may be valid, but await confirmation in additional studies. (These limitations do not apply to the findings at 3 years.)
(Click on linked authors’ names for their contact information)
- Philliber, Susan, Jackie Kaye, and Scott Herrling, The National Evaluation of the Children’s Aid Society Carrera-Model Program to Prevent Teen Pregnancy, Philliber Research Associates, May 2001. Click here for a link to this study.
- Philliber, Susan, Jackie Kaye, Scott Herrling and Emily West, “Preventing Pregnancy and Improving Health Care Access Among Teenagers: An Evaluation of the Children’s Aid Society-Carrera Program,” Perspectives on Sexual and Reproductive Health, Vol. 34, No. 5, 2002, pp. 244-251.
- Philliber Research Associates, “The Children’s Aid Society Adolescent Pregnancy Prevention Program: High Graduation and College Admission Results,” 2007, and follow-up correspondence with researcher Jackie Kaye.
Note: Panel members Ron Haskins and Steve Raudenbush did not participate in the Advisory Panel’s review of this intervention.
1 The program now works with 10-12 year-olds in addition to 13-15 year-olds. It has also developed an in-school (as opposed to after-school) model that provides similar activities but costs less to administer. The evidence in this summary, however, applies only to the program for 13-15 year-olds, delivered after school in community centers.