1. Introduction
In teenage pregnancy prevention interventions, sample attrition can seriously compromise claims of internal and external validity. When subjects lost to a study sample differ significantly from those who remain, the generalizability of findings beyond the intervention may be applicable only to those who are similar in characteristics to the final analysis sample [
1]. Further, if attrition—and specifically, differential attrition rates—become high, the analytical sample may differ considerably from the initially randomized sample, which could negatively affect claims of validity [
2]. Under such conditions, studies are likely to produce biased estimates of program effects. Therefore, given the potential threats to validity and statistical power posed by attrition, program planners and researchers tend to devote considerable resources to maximize retention and minimize attrition, particularly in longitudinal study designs [
3]. However, despite these efforts, retention continues to be a source of frustration in many preventative and health interventions involving adolescents and pre-adolescents. For example, Karlson and Rapoff’s [
4] review of 40 randomized cognitive behavioral studies in pediatric populations published between 2002 and 2007 found that the mean attrition rate was 20% for initial follow-up and 32% for extended follow-up, with a range of 0%–54%. Bennett and Assifi’s [
5] systematic review of 19 abstinence-only and abstinence-plus interventions implemented between 1998 and 2001 yielded attrition rates ranging from a low of 2% to a high of 35%.
The integrity of an intervention becomes problematic not only when attrition is high, but also when there is low adherence to treatment. Subjects randomized to a treatment are expected, under ideal conditions, to complete the full dosage. However, as research has shown, for a variety of reasons—including loss of interest and characteristics of both the subject and intervention—some subjects fail to adhere to the treatment protocol [
6]. Low compliance rates are particularly worrisome during efficacy trials, as such rates can mask the effects of an intervention [
7]. However, statistical and methodological solutions are available for dealing with problems of attrition and adherence. For example, intent to treat (ITT) models have become acceptable and recommended approaches in randomized control trials (RCTs). Under ITT models, subjects are included in the final analytic sample irrespective of whether they adhere to the intervention protocol or are lost at follow-ups. These models generally produce conservative estimates of program effects.
Notwithstanding advances in the statistical handling of loss to program and evaluation samples, researchers continually strive to identify those subjects who are most likely to leave an intervention prematurely and the reasons for their withdrawal. For example, models for predicting treatment adherence and loss to follow-ups have been extensively tested in home-based healthcare [
8] and substance abuse interventions [
9]. In the case of home-based interventions, reported attrition rates are found to be generally high, averaging 50%; with incidences of non-completion and loss to follow-up being particularly problematic among underserved and at-risk families [
8]. Brand and Jungmann [
10] investigated the factors that contribute to high attrition among disadvantaged first-time mothers in Germany enrolled in Pro-Kind, which is a home-based intervention that provides prenatal, postnatal, and social services. They conducted a series of hierarchical logistic regression analyses in which process factors (low maternal engagement, time spent on program content, participant satisfaction with the program) and demographic factors were the major predictors. The results indicated that process variables explained three times the amount of variance associated with attrition than did personal characteristics. Similar findings were noted by Damashek
et al. [
11] in a study of a home-based child maltreatment prevention program in the US.
In this regard, the recruitment and retention of minorities in intervention studies have been of particular concern among health researchers [
12,
13,
14]. However, the primary focus has been on the adult population. Results from these studies reveal that retention of adult minority participants in clinical trials is influenced by both program and participant characteristics [
13,
15]. For example, Brown
et al.’s [
15] investigation of the factors that determine minority women’s involvement in clinical trials found that practical issues such as transportation, work and family responsibilities, and economic hardships were mitigating factors in their decision to participate and remain in trials.
Although past efforts have been directed at minority adults, there is a growing body of studies devoted to understanding strategies for recruiting and retaining minority youth in both clinical trials and other programs (e.g., adolescents’ mental health). However, few studies have attempted to explain retention and treatment adherence among minority youth participating in teen pregnancy prevention interventions. On the other hand, several studies have addressed issues regarding risk-related health behaviors such as HIV prevention and drug and alcohol abuse [
16,
17,
18]. Findings from these studies have reinforced the results from those conducted on the adult population.
Villarruel
et al. [
18] examined factors associated with retention of Latino adolescents in an HIV sexual risk reduction intervention. Responses to open-ended questions suggested that major facilitators of participation were perceived support from peers, family, and teachers. On the other hand, barriers to both participation and retention included conflict with family and personal obligations, in addition to intervention characteristics. Clark
et al.’s [
19] exploration of factors affecting African-American adolescents’ willingness to be recruited and participate in health-related interventions revealed similar results. Specifically, intervention features such as the time, location, and setting of the interventions, the type of incentive offered, the characteristics of the instructors, and the adolescent’s own attitudes and beliefs, as well as family problems were identified as important determinants of whether youths would enroll in and complete an intervention. In their study of middle school students, Morean
et al. [
20] investigated how well behavioral incentives and personal characteristics predicted interest in a tobacco prevention program. Students were more likely to report interest in participating if they were younger, non-Caucasian, highly impulsive, and/or highly self-regulated students. Gender and current smoking status did not affect reported interest. Incentives that increased interest included frequent, inexpensive awards and electronic prizes, such as monthly video games. Cash incentives were not as influential in predicting participation interest.
Differences in the saliency of these predictors (family, intervention, and demographic variables) among ethnic and racial groups have been noted in various studies [
21]. In a randomized study on middle school families residing in an urban area, Coatsworth
et al. [
21] found that while family context variables were significantly associated with retention for both African-Americans and Hispanics, they were more important for Hispanic families than they were for African-American families. It was suggested that these findings underscored the importance of culture in the Hispanic community. Indeed, in a study that examined the extent to which family culture influences students and families of Mexican-American origins in a school-based intervention to prevent academic disengagement and mental health problems, cultural sensitivity, language preference, and acculturation status were significant factors in program engagement and completion [
22].
Thus, regarding the association between adherence and retention on the one hand, and those among program, family, and personal characteristics on the other, the findings from the studies reviewed in this article indicate that these variables contribute to both adherence and retention. However, their importance may vary according to the racial and ethnic characteristics of participants. Given the relative paucity of similar studies in the field of teen pregnancy interventions, the present study investigated: (i) the success of models that incorporate program, family, and individual characteristics in predicting treatment adherence and retention at six months in a sample of 1319 youths who participated in an abstinence-only intervention; and (ii) the relative importance of the predictors in explaining retention and adherence among Hispanic and non-Hispanic youths—in light of recent findings which indicate higher teen birth rates for Hispanics than for any other racial group [
23]. In fact, according to Faccio and Fish, [
23] 34% of all teen births in the US are to Hispanics. The need for such a study is compelling given the substantial investment at the federal level in teen pregnancy prevention programs, as well as the fact that despite declining rates, births among individuals under the age of 19 remain higher in the US than they are in any other Western country [
24].
2. Methods
The article is based on data from a replication teenage pregnancy prevention study in which 1319 students were randomly assigned to one of two treatment arms: an abstinence-only or general health curriculum (counterfactual). The goal of the intervention was to determine if results initially found in the efficacy trial—delay in sexual debut—could be replicated in a different context. The initial study occurred with a predominantly African-American middle school population of youth aged 10–11 in Philadelphia. The replicating study, funded by the Office of Adolescent Health in the US Department of Health and Human Services from 2010 to 2015, was conducted in New York state with a more heterogeneous population consisting of Hispanic, White, and African-American youth of similar ages. Youth were assessed at baseline, posttest, three, six, and 12 months. Prior to the onset of the intervention, Institutional Review Board approval was secured.
The Promoting Health Among Teens! Abstinence-Only Intervention (PHAT-AO), the treatment arm of the intervention, had four major content foci: (i) the relationship between goals and dreams and adolescent sexual behavior; (ii) importance of knowledge regarding the causes, transmission, and prevention of HIV, sexually transmitted diseases (STDs), and teenage pregnancy; (iii) the centrality of beliefs and attitudes about abstinence, HIV, STDs, and pregnancy; and (iv) the necessity of developing skills and self-efficacy including negotiation-refusal skills. The program provided opportunities for youths to practice and receive reinforcement and support. The content was covered over two consecutive Saturdays in eight one-hour modules. The counterfactual curriculum addressed the following: (i) the relationship between goals, dreams, and health; (ii) the importance of exercise and healthy eating for health; (iii) how exercise and healthy eating help the body to function properly, and (iv) how negative substances can be detrimental to healthy body functioning. Similar to the sexual health curriculum, time is provided for practice, reinforcement, and support for making healthy choices. Youth living more than a mile from the intervention sites (which were eight schools in the district) were provided with transportation. Breakfast and lunch were served on each program day.
2.1. Recruitment and Retention Recruitment Strategies
Both active and passive strategies were used to recruit eligible youth [
25,
26]. Eligibility was based on areas of the city with the highest incidence of teenage pregnancy. Schools were the primary venues for actively recruiting youths. Presentations were made at back-to-school nights for parents, as well as at assembly programs for eligible sixth- and seventh-graders. Recruitment packets with informed consent forms were sent home with students. In order to incentivize the return of the forms, pizza parties were held for the classes with the highest return rates. Passive recruitment strategies included spots on local television public programming and flyers in the community. Over the 5 years of the program, 6469 packets were sent home. About half (54%) of these were returned (see
Figure 1), 314 did not pass screening (students either did not speak English or had a special education classification), and 45% of parents declined to have their child participate. Of the 1612 youths with parental consent who passed screening, 82% were randomized to one of the intervention arms; 18% did not attend the program. The consort diagram presents the number of randomized youth eligible for inclusion in the follow-up data collection at the time of this study. It is instructive to note that follow-up data collection was based on the anniversary of post completion of the intervention. Thus, in the current article, only 1108 youth were eligible to be included in the six- month analytic sample.
A multiplicity of approaches was used to ensure retention. Youths were provided with incentives in the form of Barnes & Noble gift cards, opportunities to win movie tickets, iPads, and gift cards to the local supermarket when they completed the intervention and returned for each of the follow-up data collection sessions [
27]. Additionally, youth received birthday cards and congratulatory cards for accomplishments. Periodic mailings reminding youth of follow-up sessions were conducted in conjunction with personal phone calls to families both the day before and the morning of data collection. For students who lived more than a mile from the program sites, transportation to and from the sites was provided. For students who failed to attend a regularly scheduled follow-up data collection date, alternative modes of participating in the follow-up were made available, such as answering the questionnaire over the phone, participating in a follow-up session in the neighborhood library and in school during a free period, or participating before or after school.
2.2. Participants and Procedures
Approximately 63% of youth in the study were of Hispanic origin, 25% were African-American, 9.6% were White, and 8% other. Males constituted 47% of all randomized youth. The average age of participants was 11 years. Among all participants, 52% spoke English at home, 29% spoke Spanish, 14% spoke Spanish and English, and 3.4% spoke a language other than English or Spanish. Further, 56% lived with both biological parents or in families in which their guardians were married. Finally, 75% had working mothers and 80% had fathers who were employed.
On the first Saturday of the intervention, students were randomized on site after assenting. Upon completing the baseline questionnaire, randomized students received instruction from a facilitator—a graduate student in public health—in either the sexual health or general health arm of the intervention. Program classrooms consisted of no more than eight students. At the end of both Saturdays, youth completed a debriefing instrument. The PHAT-AO is based on an ITT model, and therefore, all randomized students were surveyed and resurveyed at posttest, three, six, and 12 months, irrespective of whether they completed the intervention or missed any of the follow-up data collection points.
2.3. Measures
2.3.1. Outcomes
In this study, the primary outcomes were adherence to treatment and retention. The full sample (1319) was used for the primary outcome adherence to treatment and a subsample of youths (1108) in the analyses addressed retention. Adherence to treatment was defined as completion of the two-Saturday program, and it was coded (1) to represent whether youths attended both Saturdays and completed the intervention. Retention was defined as participating in the six-month follow-up (coded 1 for yes). Under the ITT model, all youths randomized were eligible for inclusion in the six-month follow-up, regardless of whether they completed the intervention. This data point was selected because attrition was greater at six months (42%) than it was at three (33%) and 12 months (30%). Further, participation in the six-month follow-up was the strongest predictor of retention at 12 months, which was the final analytical point for determining effectiveness (β = 0.33, 0.22, and 0.19 for the six-month, three-month, and posttest measurements, respectively).
2.3.2. Covariates
All covariates were measured at baseline and before randomization. The following were the covariates for which the effects on the outcomes were estimated. For family context, two measures of youth family context were included: (1) whether the mother works (coded 1 for yes); and (2) parental disapproval of sex, as assessed by a single categorical item (coded 1 for yes and 0 for no). Intervention context was measured by an index with 17 items. Youth were asked to evaluate the intervention along a number of dimensions including the facilitators who taught the curriculum, benefits gained from participating in the intervention, level and comfort of involvement in the various activities, and whether they would recommend the program to another youth. Items were on a five-point Likert scale with five representing strong positive beliefs about the intervention. The index had a Cronbach’s reliability coefficient of 0.879. Behavioral disposition was measured by youth’ intentions to have sex in the next three months, which was a dichotomous variable (coded 0 for no and 1 for yes). Youths’ demographics were captured by two variables: age and gender.
2.4. Statistical Analysis
The analysis was carried out in two stages. First, hierarchical multivariate logistic regressions were used to predict adherence and retention at six months. Separate analyses were conducted for each of the outcomes. The same covariates were used for the two outcomes, and the models were specified similarly, except for retention, where adherence to treatment was used as a covariate. For the outcome variable of interest—treatment adherence—family context was entered in the first model, the second added student characteristics and intentions to have sex in the next three months, and the final model added youths’ reactions to the intervention. An additional model was tested for retention at six months in which treatment adherence was introduced as a covariate in the last model. Missing data were handled through multiple imputations with the pooled odds ratios (ORs) reported in the tables. Youth demographic variables (gender, age, Hispanic origins) and randomized condition were used to impute missing values for the outcome variables. Twenty imputed data sets were generated.
Hosmer–Lemeshow goodness-of-fit tests indicated that all the tested models adequately fit the data. Second, to complement the logistic regression, discriminant function analysis was conducted to test whether the predictors used in the logistic regression models were similar in importance for Hispanic and non-Hispanic youths. Box’s M test for homogeneity of the covariance matrices resulted in the use of separate group covariances in the classifications. Follow-up contrast testing based on independent
t-tests and chi-square was conducted to determine whether there were differences in predictors between completers and non-completers for Hispanics and non-Hispanics [
21]. As a precautionary step, analyses with randomized status as a covariate were run. As expected, this variable was not significant.
4. Discussions
Previous research has identified family, personal, and intervention characteristics as potential explanatory factors in treatment adherence and retention in behavioral interventions. Moreover, in some studies, the effects of these variables have been found to differ based on the ethnic and racial backgrounds of subjects. The goal of the present study was to test the relevance of these variables for understanding adherence and retention in teenage pregnancy prevention research, an approach that has been overlooked in the field. Family context variables, and in particular having a working mother, was associated with youths’ completion of the intervention and retention at subsequent data collection points. This finding corresponds with prior research showing that among low- and moderate-income families, labor market and family pressures can negatively affect children’s involvement in extracurricular activities [
8,
12,
15,
28].
Because the
PHAT-AO was a strict replication of a previous evidence-based intervention, adapting the timing of the intervention to better align with the needs of families was not initially considered, although subsequent modification to the initial start time was done. The data did not allow us to determine why having a working mother was such a critical barrier to adherence and retention in the present study, nor could we identify whether working mothers were single parents as well. We speculate that the early starting time of the program, notwithstanding the availability of transportation and subsequent change in time, coupled with the fact that the intervention occurred on a Saturday, might have conflicted with competing family obligations [
3].
The present findings also provided confirmatory support for studies in which the effects of intervention context on adherence and retention were significant. Not surprisingly, overall satisfaction with the various components of the intervention significantly predicted the likelihood of youth completing the intervention and being retained at six months; although for the latter outcome, once we introduced adherence as a covariate, its impact was lessened. There is a well-established body of literature documenting the importance of subjects’ reactions to an intervention as a measure of implementation success. Although the brevity of the
PHAT-AO provided minimal opportunity for bonding between youths and the project to occur over time, concerted efforts were made during implementation to promote a positive relationship between youth and the facilitators. In spite of this, the use of appointment reminders, extrinsic incentives, and so forth, retention remained problematic. In order to boost retention and adherence, future providers of brief teenage pregnancy interventions may consider some of the various approaches and models used to engage participants and families in other health fields. These include the Strategic Structural-Systems Engagement approach, which acknowledges resistance to treatment adherence and attrition as a natural process and incorporates this acknowledgement into the program theory [
29], and motivational interviewing, which continually addresses the ambivalence that subjects face regarding an intervention [
30].
The influence of the various covariates in the study differed by ethnicity and culture. For example, youth personal characteristics were more important in explaining the decision to withdraw from the intervention among Hispanics as compared to non-Hispanics. Conversely, attitudes toward sex, both on the part of the youth (intention to have sex) and in terms of family sexual mores (disapproval of adolescent sex) figured prominently as reasons for non-Hispanics’ withdrawal. Similarly, in addition to treatment adherence, gender was a critical discriminating variable for explaining retention at six months among non-Hispanics. For Hispanics, the critical discriminating variable was living with a working mother. An implication of these findings is that teenage pregnancy interventions targeting multi-ethnic communities should develop strategies that are sensitive to the unique culture of their diverse participants. Research has shown that the teen birth rates differ significantly among Hispanics, African-Americans, and Whites, with Hispanic females having the highest rates [
27,
31]. Further, Hispanic parents are less likely to discuss sex and reproductive health with their children than are parents from other ethnic groups [
32,
33].
Two other important findings in this study were (i) the complex relationship between adherence and retention and (ii) the inability of the models to generate comparable classification rates for non-completers and attritors to that observed for those who completed and those who were retained in the intervention at six months. It is clear that while treatment adherence and retention are related, they also measure different underlying behaviors, as evidenced by how they are differentially influenced by the variables in this study. This suggests that they should be treated as separate outcomes in future studies. Further, intervention providers should strategize on how best to address each factor individually. Finally, improving the classification rate for non-completers and attritors will require the identification of additional variables for inclusion in future models.
5. Conclusions, Limitations and Future Research
This study sought to fill a gap in the field of teenage pregnancy prevention, as there are few studies, if any, that have sought to predict youth who are most likely to complete an intervention and be retained in subsequent follow-ups. However, there are limitations to the study. First, as previously alluded to, we still do not have a thorough understanding of the reasons why some youth discontinued their participation. The use of qualitative techniques could have strengthened our understanding. Regarding family background, there were no measures of family structure, such as number of siblings, size of households, work patterns of guardians/parents, head of household status, and so forth. The inclusion of these measures would have helped us to explore further the effects of working mothers on treatment adherence and retention. Additionally, we did not incorporate measures of religiosity and parenting styles—both of which could help to explain differences between Hispanics and non-Hispanics. Finally, given the low prevalence rate of sexual initiation in the present study, we were unable to determine if sexual activity was related to either retention or adherence. Therefore, we recommend that future studies consider including this variable as well as those that were omitted from the current study in their models. Notwithstanding these limitations, the present findings help to clarify why some youth who are at the greatest risk of having a child while still in their formative years are likely to participate and engage in interventions that seek to prevent such an occurrence from happening.