Testing the Effectiveness of Transfer Interventions Using Solomon Four-Group Designs

Abstract

How to enhance the transfer of training remains an important question, and to some extent, post-training interventions can provide an answer. The purpose of this study was to validate inconclusive findings on the effectiveness of two post-training transfer interventions. This study used Solomon four-group design to filter out the effects of pretest sensitization and history, which are threats to the internal and external validity but have rarely been checked in previous transfer studies. Management study undergraduate students were randomly divided into two groups: pretested and unpretested groups. After a time management workshop, the students were randomly subdivided into three additional groups based on the following conditions: full relapse prevention (RP); proximal plus distal goal setting (GS); and the control group. Although results from both intervention groups were not significantly different from those of the control group, a significant difference was found between full RP and proximal plus distal GS in terms of self-reported time-management behavioral change. It is difficult to conclude whether post-training interventions enhance the transfer of training. Further ideas for improving research designs were explored, such as increasing the time intervals between training and interventions so that trainees have opportunities to attempt transfers before the interventions.

1. Introduction

The transfer of training is the utilization of knowledge and skills acquired through training in new settings. It is a stepping stone from thinking, learning, and developing to increasing personal and organizational performance. Thus, training transfer is critical for maximizing transfer outcomes. At the same time, however, the transfer problem—the question of how to enhance this transfer—requires further clarification [1]. In studies of different interventions (pre, during, and post), the post-training interventions have been found to have the strongest influence on transfer [2]. Consequently, investigations of post-training interventions have been increasing in transfer studies [3]. Among the post-training interventions, relapse prevention (RP) and goal setting (GS) interventions have been the most tested in recent decades. The results of these studies, however, have been mixed [4,5], with some studies having supported the interventions’ effectiveness while others have not.

Moreover, in terms of research design, we use Solomon four-group design in this study, which allows us to rule out several internal validity threats (history and maturation) as well as external validity threats (interactions between pretesting and treatments and between selection bias and treatments) [6]. By so doing, the results in the intervention effects is more likely to be convincing.

Based on the discussion above, in the present study, we investigate the effectiveness of the post-training interventions RP and GS. Subjects were undergraduate students who participated in time-management training as the main focus of learning. Testing interventions in this manner can help trainers, educators, practitioners, and managers in their considerations of whether to add post-training interventions to boost the future transfer of training. Based on the result, it is difficult to conclude whether post-training interventions enhance the transfer of training because results from both intervention groups were not significantly different from those of the control group whereas a significant difference was found between the two intervention groups: full RP and proximal plus distal GS.

1.1. Relapse Prevention

RP is a self-control strategy that helps trainees recognize and overcome potential hindrances to their transfer of newly acquired knowledge and skills [7]. RP is based on the principles of social learning theory. Originally, it was employed as a behavioral therapy to help addicted individuals predict and cope with relapse problems [8]. Generally, “relapse refers to a breakdown or failure in a person’s attempt to change or modify any target behavior” [8] (p. 261).

In 1982, Marx described a cognitive behavioral model of the relapse process and elaborated RP strategies as transfer-enhancement interventions for corporate trainings [7]. In the corporate training field, RP helps trainees to become aware of the possibility for relapse in high-risk situations (e.g., those involving time pressure or intense interpersonal emotions) and to reduce the potential for slips, which are failures to transfer trained skills. The original, full version of Marx’s RP includes seven steps for trainees: (a) selecting skill(s) to retain; (b) setting retention goal(s) by specifying frequencies for skill usage and identifying slips and relapses; (c) creating commitments to skill maintenance by determining the pros and cons of using and not using the skill(s); (d) learning and applying the 14 RP strategies, including identifying high-risk situations; (e) imagining conditions surrounding a first slip; (f) practicing coping strategies; and (g) monitoring the target skill-transfer process [9].

Several empirical studies have suggested that RP groups exhibited improved training transfer compared to control groups [10,11]. Other studies, however, have not supported the effectiveness of RP [12,13]. Two literature reviews have shed light on the inconsistent results for RP, which may stem from alternating between using full versus modified RP models [5,14]. Modified RP is a scaled-down version of full RP, usually including just two to five steps from the full version, and it has not been standardized across studies except that step (d) has consistently been included [14] (p. 14). However, we should not simply assume that the mixed RP results have arisen from the use of these different versions. Even for full RP, one study fully supported it [15] while another only partially supported it [11,16]. Likewise, for modified RP, some studies have supported it [17,18] while other studies have not [12,19]. It should also be noted that full and modified RP are not markedly different and that results for both approaches have been diverse.

As mentioned above, only three studies since 1986 have examined the use of the full version of RP. A recent study of time-management training for Indonesian employees indicated that full RP influenced transfer [15]. Moreover, a study on employee coaching skills in 78 research scientists showed that full RP modestly influenced transfer when the work environment was unsupportive [11]. An empirical study of 90 undergraduate students in an assertive communication training showed that full RP significantly increased their ability and desire to transfer skills (i.e., a direct consequence of the RP intervention before transfer). It did not, however, increase the undergraduates’ use of transfer strategies (i.e., the transfer of RP, which was expected to precede transfer of the main training) or their use of skills acquired in the main training (i.e., the transfer of the main training, which was the outcome variable of the present study) [16]. Since these studies found at least partial support for the effect of RP, we hypothesize that full RP trainees will demonstrate greater transfer than the control group trainees.

Hypothesis 1 (H1).

The full RP intervention trainees demonstrate greater transfer of training than the control group trainees.

1.2. Proximal Plus Distal Goal Setting

Locke and Latham’s goal setting theory of motivation explains why some people perform better than others on work-related tasks [20]. It maintains that a person setting a specific, challenging goal can achieve a higher outcome than one setting a vague, easily attainable goal as long as the person has the following things with respect to the goal: 1. the ability, 2. the situational resources, 3. the commitment, and 4. feedback on progress [21].

Moreover, we should consider another perspective on goals in the present study. Several types of goals have been discussed in the literature, including outcome, performance, process, learning, and behavioral goals. There has been no strong evidence to suggest, however, that intervention effects differ according to the type of goal. Since our target is to enhance behavioral change, the goal set in our intervention is a behavioral one, such that the performance of specific behaviors leads to a given outcome or goal: for example, writing a paragraph every day to finish a paper [21].

Another perspective relevant to our study is distinguishing between distal and proximal goals. A distal goal is a long-term goal, whereas a proximal goal is a shorter-term, benchmark goal. Short-term goals that cascade from a longer-term distal goal are called proximal plus distal goals [22]. The achievement of proximal goals can be enhanced by immediate incentives [23] to boost motivation and by trial-and-error feedback [24] to correct actions. These processes can also promote progress toward distal goals, toward which a person may be less motivated to act. Hence, combining these two kinds of goals is considered an effective approach in management development programs, where people are learning new and complex tasks [25] (p. 38–39). In particular, these ideas apply to the subject of the current study, which is learning time management as a new skill.

Previous empirical studies have supported that, compared to distal and do-your-best goals, proximal plus distal GS as a post-training intervention more significantly enhances transfer outcomes [26,27]. Six weeks after 72 Canadian public service employees completed a self-awareness training, the participants in the proximal plus distal goals group demonstrated greater transfer (in terms of generalization and maintenance) than did those in the distal and do-your-best goals groups [26]. In another study of 89 government employees who completed self-awareness training, the results confirmed that proximal plus distal goals enhanced transfer (maintenance) six months after training. The current study hypothesizes that the proximal plus distal goal intervention trainees exhibit greater transfer than control group trainees.

Hypothesis 2 (H2).

Proximal plus distal goal intervention trainees demonstrate greater transfer of training than control group trainees.

1.3. Comparison of RP and GS

Several scholars have compared the contributions of RP and GS to transfer maximization [15,19,28], although most did not conduct their comparisons in exactly the same manner as the present study, which compares full RP and proximal and distal GS. In a comparative study on customer-service skills training, trainees in the outcome goal GS group utilized learned behaviors to a greater extent than did those in the modified RP and control groups [17]. In a case study of Master of Business Administration students’ salary-negotiation simulations, the results of the regression analysis showed that modified RP attenuated trainees’ negotiating performance, while self-set GS accentuated it [29]. In the earliest comparative study of GS with modified RP intervention in the context of a time-management workshop [28], students in GS interventions were superior to those in modified RP and the control group in terms of maintaining behavior change over a two-month period. The only study with the same comparison as ours was a recent study of RP and GS in 160 Indonesian employees. This study showed that proximal plus distal GS boosted transfer more than full RP did twelve weeks after time-management training [15]. All the above studies concluded that RP was less effective than GS, with the exception of one conflicting result in a study of 68 MBA students involved in a dyadic negotiation skills training program. In this study, the modified RP group exhibited higher skill transfer than the self-set GS group [30]. This may be because the simulation was too novel and complex for the students, and consequently, the RP transfer strategies were more effective than the GS ones.

Based on the above discussion, GS has had more solid results than RP in the majority of studies, and this trend can be explained by examining the underlying concepts. GS is based on motivation theory, and especially in proximal plus distal GS, proximal goals serve as “small wins” [26] on the journey to distal goals. At the same time, proximal goal outcomes function as feedback [26] during the goal-striving process. Thus, it seems that the motivation to look forward to achieving goals is more activated in GS than in RP. In GS, obstacles might fade away or at least be less active in a motivated person’s cognition. RP, though, is based on coping with obstacles [7], which may cause people to focus more on the obstacles in the interest of overcoming them, which engages a passive response.

Based on the above discussion, we hypothesize that trainees in the proximal plus distal GS group will exhibit greater transfer than those in the RP group.

Hypothesis 3 (H3).

Trainees in the proximal plus distal goal intervention group demonstrate greater transfer of training than those in the full RP intervention.

2. Materials and Methods

2.1. Research Design

The present study is designed to eliminate threats to internal validity (history and maturation effects) and threats to external validity (the interaction effect of treatment) by pretesting and conducting a systematic exploration of the effects of RP and proximal plus distal GS. We used Solomon four-group experimental field research design [31]. In this design, both the treatment and control groups have two subgroups: a pretested group and an unpretested group (see

Table 1. The Solomon four-group design.

	Pretest	Treatment	Posttest
Group 1	O1	X	O2
Group 2	O3		O4
Group 3		X	O5
Group 4			O6

Note. X: treatment, O: outcomes.

).

In this study, we had a total of three Solomon four-group designs (illustrated in

Table 2. Solomon four-group comparisons.

First Solomon four-group (S1): Comparing RP with Control (Hypothesis 1)
Conditions				Pretest	Treatment	Posttest
R	Group 1	RP1	RP pretest group	B1	X	B2
R	Group 2	C1	Control pretest group	B3		B4
R	Group 3	RP2	RP unpretested group		X	B5
R	Group 4	C2	Control unpretested group			B6
Second Solomon four-group (S2): Comparing GS with Control (Hypothesis 2)
Conditions				Pretest	Treatment	Posttest
R	Group 5	GS1	GS pretest group	B7	X	B8
R	Group 2 a	C1	Control pretest group	B3		B4
R	Group 6	GS2	GS unpretested group		X	B9
R	Group 4 a	C2	Control unpretested group			B6
Third Solomon four-group (S3): Comparing RP with GS (Hypothesis 3)
Conditions				Pretest	Treatment	Posttest
R	Group 1 a	RP1	RP pretest group	B1	X	B2
R	Group 5 a	GS1	GS pretest group	B7	X	B8
R	Group 3 a	RP2	RP unpretested group		X	B5
R	Group 6 a	GS2	GS unpretested group		X	B9

Notes: R = randomization, B = behavior outcome measure, X = treatment, RP = relapse prevention, GS = proximal plus distal goal setting. ^a Group reused in repeating conditions, ₁ pretest group, ₂ unpretested group.

) involving nine groups and comparing the RP group with the control group, the GS group with the control group, and the RP group with the GS group.

2.2. Participants

Data were collected from 210 management study students at the National Management Degree College (NMDC) in Yangon, Myanmar. Participants were final-year students who had enrolled in the four-year bachelor’s degree program during the 2018–2019 academic year. Participants enrolled in a three-hour time management workshop arranged by the college and the researchers. This particular sample was selected as students were nearing graduation and about to enter the job market. Therefore, it was thought that time-management skills would be helpful for these students to complete their academic work, to prepare them to enter the job market, and at their future workplaces. Moreover, training transfer seemed more feasible for this course compared with other courses available to the students. For example, for these particular students, measurements of training transfer from a financial management course would be problematic as the students were not employed at that moment. Furthermore, two studies in the transfer literature also used time-management training as the core training. For the sake of consistency and comparability with other literature, we used a similar training to examine the effect of transfer interventions.

2.3. Procedures

2.3.1. Before the Training

Two months prior to the training, we asked the NMDC principal for permission to conduct the training. We also informed the head of the Department of Tourism and Hospitality and the head of the Department of Business Management. They granted us permission to conduct the trainings and surveys. Two weeks before training, we emailed a student representative who was introduced to us by the principal. In that email, we introduced the purpose of the training and the research, explained the expected outcomes of the training, and requested voluntary participation. A week before the workshop, we met with two student representatives and discussed effective incentive tools for students. The representatives requested that we provide a certificate of workshop completion in exchange for students’ participation, and we did so accordingly. We sent invitation letters to all final-year students via the representative through a private Facebook group chat. In the letter, we explained our objectives, procedures, training schedule, voluntary participation, and rewards. We collected consent forms before training, and the two student representatives were removed from the data analysis to avoid bias.

2.3.2. Implementation of Training

Three-hour time-management workshops were held six times to accommodate all 330 final-year students. Generally, about 50 students participated in each session, and a total of 310 students participated due to the absence of 20 students. Before the workshop, participants were randomly assigned into either the pretested or unpretested groups based on their roll numbers. Then, the time-management knowledge and behavioral questionnaires were distributed only to the pretested groups. Even though our main outcome was time-management behavior, we collected data on time-management knowledge, as it is considered an important intermediate outcome and useful for understanding the underlying mechanism of the behavior. All participants were asked to identify whether they had participated in a time-management training before starting the training; no one reported having participated in the past. The training had seven learning objectives: (1) setting goals, (2) prioritizing, (3) making lists, (4) scheduling and planning, (5) organizing desks and papers, (6) dealing with procrastination, and (7) dealing with interruptions. Training methods consisted of lectures, discussions, films, and activities such as making lists, setting goals, and role playing.

2.3.3. Intervention Sessions and Manipulation Check

At the end of each training session, participants of both pretested and unpretested groups were randomly assigned to three groups: RP, GS, and control groups. We implemented post-training intervention sessions on days other than training days, as students were not available for additional time on the training days due to their need to attend other classes. Four sessions for each intervention (a total of eight sessions) were held over three days within 10 days of the workshop. During the intervention sessions, participants were introduced to the respective interventions briefly and they worked individually as well as in groups. We also provided example cases for the transfer of learning. Lectures, power point presentations, and group discussions were used in the sessions, and at the end of each intervention session, a manipulation check was performed.

Consistent with Marx [7], the one-and-a-half-hour sessions for the full version of the RP intervention were conducted after the core time-management training. Before starting each session, participants were asked whether they had participated in RP interventions or learned RP skills before, and they reported that they did not have any related experiences. After introducing RP and explaining why it is important in training transfer, RP group trainees chose a specific newly trained skill that they wanted to use in their daily lives. Next, each participant set a specific, measurable skill-maintenance goal and quantified the parameters of slips and relapses under the supervision of the instructor. Then, using a decision matrix, each participant committed to retaining the skill by identifying the positive and negative consequences of using and not using the new skill. RP case examples were also provided to the participants [19]. Next, the participants learned the 14 RP strategies, which are designed to increase awareness of potential trouble spots and to enhance a person’s likelihood of coping with the situation. In the next step, trainees predicted the circumstances of their first slip and practiced strategies to deal with such situations. Lastly, trainees were provided with a simple chart with which to track their progress on skill maintenance goals [11,18].

As in RP, after the time-management training, the one-hour GS intervention sessions were conducted for participants. Trainees were asked whether they had previous exposure to proximal plus distal GS skills, and no one reported having related experiences. The participants were then trained to set proximal plus distal goals. Adapting the approach from previous studies [19,29,30], the training included two main parts: (1) discussion and demonstration of GS and (2) development of GS by the trainees. The first part of the training consisted of introducing the idea of a proximal plus distal goal and of explaining why GS is important. A description of the proximal plus distal GS process was provided along with a discussion and examples showing how proximal plus distal GS can be effective in one’s daily and future life. How to set specific, challenging proximal plus distal goals was demonstrated, and then, the second part of the intervention session involved each trainee developing a proximal plus distal GS plan. The participants were asked to individually set specific, challenging distal goals (e.g., goals for the entire eight weeks’ time) and to then break these eight-week goals into shorter one-, two-, and three-week goals after learning how to complete the worksheets.

To ensure that the appropriate content was covered in the interventions, immediately after the post-training session, all participants (except those in the control group) were asked what was taught in the post-training interventions using checklists for full RP and GS [19].

2.4. Measures

Students’ knowledge and time-management behavior were measured. Their initial knowledge and time-management behavior were checked before the workshop for the pretested group (time 1). Ten weeks after the workshop, their knowledge or retention of learning and post-time management behavior were measured for the both pretested and unprotested groups (time 2). Knowledge was examined by 14 questionnaire items reviewing the basic concepts covered in the study’s time-management course; these were developed from Wexley and Baldwin’s 16 short-answer questions [28]. The excluded items were related to GS to avoid the problem that the GS group members would have an advantage from their additional learning from the intervention. The behavior measure was from two sources: self and observer reports, both of which were borrowed from Wexley and Baldwin [28]. The self-reported behavior measures consisted of 30 items (e.g., “I make up a daily planner or to-do list”). For observer-reported behavior measures, all participants were asked to identify one friend, family member, or current teacher from NMDC or elsewhere who was familiar with the participants’ time-management behavior. The observer-reported measure consisted of 10 items (e.g., “The student seemed very conscious of planning and prioritizing their day”). For both behavior measures, all items used a 5-point response scale ranging from 1 (always) to 5 (never). The authors confirm that the data supporting the study findings are available in Supplementary Materials.

2.5. Data Analysis

Due to data limitations, we were forced to adopt different analytical methods for the three outcome variables—self-reported behavior, observer-reported behavior, and knowledge in terms of time management—after checks for randomization and pretest sensitization. We used a one-way ANOVA for the three pretest groups’ pretest scores (O1 and O3) and a 2 × 2 between-groups ANOVA for the four posttest scores (O2, O4, O5, and O6).

The most rigorous analysis was possible for self-reported behavior. The external factors, the potential confound between treatment effects, and the history effects could be checked by comparing the posttest outcomes of the unpretested control group (O6) with the pretest outcomes of the pretested treatment group (O1) and the pretest outcomes of pretested control group (O3). Analyses were conducted using simple independent-samples t tests. After these preliminary analyses, we examined the main effect of each treatment by performing an analysis of covariance (ANCOVA) on the posttest scores, covarying the pretest scores [32].

In contrast to the approach for analyzing self-reported behavior, we performed a one-way mixed ANOVA on matched observer-reported behavior, as we excluded this measure from the Solomon four-group design. Moreover, effects on knowledge were tested, particularly because the retention of knowledge after eight weeks is a potentially important mediator between the intervention and the transfer. We could not utilize multivariate analysis as the sample size had dropped. We conducted a one-way repeated measures ANOVA for comparing knowledge changes in the RP, GS, and control groups.

3. Results

3.1. Descriptive Statistics and Randomization Checks for Outcome Variables

Descriptive statistics, scale reliabilities, and intercorrelations of dependent variables are presented in

Table 3. Descriptive statistics, scale reliabilities, and intercorrelations of study variables.

Variables a	n	M	SD	α	1	2	3	4	5
1. Knowledge, time 1	116	1.99	1.46	__	__
2. Self-reported behavior, time 1	118	3.16	0.35	0.72	0.24 **
3. Observer-reported behavior, time 1	149	3.27	0.47	0.57	0.19 *	0.20 *
4. Knowledge, time 2	111	3.92	2.35	__	0.25 *	0.20	0.01
5. Self-reported behavior, time 2	133	3.08	0.38	0.79	0.17	0.39 **	0.21 *	0.22 *
6. Observer-reported behavior, time 2	112	3.21	0.47	0.63	0.26 *	−0.04	0.22 *	0.24 *	0.21 *

Note: only knowledge and self-reported behavior measures used Solomon four-group design. ^a Time 1 = Before the time-management workshop; time 2 = eight weeks after the workshop. *p < 0.05. **p = < 0.01.

. A one-way ANOVA performed on each dependent variable of the pretested groups taken at time 1 tested whether the random assignment of subjects was ideal. Results indicated insignificant differences across conditions with regard to knowledge F(2, 115) = 1.52, p = 0.22; self-reported time-management behavior, F(2, 117) = 0.78, p = 0.46; and observer-rated behavior F(2, 146) = 0.55, p = 0.58, indicating that the groups were not statistically different (see Appendix A).

3.2. Pretest Sensitization Checks for External Validity for Self-Reported Time Management Behavior

In terms of self-reported time management behavior, the results of a 2 × 2 between-groups ANOVA on the four posttest scores (O2, O4, O5, and O6) in each Solomon four-group indicated no interaction effects (pretesting by treatment effects) as shown in

Table 4. ANOVA results for the pretest sensitization checks (by the interaction of pretesting and treatment) for self-reported time management behavior.

Solomon Four-Group (S)	Type IIISum of Squares	df	Error	Mean Square	F	p
S1 (RP vs. Control)	0.508	1	13.347	0.508	3.461	0.066
S2 (GS vs. Control)	0.031	1	10.892	0.031	0.240	0.626
S3 (RP vs. GS)	0.236	1	12.384	0.236	1.522	0.211

Note: RP = relapse prevention, GS = proximal plus distal goal setting.

.

3.3. History Effect Checks for Self-Reported Time Management Behavior

For the history-effect checks, analyses were simple independent-samples t tests, first comparing O1 with O6 and then O3 with O6. One significant history effect was found in a pair of GS pretested and control unpretested groups (see

Table 5. Sample t test results for history-effect checks for self-reported time-management behavior.

Comparison Paired Groups	Mean Dif.	Std. Error of Mean	95% Confidence Interval of the Difference		t	df	Sig. (2-Tailed)
			Lower	Upper
B1 = B6 (RP1a vs. C2b)	0.13	0.096	−0.06	0.32	1.35	55	0.18
B7 = B6 (GS1a vs. C2b)	0.21	0.099	0.01	0.40	2.08	53	0.04
B3 = B6 (C1a vs. C2b)	−0.11	0.089	−0.29	0.07	−1.24	70	0.22

Note: RP_1a = RP, pretested group, prescore; GS_1a = GS, pretest group, prescore; C_1a = Control, pretested group, prescore; C_2b = Control, unpretested group, postscore.

).

3.4. Results of the Experiment for Outcome Variables

We proceeded to examine the main effect of each treatment on self-reported behavior after showing the descriptive statistics of the behavior for all nine groups (from B1 to B9, see

Table 6. Descriptive statistics of self-reported time management behavior.

Conditions	RP			GS			Control			Total
Pretest	Yes		No	Yes		No	Yes		No
	T1	T2	T2	T1	T2	T2	T1	T2	T2
Sample	24	24	27	27	22	22	24	24	15	207
Mean	3.11	2.93	3.12	3.11	3.13	3.02	3.15	3.17	3.15
SD	0.38	0.43	0.31	0.36	0.39	0.35	0.35	0.37	0.37

Note: RP = relapse prevention, GS = proximal plus distal goal setting.

). The main effects were not significant. We performed ANCOVA on the posttest scores, covarying the pretest scores (see

Table 7. ANCOVA results for treatment effects on time-management behavior in three Solomon four-group designs.

Solomon Four-Group (S)	Mean Difference	Std. Error	p	95% Confidence Interval for Difference
				Lower Bound	Upper Bound
S1 (RP vs. Control)	−0.180	0.104	0.090	−0.388	0.029
S2 (GS vs. Control)	0.044	0.097	0.652	−0.152	0.240
S3 (RP vs. GS)	−0.219 *	0.107	0.046	−0.435	−0.004

Note: RP = relapse prevention, GS = proximal plus distal goal setting.

). As a result, none of the treatments were statistically different from the control group and Hypotheses 1 and 2 were rejected. The comparison between the two treatments, RP and GS, was statistically different (p = 0.046). Therefore, Hypothesis 3 that the GS group would transfer more than the RP group was accepted.

The effects on the other measure of behavior (observer-reported) were analyzed as well. Specifically, a one-way mixed ANOVA on matched observer-reported behavior was performed as we excluded this measure from the Solomon four-group design. We tested only for matching between pre- and post-observer. The main effect of the within-subjects factor pre- and post-observer-rated was not significant: F (1,64) = 0.30, p = 0.57. The main effect of the between-subject factor conditions were not either: F(2,64) = 0.21, p = 0.81. Therefore, H1, H2, and H3 were not supported, partly differing from the main effect results for self-reported behavior.

Additionally, we analyzed the effect on time-management knowledge. No pretest sensitization effect or interaction effect of pretest and treatment was found for the knowledge measures: F (5,105) = 0.23, p = 0.95. To analyze the main effects and to compare knowledge changes in the RP, GS, and control groups, we conducted a one-way repeated measures ANOVA. No significant main effect was found (F(2,107) = 0.44, p = 0.65) among the three groups. This result was not consistent with that for self-reported behavior (see

Table 8. Descriptive statistics of time-management knowledge.

Conditions	RP			GS			Control			Total
Pretest	Yes		No	Yes		No	Yes		No
	T1	T2	T2	T1	T2	T2	T1	T2	T2
Sample	19	19	15	20	20	15	26	26	16	176
Mean	1.95	4.11	4.40	1.60	3.70	4.00	2.27	3.77	3.69
SD	1.75	2.35	2.41	1.43	2.13	2.59	2.50	2.50	2.33

Note: RP = relapse prevention, GS = proximal plus distal goal setting.

).

4. Discussion

The purpose of this study was to validate the effectiveness of the post-training interventions in the training literature by using Solomon four-group design. Regarding the three hypotheses for the main effects, the interventions were not statistically different from the control group for all the three outcome variables in contrast to Hypotheses 1 and 2. Only Hypothesis 3, that participants in the proximal plus distal GS group would demonstrate greater transfer than those in the RP group, was partially supported, as self-reported behaviors were different between the RP and GS groups, whereas the hypothesis was not supported for observer-rated behavior and knowledge. Discussion on the results are as follows.

The unexpected insignificance may have resulted from the implementation and evaluation of the interventions and from how trainees perceived their behavior. First, when we look back on previous studies and especially on the three studies of full RP, we see that two studies implemented transfer interventions immediately after the training [11,16], whereas the other study implemented interventions six weeks later [15]. Burke [16] measured transfer three weeks after training and Burke and Baldwin [11] measured transfer six weeks after training, while Rahyuda et al. [15] measured transfer six weeks after training and again six weeks after intervention. Rahyuda et al.’s [15] results supported that full RP enhanced transfer, whereas the first two studies did not show direct significant results for full RP interventions, though they were partially supportive, as full RP enhanced the ability and the desire to transfer. Moreover, full RP modestly influenced transfer based on the transfer climate, but the researchers did not find a direct significant effect for full RP. Based on these findings, it seems that participants who had opportunities to attempt transfer were able to review and evaluate their first transfer attempts before the intervention. After the six-week trial period, participants seemed ready for the transfer intervention, and perhaps, they more fully appreciated how to enhance transfer after learning strategies in the intervention. In our study, we implemented transfer interventions in the days immediately after training, as in the previous two studies. Post-training interventions may have been more effective if we had provided trainees with adequate time for trial and error with transfer attempts. Then, after a period of time to allow for this kind of experimentation, it may have been more effective to implement the transfer interventions.

It is possible that minimal transfer or behavior change can arise from trainees’ thoughts and feelings when confronting their first lapse. “Dichotomous thinking” or all-or-nothing thinking [33] may plague them when they fail to cope in a high-risk situation. For example, they may think, “Let it be, I don’t want to try it again” or “As I already failed in the first attempt, now I don’t want to do it anymore.” If this is the case, trainees may be discouraged from returning to prolapse (their ideal behavior) [34] (p. 3) after their first lapse or slip back into the old behavior. Moreover, because of the kinds of strategies they learn in the RP group, some trainees may be more sensitized to or aware of their relapse compared to those in the other two groups. In turn, they may be less motivated to switch back to the intended behavior, or perhaps, they change their behavior such that it becomes even more negative than before the workshop. As a possible solution for this problem, mindfulness-based RP (MBRP) should be explored as a transfer strategy in the training and development arena, as the effectiveness of RP might depend on the mindfulness of trainees when they encounter high-risk situations. MBRP encourages individuals to increase their awareness of the thoughts, emotions, and sensations that characterize high-risk situations, thereby allowing them to notice and resist urges to revert back to old behavior [35]. According to the tenets of MBRP therapy, mindfulness reduces negative effects, halts and reduces relapse, and mitigates the relationship between negative effects and actual behaviors [35].

Reflecting back on the origins of RP, it is “a self-control program designed to teach individuals who are trying to change their behavior” [8] (p. 261). RP is explicitly designed to help addicted individuals who acknowledge their problem behaviors and wish to change them. What if, in our case, students chose to ignore their current behavior or did not have any desire to change it? Simply teaching the students RP, then, may not have been as effective as anticipated. For example, it is possible that time management was not a pressing issue for the students to address at that point in time. Likewise, for the GS group, we did not monitor the students to see whether they were actually following up on their goals, even though the manipulation checks after each of the intervention sessions ensured that learning had taken place. Perhaps merely teaching the self-help strategies to the trainees was not enough, and we should have determined whether they were really following through the interventions.

In this study, we measured time-management behavior only two times, which may not have been adequate for monitoring behavior change. We collected data through survey questions on paper and via online forms, which may have failed to capture participants’ actual behaviors over time. By striving for high data reliability and by monitoring behavior daily or weekly, future studies may follow up on these discussions about how to measure human relapses [36]. It could be beneficial to utilize interactive voice response, an automated telephony system, for daily or weekly monitoring as previously discussed. Another approach would be to try implementing the timeline follow-back method, which has been used to retrospectively estimate alcoholic drinking on a day-to-day basis through the identification of major events [36]. Students could also be encouraged to keep a logbook, and feedback could be provided based on their records; this approach might assure more a positive behavior change.

5. Conclusions

The present study investigated the effects of two post-training interventions: RP and GS; the results were not supportive, excluding the difference in training transfer between RP and GS in terms of self-reported behavior. We mainly justify these unexpected results based on the specific characteristics of our subjects, who were undergraduate students. This study contributes to the literature by clarifying why results from RP have not been significant. Based on reviews and studies of RP, we originally assumed that the inconsistent results may have arisen from researchers using different RP models: the full versus the modified versions. However, it appears that inconsistencies stem not only from the utilization of different versions but also on trainees’ perceptions of their own behavior: their desire to change, their mindfulness, and their self-monitoring methods. This study also sheds light on how interventions affect transfers—how trainees think about, interpret, learn, and apply interventions for enhancing transfer.