Comparison of Measured and Observed Exercise Fidelity during a Neuromuscular Training Warm-Up

Abstract

Neuromuscular training (NMT) warm-up programs effectively prevent injuries in youth, but monitoring exercise fidelity is challenging. The purpose of this study was to compare the exercise fidelity as measured via an inertial measurement unit (IMU) with direct observations of selected exercises. Youth basketball and soccer players performed single leg jumps, squat jumps, Nordic hamstring curls, and/or single leg balance exercises as part of an NMT warm-up. An IMU was placed on the lower back of each participant and the warm-up was video recorded. A physiotherapist evaluated the volume aspect of exercise fidelity (i.e., performing the prescribed number of repetitions) using the video recordings and a checklist. Algorithms were developed to count the number of repetitions from the IMU signal. The repetitions from the algorithms were compared with the physiotherapist’s evaluation, and accuracy, precision, and recall were calculated for each exercise. A total of 91 (39 female, 52 male) athletes performed at least one of the four warm-up exercises. There was an accuracy, precision, and recall of greater than 88% for all exercises. The single leg jump algorithm classified all sets correctly. IMUs may be used to quantify exercise volume for exercises that involve both impact during landing and changes in orientation during rotations.

1. Introduction

Sport participation provides many health benefits [1,2], but it is not without adverse effects: the prevalence of sport injuries among youth is high [3], and injuries can have long-term consequences such as the increased risk of osteoarthritis [4,5]. Injury prevention programs are important in sports such as basketball and soccer that involve frequent changes of direction, pivoting turns, and landings [6,7]. It has been established in several high-quality randomized controlled trials that neuromuscular training (NMT) warm-up programs, which comprise aerobic, balance, strength, and agility exercises [8], reduce the rate of injuries in youth sports by about 40% [9,10]. However, NMT warm-up programs only prevent injuries when adequate adherence is achieved, as injury rates have been shown to be similar between players with poor adherence and players who do not participate in NMT warm-up programs [11,12].

Exercise fidelity is a key measure of adherence [13,14,15]. Exercise fidelity is defined as athletes performing the exercises with the correct technique and intensity and completing the prescribed sets and repetitions [16]. The essential criteria are defined for each exercise, which cover technique (e.g., maintaining lower limb alignment: hip-knee-ankle), intensity (e.g., knee in high position in front of body), and volume (e.g., at least six repetitions) [17]. One current limitation in determining the dose–response to NMT warm-up programs is that measures of adherence are usually limited to utilization frequency (i.e., how many NMT warm-up sessions the athletes completed in a week), which does not provide information on how many sets or repetitions the athletes performed. Previous studies on NMT warm-up program fidelity have reported that performing the correct number of repetitions was the most frequent failed criterion of exercise fidelity [17], and that the prescribed exercise volume was completed in less than 50% of NMT warm-ups [15].

Thus far, the volume aspect of exercise fidelity has been evaluated through direct observations [15,17,18,19] or a self-report of volume (i.e., completed sets and repetitions). Direct observations are very time consuming and especially difficult to conduct in a team setting: when a full team is conducting a workout, the observer can only have a clear view of the athletes at the end of the line-up [15]. Using video recordings also has limitations as it is not often feasible to record teams with a clear view of each athlete during each exercise [19], and watching videos after the fact delays feedback to athletes and coaches. Using self-report to measure volume, such as a coach or designated person recording the prescribed sets and repetitions, only provides information on what the coach included in the warm-up, not the actual number of repetitions each athlete completed [20]. Wearable technology could offer a potential solution for measuring the quantity of performed repetitions during an NMT warm-up to determine the volume aspect of exercise fidelity. Accurate measures of exercise fidelity can be used to determine the dose–response relationship between NMT warm-up program adherence and the reduction in injury risk.

Inertial measurement units (IMUs) have been shown to be effective and accurate in measuring the volume of some exercises, such as the number of jumps during basketball practices and games [21], gait event detection for running [22], and counting repetitions during gym exercises [23,24,25,26,27,28]. NMT warm-up programs present a slightly different challenge in that they consist of a variety of exercises, including those with high- and low-impact. Nevertheless, wearable technology offers a possible solution for measuring performed repetitions during an NMT warm-up on the athlete level without relying on time-consuming observations. The purpose of this study was to evaluate the accuracy of using a single IMU to measure the volume aspect of exercise fidelity during an NMT warm-up in basketball and soccer. We demonstrate the application of unobtrusive wearable technology and signal-processing algorithms for injury prevention in a real-world setting.

2. Materials and Methods

There were three stages to this study: a workshop for coaches, a youth sport practice that included data collection and post-processing steps, and data analysis. First, youth basketball and soccer coaches attended a sport-specific workshop where they learned an NMT warm-up program and were encouraged to implement the program with their teams. Participants in the current study were players on those teams. The players completed the warm-up prior to a practice while wearing an IMU on the lower back and with video recorded from multiple angles. The analysis was of the four repetition-based exercises that were common between the warm-up programs. The outcome was a comparison between the algorithm and the gold standard of observation. A flow chart of the methods is depicted in Figure 1.

2.1. NMT Warm-Up Workshop

Youth basketball and soccer coaches attended workshops where they learned an NMT warm-up program designed for their sport and were encouraged to implement the program prior to practices and games [19,29,30]. Basketball and soccer were chosen as both sports involve frequent changes of direction, pivoting turns, and landings, which are common mechanisms for lower extremity injuries [6,7]. In the workshop, the coaches were instructed on how to deliver the sport-specific NMT warm-up program as intended, which covered the correct technique for each exercise and the prescribed volume (sets and repetitions/distance covered/time) for each exercise [19]. The basketball NMT warm-up program comprised 13 exercises while the soccer NMT warm-up program comprised 9 exercises [19,29,30]. Both programs included aerobic, balance, strength, and agility exercises. Most exercises had two levels of progression, and teams were advised to complete one level in each session.

Participants in the current study were a convenience sample of youth basketball and soccer players recruited from teams whose coaches attended the workshops. The basketball players were on high school teams and the soccer players were on youth club teams in the Calgary, AB, Canada area. The study was approved by the Conjoint Health Research Ethics Board of the University of Calgary (REB16-0864, REB19-2030). All participants provided informed consent to participate.

2.2. NMT Warm-Up in Practice

2.2.1. Data Collection

Data were collected at the beginning of a regularly scheduled team practice. Participants completed their team warm-up. Participants wore their own clothing and shoes. A triaxial IMU was worn on the lower back in line with the navel and secured with an elastic belt [22,31,32]. The IMU worn by basketball players (Shimmer3 GSR+^®, Shimmer Inc., Dublin, IE, USA) was sampled at approximately 200 Hz, and the IMU worn by soccer players (Blue Trident^®, Vicon Motion Systems Ltd., Oxford, UK) was sampled at approximately 1000 Hz. While 3D accelerometer (accelerations, m·s⁻²), gyroscope (angular velocity, °·s), and magnetometer (orientation, gauss) signals were recorded, only the accelerometer and gyroscope signals were used in this analysis. The IMUs were oriented such that the positive x-axis pointed up (vertical axis), the positive y-axis pointed left (medial–lateral axis), and the positive z-axis pointed posteriorly (anterior–posterior axis).

Video of the warm-up was recorded from multiple angles to ensure that all participants were captured by at least one camera at all times [19]. Depending on the size of the field/court, two or three cameras were positioned on the baseline opposite to where the team started the warm-up exercises and captured the front view. Two cameras were held by research staff on either side of the area to capture each side view. The side camera operators were able to adjust their position throughout the warm-up to follow participants as they moved.

2.2.2. Syncing

Since multiple IMUs and multiple video cameras were used in each data collection, syncing protocols in both the data collection and post-processing stages were employed to create a common timestamp between all IMUs and all videos for each warm-up session. The procedures to sync the IMU and video data differed between basketball and soccer due to the different IMU devices used in different settings. Prior to collecting the basketball data, the IMUs were synced to each other by securing all IMUs in the same orientation in the same rigid case and striking the case five times on a hard surface. Then, a single IMU was held in view of all cameras and struck five times on a hard surface. In the post-processing stage, the impact peaks were identified from the accelerometer signal of each IMU and used to create a common IMU timestamp using a custom Matlab script (9.9.0.1467703 (R2020b), Mathworks, Inc., Natick, MA, USA) [22,31,32]. Once the IMUs were synced to each other, the video frame where the IMU struck the surface was tagged in each video using video-tagging software (Dartfish 10 Live S, 10.0, build 20405, Fribourg, Switzerland) and a custom Matlab script matched the impact frames from the video with the impact peaks from the accelerometer signal of the single IMU, resulting in a common timestamp for all videos and IMUs. During the soccer data collection, one video and all IMU data were recorded using the same iPad app (Capture.U 1.3, Vicon Motion Systems Ltd., Oxford, UK) and thus had a common timestamp. Additional cameras were used to achieve multiple angles of video recording and were synced to the iPad video by bouncing a ball on the ground five times in view of all the cameras and the iPad video. In the post-processing stage, the video frame in which the ball contacted the ground was tagged and a custom Matlab script was used to create a common timestamp for all videos and IMUs [33,34].

2.2.3. Post-Processing

The video recordings were tagged to identify the start and end of each exercise set for each participant [21]. Additionally, a stationary period was tagged, during which the participant was standing upright and not moving for at least two seconds. The video tags were used to segment the IMU data into sections for each exercise and the stationary period. The stationary triaxial accelerometer signal was used to correct the IMU tilt such that the vertical axis was aligned with gravity for all IMU data [35].

2.3. Analysis

Only exercises for which a number of sets and repetitions were defined, as opposed to a prescribed distance (e.g., 3 × 50 m) or time (e.g., 2 × 30 s), and were completed by at least half of the study participants were included in the present study. Thus, four exercises were included: single leg jumps (forward to backward), squat jumps, Nordic hamstring curls, and single leg balances [29].

In single leg jumps, the participants jumped forward and returned to the original spot with a backward jump on the same leg, completing at least 10 repetitions (5 forward and 5 backward jumps). In squat jumps, the participants started from a squatting position, performed a vertical jump with both legs, and landed in a squat, completing at least 6 repetitions. For Nordic hamstring curls, the participants started by kneeling on a mat or other similar padding with their body upright, then slowly leaned forward while their ankles were held in place by a partner, completing at least 3 repetitions. For single leg balances, the participants held a single leg stance with arms at their sides at shoulder height, hinged from the hip while remaining balanced, then returned to a starting position, completing at least 3 repetitions per leg.

An observation tool was developed to evaluate exercise fidelity across a number of criteria (Appendix A). The criteria were used to assess whether the exercises were performed as intended, including completion of the prescribed number of repetitions of each exercise (e.g., at least 10 repetitions of squat jumps). The coaches had learned about the technique and prescribed volume during the workshop. Only the exercise volume criterion was used in this analysis. The fidelity observation tool was based on previously developed observational tools [17,18,19]. A physiotherapist (AMR) with over 10 years of experience in assessments of exercise fidelity viewed the video recordings of the NMT warm-ups. The physiotherapist was permitted to view the video recordings as many times as needed. Using the fidelity observation tool, the physiotherapist evaluated each exercise set as ‘Yes’ or ‘No’ according to whether the participant performed at least the prescribed number of sets and repetitions. A ‘Yes’ or ‘No’ designation was used for all criteria in the fidelity observation tool; therefore, the exact number of repetitions observed in the video was not recorded. This evaluation was considered the gold standard.

Algorithms were developed to count the repetitions of each exercise (Appendix B). The single leg and squat jump algorithms utilized the accelerometer signal only (Figure 2), while the Nordic hamstring curl and single leg balance algorithms utilized the accelerometer and gyroscope signals (Figure 3). The number of repetitions produced by the algorithm for each exercise set was assigned ‘Yes’ or ‘No’ according to whether the algorithm counted that the participant performed at least the minimum prescribed exercise volume.

The algorithm was compared to the physiotherapist’s evaluation by calculating the number of true yes, true no, false yes, and false no, which were used to calculate the algorithm’s accuracy, precision, and recall [36]. Accuracy, precision, and recall are presented as proportions (%).

3. Results

A total of 91 youth athletes participated. There were 27 basketball players (17 F: 16.7 (0.8) years, 169.8 (7.5) cm, 64.5 (14.3) kg; 10 M: 17.6 (0.5) years, 184.2 (8.7) cm, 76.8 (9.3) kg) and 64 soccer players (22 F: 15.0 (1.2) years, 162.8 (5.9) cm, 58.6 (8.6) kg; 42 M: 14.6 (2.9) years, 165.1 (15.8) cm, 57.3 (17.7) kg).

Each participant performed at least one of the four warm-up exercises. All participants performed squat jumps, 48 athletes performed single leg jumps, 49 athletes performed the Nordic hamstring curls exercise, and 63 athletes performed the single leg balance exercise (

Table 1. Algorithm performance.

Exercise	Individual Sets					Total Sets	Accuracy	Precision	Recall
	Sport	Sex	n	R	L
Single Leg Jumps	Basketball	F	11	11	11	95	100	100	100
		M	9	9	9
	Soccer	F	6	6	6
		M	22	21	22
Squat Jumps	Basketball	F	17	-	-	91	95	97	96
		M	10	-	-
	Soccer	F	22	-	-
		M	42	-	-
Nordic Hamstring Curls	Basketball	F	9	-	-	49	96	096	100
		M	0	-	-
	Soccer	F	22	-	-
		M	18	-	-
Single Leg Balance	Basketball	F	0	0	0	126	89	90	96
		M	0	0	0
	Soccer	F	22	22	22
		M	41	41	41

R = set completed on right leg; L = set completed on left leg.

). The algorithm was tested on all sets of each exercise, including left and right sets when the single leg exercises were performed on each side individually. There was a greater than 95% accuracy, precision, and recall for all exercises except the single leg balances which had an accuracy of 89%, a precision of 90%, and a recall of 96%. The single leg jump algorithm classified all sets correctly.

4. Discussion

The high accuracy of the algorithms suggests that IMUs can be used to determine the volume aspect of exercise fidelity during NMT warm-up programs in basketball and soccer. The developed algorithms demonstrate the utility of a single IMU for detecting repetitions for both high-impact (single and squat jumps) and low-impact (Nordic hamstring curls and single leg balance) exercises. These methods may be expanded to effectively evaluate the volume aspect of exercise fidelity in other high- or low-impact exercises included in an NMT warm-up program. The use of a single IMU and simple peak detection algorithms should facilitate providing real-time feedback on exercise volume.

Comparisons with previous research are limited as previous studies determining exercise fidelity during NMT warm-ups have relied on visual observations [17], and have not compared visual observations to IMUs. There are some studies that have investigated using IMUs to identify exercise volume, with the complexity of the task dependent on whether the exercise is cyclical or acyclic, and high impact or low impact. In cyclical exercises, the event is repeated in a periodic nature, while acyclic events may occur randomly or with other events interspersed throughout the exercise period. Accelerometer-based peak detection is commonly used to identify repetitions for cyclical events with high impacts such as step detection during running [22]. Jump landings are also high-impact events, though they are not always cyclical in sports such as basketball and volleyball. The acyclic nature of jumping in basketball and volleyball [21,37], and overhead exercises such as throwing and serving [38,39], require the validation of IMU-based exercise volume with visual observation of the events.

Identifying peaks in the accelerometer signal is not useful for exercises with lower impacts. In the current study, the Nordic hamstring curls and single leg balance exercises did not involve large impacts, such as those present during landing from the single and squat jumps. The Nordic hamstring curls and single leg balance exercises involved rotation about the medial-lateral axis, which was identified by detecting peaks in the gyroscope signal about the medial-lateral axis. While not used for peak detection during low-impact exercises, the accelerometer signal can be used to identify shifts in orientation when the IMU alignment with gravity changes from primarily one axis (e.g., vertical) to another (e.g., anterior-posterior), as was the case with the hamstring curls and single leg balance exercises.

The slightly poorer performance of the single leg balance algorithm compared to the other three algorithms may be due to differences in how the single leg balance exercise was performed across teams. Some teams remained in one location while balancing, while others took steps in-between repetitions. The pace at which the exercise was completed differed, with some teams taking a long time to complete each repetition and others performing the exercise more quickly. Due to these differences, the algorithm was not constrained by the time between repetitions. Additionally, during this exercise and unlike the other exercises, a frequent loss of balance and stumbling resulted in an unexpected IMU orientation and may have influenced the efficacy of the algorithm to detect the expected movement pattern. It is likely that a better algorithm performance could be achieved with additional calibrations or subject-specific algorithms [24]; however, the purpose of this study was to demonstrate the use of generic algorithms for a number of youth athletes, including males and females playing different sports, and across multiple exercises.

The accuracy of the algorithms in the present study is similar to the performance of other algorithms to count repetitions using peak detection and/or autocorrelation techniques, with reported accuracies of 85% to 99% [24,25,26,27,28]. There are limitations to relying on peak detection for counting repetitions, as identifying which peaks are important often requires assumptions to be made about the underlying IMU signal. In the current study, the expected pattern of multiple peaks during squat jumps was included as a constraint in the algorithm. Additional methods for identifying important peaks within a signal involve prior knowledge of peak magnitude. Instead of a threshold based on the magnitude of a peak, peak prominence was used to identify major peaks that corresponded to a repetition. Peak prominence is the height of the peak above its highest adjacent trough. Thus, prominent peaks stand out from the rest of the signal, but the actual magnitude of the peak does not have to reach a specific threshold, which could vary depending on the speed or intensity with which an exercise is performed. In this study, the minimum peak prominence was set for each algorithm.

Autocorrelation is a method that requires fewer constraints for peak detection, as it identifies repeating patterns within the signal but does not search for a specific pattern (e.g., a specific peak prominence). Autocorrelation assumes repetitions are repeated at relatively consistent intervals, and is efficient for exercises such as running and walking that are highly periodic [25]. It is more difficult to account for repetitions using autocorrelation when there are pauses within the set or the number of repetitions is low [24]. For example, the peak detection-based single leg balance algorithm in the current study still performed well even though some participants took steps in-between repetitions and others did not. It is likely that an autocorrelation approach would have incorrectly identified those steps as repetitions.

There are a few limitations to this study. First, the algorithms were tested on data that were already segmented by exercise. In some sets, the time between repetitions contained movements that were not relevant to the prescribed exercise. Nevertheless, a greater challenge would be applying the algorithm to data that contain multiple exercises. An exercise recognition step would be required to identify each exercise set before the repetition-counting algorithm could be used. Several studies have created both exercise-recognition and repetition-counting algorithms in parallel [23,24,25,26,27,28,38,39], and future work should consider exercise recognition as a necessary component of automating a system for monitoring exercise volume. Alternatively, while the exercise sets in this study were segmented based on tagged videos, it is possible that an app-based system designed to record IMU signals during an NMT warm-up could receive inputs from the user or instruct the user when to start and stop each exercise. It must also be taken into account that the premise of this study was to compare the IMU-based detection to the “gold standard”, observations by a trained expert on whether or not pre-determined criteria were met [17,18,19]. Since these checklists have been based on ‘Yes’/’No’, the physiotherapist did not record the actual number of repetitions performed but only recorded if the minimum number of repetitions was completed. Therefore, based on the current data it is not possible to determine if the algorithm and physiotherapist determined the same number of repetitions. This should be explored in future studies. Another limitation of this study is that algorithms were developed for only four exercises within the NMT warm-up program. Examples of other exercises include running or skipping a prescribed distance, or acyclic exercises such as holding a plank for a prescribed amount of time. In these cases, exercise volume can be accounted for through processes other than repetition counting. Future work should focus on developing methods to evaluate exercise volume for these exercises.

The findings of the present study can be applied in future research studies focusing on measuring the exercise fidelity of NMT warm-up program interventions in order to determine the dose-response relationship of the program and reduction in injury rates. Currently, more research is needed to explore the evaluation of exercise volume during NMT warm-up exercises beyond those included in the present study, and also to use IMUs to determine the movement quality aspect of exercise fidelity. While this technology is currently not easily implemented in coaching practice, the goal is to develop technology that is accessible to coaches and others who work in a youth sport setting.

5. Conclusions

The results of this study indicate that IMUs can accurately quantify the volume aspect of exercise fidelity for both high- and low-impact exercises that are part of an NMT warm-up program for youth basketball and soccer players. Using this approach, it may be possible to develop a system that evaluates exercise fidelity during an NMT warm-up program.