Comparing the Relative Efficacy of Generalized Estimating Equations, Latent Growth Curve Modeling, and Area Under the Curve with a Repeated Measures Discrete Ordinal Outcome Variable

Abstract

Researchers are often interested in how changes in one variable influence changes in a second variable, requiring the repeated measures of two variables. There are several multivariate statistical methods appropriate for this research design, including generalized estimating equations (GEE) and latent growth curve modeling (LGCM). Both methods allow for variables that are not continuous in measurement level and not normally distributed. More recently, researchers have begun to employ area under the curve (AUC) as a potential alternative when the nature of change is less important than the overall effect of time on repeated measures of a random variable. The research showed that AUC is an acceptable alternative to LGCM with repeated measures of a continuous and a zero-inflated Poisson random variable. However, less is known about its performance relative to GEE and LGCM when the repeated measures are ordinal random variables. Further, to our knowledge, no study has compared AUC to LGCM or GEE when there are two longitudinal processes. We thus compared AUC to LGCM and GEE, assessing the effects of repeated measures of psychological distress on repeated measures of smoking. Results suggest AUC performed equally well with both methods, although missing data management is an issue with both AUC and GEE.

1. Introduction

Statistical analysis options for repeated measures designs vary depending upon the nature of the outcome variable assessed. When the outcome variable (dependent variable) is continuous and normally distributed (multivariate normal), there are more analysis options than when the dependent variable is discrete. Most students taking statistics courses in undergraduate or even some graduate programs learn little about the different statistical analysis methods available for dealing with repeated measures designs, especially when the dependent variable is discrete. The longitudinal data analysis method they are most likely to encounter is repeated measures ANOVA (RM-ANOVA). Although providing some flexibility, such as the ability to assess differences among time points and shape of change over time in a post hoc analysis (e.g., linear or quadratic), RM-ANOVA has limitations when compared to more advanced multivariate statistical methods, such as generalized estimating equations (GEE) and latent growth curve modeling (LGCM). Both of these methods, however, have advantages and disadvantages as well. A more recent method beginning to be employed in the assessment of repeated measures of a variable of interest is area under the curve (AUC). Researchers are now assessing its comparative effectiveness in relation to these methods.

GEE is a multivariate method that has several advantages over RM-ANOVA [1,2]. First, unlike RM-ANOVA, GEE permits repeated measure variables to be discrete or continuous. Second, there is no need to specify the multivariate distribution as the aim is parameter estimation instead of model testing, and GEE provides normally distributed and consistent parameter estimates by simply specifying the correct mean structure [3]. RM-ANOVA, by contrast, assumes the repeated measures are multivariate normally distributed [4]. Third, GEE permits the selection of the appropriate correlation structure (i.e., the correlations among the repeated measures), and the efficiency of parameter estimates does not suffer from an incorrectly specified correlation structure. RM-ANOVA assumes the correlations between measures remain constant [5]. Further, GEE permits estimation of more complicated relations in a single model, such as when one desires to explore the relation between two repeated measures variables (e.g., the effect of change in one variable on change in a second variable). Assessing the effects of time-varying covariates is not easily available in RM-ANOVA, although one can format the data to permit the use of a time-varying covariate [6].

Equation (1) presents a basic GEE cumulative logit model with three covariates (x₁ through x₃; the number of covariates employed in the present study), where the logit is the link function relating the covariates to the repeated-measures outcome variable Y_it, j is the level of the ordinal variable Y, where j ranges from 1 to J-1, and t represents time (i.e., the unit of time for the repeated measures). Equation (2) is the general cumulative logit model relating the vectors of predictor variables (X) and parameters (β) to the vector of outcomes (Y), with the mean structure being the probability of the j^th level of the ordinal outcome variable y for individual i at time t (Equation (3)) [3].

$$logit\left[pr\left(y_{it}\le j\right)\right]={\beta }_{0j}+{\beta }_1{x}_{i1}+{\beta }_2{x}_{i2}+{\beta }_3{x}_{i3}+{\beta }_{4}t$$

(1)

$$logit\left[pr\left(Y_{it}\le j/X_{it}\right)\right]={\beta }^{\prime }{X}_{it}$$

(2)

$$\begin{gathered} \begin{array}{cc}E(Y_{it}/X_{it};\beta )& ={\left({\mu }_{it1},\dots ,{\mu }_{it\left(J-1\right)}\right)}^{\prime },where {\mu }_{it}(mu)\hfill \\ & = pr\left(O_{it}=j/X_{it};\beta \right),\hfill \end{array} \\ and O_{it} is the ordinal response \left\{\mathrm{1,2},3,\dots J\right\} for individual i at time t. \end{gathered}$$

(3)

While there are indeed many advantages to GEE over RM-ANOVA, limitations include the inability to test models and compare different models since GEE lacks a likelihood function. Further, the treatment of missing data is a question. With GEE, one assumes that data are missing completely at random (MCAR), meaning that the available cases are a random sample of all cases (absent missing data) [3,7,8]. Although plausible, MCAR is less likely than data being missing at random (MAR). When the data are MAR, this indicates that missingness is related to a predictor variable (or predictor variables) or prior measures of the dependent variable. This is a less restrictive assumption. Although there are extensions to model missing data in GEE for cases that are not MCAR [9,10], these are less likely to be employed in a naive analysis.

Another issue related to modeling discrete dependent variables in GEE is the use of correlation coefficients to represent the relations among the repeated measures. Simulation studies suggest that the best measure of association among repeated measures of discrete data is the local odds ratio (LOR) rather than correlation coefficients [11,12]. Although we did not address this limitation in our GEE analysis in this study, as our aim is to conduct a simple (naive) analysis that most researchers would do, it is noted that there are more efficient ways to estimate associations than correlation coefficients.

Many of the restrictions encountered with GEE are not met when using LGCM with repeated measures data. LGCM is a multivariate method that employs unobserved (latent) variables to represent initial level (baseline) and rate of change from baseline (trend) [13,14]. Equation (4) presents the relations among the latent and observed variables in a prototypical LGCM with ordinal data [15].

$$\begin{gathered} y=\nu +\mathsf{\Lambda }\eta +\epsilon , \\ where y is a vector of observed ordinal outcomes, v is a vector of regression intercepts, \\ \mathsf{\Lambda }\left(lambda\right) is a matrix of factor loadings relating the observed variables to the vector of \\ latent variables \eta \left(eta\right), and \epsilon \left(epsilon\right) is a vector of residuals. \end{gathered}$$

(4)

Equation (5) clarifies Equation (4) with respect to three repeated measures, i.e., the number of repeated measures employed in the present study.

$$\left[\begin{array}{c}{y}_{i1}\\ y_{i2}\\ y_{i3}\end{array}\right]=\left[\begin{array}{c}0\\ 0\\ 0\end{array}\right]+\left[\begin{array}{cc}1& 0\\ 1& 1\\ 1& 2\end{array}\right]\times \left[\begin{array}{c}{\eta }_0\\ {\eta }_1\end{array}\right]+\left[\begin{array}{c}{\epsilon }_{i1}\\ {\epsilon }_{i2}\\ {\epsilon }_{i3}\end{array}\right]$$

(5)

Working from left to right, we have a vector of ordinal outcome variables for participant i at time t for T = 3 time points. On the right side of the equation, we have a vector of regression intercepts, set to zero. The Λ matrix includes the factor loadings for the intercept factor (first column) and the slope factor (second column). The intercept loadings are set to one, indicating an unchanged relation between the intercept and the repeated measures. The unit increasing factor loadings in the second column suggest linear growth over time. The next vector includes the two factors (η), the first representing the intercept factor and the second representing a linear trend factor. If we had proposed a quadratic trend, there would have been a third factor term. For a cubic trend, there would have been a fourth factor term. The final vector represents the residual, one for each time point. These are assumed to be uncorrelated, although at times researchers may correlate them to improve model fit to the data. However, this is generally not recommended [16,17,18].

The latent variables (η) underlying the observed ordinal variables are continuous, not discrete. To capture progression across increasing levels of the ordinal variable, LGCM uses thresholds (τ; tau). Thresholds are cut points in a continuum representing propensities to progress from one ordinal category to the next in an observed variable [15,19]. To assess change over time, these thresholds are constrained to equality across time points. As such, a change in participant proportions across ordinal levels, increasing from a lower to a higher category over time, results in an increased propensity to cross thresholds to higher levels of the dependent variable.

With respect to the latent variables, they are assumed to be multivariate normally distributed, with a mean α (alpha) and variance/covariance ψ (psi; Equation (6)) [15,19]. The residuals are assumed to be normally distributed, with a mean of 0 and a variance θ (theta; Equation (7)). For a very detailed presentation on LGCM with ordinal data, see the Mehta et al. reference [15]. See also Masyn et al. reference [19]. For a general introduction to growth modeling, see Duncan and Duncan [13].

$$\eta ~MVN\left(\alpha ,\psi \right)$$

(6)

$$\epsilon ~N(0,\theta )$$

(7)

LGCM also permits researchers to assess the effect of potential predictor variables on baseline and trend factors. In addition, one can assess how changes in another repeated measures variable influence the repeated measures of the dependent variable, what is commonly known as an associative (parallel) process in LGCM [20]. This could involve, for instance, assessing how a baseline level of one process influences a trend for a second process, as well as trend-to-trend effects. We represent effects on the growth factors in Equation (8).

$$\begin{gathered} \eta =\alpha +\Gamma \xi +{\rm B}\eta +\zeta , where \alpha is the intercept, \\ \Gamma \left(\mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}\right) is a matrix of coefficients relating observed predictors to the latent variables, \\ \xi \left(Xi\right) is the vector of observed predictor variables, \\ {\rm B} \mathrm{i}\mathrm{s} \mathrm{a} \mathrm{m}\mathrm{a}\mathrm{t}\mathrm{r}\mathrm{i}\mathrm{x} \mathrm{o}\mathrm{f} \mathrm{c}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{s} \mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{n}\mathrm{g} \mathrm{t}\mathrm{h}\mathrm{e} \mathrm{l}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t} \mathrm{v}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{a}\mathrm{b}\mathrm{l}\mathrm{e}\mathrm{s} \eta \mathrm{t}\mathrm{o} \mathrm{o}\mathrm{n}\mathrm{e} \mathrm{a}\mathrm{n}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}, \\ \mathrm{a}\mathrm{n}\mathrm{d} \zeta \left(Zeta\right) is the vector of residuals. \end{gathered}$$

(8)

Advantages of LGCM include a likelihood function, allowing for model comparisons. In addition, one can test multiple hypotheses, exploring the effects of predictors on repeated measures of a dependent variable. Moreover, one can assess the effects of initial level and rate of change on other outcome variables, whether latent or observed. Further, like GEE, one can assess change on a variety of levels of a dependent variable, whether continuous or discrete. Another advantage of LGCM, not shared by GEE, is how it handles missing data. Unlike GEE, which assumes MCAR, LGCM assumes data are MAR. This is a far more tenable assumption. Using full information maximum likelihood (FIML) estimation, LGCM uses all available data on the dependent variables to estimate parameters [21]. Thus, it has the same sample size as GEE, but with a less restrictive assumption.

Despite the advantages of both methods, conducting GEE and LGCM requires more than a rudimentary understanding of statistics. Moreover, although not necessarily the case for GEE, one needs specific software to conduct LGCM (e.g., Mplus). This can be an unnecessary impediment to researchers if they are interested in the assessment of longitudinal processes but not in the impact of selected covariates on rates of change or the shape of development (e.g., linear, quadratic, or cubic). For instance, a researcher may wish to assess whether males or females differ in psychological distress (PD) over multiple time points, but not whether males differ from females in the rate of change in PD from baseline or whether change in PD is linear or quadratic in nature. Further, one important limitation with LGCM and GEE alike is related to the nature of ordinal variables, the type of discrete random variable we are employing in our analysis. When modeling with an ordinal random outcome variable (Y) with k categories, one assumes proportional odds, such that the odds related to a set of predictor variables (X) remain constant when comparing different levels of the dependent variable [22,23,24,25]. This is seen in Equation (1), where we predict the log odds (logit) of being in category ≤ j vs. > j given a set of predictor variables (x). However, this assumption may not always hold in practice, meaning that multinomial logistic regression instead of ordinal logistic regression is the more appropriate analysis option.

One existing method that researchers are recently beginning to investigate as a viable alternative for longitudinal data analysis is area under the curve (AUC). With this method, one merely calculates the area under the curve generated by the repeated measures. Equation (9) presents the calculation of area under the curve for repeated measures analysis, known as AUC with respect to the ground [26,27,28]. Employing the trapezoid rule for calculating AUC across T time points, we have

$$\begin{gathered} \begin{array}{cc}{AUC}_{ground}& = \left[{\displaystyle \frac{\left(y_2+y_1\right)}{2}}\times \left(x_2-x_1\right)\right]+\left[{\displaystyle \frac{\left(y_3+y_2\right)}{2}}\times \left(x_3-x_2\right)\right]+\dots \hfill \\ & +\left[{\displaystyle \frac{\left(y_T+y_{T-1}\right)}{2}}\times \left(x_T-x_{T-1}\right)\right],\hfill \end{array} \\ where y_t is the y - axis value and x_t is the x - axis value at timepoint t. \end{gathered}$$

(9)

If we let t_i represent our intervals x_t − x_t−1, there will be one less interval than time points (Equation (10)).

$${AUC}_{ground}=\sum _{i=1}^{T-1}\left[{\displaystyle \frac{\left(y_{i+1}+y_i\right)}{2}}\right]\times t_i$$

(10)

If the intervals are constant in length, Equation (10) reduces to Equation (11):

$${AUC}_{ground}=\sum _{i=1}^{T-1}{\displaystyle \frac{t}{2}}\left(y_{i+1}+y_i\right)$$

(11)

Researchers have shown that AUC performs as well as LGCM when the data are continuous or discrete counts [26,27]. However, to our knowledge, no study has yet compared AUC to GEE or LGCM using discrete ordinal random variables.

AUC has several advantages over GEE and LGCM. First, instead of multiple dependent variables to form growth curves, AUC is a single variable that can be employed in simpler statistical methods, such as regression analysis, t-tests, and ANOVAs, as well as multivariate methods such as structural equation modeling (SEM). Second, one does not need advanced statistical knowledge to understand AUC, although some basic knowledge about distributions (e.g., normal or Poisson) is a plus. Further, there is no need to consider issues such as the proportional odds assumption when calculating AUC, as the primary concern is the area created by the different rectangles covering the mass or density for the T-1 intervals.

Despite these advantages, there are some key limitations. First, in calculating AUC without any modifications, individuals missing data at a given time point are deleted listwise, meaning the entire record for that participant is expunged, resulting in a potentially drastic reduction in sample size. As such, AUC assumes data are MCAR, and data imputation methods are necessary to reduce the number of missing cases. Second, one must write syntax to calculate AUC for repeated measures designs at this point, and this may be difficult for those with limited experience with syntax and coding. Acknowledging these limitations, though, AUC may provide a useful alternative to other repeated measures statistical methods. As such, the purpose of the present study was to assess the relative efficacy of AUC compared to GEE and LGCM using real data from a publicly available data source. Using data from the Panel Study of Income Dynamics (PSID) Transition to Adulthood (TA) study and all three data analysis methods, we assessed the impact of repeated measures of a continuous predictor variable, psychological distress, on repeated measures of an ordinal dependent variable, nicotine use, both measured in 2017, 2019, and 2021.

The rationale for selecting these variables is as follows: In this study, we assessed the relation between psychological distress and smoking in older adolescents and young adults (ages 18–24). Smoking is the leading cause of preventable disease and death in the United States [29], and while combustible cigarette smoking has decreased in recent years, e-cigarette smoking (vaping) has been on the rise, and both remain serious risk factors for poor health outcomes [30,31]. A potential reason for smoking in older adolescents and young adults is the rising prevalence of mental health issues [32]. Nicotine can temporarily relieve symptoms of mental illness, such as anxiety and depression, and many people suffering from these issues may use it to self-medicate, leading to nicotine dependence [33,34,35,36]. In the long term, however, smoking can increase susceptibility to anxiety and the severity of depression, exacerbating poor mental health [36,37]. Thus, studying the relation between psychological distress and smoking in older adolescents and young adults may help to find solutions to both problems. Finally, we also controlled for race and education level, as these are two factors that are known to affect smoking status [38].

2. Materials and Methods

Participants and procedures: Participants were drawn from a total sample size of n = 4222 young adults (18–24 years old) at each of three data collection waves (2017, 2019, and 2021) who were taking part in the Panel Study of Income Dynamics (PSID) Transition to Adulthood (TA) supplement. Started in 1968, the PSID is the longest continuously running cohort study [39]. Although its original purpose was to understand the intergenerational transmission of poverty, subsequent data collections have included a variety of variables covering health and other experiential domains. The final sample sizes per analysis differed due to missing data and its treatment in each statistical method, with 2510 participants for the GEE and 2511 participants for the LGCM and AUC, after multiple imputation for the AUC analysis. The large number of participants missing per wave resulted from the transitional nature of this supplement. Participants age out, and new participants enter the TA sample during each data collection wave. However, we were only interested in following the same participants (cohort) across the data collection waves to meet our aim. We found that the three waves selected maximized our cohort size, as adding additional waves resulted in a drastic decrease in sample size.

Instrumentation: To estimate the use of nicotine across time, we generated a four-level ordinal variable from two PSID questions related to one’s nicotine use behavior (i.e., “Do you smoke cigarettes?” and “Have you ever vaped?”). Values are 0—does not use a nicotine product, neither combustible cigarettes (smoking) nor electronic cigarettes (e-cigarette; vaping), 1—vaped or vapes, 2—smokes, and 3—vaped/vapes and smokes. We assessed psychological distress with a variable that is the sum of six five-point (0 through 4) Likert-style questions asking how often in the past month the respondent felt nervous, hopeless, restless, everything an effort, too sad, and worthless. Scores ranged from 0 to 24 points. Finally, we controlled for race (0—non-White and 1—White) and education (0—greater than a high school education and 1—high school education or less).

Data analysis methods: We conducted GEE, LGCM, and AUC. Our GEE analysis included four predictor variables: the time-varying covariate psychological distress, the two time-invariant covariates education and race (both measured in 2017), and time (i.e., data collection wave). We also included two interaction terms, one each for the time-invariant covariates: race by time and education by time. This allowed us to assess whether the effects of education and race smoking in 2017, if any, changed across time.

For the LGCM, we conducted an associated-processes model, with one process for repeated measures of psychological distress and a second process for smoking. Each LGCM included two factors, one for the intercept and one for a linear trend. In each LGCM, we controlled for the effects of the time-invariant covariates, education and race, on the baseline level and linear trend factors. We also controlled for the effect of the baseline level of the opposite LGCM on each trend factor (i.e., the effect of baseline smoking on the psychological distress trend and the effect of baseline psychological distress on the smoking trend). Finally, we assessed the effect of psychological distress trend on smoking trend. To assess the fit of our LGCM to the data, we used the chi-square test of model fit, the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean residual (SRMR). Heuristics for good fit include a non-significant chi-square, CFI ≥ 0.95, RMSEA ≤ 0.05, and SRMR ≤ 0.06 [40,41,42].

For AUC, we employed multiple regression analysis to assess the effects of psychological distress AUC on smoking AUC, controlling for race and education, both measured in 2017.

3. Results

3.1. Descriptive Statistics

Table 1. Descriptive statistics.

Discrete Variables
Variable	Level	N	%
Race	White	1097	26.0
	Non-White	1414	33.5
	System missing	1711	40.5
Education	≤High school	1055	25.0
	>High school	1471	34.8
	System missing	1696	40.2
Nicotine use (2017)	Does not smoke	1674	39.6
	Vapes/vaped	473	11.2
	Smokes	185	4.4
	Smokes and vapes/vaped	188	4.5
	System missing	1702	40.3
Nicotine use (2019)	Does not smoke	1642	38.9
	Vapes/vaped	642	15.2
	Smokes	122	2.9
	Smokes and vapes/vaped	142	3.4
	System missing	1674	39.6
Nicotine use (2021)	Does not smoke	1438	34.1
	Vapes/vaped	729	17.3
	Smokes	54	1.3
	Smokes and vapes/vaped	88	2.1
	System missing	1913	45.3
Continuous variables
Variable	N	Mean	SD
Distress 2017	2512	4.91	4.00
System missing	1710
Distress 2019	2519	6.10	5.35
System missing	1703
Distress 2021	2309	6.72	5.73
System missing	1913

presents the descriptive statistics for our sample, including missing data per variable.

3.2. GEE

Table 2. GEE results (n = 2510).

Variable	B	SE	95% CI
			Lower	Upper	Wald	p-Value
≤High school	0.139	0.2258	−0.304	0.581	0.377	0.539
White race	0.191	0.2368	−0.273	0.655	0.651	0.420
Psychological distress	0.052	0.0116	0.029	0.075	20.274	<0.001
Time 3	0.720	0.0932	0.537	0.903	59.705	<0.001
Time 2	0.293	0.0677	0.160	0.426	18.759	<0.001
High school*Time 3	0.117	0.1174	−0.113	0.347	0.994	0.319
High school*Time 2	0.053	0.0868	−0.117	0.223	0.378	0.539
Race*Time 3	−0.143	0.1161	−0.371	0.084	1.519	0.218
Race*Time 2	−0.049	0.0838	−0.213	0.115	0.342	0.559

presents the results of the GEE analysis. Psychological distress and time had significant and positive effects on nicotine use. Converting the logits (log odds) to odds ratios, each unit increase in psychological distress across the three time points was associated with a 5% increase in the odds of progressing to a higher level of smoking, from not smoking to smoking both combustible and e-cigarettes (OR = 1.053; 95% CI = 1.029, 1.078). Compared to baseline (time 1; 2017), time 2 (2019) and time 3 (2021) were associated with a 34% (OR = 1.34; 95% CI = 1.174, 1.531) and an over two-fold (OR = 2.054; 95% CI = 1.711, 2.467) increase in the odds of progressing to a higher level of smoking, respectively.

3.3. LGCM

Table 3. LGCM results (n = 2511).

Intercept					Slope
Smoking
	b	SE	z-Value	p-Value	b	SE	z-Value	p-Value
Distress slope	-	-	-	-	0.011	0.01	1.129	0.259
Distress intercept	-	-	-	-	0.037	0.005	6.777	<0.001
High school	0.364	0.05	7.363	<0.001	−0.082	0.027	−3.063	0.002
White	0.251	0.049	5.095	<0.001	−0.058	0.027	−2.176	0.03
Psychological distress
	b	SE	z-value	p-value	b	SE	z-value	p-value
Smoke slope	-	-	-	-	-	-	-	-
Smoke intercept	-	-	-	-	−0.008	0.055	−0.138	0.89
High school	0.295	0.159	1.852	0.064	0.226	0.152	1.488	0.137
White	0.164	0.16	1.023	0.306	0.379	0.151	2.514	0.012

presents the results of the associated-processes LGCM analysis. The model fit the data fairly well: χ²_(df=15) = 184.613, p < 0.001; CFI = 0.990; RMSEA = 0.067, 90% CI = 0.059, 0.076; and SRMR = 0.043. These results indicate acceptable fit, with the exception of the significant chi-square, suggesting some model misfit that is amplified by the larger sample size [43]. The RMSEA was above the 0.06 cutoff for good fit, suggesting an acceptable but not necessarily good fit. Regarding model effects, baseline psychological distress had a significant and positive effect on smoking trend (b = 0.037, z = 6.777, p < 0.001). The trend-to-trend path from psychological distress to smoking distress, however, was not significant (p = 0.259).

3.4. AUC

Table 4. AUC results.

	Original Data (N = 1095)				Multiple Imputation (n = 2511)
Predictor	b	SE	t-Value	p-Value	b	SE	t-Value	p-Value
Intercept	0.335	0.053	6.314	p < 0.001	0.498	0.085	5.845	0.000
High school	0.451	0.050	8.989	p < 0.001	0.512	0.069	7.392	0.000
White	0.317	0.050	6.358	p < 0.001	0.286	0.066	4.309	0.000
AUC distress	0.033	0.003	11.102	p < 0.001	0.033	0.005	7.189	0.000

presents the results of the AUC multiple regression analysis, for the original sample based on listwise deletion of missing data (n = 1095) and the multiple imputation sample (n = 2511). Figure 1 presents the standardized residual plots for the initial sample (panel a) and each of the five imputations (panels b–f) from the multiple imputation runs. Figure 2 presents the residual plots of the standardized residuals for the initial sample (panel a) and each of the five imputations (panels b–f) from the multiple imputation runs. Looking at the regression assumptions, the standardized residuals peak at lower values, with higher values approaching normality, suggesting most participants have lower levels of smoking (Figure 1), and there is apparent homoscedasticity when looking at the variance of the residuals (Figure 2). Given that, as sample size increases, there is convergence in the distribution of standardized variables to the standard normal distribution (central limit theorem) [44], possible violations of the normality assumption are unlikely. Further, there was no difference in the pattern of results between the two samples (Table 4). However, we will look at the multiple imputation sample, as its size is comparable to the sample sizes for the GEE and the LGCM analyses. Psychological distress had a significant and positive effect on smoking (b = 0.033, t = 7.189, p < 0.0001).

3.5. Comparing the Three Methods

In all three methods, psychological distress had a significant effect on smoking. The only difference was in how the effects are partitioned in LGCM. With LGCM, we were able to explore the effects of baseline psychological distress on smoking trend and the effect of psychological distress trend on smoking trend. Because of this partitioning, we saw that only the effect of baseline distress on the smoking trend was significant. As such, initial levels of psychological distress have the greatest impact on progression to higher levels of smoking, with no further impact from change in distress with time. The two covariates race and education in the LGCM, being White versus non-White and having a high school education or less versus a greater education, were associated with higher levels of smoking at baseline but had the inverse effect on the rate of change from baseline (i.e., a negative effect on smoking trend). This indicates a possible ceiling effect for these variables, meaning that their effect was strong initially, leaving little room for additional growth.

In contrast to the results for the LGCM, neither race nor education had a significant effect on smoking in the GEE analysis. The AUC results were more like those seen with LGCM, with both covariates having significant and positive effects on smoking.

4. Discussion

The aim of the present study was to compare three multivariate methods for analyzing repeated measures ordinal data: GEE, LGCM, and AUC. The results of all three methods agreed on the positive effects of psychological distress on smoking, with LGCM being able to partition the effects between baseline level and trend. This is a key difference among the methods and suggests that researchers interested in understanding how different facets of growth in a longitudinal process, whether initial level or rate of change, affect each other may be best served by LGCM over the other two methods. For researchers interested in overall effects, however, it seems that AUC provides a reasonable choice, as it incorporates the entirety of change within a single variable, whether a predictor or an outcome variable. Nevertheless, the handling of missing data in AUC and GEE can be problematic, especially if one is not well versed in handling missing data using methods such as multiple imputation.

Even though the aim of this study was to compare performance among the three different statistical methods with non-simulated data, the finding that psychological distress was associated with an increase in the propensity to smoke with all three methods is noteworthy. A potential reason for this robust association is that psychological distress may lead some to smoke to self-medicate for symptoms of mental illness. Indeed, psychological distress increased the risk of nicotine use, and the effect persisted across time. This is seen with the significant effect of baseline distress on the LGCM trend, along with a non-significant effect from the psychological distress trend to the smoking trend. This result is mirrored by the effect of psychological distress on smoking in both GEE and AUC, two methods that essentially aggregate change into a single variable (long-format data structure in GEE, calculation in AUC).

Another interesting contrast that mirrors the effects of psychological distress is seen when comparing the effects of education and race on smoking in LGCM and AUC. Both results mirror what is seen with distress, as there appears to be a ceiling effect for race and education in LGCM and a clear positive effect for the two covariates with AUC. This highlights the greater ability to partition effects with LGCM. It is notable that neither variable had a significant effect on smoking using GEE, neither on its own nor as an interaction term with time. More research is needed comparing these different methods to better understand the nature of such differing effects.

Like all studies, there are several limitations to our work. First, our smoking variable was generated based on the available questions in the PSID TA dataset. The questions related to vaping were not as well-defined as those for combustible cigarette smoking, at times failing to delineate past from current use. Creating a similar variable with different questions in another dataset may result in a different pattern of results than found here. Second, the low prevalence of smoking in this sample may have affected the results. As such, the comparisons made here should be made with different samples with a higher prevalence of smoking. This would allow researchers to see if the findings in the analyses here are replicable or merely a product of the sample. Third, we only controlled for two variables, education and race. However, as the aim of this study was to compare the three methods, we did not find it necessary to go beyond two control variables. Researchers interested in better understanding the relation between psychological distress and smoking, however, should include additional control variables. These limitations noted, this study adds to recent studies that suggest that AUC may be a viable alternative method for assessing longitudinal data, especially when the research aim is to understand overall effects rather than partitioning effects by baseline level and trend. Nevertheless, our findings suggest that all three methods are useful to researchers interested in assessing change, with AUC being more accessible because of its simplicity. In our opinion, the next steps are to develop a program to calculate AUC regardless of the number of repeated measures, develop an equation that incorporates baseline to assess change without allowing for negative values, and to add the ability to account for missing data through multiple imputation without the researcher having to write additional code.