Sunday, June 7, 2020

Latent growth mirrors linear mixed and multilevel models

[slides, data and more SAS details at https://bit.ly/LGM_Mplus_StataThis post deals with analyzing many-time-points data, intensive longitudinal (ILD, some use ecological momentary assessment, EMA (Shiffman, Stone, & Hufford, 2008), or experience sampling method (Larson & Csikszentmihalyi, 1983) an example to the right, 28 waves; note: when analyzing 1 case only, these are labeled time series).

 I extend the the excellent crosswalk between 2 less intuitive statistical models of changes for many time points of data posted by the UCLA folks: the Linear mixed Model (LMM) and the MultiLevel Model (MLM). I add here a 3rd, which I didn’t see until trying to replicate the 2 they linked: the Latent Growth Model (LGM, in structural equation modeling SEM = latent variables tradition). What I add here then are: SAS+Stata+Mplus code to run the LGM that replicates the LMM and the MLM analyses and then: set them side-by-side for direct matching that can allow for more informed interpretation. I also add some help with interpretation of results, heavily drawn from ch. 4 from the intensive longitudinal analysis (ILA) ‘bible’ by (Bolger & Laurenceau, 2013).
Several points for ease of navigation: (1). Follow the subscripts! I am using generously subscripts, and insist on keeping a dot in there when that variability dimension has been muted, either because we averaged across it, or because that variable or parameter does not vary across it; (2). In addition to using the original Greek letters in setting up the mathematical models, I then revert to their simpler labels, as the UCLA folks did too: G00 for e.g. instead of γ00; (3). I use the same data, publicly available online (in *.dat format, ready for Mplus, Stata too: this means is does not have any variable labels or codes in it, it’s raw numbers data); (4). I add the SAS LMM MIXED and LGM CALIS syntaxes, because SAS is often omitted in such head-to-head comparisons; e.g. in a ‘growth bible’ book  (Grimm, Ram, & Estabrook, 2017) (with lots of online codes) SAS (nlmixed) and R codes are used for LMM models, then Mplus and R (but not SAS anymore) for LGM ones.

Let’s dive in:
(A). The biggest barrier in easily analyzing data with many time points (>5, e.g.) with these models is the naming of things, and then the Greek letters, and then (non)use of subscripts.

(B). To interpret consistently across models, and obtain the same results, one needs to use a time setting that is common in LGM, but not otherwise: first time point is 0, last one 1, and each wave is set to the % of time it passed from 0, for 4 waves this comes out as 0, .333, .666, 1 (see ch. 4 in (Bolger & Laurenceau, 2013)).
The biggest challenge in walking across the 3 models is the language/labeling/meanings. My insights are captured in the table below; also, Mplus/LGM/SEM talks about the ‘random coefficient for the time slope’ in terms of a ‘latent slope varying across cases’ instead, although equation-wise we end up in the same spot.
Symbol
General meaning
LGM parameter meaning
                                   MLM parameter meaning
LMM parameter meaning

LEVEL 1



S^2
Y residual variance
σi2(Yit) Y0-YT S^2 Residual variances (set @=)
Residual Y variance
Residual Y variance

LEVEL 1 & 2

Group by time
Group by time
G01
Ai.  effect on Y intercept
Ai.  ->  ‘LGM Y Intercept’ beta
L2 Ai. ->Y effect
Ai.  -> Y effect
G11
Ai.  effect on time slope
Ai.  ->  ‘LGM Y Slope’ beta
L2 Ai. ->Slopes effect
Ai.  * Time interaction effect

LEVEL 2



G00
Intercept (Y level at time 0)
αI.. Intercept/mean of Y
L2 Y Intercept
Y Intercept (when all X’s are 0)
G10
Time ‘effect’
αS.. Intercept/mean of LGM slope
L2 Intercept of Y slopes
Time effect = time slope
Tau11
Variance of time slope
σ2(uIi.) Residual variance of LGM Y Slope
L2 Slopes residual variance
Time slopes variance
Tau00
Variance of Y intercept
σ2(uSi.) Residual variance of LGM Y Intercept
L2 Slopes residual variance
Y Intercepts variance
Tau01
Covariance time slope* Y intercept
σ(uIi.,uSi.) Covariance LGM Y Intercept*LGM Y Slope
L2 slopes*intercepts covariance
Time slopes with Y intercepts covariance
G20
Effect of X on Y
Contemporaneous effects set equal Xi= ->  Yi=
Within level Xit ->  Yit effect
Xit ->  Yit effect
Formally:
1. LMM and MLM setup is:
Level-1 Model
     Y = B0 + B1*(TIME) + B2*(X) + E
Level-2 Model
     B0 = G00 + G01*(A) + U0
     B1 = G10 + G11*(A) + U1   
B2 = G20                       
Which can be ‘combined’ into 1 equation by bringing B0, B1, and B2 from level-1 into the level-1 equation (which is what makes the LMM harder to intuitively decipher!)”:
Y = G00 + G20*(X) + G01*A + G10*TIME + G11*A*TIME + [U0 + U1*TIME + e]
+ with some additional parameters that appear now at level-2:
Tau00 = sigma^2(U0,U0); Tau11 = sigma^2(U1,U1); Tau01 = sigma^2(U0,U1)
or with Greek and subscripts:
     yit = β0i. + β1i.∙Time.t + β2i.∙Xit + 1∙εit
Level-2 Model
     β0i. = γ00 + γ01*Ai. + u0i.
     β1i. = γ10 + γ11*Ai. + u1i.  
β2i. = γ20
τ00 = σ2(u0i.)i  ; τ11 = σ2(u1i.)i  ; τ01 = σ(u0i., u1i.)i
(I add a ‘redundant’ last subscript outside () to indicate variability dimension that gives rise to the co/variance)

2. The LGM setup is:
Level-1 Model
Y = 1*B0 + B1*(TIME.) + E (time here are fixed ‘loading’ numbers; can also be freed)
or with Greek and subscripts:
yit = 1∙ηYIntercept,i. + ηYSlope,i λSlope,.t + βYX,..Xit   +  1∙εYit            [βYX.. forced equal across time and persons!]
Level-2 Model
B0 = G00 + G01*(A) + U0
     B1 = G10 + G11*(A) + U1
B2 = G20
or with Greek and subscripts:
ηYIntercept, i. = αIntY,.. + βAInt,..Ai.  + uYIi.                                    ~ B0 = G00 + G01*(A) + U0
ηYLSlope, i.    = αSloY,.. + βASlo,..Ai.  + uYSi.                                   ~ B1 = G10 + G11*(A) + U1
βYX,.. = βYX,..                                                                          ~ B2 = G20
The major insights gained are (this comes through in the table of meanings above too):
(1). G00 or γ00 is LGM’s  αIntY,..
(2). G10 or γ10 is LGM’s αSloY,..
(3). G01 or γ01 is LGM’s βAInt,..
(4). G11 or γ11 is LGM’s βASlo,..
(5) G20 or γ20 is LGM’s βYX,..  
(6) Tau00 or τ00 is LGM’s σ2YIntercept, i.)i 
(7) Tau11 or τ11 is LGM’s σ2YLSlope, i.)i  
(8) Tau01 or τ01 is LGM’s σYIntercept, i. YLSlope, i.)i
- I use A as a time-fixed (non-varying) predictor, A.t is sometimes used for ‘treatment’ (T would not work here, because with t time, T is maximum waves…) and Xit for a time-varying predictor (I switched the UCLA notation, sorry). Note that the gamma, beta, lambda, eta Greek labels don’t make our job of understanding easier, I would want to have a simpler labeling, common across all traditions; for now I think the UCLA G00 etc. that comes out of LMM (and MLM) is good enough to show commonalities between models: this means I will drop the LGM specific labels here, to make it clear what LGM estimates that matches the LMM/MLM approaches.
- Note that the ‘intercept/slope language becomes quickly a problem, unless we complete the phrase’: whose intercept/slope… much like a regression coefficient β for that matter: βYX, two more pointers, from what to what: e.g. βYTime,1i.is the time slope of Y, the outcome (Y and ‘Time’ can be dropped often), and because it becomes itself a variable at level-2, it will have its own intercept estimated G10 or γ10, and its own slope (if a predictor of it is modeled)… G11 or γ11, hence G10/γ10 is the ‘level-2 intercept or conditional mean, of the Y time slope’, while G11/γ11 is the ‘level-2 slope of βYTime’ or ‘the effect of the time-invariant predictor A on the Y time slope’: yes, 2 mouthfulls!

(C). The SAS Calis option is pretty flexible, see e.g. URL ,and it has several flavors, of which we showcase 2: the lineqs option, which is an equation-like coding (like y1 = 1*yintLGM + Lambda * ysloLGM  + e1), and the path option (like y <--- yintLGM + Lambda * ysloLGM)

(D). LGMs differ in Mplus from the Stata and the SAS ones; among the reasons may be … “Mplus does not consider a growth model to be a two-level model as in multilevel modeling but a single-level model. With longitudinal data, the number of levels in Mplus is one less than the number of levels in conventional multilevel modeling.” (Mplus Guide, p. 113; needs further investigation). Not sure; even a simple unconditional LGM differs (see the detailed outputs at https://bit.ly/LMM_MLM_LGM  )



(E). SAS has a ‘proc’ that is underutilized: CALIS. It of course can run LGMs (as well as LCSs). E.g.: URL1; URL2; URL3.
Mplus has a unique feature that other software needs a different procedure for: it can rename the variables you have and use in an analysis, on the spot, without changing the data itself. Meaning, one can recycle the same analysis over and over, which comes in handy for long code (like LGM, or Latent Change Score LCS models).
Q&A
You might have some questions, I suspect (so I label them ‘predicted Q…):
pQ&A 1. What if we add auto-regressive (AR) paths in LGM? Go ahead, go wild! You will then run a Autoregressive latent trajectory (ALT) (Bollen & Curran, 2004) or LV-ALT (Bianconcini & Bollen, 2018).
pQ&A 2. Can we add nonlinear slopes in this LGM? Yes, of course, or even let the trajectory be ‘free’, i.e. follow the data (go up and down as in the data). Just free the Lambda loadings of the latent slope (except the first @0 and the last @1)
pQ&A 3. Is this ‘new’? Certainly not, many before pointed to this possibility, but I have not seen a ‘how to’ and side-by-side crosswalk like this anywhere. Excellent detailed appendix (with SAS, Mplus and Mx codes) accompanies (Mehta & Neale, 2005); also see the appendix for (McNeish & Matta, 2018)
pQ&A 4. Is it really ‘this simple’? Well… the setup for LGM I was lucky to get right before figuring out the equations, there are several settings that one can miss or mess up easily: I got the ‘all Y intercepts set @0’ insight from the online codes accompanying Kevin Grimm, Nilam and Ryne’s  growth modeling ‘bible’ (Grimm et al., 2017) .
pQ&A 5. Can this be made even simpler? Likely yes, using the graphical instead of equational display/tool: e.g. for the LGM:


pQ&A 6. Is there more to this? Certainly, Tihomir Asparouhov told me first this is not so such a surprising find (but his brain resonates at other wavelengths than normal ones), it's just a lond to wide change... and in fact the Mplus team has recently merged the classical ‘very many time points’ (econometric time series) tradition with the latent variable (SEM) one recently, under the comfortably large and colorful umbrella of DSEM. Much more possible here to explore.
pQ&A 7. (I am asking this, sorry) Isn’t this a good candidate for a ‘teacher’s corner’ paper for a journal like SEM ?It sure looks like to me (thanks for asking!), however, a bad experience that got me caught in some ideological disputes that the editors seem to be part of (regarding the ‘existence’ of potential outcomes in SEM) made me promise that I won’t knock on that door ever again: so I prefer to lay out things in the open, both for critiques, and for free open use and enjoyment.

Some limitations/invitations for more:
(i). I tried for many hours to replicate the simple LMM models from Stata in SAS: there are many little quirks that made this pretty hard (and got the dreaded “WARNING: Stopped because of infinite likelihood” and another like it in response too: if someone knows the easy way to do this, email me; UCLA has a page, and URL2, and URL3, etc.); eventually Dan McNeish’s code did it for me!
(ii). The SAS MLM setup is also a next ‘must’: so that SAS users can see the link in their own preferred output format (the logic and interpretation should not change): if someone knows a quick way to do this, email me.
(iii). Can LCS replicate these, and then go wilder beyond? Most likely, that’s for another post.

References
Bianconcini, S., & Bollen, K. A. (2018). The Latent Variable-Autoregressive Latent Trajectory Model: A General Framework for Longitudinal Data Analysis. Structural Equation Modeling: A Multidisciplinary Journal, 25(5), 791-808.
Bolger, N., & Laurenceau, J. (2013). Intensive longitudinal methods: an introduction to diary and experience sampling research. New York: Guilford Press.
Bollen, K. A., & Curran, P. J. (2004). Autoregressive latent trajectory (ALT) models a synthesis of two traditions. Sociological Methods & Research, 32(3), 336-383.
Grimm, K. J., Ram, N., & Estabrook, R. (2017). Growth modeling: Structural equation and multilevel modeling approaches: Guilford Publications.
Larson, R., & Csikszentmihalyi, M. (1983). The experience sampling method. New Directions for Methodology of Social & Behavioral Science.
McNeish, D., & Matta, T. (2018). Differentiating between mixed-effects and latent-curve approaches to growth modeling. Behavior Research Methods, 50(4), 1398-1414.
Mehta, P. D., & Neale, M. C. (2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10, 259-284.

Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annu. Rev. Clin. Psychol., 4, 1-32.
Summary of outputs for illustration of identical results and how to read/what:

LMM
LMM (Stata) results:
. xtmixed y x1 a time timeBYx1  || id: time, cov(un) var mle
           y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
G01       x1 |    .611028    .062607     9.76   0.000     .4883205    .7337355
G20        a |   .2978672   .0215842    13.80   0.000      .255563    .3401715
G10     time |   3.217683   .0972144    33.10   0.000     3.027147     3.40822
G11 timeBYx1 |   .8891406   .0974282     9.13   0.000     .6981847    1.080096
G00    _cons |   .6647655   .0624683    10.64   0.000     .5423298    .7872011
------------------------------------------------------------------------------
  Random-effects Parameters  |   Estimate   Std. Err.     [95% Conf. Interval]
-----------------------------+------------------------------------------------
id: Unstructured             |
Tau11              var(time) |   3.724995   .3006731      3.179938    4.363477
Tau00             var(_cons) |   1.561595   .1239062      1.336685    1.824349
Tau01        cov(time,_cons) |   1.111405   .1396993      .8375996    1.385211
-----------------------------+------------------------------------------------
S^2            var(Residual) |   .5415599   .0242323      .4960884    .5911993
------------------------------------------------------------------------------
LR test vs. linear model: chi2(3) = 2209.97               Prob > chi2 = 0.0000            
MLM Mplus
                    Estimate       S.E.  Est./S.E.    P-Value
Within Level
 Y          ON
G20    A               0.298      0.022     13.352      0.000
 Residual Variances
S^2    Y               0.542      0.024     22.149      0.000
Between Level
 S          ON
G11    X1              0.889      0.101      8.836      0.000
 Y          ON
G01    X1              0.611      0.063      9.739      0.000
 Y        WITH
Tau01    S             1.112      0.145      7.685      0.000
 Intercepts
G00    Y               0.665      0.062     10.683      0.000
G10    S               3.218      0.097     33.025      0.000
 Residual Variances
Tau00    Y             1.562      0.118     13.202      0.000
Tau11    S             3.725      0.284     13.121      0.000                   

LGM <plus
YINTLGM  ON
G01    X1              0.611      0.063      9.760      0.000
 YSLOLGM  ON
G11    X1              0.889      0.097      9.126      0.000
 YY0-3      ON
G20    AX0-3           0.298      0.022     13.772      0.000
 YSLOLGM  WITH
Tau01    YINTLGM       1.111      0.140      7.956      0.000
 Intercepts
        YY0-3          0.000      0.000    999.000    999.000
G00    YINTLGM         0.665      0.062     10.642      0.000
G11    YSLOLGM         3.218      0.097     33.099      0.000
 Residual Variances
S^2    YY0-3           0.542      0.024     22.349      0.000
Tau00    YINTLGM       1.562      0.124     12.603      0.000
Tau11    YSLOLGM       3.725      0.301     12.389      0.000


Summary input/syntax/software codes for replications (but ZIP with full list of: data, codes, outputs are posted here https://bit.ly/LMM_MLM_LGM  )
1. SAS
1.1. SAS LMM
*     1. Read data first LONG/Vertical
proc import out= ex6long datafile = "P:\ex6.10L.dta";
run;
proc contents data=ex6long;
run;


/* NOT completely sure about this MIXED model */
proc mixed data= ex6long covtest method=reml; *Method=REML uses restricted ML;
model y = time | x1 / s ddfm=kr;*ddfm = KR uses Kenward-Roger correction;
random int time / subject=id type=vc; *Uncorrelated random effects for the intercept and slope; solution adds MANY estimates: what are they?
repeated / type=vc subject=id; *homogeneous diagonal error structure;
run;

1.2. SAS LGM
*     1. Read data first WIDE/Horizontal
proc import out= ex6wide datafile = "P:\ex6.10W.dta";
run;
proc contents data=ex6wide;
run;

title "Linear Latent Growth Model LGM Model replicating the Unconditional Linear Mixed Model LMM and the MultiLevel Model MLM";
proc calis  data= ex6wide method=ml pall;
lineqs
y1 = 0. * Intercept + 1*F_yintLGM + 0 * F_ysloLGM          + e1,
y2 = 0. * Intercept + 1*F_yintLGM + 0.3333333 * F_ysloLGM + e2,
y3 = 0. * Intercept + 1*F_yintLGM + 0.6666666 * F_ysloLGM + e3,
y4 = 0. * Intercept + 1*F_yintLGM + 1 * F_ysloLGM          + e4;
variance
F_yintLGM F_ysloLGM,
e1-e4;
mean
F_yintLGM F_ysloLGM;
cov
F_yintLGM F_ysloLGM;
run;

* OR run like: 

title " LGM replicating the LMM and the MLM";
proc calis  data= ex6wide method=ml PSHORT ;
path
y1 <--- F_yintLGM = 1, /* constrained = 1 */
y1 <--- F_ ysloLGM = 0, /* constrained = 0 */
y2 <--- F_yintLGM = 1, /* constrained = 1 */
y2 <--- F_ ysloLGM = 0.3333333, /* constrained = 0.33333334*/
y3 <--- F_yintLGM = 1, /* constrained = 1 */
y3 <--- F_ ysloLGM = 0.6666666, /* constrained = 0.66666669 */
y4 <--- F_yintLGM = 1, /* constrained = 1 */
y4 <--- F_ ysloLGM = 1; /* constrained = 1 */
pvar
y1-y4 = reserr,  /* variancee estimated but equal*/
F_yintLGM = varint,
F_ysloLGM = varSlo;
pcov
F_yintLGM F_ ysloLGM ;
mean
y1 = 0, /* intercept of the 2nd constrained = 0 */
y2 = 0, /* intercept of the 2nd constrained = 0 */
y3 = 0, /* intercept of the 2nd constrained = 0 */
y4 = 0; /* intercept of the 2nd constrained = 0 */
run;

title "Conditional x1 LGM replicating the LMM and the MLM with time varying predictor too a1-4";
proc calis  data= ex6wide method=ml PSHORT ;
path
y1 <--- F_yintLGM = 1, /* constrained = 1 */
y1 <--- F_ysloLGM = 0, /* constrained = 0 */
y1 <--- a1 = bx,
y2 <--- F_yintLGM = 1, /* constrained = 1 */
y2 <--- F_ysloLGM = 0.3333333, /* constrained = 0.33333334*/
y2 <--- a2 = bx,
y3 <--- F_yintLGM = 1, /* constrained = 1 */
y3 <--- F_ysloLGM = 0.6666666, /* constrained = 0.66666669 */
y3 <--- a3 = bx,
y4 <--- F_yintLGM = 1, /* constrained = 1 */
y4 <--- F_ysloLGM = 1, /* constrained = 1 */
y4 <--- a4 = bx,
F_yintLGM <--- x1,
F_ysloLGM <--- x1;
pvar
y1= reserr,  /* variance estimated but equal*/
y2= reserr,  /* variance estimated but equal*/
y3= reserr,  /* variance estimated but equal*/

y4= reserr,  /* variance estimated but equal*/
F_yintLGM = varint,
F_ysloLGM = varSlo;
pcov
F_yintLGM F_ ysloLGM ;
mean
y1 = 0, /* intercept of the 2nd constrained = 0 */
y2 = 0, /* intercept of the 2nd constrained = 0 */
y3 = 0, /* intercept of the 2nd constrained = 0 */
y4 = 0, /* intercept of the 2nd constrained = 0 */
F_yintLGM  = mnInt, /* declare them to be estimated*/

F_ysloLGM = mnSlo; /* declare them to be estimated*/
run;
SELECTION of output to show identical results:

Absolute Index
Fit Function
0.0724

Chi-Square
36.1498

Chi-Square DF
25

Pr > Chi-Square
0.0694
Path
Param.
Est.
SE

t Value
p
y1-4
<===
a4
G20
0.298
0.022
13.787
<.0001
F_yintLGM
<===
x1
G01
0.611
0.063
9.750
<.0001
F_ysloLGM
<===
x1
G11
0.889
0.098
9.117
<.0001
Variance Param.s
Variance
Variable
Param.
Est.
SE
t Value
p
Error
y1-4
S^2   
0.543
0.024
22.338
<.0001

F_yintLGM
Tau00
1.565
0.124
12.591
<.0001

F_ysloLGM
Tau11
3.733
0.301
12.387
<.0001
Means and Intercepts
Type
Variable
Param.
Est.
SE
t Value
p

F_yintLGM
G00 
0.665
0.063
10.620
<.0001

F_ysloLGM
G10 
3.218
0.097
33.033
<.0001
Covariances Among Errors
Error of
Param.
Est.
SE
t Value
p
F_ysloLGM
_Tau01
1.114
0.140
7.9518
<.0001
2. Stata
2.1. Stata LMM
xtmixed y x1 a time timeBYx1  || id: time, cov(un) var mle
(yes, that’s all)

2.2. Stata LGM

* one would need to switch the X and A to mirror the equations above! The original data had X as time-fixed and A as time invariant: bad choices
* Unconditional Mplus eg ex6.10W
sem (y1 <- IntY@1 SloY@0              _cons@0) ///
(y2 <- IntY@1 SloY@.33333334  _cons@0) ///
(y3 <- IntY@1 SloY@.66666669  _cons@0) ///
(y4 <- IntY@1 SloY@1                  _cons@0) , latent(IntY SloY) ///
var(e.y1@var e.y2@var e.y3@var e.y4@var ) /// /*set = means (IntY SloY ) */
cov(IntY*SloY) means (IntY SloY ) method(mlmv)

* Time-fixed & time-varying predictor Mplus eg ex6.10W 
* Beware: if not declaring _cons in Intercept & Slope line, df's will be larger by 2, and estimates off
 sem (y1 <- a1@beta IntY@1 SloY@0            _cons@0) ///
(y2 <- a2@beta IntY@1 SloY@.3333333  _cons@0) ///
(y3 <- a3@beta IntY@1 SloY@.6666666  _cons@0) ///
(y4 <- a4@beta IntY@1 SloY@1                 _cons@0) ///
(IntY <- x1 _cons) (SloY <- x1 _cons), latent(IntY SloY) ///
var(e.y1@var e.y2@var e.y3@var e.y4@var ) /// /*set = means (IntY SloY ) */
cov(e.IntY*e.SloY) method(mlmv)
estat gof

3. Mplus
3.1. Mplus MLM

3.2. Mplus LGM
! one would need to switch the X and A to mirror the equations above! The original data had X as time-fixed and A as time invariant: bad choices
! time-varying outcome part
        yintLGM BY y0-y3@1;
        yintLGM;
        [yintLGM];
        ysloLGM BY y0@0
                  y1@.3333333
                  y2@.6666666
                  y3@1;
        yintLGM on x1  ;
        ysloLGM on x1  ;
        [ysloLGM];
        intLGM WITH sloLGM; ! because sloLGM ERROR variance was>0 and is so small
        yy0-yy3 (v_u);! set equal across time
        [yy0-yy3@0]; ! set all intercepts to 0 as in Grimm
   ! make it like LMM&MLM:
  y0 on a0 (bx);
  y1 on a1 (bx);
  y2 on a2 (bx);
  y3 on a3 (bx);
    OUTPUT: SAMPSTAT standardized tech4 tech1;

No comments:

Post a Comment