Simulation-based linear mixed effect regression models with stan

In this blog, I will provide a step-by-step instruction of how we can generate data from a mixed effect model and recover the parameters of the model with the simulated dataset. This simulation-based experiment can help us better understand the structure and generative process of the multilevel model with correlated random intercepts and slopes. To proceed, I will first illustrate the general form of mixed effect models, and generate data based on a given set of design matrices and parameters (X, ß, Z, b). In the end, I will build a Bayesian model to estimate the parameters on the simulated data via stan.

1. Matrix form of the mixed effect regression model

The general form of a mixed effect model can be illustrated in the following way:

where y is a vector of dependent variables, and the two design matrices, X and Z, are related to fixed effects and random effects, respectively. The parameters of fixed effects are captured by ß, and the random effects (e.g., random intercepts and slopes) are specified in b, which is generated from a Multivariate Normal (MN) distribution with mean 0 and variance-covariance matrix ∑. The random errors are indicated by epsilon.

To be more specific, I will expand the matrix form and provide a detailed description of the process for generating the fixed effects, random effects and response. As an example, I will assume a continuous predictor x and correlated random intercepts and slopes. This can…