Live Webinar Workshop

Analyzing Repeated Measures Data: GLM and Mixed Model Approaches

There are three main approaches to analyzing repeated measures data--the multivariate GLM approach, the marginal model approach, and the mixed model approach.

You've probably heard of at least two of these, and actually used one or two. Maybe all three.

Each one has advantages and disadvantages. Each one has certain repeated measures designs that it works well for, and certain designs it does not.

The trick is figuring out which approach is best for your design and your research questions.

The GLM Approach is easy to run, but the output is not easy to understand. And it's limited to certain designs.

Most people (me included) were trained on the GLM approach, especially if you use SPSS. SPSS has a GLM Repeated Measures procedure that is pretty easy to run.

But it has severe limitations and pretty strong assumptions that are hard to meet.

• If even one observation is missing, you lose the whole subject's data.
• You can't run post-hoc tests on within-subjects factors (which is usually the whole point!).
• You must have the same number of repeated observations for all subjects. Or you have to throw away data again.
• And you have to assume that all observations within a subject have equal variance and equal correlations (that's sphericity, if you've ever wondered). It works sometimes, but often not.

You may have realized some of these limitations and tried to learn mixed models on your own.

But that's hard--really hard.

The math gets tricky, and few books spell it out for you. And every book on mixed models I've ever seen puts one chapter in the back about repeated measures data. You need to master the material before you get there or it doesn't make sense. But all of that material uses examples that don't make sense, given your design.

My goal in this workshop is that you will be able, on your own, to run repeated measures, marginal, and mixed models for repeated measures data using SPSS.

And have a solid basis for which approach to choose.

You'll learn the difference between the repeated and the random statement. You'll know what to check for, the steps to take, and how to set up your data. You will finally understand what a covariance matrix is for, and how to choose which one to use. You'll know the difference between a fixed and a random factor and how to choose.

And when you do need to look something up in a book on Mixed Modeling, you'll understand it.

In the Workshop, you will learn:

Module 1: An Overview of Repeated Measures Data

One of the things I've found troublesome over the years is that when people say repeated measures, they often mean very different things. There are actually a lot of different designs that can be considered repeated measures. There are a few I've seen people attempt to analyze as repeated measures ANOVAs over the past few years:

• Experiments with one or two within-subjects factors, with a possible between subjects design
• Pre-post designs
• Longitudinal studies with pre-, post-, and followup measures taken months apart, with some covariates measured only at baseline and some measured at each measurement
• Studies that measured patients five times over two years on an outcome variable only, where only 10% of participants had all five measurements
• Experimental studies where repeated trials were pure replicates

Consequently, we're going to start by taking away the names which may or may not communicate what we need and examine the structure of the variables and the research questions to think about the real issues:

• What is the role of time?
• Should time be considered continuous or categorical?
• What is the role of baseline measurements?
• Are correlations among a subjects measurements something of interest or a nuisance?
• Is time confounded with any predictors? Is Subject?

We'll then go through an overview of the three main approaches so you have the big picture before we delve deeply into each one.

Module 2: The Multivariate Repeated Measures Compound Symmetry Model

This is the "usual" method for running repeated measures data, and probably the one with which you're most familiar. It does work reasonably well in some situations, and although it doesn't look like it, there are many similarities between this model and the mixed model.

We'll discuss the similarities, the differences, and the assumptions this model has that we are able to relax in mixed models.

We'll go through examples of running a model with this method (using SPSS's GLM Repeated Measures procedure) and interpreting the output so that you have a new understanding of the model you've been running for years and its assumptions.

We will then be able to compare two new approaches to this one, so you see where they differ and where they're really the same.

Module 3: Transitioning to the Mixed Approach

There are a lot of advantages to running repeated measures data using mixed models. But they do require you set up your data differently (in the long, or observation-level format, as opposed to the wide, or person-level format you're used to).

In this module I'll give a tutorial on how to restructure the data set from the Wide Format required by GLM Repeated Measures (with one row per subject) to the Long Format, (with one row per observation).

I'll also show you how to graph your data, per person, so you can see what is going on. I have alway found this absolutely indispensible in choosing an approach. But you can't just use a regular means plot or scatter plot, because of the clustering of the data, so once again, I'll show you just how to do it.

Both the marginal and the mixed model use Maximum Likelihood Estimation, so we'll do a quick review, then talk at length about measuring model fit.

We'll still use F tests, but our old friend R-squared disappears as a meaningful measure. In it's place comes Deviance, and its Information Criteria buddies.

They're not so bad once you get used to them, and you'll see how necessary they become when you are choosing a model later on. Not just which predictors to include, but whether to include various random effects.

Module 4: The Marginal Model and the Repeated Statement

The Marginal Model is not actually a mixed model--there are no random effects, although it is still run using the Repeated statement in the Mixed procedure.

In this session we will explore the Marginal Model (also known as population averaged models) both conceptually and for Repeated Measures Data.

In many repeated measures situations, the marginal model fits the data better than a Mixed Model. It has many of the same advantages as Mixed Models, and is in many ways more intuitive.

It's a nice bridge because conceptually, it's very similar to the Repeated Measures ANOVA you're used to. It too takes into account the correlation among each subject's observations. But it's more flexible than the Repeated Measures ANOVA because it allows those correlations to be unique (goodbye, Sphericity assumption! Hello, post-hoc tests on within subject factors!).

The result is more choices, and often better model fit.

Module 5: The Mixed Model: Random Intercept Models

If we stopped before this module, you would already have a whole new set of tools that greatly expand the types of analyses you can do, and do well.

But being able to run mixed models will exponentially expand the types of research questions you can answer.

For repeated measures data, there are two types of mixed models you'll use--random intercept models and individual growth curve models.

Random intercept models take care of the correlation among a subject's responses as do marginal models, but they do it in a different way. Instead of measuring correlations, they actually measure how much responses vary across individuals. If one individual generally has high responses, and another always low, the random intercept measures and accounts for that. It literally redefines the residuals.

We can then measure how much of the variation in the outcome is due to the covariates, the treatment conditions, or the subjects themselves. So here you will learn how to use statistics like variance components and Interclass Correlation to understand the variation in responses over time, conditions, or individuals.

This module introduces the linear mixed model in depth before going on to random slope models.

Module 6: The Mixed Model: Individual Growth Curve (Random Slope) Model

Individual growth curve models go a step further--they allow each subject to have a different growth trajectory over time--or over any other covariate that changes over time.

In many, many designs, the random intercept model is enough to take care of similarities in individuals' responses.

But when the real research questions are how factors affect growth (or decline) over time, growth models answer the question directly.

We'll talk about what individual growth curve models are, how to run them, when to use them, and when not to.

We'll also explore issues like improving interpretation through centering, when missing data causes no problems at all, and and how the models differ when time is measured as continuous or categorical.

We'll also introduce covariance structures for random effects.

Module 7: Model Refinements, Extensions, and Assumptions

Now that you've got a good understanding of the model, what it measures, and what the various statistics mean, we're going to talk about ways to improve it.

We'll start by checking whether or not the newly-defined residuals meet assumptions of normality and independence. If they don't, we can account for it by refining the residual covariance structure with a repeated statement. Nice, eh?

Then we'll go over a few common extensions of the model

• three-level models for when individuals are clustered into classrooms or companies or some other grouping
• and Random Curvature models, which allow growth to vary from a linear trajectory

Module 8: Building and Choosing a Model

Okay, now that you've learned how to create and refine linear mixed models, how do you choose?

There are a lot of choices along the way--the type of model, which random effects to include, which covariance structures--before you even get to choosing which predictors to put into the model.

There are two main approaches to take to make all these decisions--you either start with the most complex model and simplify, or start with the simplest and add complexity.

Here we go through examples start to finish to bring together everything you've learned. You will learn the exact steps to take to build the model and choose the best one.

------------------------------------------------------------------------------

But can I keep up?

This starts out at a moderate level, but moves into an advanced level workshop. Not need-a-Phd-in-statistics advanced. It is for researchers, not statisticians.

But you should be pretty statistics savvy and have solid experience running linear models. Centering, least-squares estimation, dummy variables, interactions, residuals, and variance should not be new concepts.

Concepts that are unique to mixed modeling? I'll explain those--information criteria, maximum likelihood, deviance, covariance matrices, variance parameters, sphericity, growth models, random slopes. All those terms that books on mixed models seem to skim over, you'll learn here.

I'm assuming you've never had linear algebra, don't want to see things derived, and you're here because you want to see explained in English.

We'll look at matrices (we have to), but we won't manipulate them--no Kroniker products here.

We will look at model equations (we have to do that, too). To stay consistent with books and articles on the topic, they will include some Greek letters. But I'll walk you through them--I won't assume you automatically know what everything means or why it's important.

You do need to spend some time each week. You will learn concepts and get some clarity if you don't do the exercises and try to rerun my examples on your own. But you won't entirely get it.

This is a workshop where you want to get your hands dirty with some data. Please expect to spend 6-10 hours per module just doing the exercises.

This workshop is for you if you:

• Have always run regressions and now you're facing longitudinal or repeated measures data for the first time
• Have heard you should use Mixed Models for your repeated measures data, and want to learn how
• Have used Mixed Models before, but never quite got the difference between the random and repeated statement, or what those covariance structures really are
• Have tried to learn mixed models on your own, and have found that, well, that's really, really hard to do
• Have always run repeated measures using the SPSS GLM Repeated Measures, but now you're facing a design where it doesn't work--unequal sample sizes per person, time as continuous, or lots of missing data
• Use SPSS. I am focusing on SPSS this time around. The workshop is not all about SPSS--I do plan on expanding it to include SAS the next time around, and you can also run these models in Stata or R (I'm not so sure about JMP). But I have found that for mixed models, even more than other methods, knowing the software inside and out is crucial. The techniques are so flexible, and it's really easy to mis-specify a model without realizing it, and get different results. You are welcome to try it on another software package, but I can't help too much this time around.
• Have the time over the next few months to really invest in learning. It will require about 6-10 hours per week.

It is not for you if you:

• Want to learn Bayesian approaches to Mixed Modeling
• Are new to statistics or to regression and anova
• Have only run regressions with continuous variables and have never run a repeated measures ANOVA
• Already understand linear mixed models and want to learn advanced topics like GLMM
• Want a general linear mixed models course and don't run repeated measures data. You'll still learn a lot of relevant info, but our focus is the specific application to repeated measures data.

Prerequisites:

• You will get the most out of the workshop if you have had at a bare-bones minimum two statistics classes, and one must include a class on linear models--either multiple regression or Analysis of Variance (ANOVA).  Four or more would be better.
• Real experience doing some sort of linear modeling. Familiarity with the GLM procedure will be very beneficial.
• You should be familiar with SPSS Syntax. I will demonstrate some parts using the menus, but once you get to this level of modeling, you need to have control of the details that syntax allows. There are just too many parts that the menus can't do. And honestly, I have found the SPSS Mixed dialog windows to be the most unintuitive, confusing ones I've come across. Even after years of using Mixed in SAS (and knowing what I was doing), I was completely lost. So I switched to syntax for SPSS Mixed and never looked back.

The workshop is offered as a Webinar live online workshop + Membership Site

I will be conducting the workshop live via webinar. You attend online and listen and speak either by phone or over the internet as you see what is happening on my computer screen. Webinars are highly interactive--you can see my power point and ask questions during the session, yet you never have to leave your house or office. 

During the webinar sessions, I will present concepts and explain the meaning of the techniques in that module, demonstrate how to implement those techniques in SPSS on different examples, and answer your questions.

I will be opening the workshop to only 40 people. I want to make sure everyone has a chance to ask questions and get plenty of feedback.

We've also created a workshop site that you become a member of. For a year.

The site is our home-base for the workshop. It's where you'll find everything you need to support your learning of each module:

• SPSS data files. These are real, true, not-textbook-perfect data files from real research projects I've worked on with clients, who have graciously allowed me to share them with you. You get full access to use the data to try everything I demostrate in the workshop and try things on your own.
  
  
• SPSS syntax code to run and explore all of the examples yourself. You won't learn it unless you try it. So I'm giving you my syntax so you can see exactly how I got all the output I'm showing you.
  
  
• PDF handouts of the presentation slides, which will be available ahead of each session, on which you can take notes.
  
  
• Video Screenshot recordings of each workshop session made available within 48 hours. So you can review the material right away, a year later, or while doing the exercises. Or if you need to miss one (or all!) of the live sessions, you can still participate on your own schedule.
  
  
• Exercises.  (Yes, homework!).  You really need to practice this stuff. Get your hands dirty. So we're giving you the data to try it on your own with new models to try out. But don't worry--you won't be on your own stuck on some coding error that won't work. I'm also giving you the syntax I used to do the exercises and the answers.
  
  
• A place to submit written questions between sessions. So as you're reviewing videos afterward (or if you missed a live session), just submit a question. I'll answer it there if I can, or if it's something I need to show you, I'll answer it in the next session.
  
  
• Video Recordings of all Q&A sessions. So if you have to miss one, just submit your questions ahead of time and I'll answer in the Q&A. Just watch it later.

• A list of helpful resources and suggestions for further reading. I'm not requiring a text book. But there are definitely some good books and articles out there on this topic, and you will learn the techniques better with some background reading. I'm selecting the chapters from each book that best explains each topic, and I'll make the list available when you first register, so you have time to interlibrary loan them.
  
  
• Bonus videos. I've included a number of videos from some webinars on relevant topics to help jog your memory or clear up misunderstandings. Included are:
  • Getting Started with SPSS Syntax
    
  • Dummy and Effect Coding
    
  • Running Repeated Measures as a Mixed Model
    
  • What Happened to R-squared
    
  • Random Intercept and Random Slope Models
    

You'll have access for a year to this workshop membership site. So as we add resources, re-record sessions, and answer more questions each time we offer the workshop, you have access to the updated material.

You may also find that you learn a LOT more the second time through. Or third. This stuff is hard, and there is a lot to it. You'll learn more every time.

Comments from past participants in this workshop:

The Details:

The Workshop Webinar Sessions will meet 8 times.  You may have noticed you're going to learn a lot. But we know you have other things going on in your life. In order to balance keeping momentum going and giving you enough time so you don't get behind, we've devised this schedule based on feedback from previous participants.

We’ll meet for two weeks in a row, then we'll take a week off. Before we start again, we'll have one day and one evening Q&A session, so you get any questions answered before we move on.

• Module 1 April 19
• Module 2 April 26
  Break week
• Module 3 May 10
• Module 4 May 17
  Break week
• Module 5 May 31
• Module 6 June 7
  Break week
• Module 7 Jun 21
• Module 8 June 28

We will meet at 12:00 noon Eastern (UTC-4). The workshop sessions are approximately 2 hours long, including time for questions.

We will also meet for eight Q&A sessions throughout this time, at 12:00 noon Eastern (UTC-4) and 7:00pm Eastern (UTC-4), on the following dates. We’ve just added these four evening sessions to accommodate participants with varying schedules and time zones.

• May 8
• May 29
• June 19
• July 10

Full Workshop Price: $497

Sorry, this workshop is full, and registration has closed.

To get first notice when it opens again, please join our Advance Discount List.

Refund Policy:  Your registration fee is fully refundable up to 72 hours in advance minus a $35 administration fee. Because enrollment is limited, no refunds will be granted after the program begins.