Marginal Models Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Those are fine if each person is measured only once. Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Those are fine if each person is measured only once.

Example: Do Southerners lose their accent when they live abroad? Correlation is just fine Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Those are fine if each person is measured only once. Example: Do surfers front the /u/ in dude more than non-surfers? T-test is just fine Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? They assume one measurement per person If there are repeated measures, they may produce invalid results

You want valid results, right? Why do I need to use a different kind of statistics? Whats wrongwith using correlation, ANOVA, t-tests, logistic regression? Stats that handle repeated measures Paired t-test Repeated measures ANOVA Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Stats that handle repeated measures Paired t-test Repeated measures ANOVA

Mixed effects and marginal models handle paired t-test, and repeated measures ANOVA, and regression models, and missing data. Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Linguistic data often gets multiple measurement from a single person Example: Sociolinguistic interview to measure ing > -in Each case of ing or in is viewed in its social and linguistic context Participant Variant

Gender Phone after Addressee Jeff -ing M [b] Jill

Jeff -in M Pause Jill Jeff -in M

[i] Jill Jeff -ing M [l] Bob Sally

-in F Pause Carl Sally -in F [r]

Ross Sally -in F [ej] Ross Why do I need to use a different kind of statistics?

But wait! This is just how its done in sociolinguistics! Why do I need to use a different kind of statistics? But wait! This is just how its done in sociolinguistics! True, but it has been wrong Many warning voices have been raised recently:

Daniel Ezra Johnson, Harald Baayen, etc. Computers couldnt handle mixed effects and marginal models until recently Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? What if analysis shows men use more in forms, but you dont pay attention to repeated measures? Participant Gender

% -in 1 M 53 2 M 22 3

M 34 4 M 30 5 F 29

6 F 31 7 F 33 Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? What if analysis shows men use more in forms, but you dont pay

attention to repeated measures? Maybe its not men in general, but participant 1 skews the results Mixed effects controls for this kind of thing Participant Gender % -in 1 M 53 2

M 22 3 M 34 4 M 30

5 F 29 6 F 31 7 F

33 Why do I need to use a different kind of statistics? Linguistic data correlation, often gets multiple measurement from a single Whats wrong with using ANOVA, t-tests, logistic regression?

person Example: Psycholinguistic study where participants give reaction times to several different verbs, nouns, and adjectives. Gender is an independent variable Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Linguistic data often gets multiple measurement from a single person Example: Psycholinguistic study where participants give reaction times to several different verbs, nouns, and adjectives. Gender is an independent variable One participant had two lattes before test and was really fast

If he only gave one response he wont affect mean male response time much Why do I need to use a different kind of statistics? Whats wrong with using correlation, ANOVA, t-tests, logistic regression? Linguistic data often gets multiple measurement from a single person Example: Psycholinguistic study where participants give reaction times to several different verbs, nouns, and adjectives. Gender is an independent variable One participant had two lattes before test and was really fast If he only gave one response he wont affect mean male response time much If he gives many responses hell skew the mean for males and make

the results inaccurate. Mixed-effect and marginal models control for this kind of problem Repeated Measures Introduce Correlated Residuals ANOVA, correlation, etc. give inaccurate results when there are correlated residuals What the freak are residuals anyway? Repeated Measures Introduce Correlated Residuals

ANOVA, correlation, etc. give inaccurate results when there are correlated residuals What the freak are residuals anyway? Weight of different volumes of water Residual is the distance from a data point to the regression line Repeated Measures Introduce Correlated Residuals Frequency of end up Verbing expressions by decade

E.g. ended up going, ends up working, ending up in China, etc Repeated measures taken from four corpora Residuals are distances from data point to regression line (blue lines) Repeated Measures Introduce Correlated Residuals Correlated residuals

All COHA residuals are positive (above line). They are highly correlated All GoogleUS are negative (below line). They are highly correlated Repeated Measures Introduce Correlated Residuals Correlated residuals The reaction times the latte hyped guy gave in the experiment would be like the GoogleUK data points (if you switch decade for experimental word and frequency for reaction time) How do you Eliminate Correlations Caused by Repeated Measures?

In a mixed-effects model add a RANDOM INTERCEPT Make an individual intercept for every unit that has repeated measures The residuals are now calculated to the units personal line, not the overall line How do you Eliminate Correlations Caused by Repeated Measures?

Individual intercepts are the mean amount the corpus deviates from the overall regression line. The individual intercept is parallel to the overall regression line It is called an intercept because it is measured at the point the line crosses (intercepts) the vertical axis at zero. How do you Eliminate Correlations Caused by Repeated Measures? Look at circles representing GoogleUS. When there is no random intercept, all residuals are negative. They are correlated When there is a random intercept, some are above the line and others

are below. There is no longer a correlation. (Generally, not always) How do you Eliminate Correlations Caused by Repeated Measures? Look at circles representing GoogleUS. When there is no random intercept, all residuals are negative. They are correlated When there is a random intercept, some are above the line and others are below. There is no longer a correlation. (Generally, not always) The residuals are smaller, so the model fits the data better.

Repeated measures in corpus study Random effects modify the residuals by modifying the intercept or slope (mixed-effects) Repeated measures in corpus study Random effects modify the residuals by modifying the intercept or slope (mixed-effects) Repeated effects don't change the residuals, they just estimate them (marginal model) Mixed effects versus marginal models When do you use each? Try a mixed effects model first. Mixed effects versus marginal models

When do you use each? Try a mixed effects model first. If your mixed effects model tells you: Hessian matrix not positive definite Model failed to converge It is too computationally complex for a mixed-effects model Use a marginal model instead Repeated effects in a marginal model Two estimates are made of the residuals First: How spread out they are from the regression line at each point of measurement (variance)

Point of measurement: Point in time, experimental condition How close or far from the line are the residuals? Repeated effects in a marginal model Two estimates are made of the residuals

First: How spread out they are from the regression line at each point of measurement (variance) Point of measurement: Point in time, experimental condition How close or far from the line are the residuals? Second: How do the residuals change at each point of measurement? (covariance) They get larger over time This information is put in the R

matrix Covariance structures There are different estimates of the variance and covariance of the residuals Covariance structures There are different estimates of the variance and covariance of the

residuals They have funky names Covariance structures There are different estimates of the variance and covariance of the residuals They have funky names Do not use them in public or you may not get any dates

Covariance structures What do they mean? Covariance structures What do they mean? They are different kinds of patterns Covariance structures What do you do with them? Covariance structures What do you do with them? Find the one that fits your residuals the best

Covariance structures What do you do with them? Find the one that fits your residuals the best How do you do that? Compare the -2RLLs with a likelihood test Covariance structures What do you do with them?

Find the one that fits your residuals the best SPSS gives you a table like this for every model you run Information Criteriaa -2 Restricted Log Likelihood -24.806 Akaike's Information Criterion (AIC)

-20.806 Hurvich and Tsai's Criterion (AICC) -20.362 Bozdogan's Criterion (CAIC) -16.004 Schwarz's Bayesian Criterion (BIC) -18.004 The information criteria are displayed in smaller-isbetter forms. a. Dependent Variable: CubeRootFreq.

How do you do that? Compare the -2RLLs with a likelihood test Covariance structures What do you do with them? Find the one that fits your residuals the best How do you do that?

SPSS gives you a table like this for every model you run Information Criteriaa -2 Restricted Log Likelihood -24.806 Akaike's Information Criterion (AIC) -20.806

Hurvich and Tsai's Criterion (AICC) -20.362 Bozdogan's Criterion (CAIC) -16.004 Schwarz's Bayesian Criterion (BIC) -18.004 The information criteria are displayed in smaller-isbetter forms. a. Dependent Variable: CubeRootFreq.

Compare the -2RLLs with a likelihood test Compare the results with different covariance structures Lowest -2RLL with fewest parameters wins! Covariance structures Now that repeated measures are accounted for, report the results of the fixed effects

Example: The effect of time (by decade) on the frequency of end up verbing expressions was significant (F (1, 5.173) = 133.517, p < .0005). The fact that repeated measures were taken from the four corpora was accounted for by running a marginal model with a heterogenous autoregressive covariance structure. Running a marginal model in SPSS Using a repeated effect

Open the file endupVERBING.sav Running a marginal model in SPSS Using a repeated effect Put the variable with repeated measures in the subjects box. Running a marginal model in SPSS Using a repeated effect

Put the variable with repeated measures in the subjects box. Specify what the different points of measurement were taken from Running a marginal model in SPSS Using a repeated effect Put the variable with repeated

measures in the subjects box. Specify what the different points of measurement were taken from Choose the covariance structure Running a marginal model in SPSS Using a repeated effect Specify the dependent variable Running a marginal model in SPSS Using a repeated effect

Specify the dependent variable Put categorical independent variables in the factors box Put continuous independent variables in the covariates box Both the independent variables and the repeated effects variables are chosen here Running a marginal model in SPSS Using a repeated effect

Click on Fixed Running a marginal model in SPSS Using a repeated effect After clicking on Fixed, click on DecadeNumeric (or DecadeNUM) Move it to the Model box by

clicking on the Add button > Continue From the variables chosen before (Decade and Corpus), this specifies Decade as the independent variable Running a marginal model in SPSS Using a repeated effect Click on Statistics Choose Parameter estimates and Covariances of residuals > Continue > OK

Running a marginal model in SPSS Using a repeated effect Go to output window Note number of parameters and -2 Restricted Log Likelihood Running a marginal model in SPSS Using a repeated effect Lets compare the heterogenous autoregressive covariance

structure we just ran with a nonheterogenous autoregressive covariance structure Running a marginal model in SPSS Using a repeated effect Lets compare the Autoregressive covariance structure we just ran with a heterogenous autoregressive covariance structure Chose AR(1) in Repeated Covariance Type box >

Continue > OK Running a marginal model in SPSS Using a repeated effect Go to results window Note the number of parameters and -2LL for this model Running a marginal model in SPSS Using a repeated effect Model # parameters

-2LL Hetero AR1 11 -24.806 AR1 4 -31.833 difference 7

7.027 Let's compare using -2LL At difference of 7 parameters, difference in -2LL needs to be 14.067 or greater for there to be a significant difference between models We go with simpler model (with fewer parameters) even though

the -2LL is not the smallest Running a marginal model in SPSS Using a repeated effect Model # parameters -2LL Hetero AR1 11 -24.806 AR1

4 -31.833 difference 7 7.027 The best fit is heterogenous autoregressive covariance structure. Why?

Running a marginal model in SPSS Using a repeated effect The best fit is heterogenous autoregressive covariance structure. Why? Variance (of residuals) isnt constant across time. Running a marginal model in SPSS Using a repeated effect

The best fit is heterogenous autoregressive covariance structure. Why? Variance (of residuals) isnt constant across time. Covariance goes up with time. (More variance in recent decades.) Running a marginal model in SPSS Using a repeated effect

The best fit is heterogenous autoregressive covariance structure. Why? Variance (of residuals) isnt constant across time. Covariance goes up with time. (More variance in recent

decades.) This is what heterogenous autoregressive models. Running a marginal model in SPSS Using a repeated effect Open the file Dabrowska2010.sav Linguists and non-linguists judged sentences for grammatical correctness Scale of 1-5 (1 is bad 5 is good) Running a marginal model in SPSS

Using a repeated effect Open the file Dabrowska2010.sav Linguists and non-linguists judged sentences for grammatical correctness Scale of 1-5 (1 is bad 5 is good). Some sentences were grammatical and others ungrammatical. Running a marginal model in SPSS Using a repeated effect

Open the file Dabrowska2010.sav Linguists and non-linguists judged sentences for grammatical correctness Scale of 1-5 (1 is bad 5 is good). Some sentences were grammatical and others ungrammatical. Dependent variable: Judgment (Score) Independent variables: Linguist or not (Group) Grammatical or not (GramUngram) Interaction of Group and GramUngram

Running a marginal model in SPSS Using a repeated effect Open the file Dabrowska2010.sav Linguists and non-linguists judged sentences for grammatical correctness Scale of 1-5 (1 is bad 5 is good). Some sentences were grammatical and others ungrammatical. Dependent variable:

Judgment (Score) Independent variables: Linguist or not (Group) Grammatical or not (GramUngram) Interaction of Group and GramUngram Repeated effect: Participant by GramUngram with UN, CS, or CSH covariance structures Running a marginal model in SPSS

Using a repeated effect Running a marginal model in SPSS Using a repeated effect Repeated effect of Participant by GramUngram Running a marginal model in SPSS Using a repeated effect

Repeated effect of Participant by GramUngram Compound Symmetry covariance structure Press Contiue Running a marginal model in SPSS Using a repeated effect

Specify the dependent variable Put categorical independent variables in the factors box Put continuous independent variables in the covariates box Both the independent variables and the repeated effects variables

are chosen here Click OK Running a marginal model in SPSS Using a repeated effect Chose Group, GramUngram as independent variables Chose Factorial to make an interaction variable

between the two Click Add Running a marginal model in SPSS Using a repeated effect Click on Continue Running a marginal model in SPSS Using a repeated effect

Click on Statistics Choose Parameter estimates and Covariances of residuals > Continue > OK Running a marginal model in SPSS Using a repeated effect You get this error. Warnings The levels of the repeated effect are not different for each observation within a repeated subject.

Execution of this command stops. Running a marginal model in SPSS Using a repeated effect You get this error. Participant by GramUngram is problem There are more than one item in the Gram and

Unbram groups We need to specify this Warnings The levels of the repeated effect are not different for each observation within a repeated subject. Execution of this command stops. Running a marginal model in SPSS Using a repeated effect This change is done in the

syntax editor. Click Paste to access it. Running a marginal model in SPSS Using a repeated effect This is under the hood. Change the last line to this: Running a marginal model in SPSS Using a repeated effect

Now place the curor in the text somewhere and press the green arrow.