class: center, middle, inverse, title-slide # t-tests ## Independent Sample ### Matthew Crump ### 2018/07/20 (updated: 2019-03-18) --- #overview 1. Independent samples t-test calculation 2. Example from lab 3. Directional and non-directional test 4. t-test assumptions --- class: pink, center, middle, clear # t-tests and designs --- # Three kinds of t-tests 1. one-sample 2. paired-sample 3. Independent sample --- # Independent Sample t-test Purpose: Compare two sample means in a between-subjects design Between-subjects design: **Different** subjects are measured across both levels of the experimental manipulation (independent variable) --- # Consider this Between-subjects experiment, n=5, different subjects are measured in group A and B of the experiment. <table> <thead> <tr> <th style="text-align:right;"> subjects_A </th> <th style="text-align:right;"> A </th> <th style="text-align:right;"> subjects_B </th> <th style="text-align:right;"> B </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 4 </td> </tr> <tr> <td style="text-align:right;"> 2 </td> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 7 </td> <td style="text-align:right;"> 8 </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 8 </td> <td style="text-align:right;"> 7 </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 9 </td> <td style="text-align:right;"> 9 </td> </tr> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 10 </td> </tr> </tbody> </table> --- # Empirical question Did the manipulation (A vs. B) cause a difference in the measure? <table> <thead> <tr> <th style="text-align:right;"> subjects_A </th> <th style="text-align:right;"> A </th> <th style="text-align:right;"> subjects_B </th> <th style="text-align:right;"> B </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 4 </td> </tr> <tr> <td style="text-align:right;"> 2 </td> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 7 </td> <td style="text-align:right;"> 8 </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 8 </td> <td style="text-align:right;"> 7 </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 9 </td> <td style="text-align:right;"> 9 </td> </tr> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 10 </td> </tr> </tbody> </table> --- # Can't use Difference scores Unlike a paired samples t-test, we can't look at the difference scores for a between-subjects design. Why not? <table> <thead> <tr> <th style="text-align:right;"> subjects_A </th> <th style="text-align:right;"> A </th> <th style="text-align:right;"> subjects_B </th> <th style="text-align:right;"> B </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 1 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 4 </td> </tr> <tr> <td style="text-align:right;"> 2 </td> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 7 </td> <td style="text-align:right;"> 8 </td> </tr> <tr> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 3 </td> <td style="text-align:right;"> 8 </td> <td style="text-align:right;"> 7 </td> </tr> <tr> <td style="text-align:right;"> 4 </td> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 9 </td> <td style="text-align:right;"> 9 </td> </tr> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 10 </td> </tr> </tbody> </table> --- # Independent samples t-test .pull-left[ Purpose: Compare two means from different samples - `\(\bar{X}_1\)` = mean of sample 1 - `\(\bar{X}_2\)` = mean of sample 2 - `\(s_p\)` = pooled standard deviation - `\(s_p^2\)` = pooled variance - `\(n\)` = sample-size - `\(df\)` = `\(n_1 + n_2 - 2\)` ] .pull-right[ `\(t = \frac{\bar{X_1}-\bar{X_2}}{s_p\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\)` `\(s_p^2 = \sqrt{\frac{(n_1-1)s_\text{X1}^2+(n_2-1)s_\text{X2}^2}{n_1+n_2-2}}\)` ] --- # Pooled standard deviation `\(s_p\)` (standard deviation) and `\(s_p^2\)` (variance) are **pooled estimates** (combined). We have two samples 1. The basic idea is to find the average of the standard deviations 2. The formula makes the estimate unbiased --- # R: Calculating t The `t.test()` function assumes independent samples by default. We do need to set the `var.equal=TRUE` option. ```r A <-c(1,4,3,6,5) B <-c(4,8,7,9,10) t.test(A,B,var.equal = T)$statistic ``` ``` ## t ## -2.832353 ``` --- # Calculating pooled estimate ```r A <-c(1,4,3,6,5) B <-c(4,8,7,9,10) mean_dif <- mean(A)-mean(B) numerator <- (4*var(A)) + (4*var(B)) denominator <- 5+5-2 sp2 <- sqrt(numerator/denominator) t<- mean_dif/(sp2*sqrt((1/5)+(1/5))) t ``` ``` ## [1] -2.832353 ``` --- class: pink, center, middle, clear # Lab Example --- # Schroeder and Epley (2015) <img src="figs/ttest/Lab7a.png" width="2925" /> --- # Research Question Are evaluations of a person's intellect better conveyed through writing or speaking? --- # Method - Job applicants in an interview give both a written statement, and an audio version of them reading the statement - Interviewers (profesional recruiters) read or listen to each statement, and rate each applicant (intellect, general impression, Hiring likelihood) --- # IVs and DVs IVs: - Read written transcript vs. Listen to spoken audio transcript DVS: - Intellect rating (0-10) - general impression rating (0-10) - hiring likelihood rating (0-10) --- # The results <img src="figs/ttest/Lab7b.png" width="80%" /> --- # The write-up <img src="figs/ttest/Lab7c.png" width="1416" /> --- # R analysis <img src="figs/ttest/Lab7d.png" width="1648" /> --- class: pink, center, middle, clear # Directional vs. non-directional tests --- # Critical t Critical t is set by two properties: 1. the alpha criterion 2. whether the test is directional (one-tailed) or non-directional (two-tailed) --- # Directional test (reminder) A directional test assumes that the experimental manipulation will cause a difference in a particular direction. - mean for A > (greater than) mean for B - mean for A < (less than) mean for B --- # Critical t (one-tailed) example Critical t for a directional (one-tailed) test - alpha = 0.05, or 5% Critical t is the t-value associated with a null-distribution where this t-value or larger occurs 5% of the time. --- # Critical t (one-tailed) <img src="figs/ttest/6critT-1.png" width="1792" /> --- # Critical t depends on df and alpha .pull-left[ The table shows values of critical t for a one-tailed test - alpha values of .10, .05, and .01 - degress of freedom from 5 to 100 ] .pull-right[ <table> <thead> <tr> <th style="text-align:right;"> df </th> <th style="text-align:right;"> p_10 </th> <th style="text-align:right;"> p_05 </th> <th style="text-align:right;"> p_01 </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 1.48 </td> <td style="text-align:right;"> 2.02 </td> <td style="text-align:right;"> 3.36 </td> </tr> <tr> <td style="text-align:right;"> 6 </td> <td style="text-align:right;"> 1.44 </td> <td style="text-align:right;"> 1.94 </td> <td style="text-align:right;"> 3.14 </td> </tr> <tr> <td style="text-align:right;"> 7 </td> <td style="text-align:right;"> 1.41 </td> <td style="text-align:right;"> 1.89 </td> <td style="text-align:right;"> 3.00 </td> </tr> <tr> <td style="text-align:right;"> 8 </td> <td style="text-align:right;"> 1.40 </td> <td style="text-align:right;"> 1.86 </td> <td style="text-align:right;"> 2.90 </td> </tr> <tr> <td style="text-align:right;"> 9 </td> <td style="text-align:right;"> 1.38 </td> <td style="text-align:right;"> 1.83 </td> <td style="text-align:right;"> 2.82 </td> </tr> <tr> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 1.37 </td> <td style="text-align:right;"> 1.81 </td> <td style="text-align:right;"> 2.76 </td> </tr> <tr> <td style="text-align:right;"> 20 </td> <td style="text-align:right;"> 1.33 </td> <td style="text-align:right;"> 1.72 </td> <td style="text-align:right;"> 2.53 </td> </tr> <tr> <td style="text-align:right;"> 50 </td> <td style="text-align:right;"> 1.30 </td> <td style="text-align:right;"> 1.68 </td> <td style="text-align:right;"> 2.40 </td> </tr> <tr> <td style="text-align:right;"> 100 </td> <td style="text-align:right;"> 1.29 </td> <td style="text-align:right;"> 1.66 </td> <td style="text-align:right;"> 2.36 </td> </tr> </tbody> </table> ] --- # Non-Directional test A non-directional test assumes that the experimental manipulation will cause **any** difference. - mean for A != (will not equal) mean for B E.g., - Mean for A could be bigger or smaller than mean for B --- # Non-directional test (2-tailed) <img src="figs/ttest/6twotailedt-1.png" width="1792" /> --- # Comparing critical t (1 vs 2 tailed) .pull-left[ One-tailed <table> <thead> <tr> <th style="text-align:right;"> df </th> <th style="text-align:right;"> p_10 </th> <th style="text-align:right;"> p_05 </th> <th style="text-align:right;"> p_01 </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 1.48 </td> <td style="text-align:right;"> 2.02 </td> <td style="text-align:right;"> 3.36 </td> </tr> <tr> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 1.37 </td> <td style="text-align:right;"> 1.81 </td> <td style="text-align:right;"> 2.76 </td> </tr> <tr> <td style="text-align:right;"> 20 </td> <td style="text-align:right;"> 1.33 </td> <td style="text-align:right;"> 1.72 </td> <td style="text-align:right;"> 2.53 </td> </tr> <tr> <td style="text-align:right;"> 50 </td> <td style="text-align:right;"> 1.30 </td> <td style="text-align:right;"> 1.68 </td> <td style="text-align:right;"> 2.40 </td> </tr> <tr> <td style="text-align:right;"> 100 </td> <td style="text-align:right;"> 1.29 </td> <td style="text-align:right;"> 1.66 </td> <td style="text-align:right;"> 2.36 </td> </tr> </tbody> </table> ] .pull-right[ Two-tailed <table> <thead> <tr> <th style="text-align:right;"> df </th> <th style="text-align:right;"> p_10 </th> <th style="text-align:right;"> p_05 </th> <th style="text-align:right;"> p_01 </th> </tr> </thead> <tbody> <tr> <td style="text-align:right;"> 5 </td> <td style="text-align:right;"> 2.02 </td> <td style="text-align:right;"> 2.57 </td> <td style="text-align:right;"> 4.03 </td> </tr> <tr> <td style="text-align:right;"> 10 </td> <td style="text-align:right;"> 1.81 </td> <td style="text-align:right;"> 2.23 </td> <td style="text-align:right;"> 3.17 </td> </tr> <tr> <td style="text-align:right;"> 20 </td> <td style="text-align:right;"> 1.72 </td> <td style="text-align:right;"> 2.09 </td> <td style="text-align:right;"> 2.85 </td> </tr> <tr> <td style="text-align:right;"> 50 </td> <td style="text-align:right;"> 1.68 </td> <td style="text-align:right;"> 2.01 </td> <td style="text-align:right;"> 2.68 </td> </tr> <tr> <td style="text-align:right;"> 100 </td> <td style="text-align:right;"> 1.66 </td> <td style="text-align:right;"> 1.98 </td> <td style="text-align:right;"> 2.63 </td> </tr> </tbody> </table> ] --- # Making decisions .pull-left[ One-tailed - reject null - observed t in green area - fail to reject null - observed t in white area ] .pull-right[ <img src="figs/ttest/6critT-1.png" width="100%" /> ] --- # Making decisions .pull-left[ Two-tailed - reject null - observed t in green area - fail to reject null - observed t in white area ] .pull-right[ <img src="figs/ttest/6twotailedt-1.png" width="100%" /> ] --- class: pink, center, middle, clear # t-test assumptions --- class: center, middle, clear <iframe style="width:100%;height:100%;border-style:none;", src="https://crumplab.shinyapps.io/indTtest/" /> --- # Next class: Power and Effect-size 1. Quiz on t-tests starts today, due next Monday 2. Midterm review next Monday 3. Midterm review sheet and info is posted on Blackboard