Student Learning Outcomes
At the completion of this instructional unit students will be able to:
Review
We've learned that statistics can be used to describe the characteristics of data (means, medians, SDs etc.), to show relationships (r, R, regression), and to show differences among groups.
Q. In what type of research settings are we most likely to encounter statistics to show differences among groups?
A. In experimental and quasi-experimental settings, when interested in evaluating effects of IV on DV.
Q. What do we mean by quasi-experimental settings?
A. Real world settings that lack true randomization
Q. Statistics never establish cause and effect. Why? What do they tell us?
A. Because cause and effect depends on consideration of the total experimental design. Statistics tell us what occurred, not why it occurred. For example, there was a difference between two training methods on strength improvement - why this occurred is not explained by statistics.
Q. In correlational studies we learned that it was important both to find significance and also to evaluate the strength or meaningfulness of the finding. The same is true in experimental settings. What types of statistics do we use?
A. For significance we use t and F ratios and for strength or meaningfulness we use omega squared (w2) and effect size (ES)
Q. There are three types of t test. Name them and give an example of their possible use.
A.
1. t test between sample and population mean, e.g. if a national fitness test existed we could compare the scores of this class to those national norms.
2. independent t test, e.g. comparing two samples
3. dependent t test, e.g. when the group scores are somehow related, such as in pre and posttest situations. Suppose I wanted to show how much your knowledge of research methods improved over the quarter I might test you on the first day then again during finals. I'd use a dependent t test.
t tests and power
Q. What does the word "power" mean in reference to research? And what do we know about ways in increase power?
A. Power refers to the probability of rejecting the Ho when it is false. Obviously this is highly desirable. There are three key ways to increase power that can be easily understood by examining the formula for the t ratio:
t = M1 - M2 1
/s12 + s22 2
/ n1 n2 3
1. Increasing the difference between the means increases the size of the t. How to do? Give stronger treatments.
2. Decreasing the standard deviation reduces the denominator. How to do? Apply treatments consistently so subjects responses to the treatment will be similar. Remember the SD is affected by variations in response between subjects in a group.
3. Increasing the #s will decrease the denominator and help to increase the t.
ANOVA
Q. t test is one statistical technique to examine differences between groups, ANOVA is another. What is the difference?
A. t test is used with two groups. If there are more than two groups you must use ANOVA
Q. So let's suppose that we run a simple ANOVA (Q. Why called simple? A. Because there's only one IV) on three dieting classes and find a significant F ratio. What does this tell us? What does this not tell us?
A. That there is a difference between the three groups but not where the differences lie.
Q. What would probably be our next step once we've determined a significant F exists?
A. Follow up tests - Post-hoc comparisons.
Q. Several follow up tests exist. Name them?
A. Scheffe, Neuman-Keuls, Duncan's. Try to remember these names because researchers often just refer to the name when discussing their analysis
Q. Rather than doing a follow up test a researcher anticipating the emergence of differences would use what kind of analysis?
A. Planned comparisons
Factorial ANOVA
Q. We use simple ANOVA rather than a t test when? So if we had three weight training groups and wanted to compare them we could use simple ANOVA. What about if we also wanted to make comparisons between males and females using three different weight training methods? How would we analyze this?
A. Factorial ANOVA
Q. What I described would use a (3x2) factorial ANOVA. In this situation what are some of the findings I would be interested in?
A. Main effects but primarily the interactions
Q. Who can explain what an interaction is? Using my example and supposing that there was indeed a significant interaction make up a possible finding.
A. Although in general free weights produced greater strength gains than either machines or conditioning exercises, the difference was only significant for males.
Researcher usually checks first for interactions. If no interaction then main effects. If interaction exists often main effects will not be of great interest.
Q. Check understanding of what the IVs and DVs are in this example.
Repeated Measures
Q. Repeated measures are common in PEHLS because we are interested
in learning and seeing improvement. For example, suppose we want
to see the extent to which a person improves between the start
of our weight training class and the end. How might this be measured?
A. Notice I said person. You hardly need statistics to see individual improvement. Remember that the statistics we are discussing here are designed to deal with group data. Let me rephrase the question to note that I am interested in strength improvement of the group. What statistics might be appropriate here?
A. Dependent t test
Q. What about if I was interested in the change of improvement over the course of the quarter. How might you design an experiment to measure this?
A. Repeat measurements every two weeks which would mean 5 sets of measurements for each individual. This is an ANOVA design with repeated measures on the IV variable of time. This design is also referred to as a split plot ANOVA or subject x trials ANOVA.
Repeated measures designs are often applied to factorial ANOVAs where there are more than one IV
Q. Advantages of repeated measure designs?
A.
1. can control individual differences which acts to reduce the error term.
2. fewer subjects required
3. useful when studying changes in a variable over time
Q. Disadvantages?
A.
1. carryover effects
2. practice effects
3. fatigue
4. sensitization to experimental arrangements which produces a change in response
Notice that these threaten the internal validity of the experiment. Of course in some situations these are precisely the effects of interest.
Analysis of Covariance
Q. Suppose you run your weight training program in the hope of seeing strength improvements as measured by chin ups. In this situation an ANCOVA might be used to analyze your findings. Why?
A. Because of the influence of body weight. ANCOVA would allow you to evaluate your programs as if body weights were the same.
ANCOVA often used when researcher wants to remove the influence of a pretest. Of course if the pretest is of interest you'd use a repeated measures design.
Although it sounds a neat idea there are concerns about using ANCOVA. When you adjust scores you can come to misleading conclusions. For example, you might conclude a program is effective when in reality you would not see this effectiveness. (i.e. they would not do any more chins)
Also you have to be sure that the effects of the covariate are similar across treatment groups.
Practice
To check your understanding, I encourage you to use a research review form to briefly outline a plausible experimental study that involves examining differences between groups. Bring questions and let's discuss in class.
(Revised 2/3/99)