PEHL 557
Class Notes
Formulating the Method
Student Learning Outcomes
Students will be able to:
Introduction
The second or third chapter in a thesis is usually entitled "Methods" or "Procedures." If you remember that one of the distinguishing characteristics of scientific research was replicability you will understand that the purpose of a Methods chapter is to explain a study in sufficient detail to enable duplication of the study.
Q. Typically, a Methods chapter has four sections. What are they?
A.
- Subjects
- Instruments/apparatus
- Procedures
- Design and Analysis
While all chapters of your study are important, the Methods chapter is especially important because you want to ensure that your study is really going to test what you want to test.
Q. What was this characteristic called?
A. Oh, yes - internal validity.
Ideally, you should write this chapter before you begin your study and distribute it among your committee for their comments. Now is the time to catch procedural errors before you invest beaucoup time in the study.
Example: Suppose as part of a study a graduate wanted to quantify the annual use of a city's recreational facilities and proposed to tally participation during the break between spring and fall quarters. Hopefully, committee members would point out that the summer months would not by themselves provide typical usage figures.
All studies experience problems that must be resolved or accepted. Your goal is to ensure that any problems affecting your study do not have any significant effect on the study's results.
Importance of MaxMinCon Principle
In the text you are introduced to what is called the MAXICON principle. Well, we're going to call it the MAXMINCON principle because it makes more sense.
Q. This principle reminds us of what need?
A. MAXIMIZE true variance
MINIMIZE error variance
CONTROL extraneous variance
Q. What does this all mean? Try to work it out by yourself or chat with peers and come up with examples. Illustrate the importance of this principle.
Subjects
Q. What information would you expect in this section?
1. How and why the subjects were selected
2. What pertinent characteristics these individuals had that made them targets of selection
3. The number of subjects - supplemented later by information on any drop outs.
Clearly, all of this information can be written before beginning the study then changed or expanded upon later.
Instruments
Q. Give me an example of information you might include here.
A. Here you will describe any apparatus, instruments, or tests used.
For example, if you choose to use a certain personality questionnaire you must give information on the test's reliability and validity. Information on scoring and a copy of the test should be included (Q. Where?) in your appendix. If you are using equipment you must give specific details of the make and models. In some instances you may want to include diagrams of equipment and facility layout. In short, include anything that will help later replication of your study.
Procedures
Q. Procedures are what? Give an example.
A. What did you do and in what order did you do things? Who was involved in the experiment? The steps outlined in your text provide a helpful checklist:
1. Explain the specific order of steps taken.
2. Include details on the timing of the various steps.
3. What were the subjects told? If the instructions were written include a copy (possibly in appendix).
4. Detail any briefings, debriefings, and safeguards.
Piloting: Hidden among the various humorous comments of the text authors is the statement that more than 75% of all the studies they have reviewed contain major methodological flaws that could have been avoided. Their recommendation, and certainly worth reinforcing is to Pilot all procedures before beginning the study.
Design and Analysis
As noted earlier, through careful design you must attempt to maximize the treatment variance, minimize error variance, and control extraneous variance. In this section of your chapter you must describe the design and any statistical techniques you plan to use. The section is often quite short containing little more than, "A t-test was used to determine differences in strength improvement between athletes and non-athletes following completion of a 6-week training program." Or, "An ANOVA was used to determine differences in mental ability of three groups of college age drug dependent students following a 30-day stress reduction program."
To complete this section you do need some idea of the appropriate way in which your data should be analyzed even though the actual analysis may be performed by a computer program.
And do remember - cause and effect are not established by statistical results but must include consideration of the entire experimental design. This relates to internal validity I think...does it? Be prepared to explain!
Bonus Advice
Writing /Review Procedures: When you have completed a draft of this chapter you should distribute copies to your committee and request input. Schedule a meeting for discussion. Not only might you receive ideas to improve your study but you are protecting yourself from later criticism by giving committee members an early chance to identify possible problems - and that's smart thinking!
(Revised 12/15/01)