PEHL 557
Class Notes
Introduction to Research in PEHLS
Student Learning Outcomes
At the completion of this unit of instruction students will be able to:
What is research?
The word "research" can be confusing because it is frequently used in conversation to describe a non-scientific searching process. For example, you might say that you conducted "research" to find the best price for a computer, or maybe you "researched" the best chocolate chip cookie recipe. In both instances you clearly did not follow a rigorous, scientific method of solving problems.
Q. In what ways is the word "research" as used in a class entitled "Research Methods" different from the kind of research done when looking for a better way to make chocolate chip cookies?
Research in PEHLS
According to Day, scientific problem solving involves answering four questions. The answers to these questions comprise the typical four chapters in a masters theses.
Q. What were these four questions - guess if you don't know
1. What was the problem? (Introduction)
2. How was the problem studied? (Methods)
3. What did you find? (Results)
4. What do the findings mean? (Discussion)
These four questions constitute a logical problem solving approach and should be your guide when designing and conducting research.
Five Characteristics of HPER Research
1. Systematic - research begins with the identification and labeling of variables followed by the design of research that tests relationships between these variables. Data are collected and related to variables to permit evaluation of the problem.
Identification and labeling of variables
Design of research
Collection of data
Analysis of data
Evaluation of the problem
2. Logical - as you can see the procedures follow a logical pattern that permits the results to be evaluated.
3. Empirical - data (in some form e.g. numbers, statements, scores, etc.) are collected on which to base decisions. Your decisions are not based on guesswork or intuition but on solid evidence.
4. Reductive - scientific research takes many individual events (in the form of data) and uses them to establish more general relationships. Lots of data is "reduced" to specific conclusions. For example, if on New Year's Eve a police officer took your blood pressure, pulse rate, performed a blood chemical analysis, gave you a breath test, and asked you to walk along a straight line the results of his findings might be "reduced" to the conclusion that you were drunk! When we collect data from many subjects we also reduce it to drawn common conclusions about the group as a whole.
Q. Think of a study you might be interested in. In what way might your study be reductive? Write down an example in your notes and be prepared to share in class.
5. Replicable - if you've followed the preceding steps and if you have recorded what you've done it should be possible for another person to replicate your study. Replication of studies is often performed to test the findings of original research especially if the findings are construed as controversial.
From where do problems come?
Problems are anywhere and everywhere. For example, consider this class and the goal I would like which is to provide the best possible learning experience. I could teach it many ways. I could give three hour lectures, I could lecture a bit then give some practical assignments, and I could write a self-study guide and tell you to follow the guide and see me when something is unclear. Which would be the most effective from the viewpoint of your learning? I could guess or I could design a research study to test the effectiveness of the various methods.
Q. Write down a sample problem. It can be something of personal interest or just a topic related to HPER.
I'm also interested in how motivation affects performance. In other words do people perform better when highly motivated or is the relationship a little more complex? To discover the truth I could design a study in which the relationship is examined closely.
Q. Take a moment to think about these two examples. Obviously they are different topics but they also differ in a significant way that was explained in the text. Be prepared to explain this difference in class.
Notice that in the first study I proposed I was seeking the solution to a practical problem that I face as a teacher. In the second study I did not give any indication that I was looking for solutions to problems but rather that I wanted to learn more about human behavior. It's possible (in fact likely) that the results of the second study might have implications for practice but I was not trying to solve any practical problem in conducting the study. So what? Read on.
Differences between basic and applied research
The examples given above illustrate some of the differences between studies that people might characterize as "basic" research and research that is viewed as "applied."
Q. What are some of the major differences between applied and basic research?
Here's the list of distinctions discussed in your text.
Applied -----------------------------Basic
1. Answers immediate problems Deals with theoretical issues
2. Subjects usually humans Often uses animals as subjects
3. Uses real-world settings Conducted in laboratories
4. Lacks rigorous controls Study is carefully controlled
5. Results are directly useful Results often lack application
Q. Which is the most valid?
There exists some controversy over which form of research is most valid. In practice most research involves some compromise of the extremes indicated in the model above. Unfortunately, the arguments often involve attempts to assert that one type of research is more valid than another.
For example, for years this was true in the field of sport psychology. The elite researchers were those people who conducted research in laboratories and developed sophisticated theories that they then tried to apply to competitive sports. Of course, the theories rarely worked well because of the complexity of sport situations. Many of these researchers later recognized the need to conduct their projects in real world settings. New professional associations were created - hence the existence now of several professional organizations for sport psychologists and the creation of several different journals.
Clearly, it's just opinion as to what problems are more important. Who is the better researcher? Is it the person who goes to New York city and begins an experimental drug rehabilitation program or the person develops a theory for breaking drug dependency based on laboratory research with rats?
Scientific and non-scientific approaches to problem solving
Q. What are they? Here is the list and some more examples of non-scientific approaches that you should endeavor to avoid.
1. Tenacity - when a person clings to a belief that lacks any supporting evidence. Years ago Bjorn Borg the Swedish tennis star would never cut his hair during the Wimbledon tennis tournament believing it would affect his playing performance. He was probably right wasn't he - however, not for reasons related to hair length.
2. Intuition - sometimes beliefs are held because they appear to be self-evident or just common sense. For example, clearly it is not a good idea to exercise a knee immediately after surgery but much better to give the knee time to rest and recover from the trauma...or is it? In fact now many surgeons are encouraging rapid rehabilitation. Patients are told to exercise immediately following surgery before the knee gets stiff and sore. So much for intuition!
3. Authority - we often believe something because we are told it by an authority figure. It may be someone we respect or simply a person who has authority over us. Suppose I tell you that research on smoking cessation and exercise is a waste of time because of too many uncontrollable factors. If you believe me and give up on your study idea you have succumbed to a non-scientific opinion. You would be much wiser to question the basis of my advice and seek other opinions.
In all your classes at this or any university you should be politely skeptical of facts that appear to not have any scientific basis. One of my current favorites involves stretching. We are told to stretch before exercising to avoid injury. I have never seen any evidence showing that stretching reduces injury. It makes sense but where's the evidence. Some people actually injure themselves while stretching! And what about strengthening the stomach muscles to reduce lower back pain! That might work if the cause of the pain is weak muscles but will have no impact on other causes of pain. Be wary of authority!
4. Rationalistic - this is knowledge derived from reasoning. We know that lawyers earn high salaries, so if Sally is a lawyer we might reason that Sally earns a high salary. On the other hand if women like wearing dresses and Frank likes to wear dresses does it follow that Frank is a woman? In the first example, the premise was probably valid therefore the conclusion was trustworthy. In the second example, the premises were not related thereby making the conclusions not reliable. An example of faulty reasoning in health might be that smoking causes cancer, cancer is life threatening, therefore people who know that smoking causes cancer will not smoke...unfortunately they do!
While the ability to reason is fundamental in the scientific method of problem solving we need to be careful with the conclusions we draw and always look for supporting evidence.
5. Empirical Method - although everything that's been said so far appears to emphasize the need for data it's also possible to draw erroneous conclusions because of faulty data collection and interpretation. Suppose we test a new drug on Amazonians and find a reduction in skin cancer. Can we realistically conclude that the drug would also be effective with Caucasian Europeans? Or suppose I test the effectiveness of a device for improving the physical mobility of handicapped children but use non handicapped children for my subjects because of the lack of handicapped children. How valid are the findings. It's easy to abuse and misuse data to promote a belief. We must be careful to remain objective when evaluating data.
The five non-scientific methods just described show us how not to conduct effective research. So, now that we know what not to do, what should we do?
There are four key steps involved in the scientific method of problem solving.
Step 1: Developing the problem (defining and delimiting it)
For now we'll assume you've come up with a reasonable problem (we'll return to this task later). You should be able to identify what exactly you plan to study. What are your independent and dependent variables? The independent variable (IV) is what you plan to manipulate. For example, suppose you want to discover which technique for weight training produces the greatest strength development. You might choose to test three different techniques (free weights, weight machines, and a combination of both).
Q. In this example what were your independent variables?
In this instance the training method is your independent variable. There was only one IV
Now don't get confused. I know I mentioned three different techniques of weight training. You might be thinking this means there are three different IVs - it doesn't. Why not? Because each subject in your study will only train using one of the three techniques. You have divided your subjects between the three techniques but your IV remains the training method. The three techniques are all forms of a training method. Kind of confusing huh? Incidentally, the IV is also often referred to as the experimental or treatment variable.
Q. What might be your DV in this example?
What will your dependent variable (DV) be in this example - do you know without reading on? Well the DV is what you are trying to influence by manipulating the IVs. It's the quality that you want to change. In the example just discussed the researcher wants to improve strength - thus, some measurement of strength will be your DV. Get the idea?
Q. To check your understanding why don't you note down a couple of examples of your own then mention them in class.
Step 2: Formulating the hypothesis
Q. What is a hypothesis?
It's the anticipated result of your research and is usually based on previous research. Hypotheses (there may be more than one) must be testable and your study should be planned in a way that allows you to support or refute your hypotheses.
I emphasize the word testable because you must be able to make an valid measurement of the quality you claim to be influencing. For example, suppose you wanted to improve strength and chose to use pull-ups as your test of strength. Many people cannot do a single pull up. At conclusion of your study you might find half your subjects unable to score even one pull up. Does this mean your IV was ineffective? What if a person can do two pull ups; does this mean the person is 100% stronger than the individual who can do only one? Clearly, your study did not allow you to test your hypothesis in any truly meaningful way. Same is true when people use the 1.5 mile run to evaluate fitness. If people don't try hard and walk what do the results indicate? (big failing in AAHPERD fitness test)
There will be more on hypotheses later!
Step 3: Gathering the data
As we've learned, data needs to be collected to support or refute a hypothesis. How you collect data is part of your experimental design. Care is essential to select appropriate data collecting methods and to avoid measurement errors. During this stage you must plan to maximize internal and external validity. What are these factors?
Q. Be prepared to discuss differences between internal and external validity. Give me an example of studies that lacked these types of validity. Make notes. Common research error to try to minimize.
Internal Validity - to what extent are the results of the study attributable to the treatment? For example, let's suppose a PE teacher wants to test the effects of her fitness program. She tests 2nd and 5th graders, finds the 5th graders score higher so concludes that her fitness program was effective. Do you see any problems? Is it okay to compare 2nd and 5th graders?
External Validity - to what extent are the results generalizable outside the study? For example, if health educators in Washington state report that AIDS knowledge among 12th graders is below a minimum acceptable level is it okay to conclude that American children are insufficiently educated on AIDS? Do you think this concern is a greater concern with basic or applied research?
Step 4: Analyzing and Interpreting Results
This part of a study often generates the most anxiety among graduate students because it may involve using statistics. In most instances the actual computations can be done by computer. The tough part is deciding what type of analysis to perform, then later making sense of the figures the machine spews out! It's at this stage you begin to examine the extent to which your data supports or refutes your experimental hypotheses. You will also relate your findings to those of previous investigators and to existing theories.
Some different ways to conduct research in PEHLS
In your text there is a section on Alternative Models for Research. Normal or naturalistic scientific problem solving methods existed for many years as the standard for "good" and "objective" scientific problem solving. Researchers in exercise science typically have followed this type of approach. As you can see however, there have been questions raised about the supposed objectivity of normal scientific methodology as well as criticism regarding the value of its findings. I like the following quote from p. 17:
The bottom line is that different problems require different solutions....The nature of the research questions and setting should drive the selection of approaches to acquiring knowledge.
No one method is better than another and your choice should depend on the type of question you are seeking to answer. However, each type of research does involve different procedures and expects the user to develop skills in application. It's probably obvious that you can't conduct an experiment unless you have some understanding of statistics. The same is also true for analytical and descriptive research. Conducting an interview, for example, requires certain skills that must be developed through knowledge and practice. Historical studies demand specific analytical approaches. What this means is that if you want to pursue a particular type of research you would be wise to learn more about the specific methodology demanded in that research.
History of PEHLS research
Researchers in PEHLS have been prolific. This would be great if it could be concluded that we now know a great deal about the subject matter in our fields. Unfortunately, many people would argue that we still know very little for certain about our fields. Reviewing the past 30 years in science and medicine it could be argued that tremendous advances have been made. Computers are now a part of everyday life, people have walked on the moon, heart transplants have proven successful, and much more. In contrast, if you were to walk into a school and view a PE lesson in most instances it probably wouldn't look any different from the time you went to school. In fact, someone (a lot smarter than me!) once made the comment that if you took away all the research that has been conducted in physics over the past 50 years the world would be a vastly different place. In contrast the removal of research by professionals in PEHLS would probably not even be noticed! (Sad but maybe true.) Not much has changed despite volumes of research. Why not?
One reason I'd propose is because much of our research is flawed in design (therefore the results are equally flawed), and a second reason is that our research if often unnecessarily repetitious. Unfortunately, because a lot of research is conducted for masters theses or doctoral dissertations it is "one-of" kind. Not only does the research often contain design flaws but it does not build on existing knowledge. In counter to this argument some faculty would say that it is unrealistic to expect students to conduct original research and that it is the process of conducting the research that is most important rather than the results. Although this argument certainly has validity it's too bad that graduate students spend a lot of time on research that rarely contributes significantly to our knowledge.
These are ideas to think about as you begin planning your thesis or project...
Types of Research
Because there are very many different types of problems that can be studied in PEHLS there is an equally diverse array of approaches to solving these problems. As noted in the text there are the traditional "scientific" approaches that use experimental (predictive) statistics as well as many approaches that use statistics in descriptive ways. Read these examples and be thinking about how you might use some of these approaches in your area of interest. We will discuss many of these different types of research over the quarter.
Research methods process overview
Review the chart on p. 22 and be prepared to explain it to me in the form of a practical example.
The parts of a study (thesis)
In the first chapter of the text an overview has been presented on the steps involved in the research process. If you decide to conduct an experimental research project your thesis will usually follow a generally accepted format. The next few chapters in the text model this format taking you through the beginning steps in a standard experimental thesis. Notice I've emphasized that the format presented is that accepted for experimental studies. Different types of research have different methodology and may be presented in alternative formats. However, whenever you read an experimental research study you can anticipate that information will be presented in the following order:
1. Introduction
2. Review of literature
3. Method
4. Results
5. Discussion and conclusions
(Revised 12/15/01)