Student Learning Outcomes
At the completion of this unit of instruction students will be able to:
Descriptive and Predictive Research
You will remember from earlier readings that there are many types of research that can be categorized as "descriptive." These included the following:
Surveys (questionnaires, Delphi method, interviews, normative)
The commonality that each of these share is that they are a measure of status - rather than prediction. Of course we might choose to predict future events based on our findings but that is not the intent of the techniques described in this chapter.
Imagine yourself the athletic director at Central Washington University. In addition to creating successful program you would clearly want a program that attracts the support of the student population at the university. While you might suppose that students would flock to support winning programs, this assumption may not be true. There have been many instances elsewhere of winning programs failing to attract support. Rather than let this important component of your program rest with fate, you might wonder what you could do to generate student interest for your athletic program. Although you could initiate a variety of non-scientific methods to "research" the problem, as an individual knowledgeable about descriptive research methods you could design a more logical and systematic approach to investigating this challenge.
Distributing carefully designed questionnaires to all or a sample of your student population would be one possible approach. As noted in your text there are some key steps to follow when constructing questionnaires. By following these steps you enhance the quality of the information you are able to obtain and also ensure that this information is in a form that can be objectively analyzed. Do remember however, that an important limitation to questionnaires is that they report what people say and not necessarily what they DO. Below the key steps involved in constructing a questionnaire are listed:
1. Determine the Objectives: From the example described earlier, it would appear that one of our principal objectives might be to determine why students either choose to attend or avoid athletic events. We might also want to know what could be done to make attendance more enjoyable. A recommended step is to list objectives, then think about the kind of responses you might anticipate, and plan how you might analyze these data. Planning the questionnaire is obviously a vital first step, and failure to plan well will likely undermine the value of the entire study.
2. Delimiting the Sample: Hopefully, you remember our earlier discussions about sampling. Some of the considerations in our example might be whether or not to distribute questionnaires to the entire student population or to select a representative sample. What would a representative sample look like? How will we sample? Do we need equal number of males and females? Do we want equal representation from freshmen, sophomores, juniors, and seniors? Do we need to consider racial composition? How many students do we need? Answers to some of these questions will depend on time and money. Certainly however, you can see that we need to think carefully about the sample we use in the study.
3. Constructing the Questionnaire: When people first attempt to build questionnaires they quickly discover that questions that appear clear to them are often open to many different interpretations. (Professors face the same challenge when constructing multiple choice exams.) It is a time consuming task and once again you would need to consider how you plan to analyze the possible responses. As you formulate your questions you must consider the most appropriate format
Open-Ended Questions allow the responder a variety of response options. The good part is that responders are free to say what they like. The bad part is that they take more time to answer and are tougher to analyze. In our example, we could ask "In what ways could the Athletic Department make your attendance at an athletic event more enjoyable?"
Closed Questions direct responders to certain choices among provided options. We can ask responders to rank order choices, select a score on a scale, or respond to provided categories. A closed version of the question posed above might be, "Should the Athletic Department sell concessions at athletic events?" (yes/no). Alternatively, "Which would be your preferred choice of days to attend basketball games? (Thursday/Friday/Saturday).
Whichever format you choose the wording of the questions, and the length and appearance of the questionnaire requires careful consideration if you are to maximize your returns.
4. Conducting a Pilot Study
A pilot study is an essential first step. Some time ago I wanted to know what motivate children to participate in ski school trips. I constructed a questionnaire with the reasons I believed would explain the childrens motivation. Fortunately, before I distributed the questionnaire I shared it with a colleague who pointed out that I did not include the option of "Was signed up by parents." As it turned out several children did not actually choose to learn to ski but were simply signed up by their parents. If I had not asked this question I would have missed a significant reason. As it turned out I made a second error by not piloting the questionnaire with kids. On the questionnaire students were asked to rank order their reasons for taking ski classes. We discovered that the younger students did not understand what was meant by "rank order" and other students were not sure what to do if they wanted to score two choices at the same rank.
5. Cover Letters
Many questionnaires will not be returned. Some people will discard them immediately upon receipt because they feel they dont have time. Others will look at the length or the type of information requested and then ignore or discard. The cover letter you include with the questionnaire creates a first impression and may sway whether or not you get a return. A brief, grammatically well written letter outlining clearly why you are requesting assistance may keep the questionnaire out of the discard pile (at least for the time being!). Usually, the cover letter will include a requested date to receive the response.
6. Sending the Questionnaire
As you can imagine your response rate from university students would probably not be high if they received the questionnaire during finals week. A person sending a questionnaire to coaches would be well advised to avoid the height of the coaching season. To increase the response rate a stamped, self-addressed envelope is advised. Some people will feel guilty not using the stamp and may be more inclined to respond to your questionnaire.
7. Follow up
Response rates are typically much lower than expected. Some types of questionnaires with certain samples might elicit much higher than average responses. A 50% response rate is often as good as it gets. To increase responses many researchers develop a system of follow-ups. These might begin with a postcard reminder, then be followed up with another questionnaire, and finally - if the response is vital - with a phone call. The authors of your text note that when response rates are extremely low (10-20%), the value of the findings is highly questionable.
8. Analyzing the Results and Preparing the Report
Once the questionnaires have been returned with a satisfactory response rate, the data must be analyzed and reported. Most often descriptive statistics (means, medians, modes, %, demographic data etc) will be used. Remember however, that methods of analysis will of course have been decided in the planning phase of the study. To a great extent, the nature of the discussion section in your study will depend on the results you obtain.
The Delphi Method
As explained in your text the Delphi method is a survey technique that involves the use of questionnaires in an attempt to get consensus on a topic. Subjects respond to a first questionnaire, then based on these responses a second questionnaire is developed and administered. Each time the questionnaire is administered is called a "round." Suppose for example, you were interested in investigating the impact of the 1980 Soviet boycott of the Olympic Games. Using the Delphi method you could survey knowledgeable Russians and Americans, each time sharing the different perspectives obtained, in an attempt to identify critical issues and perspectives.
Personal interviews are similar to questionnaires except in the manner in which they are administered. Some of the advantages and disadvantages (or challenges) to this method are indicated below:
|Greater confidentiality possible because of personal contact||Fewer subjects can be sampled|
|Flexibility to give follow-up questions||More expensive because of travel or phone|
|Opportunity to clarify questions||Need to be able to take notes quickly or get permission to tape|
|Can judge adequacy (honesty?) of replies||Need to be able to listen to one reply and be ready to follow-up immediately with the next question|
|Higher return rate||Requires skilled interviewer|
As noted in the text, conducting an interview effectively requires practice. You will notice from watching TV interviews how the interviewer usually tries to control the pace of the interview. Some people will talk for a long time but say very little if permitted to do so. Another problem is that some people tend to give an opinion when they really are not very knowledgeable about the topic. With practice the researcher quickly identifies whether a subject truly is qualified to respond to a particular question.
Although it is highly unlikely that you as graduate students would be involved in normative types of surveys, you should be aware that they have been widely used in physical education and health. Two sets of norms often used in public school physical education are the Presidents Physical Fitness Challenge and AAHPERDs Physical Best. Both consist of fitness data collected through nationwide normative surveys that teachers use to compare the fitness levels of children in their schools. In these types of surveys, the intent of the researchers is typically to establish performance norms to which the performance of others can be compared. As you can imagine this endeavor is often the subject of criticism by those who claim that scores taken in one environment lack validity when applied to a different environment.
If you are considering research involving questionnaires or interviews be sure to carefully examine the many aspects involved in the successful application of these types of research methods. An excellent resource for further reading is a book entitled Research Processes in Physical Education, by Clark and Clark.
Some years ago a researcher at the University of Oregon named H. Harrison Clarke initiated a growth study in the Oregon town of Medford. This project spurned many research papers, professional presentations, and graduate theses and dissertations. For several years researchers would visit Medford and track the growth and development of children in the public schools. This project provides an example of a longitudinal study - in other words a study of the same subjects over a period of several years. Were these same researchers to conduct a cross-sectional study we might anticipate that they would have visited Medford on one occasion and taken growth measurements of different subjects at several grade levels.
Developmental studies (despite the limitations noted in the text) can provide some fascinating insights and although only descriptive in nature can spark the researchers curiosity for more controlled experimental studies.
Some years ago a graduate student at our university was interested in examining the topic of teenage pregnancy. Although she could have researched the topic with a questionnaire she decided to use a case-study approach. The advantage of this method was that the topic could be examined in-depth, albeit gathering information from a much more limited sample of subjects than would have been possible with a questionnaire. In contrast to the questionnaire approach - which requires the researcher to have excellent knowledge of the topic when designing questions - case study researchers often approach their subjects with an inquisitive mind and an openness that permits subjects to respond in an unlimited number of directions. As you can imagine, this less structured approach may take researchers down avenues they did not anticipate traveling and open doors to new kinds of understanding.
In terms of the types of case study identified in your text, in the example given above the graduate student was probably conducting an interpretive study. Certainly, however she included a great deal of description. Were she to have approached the topic with the intent of identifying better ways of preventing teenage pregnancies then she would probably have conducted more of an evaluative study.
In summary, case studies tend to provide in-depth information about a limited number of subjects, and may produce new insights that generate additional studies.
A topic of great interest to those of us in teacher education has been in identifying those factors that contribute to effective teaching. Research on effective teaching has been through several stages. Initially, it was believed that the most effective teachers had special personality characteristics. Later it was suggested that the key to effective teaching might lie in the methods used.
Research in this area has often involved the observation of teachers and the categorization of behaviors. In observing the behaviors of teachers in classrooms in which learning is apparently occurring, certain commonalties have been identified.
The researchers who have investigated this area have engaged themselves in a type of observational research. Instead of approaching the question of effective teaching by asking questions, they chose to observe the behaviors of teachers. As noted in your text observational research necessitates adherence to certain guidelines in order to be considered as valid and reliable. As some of you know, here at CWU we use several of these observational methods (e.g. videotaped lesson analysis) in our undergraduate preparation of PE majors.
Although we discussed the techniques of correlational research in our discussion about statistics, it is important to appreciate that correlational research is descriptive. Because there is no manipulation of variables or controls, in correlational research is impossible to conclude that something "caused" something to occur. Remember that correlations are indicators of a relationship and not an effect.