PEHL 557

Class Notes

Descriptive Research

Student Learning Outcomes

At the completion of this unit of instruction students will be able to:

  1. Identify examples of descriptive research (surveys, case studies, documentary analyses, developmental studies, correlational studies)
  2. Identify the steps and key principles involved in constructing a questionnaire (determining objectives, delimiting the sample, constructing the questionnaire, conducting a pilot study, writing cover letters, sending the questionnaire, following up, analyzing the results, and preparing the report)
  3. Understand the purpose and value of the Delphi Method (repeated surveys to get consensus)
  4. Identify advantages and disadvantages of conducting personal interviews in comparison to questionnaires (see table).
  5. Recognize examples of normative surveys (Physical Best, President's Challenge)
  6. Distinguish between longitudinal and cross-sectional types of developmental research (study across time versus snap shot)
  7. Identify examples of observational research (e.g. ALT-PE)
  8. Understand the distinction between causation and association in correlational research.

Descriptive and Predictive Research

You will remember from earlier readings that there are many types of research that can be categorized as "descriptive." These included the following:

Surveys (questionnaires, Delphi method, interviews, normative)

Case Studies

Job Analyses

Documentary Analysis

Developmental Studies

Correlational Studies

The commonality that each of these share is that they are a measure of status - rather than prediction. Of course we might choose to predict future events based on our findings but that is not the intent of the techniques described in this chapter.

Survey Techniques

Imagine yourself the athletic director at Central Washington University. In addition to creating successful program you would clearly want a program that attracts the support of the student population at the university. While you might suppose that students would flock to support winning programs, this assumption may not be true. There have been many instances elsewhere of winning programs failing to attract support. Rather than let this important component of your program rest with fate, you might wonder what you could do to generate student interest for your athletic program. Although you could initiate a variety of non-scientific methods to "research" the problem, as an individual knowledgeable about descriptive research methods you could design a more logical and systematic approach to investigating this challenge.

Questionnaires

Distributing carefully designed questionnaires to all or a sample of your student population would be one possible approach. As noted in your text there are some key steps to follow when constructing questionnaires. By following these steps you enhance the quality of the information you are able to obtain and also ensure that this information is in a form that can be objectively analyzed. Do remember however, that an important limitation to questionnaires is that they report what people say and not necessarily what they DO. Below the key steps involved in constructing a questionnaire are listed:

1. Determine the Objectives: From the example described earlier, it would appear that one of our principal objectives might be to determine why students either choose to attend or avoid athletic events. We might also want to know what could be done to make attendance more enjoyable. A recommended step is to list objectives, then think about the kind of responses you might anticipate, and plan how you might analyze these data. Planning the questionnaire is obviously a vital first step, and failure to plan well will likely undermine the value of the entire study.

2. Delimiting the Sample: Hopefully, you remember our earlier discussions about sampling. Some of the considerations in our example might be whether or not to distribute questionnaires to the entire student population or to select a representative sample. What would a representative sample look like? How will we sample? Do we need equal number of males and females? Do we want equal representation from freshmen, sophomores, juniors, and seniors? Do we need to consider racial composition? How many students do we need? Answers to some of these questions will depend on time and money. Certainly however, you can see that we need to think carefully about the sample we use in the study.

3. Constructing the Questionnaire: When people first attempt to build questionnaires they quickly discover that questions that appear clear to them are often open to many different interpretations. (Professors face the same challenge when constructing multiple choice exams.) It is a time consuming task and once again you would need to consider how you plan to analyze the possible responses. As you formulate your questions you must consider the most appropriate format

Open-Ended Questions allow the responder a variety of response options. The good part is that responders are free to say what they like. The bad part is that they take more time to answer and are tougher to analyze. In our example, we could ask "In what ways could the Athletic Department make your attendance at an athletic event more enjoyable?"

Closed Questions direct responders to certain choices among provided options. We can ask responders to rank order choices, select a score on a scale, or respond to provided categories. A closed version of the question posed above might be, "Should the Athletic Department sell concessions at athletic events?" (yes/no). Alternatively, "Which would be your preferred choice of days to attend basketball games? (Thursday/Friday/Saturday).

Whichever format you choose the wording of the questions, and the length and appearance of the questionnaire requires careful consideration if you are to maximize your returns.

4. Conducting a Pilot Study

A pilot study is an essential first step. Some time ago I wanted to know what motivate children to participate in ski school trips. I constructed a questionnaire with the reasons I believed would explain the children’s motivation. Fortunately, before I distributed the questionnaire I shared it with a colleague who pointed out that I did not include the option of "Was signed up by parents." As it turned out several children did not actually choose to learn to ski but were simply signed up by their parents. If I had not asked this question I would have missed a significant reason. As it turned out I made a second error by not piloting the questionnaire with kids. On the questionnaire students were asked to rank order their reasons for taking ski classes. We discovered that the younger students did not understand what was meant by "rank order" and other students were not sure what to do if they wanted to score two choices at the same rank.

5. Cover Letters

Many questionnaires will not be returned. Some people will discard them immediately upon receipt because they feel they don’t have time. Others will look at the length or the type of information requested and then ignore or discard. The cover letter you include with the questionnaire creates a first impression and may sway whether or not you get a return. A brief, grammatically well written letter outlining clearly why you are requesting assistance may keep the questionnaire out of the discard pile (at least for the time being!). Usually, the cover letter will include a requested date to receive the response.

6. Sending the Questionnaire

As you can imagine your response rate from university students would probably not be high if they received the questionnaire during finals week. A person sending a questionnaire to coaches would be well advised to avoid the height of the coaching season. To increase the response rate a stamped, self-addressed envelope is advised. Some people will feel guilty not using the stamp and may be more inclined to respond to your questionnaire.

7. Follow up

Response rates are typically much lower than expected. Some types of questionnaires with certain samples might elicit much higher than average responses. A 50% response rate is often as good as it gets. To increase responses many researchers develop a system of follow-ups. These might begin with a postcard reminder, then be followed up with another questionnaire, and finally - if the response is vital - with a phone call. The authors of your text note that when response rates are extremely low (10-20%), the value of the findings is highly questionable.

8. Analyzing the Results and Preparing the Report

Once the questionnaires have been returned with a satisfactory response rate, the data must be analyzed and reported. Most often descriptive statistics (means, medians, modes, %, demographic data etc) will be used. Remember however, that methods of analysis will of course have been decided in the planning phase of the study. To a great extent, the nature of the discussion section in your study will depend on the results you obtain.

The Delphi Method

As explained in your text the Delphi method is a survey technique that involves the use of questionnaires in an attempt to get consensus on a topic. Subjects respond to a first questionnaire, then based on these responses a second questionnaire is developed and administered. Each time the questionnaire is administered is called a "round." Suppose for example, you were interested in investigating the impact of the 1980 Soviet boycott of the Olympic Games. Using the Delphi method you could survey knowledgeable Russians and Americans, each time sharing the different perspectives obtained, in an attempt to identify critical issues and perspectives.

Personal Interviews

Personal interviews are similar to questionnaires except in the manner in which they are administered. Some of the advantages and disadvantages (or challenges) to this method are indicated below:

Advantages Disadvantages/Challenges
Greater confidentiality possible because of personal contact Fewer subjects can be sampled
Flexibility to give follow-up questions More expensive because of travel or phone
Opportunity to clarify questions Need to be able to take notes quickly or get permission to tape
Can judge adequacy (honesty?) of replies Need to be able to listen to one reply and be ready to follow-up immediately with the next question
Higher return rate Requires skilled interviewer

As noted in the text, conducting an interview effectively requires practice. You will notice from watching TV interviews how the interviewer usually tries to control the pace of the interview. Some people will talk for a long time but say very little if permitted to do so. Another problem is that some people tend to give an opinion when they really are not very knowledgeable about the topic. With practice the researcher quickly identifies whether a subject truly is qualified to respond to a particular question.

Normative Surveys

Although it is highly unlikely that you as graduate students would be involved in normative types of surveys, you should be aware that they have been widely used in physical education and health. Two sets of norms often used in public school physical education are the President’s Physical Fitness Challenge and AAHPERD’s Physical Best. Both consist of fitness data collected through nationwide normative surveys that teachers use to compare the fitness levels of children in their schools. In these types of surveys, the intent of the researchers is typically to establish performance norms to which the performance of others can be compared. As you can imagine this endeavor is often the subject of criticism by those who claim that scores taken in one environment lack validity when applied to a different environment.

If you are considering research involving questionnaires or interviews be sure to carefully examine the many aspects involved in the successful application of these types of research methods. An excellent resource for further reading is a book entitled Research Processes in Physical Education, by Clark and Clark.

Developmental Research

Some years ago a researcher at the University of Oregon named H. Harrison Clarke initiated a growth study in the Oregon town of Medford. This project spurned many research papers, professional presentations, and graduate theses and dissertations. For several years researchers would visit Medford and track the growth and development of children in the public schools. This project provides an example of a longitudinal study - in other words a study of the same subjects over a period of several years. Were these same researchers to conduct a cross-sectional study we might anticipate that they would have visited Medford on one occasion and taken growth measurements of different subjects at several grade levels.

Developmental studies (despite the limitations noted in the text) can provide some fascinating insights and although only descriptive in nature can spark the researcher’s curiosity for more controlled experimental studies.

Case Studies

Some years ago a graduate student at our university was interested in examining the topic of teenage pregnancy. Although she could have researched the topic with a questionnaire she decided to use a case-study approach. The advantage of this method was that the topic could be examined in-depth, albeit gathering information from a much more limited sample of subjects than would have been possible with a questionnaire. In contrast to the questionnaire approach - which requires the researcher to have excellent knowledge of the topic when designing questions - case study researchers often approach their subjects with an inquisitive mind and an openness that permits subjects to respond in an unlimited number of directions. As you can imagine, this less structured approach may take researchers down avenues they did not anticipate traveling and open doors to new kinds of understanding.

In terms of the types of case study identified in your text, in the example given above the graduate student was probably conducting an interpretive study. Certainly, however she included a great deal of description. Were she to have approached the topic with the intent of identifying better ways of preventing teenage pregnancies then she would probably have conducted more of an evaluative study.

In summary, case studies tend to provide in-depth information about a limited number of subjects, and may produce new insights that generate additional studies.

 

Observational Research

A topic of great interest to those of us in teacher education has been in identifying those factors that contribute to effective teaching. Research on effective teaching has been through several stages. Initially, it was believed that the most effective teachers had special personality characteristics. Later it was suggested that the key to effective teaching might lie in the methods used.

Research in this area has often involved the observation of teachers and the categorization of behaviors. In observing the behaviors of teachers in classrooms in which learning is apparently occurring, certain commonalties have been identified.

The researchers who have investigated this area have engaged themselves in a type of observational research. Instead of approaching the question of effective teaching by asking questions, they chose to observe the behaviors of teachers. As noted in your text observational research necessitates adherence to certain guidelines in order to be considered as valid and reliable. As some of you know, here at CWU we use several of these observational methods (e.g. videotaped lesson analysis) in our undergraduate preparation of PE majors.

Correlational Research

Although we discussed the techniques of correlational research in our discussion about statistics, it is important to appreciate that correlational research is descriptive. Because there is no manipulation of variables or controls, in correlational research is impossible to conclude that something "caused" something to occur. Remember that correlations are indicators of a relationship and not an effect.

(Revised 2/10/99)


BACK TO TOP OF PAGE

BACK TO PEHL 557 CLASS NOTES


Page constructed by Stephen C. Jefferies

jefferis@cwu.edu