Standard Lesson

And the Question Is... Evaluating the Validity of a Survey

And the Question Is... Evaluating the Validity of a Survey
Grades
9 - 12
Lesson Plan Type
Standard Lesson
Estimated Time
Three 60-minute sessions
Author
Publisher
ILA
  • Preview
    • |
  • Standards
    • |
  • Resources & Preparation
    • |
  • Instructional Plan
    • |
  • Related Resources
    • |
  • Comments

Overview

Surveys are an important tool when doing research and learning to evaluate information. In this lesson, students consider the purpose and meaning of surveys, learn what types of questions are asked, evaluate the validity of a specific survey, and write in their journals to reflect on what they have learned.

From Theory to Practice

Measuring the acquisition of media literacy skills (Hobbs)

A study of eleventh-grade students showed instruction in critical media analysis improved students' abilities to identify main ideas in print, audio, and visual media messages as well as information omitted from a news broadcast. Participants were also more likely to identify a message's intended audience, construction techniques, point of view, and intended purpose than those students who did not receive the instruction.

Bradburn, N.M., & Sudman, S. (1988). Polls and surveys: Understanding what they tell us. San Francisco: Jossey-Bass.

  • Researchers should recognize proper and improper uses of surveys.

  • Critical thinkers understand how the wording of questions can affect responses.

  • Readers should analyze what the answers mean in survey results.

Common Core Standards

This resource has been aligned to the Common Core State Standards for states in which they have been adopted. If a state does not appear in the drop-down, CCSS alignments are forthcoming.

State Standards

This lesson has been aligned to standards in the following states. If a state does not appear in the drop-down, standard alignments are not currently available for that state.

NCTE/IRA National Standards for the English Language Arts

  • 3. Students apply a wide range of strategies to comprehend, interpret, evaluate, and appreciate texts. They draw on their prior experience, their interactions with other readers and writers, their knowledge of word meaning and of other texts, their word identification strategies, and their understanding of textual features (e.g., sound-letter correspondence, sentence structure, context, graphics).
  • 8. Students use a variety of technological and information resources (e.g., libraries, databases, computer networks, video) to gather and synthesize information and to create and communicate knowledge.
  • 11. Students participate as knowledgeable, reflective, creative, and critical members of a variety of literacy communities.

Materials and Technology

  • Computers with Internet access

  • Overhead projector and transparencies, LCD projector, or a whiteboard (optional)

Printouts

Websites

Preparation

1. Decide whether to use a whiteboard, LCD, or overhead projector for prompts, and prepare the following questions for Session 1:

  • What is the purpose and meaning of a survey?

  • Do you think that surveys are a valid method of gathering information?

  • What kinds of questions are used in surveys?

  • Have you ever participated in a survey?

  • Why do you think it is important to evaluate surveys?
2. If you do not have classroom computers with Internet access, arrange to use the school's computer lab for Sessions 1 and 2 (and optionally for Session 3).
3. Access and familiarize yourself with Encyclopedia of Educational Technology: Types of Survey Questions, Survey How To, and Indian Women & Gender Roles. Make sure you read the general description of a survey on the Survey How To website. Bookmark the sites on the computers students will be using. Print off the Indian Women & Gender Roles survey and make one copy for every three or four students in your class.
4. Review the Survey Evaluation Form and the explanations of terms. Make one copy of this form for every two students in your class.
5. You might choose to review (and possibly teach) the lessons "Exploring Literacy in Cyberspace" and "Identifying and Understanding the Fallacies Used in Advertising" to prepare for this lesson.

Student Objectives

Students will

  • Explore the meaning and purpose of surveys in writing and through discussions

  • Improve comprehension of how surveys work by learning what types of questions are asked when conducting surveys

  • Determine the validity of a survey and its conclusions by reading, discussing, and analyzing the results

  • Improve skills in critical thinking, respectful dialogue, and reflective writing

Session 1: Understanding Surveys

1. Ask students to respond in their journals to the prompt questions (see Preparation, Step 1). After a few minutes, invite students to share their answers with the class.
2. Have students go to the Encyclopedia of Educational Technology: Types of Survey Questions website to learn about surveys and the types of questions asked. Allow time for them to read and digest this information.
3. Lead a class discussion, asking students to share what they have learned from the reading. Ask students to explain how the information on the website agreed with or differed from the question responses they wrote in their journals. For example:

  • What is the purpose and meaning of a survey?

  • What kinds of questions are used in surveys?

  • Have you learned any new information from the reading?
4. Have students get into small groups of no more than four. (Note: You can let students choose their own groups, although you want them to be as heterogeneous as possible and may need to intervene accordingly.) Ask each group to visit the Indian Women & Gender Roles survey website.
5. Ask students in each group to read the Introduction, Data/Methods, and Questions of the survey. Ask them to think about the following while reading:

  • What is the purpose of the survey? How do you know?

  • What type of questions were asked?

Tell students that, if necessary, they can return to the Encyclopedia of Educational Technology: Types of Survey Questions. Encourage students to refer to the chart outlining which types of questions work best for particular research goals.
6. Ask the groups to share their findings in a whole-class discussion. Points to consider include the purpose of the survey and the types of questions asked.


Homework: Students should record their new findings in their journals. Journal entries might consider the following:

  • What I knew before this session

  • What I learned from this session

Session 2: Survey Result Analysis

1. Have students return to the groups from Session 1. Ask each group to visit the Survey How To website. This site gives a very thorough explanation of survey methods. 
2. Distribute copies of the survey students started analyzing in Session 1 and post the following questions on the board or overhead projector:

  • How was the data analyzed?

  • What is the survey telling you?

  • Do you think the types of questions asked were appropriate for this kind of survey? Why? Why not?

  • Would you have added or eliminated questions? Which ones?
Ask students to discuss these questions in their small groups.
3. Circulate among the groups and answer any questions students might have. Make sure students understand that charts were the format used for data comparison.

Session 3: Survey Validity

1. On the board, write the word validity and ask students what they think it means. Then allow them to look up the definition in the dictionary. Discuss.
2. Ask students how, in their opinions, validity applies in the case of surveys. How do we determine the validity of a survey? Let students discuss this question freely in their small groups. If necessary, provide the answer to the question: Validity is the degree to which a study accurately reflects or assesses the specific idea that the researcher is measuring.
3. In terms of the particular survey students are analyzing, ask them to consider whether it really assesses Indian women and gender roles. Have students work in pairs to analyze the purpose, subjects, questions, conclusions, and validity of the survey using the Survey Evaluation Form. Allow a few minutes for this activity, and then ask students to share their opinions with the class.
4. As a concluding activity, lead a whole-class discussion about the importance of media literacy, using the following prompts for the discussion:

  • Why is it important to use statistics such as survey data?

  • What bias might affect how the data is gathered, analyzed, or presented? Think about the survey you have just analyzed.

  • Next time you analyze a survey, how will you know whether the survey is valid?

  • What elements will you consider in this analysis?
5. Ask students to reflect in their journals on what they think they have learned from this lesson and how it might affect their own research in the future.

Extensions

  • Conduct the companion lesson "And the Question Is... Writing Good Survey Questions" to enable students to continue examining effective survey questions and to create their own survey to administer to classmates.

  • To reinforce real-life application of the knowledge and skills attained in this lesson, have students analyze other surveys. Some examples are available from The Pew Research Center for the People and the Press.

  • Ask students to analyze a news broadcast (perhaps using an online transcript) to determine what could have been added or eliminated to present the information differently. Also, what kind of survey information might have contributed to more information for the broadcast?

Student Assessment / Reflections

  • Monitor students’ participation in class and group discussions.

  • Review students’ journals and the Survey Evaluation Form to ensure that they can identify the purpose, types of questions, and validity of a survey. Were students able to critically evaluate a survey? Did they understand the terminology?

  • Assess what students learned using the Survey Analysis Rubric.