Step 8: Testing Your Materials

In this page

Refine Your Approach Through Audience Testing

You've worked hard to create your tobacco education materials and you want them to be successful. The best way to find out whether your materials will achieve your goals is by testing them with your key audiences at various stages in your project's development. You should also be aware that the California Tobacco Control Program requires testing of all tobacco education materials.

Conduct an audience test during the early concept development stage; test again when you have a solid draft; and, finally, test again when you have a close-to-final piece. Expect to make revisions after each test. The more times you test and revise, the better your final product will fit with your target audience.

Constructing a Test: What Do You Want to Learn From Your Audience?

Use this guide to devise a script or questionnaire of open-ended questions for your test audience.

  • Attention: What are the first things you notice? What is your first reaction? Which concepts or parts appeal to you the most?
  • Comprehension: What do you think is the main message? What does [this word] mean to you? What parts are difficult to understand?
  • Relevance: Who do you think is the intended audience? In what ways are the materials relevant to you? In what ways are the language and images culturally relevant, or not?
  • Outcome: What do the materials accomplish? What do you think this is asking you to do [or consider]? What action should you take as a result of using these materials?
  • Strengths and weaknesses: Which parts accomplish the intended outcome best? Which parts are less helpful, or not helpful at all?
  • Sensitivity: Are there any parts that make you uncomfortable? Do the jokes or humor work as intended? Are colloquialisms clear?
  • Other possible questions: Who do you think is the source of the material and what do you think of the source? Where would be the best place to see or get these materials? What are some alternative ways the materials could be designed or delivered?

TEAM Lab Pretesting Questions (Sample)

Finding Test Subjects

You'll want to approach two groups for testing your materials: content experts and your intended audience.

Content experts

Content experts include those who have knowledge and experience with the content or topic, the audience, legal and policy issues, organizational issues and communication issues. To find content experts, check with your agency or make contact with experts in your field who you've worked with before. You can also ask TEAM Lab for the names of experts. Content experts sometimes will suggest adding a lot of information. As you take their suggestions into account, remember that it's usually best to keep your materials as simple as possible and stick with just one to three key messages.

Intended audience

Next test your materials with consumers, or your intended audience. You could create a volunteer advisory group of people from your target audience or find individuals as needed. You'll need people who are truly representative in terms of education, income, cultural background and risk factors for the problem you are tackling. Go to organizations, clinics, stores, churches or community events that draw your target population. Ask sites and organizations for permission to talk to their patrons or members. Then ask individuals a brief, initial set of screener questions to make sure they represent key characteristics of your target audience. Then test your idea or draft with your chosen subjects.

How to Pretest

Prepare an early creative brief, mock-up or close-to-final draft of your materials, depending on the stage of development you're testing. Ensure that it’s laid out in an easy-to-read way and is grammatically correct. Find people who match the characteristics of your target audience, and make sure they have not seen the materials before. Then give them adequate, unpressured time to review your materials. Ask them to be honest with you. Here are some effective settings for pretesting:

Individual interviews

Intercept individuals at a site or event and arrange a short time for them to review your materials and respond to your questions verbally or in a questionnaire. Make sure you have a table and chairs or clipboards, and that they have adequate time to review the materials. You could also conduct your assessment by phone after an individual has had time to review the materials at home.

Focus groups

  • Present the materials in an interactive group of six to eight target audience members in a quiet place, for no more than an hour.
  • Provide each individual with a copy or have a large screen or poster that everyone can see.
  • Prepare a script of eight to 15 open-ended questions to ask your subjects (that is, don't ask questions that require an answer of simply "yes" or "no").
  • Give people time to collect their thoughts and ideas and respond out loud to each question.
  • Have a moderator guide the discussion and have someone else record responses or take good notes. If possible, have others, especially your creative staff, observe the groups to spot issues that need to be changed in the materials or in future pretesting.

Materials Review Score Card

This is TEAM Lab's modified version of Suitability Assessment of Materials.
Materials Review Score Card (Excel)
Materials Review Score Card (Word)

Theater testing

Pass out the sample materials to a group of consumers gathered in a classroom. Give them time to review the materials, then answer a paper questionnaire about what got their attention, comprehension, etc.

How to Pilot Test

Pilot tests are used for testing your final drafts before printing in large quantities. You can use the same methods you used for the pretests described above, having consumers review a final draft of the materials individually or in groups. Or you can conduct a field test or dry run. In a field test, you distribute the materials in much the way they will be distributed in the real world, and then interview the first 10 to 20 people who get them, asking questions similar to the ones listed above for the pretests.

At this point, it is vitally important to learn whether your main objectives for the materials have been achieved. For example, if the main objective is to change attitudes or prompt actions or teach skills, then you should assess how well the outcome may have been achieved after the user sees the materials.

Making Changes

After each test, from the early concept pretesting to field tests, you will find things that need to be changed. Keep good notes of comments and suggestions from all of your experts and consumers and decide which ones seem valid. If you hear the same critique from two or more consumers, you need to seriously consider making that change. Don't forget to test again after the change.

Even once you've completed, printed and distributed your materials, consider periodically testing them to see whether they are still accomplishing what you intended.

Additional Resources

We Can Help

If you need additional guidance about any aspect of testing your materials with your audiences, TEAM Lab is here to help. Contact us.

Learn How

Resources

Evaluation

Materials Development Tips