Mary works for a pharmaceutical company that has just tested the effectiveness of a new over the counter medicine to help people with heartburn.
There have been extensive tests on the safety of the medication and all show that the medication is virtually 100% safe with only very minor problems reported in one of every 10,000 uses. Mary’s team is testing the “taste” of the medicine in an effort to help the advertising company promote the new medication.
The taste test was done on a scale of 1-5 with respondents selecting the number that corresponded to their choice of word anchors. The question was “How do you like the taste of this medication?” Here are the responses and the % of respondents that answered to each:
1 – Liked it a lot 5%
2 – Liked it a little 5%
3 – No Opinion 60%
4 – Did not like it a little 20%
5 – Did not like it a lot 10%
Bill, a member of the advertising team, wants to make the statement, “after extensive taste testing, fully 70% of the people who tested the medicine said nothing bad about the taste”. Conversely, Mary says that that might be false advertising since only 10% of the people actually had anything good to say about the taste.
What is your feeling about this issue?
At first glance of the issue, we can say that there is really nothing wrong with Bill’s statement that “70% of the people who tested the medicine said nothing bad about the taste”. Based of the breakdown of responses, 5% of the individuals liked the taste a lot, 5% liked it a little, whereas 60% had no opinion with the taste of the medication.
In this manner, 10% of individuals can be said to have given a clear response as they expressed their perception that they liked the taste in varying degrees. Furthermore, though the 60% may seem to have “unclear” response, they still did not say anything bad about it, which would coincide to Bill’s statement or assumption.
On the other hand, Mary also had a point when she expressed her skeptical position about Bill’s statement. This is because of the fact that the 60% did not have a clear opinion on what really is their stand about the taste, whether the taste of the medication was good or bad in their respective opinion.
Nevertheless, according to the literature, the “don’t know” or “no opinion” entry of responses in a questionnaire or evaluation material are best discouraged (Martin, 2006). This is because it can bring about meaningless responses, thereby resulting in loss of data. Hence, the “don’t know” or “no opinion” responses do not contribute to improving the quality and reliability of the gathered data (Martin, 2006, p. 8).
Consequently, through careful and objective analysis, I felt that Mary’s argument that it might just be a false advertising was reasonable due to the small percentage of actual positive responses. For this reason, the quality and reliability of Bill’s assumption may be compromised because of the fact that 60% had “no opinion” about the taste, which can be equated as almost meaningless responses.
Thus, “when given the opportunity, any respondents who take the easy out by saying ‘don’t know’ or ‘no opinion’ are capable of providing meaningful and valid responses” (Martin, 2006, p. 8), thereby increasing the accuracy and reliability of the assumption and interpretation of the responses.
Martin, E. (2006). [Research Report Series (Survey Methodology #2006-13)] Survey Questionnaire Construction. U.S. Census Bureau – Director’s Office. Retrieved June 20, 2009, from http://www.census.gov/srd/papers/pdf/rsm2006-13.pdf