Hobbies And Interests
Phrasing
The phrasing of a question can have large effects on the answer. For example, people have more negative responses about beef that is 20-percent fat than they do about beef that is 80-percent lean. They give inconsistent answers when asked their age than when asked their birth date and year (the latter tends to be more accurate). In a CNN poll, Hillary Rodham Clinton did worse than Hillary Clinton. And, when asked if speeches against democracy should be allowed, 75 percent of those surveyed said they should be allowed, but at the same time, 54 percent thought they should be forbidden.
Response Choice
People give inconsistent responses when asked open-ended questions versus questions with fixed choices. People are also more likely to choose "moderate" positions than extreme ones, regardless of how those positions are defined. For example, when asked about how much TV they watch in a typical day, responses differed when 2.5 hours was defined as "low" versus when it was defined as "high." Similar effects occur for questions about sexual behavior, alcohol use and other touchy subjects.
Order of Questions
People give inconsistent responses when the same question is asked following different sets of questions. People express higher satisfaction with life when general questions about satisfaction are asked before specific questions than when the specific questions are asked first. They also interpret these questions differently. People who are shopping for a product express different opinions about the price of a product when it is preceded by more expensive products than when cheaper products come first.
Summary
Designing a study is one of those subjects that seems simple, but is not. There are many traps to watch out for, and, if you are not expert in the design of surveys, you may get misleading responses.