How to Write Unbiased Survey Questions

Imagine asking a room of people, “Don’t you agree that our fantastic new café is a great improvement?” Most will nod. You have learned nothing except that people are polite. The wording of a question nudges answers more than most authors realise, which is why learning to write unbiased survey questions is the single most valuable skill in survey design. Below are the common kinds of bias, with before-and-after examples you can adapt.

Why wording matters so much

Respondents do not read questions in a vacuum. They look for clues about what the author wants, they interpret ambiguous words in their own way, and they try to be agreeable. Small changes in wording can shift results noticeably, and you will never see the damage in the data itself, because biased answers look just as tidy as honest ones.

The aim is not perfect neutrality, which is impossible, but to remove obvious pushes and make questions mean the same thing to everyone.

Type 1: Leading questions

A leading question suggests the “right” answer.

Biased: “How excellent was your experience with our friendly team?”
Better: “How would you rate your experience with our team?”

Watch for adjectives and adverbs that carry judgement, such as “excellent”, “helpful”, “unfortunately” and “obviously”. Strip them out. If the thing being rated needs describing, describe it in plain facts.

Type 2: Loaded questions

A loaded question contains an assumption the respondent may not share.

Biased: “Where do you usually park when you drive to our shop?”
Better: “How do you usually travel to our shop?” with options including walking, bus, cycling, driving and other.

The first version assumes everyone drives. Check each question for hidden premises. Where a premise applies only to some people, use a screening question first, then ask follow-ups only of those it applies to.

Type 3: Double-barrelled questions

These ask two things in one.

Biased: “How satisfied are you with the price and quality of our products?”
Better: Ask about price and quality separately.

A person who loves the quality but thinks the price is high cannot answer honestly. The quick test is to look for the word “and”. Not every “and” is a problem, but each one deserves a second look.

Type 4: Absolute wording

Words such as “always”, “never”, “all” and “every” force extremes. Very few people always do anything.

Biased: “Do you always read the terms and conditions?”
Better: “How often do you read the terms and conditions before agreeing?” with options such as never, rarely, sometimes, often and every time.

Frequency scales with defined labels give a more accurate and honest picture.

Type 5: Vague or ambiguous terms

Words like “regularly”, “recently” and “affordable” mean different things to different people.

Vague: “Do you shop with us regularly?”
Clearer: “In the past three months, how many times have you shopped with us?”

Anchor questions to a time frame, a number or a clear example. If a term is technical, define it in a short note or choose a simpler word.

Type 6: Unbalanced answer options

Bias hides in the answers as much as the question.

Unbalanced: Excellent, very good, good, fair.
Balanced: Very good, good, neither good nor poor, poor, very poor.

An imbalance of positive over negative options tilts results. Make sure scales have an equal number of positive and negative points, with a clear midpoint if a neutral view is possible.

Other scale points to check:

• Label the ends at least, and ideally every point, so that people interpret numbers consistently.
• Keep direction consistent. If 1 means “strongly disagree” in one question, do not make it mean “strongly agree” later.
• Include “don’t know” or “not applicable” where a respondent might genuinely have no view.
• Keep lists complete. Add “other (please specify)” to multiple-choice questions where your list may not cover everything.

Type 7: Order effects

The sequence of questions and answers affects results. A question about a specific complaint immediately before a general satisfaction rating will lower the rating. Ask general questions first, specific ones later. For lists of options, consider randomising the order where your tool supports it, so no item benefits from being first. Keep ordinal scales such as “never to always” in logical order, since randomising them would confuse people.

Type 8: Social desirability

People tend to give answers that make them look good, particularly on topics like health habits, finances, recycling or voting. You cannot eliminate this, but you can soften it:

• Use anonymous surveys where possible, and say so truthfully.
• Phrase questions in a normalising way. “Some people find it hard to remember to take medication. How often does this happen to you?” is gentler than “Do you forget your medication?”
• Offer a “prefer not to say” option.
• Keep sensitive questions towards the end.

A simple revision process

1. Draft your questions quickly without worrying about polish.
2. Mark each one with the checks above: Leading? Loaded? Two questions in one? Absolute? Vague?
3. Rewrite flagged questions in plain, neutral language.
4. Read each aloud. Awkward phrasing is easy to hear.
5. Ask a colleague to explain, in their own words, what each question is asking. Differences in interpretation reveal ambiguity.
6. Pilot with a few real respondents and ask them where they hesitated.

This takes an hour or two, and it is the cheapest quality control you will ever do.

Worked example

Suppose a local library wants to know how people feel about its new opening hours.

Original: “Do you agree that our convenient new opening hours are better for everyone?”

Problems: it leads (“convenient”), it is loaded (assumes better), it uses an absolute (“everyone”), and it limits the answer to agreement.

Revised set:
1. “Which of these times do you usually visit the library?” (list of time bands, plus “I don’t visit”)
2. “Compared with the previous opening hours, how well do the new hours suit you?” (much better, slightly better, no different, slightly worse, much worse, I did not use the library before)
3. “What, if anything, would you change about the opening hours?” (open text, optional)

The revised version asks about behaviour first, uses a balanced scale and invites criticism.

When bias still creeps in

Even careful wording cannot fix who answers. If only enthusiastic users reply, your results will lean positive no matter how neutral the questions. State your recruitment method when reporting, and avoid overclaiming. If the decision is important, think about reaching less engaged people by other means.

How to Design a Short Survey People Will Finish

Frequently asked questions

Can a question ever be completely neutral?

Not entirely, but you can remove obvious cues and ambiguity. The aim is that different people read the same question in the same way and feel free to answer honestly.

Is a neutral midpoint always a good idea?

Usually, when a neutral view is plausible. Removing it forces people to choose a side. If you do include it, label it clearly.

Should I use agree/disagree scales?

They are common but prone to acquiescence, where people lean towards agreeing. Where possible, ask direct questions with answer labels that describe the topic, such as “very easy” to “very difficult”.

How many people should pilot the survey?

Even three to five people from your audience will uncover most wording problems.

Conclusion

To write unbiased survey questions, remove loaded and leading words, ask one thing at a time, define vague terms and balance every scale. Then test on real people. Neutral wording will not fix every flaw in a survey, but it protects the one thing you cannot recover afterwards: honest answers.