How can you use language to avoid bias in a questionnaire?

I would recommend adopting a mindset of thinking about every question from the perspective of the participant. "What is your highest level of education?"...how would I feel if I didn't have a "high" education and so how then would I be inclined to answer this question? How old are you?... What is your job status?"... How much do you earn? "Old" , "status" , "much" could all be interpreted as pejorative.

Use straightforward language that is easy to understand. Complex or technical terms can confuse respondents or lead to misinterpretation. If using scales (like Likert scales), ensure consistency in how they are presented. Changing the scale direction or meaning partway through can introduce bias.

Once you have drafted your survey questions, consider using a grade-level checker to be sure your audience will readily understand the words and phrases you use. In many industries the common jargon we use becomes so familiar internally that we forget external audiences may not know those terms! Avoid abbreviations, industry jargon, text-speak, and local usage idioms - a quick readability checker can help you optimize your survey. And don't forget the value of internal subject matter experts' review and pilot testing with real people!

Large language models, such as Chat GPT, can help test for bias in your question. You can prompt them to act as a certain persona e.g. "Act as a sceptical customer who regularly buys..." Then you can refine your question by getting feedback and reprasing it appropriately to your audience.

This is really common. We often see questions starting with "Do you think..." or "Do you agree that..." which are most likely leading questions. Instead, try to use more "open" phrases like: -"What do you think about..." -"Which of the following best describes..." -"How.." so you ask them in an unbiased way. Instead of asking "Do you agree that..." ask "How much do you agree or disagree with the following statement?..." Instead of "How happy were you with your recent experience?" ask "How was your recent experience?" etc. Sounds obvious but when we translate business questions into survey questions we need to remember to phrase them in the most unbiased, clear, and neutral way possible.

Be cautious about including color-coded scales. Consider examples of Net Promoter Scales that color-code detractors (0-6) in red, passives (7-8) as yellow, and promoters (9-10) in green. Some would argue that these clarify the classifications for the respondent's use; others might argue that the prompts lead the respondent to an answer you want. Whichever camp you fall into, make using or not using color coding an intentional choice.

Be as consistent as possible with scaling, respondants get used to one set and then switch to another. For the Likert always include labels at each level, treat it a bit like a hybrid of multiple choice selection and scale rating. Customize where it is needed to get clear directions on numbers.

Recognize that the order of your scale is important too. In the US, we read left to right and down the page - so offering negative scale options to the left (or top in a drop-down box) is what the respondent sees first. At the end of the day, if potential presentation order bias is an issue in the question, I'd rather understate the outcome.

I agree that pilot testing is essential, but how far to take the process is something to think through for each project. For smaller simple studies, a quick review by SMEs not directly involved in the project may be sufficient. For huge or complex jobs it may make sense to take the pilot through small sample data collection/soft launch, and preliminary analysis. That's when you may find the great "ah ha" moment, discovering some aspect of planned reporting you may have missed - or some unintended consequence of process not considered in design. A good rule of thumb might be "the bigger or more complex the project, the more critical the pilot test."

One great way to do this is to conduct pilot tests. Pre-testing of surveys helps to ensure that target respondents think all survey questions and attributes are clear and well phrased. If there is an issue with the design, it often can be checked and evaluated during this step.

Keeping the survey brief is often a challenge. When working with tracking studies you really want to keep the core content consistent over time - so what do you do with other "then current" questions clients want to add? One solution is to tag them on the end with an intentional limit of perhaps three questions. When you have a lot of extra questions being requested - and when you have a large enough sample - you might split your sample and ask a few items to each split. You might also ask some of the extra questions periodically (in waves 1, 3, and 5, for instance) - and other questions in the alternating waves. Or just say no! Regardless, let's keep the surveys short for our participants.