Research: Designing questionnaires (continued)
Creating your questions
You must make sure that questions are clear, unambiguous, and useful. The wording is fundamental to both the validity and reliability of any study, so always pilot your questions first. Keep the following points in mind:- Keep questions simple and short.
- Don’t use vague terms. Avoid ambiguity. Be precise.
- Avoid ‘loaded’ or ‘leading’ questions that hint at the answer you want to hear. e.g. Do you think that hard-working nurses should be forced to take on this thankless task? It's hard to answer 'yes' to this one!
- Avoid ‘double-barrelled’ questions, asking more than one question. e.g.What do you think of the design and content of this teaching pack? The design might be brilliant, but the content could be awful.
- Avoid ‘double-negative’ questions as they can confuse people e.g. Is it ever unacceptable to not use this medicine? Whether you answer 'yes' or 'no', the meaning isn't really clear.
- Avoid ‘presumption’ questions: do not assume that everyone practises at the same level or has the same standards. Don’t use jargon or abbreviations that some people may not know.
- Watch out for prestige bias: even if responses are anonymous, respondents may not want to be portrayed in a bad light e.g. The pharmacist’s role is to protect the patient, but have you ever made a serious error?
1. Two-way question
Here there are only two alternatives: yes/no, good/bad, for/against and so on e.g. Do you think the pharmacy on-call service should be accessible to doctors only? Answer: yes or no.
2. Likert scale
You provide your respondent with a scale typically comprising 5 or 6 points, and ask them to rate their experience somewhere within it. You may say something like: What did you think of our training? Rate our service on a scale of 1 to 6 where 1 is poor and 6 is excellent.
Alternatively, you may offer a selection of ranked statements to choose from, rather than numbers: Excellent – Good – Satisfactory – Below average – Poor. Be wary of unhelpful labels such as asking respondents to choose between: Vitally important – very important – important – unimportant – totally unimportant. This is simply a two-way question transferred into a 5 point scale.
Whether to offer a 'don’t know' category is an issue of some controversy when using Likert scales: should you omit it and force respondents to make a choice from the other options?
3. Ranking questions
A set of items is provided and the respondent is asked to list them in order of preference, importance, merit, etc., or to pick their 'top three'. No more than ten options should be offered. Remember that ranking does not tell you anything about the distance between the ranks: i.e. by how much respondents prefer their first option over their second one.
Here there are only two alternatives: yes/no, good/bad, for/against and so on e.g. Do you think the pharmacy on-call service should be accessible to doctors only? Answer: yes or no.
You provide your respondent with a scale typically comprising 5 or 6 points, and ask them to rate their experience somewhere within it. You may say something like: What did you think of our training? Rate our service on a scale of 1 to 6 where 1 is poor and 6 is excellent.
Alternatively, you may offer a selection of ranked statements to choose from, rather than numbers: Excellent – Good – Satisfactory – Below average – Poor. Be wary of unhelpful labels such as asking respondents to choose between: Vitally important – very important – important – unimportant – totally unimportant. This is simply a two-way question transferred into a 5 point scale.
Whether to offer a 'don’t know' category is an issue of some controversy when using Likert scales: should you omit it and force respondents to make a choice from the other options?
3. Ranking questions
A set of items is provided and the respondent is asked to list them in order of preference, importance, merit, etc., or to pick their 'top three'. No more than ten options should be offered. Remember that ranking does not tell you anything about the distance between the ranks: i.e. by how much respondents prefer their first option over their second one.
Presenting your questionnaire
Good design can increase your response rate. Most people send surveys out electronically using a resource like Survey Monkey or Smart Survey; there are many to choose from. They have the advantage of providing a professional presentation as well as doing some of the analysis for you. However, even with online methods like these, there are a variety of ways to improve the success of a questionnaire:- Explain what you’re asking for and why; thank people for giving their time; give a clear deadline for responses.
- Generally, start with broad, straightforward questions and include more complicated, specific or sensitive ones later.
- The questions should proceed in a logical manner.
- Make sure instructions are clear, and that the method of answering is obvious.
- Offer options such as 'don’t know' or 'not applicable' if they are likely to apply to at least some of your respondents.
- Vary the question format to add interest.
- Think about issuing a reminder to complete the questionnaire before your deadline.
Piloting the questionnaire
It is absolutely crucial to pilot your questionnaire. You will want to test how long it takes to complete the questionnaire, check that all questions and instructions are clear, and try to expose any items that will not generate usable data. In practice it is not always possible to pilot on your real audience. If so, try the questionnaire out on friends or colleagues. You could ask them:- How long did it take to complete?
- Were the instructions clear?
- Were any questions ambiguous, difficult to answer, or objectionable?
- Was the layout clear and easy to follow?
- Were any questions repetitive or unnecessary?
- Did you think any questions were missing?
PAGE 7 OF 11. NEXT PAGE ›