Consulting Journal: Methods - Using questionnaires

Methods: Using questionnaires

by David Blakey

Questionnaires can be very useful as a means of getting answers to a set of closed questions. Here's how.

[Monday 25 June 2001]

There are several ways of getting information from various numbers of people throughout a client organization. One of these is the questionnaire. It is a good method of getting the answers to a set of closed answers from a large group of people.

Questionnaires seem to be used by a great number of organizations these days. You can be presented with a questionnaire by a university web site, in a store, or even - as happened to me this morning - in a public swimming pool. People are familiar with filling in questionnaires and selecting one answer out of a number or checking all the boxes that apply. They do it on questionnaires and on web sites, regularly.

Questionnaires do have their faults.

Questionnaires can be tiresome

As a result of this, people may view questionnaires as a nuisance. They may have filled in so many that another one is simply tiresome, if not annoying. As a consultant, managing the questionnaire, this must concern you. If people are not giving the questions their full and interested attention, then they may not be giving you the most appropriate answers.

This is probably not an issue on web sites. If a site about model railways is asking you questions about your preferences, then you are probably visiting that site because you have an interest in model railways. As a result, the questionnaire may interest you. It may even cause you to think of ideas. As an example, you may have visited the model railway site because you wanted information about building a model railway in your house. The questionnaire may have asked if you preferred indor or garden railways. As a result, you might reconsider your plan and start thinking about building a model railway in your garden. So questionnaires on web sites are not usually tiresome for the people who visit those sites.
Questionnaires are unfocused

Questionnaires can not only be tiresome if they are of little interest to the person answering the questions but they can also miss important points.

As an example of this second point, consider some of the questionnaires about ‘customer service’ that are asked by supermarkets. Because the questions are closed, they may not be relevant to the individuals being asked. One favourite used to be asking mothers of young children if the changing facilities - the ‘mother and baby room’ - were poor, satisfactory, good or excellent. Fathers were not asked about the ‘father and baby room’, usually because there wasn't one. Fathers also were not asked if they felt the need for one. So the questionnaire could make the store owner believe that everyone who needed to clean and change babies was happy.
Questionnaires can be irrelevant

Then, as well as being uninteresting and unfocused, questionnaires can be seen as irrelevant. Many people, on being presented with a questionnaire, imagine that its main purpose is to reassure the organization asking the questions that all is well. They may also imagine that, if the answers are not what the organization wants to hear, then nothing will happen to set things straight. If a supermarket does not get a ‘thumbs up’ from its customers, it may ignore the questionnaire results entirely.

Advice to consultants

Here are the conditions under which I use a questionnaire.

I cannot interview everyone personally.

This happens when there are more people than I and my team can interview individually in the time available (which is often dictated by the client's budget). On some assignments, this can be as few as five people. On other assignments, it needs to be at least twenty.
I cannot run workshops.

This happens when the information needed may provoke arguments. The aim of a workshop is to build a consensus view, with some dissension being acceptable over minor points. If the subject is one that is unlikely to have a consensus view, then a workshop should be avoided. Another reason is that not enough people may be available for a workshop at a given time. A third reason is that a workshop will generally occupy more time that individual interviews, for each individual concerned. Although workshops may save the consultant's time, they will probably take more of the client's time.

To me, building a questionnaire is an option that I choose if my two primary options - mass interviews and workshops - and impractical.

Here are the things that I build into the questionnaire.

Context

The respondents must be told what the purpose of the questionnaire is, and what its limitations and constraints are. If the questionnaire is designed for you to gain information, then state this clearly in a briefing paper attached to the questionnaire. If you do not consider that any immediate changes will result from the questionnaire responses, then you should state this, clearly and unequivocably. Do not raise expectations unnecessarily.
Relevance

The respondents must also be aware of the relevance of the questionnaire. As well as providing the context of the questionnaire, you should describe its relevance. So, if the questionnaire asks for information about current projects to improve operational effectiveness, the context might be that you - the consultant - needed an inventory of all such ongoing projects, and the relevance might be that your assignment is to prioritize them. The respondents can then see ‘what's in it for them’.
Focus

Your questionnaire should be checked - preferably by a member of the client's staff - to ensure that your questions focus on the real issues and that you are not asking questions that have only one possible answer. If you were asked to rate yourself as a consultant, on a scale from poor to excellent, you really should be in some other occupation of your answer is other than ‘excellent’.

Also, following on from this, there is no point in asking ‘trick’ questions. Asking people to rate their performance against customer expectations on a scale from 0% to 110% is not fair. There may be a belief that a 110% performance against expectations is a good thing, whereas, in economic terms, it may be worse than an 80% performance. You should not use the questionnaire for your own self-aggrandizement, however excellent a consultant you believe yourself to be.
Interest

If the briefing for the questionnaire does not provide sufficient interest, then you have got a problem. The respondents should believe that its intelligent focus and its relevance are sufficient for them to give it serious attention. If these messages do not reach them, then you should consider first whether the questionnaire is necessary and second, if it is, how the briefing can be improved.

One final point about the design of questionnaires. You should avoid asking for additional detail within the questionnaire. Keep to checklists as much as possible. You can ask whether the respondent would like to provide additional details or documents on specific topics. Then you can use their answers to set up individual or group interviews with particular people about particular topics. Whenever someone volunteers additional infomation, you should always reply to them, and tell them whether or not you will meet them or ask for that information.

[ List articles on Methods ] [ View printable version ]