Ten commonly made mistakes in surveys

A survey is one of the ways to involve stakeholders and or experts in a project. The CEPA toolkit was developed with the input of more than a hundred experts worldwide through web based surveys, telephonic interviews and expert meetings. Gillian interviewed me what works and what does not work in involving large networks. The result is the posting Time, Technology and Tangibility on her blog. For the more practical aspects I remember that I used an old checklist from my organization. Here is a ‘brushed up’ version of the ten commonly made mistakes in surveys:

1. Working in isolation – trying to have results first before engaging a wider community, instead of asking right from the start the largest possible group for ideas, feedback and advice and use that as a staring point.
2. Forgetting to offer clear benefits for participation and providing information. We ask their time, knowledge and experience – can we offer them credits, exposure, access to new information or other benefits?
3. Not making the invitation to participate as personal as possible – investing time in personal emails, especially to those people we know. Using intermediaries – respondents know - as senders also helps.
4. Forgetting to give timely feedback to great individual contributions – a personal email to thank people who have made an extraordinary effort and or ask them for a phone interview to get more details and examples.
5. Forgetting to give timely feedback to the whole group of respondents on the results of the survey. This causes the decline of interest and less response in a next round.
6. Not keeping the pressure up – it is better to give a rather short lead time to respond and to send halfway that period a reminder.
7. Not testing the questions first in a small group. The questions have to be concise, clear, not too many and fun to do. This to avoid the feeling that responding is actually a waste of time or that important aspects of culture, language or gender are overlooked.
8. Having too many open questions – this can be avoided by having a qualitative survey first and formulate multiple choice questions based on that survey.
9. Not having a simple procedure for processing and interpreting the answers – the design of the survey should make it easy to see the overall results. Proper time management is the basis of the time consuming task of interpretation.
10. Not making a proper project planning – involving all survey project team members - for all the phases of the survey from design to evaluation. A (joint) planning helps to save time and reduce risks, especially when you forget to involve secretariat support staff.