A weekend in the Bush?
The banner photo above shows view of Australian landscape from our holiday lodge. The Retreat Bushland Lodge
Home » Resources » Articles » Ten Deadly Sins of Surveying

Ten Deadly Sins of Surveying

  1. Using survey results for bonus or balanced scorecard decisions
When employees know their survey scores will influence their (or their manager’s) bonus or balanced scorecard results, they often artificially raise their ratings. We’ve seen exceptionally high satisfaction scores totally inconsistent with the employees’ ratings of other survey factors.
 
  1. Using too few items for measurement to be sufficiently robust
You should use at least 3-4 items for each construct you wish to measure. Using one item is not sufficient to provide satisfactory results, as some respondents can interpret individual items in different ways.
 
  1. Not including all the main factors that may prove to be important
If you wish to find the key correlates of satisfaction you can’t get an accurate picture if you omit possibly key variables (e.g. senior executives’ communication).
 
  1. Keeping the survey too short – you don’t get a comprehensive survey
There is no evidence that people will not complete long surveys – they will if the content is relevant to them and layout is clear – if you miss key areas you won’t get the “complete” picture. (Research shows that the response rate is most influenced by not communicating findings, or taking action, or explaining why action is not taken.)
 
  1. Changing response scales unnecessarily
This confuses and annoys respondents – if you can, it’s best to use a five point Likert scale (strongly agree, agree, neutral, disagree, strongly disagree) with all items expressed positively.
 
  1. Asking double-barrelled questions (or worse)
Each item should have one proposition only – otherwise the results will be unclear. This is a very common error made in surveys.
 
  1. Conducting the survey electronically when not all have access
In most organisations there will be some employees who you will disenfranchise if you conduct the survey in a way that misses them.
 
  1. Inadequate sample – too small to provide an adequate confidence level
Generally surveyors aim to get a 95% confidence level (i.e. you’re 95% sure that you would get the same result if you surveyed everyone in the population). If you get 400 surveys back you will achieve this.
 
  1. Not doing a “key driver” or correlational analysis
You will miss a key advantage of conducting a survey if you don’t find out the pattern of statistical relationships – exactly what factors are driving satisfaction or whatever other criteria (e.g. retention) you wish to understand.
 
  1. Not benchmarking your findings against appropriate norms
If you don’t know how your results compare (e.g. to Australian or other country norms) you won’t know if you have a relatively good or poor score in regard to various factors.
 
Rodney Gray
November 2005

Published in Journal of Employee Communication Management, July/August 2006.