DIY Survey Risks and Rewards
With tools like SurveyMonkey and Google Forms readily available, it's easy to understand why many associations consider conducting customer and member satisfaction surveys themselves. While the DIY approach might seem tempting for its convenience and cost savings, it's crucial to understand the balance between potential benefits and hidden pitfalls.
The benefits of DIY surveys
Let's start by acknowledging why DIY surveys appeal to many associations:
Cost-effectiveness – Low-cost or free online survey tools make gathering feedback financially accessible, especially for smaller or budget-conscious associations.
Flexibility and speed – Conducting surveys internally allows you to quickly roll out questionnaires, adjust them in real-time, and rapidly access results. This agility can significantly streamline decision-making processes.
Direct member engagement – When your members see surveys coming directly from your organisation, they may feel more valued and heard, potentially strengthening relationships and boosting loyalty.
Hidden risks of going solo
Despite these attractive benefits, DIY surveys can carry considerable risks, particularly around bias, data quality, and interpretation- which could inadvertently undermine your strategy.
Confirmation Bias – Associations naturally risk framing questions that reinforce existing beliefs. Without an independent perspective, DIY surveys often produce biased results aligned with preconceived expectations rather than revealing a true snapshot of member sentiment.
Poorly constructed questions – Crafting effective surveys requires professional expertise. Ambiguous phrasing, leading questions, or complex "double-barrelled" queries (addressing two points simultaneously) can severely distort responses, leading to unreliable and confusing data.
Sampling and response bias – Convenience sampling, common in DIY surveys, often results in feedback from only the most engaged or accessible respondents. This leaves dissatisfied or less engaged segments underrepresented, painting an inaccurately positive picture of your associations’ satisfaction levels.
Social desirability bias – Respondents might provide answers they believe the association expects, rather than their genuine opinions, especially when the survey is directly tied to the association’s leadership.
Misinterpretation of data – Analysing and interpreting survey data accurately is a specialised skill. Without statistical expertise, associations risk drawing incorrect conclusions or missing critical insights, ultimately impairing strategic decision-making.
Why independent research agencies add value
Engaging an independent research provider mitigates these risks by offering:
Neutral survey design – Objective question framing eliminates confirmation bias and ensures responses genuinely reflect your audience’s views.
Balanced and representative sampling – Professional research approaches ensure broader respondent engagement, avoiding skewed data that comes from convenience sampling.
Enhanced data credibility – Members typically feel freer to offer honest feedback to an independent third party, providing richer, more genuine insights.
Expert data analysis – Statistical expertise and professional analysis turn raw data into meaningful, actionable insights that support informed strategic decisions.
Finding the right balance
DIY surveys have their place, particularly for quick, informal checks or minor internal feedback loops. However, when member satisfaction is central to your strategy, leveraging professional research expertise ensures your data is robust, credible, and truly valuable.
At Survey Matters, we're dedicated to supporting associations with reliable, actionable insights to drive confident decisions. If accurate, insightful data matters to your organisation, we're here to help.