A lot of organizations automatically opt for the cheapest and easiest method, but it’s more important to consider which tool will help you get the information and insights you want to get out of your survey.

Response rates and the size of your membership are two of the key factors to consider when choosing your survey tool. In a recent 123Signup webinar with Matt Braun of Loyalty Research, he estimated that response rates for phone interviews are the highest (50-60%), paper surveys tend to get a response rate of around 20%, and web-based surveys get the lowest response rates of 8-12%.

A web-based survey sent to 10,000 people with a response rate of 10% gives you 1000 responses – a big enough sample size to give you good insights on your members’ attitudes and behaviors. But sending the same survey to 100 people would result in only 10 responses, and that’s just not enough to give you reliable, actionable data.

(Read The Right Questions to Ask on Your Next Membership Survey for ideas on putting together an effective questionnaire.)

For smaller organizations looking to get more detailed information, a phone survey may well be the way to go – there’s more to think about before choosing your methodology. Here are some of the pros and cons of each survey tool to help you choose the right method for your next research project.

Paper Surveys

Pros:

• Easy to design as a Word document.
• Can be easy to collect by mail or if handed out at an event.

Cons:

• Entering data from the back end can be time consuming and result in error.
• People can write in answers that are difficult to interpret and categorize.
• May get a lower response rate among younger members.

Phone Surveys

Pros:

• Information quality is the best of all the methods because it captures the true voice of the member. Phone interviewers have the ability to probe respondents and get them to expand on their answers so you get more in-depth answers and insights.
• Better participation than paper or web-based surveys, and a good choice for smaller memberships.

Cons: 

• The most difficult method to get right.
• Cost is high as it may require that you higher an external company to help to design and implement it.
• It takes longer to collect information.
• Can be difficult to reach people at a convenient time for a 10 to 20 minute survey.

Web Surveys

Pros:

• Highest value for the cost.
• Inexpensive applications make it easy to design and collect information.
• Can be distributed through various channels, including email, social media, and on your website.
• Many applications offer basic analysis and reporting tools to help you get insights from your data.
• Gives every member with access to the web the ability to participate.
• Allows for longer surveys, and give respondents the ability to save and continue.
• Gives you the ability to assign a unique identifier to each respondent for tracking purposes.

Cons:

• Have lower response rates than paper or phone surveys, which may not be ideal for smaller organizations.