I'm often asked what's the best way to get feedback from customers and it's really about finding the sweet spot between these three variables:
- signal quality
Here's how you do it.
Why, not just what or if
I think we all agree that getting feedback from prospective customers is the best way to ensure we don't waste time and money building something nobody wants.
I want however to re-iterate one important thing:
learning that something works is much less useful than learning why something works.
Remember, the definition of a startup is a learning organization looking for a repeatable business model.
If you are early stages you are not trying to make money, you are trying to figure out how to make money on a consistent basis. That is your objective.
So figuring out that your ideal customer, let's call her Jenny, would be willing to buy your product, is actually not as interesting as learning how Jenny thinks about your product and why she's buying it over the competition (which includes whatever she's doing today instead of using your thing).
Only when you understand your customer's thinking process you have a good shot at building copy for your website and marketing material that will resonate consistently with that market segment, retain customers, work on the most important features, etc.
The right tool for the job
Imagine you briefly meet someone for the first time at an event and exchange contacts. You have their phone number, their email, you connected on Facebook and learned that they work in the same office building.
Two weeks later something comes up where you think it might be worthwhile to have a conversation with that person. You now have 5 options:
- you can call them on the phone
- you can text them
- you can message them on Facebook
- you can email them
- you can meet them
Granted that it depends on the context (age, job, request, etc), most people will send a message to arrange for a coffee meeting or similar. It comes natural. Why? because it's human. We crave the face to face contact, it's more personal, it allows you to better understand the situation and the conversation feels richer.
It happened to all of us to exchange emails or being on the phone and struggling to figure out if the other person is paying attention and what's really going through their minds.
So it blows my mind when, having talked to hundreds of founders, face to face interviews are almost everybody's last choice.
There's actually an obvious reason for that, one that's at the root of most startup's failure, but it's out of scope so I'll deal with it in another post.
Face to face is not necessarily always the best approach, but it's important to be conscious when we decide what tool to use for the job.
The 3 key criteria + 1
There are 3 variables that you want to consider when planning interviews:
- signal quality
Thinking about the above example and assuming you want to talk with 10 people you met at an event, this is how you can analyze your options.
If you are meeting them for coffee that's gonna cost you 20 coffees (assuming you're always having one yourself and paying for them) and at a least 45 mins (30 mins together plus commute). In our example example that would total to ~$60 and 7.5hrs.
Also, it may be that people flew in for the event and you cannot physically meet them or that you don't have 7.5 hrs to put toward this.
It's Monday and you need to make a decision by Friday. You want to get these 10 people's opinion, but they are busy folks so when you reach out you figure out that half of them won't be able to meet before next week.
On the other hand they could all get on the phone with you in their evenings or during lunch break. Arranging and doing phone calls is much faster than organizing a face to face meeting.
Literally and metaphorically.
I bet you've been at the end of a noisy phone line or lousy internet connection, having problems to make out half of the words the other person is saying. Nothing is clearer than sitting in front of someone else.
Whether you buy into it or not, there's also a substantial amount of research supporting some mind blowing numbers regarding non verbal communication:
Professor Mehrabian combined the statistical results of the two studies and came up with the now famous—and famously misused—rule that communication is only 7 percent verbal and 93 percent non-verbal. The non-verbal component was made up of body language (55 percent) and tone of voice (38 percent).
If that's even remotely true any time you're not sitting in front of the person you're talking to you're missing half of information. If you're using text mediums like email you're giving up 93% of it. N-i-n-e-t-y t-h-r-e-e.
Think about you experience. Compare email conversations with chats over coffee break.
If you are trying to understand the reasons why someone is choosing, or not, your product, you better make sure you have as much signal bandwidth available to you as possible.
Qualitative Vs Quantitative
At different stages of testing your idea the validation needs change. A good rule of thumb is to first validate qualitatively on small numbers and then quantitatively on large numbers.
It's important to keep this in mind as some tools are harder to scale than others - for sure you need to watch how many cups of coffee you drink in a week if you want to avoid a heart attack.
Let's look at the tools
Armed with a goal and a set of criteria we can make a solid decision of what tools to use, but what are the options to choose from? Sorted by signal quality here they are:
Face to face
If you are a functioning member of the human race this should likely always be your first option for qualitative validation. Nothing beats the signal quality of f2f interviews and done well they outweigh their higher cost and lower speed.
Also, as Roger Cauvin reminded me in the comments, they are the ideal tool to discover unknown unknowns. By asking broad questions it's possible to learn completely unexpected things about your subject leading to breakthrough insights.
The biggest issue is to get the other party to agree to it since the logistics and time commitment are often a problem, especially if you are in B2B or dealing with busy people. Another issue is if you live in a region where you don't have access to your target market, but that's often a signal of bigger problems.
And it obviously won't work for quantitative validation, it's just too costly.
Easier to set up, they are still pretty good in terms of signal quality as you can hear and see the person.
Depending where they are connecting from you are potentially competing with a larger number of distractions, but overall you can still get visual and audio cues.
The biggest risk is quality of link, a lot of video calls are still an issue for me and I'm often having to turn off video to have good voice quality.
Phone occupies a very special place in the toolbox. While the temptation is to think of it like video conferencing minus the video part, the reality is much different.
First of all a lot of people comfortable with voice aren't comfortable with video, so a phone call is lowering the barrier in that sense. Likewise everybody has a phone and it's in general easier for people to hop on a call, maybe even from their cars while commuting (done quite a few of those myself).
The combination of ease of use, easier scheduling and lower entry barrier makes it a very appealing option, but don't forget that you're losing signal and without video the chance of being multitasking and distracted go up tenfold.
If you recall the numbers from earlier, here you just went down from 45% down to 7%. you've lost tone of voice, body language, everything, you just have a bunch of words.
Good luck with that.
But at least you have the realtime component so if you realize someone hasn't understood he question or something isn't clear you can tackle it right away.
Non-realtime chat (social channels, forums)
Public forums, groups and similar outlets can be useful to reach larger groups. Again signal goes down, but speed and cost is really low.
A few important things to keep in mind:
- you can very easily run into group think and individuals conforming to the opinion of the group
- forum admins don't look positively to people trying to take the conversation out of forums and most definitely don't like newcomers contributing nothing and just asking others to fill out a survey
- outreach of this kind works best when you've been a long time member of that community and contributed to it in the past
Initially I would avoid this approach, but it can become handy during a transition from qualitative to quantitative feedback.
You think I'm kidding, I'm not. I believe you'd get more value, if anything in terms of attention through novelty/weirdness, by sending a message written in very tiny characters on a mangled piece of paper carried by a pigeon than to interview someone via email.
Stats from Mailchimp suggest only a small percentage of people actually allow images that make the tracking pixel work so it's safe to say that you're unlikely to get an idea if your email was even opened most of the times, let alone read.
You also have no idea of the situation in which it was read, what amount of attention was given to it to even understand the content, let alone think through the answer.
For busy people, and most of us are busy, it can take up to a week or more to respond so it might be fast for you to send them out, but it can take a long time to see anything back. And good chances are the response will ask for clarifications or miss the point, leading to more emails and more time.
Oh boy, surveys are such a thing that they deserve their own macro section (see below)
Surveys, the only 3 good uses for it
If the world was about to end and surveys were the only way leaders could communicate to find a solution we'd be screwed.
Surveys are the worst and yet most popular way for founders to get feedback on their idea. Nine out of ten of the early stages entrepreneurs that call me for help, when I ask them if they got any feedback, will tell me two things:
- I asked my friends if they liked my idea
- I ran a survey on Facebook/Linkedin/Reddit/whatver
The thing is, you can't blame anybody, surveys are damn appealing: takes a minute to create one, you get the perception of potentially getting feedback from thousands of people and most important of all you can do it from home without putting yourself out there and risking to be in the uncomfortable situation of sitting in front of someone who just told you your idea sucks.
It's a winner no doubt.
Unfortunately it's worse than just having lost 93% of the signal, having no clue if somebody is going to fill it out or not and with what level of attention, but, you're also committing the cardinal sin of priming your subjects by giving them multiple options questions that you wrote with your own understanding of the problem.
And if you use a lot of open questions to avoid that your response rate will plummet too the point that doing a survey has very little value.
But not all is bad.
Earlier we talked about qualitative Vs quantitative validation. If you have done a number of interviews and have a very solid understand of how your customers think and talk about the problem, it's perfectly ok, and in fact recommended, that you try to validate that quantitatively. And surveys are perfect for that.
Another case where surveys shine is screening. If you are trying to find people to interview there's a number of approaches and if you've casted a rather wide net to find people surveys are perfect to sift through them so that only the right ones you want to talk to show up at your door.
The last way a survey can be successfully leveraged is as an artificial barrier. Say you have a service for which you're trying to gauge interest. You have some feedback and want to open for sign-ups but realize that just leaving an email doesn't mean much. At the same time you're not ready or want people to pay. This is a good place to create some artificial difficulties and see if potential users are willing to invest their time to overcome them. A survey can be just right here, allowing you to learn something interesting while validating commitment.
What's best for your startup?
It might not be still obvious which way to go or how to exactly implement some of those solutions so if you'd like to chat about it just go ahead and book a call, I'd love to work on it together.