Having a solid hypothesis before you run a test is essential. Without it you don’t know what you are trying to prove or disprove, and you risk wasting valuable time and resource.
A hypothesis helps to clarify why you are running a test and aligns your team on a clear objective.
There are three questions you should ask yourself when developing a test idea:
- What is the problem you are trying to solve?
- What is your proposed solution?
- What is your predicted outcome?
If you can’t answer these questions then you don’t have a test worth running. If you can answer these questions then you have your hypothesis.
Problem
Most test ideas evolve off the back of data. For example, you might spot that step 2 in your sign up funnel has a significant drop off rate.
Solution
You can see what is happening but you might not necessarily know why. It’s time to get creative and come up with some assumptions about why the problem is occurring. Armed with these assumptions you can create some solutions that you think will fix the issue.
For example, I think users are dropping off from step 2 in the sign in funnel because we are asking for their date of birth and the request for personal data is putting them off from continuing. We should remove this mandatory field.
Outcome
What’s the expected outcome of the test? In this example, I predict by making the changes above we will increase the sign in rate.
I can now structure this thought process into a written hypothesis for everyone to see:
“60% of users are dropping out of step 2 of the sign in funnel. This is because we are asking for a date of birth, which is deterring people. By removing this mandatory field we will remove this barrier and increase sign in rate”
This hypothesis has even given me a success metric to measure the outcome of my test – sign in rate.
I spend alot of time preaching to my colleagues about this approach. I even have it on the wall above my desk for the whole office to see:
Early on in my testing days I made some epic mistakes. Running tests without a solid hypothesis was one of them, so I encourage you to take the right approach in order to maximise your ab testing efforts.
The featured image was Designed by Freepik
Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.
1,840 thoughts on “How To Form A Hypothesis for A/B Testing”
Comments are closed.