Many are talking about growth and product analytics, but how many well established companies are leaving behind their old ways of design by committee? In this post I want to highlight a common scenario and promote the modern approach to product development.

Once upon a time…

I received a heads-up from a UX team about an AB test they wanted to run for a new page they had designed. My first reaction was “great, you don’t want to just roll out a new page, you want to make sure it has the desired outcome.”

So what was the desired outcome? By digging a bit deeper, I discovered that…erm, there wasn’t one. The “KPIs” had been plucked out of thin air and they were not relevant to the page in question.

For example, one of the success metrics for the test was bounce rate. This suggests there was a high bounce rate to begin with, which in-turn prompted the redesign. But this page is rarely an entry page, so why were the team redesigning a page to reduce bounce rate?

It occurred to me that the team had no idea why they had created this new page and had no clear objectives for the redesign. They were trying to solve a problem that didn’t exist…or did it?

The page in question, was the top of a funnel. By digging into the data, we discovered that the drop off rate from the top of the funnel to the bottom was significant.

A website funnel

Funnel

Ok, so now we have a problem to solve. And what a great place to start optimising — at the top!

So why were certain decisions made for the redesign?

In my experience this example of product development is not uncommon. Its the difference between having HIPPO (highest paid person’s opinion) led decisions, and data led decisions.

It is a deeply routed cultural issue in many organisations, where areas of the business lose sight of why they are creating something, especially when it involves jumping into a web analytics tool they don’t know how to use, and putting opinions (and sometimes egos) to one side, by listening to the data.

I’m sure if I asked you to post some thing on Facebook that will receive lots of ‘likes’, you’d know what to post. This is because you have learned over time what resonates with your friends, by looking at the numbers. Ok, this is a very basic example, but my point is data is now so deeply engrained in our every day lives — sometimes without even realising it.

The New Approach to Product Development

This is why multi disciplined growth teams (designers, analysts, developers…) are now being formed across the tech industry — to fulfil a desperate need to understand audiences and make smarter decisions.

The importance of a correct web analytics implementation and test & learn function cannot be understated. Without them, product teams are just shooting into the wind, in the hope that something sticks, without ever really knowing if it does in-fact stick.

Going back to my original example, of course there is nothing wrong with testing a new design just because you feel a particular page needs a refresh, as long as every one remains focused on why they are changing/updating certain elements of the experience and what metric(s) they are trying to shift.

Maximise the impact of your changes

If you launch a test where you have changed multiple elements on a page, without any prior iterative testing, then the chances are you won’t see the positive results you’d hoped for. And if you are fortunate enough to see an uplift you will have no way of knowing which changes have been the drivers.

It’s a great headline to share with the business: “We tested this new shiny page and we saw an increase of 2% in sign ins”. High fives all round… But how do you know which element of the new design drove this uplift? Was it the new title/new CTA on the button/larger fields/more emotive images…? These are all valuable questions you could and should have answers to.

Your web analytics data gives you insights into how your users are behaving on your site or app. Are they behaving in the way you intended? If not, then you have identified a problem. Make some assumptions and test them.

The product development testing cycle

Continuous experimentation cycle

So if your team wishes to refresh a page / product development, break down the various elements you think will impact your specific KPI for that page, and test them individually. Slowly you will gather valuable insights that will feed into your redesign as well as future changes. With a more iterative approach you are likely to see even bigger wins.

Finally

Whether you’re a designer, developer, analyst or marketer, we are all striving to see our companies succeed.

In order to succeed we need sustainable growth. To achieve sustainable growth we need to understand our audience. To understand our audience we need data.

Key takeouts for upstream product development testing

  • Know why you are redesigning a page / an experience — what problem are you trying to solve?
  • Have data to back up your new design
  • Test whatever you want to test, but know what you are testing and why
  • For gathering insights, break the elements of the page down into bite size pieces
  • Make assumptions with a sound hypothesis
  • Treat your testing program like any other analytics tool. Use it in conjunction with your web analytics, to find answers quickly, to important questions. Testing tools are effectively web analytics tools

For further reading see my infographic 7 Steps for Effective A/B Testing

Feature image Designed by Freepik

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

85 thoughts on “How AB Testing and Product Development Work Together

  1. Pingback: Maria Smith
  2. Pingback: Melanie Bowen
  3. Pingback: buy cialis
  4. Pingback: cialis pills
  5. Pingback: viagra pills
  6. Pingback: viagra
  7. Pingback: cialis
  8. Pingback: viagra online
  9. Pingback: cialis online
  10. Pingback: buy cialis pills
  11. Pingback: cialis black
  12. Pingback: cialis from canada
  13. Pingback: cialis 20mg
  14. Pingback: viagra from india
  15. Pingback: buy cialis canada
  16. Pingback: cialis coupon
  17. Pingback: viagra 100mg
  18. Pingback: viagra for sale
  19. Pingback: best ed pills
  20. Pingback: otc ed pills
  21. Pingback: cialis 20 mg
  22. Pingback: rx pharmacy
  23. Pingback: pharmacy online
  24. Pingback: cialis mastercard
  25. Pingback: cialis mastercard
  26. Pingback: levitra online
  27. Pingback: levitra canada
  28. Pingback: slot games
  29. Pingback: order viagra
  30. Pingback: online casino
  31. Pingback: online cialis
  32. Pingback: personal loans
  33. Pingback: instant loans
  34. Pingback: casino city
  35. Pingback: viagra 100mg
  36. Pingback: cialis buy
  37. Pingback: 5 mg cialis
  38. Pingback: cialis generic
  39. Pingback: generic for cialis
  40. Pingback: 20 cialis
  41. Pingback: viagra online
  42. Pingback: free slots online
  43. Pingback: casino real money
  44. Pingback: viagra reviews
  45. Pingback: buy viagra
  46. Pingback: buy cialis online
  47. Pingback: canadian viagra
  48. Pingback: buy viagra pills
  49. Pingback: cialis tadalafil
  50. Pingback: mens ed pills
  51. Pingback: buy viagra cheap
  52. Pingback: viagra
  53. Pingback: cialis coupon
  54. Pingback: generic cialis
  55. Pingback: casino slots
  56. Pingback: online slots
  57. Pingback: viagra online usa

Comments are closed.