Creating a testing culture in an organisation that is unfamiliar with the concept is not easy, especially in large companies. However, when you do eventually make an impact, which can be extremely rewarding, it can begin to snowball. And by this I mean lots of people getting wind of the benefits of testing and wanting a piece of the pie. It is fantastic to reach this point, but it can come with its own problems.

Depending on how you’re structured, very quickly you can have a queue of business owners wanting to run tests on their areas of the product. If you and your team are in charge of testing, then it is your responsibility to maintain law and order. It’s all well and good implementing a shiny new testing tool, but your job should not just be about becoming the gatekeeper for this tool — it is so much more than that.

It is very easy to get carried away and just launch a test when another one has finished, because you have a Business Owner bugging you, asking “when will my test be prioritised”. Before you know it, you have run a multitude of tests without any time to breath before or afterwards. This may not be the most effective way for you to run your testing program. Here is a few points to be aware of:

Testing Culture Maintenance

Build a roadmap

You hear this alot in the world of AB testing and MVT, but I think it is easy to get carried away and forget the importance of careful planning and maintenance. Ensure you have a roadmap on a single document or wiki page, which gives you a clear overview of all the tests you want to run over the next quarter or half year. This plan will not stay the same, it will evolve and grow as you start testing.

If you don’t maintain a roadmap, then before you know it, there will be no strategy and you will just be answering ad hoc requests from around the business.

Prioritise

Once you have built a roadmap with lots of test ideas, you then need to prioritise them. I find that a good way to do this is to rank your ideas in order of desirability and feasibility. It might be that the easier tests with quick, small wins is where you want to start. Or maybe you want to run with a test that has huge desirability and low feasibility, and you have the allocated resource to develop the experiences and launch the test. This is individual to you and your business, but the point is, you must prioritise your tests to maintain law and order

Iterate

Testing can raise as many questions as it answers, so It is not wise to just analyse the data, form a conclusion and move onto the next test. It may be that you don’t have a winning experience but there could still be some valuable insights you can take from the test in order to build out an iterative test, and help find answers. Or you might have a test result which is close to reaching statistical significance and needs a bit more exploring.

This would not be possible if you just jumped onto the next test because a Business Owner is banging on your door

Create traffic groups

When a user enters a test, the last thing you want to do is contaminate the data by allowing them into another test that’s running at the same time. This is where traffic groups are extremely necessary. All 3rd party testing tools have their own terminology and ways of creating traffic groups, but the outcome is the same — creating rules in the UI that says only this group of users are allowed to enter this test.

Providing your site has the traffic volumes; traffic groups allow you to run multiple tests at the same time. If you have enough traffic to the site, you could allocate a certain percentage of your traffic to those adhoc test requests. For example, a redesign that’s due to be pushed live (and needs to be tested) tomorrow, that nobody has communicated to you. This happens alot in some organisations, especially large ones. It’s difficult to get a handle on everything that is going on across the business, but ultimately the testing should become part of the design process, so it can be built into the roadmap and scheduled as best as possible. However, for when that doesn’t happen, having a dedicated traffic group could prove useful.

Hypothesise

When teams get the testing bug and they start chucking tests they’d like to run at you, they often forgot to hypothesise or they are unaware of the importance of forming a solid hypothesis before running an experiment. This is where strategy and best practices really need to be communicated internally. It might even be necessary to run a training session to explain why we need a solid hypothesis and how we can form our hypothesis.

In Conclusion

Ultimately, it is your responsibility to structure your plan so you can either accommodate those ad hoc “emergency” requests or have the processes in place for prioritising them amongst your other planned tests. The key to success is to maintain a fluid plan and ensure all areas of the business it impacts, are aware of it and your processes. You need to take charge, so you can be strategic about your test and learn programme. This is only going to benefit your team and the business in the long run.

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

118 thoughts on “How to Maintain A Testing Culture

  1. Pingback: viagra walmart
  2. Pingback: cialis coupons
  3. Pingback: cialis for sale
  4. Pingback: viagra 50mg
  5. Pingback: men's ed pills
  6. Pingback: cialis generic
  7. Pingback: online pharmacy
  8. Pingback: Buy cialis online
  9. Pingback: vardenafil generic
  10. Pingback: levitra generic
  11. Pingback: generic vardenafil
  12. Pingback: online casinos
  13. Pingback: installment loans
  14. Pingback: cash loan
  15. Pingback: viagra 100mg
  16. Pingback: buy cialis
  17. Pingback: cialis 5 mg
  18. Pingback: 20 cialis
  19. Pingback: cialis buy
  20. Pingback: generic for cialis
  21. Pingback: cialis 5 mg
  22. Pingback: gambling games
  23. Pingback: online slots
  24. Pingback: purchase viagra
  25. Pingback: viagra price
  26. Pingback: viagra online
  27. Pingback: tadalafil generic
  28. Pingback: tadalafil 10 mg
  29. Pingback: cheap viagra
  30. Pingback: real casino online
  31. Pingback: viagra pill
  32. Pingback: viagra generic
  33. Pingback: cialis prices
  34. Pingback: buy Viagra 120mg
  35. Pingback: buy generic viagra
  36. Pingback: Cialis 80 mg otc
  37. Pingback: Cialis 40mg pills
  38. Pingback: order Cialis 20 mg
  39. Pingback: online viagra
  40. Pingback: Cialis 10 mg nz
  41. Pingback: Cialis 80mg cost
  42. Pingback: cialis pills
  43. Pingback: Cialis 60mg prices
  44. Pingback: Cialis 60 mg pills
  45. Pingback: cialis online
  46. Pingback: levitra 40mg cheap
  47. Pingback: lasix 40mg tablets
  48. Pingback: buy generic viagra
  49. Pingback: viagra sale
  50. Pingback: allegra 120mg otc
  51. Pingback: amaryl 4mg canada
  52. Pingback: sildenafil
  53. Pingback: arava 20 mg otc

Comments are closed.