Creating a testing culture in an organisation that is unfamiliar with the concept is not easy, especially in large companies. However, when you do eventually make an impact, which can be extremely rewarding, it can begin to snowball. And by this I mean lots of people getting wind of the benefits of testing and wanting a piece of the pie. It is fantastic to reach this point, but it can come with its own problems.

Depending on how you’re structured, very quickly you can have a queue of business owners wanting to run tests on their areas of the product. If you and your team are in charge of testing, then it is your responsibility to maintain law and order. It’s all well and good implementing a shiny new testing tool, but your job should not just be about becoming the gatekeeper for this tool — it is so much more than that.

It is very easy to get carried away and just launch a test when another one has finished, because you have a Business Owner bugging you, asking “when will my test be prioritised”. Before you know it, you have run a multitude of tests without any time to breath before or afterwards. This may not be the most effective way for you to run your testing program. Here is a few points to be aware of:

Testing Culture Maintenance

Build a roadmap

You hear this alot in the world of AB testing and MVT, but I think it is easy to get carried away and forget the importance of careful planning and maintenance. Ensure you have a roadmap on a single document or wiki page, which gives you a clear overview of all the tests you want to run over the next quarter or half year. This plan will not stay the same, it will evolve and grow as you start testing.

If you don’t maintain a roadmap, then before you know it, there will be no strategy and you will just be answering ad hoc requests from around the business.

Prioritise

Once you have built a roadmap with lots of test ideas, you then need to prioritise them. I find that a good way to do this is to rank your ideas in order of desirability and feasibility. It might be that the easier tests with quick, small wins is where you want to start. Or maybe you want to run with a test that has huge desirability and low feasibility, and you have the allocated resource to develop the experiences and launch the test. This is individual to you and your business, but the point is, you must prioritise your tests to maintain law and order

Iterate

Testing can raise as many questions as it answers, so It is not wise to just analyse the data, form a conclusion and move onto the next test. It may be that you don’t have a winning experience but there could still be some valuable insights you can take from the test in order to build out an iterative test, and help find answers. Or you might have a test result which is close to reaching statistical significance and needs a bit more exploring.

This would not be possible if you just jumped onto the next test because a Business Owner is banging on your door

Create traffic groups

When a user enters a test, the last thing you want to do is contaminate the data by allowing them into another test that’s running at the same time. This is where traffic groups are extremely necessary. All 3rd party testing tools have their own terminology and ways of creating traffic groups, but the outcome is the same — creating rules in the UI that says only this group of users are allowed to enter this test.

Providing your site has the traffic volumes; traffic groups allow you to run multiple tests at the same time. If you have enough traffic to the site, you could allocate a certain percentage of your traffic to those adhoc test requests. For example, a redesign that’s due to be pushed live (and needs to be tested) tomorrow, that nobody has communicated to you. This happens alot in some organisations, especially large ones. It’s difficult to get a handle on everything that is going on across the business, but ultimately the testing should become part of the design process, so it can be built into the roadmap and scheduled as best as possible. However, for when that doesn’t happen, having a dedicated traffic group could prove useful.

Hypothesise

When teams get the testing bug and they start chucking tests they’d like to run at you, they often forgot to hypothesise or they are unaware of the importance of forming a solid hypothesis before running an experiment. This is where strategy and best practices really need to be communicated internally. It might even be necessary to run a training session to explain why we need a solid hypothesis and how we can form our hypothesis.

In Conclusion

Ultimately, it is your responsibility to structure your plan so you can either accommodate those ad hoc “emergency” requests or have the processes in place for prioritising them amongst your other planned tests. The key to success is to maintain a fluid plan and ensure all areas of the business it impacts, are aware of it and your processes. You need to take charge, so you can be strategic about your test and learn programme. This is only going to benefit your team and the business in the long run.

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

445 thoughts on “How to Maintain A Testing Culture

  1. Pingback: viagra walmart
  2. Pingback: cialis coupons
  3. Pingback: cialis for sale
  4. Pingback: viagra 50mg
  5. Pingback: men's ed pills
  6. Pingback: cialis generic
  7. Pingback: online pharmacy
  8. Pingback: Buy cialis online
  9. Pingback: vardenafil generic
  10. Pingback: levitra generic
  11. Pingback: generic vardenafil
  12. Pingback: online casinos
  13. Pingback: installment loans
  14. Pingback: cash loan
  15. Pingback: viagra 100mg
  16. Pingback: buy cialis
  17. Pingback: cialis 5 mg
  18. Pingback: 20 cialis
  19. Pingback: cialis buy
  20. Pingback: generic for cialis
  21. Pingback: cialis 5 mg
  22. Pingback: gambling games
  23. Pingback: online slots
  24. Pingback: purchase viagra
  25. Pingback: viagra price
  26. Pingback: viagra online
  27. Pingback: tadalafil generic
  28. Pingback: tadalafil 10 mg
  29. Pingback: cheap viagra
  30. Pingback: real casino online
  31. Pingback: viagra pill
  32. Pingback: viagra generic
  33. Pingback: cialis prices
  34. Pingback: buy Viagra 120mg
  35. Pingback: buy generic viagra
  36. Pingback: Cialis 80 mg otc
  37. Pingback: Cialis 40mg pills
  38. Pingback: order Cialis 20 mg
  39. Pingback: online viagra
  40. Pingback: Cialis 10 mg nz
  41. Pingback: Cialis 80mg cost
  42. Pingback: cialis pills
  43. Pingback: Cialis 60mg prices
  44. Pingback: Cialis 60 mg pills
  45. Pingback: cialis online
  46. Pingback: levitra 40mg cheap
  47. Pingback: lasix 40mg tablets
  48. Pingback: buy generic viagra
  49. Pingback: viagra sale
  50. Pingback: allegra 120mg otc
  51. Pingback: amaryl 4mg canada
  52. Pingback: sildenafil
  53. Pingback: arava 20 mg otc
  54. Pingback: cialis price
  55. Pingback: buy baclofen 10 mg
  56. Pingback: cialis 5mg
  57. Pingback: cialis pink
  58. Pingback: generic sildenafil
  59. Pingback: viagra 120 pills
  60. Pingback: catapres prices
  61. Pingback: sildenafil generic
  62. Pingback: red dog casino
  63. Pingback: slot machine games
  64. Pingback: play casino
  65. Pingback: casinos
  66. Pingback: kako deluje viagra
  67. Pingback: top car insurance
  68. Pingback: buy cialis
  69. Pingback: abc car insurance
  70. Pingback: au cialis
  71. Pingback: Buy viagra in us
  72. Pingback: cheap payday loans
  73. Pingback: buy cialis
  74. Pingback: cbd oil for pain
  75. Pingback: write essay for me
  76. Pingback: viagra uk cost
  77. Pingback: ace homework
  78. Pingback: ace homework
  79. Pingback: paper writer
  80. Pingback: clomid price
  81. Pingback: cialis generic uk
  82. Pingback: clonidine purchase
  83. Pingback: viagra or cialis
  84. Pingback: cialis miami
  85. Pingback: Order viagra usa
  86. Pingback: coreg pharmacy
  87. Pingback: everyday cialis
  88. Pingback: generic cialis
  89. Pingback: cozaar otc
  90. Pingback: dissertation topic
  91. Pingback: cymbalta prices
  92. Pingback: thesis advice
  93. Pingback: buy thesis online
  94. Pingback: ddavp 10 mcg cost
  95. Pingback: depakote usa
  96. Pingback: diamox nz
  97. Pingback: cheap dramamine
  98. Pingback: erythromycin otc
  99. Pingback: geodon purchase
  100. Pingback: cheap hyzaar
  101. Pingback: imdur 20mg online
  102. Pingback: cialis and alcohol
  103. Pingback: cialis for sale
  104. Pingback: buy cialis online
  105. Pingback: viagra online
  106. Pingback: imodium 2mg tablet
  107. Pingback: this post
  108. Pingback: go to these guys
  109. Pingback: buy viagra
  110. Pingback: imuran generic
  111. Pingback: order lopressor
  112. Pingback: luvox coupon
  113. Pingback: macrobid coupon
  114. Pingback: meclizine uk
  115. Pingback: pharmacy discount
  116. Pingback: Viagra Gold
  117. Pingback: order micardis
  118. Pingback: motrin 400mg uk
  119. Pingback: Brand Cialis
  120. Pingback: prevacid 15 mg nz
  121. Pingback: prilosec tablets
  122. Pingback: procardia purchase
  123. Pingback: proscar generic
  124. Pingback: pulmicort online
  125. Pingback: pyridium 200mg otc
  126. Pingback: order risperdal
  127. Pingback: seroquel coupon
  128. Pingback: spiriva generic
  129. Pingback: tenormin canada
  130. Pingback: toprol otc
  131. Pingback: tricor nz
  132. Pingback: vantin purchase
  133. Pingback: voltaren pharmacy
  134. Pingback: click over here
  135. Pingback: my explanation
  136. Pingback: zocor generic
  137. Pingback: zovirax australia
  138. Pingback: buy viagra where
  139. Pingback: zyvox nz
  140. Pingback: sildenafil cheap
  141. Pingback: cost of tadalafil
  142. Pingback: cheap furosemide
  143. Pingback: escitalopram otc
  144. Pingback: atomoxetine pills
  145. Pingback: olmesartan pills
  146. Pingback: buspirone price
  147. Pingback: cialis sale 20mg
  148. Pingback: trazodone cost
  149. Pingback: buy tolterodine
  150. Pingback: buy cialis rush
  151. Pingback: cialis online
  152. Pingback: phenytoin canada
  153. Pingback: bisacodyl cheap
  154. Pingback: permethrin nz
  155. Pingback: ityyouwo
  156. Pingback: cialis sale 20mg
  157. Pingback: xgwyobxh
  158. Pingback: buy viagra in uae
  159. Pingback: viagra
  160. Pingback: buy cialis rush
  161. Pingback: buy cialis now
  162. Pingback: indomethacin usa
  163. Pingback: cialis in europe
  164. Pingback: help with an essay

Comments are closed.