Many are talking about growth and product analytics, but how many well established companies are leaving behind their old ways of design by committee? In this post I want to highlight a common scenario and promote the modern approach to product development.

Once upon a time…

I received a heads-up from a UX team about an AB test they wanted to run for a new page they had designed. My first reaction was “great, you don’t want to just roll out a new page, you want to make sure it has the desired outcome.”

So what was the desired outcome? By digging a bit deeper, I discovered that…erm, there wasn’t one. The “KPIs” had been plucked out of thin air and they were not relevant to the page in question.

For example, one of the success metrics for the test was bounce rate. This suggests there was a high bounce rate to begin with, which in-turn prompted the redesign. But this page is rarely an entry page, so why were the team redesigning a page to reduce bounce rate?

It occurred to me that the team had no idea why they had created this new page and had no clear objectives for the redesign. They were trying to solve a problem that didn’t exist…or did it?

The page in question, was the top of a funnel. By digging into the data, we discovered that the drop off rate from the top of the funnel to the bottom was significant.

A website funnel

Funnel

Ok, so now we have a problem to solve. And what a great place to start optimising — at the top!

So why were certain decisions made for the redesign?

In my experience this example of product development is not uncommon. Its the difference between having HIPPO (highest paid person’s opinion) led decisions, and data led decisions.

It is a deeply routed cultural issue in many organisations, where areas of the business lose sight of why they are creating something, especially when it involves jumping into a web analytics tool they don’t know how to use, and putting opinions (and sometimes egos) to one side, by listening to the data.

I’m sure if I asked you to post some thing on Facebook that will receive lots of ‘likes’, you’d know what to post. This is because you have learned over time what resonates with your friends, by looking at the numbers. Ok, this is a very basic example, but my point is data is now so deeply engrained in our every day lives — sometimes without even realising it.

The New Approach to Product Development

This is why multi disciplined growth teams (designers, analysts, developers…) are now being formed across the tech industry — to fulfil a desperate need to understand audiences and make smarter decisions.

The importance of a correct web analytics implementation and test & learn function cannot be understated. Without them, product teams are just shooting into the wind, in the hope that something sticks, without ever really knowing if it does in-fact stick.

Going back to my original example, of course there is nothing wrong with testing a new design just because you feel a particular page needs a refresh, as long as every one remains focused on why they are changing/updating certain elements of the experience and what metric(s) they are trying to shift.

Maximise the impact of your changes

If you launch a test where you have changed multiple elements on a page, without any prior iterative testing, then the chances are you won’t see the positive results you’d hoped for. And if you are fortunate enough to see an uplift you will have no way of knowing which changes have been the drivers.

It’s a great headline to share with the business: “We tested this new shiny page and we saw an increase of 2% in sign ins”. High fives all round… But how do you know which element of the new design drove this uplift? Was it the new title/new CTA on the button/larger fields/more emotive images…? These are all valuable questions you could and should have answers to.

Your web analytics data gives you insights into how your users are behaving on your site or app. Are they behaving in the way you intended? If not, then you have identified a problem. Make some assumptions and test them.

The product development testing cycle

Continuous experimentation cycle

So if your team wishes to refresh a page / product development, break down the various elements you think will impact your specific KPI for that page, and test them individually. Slowly you will gather valuable insights that will feed into your redesign as well as future changes. With a more iterative approach you are likely to see even bigger wins.

Finally

Whether you’re a designer, developer, analyst or marketer, we are all striving to see our companies succeed.

In order to succeed we need sustainable growth. To achieve sustainable growth we need to understand our audience. To understand our audience we need data.

Key takeouts for upstream product development testing

  • Know why you are redesigning a page / an experience — what problem are you trying to solve?
  • Have data to back up your new design
  • Test whatever you want to test, but know what you are testing and why
  • For gathering insights, break the elements of the page down into bite size pieces
  • Make assumptions with a sound hypothesis
  • Treat your testing program like any other analytics tool. Use it in conjunction with your web analytics, to find answers quickly, to important questions. Testing tools are effectively web analytics tools

For further reading see my infographic 7 Steps for Effective A/B Testing

Feature image Designed by Freepik

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

1,003 thoughts on “How AB Testing and Product Development Work Together

  1. Pingback: Maria Smith
  2. Pingback: Melanie Bowen
  3. Pingback: buy cialis
  4. Pingback: cialis pills
  5. Pingback: viagra pills
  6. Pingback: viagra
  7. Pingback: cialis
  8. Pingback: viagra online
  9. Pingback: cialis online
  10. Pingback: buy cialis pills
  11. Pingback: cialis black
  12. Pingback: cialis from canada
  13. Pingback: cialis 20mg
  14. Pingback: viagra from india
  15. Pingback: buy cialis canada
  16. Pingback: cialis coupon
  17. Pingback: viagra 100mg
  18. Pingback: viagra for sale
  19. Pingback: best ed pills
  20. Pingback: otc ed pills
  21. Pingback: cialis 20 mg
  22. Pingback: rx pharmacy
  23. Pingback: pharmacy online
  24. Pingback: cialis mastercard
  25. Pingback: cialis mastercard
  26. Pingback: levitra online
  27. Pingback: levitra canada
  28. Pingback: slot games
  29. Pingback: order viagra
  30. Pingback: online casino
  31. Pingback: online cialis
  32. Pingback: personal loans
  33. Pingback: instant loans
  34. Pingback: casino city
  35. Pingback: viagra 100mg
  36. Pingback: cialis buy
  37. Pingback: 5 mg cialis
  38. Pingback: cialis generic
  39. Pingback: generic for cialis
  40. Pingback: 20 cialis
  41. Pingback: viagra online
  42. Pingback: free slots online
  43. Pingback: casino real money
  44. Pingback: viagra reviews
  45. Pingback: buy viagra
  46. Pingback: buy cialis online
  47. Pingback: canadian viagra
  48. Pingback: buy viagra pills
  49. Pingback: cialis tadalafil
  50. Pingback: mens ed pills
  51. Pingback: buy viagra cheap
  52. Pingback: viagra
  53. Pingback: cialis coupon
  54. Pingback: generic cialis
  55. Pingback: casino slots
  56. Pingback: online slots
  57. Pingback: viagra online usa
  58. Pingback: buy cialis cheap
  59. Pingback: Viagra 130mg price
  60. Pingback: viagra buy
  61. Pingback: Viagra 50mg canada
  62. Pingback: cheap Cialis 40 mg
  63. Pingback: Cialis 40mg online
  64. Pingback: sildenafil
  65. Pingback: Cialis 20mg cost
  66. Pingback: rx trust pharm
  67. Pingback: viagra cheap
  68. Pingback: buy Cialis 20mg
  69. Pingback: buy cialis
  70. Pingback: levitra 40mg otc
  71. Pingback: otc viagra
  72. Pingback: viagra dosage
  73. Pingback: actos 30 mg cheap
  74. Pingback: amaryl 4 mg usa
  75. Pingback: cialis prices
  76. Pingback: arava 10 mg nz
  77. Pingback: viagra brand
  78. Pingback: arimidex 1 mg nz
  79. Pingback: viagra kennels
  80. Pingback: baclofen 10mg cost
  81. Pingback: rxtrust pharm
  82. Pingback: shelf life cialis
  83. Pingback: tadalafil citrate
  84. Pingback: rxtrustpharm
  85. Pingback: casodex 50mg cheap
  86. Pingback: ceclor otc
  87. Pingback: celexa pharmacy
  88. Pingback: cephalexin pills
  89. Pingback: claritin 10 mg otc
  90. Pingback: casino slots
  91. Pingback: doubleu casino
  92. Pingback: casino real money
  93. Pingback: cialis miami
  94. Pingback: buy cialis usa
  95. Pingback: sr22 insurance
  96. Pingback: car insurance usaa
  97. Pingback: buy cialis usa
  98. Pingback: quick payday loans
  99. Pingback: buy viagra online
  100. Pingback: check into cash
  101. Pingback: cialis for women
  102. Pingback: top personal loan
  103. Pingback: avanafil vs viagra
  104. Pingback: buy cialis canada
  105. Pingback: levitra reviews
  106. Pingback: sildenafil cost
  107. Pingback: sildenafil female
  108. Pingback: viagra dosage
  109. Pingback: who makes cialis
  110. Pingback: cbd capsules
  111. Pingback: sildenafil 50 mg
  112. Pingback: assignment define
  113. Pingback: viagra cock
  114. Pingback: atorvastatin 1
  115. Pingback: viagra vs levitra
  116. Pingback: sildenafil reviews
  117. Pingback: Generic viagra us
  118. Pingback: colchicine generic
  119. Pingback: tadalafil dosage
  120. Pingback: buy coreg
  121. Pingback: viagra wiki
  122. Pingback: generic cialis
  123. Pingback: compazine uk
  124. Pingback: coumadin cheap
  125. Pingback: US viagra sales
  126. Pingback: cozaar purchase
  127. Pingback: crestor nz
  128. Pingback: dissertation paper
  129. Pingback: 50mg viagra
  130. Pingback: doxycycline prices
  131. Pingback: herbal viagra gnc
  132. Pingback: flomax price
  133. Pingback: cialis ordering
  134. Pingback: viagra naturale
  135. Pingback: geodon uk
  136. Pingback: where to buy imdur
  137. Pingback: cialis 5mg
  138. Pingback: cialis dose
  139. Pingback: cialis price
  140. Pingback: how to buy imodium
  141. Pingback: Find Out More
  142. Pingback: imuran 50 mg pills
  143. Pingback: lamisil for sale
  144. Pingback: lopid generic
  145. Pingback: meclizine purchase
  146. Pingback: mestinon purchase
  147. Pingback: cheap drugs online
  148. Pingback: atorvastatin ddi
  149. Pingback: micardis generic
  150. Pingback: mobic canada
  151. Pingback: motrin canada
  152. Pingback: baclofen gi reflux
  153. Pingback: periactin for sale
  154. Pingback: viagra amazon
  155. Pingback: provigil coupon
  156. Pingback: loviagraosn viagra
  157. Pingback: reglan 10mg tablet
  158. Pingback: cost of remeron
  159. Pingback: sildenafil viagra
  160. Pingback: cialis for sale
  161. Pingback: cost of revatio
  162. Pingback: viagra stock
  163. Pingback: cheap rogaine 5%
  164. Pingback: skelaxin 400mg usa
  165. Pingback: tricor for sale
  166. Pingback: vantin purchase
  167. Pingback: voltaren for sale
  168. Pingback: wellbutrin tablets
  169. Pingback: read here
  170. Pingback: cheapest zocor
  171. Pingback: wild viagra uk
  172. Pingback: sildenafil tablets
  173. Pingback: tadalafil coupon
  174. Pingback: pioglitazone nz
  175. Pingback: viagra for females
  176. Pingback: meclizine cheap
  177. Pingback: atomoxetine pills
  178. Pingback: order donepezil
  179. Pingback: buy cialis doctor
  180. Pingback: cialis 20mg uk
  181. Pingback: warfarin for sale
  182. Pingback: trazodone online
  183. Pingback: tolterodine nz
  184. Pingback: acetazolamide uk
  185. Pingback: byy viagra online
  186. Pingback: buy cialis 36 hour
  187. Pingback: order oxybutynin
  188. Pingback: azithromycin wiki
  189. Pingback: bisacodyl tablet
  190. Pingback: keflex in dogs
  191. Pingback: venlafaxine coupon
  192. Pingback: duloxetine 6 mg
  193. Pingback: liquid amoxicillin
  194. Pingback: gcgwsnfb
  195. Pingback: zithromax price
  196. Pingback: xelhleyz
  197. Pingback: how safe is viagra
  198. Pingback: wat doet viagra
  199. Pingback: viagra review
  200. Pingback: precio cialis
  201. Pingback: cheap cialis pills
  202. Pingback: buy viagra
  203. Pingback: indomethacin price
  204. Pingback: terbinafine prices
  205. Pingback: digoxinmg tablet
  206. Pingback: augmentin 400
  207. Pingback: lasix 40 mg buy
  208. Pingback: ivermectin 5ml
  209. Pingback: tadalafil generic
  210. Pingback: cialis 20mg
  211. Pingback: getroman viagra
  212. Pingback: gemfibrozil pills
  213. Pingback: viagra sample pack
  214. Pingback: buy generic viagra
  215. Pingback: vardenafil
  216. Pingback: real viagra
  217. Pingback: clomid ovary pain
  218. Pingback: cialis from india
  219. Pingback: cialis dosages
  220. Pingback: cost of viagra
  221. Pingback: Google
  222. Pingback: price of viagra
  223. Pingback: sildenafil citrate
  224. Pingback: custom essays usa
  225. Pingback: thc vape juice
  226. Pingback: canada vs usa
  227. Pingback: buy college paper
  228. Pingback: viagra effect time
  229. Pingback: definews
  230. Pingback: pranks with viagra
  231. Pingback: neurontin 100 mg
  232. Pingback: virtual card buy
  233. Pingback: bubble hockey
  234. Pingback: paxil uso
  235. Pingback: buy delta 8 THC
  236. Pingback: cialis in manitoba
  237. Pingback: 우리카지노
  238. Pingback: 퍼스트카지노
  239. Pingback: 샌즈카지노
  240. Pingback: 코인카지노
  241. Pingback: Bitcoin SV (BSV)
  242. Pingback: bersa thunder 380
  243. Pingback: Dank Vapes
  244. Pingback: lasix in mexico
  245. Pingback: 18 Gauge Needle
  246. Pingback: DEXEDRINE 20mg
  247. Pingback: crossdressers cams
  248. Pingback: Zakhar Berkut
  249. Pingback: movers burnaby
  250. Pingback: 4569987
  251. Pingback: buy weed
  252. Pingback: news news news
  253. Pingback: موقع سكس
  254. Pingback: psy
  255. Pingback: psy2022
  256. Pingback: projectio-freid
  257. Pingback: bitcoin atm
  258. Pingback: Tinder northampton
  259. Pingback: kinoteatrzarya.ru
  260. Pingback: topvideos
  261. Pingback: video
  262. Pingback: seo canada
  263. Pingback: buy online viagra
  264. Pingback: gas fireplace
  265. Pingback: buy online viagra
  266. Pingback: lovoo einloggen
  267. Pingback: Ukrainskie-serialy
  268. Pingback: site
  269. Pingback: top
  270. Pingback: cialis wirkung
  271. Pingback: Best Dildo
  272. Pingback: umraniye escort
  273. Pingback: vibrator
  274. Pingback: thrusting dildo
  275. Pingback: porn
  276. Pingback: alex more
  277. Pingback: pharmacy canada
  278. Pingback: masturbator
  279. Pingback: buy virtual card
  280. Pingback: turk escort
  281. Pingback: lifelike dildo
  282. Pingback: buy kratom
  283. Pingback: realistic dildos
  284. Pingback: Childrens haircut
  285. Pingback: cialis 200mg
  286. Pingback: super viagra
  287. Pingback: sildenafil sandoz
  288. Pingback: g spot vibe
  289. Pingback: tadalafil canada
  290. Pingback: cialis drug prices
  291. Pingback: levitra mexico
  292. Pingback: buy kratom online
  293. Pingback: best delta 8
  294. Pingback: chelovek-iz-90-h
  295. Pingback: podolsk-region.ru
  296. Pingback: meloxicam for cats
  297. Pingback: metoprolol
  298. Pingback: cytotmeds.com
  299. Pingback: viagra soft gel
  300. Pingback: cialis samples
  301. Pingback: vardenafil paypal
  302. Pingback: cymbalta reviews
  303. Pingback: bupropion 150 mg
  304. Pingback: buspar dosage
  305. Pingback: citalopram 10 mg
  306. Pingback: zanaflex
  307. Pingback: wellbutrin dosing
  308. Pingback: what is coreg
  309. Pingback: bender na4alo 2021
  310. Pingback: blogery_i_dorogi
  311. Pingback: strapon pegging
  312. Pingback: sildenafil dosage
  313. Pingback: real cialis pills
  314. Pingback: sildenafil 25 mg
  315. Pingback: THC Gummies
  316. Pingback: buy cbd gummies
  317. Pingback: bachelor's degree
  318. Pingback: acyclovir
  319. Pingback: amoxicillin uses
  320. Pingback: amoxicillin dosage
  321. Pingback: keflex antibiotic
  322. Pingback: clindamycin class
  323. Pingback: zithromax tablets
  324. Pingback: makeup tutorial
  325. Pingback: top vibrators
  326. Pingback: BANANA RUNTZ CART
  327. Pingback: SF Examiner
  328. Pingback: cialis forms
  329. Pingback: levitra vs viagra
  330. Pingback: Kelowna movers
  331. Pingback: Takeout Food
  332. Pingback: MILF Porn Land
  333. Pingback: chernaya vodova
  334. Pingback: 66181
  335. Pingback: viagra substitute
  336. Pingback: viagra wikipedia
  337. Pingback: pharmacy canada
  338. Pingback: Sig P365 Xl
  339. Pingback: how levitra works
  340. Pingback: metformin er500
  341. Pingback: Porno
  342. Pingback: vechernyy urgant
  343. Pingback: ukraine
  344. Pingback: Elliptical Trainer
  345. Pingback: priligy price usa
  346. Pingback: A3ixW7AS
  347. Pingback: clomid 50mg bula
  348. Pingback: bunnings shed
  349. Pingback: software reviewer
  350. Pingback: Buy Guns Online
  351. Pingback: link
  352. Pingback: cialis trial
  353. Pingback: Canada Pharmacy
  354. Pingback: cialis sales
  355. Pingback: buy weed in Hawaii
  356. Pingback: KremlinTeam
  357. Pingback: medunitsa.ru
  358. Pingback: kremlin-team.ru
  359. Pingback: psychophysics.ru
  360. Pingback: yesmail.ru
  361. Pingback: ptsd
  362. Pingback: Buy Cocaine Online
  363. Pingback: Suicide Squad 2
  364. Pingback: learn more
  365. Pingback: psiholog
  366. Pingback: Knee Pain
  367. Pingback: discount viagra
  368. Pingback: cialis dosis
  369. Pingback: happyluke
  370. Pingback: happyluke
  371. Pingback: Ventolin inhalator
  372. Pingback: regcialist.com
  373. Pingback: kadikoy escort
  374. Pingback: Shrooms Vancouver
  375. Pingback: gay sex toy
  376. Pingback: mersin escort
  377. Pingback: Takeout
  378. Pingback: pocket masturbator
  379. Pingback: vibrator review
  380. Pingback: Canadian website
  381. Pingback: hosting deals
  382. Pingback: ground leases
  383. Pingback: ruay lotto
  384. Pingback: prostate massager
  385. Pingback: dildo and balls
  386. Pingback: Thc jull pods
  387. Pingback: aksaray escort
  388. Pingback: generic viagra
  389. Pingback: streamcomplet
  390. Pingback: car for export
  391. Pingback: french streaming
  392. Pingback: streaming gratuit
  393. Pingback: CTRLCOIN
  394. Pingback: twitter downloader
  395. Pingback: pocket pussy
  396. Pingback: cialis uk
  397. Pingback: omeprazole dr 20
  398. Pingback: sex education
  399. Pingback: masturbating
  400. Pingback: 1xbit mobile
  401. Pingback: pocket masturbator
  402. Pingback: best in ottawa
  403. Pingback: pregabalin buy
  404. Pingback: viagra generic
  405. Pingback: 피망머니상
  406. Pingback: russian shirt
  407. Pingback: viagra pill
  408. Pingback: viagra sale
  409. Pingback: therme skincare

Comments are closed.