Implementing an ab testing tool is the painless step in embracing Conversion Rate Optimisation (CRO). Perhaps you have a new shiny ab testing tool implemented on your site and you don’t know where to start. Or maybe you’ve heard everyone else is doing it (not to mention the awesome benefits), and you want to kick off the conversation with your team/boss.

Whatever tool you have chosen or you are exploring, it is important to understand that it is just an enabler. Without people and process, the tool is useless. So here is a quick guide to Conversion Rate Optimisation (CRO).

What is AB Testing?

Serving multiple versions of a web page or experience at the same time. This allows you to:

  • Test hypothesis and new ideas
  • Quantify your decisions
  • Measure website performance
  • Provide insights about your visitors to feed into future features and designs

 

Why a Testing Culture is Important

Opinions are like noses…every one has one. Whether this is what features you should be developing or the look and layout of your platforms; ab testing challenges these opinions and lets your audience decide through their behaviour. By implementing the right processes you can continually optimise to make your product(s) better.

(For more detail please see my post How AB Testing and Product Development Work Together)

What to Test

Knowing where to start can be difficult and sometimes overwhelming. If you work on a website or mobile app that has thousands of pages/screens, where do you start? Firstly you should have some clearly identified (micro) KPIs for every page on your site, which impact your more high level (macro) KPIs, so you know which metrics to focus on page by page.

I always start by looking at the 2–3 highest volume landing pages and/or the top of a funnel. I measure the performance of the page(s) next to the relevant goals. How is the page performing? Could it do better? What elements of the page could be impacting the success of the page? From here you can begin to build out some test ideas.

Here is a few scenarios where you might want to run a test:

  • A Hunch — when there is an element on a platform that you believe isn’t working and you want to change it. Having a hunch is fine and is a good basis to run a test, but it’s important to back up your hunch with data, to make sure you’re ab testing the right page(s)
  • Data — when you spot a problem in the data. For example, you notice that a high drop off from step 1 of the sign up funnel. The data tells us WHAT is happening, but it doesn’t tell us WHY. This is where you create a hypothesis and then test it
  • A New Feature — when a new feature is developed, there should always be clear KPIs, so you know exactly what metric(s) you are trying to impact. You can use this KPI to run tests and let your users decide whether it is the right decision or not
  • Iteration — sometimes when you run a test you will see results you are unable to explain. This might create a new hypothesis which you might want to test further, to find answers

 

AB Testing Process

When running multiple tests, it can get messy. Here is 7 steps to consider to keep your ab testing program as effective as it can be:

1. Hypothesis

Every test MUST have a hypothesis, so you establish why you’re running the test in the first place and then it can be proved or disproved. A hypothesis should have a problem, solution and result

2. Design

  • Create variations
    Create as many variations for test as makes sense, based on the assumptions you have made in your hypotheses and the traffic volumes you have available
  • Tracking
    How are you going to track your goals? What custom events do you need to implement in your testing tool and/or analytics tool?
  • Audience Segments
    What segments do you want to test on? For example, is there an specific country or mobile device you want to target?
  • Approval
    Get internal approval for the changes

3. Build & QA

  • Using your preferred ab testing tool, build the test inside the UI.
  • QA every variation in all the major browsers

4. Launch

  • Push the test live

5. Analyse

  • Review the results of the test in your chosen ab testing or analytics tool
  • Don’t forget to segment your data. You may see zero uplift in the data when looking at the high level data, but when you segment by – for example – traffic source / device / country, you might see an uplift. Don’t treat every user the same

6. Action

  • Make recommendations off the back of the data:
  • Is there a clear winner? If so, should you roll out the new experience or run an iterative test to build on the success of the test and try to drive even greater uplift?
  • Is the test generating more questions than answers? This is ok. These questions may point to more assumptions that you want to test (even if you don’t see a clear winner, this could still apply)

7. Document

  • Record test results somewhere organised and accessible.Test results are another source of data insights, like some one logging into an analytics tool to find insights

Below is an infographic that helps to illustrates the process above for running ab tests, designed by Seb Wals

Summary

  • It needs planning and sound methodology
  • It needs a hypothesis, so you have something to prove or disprove
  • It should always be data driven
  • It needs clear KPIs, so you can clearly measure success
  • It should not have too many KPIs. Keep it simple
  • When thinking about success metrics, also think about whether your test could cannibalise other KPIs on the platform — these may be worth measuring as well
  • You can’t fail when testing — there is always insights
  • AB Testing must be statistically significant before any results can be acted upon

ab testing process

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

1,319 thoughts on “AB Testing: A Quick Guide for Getting Started

  1. Pingback: viagra 5mg price
  2. Pingback: cialis 20 mg price
  3. Pingback: cialis 5 mg
  4. Pingback: how much is cialis
  5. Pingback: goodrx cialis
  6. Pingback: viagra for sale
  7. Pingback: ed pills
  8. Pingback: ed pills
  9. Pingback: sale cialis
  10. Pingback: Cialis in usa
  11. Pingback: vardenafil usa
  12. Pingback: order vardenafil
  13. Pingback: generic sildenafil
  14. Pingback: personal loan
  15. Pingback: instant loans
  16. Pingback: real money casino
  17. Pingback: instant loans
  18. Pingback: viagra 100mg
  19. Pingback: cialis 5 mg
  20. Pingback: generic for cialis
  21. Pingback: cialis 20
  22. Pingback: cialis 20
  23. Pingback: chumba casino
  24. Pingback: viagra for women
  25. Pingback: sildenafil dosage
  26. Pingback: viagra pills
  27. Pingback: generic for viagra
  28. Pingback: viagra sample
  29. Pingback: brand viagra
  30. Pingback: buy cialis
  31. Pingback: viagra canada
  32. Pingback: best ed medication
  33. Pingback: buy cheap viagra
  34. Pingback: jueriy.com
  35. Pingback: order Viagra 200mg
  36. Pingback: buy cialis online
  37. Pingback: Viagra 200mg nz
  38. Pingback: Cialis 40 mg uk
  39. Pingback: cialis prices
  40. Pingback: buy Cialis 20mg
  41. Pingback: Cialis 10 mg nz
  42. Pingback: buy Cialis 40mg
  43. Pingback: viagra coupons
  44. Pingback: tadalafil 20mg nz
  45. Pingback: viagra price
  46. Pingback: amaryl 1 mg usa
  47. Pingback: online viagra
  48. Pingback: cialistodo.com
  49. Pingback: buy cialis toronto
  50. Pingback: benicar 20mg otc
  51. Pingback: cialis dosage
  52. Pingback: buspar 10mg uk
  53. Pingback: buy cialis bangkok
  54. Pingback: tadalafil generic
  55. Pingback: viagra sold in usa
  56. Pingback: casodex tablets
  57. Pingback: buy ceclor
  58. Pingback: celebrex 100 mg nz
  59. Pingback: celexa nz
  60. Pingback: casino game
  61. Pingback: rivers casino
  62. Pingback: gambling casino
  63. Pingback: best casino online
  64. Pingback: play casino
  65. Pingback: casinos online
  66. Pingback: accc car insurance
  67. Pingback: cialis 20mg pills
  68. Pingback: payday loan today
  69. Pingback: tadalafil cialis
  70. Pingback: quick loans
  71. Pingback: cialis 10mg
  72. Pingback: hims viagra
  73. Pingback: 5mg cialis
  74. Pingback: viagra boner
  75. Pingback: how to take viagra
  76. Pingback: buy cialis canada
  77. Pingback: tadalafil 5 mg
  78. Pingback: viagra women
  79. Pingback: viagra costs
  80. Pingback: online viagra
  81. Pingback: cbd massage oil
  82. Pingback: viagra and alcohol
  83. Pingback: cost of cialis
  84. Pingback: Discount viagra
  85. Pingback: viagra twitter
  86. Pingback: atorvastatin and
  87. Pingback: buy essay
  88. Pingback: clomid prices
  89. Pingback: clonidine pills
  90. Pingback: Testogen
  91. Pingback: Buy cheap viagra
  92. Pingback: female viagra sex
  93. Pingback: tadalafil
  94. Pingback: cialis price india
  95. Pingback: viagra prices
  96. Pingback: cialis online
  97. Pingback: cheap cozaar 100mg
  98. Pingback: crestor 5mg cheap
  99. Pingback: thesis editing
  100. Pingback: write my paper
  101. Pingback: cost of ddavp
  102. Pingback: doxycycline otc
  103. Pingback: elavil tablet
  104. Pingback: Testing60
  105. Pingback: flomax 0,2mg pills
  106. Pingback: levitra coupon
  107. Pingback: geodon purchase
  108. Pingback: hyzaar prices
  109. Pingback: imdur tablets
  110. Pingback: viagra online
  111. Pingback: imitrex prices
  112. Pingback: oxybutynin 10 mg
  113. Pingback: try this
  114. Pingback: viagra otc
  115. Pingback: look here
  116. Pingback: zanaflex picture
  117. Pingback: online viagra
  118. Pingback: imuran for sale
  119. Pingback: lamisil 250mg cost
  120. Pingback: lopid 300 mg usa
  121. Pingback: amlodipine 25 mg
  122. Pingback: order luvox
  123. Pingback: macrobid canada
  124. Pingback: mestinon price
  125. Pingback: atorvastatin 152
  126. Pingback: motrin usa
  127. Pingback: bupropion hcl er
  128. Pingback: viagra
  129. Pingback: cosopt and coreg
  130. Pingback: procardia pharmacy
  131. Pingback: 10 mg celexa
  132. Pingback: provigil canada
  133. Pingback: viagra sister
  134. Pingback: viagra mistake
  135. Pingback: remeron usa
  136. Pingback: cialis from canada
  137. Pingback: viagra roman
  138. Pingback: cheapest revatio
  139. Pingback: risperdal pills
  140. Pingback: snorting viagra
  141. Pingback: rogaine canada
  142. Pingback: seroquel coupon
  143. Pingback: cost of singulair
  144. Pingback: skelaxin for sale
  145. Pingback: toprol price
  146. Pingback: how to buy tricor
  147. Pingback: voltaren tablet
  148. Pingback: zestril tablets
  149. Pingback: her explanation
  150. Pingback: additional info
  151. Pingback: where to buy zocor
  152. Pingback: types of viagra
  153. Pingback: furosemide nz
  154. Pingback: sophia viagra nu
  155. Pingback: clonidine prices
  156. Pingback: cialis in europe
  157. Pingback: clozapine for sale
  158. Pingback: buy cialis drug
  159. Pingback: divalproex cheap
  160. Pingback: tolterodine online
  161. Pingback: acetazolamide cost
  162. Pingback: phenytoin pharmacy
  163. Pingback: oxybutynin generic
  164. Pingback: bisacodyl canada
  165. Pingback: permethrin canada
  166. Pingback: 141genericExare
  167. Pingback: cialis cost at cvs
  168. Pingback: pbhkvdjj
  169. Pingback: cheap tamsulosin
  170. Pingback: Google
  171. Pingback: tnkfrhtk
  172. Pingback: celebrex and dogs
  173. Pingback: keflex anti biotic
  174. Pingback: dp viagra
  175. Pingback: top broker
  176. Pingback: 샌즈카지노
  177. Pingback: 메리트카지노
  178. Pingback: 코인카지노
  179. Pingback: isosorbide usa
  180. Pingback: AnaGlobal
  181. Pingback: container ship
  182. Pingback: loperamide nz
  183. Pingback: best cbd gummies
  184. Pingback: cbd oil for dogs
  185. Pingback: canada viagra
  186. Pingback: best cbd gummies
  187. Pingback: prostate massager
  188. Pingback: cialis 20mg usa
  189. Pingback: buy indomethacin
  190. Pingback: viagra 100
  191. Pingback: working visa
  192. Pingback: buy cbd oil
  193. Pingback: software buy
  194. Pingback: essay on the help
  195. Pingback: Mzadaty
  196. Pingback: buy kratom online
  197. Pingback: kratom for anxiety
  198. Pingback: cheap lamotrigine
  199. Pingback: 우리카지노
  200. Pingback: 퍼스트카지노
  201. Pingback: DEATH BUBBA
  202. Pingback: lasix 5 mg
  203. Pingback: buy levitra
  204. Pingback: viagra canada
  205. Pingback: 150 mg viagra
  206. Pingback: riley reid viagra
  207. Pingback: cialis from india
  208. Pingback: levitra online
  209. Pingback: canadian cialis
  210. Pingback: clomid drug test
  211. Pingback: sildenafil 100 mg
  212. Pingback: cialis for sale
  213. Pingback: avanafil vs viagra
  214. Pingback: mp3juices
  215. Pingback: synthroid goodrx
  216. Pingback: tadalafil 20 mg
  217. Pingback: pure amphetamine
  218. Pingback: click here
  219. Pingback: essay writers net
  220. Pingback: Herbal incense
  221. Pingback: Home, Garden
  222. Pingback: viagra community
  223. Pingback: cialis
  224. Pingback: cialis 20 mg price
  225. Pingback: low cost cialis
  226. Pingback: bitcoincard
  227. Pingback: 600 neurontin
  228. Pingback: Craft wire
  229. Pingback: virtual card buy
  230. Pingback: delta 8 near me
  231. Pingback: online consultants
  232. Pingback: Vidalista 40mg
  233. Pingback: albino penis envy
  234. Pingback: Testing62
  235. Pingback: Guns for sale
  236. Pingback: propecia purchase
  237. Pingback: viagra india price
  238. Pingback: cialis 20mg usa
  239. Pingback: Surgical Gloves
  240. Pingback: cost of furosemide
  241. Pingback: ktm duke 390
  242. Pingback: Nortriptyline
  243. Pingback: Zakhar Berkut
  244. Pingback: 4569987
  245. Pingback: mexico cialis
  246. Pingback: news news news
  247. Pingback: learn more
  248. Pingback: learn more
  249. Pingback: psy
  250. Pingback: Strap on sex toy
  251. Pingback: psy2022
  252. Pingback: projectio-freid
  253. Pingback: about
  254. Pingback: refund tinder gold
  255. Pingback: kinoteatrzarya.ru
  256. Pingback: canadian services
  257. Pingback: topvideos
  258. Pingback: video
  259. Pingback: generic canadian
  260. Pingback: 50 Beowulf
  261. Pingback: bàn học cho bé
  262. Pingback: generic canadian
  263. Pingback: Ukrainskie-serialy
  264. Pingback: site
  265. Pingback: cougar dating free
  266. Pingback: top
  267. Pingback: cialis in europe
  268. Pingback: sex toys
  269. Pingback: bondage spank
  270. Pingback: cialis soft tablet
  271. Pingback: alex more
  272. Pingback: realistic dildo
  273. Pingback: reasons for ed
  274. Pingback: slot online
  275. Pingback: anal sex kit
  276. Pingback: Guns for Sale
  277. Pingback: lesbian strapon
  278. Pingback: cialis daily dose
  279. Pingback: viagra sildenafil
  280. Pingback: Dank Vapes
  281. Pingback: price of viagra
  282. Pingback: PARROTS FOR SALE
  283. Pingback: Best software
  284. Pingback: MASALA MACHINE
  285. Pingback: cialis pills
  286. Pingback: canada viagra cost
  287. Pingback: extra super viagra
  288. Pingback: gabapentin 300 mg
  289. Pingback: levitra tab 20mg
  290. Pingback: kratom capsules
  291. Pingback: kratom for sale
  292. Pingback: chelovek-iz-90-h
  293. Pingback: drugs like viagra
  294. Pingback: podolsk-region.ru
  295. Pingback: norvasc amlodipine
  296. Pingback: atorvastatin 40mg
  297. Pingback: meloxicam 10
  298. Pingback: losartan dosage
  299. Pingback: does cialis expire
  300. Pingback: metformin hcl
  301. Pingback: buspirone reviews
  302. Pingback: celexa reviews
  303. Pingback: bupropion overdose
  304. Pingback: voltaren tablets
  305. Pingback: clonidine dose
  306. Pingback: cytotmeds.com
  307. Pingback: coreg dosage
  308. Pingback: how to help ed
  309. Pingback: bender na4alo 2021
  310. Pingback: blogery_i_dorogi
  311. Pingback: acheter cialis
  312. Pingback: viagra sales usa
  313. Pingback: Cannabis Edibles
  314. Pingback: delta 8 for sale
  315. Pingback: remedies for ed
  316. Pingback: stromectol 3mg
  317. Pingback: bachelor's degree
  318. Pingback: stromectol 3 mg
  319. Pingback: acyclovir dosing
  320. Pingback: cefdinir 300mg
  321. Pingback: erythrocin
  322. Pingback: zithromax 500 mg
  323. Pingback: plaquenil canada
  324. Pingback: doc johnson
  325. Pingback: best butt plug
  326. Pingback: plaquenil sulfate
  327. Pingback: buy weed online
  328. Pingback: Seattle Weekly
  329. Pingback: cialis samples
  330. Pingback: viagra walmart
  331. Pingback: tiktok followers
  332. Pingback: viagra like pills
  333. Pingback: b+ mushrooms
  334. Pingback: hemp oil for sale
  335. Pingback: ruger guns
  336. Pingback: keto shrimp scampi
  337. Pingback: movers Kelowna
  338. Pingback: bayer levitra
  339. Pingback: Obligasi.com
  340. Pingback: cialis price usa
  341. Pingback: best cure for ed
  342. Pingback: viagra memes
  343. Pingback: canada ed drugs
  344. Pingback: MILF Porn Land
  345. Pingback: chernaya vodova
  346. Pingback: 66181
  347. Pingback: goztepe escort
  348. Pingback: istanbul eskortlar
  349. Pingback: viagra paypal
  350. Pingback: pharmacy in canada
  351. Pingback: Sig P229
  352. Pingback: Study Table
  353. Pingback: causes for ed
  354. Pingback: cialis generic
  355. Pingback: metformin benefits
  356. Pingback: cialis everyday
  357. Pingback: Porno
  358. Pingback: vechernyy urgant
  359. Pingback: ukraine
  360. Pingback: how much is viagra
  361. Pingback: xenical price
  362. Pingback: bimatoprost acid
  363. Pingback: A3ixW7AS
  364. Pingback: cialis soft tablet
  365. Pingback: bunnings shed
  366. Pingback: adult toys
  367. Pingback: learn more
  368. Pingback: best software list
  369. Pingback: HBCUC
  370. Pingback: Canada pharmacy
  371. Pingback: link
  372. Pingback: weed online usa
  373. Pingback: fucking on viagra
  374. Pingback: KremlinTeam
  375. Pingback: medunitsa.ru
  376. Pingback: kremlin-team.ru
  377. Pingback: psychophysics.ru
  378. Pingback: yesmail.ru
  379. Pingback: ptsd
  380. Pingback: Suicide Squad 2
  381. Pingback: psiholog
  382. Pingback: SHOULDER PAIN
  383. Pingback: silicone vibrator
  384. Pingback: penis extender
  385. Pingback: male sex toys
  386. Pingback: viagra strain
  387. Pingback: erotica stories
  388. Pingback: viagra u.s.a
  389. Pingback: viagra gold 800 mg
  390. Pingback: definition cialis
  391. Pingback: eskort bayanlar
  392. Pingback: Crack For Sale
  393. Pingback: US News
  394. Pingback: best wand vibrator
  395. Pingback: best male strokers
  396. Pingback: regcialist.com
  397. Pingback: bayrampasa escort
  398. Pingback: ben wa balls
  399. Pingback: sexual health
  400. Pingback: gay sex toys
  401. Pingback: ivermectin 18mg
  402. Pingback: mersin escort
  403. Pingback: sensuelle massager
  404. Pingback: drug pharmacy
  405. Pingback: ivermectin 12 mg
  406. Pingback: buy weed in Ohio
  407. Pingback: Food Delivery
  408. Pingback: fun88
  409. Pingback: panty vibe
  410. Pingback: vagina enlarger
  411. Pingback: visit website
  412. Pingback: hosting deals
  413. Pingback: ruay vip
  414. Pingback: pocket masturbator
  415. Pingback: how to use dildo
  416. Pingback: لحس الكس
  417. Pingback: Bulldogs for sale
  418. Pingback: turk escort
  419. Pingback: streamcomplet
  420. Pingback: site de streaming
  421. Pingback: streaming gratuit
  422. Pingback: prozac dose
  423. Pingback: pills from canada
  424. Pingback: best in ottawa
  425. Pingback: mango nac
  426. Pingback: escitalopram 40mg
  427. Pingback: 안전놀이터
  428. Pingback: 피망슬롯머니
  429. Pingback: ivermectin 50
  430. Pingback: ivermectin rx
  431. Pingback: jewelry gift boxes
  432. Pingback: sildenafil citrate
  433. Pingback: medications for
  434. Pingback: gillette
  435. Pingback: Thc juul pods
  436. Pingback: cialis shop
  437. Pingback: cialis samples
  438. Pingback: OXYCODONE FOR SALE
  439. Pingback: Duna 2021
  440. Pingback: about
  441. Pingback: www.viagrakari.com
  442. Pingback: masturbation
  443. Pingback: anal sex toy
  444. Pingback: 1
  445. Pingback: cheapest viagra
  446. Pingback: apps for pc
  447. Pingback: mp3 juices
  448. Pingback: Monkeys For Sale
  449. Pingback: order viagra us
  450. Pingback: clomid 50mg
  451. Pingback: ivermectin drug
  452. Pingback: ivermectin 1mg
  453. Pingback: sildenafil 20mg
  454. Pingback: aromatherapy
  455. Pingback: viagra 800mg price
  456. Pingback: stromectol 12mg
  457. Pingback: ventolin nebuliser
  458. Pingback: viagra 100 mg
  459. Pingback: viagra spider
  460. Pingback: landor arms tx 801
  461. Pingback: sig m17
  462. Pingback: discount cialis
  463. Pingback: magic mushrooms
  464. Pingback: cialis wirkung
  465. Pingback: aspirin and ed
  466. Pingback: mint juul pods
  467. Pingback: rosy boa for sale
  468. Pingback: cockatoo for sale
  469. Pingback: d33006 video card
  470. Pingback: buy cross trainer
  471. Pingback: Vertise
  472. Pingback: Beretta CX4 STRM
  473. Pingback: prednisone daily
  474. Pingback: ivermectin humans
  475. Pingback: RissMiner
  476. Pingback: dizziness
  477. Pingback: bitcoin
  478. Pingback: otc z pack
  479. Pingback: Uganda Safari
  480. Pingback: canadian drugs
  481. Pingback: viagra cheap
  482. Pingback: beretta shortguns
  483. Pingback: raw garden extract
  484. Pingback: sig mcx virtus
  485. Pingback: bacimi sikin benim
  486. Pingback: viagra for men
  487. Pingback: cleantalkorg2.ru
  488. Pingback: Anne Klein Watch
  489. Pingback: viagra cock
  490. Pingback: levitra vs viagra
  491. Pingback: viagras
  492. Pingback: ivermectin 6mg
  493. Pingback: amoxicillin
  494. Pingback: roman viagra
  495. Pingback: best lasix
  496. Pingback: 12 gabapentin
  497. Pingback: gabapentin 1000mg
  498. Pingback: 24 mg prednisone
  499. Pingback: viagra sildenafil
  500. Pingback: avana 3131
  501. Pingback: stromectol pills
  502. Pingback: ventolin 70
  503. Pingback: ventolin hfa
  504. Pingback: zithromax online
  505. Pingback: viagra canada
  506. Pingback: lasix 40 mg daily
  507. Pingback: quineprox 30
  508. Pingback: buy modafinil
  509. Pingback: Rentals car Bar
  510. Pingback: buy backlinks
  511. Pingback: mining rigs

Comments are closed.