UX teams must use data. They must use data to identify a problem, to help them to find a solution, and finally, to help quantify and tweak their efforts.
Not so long ago I received a heads up about an A/B test a UX team wanted to run for a new page they had designed. My first reaction was “great, they don’t want to just roll out a new page, they want to make sure it has the desired outcome.”
So what was the desired outcome? By digging a bit deeper into this planned test, I discovered that...erm, there wasn’t one. The ”KPIs” had been plucked out of thin air and they were not relevant to the page in question. For example, one of the success metrics for the test was bounce rate. But this page is rarely an entry page, so why are we redesigning a page to reduce bounce rate? That surely makes the assumption that there was a high bounce rate to begin with, which in-turn prompted this redesign? No, this was not the case - clearly, no one had looked at any data to gather insights.