Product Development: Experimentation is Where it Begins

product development illustration

Many are talking about growth and product analytics, but how many well established companies are leaving behind their old ways of design by committee? In this post I want to highlight a common scenario and promote the modern approach to product development.

Once upon a time…
I received a heads-up from a UX team about an AB test they wanted to run for a new page they had designed. My first reaction was “great, you don’t want to just roll out a new page, you want to make sure it has the desired outcome.”

So what was the desired outcome? By digging a bit deeper, I discovered that…erm, there wasn’t one. The “KPIs” had been plucked out of thin air and they were not relevant to the page in question.

For example, one of the success metrics for the test was bounce rate. This suggests there was a high bounce rate to begin with, which in-turn prompted the redesign. But this page is rarely an entry page, so why were the team redesigning a page to reduce bounce rate?

It occurred to me that the team had no idea why they had created this new page and had no clear objectives for the redesign. They were trying to solve a problem that didn’t exist…or did it?

The page in question, was the top of a funnel. By digging into the data, we discovered that the drop off rate from the top of the funnel to the bottom was significant.

A website funnel


Ok, so now we have a problem to solve. And what a great place to start optimising — at the top!

So why were certain decisions made for the redesign?

Continue reading → Product Development: Experimentation is Where it Begins

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

Amplitude – A Review of What They are Doing Well

Amplitude celebrating success

Amplitude have just announced $30 million of Series C funding, and I wanted to look at what they are doing well, in an increasingly competitive market.

Hopefully if you’re reading this you’ve already heard of Amplitude. For those of you that haven’t, it is a relatively new mobile and web analytics tool.

On first hearing about Amplitude I was very intrigued. I was desperate to find out what they were doing so differently to their well established rivals, that made them think they could compete. What could they possibly have that the others don’t? In hindsight, the same question could have been asked of other tech companies like Airbnb and Slack (holiday lettings and instant chat? Yeah, because thats never been done before!).

At a time when ‘growth’ is the buzz word of the tech start up scene, Amplitude have cleverly positioned their solution as a growth tool and moved away from simply being a marketing /reporting tool, which is where many of their competitors come from.

With regards to the use of analytics, companies tend to fall into one of two buckets:

Vanity Analytics

  • Reporting vanity metrics (sessions, page views, avg. time on site) back to the business, with minimal actionable insights and superficially measuring growth
  • Creating loads of dashboards that no one looks at

Growth Analytics

  • Proactively analysing and identifying problems and opportunities
  • Understanding user behaviour — how do users fall in love with the app/site, and what keeps them coming back again and again?

I’ve seen vanity analytics alot and its a difficult trend to break. What do you do when you have an exec asking you “how many page views has page x had?”.

There is no doubt that vanity analytics is a cultural issue for alot of organisations, but it does feel like many analytics tools on the market haven’t helped this mindset. Perhaps it’s due to legacy features that has left some vendors behind, or maybe it’s the analytics vendors actually giving their customers what they want. Either way, Amplitude have taken the decision to do things differently.

Amplitude is a reporting tool secondary to being a product insights tool. It gives product teams easy access to insightful, actionable data, without the need to run any complex sql queries. The straight forward event tracking gives lots of flexibility for manipulating the data, with plenty of helpful out of the box charts. The most impressive feature is a predictive tool called ‘Compass’, which analyses behaviours that are closely correlated with retention.

Amplitude are in the privileged position of having been able to build their product from scratch and make it relevant for the twenty-tens, without any noisy legacy features. For example a lot of their competitors have been around for a comparatively long time, and they were designed for web. Along came mobile and they adapted their tech for native apps, with what appears to be little innovation — with the same tracking methods and reports as web.

Amplitude is a mobile first tool by focusing on helping to answer the increasingly difficult question of ‘how the hell do I retain my native app users?’.

It is clear that Amplitude have created a product that it’s customers have been (knowingly or unknowingly) crying out for (helped by the story behind the birth of the company), and they have reached this point in their journey by listening intently to their users. They have literally put into practice the culture they are trying to promote to their prospects and existing customers, of making smarter data led product development decisions.

Amplitude certainly know their industry and audience. This is also demonstrated by their content marketing — it is some of the best content I’ve seen. As an example, one of my early reads of their blog was Behavioral Cohorts: find your most engaged users. It’s heavily focused on the capabilities of their product, but it did the job of getting me thinking about the holes in our existing tool.

I also encourage you to read their Product Analytics Playbook. This is a masterclass in content marketing. It’s not just hot air; it gives a framework for developing a retention strategy with data. It demonstrates how in-tune they are with their market.

A little while ago I met their Co-founder and CEO, Spencer Skates. In the meeting he presented some key elements from the playbook and demonstrated how his team is applying them to their own retention strategy. I enjoyed Spencer’s transparency, and appreciated his understanding of what companies like mine are looking for.

Amplitude is an extremely exciting company. Of course the product is not perfect, but their appetite for customer feedback, and pace of development can only benefit their product iteration, and in-turn their customers. I personally want a partner that lets us lead them, and not the other way round. My hope is that as they grow in size, and eventually move away from ‘start-up’ status, they can maintain their way of working.

If you are in the market for a new web analytics tool, and Amplitude are on your hit list, please feel free to contact me for some impartial feedback.

(My company has recently migrated to Amplitude. Amplitude have had no involvement in the writing of this post).

Featured image Designed by Freepik

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

7 Step Process for Effective A/B Testing

Implementing a testing tool is easy. The challenge is building a testing culture. Below is an infographic to help you apply some process to your experimentation.

For more detailed information about the below steps, please see AB Testing – A Quick Guide For Getting Started

Share this Image On Your Site

Share this Image On Your Site

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.

App Store Optimization: How to Increase Store Ratings

App Store Optimization is the art of getting your app visible in the app store. Much like SEO, it is about getting your app ranking high for certain search terms, and there are various factors which impact your ranking. I’m not going to attempt to cover all of these factors, but I wanted to share one element which I recently had some success with – store ratings and reviews.

65% of all downloads follow an app store search, so the importance of getting your app ranking high is very clear.

Beyond the fact that apps with higher ratings rank higher for competitive keywords, it is clearly good for raising your profile and could potentially help drive your download conversion rate.

App Store Optimization Campaign –  ask your users to leave you a review

As consumers we are most likely to leave a review or give feedback for a service or product thats been really bad. When we are mad and/or frustrated we often take to the internet to voice our frustration — it can be therapeutic! On the flip side we might also feel inclined to leave a review when we’ve had an amazing experience. However, its true that the majority of positive experiences go unmentioned.

Not every single user is going to love your app (shocking I know). If you have a decent amount of repeat traffic, it’s safe to assume you have a pool of users who are getting value out of your app and therefore might be willing to spare you a couple of minutes of their time to leave you a review. For the sake of this exercise, these are the people you want to identify and target.

Continue reading → App Store Optimization: How to Increase Store Ratings

Growth Lead at Dailymotion, Richard has a passion for improving user experience and ROI through data and experimentation.