In the previous post I talked about introducing agile working practices to non-tech teams to help us go faster — increased speed being a key outcome of business agility.
Another key outcome we sought to achieve was “better”. In this post I’ll talk about how we started to introduce experimentation to non-tech teams to optimise the value the squads created.
The case for experimentation
As with agile working practices, within the world of digital product development, experimentation is widely accepted as the optimal way to work. Testing assumptions and creating minimum viable products is recognised as a sensible way to minimise risk and figure out exactly what your customers want. The growth marketing and growth hacking movement have also introduced experimentation to the world of marketing – digital marketing in particular.
However, beyond the world of product development, the concept of experimentation is still quite new. Yet as with any element of agility, it’s a set or principles and processes that are agnostic of product and technology and can be applied in other contexts.
Using experimentation to increase retention
To describe how we started to introduce experimentation, I’ll use the case study of the Retention Pod. In an earlier post I described the retention pod and their working process. The product engineering people in the retention pod had been experimenting for some time, but for their CRM colleagues, experimentation was something new. I’ll explain how we set about introducing experimentation to the CRM Retention Squad.
The backstory
Historically Moonpig had made great use of TV marketing and it had served them very well. Within the UK, Moonpig has huge brand awareness — 75%+. However, the leadership were keen to boost our digital marketing efforts, both to maintain growth and improve ROI.
Skilling up
Within marketing the benefits of experimenting were quickly recognised, but what the teams needed was help to get started. To kick-start our lean marketing efforts we invested in some training with the excellent Growth Tribe Academy. We organised 3 days in-house training for 24 members of our growth tribe. Over the course of the training they learned how to use data to gain insight, how to gather and rank ideas, how to craft hypotheses and how to run experiments. This training provided the basic knowledge of how to run experiments.
Experimenting in practice
Within the Retention Pod the aspiration was to run experiments at high tempo. We adopted the growth process advocated by Sean Ellis in the excellent Hacking Growth, supplemented with some of the great tools we acquired from our Growth Tribe training.
To begin with the team needed to decide on the initial focus for experimentation. Having done this, they held a brainstorm to come up with ideas. They then ranked the ideas and kicked off the first few experiments.
Gathering and ranking ideas with Growth Tribe Academy’s GrowsApp
Having got started, they were now able to adopt an ongoing process. Each week the team held a growth meeting. They would review the results of the previous week’s experiments, discuss learnings and insights, and pick which experiments to run next.
Planning, running and analysing experiments in the GrowsApp
In addition to the weekly growth meeting they would hold regular brainstorms to keep building a backlog of ideas.
In the case of our email marketing, we started with fairly simple experiments — testing subject line headings and email timings. Later they also started to test email content across both triggers and newsletters.
Managing experiments
One benefit of our training with Growth Tribe was access to their Grows App. This tool, developed by Growth Tribe, provides a way to gather and rank ideas, plan and visualise current and future experiments, and capture insights from past experiments. Everyone in the pod had access to it so it provided a great way to collaborate and share learnings.
Building a culture of experimentation
The pod model really helped drive the uptake of experimentation within the CRM squad, because they were able to learn from their more experienced product engineering colleagues.
Elsewhere experimentation was also being used to influence our product range development. Squads developing physical product ranges started to use data insights to influence the new ranges they developed.
As with optimising speed, there is still plenty of room to increase experimentation, but the crucial factor is that the value of experimentation has been realised and it now happens across the organisation. It is no longer a “tech thing”!
What’s next?
When I outlined the key outcomes we hoped to achieve from adopting business agility, I described how we wanted to be better, faster and happier. Thus far I have covered how we sought to be better and faster. In the next postI’ll talk about how we went about getting happier.
If you missed it, you can read Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7 and Part 8 here.
