27th of May was World Product Day and to celebrate I virtually attended the online Tokyo meet-up, which was fantastic not just for its content but for another surprising reason not relevant to this post. You too can watch the whole meet-up on YouTube.
What I loved about the talk itself was how it did not paint a picture of a perfectly idealised product process that is unreachable for most teams. Instead, it focused on practical, prioritised tips to make any product process a bit better.
The vast majority of us do not work within a perfect Discovery & Delivery culture. We work in situations that exhibit one of more of the old ways of doing things. Furthermore, the speaker, Todd Birzer, recognised not every team should be working in the new way. The old way can be appropriate in situations where there is a high degree of safety risk, or where it is expensive to fix mistakes.
In spite of this, there are things all product teams can learn from the new way of product development, even those bound (for sensible reasons or otherwise) by sequential processes.
Old vs. new
Product teams tend to work in one of two ways. Birzer uses the terms Stage-Gates and Discovery & Delivery. In my head I was substituting them with the frequently used terms waterfall and agile, but they are not exactly equivalent — not least of all because the terms ‘waterfall’ and ‘agile’ come with a lot of baggage.
‘Stage-Gates’ are where a project moves sequentially through the stages of:-
Scoping → Business case → Development → Testing → Launch
Each Stage is a phase of a new product or service development and each Gate is where product development is reviewed, go/kill decisions are made and the next-stage’s funding is allocated.
Stage-Gate processes are very common in companies around the world. For some types of products it can even be the right approach.
Discovery & Delivery
‘Discovery & Delivery’ is the newer approach. Rather than pushing the entire product’s development process through distinct steps, it breaks the product down into ideas that each, individually, go on their own journeys at their own speed through:-
Ideas → Discovery → Delivery → Optimisation
It works like a funnel: starting with a lot of ideas, picking a few of them to put through discovery, then filtering them further to choose only those that have tested very well to take to delivery.
Unlike Stage-Gates, Discovery & Delivery is non-linear. Ideas that test well will move onto delivery. At the same time, ideas that do not test well will go back to the beginning. New ideas will continuously be added. Often, all stages will be happening concurrently.
Discovery & Delivery is clearly a more lean, agile process. Requirements are defined immediately before (not months before) delivery. Very few resources are devoted at the beginning as you’re just quickly sorting through ideas. Significant resources are only allocated at the end when product-market fit has really been found.
Comparing the two processes
The Stage-Gates process forces requirements to mostly be defined upfront and decisions on what will be built will be made at the beginning. It might be a year later when the product is finally released and shown to customers.
With Discovery & Delivery, requirements are figured out as you go. You’re rapidly prototyping, learning as you go along, developing requirements at the same time as doing development.
Following a Stage-Gates approach risks being beaten by faster Discovery & Delivery competitors as they are more flexible and therefore better able to respond to customers.
Whereas those following the Discovery & Delivery process suffer risk of poor coordination as their progress is less easy to track and increased financial risk as the process is more chaotic.
How to make your waterfall process a little more agile
Not every team can (or should) move from Stage-Gates to Discovery & Delivery. Besides, it’s not realistic to try to change everything at once. Changing how people work, much like product management itself, is best done incrementally, but where to start?
There are 3 best practises every product team can learn and apply from the new process:
- Talking directly to real customers
- Testing product concepts
- Releasing incrementally
1. Talk directly to real customers
The first rule of excellent product management is interview customers early and often.
We should understand our customers in many ways. Customer satisfaction surveys, net promoter scores, third party research reports, product usage data, etc.
In particular we need to understand not just what the customer wants but what they are trying to achieve. The underlying needs your product is aiming to fulfil.
And because of that our most insightful source is direct conversations with customers.
Talk directly with your customers, not to (or through) your sales team
Customers are defined as the companies, groups and individuals who are benefiting from your product. Or competitors’ products. Or potential customers. Not your sales reps, not your sales channels.
Sales reps have a lot of bias — they will often filter the information from customers to highlight the demand for features they think they need to meet their own sales goals. We need to have product conversations with customers. We need to understand the customers without trying to make a sale at the end of the conversation. We need to test our product concepts. In order to do that, you need to talk directly to your customers and not go through sales channels.
How to interview customers remotely
Through video conferencing you need to be more disciplined. When you’re face-to-face you can be a bit more casual, go off on tangents because you have the personal rapport. It’s a little tougher when you’re on the video conference to develop that. You have to be more strict and focused. Your subjects might not be as patient with you. Stick to the time.
With a video conference it’s easy to invite a lot of colleagues but be careful. Make sure they’re not disruptive because if too many people are talking in at once it gets a little out of control.
Think about security. Use password protection. Ask your subjects not to show confidential information on their screen. Only record when you have permission.
2. Test your product concepts
We have many ideas for product enhancements. Some are excellent and some are worthless. The goal is to quickly screen ideas, discarding the bad and keeping the good.
Do not decide at the beginning on a set of features to build, then go through the full development cycle only to find out it doesn’t have a good product-market fit as this would be a huge waste of effort.
Instead, aim to filter out bad ideas and refine the good ideas with minimal investment by going through the test-feedback-test loop cycle, refining concepts to find product-market fit.
It’s a funnel process, you’re getting rid of a lot of ideas early on and only moving the ones that are truly testing well to the next stage.
Product concepts should start as simple as possible, such as a paper print out. Then a video. A clickable prototype. Only when you’ve gathered significant evidence that there is demand for your feature should you start to write code for beta software.
It’s never too late to start. Even if you’ve almost completely finished the new feature without any customer input, find a way to test it.
If you’re struggling to decide which approach to use to test, also don’t worry. Just pick one. Not every kind of user testing is ideal for every scenario, but imperfect user testing is better than no user testing.
3. Find ways to release incrementally
We often think it’s the first-to-market company that has all the advantage but it’s often the company that evolves the fastest that ends up winning in the long run.
Smaller and sooner is better than bigger and later.
Why do we want to release incrementally?
- Offer value to customers sooner
- Get revenue faster
- Course correct sooner if we are off-track
- Learn and evolve faster than our competitors
My own experience with FT.com resonated very strongly with this — I’m proud to say that we deployed code multiple times a day. New feature releases would be controlled by switches, instantly turning them on or off, or showing to customers through an A/B test. New feature ideas would typically be released this way every week or so.
A/B testing is one of the ways that product teams can accelerate the evolution of their product. But just looking at the data is not enough, you need a deeper understanding of the qualitative reasons behind it. Why are people leaning one direction or the other?
If you do just one thing differently after reading this post make it talk directly to real customers.
If you’re already doing that, then find ways to test your product concepts with customers and release new features incrementally.