Last week, I saw Jeff Gothelf speak on Better Product Definition Through Lean UX and Design Thinking. Jeff Gothelf is a product designer and speaker who wrote Lean UX and is currently working at Neo.
We can’t know if our products will work or if customers will use them in the way that we intend until we actually put release them into the wild. As designers, it’s our responsibility to mitigate the risk of building things that don’t work by testing them early and often with actual customers. Very often, products end up over-engineered and under-tested. By combining the principles and practices of design thinking and Lean UX, we can mitigate much of this risk by framing our ideas as experiments and validating them as cross-functional teams.
What happens when you don’t validate value?
People often tell me, “I like Plancast, but I never have any plans to share.”
Mark Hendrickson
Plancast launched in 2010 as a social event sharing service. It launched on TechCrunch to terrific fanfare with great initial growth but ultimately flopped. In a post-mortem from the founder, he shared that they spent 5 months building before putting anything into the market. In their early adoption phase, they were excited by a rapid growth in vanity metrics: lots of signups and visits. But they were misled by these as they don’t necessarily represent engagement or repeat use. After signing up, most people didn’t do anything of value with the product.
They discovered quite a few issues after the fact. Social networks feed on vanity, and plans aren’t as psychologically rewarding to show off as photos. They chose to explicitly avoid an invitation feature so they weren’t directly competing with Evite and similar tools; when people shared events, it felt as though they were antisocially not welcoming others to join them.
In another example, Formspring let you ask interesting people interesting questions. Like Plancast, they had early success but later fizzled out. In a post mortem from one of their designers, one of their early projects was a social media sharing button much like Facebook and Twitter sharing buttons. The team spent two months building out infrastructure to handle traffic, designing flows to account for the difference from normal social sharing and their model, and cultivating business partners to implement the button. After launching, no one clicked the button. They put a lot of effort in to building a feature that no one found valuable.
The shift
There has been a major shift in the last decade. Software has become continuous: there is no final version, and it can be updated at any time. We can push updates to products at any time, which enables conversations with our customers in real-time through our software. For instance, Amazon pushes code to production every 11.6 seconds. They make minor changes and push them to a small portion of the user base, then sit back and observe how successful the changes are. If it’s working, they scale it out to others; if it fails, they can roll it back with little impact.
Key questions to ask
This shift leads to new questions we should ask to inform our process:
- How long should we wait before we launch?
- How do we define the right requirements for our product?
- How do we determine success? What signals are we looking for from the market?
Shipping no longer means success; the key is to discover whether what we’ve launched has value. This means a fundamental rethinking of defining requirements. We need to think of requirements as assumptions or guesses, and attempt to validate those assumptions through testing. Building requirements as anything but educated guesses is unfair to product managers and teams; they can’t foresee every possible result of a given requirement without first testing it either.
When you start to think of requirements as assumptions, our vernacular changes. Requirements become assumptions, “we know” changes to “we believe”, “let’s build it” becomes “let’s test it”. Learning first is crucial to building the right product.
Design thinking can help
Design thinking puts the customer at the center of the process. Conceived at IDEO, design thinking is a style of thinking with three components:
- Empathy for the context of a problem - This means understanding the customer, the situation they’re in, and the problem they’re trying to solve.
- Creativity in the generation of insights and solutions - Brainstorm a bunch of big ideas about how to solve the pain points.
- Rationality to analyze how well the solutions solve the problem - If the ideas don’t solve the problem, we stop working on them.
The ultimate goal of design thinking is to understand the customer’s problem and to solve it.
Now throw in a dash of Lean UX
Lean UX is inspired by the Lean Startup (written by Eric Ries) and Agile, both of which focus on mitigating the risk of building the wrong ideas by focusing on a minimum viable product. The object of Lean UX is to encourage doing so “in a collaborative, cross-functional way with less emphasis on deliverables and greater focus on shared understanding of the actual experience being designed.”
This means getting something into the customer’s hands quickly, in a way that the entire team is involved in the process. This helps everyone to understand why some solutions are valuable, why certain decisions are made, and ultimately gives every team member empathy for the customer and their problems.
Lean UX prioritizes learning over growth. Is this the right thing to work on? If it’s not, stop and find the right thing. The first step in the process is to declare assumptions - these are the things everyone on the team believes to be true about customers, their problems, and everything else related to potential solutions. Have everyone separately work through these assumptions and then come back together to share them.
Next, use those assumptions to build hypotheses with the following format:
“We believe that [building this feature] [for these people] will achieve [this outcome].
We will know we are successful when we see [this signal from the market].”
When you start to think about things in this way, the measure of progress changes from output (what you’ve produced) to outcome (a measurable change in customer behavior attributable to a specific feature). There’s a further metric called impact that is a high-level measure of the business’ health: NPS, revenue, profit, sales, etc. Every business measures these, but it’s often hard to correlate a change in impact as a result of outputs or outcomes.
Works as designed is no longer good enough
You can launch more features and they can still suck.
The ultimate result of these changes is a fundamental shift in thinking. The measure of success switches from “was it produced?” to “was it valuable?” We have to continually ensure that people want to use the product and that they enjoy using it.
Most companies currently manage by focusing on output because it’s easy. Managers can quickly determine whether something was shipped or not. It’s better to focus on delivering outcomes, but harder for management to measure success because it’s not binary. And while impact metrics are most meaningful to the business, they should not be used to measure the success of individual teams or features because it’s so difficult to tie them directly.
Focusing on outcomes allows teams to make decisions based on objective observations, rather than the subjective whims of individuals. When you collect enough data, you end up with three options:
- If the data reveals that it’s a bad idea, you should kill the idea.
- The data may indicate some success, but not as much as you’d like. At that point, you pivot. Keep one foot in the strategy and change your tactics.
- When you’re hitting the numbers you hope to achieve, that’s the time to double down on the idea and scale it out.
This thinking should also affect product roadmaps:
product roadmaps should be lists of questions, not lists of features
Kent Beck (@KentBeck) May 28, 2013
The easy parts of Lean UX include:
- Measuring: Analytics, usage, metrics
- Talking to customers: gain insights to “why” customers behave in specific ways
- Pausing and reflecting: team-wide discussion of learnings and progress
The hard part is actually changing course: admitting that what you attempted isn’t working and moving on to try the next idea. Being wrong should be okay, especially if the risk you take is smaller. Your aim is to mitigate risk by not creating things customers don’t want.
This is not just for designers
Lean UX is not an exercise just for designers. If everyone is involved, you get perspective from different viewpoints about the hypotheses you are testing: feasibility, usability, business objectives, etc. Everyone on the team empathizes with users and understands the why behind key decisions. Your teammates may have specific titles and roles, but they often have experience and expertise outside of those. Why limit them to their existing roles when they can potentially bring even more value?
How is this funded?
Think of cross-functional teams like an internal startup. Given a specific outcome to achieve, limit the team’s hypothesis and validation cycle to a specific time frame (a few weeks to a few months depending on the desired outcome) and budget. The team comes up with one or more hypotheses, validates the business model and customer value, and presents the results at the end of the time period. If successful, they continue scaling out or building on the hypothesis. This incremental funding allows for more testing and less sunk costs into exploring ideas that don’t work.
In summary
Defining the right product reduces wasteful time spent working on the wrong thing for customers, builds better team-wide momentum and shared understanding, and prevents teams from spending too much time and money on the wrong initiatives. This is done by shifting the way we work to:
- Assumptions, not requirements
- Outcomes, not output
- Cross-functional collaboration, not silos
- Ruthless testing, not egos
- Changing course, not fallacy of sunk costs
- Incremental funding, not annual planning
How can you focus on making these changes in your organization or team?
Want more? You can review the slides from his talk, watch the video, or get a copy of Lean UX.