THREE LAYERS OF BETTER PRODUCT DECISIONS

Throughout our lives, we make decisions of any kind. As always, we are questioning ourselves — am I doing it right or not? How can I make the best decision in the existing situation? What should I know before making any decisions? We are looking for these important answers to these questions to become great decision-makers in the end.

Based on her experience, Anna Buldakova (Product Manager at the Workplace by Facebook Machine Learning team) delivered an enriching talk from the BUILD stage at EMERGE 2019 about her journey to grow as a decision-maker and a product manager.

The cheapest and best tool for data gathering is Google search. Anna shared her own experience while she was working on a new product for sales and trying to understand what kind of problems should be addressed first. To do a very thorough search of trends in sales, she spent a couple of days in sales forums, Facebook groups, online communities, etc.

She was interested to see what people talk about, understand their concerns and complaints. Later on, she investigated the list of competitors who failed in the past or who will probably be competing with the product she was working on to have a clue about what people like or dislike about them. Lastly, Anna was researching the latest market trends — what is happening on the market in general, what are the changing forces there, if any new professions appearing, any changes in the job structure, mode of work, etc. All of that helped her greatly to get a better understanding of the domain, including sales-specific terminology.

Also, it’s important to remember that collecting the data is not just about collecting the data itself, it’s more about understanding the impact of it. Your goal is not to always make the right decision, but to invest the right amount of time into making this decision with regards to its importance. To sum up, the higher the impact of your decision is, the higher the investment in the decision-making process should be.

In the fascinating study done by Paul Slovic, a group of professional horse race handicappers were asked to make predictions about the outcome of a horse race. In the beginning, they were allowed to pick only five pieces of information and they were able to accurately predict the outcome of the race nearly 17% of the time, and their confidence in their decisions was around 19%. Not bad, right? But the most interesting thing happened when each handicapper was allowed to pick more data until they reached 40 pieces of the information and their confidence nearly doubled.

In fact, the accuracy of the predictions hasn’t changed. On the one hand, the additional amount of data made people believe that they’re making more informed decisions, but on the other hand, it didn’t make their predictions better.

Why is it a really bad situation?
1. You collect the data that is unnecessary and redundant.
2. You collect so much data, that, at some point, you might loose sight of what matters.
3. Eventually, you might fall into this analysis paralysis trap, and end up fearing to make a decision at all.

Then, how to understand how much time to invest and how many people have to be involved in the data gathering to make a decision? To answer this question, we first need to understand what the impact of this decision will be. Anna defines impact through the sum of negative and positive outcomes. What do we expect to happen when this decision will be made? Do we expect to unlock new markets? Do we expect to significantly improve user experience? Can this product potentially harm our brand or reputation? Is that a decision that can be reversed, or is it something that cannot be undone?

All of these questions matter when you are trying to understand how much time you are ready to invest in. The truth is that you never know for sure what the impact would be unless you invest at least some resources into receiving the information.

Intuition is a mechanism that allows to assess the impact, even before digging deeply into data. Let's consider two examples:

1. New product for sales that was mentioned before
2. Changing the color of a button on the existing website

Which of these examples will make a higher impact on your business? A new product can unlock new market opportunities; thus, companies should invest more resources in decisions related to it.

Firstly, you should understand how to handle the information you have. In the real life, you would always have to juggle situations based on your intuition. The question is how you develop this kind of intuitive expertise and when exactly to rely on it.

According to Daniel Kahneman, there are three conditions that need to be met:

1. Regularity. There should be some regularity in the world that you can pick up and learn. For example, in chess there are some rules that you can learn and the more you play the better and faster your decisions become
2. Lots of practice.
3. Immediate feedback. Daniel Kahneman says that you should know almost immediately whether you got it right or wrong.

What all of it means for us? First of all, product managers usually require at least some experience in the industry before becoming a professional product manager. The more complicated the product is, the more appreciated is the product manager's experience.

Second, and most important, despite all the experience, there are some areas where you just can't develop an intuition. Complex research problems have a very unique set of constraints and unknowns that are unlikely to be repeated in the exact same way. In this case, even if you've been working on such problems for years, you are unlikely to develop a very good intuition.

Sometimes people who work on new products are trying not only to make an assumption about the impact of solving the problem, but also to come up with a solution to this problem. According to D. Kahneman, that's exactly the case where we shouldn't do this. Moreover, it's his highest point of uncertainty that you possibly could have in your product. The impact of your decisions might be game-changing for your business. If we go back to the sales product example mentioned above, while developing a new product we shouldn't use intuition to define which features to build in there.

We should use our intuition only to recognize the situation we are in and trying to gain as much knowledge as possible to experimentation. Intuition doesn't give us a solution, but it will help us ask better questions and consider more possibilities and risks.

Both intuition and data orient towards the past or the present, and it's definitely important. But if you want to reshape the future, you need to be creative.

Following this definition proposed by Dr. E. Paul Torrance, Anna highlighted three components that should be developed to foster creativity:

1. Humility. Being humble is the first step towards being creative. It makes us vulnerable, but also receptive to new information. It makes us wonder and ask questions.
2. Diversity. A lot of discoveries and business strategies come from linking to previous independent areas. It means that sometimes instead of going deep, you should go broad.
3. Courage. You shouldn't be afraid to look stupid when you are questioning conventional wisdom or foundational concepts, and you should always be ready to fail.

In a well-defined situation, you can rely on intuition and data. But the more uncertainties you have, the more you need to be creative, the more you need to go back again and start from the first principles. Creativity is one of the most important skills of the 21st century. But it isn't something that you can just go and learn. It's a mindset and a way of thinking that you need to adopt. Thus, the next time you have a very complex problem to deal with, you should try to apply creativity to it.

Adam Brandenburger (Director of Shanghai creativity and innovation program) suggests four creative approaches to get you started:

1. Contrast. What pieces of conventional wisdom are ripe for contradiction? Consider the video rental industry in 2000s. A company called Blockbuster is a leader in this industry and its model is based on two assumptions. The first one is that people rent videos in locations near home and the second one is that inventory must be limited, because new videos are expensive. At the time an ambitious startup called Netflix decided to challenge those assumptions. Why do you need a physical location, isn't mailing videos  much cheaper and more convenient? And why do new videos have to be expensive? Maybe you could reach a deal with studios and have a revenue-sharing deal with them. Doing those two changes enabled Netflix to remake the industry.

2. Combination. How could you connect products or services that have been separated like social messaging and financial services? At some point WeChat decided to build and integrate a mobile payment platform to enable its customers to sell and buy products within their social networks. As we know, it all worked out.

3. Constraint. How could we turn liabilities or limitations into opportunities? When IKEA just started, they were working with a lot of other manufacturers to build their inventory. One of their policies was low prices for their users. No wonder that at some point, the manufacturers decided to boycott IKEA and just refused to continue the contract with them. However, at that right moment, the founder decided to design items in-house and came up with these universal styles that IKEA is famous for now.

4. Context. How can far-flung industries, ideas of disciplines shed light on our most pressing issues? All of you know at least one example from nature or animal world that inspired technological innovation. For example, right now AI researchers look at how children learn in order to inspire and inform machine learning processes.

Making decisions is a very complex process. What even more difficult is that you never know what would have changed if you decided otherwise. Nevertheless, Anna shared that using these three components — data, intuition, and creativity, made her decision making process more weighted and multi-faceted.

EMERGE Team hopes this article will be useful to you and your team. Stay tuned! New blogposts are coming soon! ;)