Product Goals and Metrics: Part 5

Defining multiple OKRs and understanding the meaning and trade-offs of goal-setting

In Part 1,2, 3 and 4, we have covered the following topics:

• defining a clear vision for your product
• selecting key customer outcomes and corresponding “external” metrics for the product
• defining “internal” metrics that help move “external” metrics and achieve the customer outcomes
• defining a goal or OKR for the product

In Part 5, we will define another objective for the product, talk about tracking OKRs, and the benefits and trade-offs of goal-setting.

In the previous story, we defined the following OKR:

Objective: Improve the percent of “search queries that lead to a click on the first search result” by 20% by end of second quarter

Key Results:

• Improve page load time by 50% by week 6 of the quarter
• Improve search query understanding accuracy by 25% by week 8 of the quarter
• Improve the matching and ranking algorithms to account for customer preferences by week 10 of the quarter

In this story, we will define a second objective for ourselves. We will dive into another metric that we had looked at earlier i.e. “percent of search sessions that lead to a search result added to cart”.

First, we need to understand the metric. We know when users search or query, they can scroll and view search results, filter by brand or price, click on results and add them to cart. A lot of grocery shopping sites allow customers to add to their cart from the search or home page itself, while others only allow this action on detail pages. Let’s assume the business in consideration allows both. This means, the user could add a search result to their cart from the search page or wait to click on it, visit the details page, maybe visit a few details pages, and then add one or more items or objects to cart.

This metric depends on a variety of factors:

• the page load time for search and details pages and percent of page errors or failures
• the relevance of the results shown on search
• the details of the result shown on search and the detail page that help in purchase decision making
• the tools needed to add to cart; for example, selecting quantity, size or other variables

Before setting objectives and key results, one must evaluate each of the above to understand the potential and possibilities of improvement. For example, if we already have separate OKRs for load times and relevance, we should understand the impact of those improvements on “percent of search sessions that lead to a result added to cart”. Once we are good with those metrics being taken care of, we will focus on improvements in other elements of decision making including detail information and tools. We might have done previous experiments or analysis to understand the impact of these on the metric in consideration. For example, a recent experiment to highlight top search results as “best sellers” might have caused more users to click and purchase those products, thereby, improving the metric we are talking about. We know that customers do not want to scroll much and they typically like to make purchase decisions from the top results. Following is a version of our second objective.

Objective: Improve the percent of “search sessions that lead to a search result added to cart” by 20% by end of second quarter

Key Results:

• Highlight best sellers, top rated and top deals on the search results and the details page to improve the above metric by 10%
• Improve coverage for add-to-cart button on search pages to 100%
• Move the ratings and reviews module to top of the details page (so customers have to scroll less) and improve module interaction by 25%

The new OKR aim to make the product better. Ideally, product owners should or would have a few OKRs for the quarter or the year that they can chase. Once you have defined your OKRs, the following are important:

1. Sharing them with the team, organization and stakeholders, so there’s full transparency on the goals we are going after — this helps build trust and excitement around the product
2. Tracking the goals at least quarterly, if not more often, to ensure the team and the product is on the right path to success and we address any gaps or challenges
3. Pivoting or making changes as and when necessary; on occasions when one learns there is a better metric to go after or if it is not working, it is important to evaluate and if needed, pivot to a different metric. after all, goals exist so we can make the product better. in addition, companies change strategy often and that might require product owners to focus on new opportunities and look at customer value in new ways. this will also require a re-thinking of goals.

Goal-setting is a tedious but a very rewarding process. Without goals for our products, our teams and ourselves, we would not have clarity on where we are headed and without clarity, it is tough to invest time on the right problems and activities and it is impossible to bring everyone along and create value for users and business. As you and your team engage in this process, have fun with it, challenge status-quo and maybe define something new and better for all of us to use.

This brings me to the end of the five part series on product goals and metrics. Thank you for reading it and I hope you enjoyed the stories.