Product Goals and Metrics: Part 3

Defining key internal metrics to achieve your customer outcomes

In Part 1 and 2, we have covered the following topics:

• defining a clear vision for your product
• selecting key customer outcomes and corresponding “external” metrics for the product

In Part 3, we will discuss what we can call “internal” or “secondary” metrics for the product. Examples include “site load time” or “error pages returned”.

In order to define “internal” metrics, it is important to connect them to the vision and the primary customer outcomes we have talked about in Part 2 of the series. The metrics we had outlined include:

1. Percent of sessions which use the search engine
2. Percent of search queries that lead to a click on the first search result
3. Percent of search sessions that lead to a search result added to cart

Each of these metrics depends on several others that define the overall customer experience. For example, let’s take first one “percent of sessions which use the search engine”. We could assume a large percent of customers landing on a retailer site are interested in finding or purchasing a product but not all. Some might be looking for the status of their order they just placed, others might be looking for at their store’s opening hours, while some might simply want to see if they have any deals running. If we believe the best way for customers to find any of this information is search, we could say close to 100% site sessions must use the search engine.

In addition, most sites have users directly opening the site or app to use it and those that come in via emails, external advertising campaigns, and from search engines like Google and Bing. A lot of this traffic might be transient. They might not find what they need or might have no familiarity with the brand and might leave immediately. Hence, we need to understand the nuances of each metric and what impacts the numbers, before we set goals. Having said that, if the search engine is amazing i.e. it is fast, simple and easy to use, and gets customers what they need, it will have a positive impact on customers and they will use it more.

The first metric above depends on multiple internal factors including:

1. speed of the product or “time it takes for customers to completely see the top or all search results” or “percent times search does not load or throws an error”; this, in turn, depends on how the search engine is built, the technology stack used, what device and bandwidth the user is on, and a variety of other factors.
2. the quality of the search results or “relevance of top results”; this depends on how good we are at understanding user intent and matching it with their preferences and the best products. it also depends on how well we rank these matches putting the best of the best at the top.
3. the ease of use of the product; this could be measured in several ways including “user satisfaction scores” in surveys or qualitative studies, or “number of steps it takes for search users to click on a result” where steps could be many activities including querying, sorting, filtering and more. the number of steps would vary by the shopping category, the device the user is on and more. hence, it is important to look at these metrics from those angles too.

The above metrics are a few examples of “secondary” or “internal” metrics that product owners need to think about. In order to define goals, both “external” and “internal” metrics are important. “External” metrics help define clear customer outcomes while “internal” ones define how to achieve them. Even “internal” metrics will have several smaller metrics helping move it in the right direction. Depending on what part of the product one owns, it will key to understand these and find which ones fit in one’s situation.

In Part 4, next, we will start setting goals for the product in consideration and walk through an example.