︎

Benchmark Testing Process

Establishing regular benchmark testing as an internal process and a way to track metrics across The North Face’s digital products to deliver continuous value against them.



Background

Within The North Face work has traditionally been brought to the design team with the idea of creating specific tools. I brought this project to our leadership team with the mission of shifting how the team generates work, moving from a process based on requested outputs, to one that identifies user behaviors and generates work based on shifting or optimizing them.


Role

My role involved conducting interviews with stakeholders to determine what products and tasks across the site to focus on. Working with site testing coordinators and the analytics team to collect measurements and conduct testing. Partnering with teams to interpret findings and prioritize work.


Process

I created a process overview that would map the individual phases of the benchmarking program. For the purposes of establishing the practice, most of the effort in this body of work revolved around determining what to measure and how to measure it. Once that was completed measurements were collected to establish a baseline. 

Based off NN Group’s benchmarking vertical

Future work in this program will largely happen in steps 3-5 using the findings found here as a baseline to gauge performance and identify areas for product owners and stakeholders to generate work against.


Measurements

The goal of this step was to determine what 4-5 actions or objectives we want users to take or complete when they reach the site. I conducted interviews with stakeholders and product owners to answer the following questions, then worked with leadership to prioritize the top tasks.

  1. What user group to target?
  2. What product(s) to focus on?
  3. Which tasks or features to measure?

Product to evaluate
Loyalty
BOPIS
Favoriting
Product filters
Potential Task
Signing up for loyalty
Make a purchase using BOPIS
Favoriting several items
Make a purchase using product filters

Using Google’s HEART framework I selected metrics that can be repeatedly collected to track each task.


Outcomes

After determining which tasks to track, we collected measurements. I am using product filters as an example to show tested and tracked metrics for an individual task.

Product
Product filters
Task
Make a purchase using product filtering
Metric
Time on task
Click rate
Methodology
Qualitative user testing
Analytics

I worked with the analytics team to understand click rate of the filters and conducted unmoderated user testing to gather qualitative insights. Testing suggested that despite expressed interest in temperature rating as a product characteristic, users engagement with the filter was low as a result of unclear language. Analytics supported this showing temperature rating as the third most clicked filter(30%) but with a highly skewed click distribution within the sub-categories warm(>9%) and warmer(>9%) warmest(48%).



This established metrics for the click rate on product filters to serve as a baseline. Along with insights from our usability testing we can now write a user problem statement to create work against with clear metrics to target.


Future Iterations

A user problem statement and associated metrics allows stakeholders and product owners a way to create a roadmap for product improvements across the site. Test results and analytics for future iterations of product filters can be looked at in relation to our baseline metrics collected here.


Process Impact

The North Face collects a lot of data and analytics across it’s digital properties. Harnessing this information to guide work conducted by design teams and others to improve the user experience is still being realized. This work will help teams use metrics to identify areas of need, frame problems and track the delivered value of iterations through repeated testing.