Measure impact of changes

This guide details how to use the Impact measurement tab in the Performance dashboard to measure how your merchandising changes affect site performance. The tab tracks the changes you make most often: ranking rules for queries and categories, synonyms, and redirects.

Prerequisites

Permissions: Your Users Admin must assign you the Insights reports editor permission.

Review merchandising changes

The Impact measurement tab logs every change automatically and compares performance after it with performance before it. Metrics for a change appear the day after you create or update it and stay in the tables for 45 days.

The system removes a change from the tables after 45 days. If you made no changes in the last 45 days—for example, during a holiday period—the tables remain empty.

1. Go to Insights > Performance dashboard and select the Impact measurement tab.

2. Changes are grouped into Ranking rules, Synonyms, and Redirects.

   

Search and filter rules

• Search for a rule, term, or redirect.
• Click Filter rules to filter the table. See Filter tables.
• Toggle Show trends ON to show the change against the period before it next to each value. See View trends.
• Click the download icon to export the table as a one-time CSV. See Export your data.

Measure impact of a specific change

Hover over a ranking rule or synonym and click View to open the details of that change. The details page includes:

• Impact analysis: The comparison method for the change.

• Summary: The details of the change.

• Metric cards: Revenue, Conversion rate, Average order value, Revenue per visit, Visits, Product page views, Add-to-carts, Add-to-carts rate, and Conversions, each with its trend against the compared period.

• Metrics chart: The selected metric for the current period and the compared period, plotted day by day.

• Breakdown: For multi-query/category rules and two-way synonyms, the performance of each term, shown below the chart.

  

  To assess a redirect, use its row in Redirects. The metrics measure the destination page, usually a category page, before and after the redirect. They don't measure the query itself.

Choose comparison period

Both periods are counted from the Date modified. The current period starts the day after the change, and the compared period ends the day before it. Select how to compare the periods in the Impact analysis drop-down:

• Automatic period (default): Compares the days since the change with the same number of days before it, up to 45 days. For example, 13 days after a change, you see Automatic period (13 days). The date pickers at the top of the page don't apply.

• Custom periods (Date selector): Keeps the start of the Current period fixed and lets you choose when it ends. Pick any Compared to range that ends before the change. Use it to compare against a longer period before the change, such as 1 month.

Read summary

Click the arrow on Summary to expand or collapse the change summary.

Compare metrics in chart

Select a metric card to plot it. Use Compare periods or Compare metrics to change how the chart plots the data. See Compare metrics in chart. Click the download icon to export the data as a one-time CSV.

Build impact measurement use cases

Pick the metric that matches the goal of your change, such as visits, add-to-carts, conversion rate, or revenue per visit.

Check if a ranking rule change performs well

See how a query or category performed after you re-ranked its results.

1. In Ranking rules, search and open the rule.
2. Select the metric card that matches your goal and compare the current period against the compared period in the chart.

Validate a new redirect

Check whether the traffic you redirect helps the destination page. For example, you redirect queries with no results to a sale page, or queries with few results to a category page so shoppers can browse.

1. In Redirects, find the redirect by its term or phrase.
2. Compare the destination page's Visits, Conversion rate, and RPV against the period before the redirect.

Compare a change against a longer period

Holidays, sales, or low-traffic days before a change can skew a same-length comparison. Compare against a longer period to even them out.

1. Open the change from its table.
2. In Impact analysis, select Custom periods (Date selector).
3. Set Compared to to a longer range before the change, such as the previous 1 to 3 months.

Analyze a change after 45 days

After 45 days, analyze the change in the Site search or Category tab. Create a custom scope for the affected queries or categories, and set the Current period to start the day after the change.