Compare the same customer-success measure before and after a release, then inspect conversations that explain the change. The fictional example reviews V1.01, which clarified returns guidance in Main on November 8, 2026. Example dates use America/Toronto time.

Before you begin

  • The released Environment and release time, available in Environments.
  • Completed conversations and scored Evaluations in both review periods.
  • Unchanged Evaluation criteria across the comparison. A changed criterion changes what the score means.

Compare the release periods

1

Locate the release

Open Analytics, choose the released Environment, leave All versions selected and click Apply. Choosing only the new version would exclude earlier-version conversations from the comparison.

Choose dates spanning the release and hover its chart marker to inspect it. In this example, November 5–10 shows both sides of V1.01.

Locate V1.01 between the pre-release and post-release review days. Photo:
2

Set exact comparable windows

Use the date picker to select the three complete days November 8–10 after the release. The equal-length preceding period is November 5–7. Both use the same Environment and timezone; neither includes the unfinished current day.

The built-in change compares the selected period with the preceding period. Confirm these windows before treating that figure as a release comparison.

Compare November 8–10 with the preceding November 5–7 period. Photo:
3

Check outcomes, coverage and cost

Read Resolution rate, the custom policy Evaluation and the conversation counts. Compare workload and operating cost alongside the quality change.

  • Outcomes: use the same definition before and after the release.
  • Coverage: confirm that conversations in both periods were scored under the same criteria.
  • Workload and cost: check whether the amount or mix of work changed.

A post-release improvement is a finding to investigate; the chart alone does not establish that the release caused it.

4

Inspect the conversations behind the change

Open Evaluations → Return policy accuracy. Confirm the detail view’s Environment and dates, then inspect individual results from both periods. Check whether later responses follow the policy change and whether earlier failures still occur.

Review policy judgments from the same Environment and post-release dates. Photo:

For an unexpected result, use the conversation investigation workflow.

Record the release finding

Record the Environment, release time, exact windows, conversation counts, outcome rates and representative conversation evidence. State the observed change separately from an explanation for its cause.

The fictional data produces this finding for Main:

Resolution improved by 15.5 percentage points, while conversation volume fell. The policy score also improved, but low-scoring exceptions remain. Inspect those conversations before attributing the change to V1.01 or deciding whether it needs another revision.

Verify the comparison

  • Both periods use Main, the same timezone and unchanged scoring definitions.
  • The before and after windows contain complete days on the correct side of the release.
  • The finding includes individual conversation evidence and identifies any scoring gap.

Use the up and down arrow keys to select a result, Enter to open it, and Escape to close the search.