> ## Documentation Index
> Fetch the complete documentation index at: https://docs.signalmd.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Compare versions

> Check whether a change helped: where the comparison lives, how to read each verdict, what to do next, and how to get it over the API.

When you publish a change in Retell, Signal compares production calls on the new version with calls on the version before. This guide shows where to read that comparison and what to do with each result. What each metric and verdict means is in [Versions](/concepts/versions).

## 1. Let calls arrive on the new version

Signal picks up a new version within about 10 minutes, or at once when you publish from **Build**. The new version starts with no calls and fills as calls arrive on their own. To bring in calls from the days since the change at once, use **Import more calls** on **Agent config** with a window that covers them. Calls already imported are skipped, so you can import again as often as you like.

## 2. Open the comparison

Open **Agent config**. **History** lists every version with its **First call**, **Calls** and **Got it done**, plus **Handed to a person** when any call ended in a transfer. A version whose configuration Retell never gave back is marked "no config".

Below it, each change has its own section titled with the two versions, such as **v7 → v8**:

* **Changed** lists what differs in the configuration, with the old and new text for prompts. When one side was not captured, it reads "Unknown: the configuration of v6 was not captured." and the section is tagged **Calls only**.
* **Effect** shows how many calls each version has, then a row per metric with both values and a verdict.
* **Issues that moved**, when there are any, lists up to 5 issues whose share of calls changed by a real amount.
* The tag next to the title is the conclusion, and the sentence under the table says what moved and which row decided it.

From an issue, **Compare versions** under **By version** opens the same page.

## 3. Read it in order

<Steps>
  <Step title="Check the calls">
    The **Effect** heading shows calls on each version, such as "(210 → 220 calls)". Each row has its own count: **Got it done** counts only calls where the caller spoke and the call has been read. Hover a value to see its count. A row without enough calls says **Not enough calls** rather than guessing.
  </Step>

  <Step title="Read the conclusion tag">
    **Got it done** leads. The guardrails and the issues that moved decide only when it did not change. **Handed to a person** never decides.
  </Step>

  <Step title="Read each row">
    A real change shows as a signed difference, green or red. Everything else shows a grey label (see below). Rows Signal has not finished reading on these calls are left out and named after "Not shown:" under the table.
  </Step>

  <Step title="Read the summary">
    It names the row that carried the conclusion. A conclusion carried by **Reply time** alone is a faster agent, not proof that more callers got what they came for.
  </Step>

  <Step title="Check the target issue">
    If the change was aimed at an issue, open it and read **By version**. The comparison is about the whole agent; **By version** is about the one problem.
  </Step>
</Steps>

## 4. Act on the result

### Row verdicts

| You see | What to do |
| - | - |
| A signed difference, such as **+10.0 points** | A real change. Read some calls on both versions before crediting your change, and check the traps below |
| **Within noise** | Treat it as unchanged. If you expected a large effect and the row has few calls, import more and look again |
| **Too small to matter** | A real movement, but too small to act on. Treat it as unchanged |
| **Not enough calls** | One version does not have enough calls in that row yet. Import more calls on the new version, or wait for traffic |
| **No earlier calls** | The earlier version has no calls in that row, so there is nothing to compare against |
| **Not comparable** | The two sides were read by different versions of Signal's analysis. Do not read the gap as a result until the older calls are read again |

### Conclusions

| Tag | What to do next |
| - | - |
| **Observed improvement** | Read a sample of calls on the new version. If an issue was the target, read its own verdict above **By version**, which waits for the grace period |
| **Observed regression** | Read the calls behind the row that got worse. If the change caused it, revert in Retell or fix forward, and compare again |
| **Mixed result** | Decide whether the trade is acceptable. The guardrail or issue that got worse often points at a new issue on the board |
| **No better or worse** | The change did not move production numbers by a real amount. That is common for a fix aimed at a small issue: check its **By version** share, or [test the fix](/guides/test-a-fix) directly |
| **Not comparable** | Nothing changed except rows that cannot be compared yet. Wait until the older calls are read again |
| **Not enough calls** | **Got it done** does not have enough calls on one version yet. The summary says how many each has |

Enough calls to get a verdict is not the same as enough to see a small effect. The fewer calls a version has, the larger a change has to be to show as real, so a modest improvement on a quiet agent can read **Within noise** for a while (see [Why a small change can read as unchanged](/concepts/versions#why-a-small-change-can-read-as-unchanged)). If you expect a small effect, plan to wait, and use [tests](/guides/test-a-fix) for an earlier answer.

## Common traps

<AccordionGroup>
  <Accordion title="The traffic mix changed between versions">
    A comparison is before and after, not a controlled test: every caller after the change meets the new version. If a campaign, a new phone number or a change upstream sent a different kind of caller, the numbers move whatever your change did. On **Overview**, select **Edit** and group the **Got it done** chart by **Use case** to see whether the mix shifted at the version marker. Editing changes the board for everyone who can see this agent.
  </Accordion>

  <Accordion title="A fix moved a number in the wrong lane">
    A holding line in the prompt shortens the silence the board measures, so **Reply time** improves. If the cause was a slow lookup, the caller still waits for the data. Check that the number that moved belongs to the [lane](/concepts/lanes) of the cause, and that the target issue's **By version** share fell.
  </Accordion>

  <Accordion title="A holiday or a seasonal spike">
    If the new version went live just before a spike or a quiet period, the comparison mixes your change with the calendar. Wait until the new version has a normal week of calls, then compare again.
  </Accordion>

  <Accordion title="Several changes in one version">
    **Changed** may list a prompt edit, a tool change and a new voice. The comparison cannot say which one moved a number. Ship one change per version when you can.
  </Accordion>

  <Accordion title="One changed guardrail among many">
    With several rows checked at once, one will now and then move by chance alone. A single guardrail moving, with nothing else, is weak evidence. Read its calls before acting.
  </Accordion>

  <Accordion title="An issue shows Regressed after the fix">
    A resolved issue regresses only when a call that started after it was resolved shows it again. Open the issue and check **By version** for that call's version: if it is the new version, the fix did not hold; if it is an older one, the change may not be live for every caller yet.
  </Accordion>
</AccordionGroup>

## Get the same comparison over the API

The [Signal API](/api/introduction) serves version comparisons to your own tools, such as a release script that checks a change after it ships. You need an API key; see [Authentication](/api/authentication).

```bash Terminal theme={null}
curl "https://first-signal-api.fly.dev/v1/agents/$AGENT_ID/versions/compare?from=7&to=8" \
  -H "Authorization: Bearer $SIGNAL_API_KEY"
```

[Compare two versions](/api-reference/versions/compare-two-versions) takes any two versions, not only neighbours. It returns the configuration changes between them (`changes` is null when either configuration was not captured, which is not the same as nothing changed) and the comparison of production calls in `comparison`. Its `result` holds the `conclusion` with the app's wording in `conclusionLabel`, the calls on each side, a row per metric, and the `narrative` the app prints under the table. `isRandomized` is always false: a version comparison is never a controlled experiment.

To list comparisons, newest first, use [List version comparisons](/api-reference/comparisons/list-version-comparisons), and fetch one with [Get a version comparison](/api-reference/comparisons/get-a-version-comparison). Get the agent ID from [List agents](/api-reference/agents/list-agents). From an AI assistant, the [MCP server](/mcp/overview) offers the same through its `compare_versions` tool.

## Next steps

<CardGroup cols={2}>
  <Card title="Test a fix" icon="flask" href="/guides/test-a-fix">
    Get an answer before callers meet the change.
  </Card>

  <Card title="Versions" icon="layer-group" href="/concepts/versions">
    What each metric and verdict means.
  </Card>
</CardGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.