> ## Documentation Index > Fetch the complete documentation index at: https://docs.signalmd.dev/llms.txt > Use this file to discover all available pages before exploring further. # Replace a test's spec > Replaces what the spec covers and makes a new revision. An edit that changes what is played or checked clears the calibration, so the test stops gating until calibrated again. An archived test is refused (422). ## OpenAPI ````yaml /api-reference/openapi.json patch /v1/agents/{agentId}/tests/{testId} openapi: 3.1.0 info: title: Signal API version: 1.0.0 description: >- Signal reads a voice agent's production calls and says what to fix, then tests whether the fix worked. This API exposes the same data the web app shows: agents, calls and their traces, signals, issues, versions and their comparisons, tests, and test runs. Authenticate with `Authorization: Bearer sig_...` (an API key from the API keys page). Read keys may call every `read` operation; write keys may also call `write` operations, within their monthly budget. Each operation's scope is in `x-signal-scope`. Errors share one envelope (`Error`), lists share one cursor scheme (`limit`, `cursor`, `hasMore`, `nextCursor`), and every response carries `Request-Id` and `RateLimit-*` headers. servers: - url: https://first-signal-api.fly.dev description: Production - url: http://localhost:43124 description: Local development security: - bearerAuth: [] tags: - name: Account description: Who the token belongs to and what it may do. - name: Agents description: Voice agents connected to the workspace. - name: Calls description: Production calls, their traces and per-call signals. - name: Signals description: What Signal noticed across calls. - name: Issues description: 'Problems worth fixing: where, why, and what to change.' - name: Versions description: Agent versions and how each one did. - name: Comparisons description: Stored comparisons between two versions. - name: Tests description: Tests, test sets, personas and coverage. - name: Runs description: Test runs and the attempts inside them. paths: /v1/agents/{agentId}/tests/{testId}: patch: tags: - Tests summary: Replace a test's spec description: >- Replaces what the spec covers and makes a new revision. An edit that changes what is played or checked clears the calibration, so the test stops gating until calibrated again. An archived test is refused (422). operationId: updateTest parameters: - schema: type: string format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: The agent's id. required: true description: The agent's id. name: agentId in: path - schema: type: string format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: The test's id. required: true description: The test's id. name: testId in: path requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateTestRequest' responses: '200': description: The test as stored. content: application/json: schema: $ref: '#/components/schemas/TestWriteResult' '400': description: 'Invalid request: a parameter or the body failed validation.' content: application/json: schema: $ref: '#/components/schemas/Error' '401': description: Missing, invalid or revoked credentials. content: application/json: schema: $ref: '#/components/schemas/Error' '403': description: A read-only key, or a write to the shared demo agent. content: application/json: schema: $ref: '#/components/schemas/Error' '404': description: >- The agent, or a resource under it, does not exist or is not visible to this key. content: application/json: schema: $ref: '#/components/schemas/Error' '422': description: The test is archived; restore it before editing. content: application/json: schema: $ref: '#/components/schemas/Error' '429': description: Rate limited. See Retry-After. content: application/json: schema: $ref: '#/components/schemas/Error' '500': description: Server error. The message is generic; quote the request id. content: application/json: schema: $ref: '#/components/schemas/Error' x-codeSamples: - lang: bash label: curl source: >- curl -X PATCH "https://first-signal-api.fly.dev/v1/agents/6f1d2c3b-4a5e-4f60-8b7a-9c0d1e2f3a4b/tests/3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f" \ -H "Authorization: Bearer $SIGNAL_API_KEY" \ -H "Content-Type: application/json" \ -d '{"spec":{"specVersion":1,"source":{"kind":"custom","basedOnTestId":null,"basedOnCallId":null},"caller":{"persona":"golden-cooperative","goal":"Ask what the cancellation policy is","lines":["What is your cancellation policy?"],"behaviours":[]},"fixtures":{"lookup_policy":[{"response":{"policy":"Free cancellation up to 24 hours before."}}]},"criteria":[{"text":"The agent states the cancellation window","decidedBy":"judge"}]}}' components: schemas: UpdateTestRequest: type: object properties: spec: type: object additionalProperties: {} description: >- A TestSpec (see the TestSpec component and GET .../tests/schema). Checked as a whole: a refused spec names the first problem's path in `param`. required: - spec example: spec: specVersion: 1 source: kind: custom basedOnTestId: null basedOnCallId: null caller: persona: golden-cooperative goal: Ask what the cancellation policy is lines: - What is your cancellation policy? behaviours: [] fixtures: lookup_policy: - response: policy: Free cancellation up to 24 hours before. criteria: - text: The agent states the cancellation window decidedBy: judge TestWriteResult: allOf: - $ref: '#/components/schemas/Test' - type: object properties: warnings: type: array items: $ref: '#/components/schemas/SpecProblem' description: Problems that did not stop the spec being stored. unfixturedTools: type: array items: type: string description: >- Tools left without a mocked answer, for a test made from a call. They answer with a fixed 'not available' error. required: - warnings - unfixturedTools Error: type: object properties: error: type: object properties: type: $ref: '#/components/schemas/ErrorType' code: type: string example: agent_not_found description: The specific reason; stable within v1. message: type: string example: Agent not found. param: type: - string - 'null' description: The request field at fault, for a validation error. requestId: type: - string - 'null' example: req_8c1f0e2b4a6d9e3f7a5c1b0d quota: type: object properties: metric: type: string example: test_calls description: analysed_calls, agents, copilot_messages or test_calls. limit: type: integer used: type: integer plan: type: string example: free_org description: The workspace's plan slug. upgradePath: type: string example: /billing description: Where in the web app the plan is changed. required: - metric - limit - used - plan - upgradePath description: >- Present when code is quota_exceeded: the plan limit the request would pass. required: - type - code - message - param - requestId required: - error example: error: type: not_found_error code: agent_not_found message: Agent not found. param: null requestId: req_8c1f0e2b4a6d9e3f7a5c1b0d Test: allOf: - $ref: '#/components/schemas/TestSummary' - type: object properties: spec: $ref: '#/components/schemas/TestSpec' resolvedCriteria: type: array items: $ref: '#/components/schemas/ResolvedCriterion' description: The criteria as they will be checked against the current brief. required: - spec - resolvedCriteria SpecProblem: type: object properties: path: type: string description: >- Where in the spec, as a JSON path: criteria[2].ref, fixtures.lookup[0].args. code: type: string description: >- The problem's kind, such as unknown_brief_ref, unknown_tool, personal_data. message: type: string severity: type: string enum: - error - warning description: An error stops the spec being stored; a warning does not. required: - path - code - message - severity ErrorType: type: string enum: - invalid_request_error - authentication_error - budget_error - permission_error - not_found_error - conflict_error - rate_limit_error - api_error TestSummary: type: object properties: object: type: string enum: - test id: type: string format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f title: type: - string - 'null' description: What the test list calls it. Null on tests made before titles. source: type: string enum: - production - custom - probe description: >- production: a stored call replayed. custom: written as a spec. probe: written for a brief line no call exercises; never gates. lane: type: - string - 'null' enum: - hearing - understanding - flow - speaking - integrations - null description: >- The lane a defect lives in: hearing, understanding, flow, speaking or integrations. intentSlug: type: - string - 'null' description: The caller's intent the test covers. tier: type: string enum: - text - phone description: text runs a simulated chat; phone is not built yet. sourceCallId: type: - string - 'null' format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: The production call it was made from, or based on. sourceIssueKey: type: - string - 'null' description: The issue it reproduces, for a test made from an issue. persona: type: object properties: slug: type: - string - 'null' description: >- The persona slug of the test's own caller. Null when the caller had none. label: type: - string - 'null' required: - slug - label description: The caller the test plays when a run does not name personas. revision: type: integer description: Bumped by every accepted edit. GET .../revisions has each one. expect: type: string enum: - pass - fail description: >- What the test claims: an issue test expects the defect (fail) until its fix ships. guard: type: boolean description: >- Marked a guard test: it runs 3 times every release and passes only at 3 of 3. gates: type: boolean description: >- Whether its results count toward a release. Only a calibrated test in the current scoring epoch gates. gateReason: type: string description: Why it gates or does not, in a few words. written: type: boolean description: >- Signal generated the caller (a generated test). Such a test reports and never gates. calibration: $ref: '#/components/schemas/TestCalibration' inRelease: type: boolean description: In the release set, which runs on every new version and from CI. testSets: type: array items: type: object properties: id: type: string format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f name: type: string required: - id - name description: The test sets it belongs to. tags: type: array items: type: string criteriaCount: type: integer description: Criteria lines after resolving against the current brief. fixtureCounts: type: object properties: total: type: integer description: Tools the test answers for. recorded: type: integer description: Answered from the source call's own recording. fromOtherCall: type: integer description: Answered from another stored call. ownerSet: type: integer description: Answered with a response someone wrote. none: type: integer description: 'No answer: the tool returns a fixed ''not available'' error.' required: - total - recorded - fromOtherCall - ownerSet - none description: Where each tool's mocked answer comes from. lastAttempt: type: - object - 'null' properties: result: type: - string - 'null' description: passed, failed, flaky, error or drift; null while unscored. startedAt: type: string format: date-time example: '2026-10-02T14:03:07.120Z' required: - result - startedAt description: >- The latest attempt with the test's own caller. Null when it never ran. archivedAt: type: - string - 'null' format: date-time example: '2026-10-02T14:03:07.120Z' description: Set when retired. An archived test is not run. createdAt: type: string format: date-time example: '2026-10-02T14:03:07.120Z' updatedAt: type: string format: date-time example: '2026-10-02T14:03:07.120Z' required: - object - id - title - source - lane - intentSlug - tier - sourceCallId - sourceIssueKey - persona - revision - expect - guard - gates - gateReason - written - calibration - inRelease - testSets - tags - criteriaCount - fixtureCounts - lastAttempt - archivedAt - createdAt - updatedAt example: object: test id: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f title: Books a cleaning when the lookup is slow source: production lane: integrations intentSlug: book_appointment tier: text sourceCallId: 0b9e8d7c-6f5a-4b3c-9d2e-1f0a9b8c7d6e sourceIssueKey: tool_timeout:check_availability persona: slug: '@source' label: The call's own caller revision: 2 expect: pass guard: false gates: true gateReason: 'Calibrated: reproduces the failure on version 7 and passes on 8.' written: true calibration: state: stable expect: pass version: 8 k: 3 'n': 3 results: - passed - passed - passed runId: d1e2f3a4-b5c6-4d7e-8f9a-0b1c2d3e4f5a revision: 2 epoch: simulatorModel: claude-haiku-4-5 judgeModel: claude-sonnet-4-5 judgePromptHash: 9f2c1a7e scorerVersion: '2026-09-18' at: '2026-10-02T11:07:41.000Z' reason: Passed 3 of 3 on version 8. inRelease: true testSets: - id: c7d8e9f0-a1b2-4c3d-8e4f-5a6b7c8d9e0f name: Release tags: - booking criteriaCount: 2 fixtureCounts: total: 1 recorded: 1 fromOtherCall: 0 ownerSet: 0 none: 0 lastAttempt: result: passed startedAt: '2026-10-02T11:01:15.000Z' archivedAt: null createdAt: '2026-09-20T16:12:00.000Z' updatedAt: '2026-09-28T09:40:31.000Z' TestSpec: type: object properties: specVersion: type: number enum: - 1 testId: type: string format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: Output only. revision: type: integer description: Output only. source: type: object properties: kind: type: string enum: - production - custom - probe description: >- production: a stored call replayed. custom: written as a spec. probe: written for a brief line no call exercises; never gates. basedOnTestId: type: - string - 'null' format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: >- The test whose start, masking map and issue criterion this one borrows. basedOnCallId: type: - string - 'null' format: uuid pattern: >- ^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$ example: 3f1c2d4e-5a6b-4c7d-8e9f-0a1b2c3d4e5f description: The stored call it is based on. briefLine: type: - object - 'null' properties: section: type: string enum: - out_of_scope - guardrails - intent key: type: string text: type: string required: - section - key - text description: A probe's brief line. generated: type: - object - 'null' properties: generationId: type: string kind: type: string enum: - from_call - written cell: type: string required: - generationId - kind - cell description: Set when a test generation made it. Never read from input. required: - kind - basedOnTestId - basedOnCallId intentSlug: type: - string - 'null' lane: type: - string - 'null' enum: - hearing - understanding - flow - speaking - integrations - null description: >- The lane a defect lives in: hearing, understanding, flow, speaking or integrations. tier: type: string enum: - text - phone start: type: object properties: direction: type: string enum: - inbound - outbound agentSpeaksFirst: type: - boolean - 'null' variables: type: object additionalProperties: type: string description: Dynamic variables the call starts with, personal values masked. metadata: type: object additionalProperties: {} required: - direction - agentSpeaksFirst - variables - metadata caller: type: object properties: persona: type: string description: >- A persona slug from GET .../personas, or @source for the source call's own caller. goal: type: string lines: type: - array - 'null' items: type: string description: >- What the caller says. Null: the source call's own lines, read at run time. behaviours: type: - array - 'null' items: type: object properties: kind: type: string enum: - refuses - asks_for_person - off_topic - repeats - short_answers - silent_after_question - profile atLine: type: - integer - 'null' description: 1-based caller line; null for the whole call. detail: type: string required: - kind - atLine - detail description: 'Scripted caller behaviour. Null: the source call''s own.' required: - persona - goal - lines - behaviours fixtures: type: object additionalProperties: type: array items: $ref: '#/components/schemas/TestSpecFixture' description: Tool name to its mocked answers. criteria: type: array items: type: object properties: id: type: string description: Stable across revisions. ref: type: string description: A brief reference, goal or issue. Exactly one of ref and text. text: type: string polarity: type: string enum: - must - must_not decidedBy: type: string enum: - detector - judge - unmeasured description: >- What decides the criterion: a check on the transcript and tool calls, a reading of the conversation, or nothing a text run can measure. required: - id - decidedBy expect: type: string enum: - pass - fail guard: type: boolean sets: type: array items: type: string description: Names of test sets it belongs to. tags: type: array items: type: string title: type: - string - 'null' required: - specVersion - source - intentSlug - lane - tier - start - caller - fixtures - criteria - expect - guard - sets - tags description: >- A test as a JSON document: who calls, what each tool answers, and what must hold. GET .../tests/schema has its JSON Schema; POST .../tests/validate checks one. ResolvedCriterion: type: object properties: id: type: string criterionId: type: string description: >- The spec criterion it came from. A list reference resolves to one line per item. text: type: string description: The line checked, as the current brief words it. polarity: type: string enum: - must - must_not decidedBy: type: string enum: - detector - judge - unmeasured description: >- What decides the criterion: a check on the transcript and tool calls, a reading of the conversation, or nothing a text run can measure. source: type: object properties: kind: type: string enum: - brief - goal - issue_reversed - stated required: - kind additionalProperties: {} description: >- Where the criterion comes from: a brief reference, the call's goal, the source issue reversed, or a line written in the spec. changed: type: boolean description: >- The brief line it points at is no longer in the brief. Never met until someone fixes it. required: - id - criterionId - text - polarity - decidedBy - source - changed TestCalibration: type: - object - 'null' properties: state: type: string enum: - uncalibrated - calibrating - reproduces - does_not_reproduce - stable - unstable - inconclusive - not_run description: >- Whether the test gates: only `reproduces` (an issue test that fails 2 or 3 of 3 on its source version) and `stable` (passes 3 of 3) do. expect: type: string enum: - pass - fail description: What calibration checked the test against. version: type: - integer - 'null' description: The agent version it was calibrated on. k: type: integer description: Runs that behaved as expected. 'n': type: integer description: Runs scored. results: type: array items: type: string description: Each calibration run's result, in order. runId: type: - string - 'null' description: The calibration run. revision: type: - integer - 'null' description: >- The test revision it measured. Null on calibrations stored before revisions were recorded. epoch: $ref: '#/components/schemas/ScoringEpoch' at: type: string description: When it was decided (ISO 8601). reason: type: string description: Why the state is what it is, in one line. required: - state - expect - version - k - 'n' - results - runId - revision - epoch - at - reason TestSpecFixture: type: object properties: args: description: >- The arguments this answer is for. Absent or null: the answer for any arguments no other entry matches. response: description: What the tool returns. success: type: boolean description: Whether the tool call succeeds. Default true. provenance: type: object properties: from: type: string enum: - source_call - other_call - owner callId: type: string required: - from description: 'Read only: where the answer came from. Ignored on input.' ScoringEpoch: type: - object - 'null' properties: simulatorModel: type: - string - 'null' description: What played the caller. judgeModel: type: - string - 'null' description: What scored the criteria. judgePromptHash: type: - string - 'null' description: >- Identifies how results were scored. Results with different values were scored differently. scorerVersion: type: - string - 'null' description: The version of Signal's scoring. required: - simulatorModel - judgeModel - judgePromptHash - scorerVersion description: >- What scored a result. Results scored in different epochs are not compared. securitySchemes: bearerAuth: type: http scheme: bearer description: An API key (`sig_...`), or a session token from the web app. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.