Skip to content

Reference

API: Datasets and Comparisons

Upload or replay Datasets and compare two releases over them.

Part of the API reference: requests carry a personal access token as Authorization: Bearer <token>, and bodies and answers are JSON.

Datasets

GET /v1/organisations/{org}/decisions/{slug}/datasets

List Datasets

Path

  • orgRequiredstring
  • slugRequiredstring

Answer (200)

  • datasetsarray
Terminal
curl "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/datasets" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

POST /v1/organisations/{org}/decisions/{slug}/datasets

Upload a Dataset

JSON { name, rows }, or JSONL as application/x-ndjson with ?name=.

Path

  • orgRequiredstring
  • slugRequiredstring

Body

  • nameRequiredstring
  • rowsRequiredarray
    One input (a JSON object) a row.

Answer (201)

  • idstring
  • namestring
  • sourcestring
    "upload": rows sent by a person or agent; "replay": a snapshot of recorded production calls.One of upload, replay
  • replayFilterobject or null
    How a replay Dataset chose its recorded runs.
  • replayFilter.environmentstring
  • replayFilter.versionNumberinteger or null
  • replayFilter.sincestring or null
    ISO times bounding when the recorded calls were answered.
  • replayFilter.untilstring or null
  • replayFilter.limitinteger
  • rowCountinteger
  • skippedNotStoredinteger
    For a replay: matching recorded calls left out because their input is no longer stored.
  • uniqueInputCountinteger
  • bytesinteger
  • createdByinteger
  • createdByNamestring
  • createdAtstring
  • refusedarray of { "line": integer, "message": string }
    Rows that were not JSON objects (or were too large), by line; the rest were kept.
Terminal
curl -X POST "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/datasets" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"Refunds","rows":[]}'

POST /v1/organisations/{org}/decisions/{slug}/datasets/replay

Replay production calls as a Dataset

Snapshots recorded production calls. Owners and admins (and full-scope tokens) only.

Path

  • orgRequiredstring
  • slugRequiredstring

Body

  • environmentOptionalstring
    The environment whose recorded calls to snapshot; default "production".
  • versionNumberOptionalinteger
    Only calls answered by this version.
  • sinceOptionalstring
    ISO time; or days for the last N days.
  • untilOptionalstring
  • daysOptionalinteger
  • limitOptionalinteger
    At most this many of the most recent calls.
  • nameOptionalstring

Answer (201)

  • idstring
  • namestring
  • sourcestring
    "upload": rows sent by a person or agent; "replay": a snapshot of recorded production calls.One of upload, replay
  • replayFilterobject or null
    How a replay Dataset chose its recorded runs.
  • replayFilter.environmentstring
  • replayFilter.versionNumberinteger or null
  • replayFilter.sincestring or null
    ISO times bounding when the recorded calls were answered.
  • replayFilter.untilstring or null
  • replayFilter.limitinteger
  • rowCountinteger
  • skippedNotStoredinteger
    For a replay: matching recorded calls left out because their input is no longer stored.
  • uniqueInputCountinteger
  • bytesinteger
  • createdByinteger
  • createdByNamestring
  • createdAtstring
Terminal
curl -X POST "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/datasets/replay" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{}'

GET /v1/organisations/{org}/decisions/{slug}/datasets/{datasetId}

Get a Dataset

Path

  • orgRequiredstring
  • slugRequiredstring
  • datasetIdRequiredstring

Answer (200)

  • idstring
  • namestring
  • sourcestring
    "upload": rows sent by a person or agent; "replay": a snapshot of recorded production calls.One of upload, replay
  • replayFilterobject or null
    How a replay Dataset chose its recorded runs.
  • replayFilter.environmentstring
  • replayFilter.versionNumberinteger or null
  • replayFilter.sincestring or null
    ISO times bounding when the recorded calls were answered.
  • replayFilter.untilstring or null
  • replayFilter.limitinteger
  • rowCountinteger
  • skippedNotStoredinteger
    For a replay: matching recorded calls left out because their input is no longer stored.
  • uniqueInputCountinteger
  • bytesinteger
  • createdByinteger
  • createdByNamestring
  • createdAtstring
  • previewarray
    The first 50 rows.
Terminal
curl "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/datasets/<datasetId>" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

DELETE /v1/organisations/{org}/decisions/{slug}/datasets/{datasetId}

Delete a Dataset

Soft delete: Dataset runs and Comparisons keep their evidence.

Path

  • orgRequiredstring
  • slugRequiredstring
  • datasetIdRequiredstring

Answer (204)

It answers with no body.

Terminal
curl -X DELETE "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/datasets/<datasetId>" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

Comparisons

GET /v1/organisations/{org}/decisions/{slug}/comparisons

List Comparisons

Path

  • orgRequiredstring
  • slugRequiredstring

Answer (200)

  • comparisonsarray
Terminal
curl "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

POST /v1/organisations/{org}/decisions/{slug}/comparisons

Start a Comparison

Starts the Comparison, running the rows it needs in the background; maxCalls is optional (without it, every chosen row runs). With X-TypeSafe-Key its rows run on your own TypeSafe key, sealed while they wait and never stored.

Path

  • orgRequiredstring
  • slugRequiredstring

Headers

  • X-TypeSafe-KeyOptionalstring

Body

  • datasetIdRequiredstring
  • aRequiredobject
    One side of a Comparison: exactly one of version (a published version by number), draft (the draft as it is now, frozen when started), environment (the release live there when started) or recorded (production's recorded answers; replay Datasets only).
  • a.versionOptionalinteger
  • a.draftOptionalboolean
  • a.environmentOptionalstring
  • a.recordedOptionalboolean
  • bRequiredobject
    One side of a Comparison: exactly one of version (a published version by number), draft (the draft as it is now, frozen when started), environment (the release live there when started) or recorded (production's recorded answers; replay Datasets only).
  • b.versionOptionalinteger
  • b.draftOptionalboolean
  • b.environmentOptionalstring
  • b.recordedOptionalboolean
  • sampleOptionalinteger
  • seedOptionalinteger
  • batchSizeOptionalinteger
  • freshOptionalboolean
  • maxCallsOptionalinteger
    The most model calls both sides together may make; rows beyond it are not run. Without it, every chosen row runs.

Answer (202)

  • comparisonobject
  • comparison.idstring
  • comparison.datasetIdstring
  • comparison.datasetNamestring or null
  • comparison.datasetSourcestring or null
    One of upload, replay
  • comparison.statusstring
    One of running, completed, cancelled
  • comparison.aobject
  • comparison.bobject
  • comparison.maxCallsinteger
  • comparison.progressobject
    Rows finished across both sides, of the rows both will run.
  • comparison.headlineobject or null
    The headline numbers; null until analysed.
  • comparison.summaryobject or null
    The full analysis; null until analysed, and left out of lists.
  • comparison.projectionobject or null
    Each side's cost per call at the Decision's last 30 days of production traffic; null when there was none, before the analysis, and in lists.
  • comparison.costWordingobject or null
    The cost headline in words, as the app shows it: said compares the cost per call (such as "48% less per call") and projected puts it at production's recent traffic. Null before the analysis and in lists; either line is null when there is nothing to say.
  • comparison.replayobject or null
    For a Dataset replayed from production: the recorded calls it matched and how many could not be replayed, because their input was no longer stored or (once analysed) no longer fits the input a side accepts. Null for uploaded Datasets.
  • comparison.createdByinteger
  • comparison.createdByNamestring
  • comparison.createdAtstring
  • comparison.completedAtstring or null
  • estimateobject
  • estimate.datasetIdstring
  • estimate.rowsinteger
  • estimate.skippedNotStoredinteger
    The Dataset's recorded calls left out of its replay snapshot because their input is no longer stored.
  • estimate.uniqueInputsinteger
  • estimate.sampleinteger or null
  • estimate.seedinteger
  • estimate.batchSizeinteger
  • estimate.aobject
  • estimate.bobject
  • estimate.callsWithoutCapinteger
    Calls without a cap.
  • estimate.maxCallsinteger or null
  • estimate.callsinteger
    Calls within maxCalls (or without a cap when none was given).
  • estimate.rowsCoveredinteger
  • estimate.inputTokensinteger
  • estimate.minutesinteger
  • estimate.allowedboolean
  • estimate.refusalobject or null
Terminal
curl -X POST "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"datasetId":"<datasetId>","a":{},"b":{}}'

POST /v1/organisations/{org}/decisions/{slug}/comparisons/estimate

Estimate a Comparison

What a Comparison would cost and whether it may start. Calls no model.

Path

  • orgRequiredstring
  • slugRequiredstring

Headers

  • X-TypeSafe-KeyOptionalstring

Body

  • datasetIdRequiredstring
  • aRequiredobject
    One side of a Comparison: exactly one of version (a published version by number), draft (the draft as it is now, frozen when started), environment (the release live there when started) or recorded (production's recorded answers; replay Datasets only).
  • a.versionOptionalinteger
  • a.draftOptionalboolean
  • a.environmentOptionalstring
  • a.recordedOptionalboolean
  • bRequiredobject
    One side of a Comparison: exactly one of version (a published version by number), draft (the draft as it is now, frozen when started), environment (the release live there when started) or recorded (production's recorded answers; replay Datasets only).
  • b.versionOptionalinteger
  • b.draftOptionalboolean
  • b.environmentOptionalstring
  • b.recordedOptionalboolean
  • sampleOptionalinteger
    A seeded random sample of this many rows; the same seed draws the same rows.
  • seedOptionalinteger
  • batchSizeOptionalinteger
  • freshOptionalboolean
    Run both sides even when a Dataset run for the same plan and rows exists (a noise floor).
  • maxCallsOptionalinteger
    When given, the estimate covers only the rows this many calls reach.

Answer (200)

  • datasetIdstring
  • rowsinteger
  • skippedNotStoredinteger
    The Dataset's recorded calls left out of its replay snapshot because their input is no longer stored.
  • uniqueInputsinteger
  • sampleinteger or null
  • seedinteger
  • batchSizeinteger
  • aobject
  • a.labelstring
  • a.modestring
    "recorded": answers read from production; "reuse": an existing Dataset run; "run": new calls.One of recorded, reuse, run
  • a.reuseRunIdstring or null
  • a.callsinteger
    Calls this side makes for every selected row.
  • bobject
  • b.labelstring
  • b.modestring
    "recorded": answers read from production; "reuse": an existing Dataset run; "run": new calls.One of recorded, reuse, run
  • b.reuseRunIdstring or null
  • b.callsinteger
    Calls this side makes for every selected row.
  • callsWithoutCapinteger
    Calls without a cap.
  • maxCallsinteger or null
  • callsinteger
    Calls within maxCalls (or without a cap when none was given).
  • rowsCoveredinteger
  • inputTokensinteger
  • minutesinteger
  • allowedboolean
  • refusalobject or null
  • refusal.codestring
  • refusal.messagestring
Terminal
curl -X POST "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons/estimate" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"datasetId":"<datasetId>","a":{},"b":{}}'

GET /v1/organisations/{org}/decisions/{slug}/comparisons/{comparisonId}

Get a Comparison

Path

  • orgRequiredstring
  • slugRequiredstring
  • comparisonIdRequiredstring

Answer (200)

  • idstring
  • datasetIdstring
  • datasetNamestring or null
  • datasetSourcestring or null
    One of upload, replay
  • statusstring
    One of running, completed, cancelled
  • aobject
  • a.labelstring
  • a.reusedboolean
    An existing Dataset run answered this side; it made no new calls for this Comparison.
  • a.runobject
    One target executed (or read) over a Dataset: the evidence.
  • bobject
  • b.labelstring
  • b.reusedboolean
    An existing Dataset run answered this side; it made no new calls for this Comparison.
  • b.runobject
    One target executed (or read) over a Dataset: the evidence.
  • maxCallsinteger
  • progressobject
    Rows finished across both sides, of the rows both will run.
  • progress.doneinteger
  • progress.totalinteger
  • headlineobject or null
    The headline numbers; null until analysed.
  • headline.rowsinteger
  • headline.comparedinteger
  • headline.differingRowsinteger
  • headline.questionsComparedstring
    See ComparisonSummary; absent on Comparisons analysed before it existed and not read since.One of same, changed, different
  • summaryobject or null
    The full analysis; null until analysed, and left out of lists.
  • summary.rowsinteger
  • summary.comparedinteger
    Rows both sides answered: the distributions and differences count these.
  • summary.notAcceptedobject
  • summary.errorsobject
  • summary.skippedobject
  • summary.differingRowsinteger
  • summary.perQuestionobject
  • summary.onlyInobject
    Questions only one side asks: a contract change, not compared.
  • summary.retypedarray of string
    Questions both sides ask by name whose answers are of a different kind: not compared.
  • summary.questionsComparedstring
    Whether the answers could be compared: same questions, changed (some compared, some only on one side or retyped) or different (nothing in common: answers not compared, and differingRows is 0).One of same, changed, different
  • summary.differingarray of { "row": integer, "questions": array of string }
    Differing rows by Dataset row, with the questions that differ; at most 2,000.
  • summary.differingTruncatedboolean
  • summary.sidesobject
  • summary.differenceobject
    Cost, tokens and latency of each side and the change from A to B, over the rows both sides answered only, so a call limit or an error on one side doesn't skew them.
  • projectionobject or null
    Each side's cost per call at the Decision's last 30 days of production traffic; null when there was none, before the analysis, and in lists.
  • projection.environmentsarray of string
    The environments counted.
  • projection.daysinteger
  • projection.callsinteger
    Calls those environments answered in the last days days (shadow calls excluded).
  • projection.anumber or null
    Side A's cost per call times calls; null when A has no recorded cost.
  • projection.bnumber or null
  • costWordingobject or null
    The cost headline in words, as the app shows it: said compares the cost per call (such as "48% less per call") and projected puts it at production's recent traffic. Null before the analysis and in lists; either line is null when there is nothing to say.
  • costWording.saidstring or null
  • costWording.projectedstring or null
  • replayobject or null
    For a Dataset replayed from production: the recorded calls it matched and how many could not be replayed, because their input was no longer stored or (once analysed) no longer fits the input a side accepts. Null for uploaded Datasets.
  • replay.recordedCallsinteger
    Recorded calls the replay matched, stored or not.
  • replay.notStoredinteger
    Of those, calls whose input was no longer stored, so they were left out of the Dataset.
  • replay.notAcceptedinteger
    Rows the Comparison ran whose input a side did not accept (the larger side's count).
  • replay.messagestring or null
    The two above in one sentence; null when every call was replayed.
  • createdByinteger
  • createdByNamestring
  • createdAtstring
  • completedAtstring or null
Terminal
curl "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons/<comparisonId>" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

POST /v1/organisations/{org}/decisions/{slug}/comparisons/{comparisonId}/cancel

Cancel a Comparison

Path

  • orgRequiredstring
  • slugRequiredstring
  • comparisonIdRequiredstring

Answer (200)

  • idstring
  • datasetIdstring
  • datasetNamestring or null
  • datasetSourcestring or null
    One of upload, replay
  • statusstring
    One of running, completed, cancelled
  • aobject
  • a.labelstring
  • a.reusedboolean
    An existing Dataset run answered this side; it made no new calls for this Comparison.
  • a.runobject
    One target executed (or read) over a Dataset: the evidence.
  • bobject
  • b.labelstring
  • b.reusedboolean
    An existing Dataset run answered this side; it made no new calls for this Comparison.
  • b.runobject
    One target executed (or read) over a Dataset: the evidence.
  • maxCallsinteger
  • progressobject
    Rows finished across both sides, of the rows both will run.
  • progress.doneinteger
  • progress.totalinteger
  • headlineobject or null
    The headline numbers; null until analysed.
  • headline.rowsinteger
  • headline.comparedinteger
  • headline.differingRowsinteger
  • headline.questionsComparedstring
    See ComparisonSummary; absent on Comparisons analysed before it existed and not read since.One of same, changed, different
  • summaryobject or null
    The full analysis; null until analysed, and left out of lists.
  • summary.rowsinteger
  • summary.comparedinteger
    Rows both sides answered: the distributions and differences count these.
  • summary.notAcceptedobject
  • summary.errorsobject
  • summary.skippedobject
  • summary.differingRowsinteger
  • summary.perQuestionobject
  • summary.onlyInobject
    Questions only one side asks: a contract change, not compared.
  • summary.retypedarray of string
    Questions both sides ask by name whose answers are of a different kind: not compared.
  • summary.questionsComparedstring
    Whether the answers could be compared: same questions, changed (some compared, some only on one side or retyped) or different (nothing in common: answers not compared, and differingRows is 0).One of same, changed, different
  • summary.differingarray of { "row": integer, "questions": array of string }
    Differing rows by Dataset row, with the questions that differ; at most 2,000.
  • summary.differingTruncatedboolean
  • summary.sidesobject
  • summary.differenceobject
    Cost, tokens and latency of each side and the change from A to B, over the rows both sides answered only, so a call limit or an error on one side doesn't skew them.
  • projectionobject or null
    Each side's cost per call at the Decision's last 30 days of production traffic; null when there was none, before the analysis, and in lists.
  • projection.environmentsarray of string
    The environments counted.
  • projection.daysinteger
  • projection.callsinteger
    Calls those environments answered in the last days days (shadow calls excluded).
  • projection.anumber or null
    Side A's cost per call times calls; null when A has no recorded cost.
  • projection.bnumber or null
  • costWordingobject or null
    The cost headline in words, as the app shows it: said compares the cost per call (such as "48% less per call") and projected puts it at production's recent traffic. Null before the analysis and in lists; either line is null when there is nothing to say.
  • costWording.saidstring or null
  • costWording.projectedstring or null
  • replayobject or null
    For a Dataset replayed from production: the recorded calls it matched and how many could not be replayed, because their input was no longer stored or (once analysed) no longer fits the input a side accepts. Null for uploaded Datasets.
  • replay.recordedCallsinteger
    Recorded calls the replay matched, stored or not.
  • replay.notStoredinteger
    Of those, calls whose input was no longer stored, so they were left out of the Dataset.
  • replay.notAcceptedinteger
    Rows the Comparison ran whose input a side did not accept (the larger side's count).
  • replay.messagestring or null
    The two above in one sentence; null when every call was replayed.
  • createdByinteger
  • createdByNamestring
  • createdAtstring
  • completedAtstring or null
Terminal
curl -X POST "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons/<comparisonId>/cancel" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"

GET /v1/organisations/{org}/decisions/{slug}/comparisons/{comparisonId}/rows

List a Comparison's rows

Path

  • orgRequiredstring
  • slugRequiredstring
  • comparisonIdRequiredstring

Query

  • filterOptionalstring
    One of differs, all, errors, not_accepted
  • questionOptionalstring
  • fromOptionalstring
  • toOptionalstring
  • cursorOptionalinteger
  • limitOptionalinteger

Answer (200)

  • rowsarray
  • nextCursorinteger or null
    Pass as cursor for the next page; null when there are no more rows.
Terminal
curl "https://api.typefuse.dev/v1/organisations/acme/decisions/refunds/comparisons/<comparisonId>/rows" \
  -H "Authorization: Bearer $TYPEFUSE_TOKEN"