The Decisions API Returned 0.62. Now What?

A 0.62 from OpenAI's Decisions API on a refund request is the model's estimate that a statement like "this is a manufacturing defect" is true, and both OpenAI and TypeSafe, which makes a similar model called Jev, leave the action to your application. Put the threshold in a business rule that also reads the refund amount and the customer's refund history, and store the model's number with every decision, so you can test a new threshold later without asking the model again.

Imagine an online kitchenware store that refunds items with a manufacturing defect without requiring a return, and sends items damaged by normal wear or misuse through its standard return process. A customer who bought a $60 blender asks for a refund: "I've only used it twice and the jar cracked." The store's application sends the customer's message and the order record to POST /v1/decisions with a predicate question, OpenAI's question type that asks how likely a statement is to be true. The question's instructions read: "Does the message describe a manufacturing defect rather than normal wear or misuse?" The response has this shape:

{
  "model": "gpt-6-luna",
  "answers": [
    { "type": "predicate", "name": "defect", "probability": 0.62 }
  ],
  "usage": { ... }
}

The probability of 0.62 is made up for this example. At 0.62, the customer's message could describe a defect or misuse.

Neither provider acts on the number. OpenAI's guide says "your application runs any actions those answers select." TypeSafe says in its guardrails cookbook: "TypeSafe supplies the assessment; your application owns the decision." Whether to refund the blender now, pass the case to a person, or send it through the standard return process is up to the store that runs the application.

What OpenAI's Decisions API and TypeSafe's Jev return

TypeSafe is the company behind Jev, its flagship model. A typical LLM API is built to produce text for people to read, so your code has to parse the result. Like the Decisions API, Jev writes no prose and gives typed answers: a number, one of your options, or a rating. Your code can put that value straight into an if statement. This post cites TypeSafe's docs often because they say more than OpenAI's about thresholds and how to read the numbers. A Noul is TypeSafe's yes/no question, the counterpart of OpenAI's predicate. Side by side, the two APIs look like this:

The same 0.62 means different things in different fields:

The 0.62 in this post belongs to the first row. Predicate and Noul answers carry no confidence, so your application has one number to work with. TypeSafe's Noul page suggests starting values: "A good starting point is 0.7 for yes and 0.3 for no." It also says, "Values between 0.3 and 0.7 can go to a person." By that guidance, 0.62 is neither a yes nor a no, so the case can go to human review.

Why one threshold isn't enough

Human review isn't always a good choice for a 0.62. Whether an automatic refund or a human review costs less depends on the refund amount relative to the review cost, so you can calculate the answer. Suppose a review costs $6 per case and is always right, and a wrong automatic refund loses the whole amount, because the item doesn't come back.

A real defect gets refunded either way, so an automatic refund loses money only when the problem isn't a manufacturing defect. The model's probability is the chance of a defect, which makes the chance of no defect 1 - defect probability. Multiply the chance of no defect by the refund amount and you get the expected loss, the average amount the store loses per automatic refund. With human review, the store pays the review cost instead of that loss. So when (1 - defect probability) × amount is less than the review cost, an automatic refund is the lower-cost choice.

Put the same 0.62 defect probability into this formula for a $15 blender and a $60 blender. The chance of no defect is 1 - 0.62 = 0.38, so the two compare like this:

At the same 0.62 defect probability, refunding the $15 blender right away without human review is the better choice, and the $60 blender is worth the $6 review. The break-even amount, where 0.38 × amount equals the review cost, is $6 ÷ 0.38 = $15.79. At a defect probability of 0.62, an automatic refund costs less below $15.79, and human review costs less above it.

Run the formula the other way and each amount gets its own threshold. You need the probability where the expected loss equals the review cost, so the threshold is 1 - (review cost ÷ amount). For an automatic refund of $60, the probability must reach 1 - ($6 ÷ $60) = 0.90. For $15, 1 - ($6 ÷ $15) = 0.60 is enough. The bigger the refund, the more a wrong one costs, so the threshold goes up.

A single 0.7 in the code fits neither amount. With that 0.7, a $60 refund goes out automatically at a defect probability of 0.75, where the expected loss is 0.25 × $60 = $15, more than twice the human review cost. A $15 refund at a defect probability of 0.65 goes to human review, even though an automatic refund's expected loss is 0.35 × $15 = $5.25. That's less than the $6 human review cost, so the store pays for a review when an automatic refund would cost less.

A threshold comes from the amount and the review cost, so recalculate it when either changes. If the review cost goes up to $12, the $60 threshold drops to 1 - ($12 ÷ $60) = 0.80. TypeSafe's Noul page says which way to move it: "Raise it when acting on a false yes is expensive, such as paging someone or issuing a refund. Lower it when missing a true yes is expensive, such as failing to flag a safety issue."

That math trusts 0.62 to behave like a real 62%. The word for this is calibration: of all the statements a model puts near 0.62, about 62% turn out to be true. For the store, about 62 of every 100 refund requests at 0.62 should involve real defects. TypeSafe's docs say models like Jev "are trained for calibrated decisions." The same page warns: "Calibration is measured across groups of predictions; it does not guarantee that an individual answer is correct." OpenAI's guide doesn't mention calibration. It calls probability "the model's estimate" and says: "Use labeled examples from your application to set thresholds for routing, filtering, or review. Choose thresholds based on the cost of false positives and false negatives."

Your reviewers' past decisions are those labeled examples. Take the cases the model placed between 0.55 and 0.65 and count the real defects. If 40% were real defects, put 0.40 in the formula. The expected loss on a $15 refund then goes from $5.70 to (1 - 0.40) × $15 = $9, which is more than a review costs.

The number can also change from call to call. TypeSafe published a test that shows by how much. TypeSafe took one auto-insurance claim and asked Jev 14 yes/no questions about it, such as "Is the loss covered under the policy's collision coverage?" TypeSafe repeated that call 15 times on 2026-09-11, with jev-1.13.0 answering every time, and asked several LLMs the same questions for comparison. The claim and the questions never changed. To make every repeat a separate, independent trial, TypeSafe also added a unique throwaway value to each request. The value had nothing to do with the claim. So the input wasn't perfectly identical, and TypeSafe says the test can't separate that value's effect from any variation between identical requests.

Across the 14 questions, the numbers themselves didn't change much. Standard deviation measures how far values sit from their average. TypeSafe calculated it for each question and averaged the results. The average was 0.0102, so a typical answer sat about 0.01 from its average. That was lower than for any of the tested LLMs that returned probabilities. Two questions moved the most. The answer to whether a policy exclusion applies ranged from 0.53 to 0.62. The answer to the collision coverage question ranged from 0.43 to 0.53, which crosses 0.5. An application with a 0.5 threshold would have said yes to the same claim on some calls and no on others.

gpt-6-luna wasn't in the test, and OpenAI's guide doesn't say whether the model's numbers change between identical requests. The test also used an insurance claim, not a refund. Still, if a refund request's number moved by about 0.1, as the numbers for those two questions did, a $15 refund with a defect probability near the 0.60 threshold could get different results from call to call.

So the store leaves a wide band for human review between its two automatic outcomes. A $60 refund goes out automatically only at a defect probability of 0.9 or higher, and it goes to the standard return process only under 0.3. Everything in between goes to human review. A case near a threshold can land on either side of it, but a small change in the number can't turn a refund into a standard return. The store also keeps the number it used, because asking again may not return the same number.

Where the threshold should live

TypeSafe's build guide says, "Keep control flow, deterministic rules, and side effects in code." In other words, the order of steps, any check with one right answer, and the act of issuing the refund don't belong to the model. With a few thresholds that rarely change, that advice is right, and TypeSafe's guardrails cookbook keeps them in a Python POLICIES dict. The cost shows up when they change: each change is a deploy, and working out what a new value would have done to past calls is your job. The right place depends on how many thresholds you have, how often they change, and who asks about them:

The store fits the third row. Its thresholds differ by amount, its reviewers want to change them, and support asks why the $60 blender's refund went to human review.

The refund check in LexQ

LexQ, which publishes this blog, is a Decision Operations Platform for engineering teams: rules are versioned and kept outside your code, and every decision is recorded. The application sends named input values called facts, such as refundAmountUsd, to LexQ. One refund request goes like this:

1. The application sends the customer's message and the order record to OpenAI or TypeSafe and gets a probability back.
2. It sends that probability, the model name, the refund amount, and the refund count to LexQ as facts.
3. LexQ runs the rules, records the decision with its inputs, and returns the decision.
4. The application acts on the decision: it issues the refund, passes the case to human review, or starts a standard return.

LexQ never calls a model. With no outbound call in the decision path, the same facts and the same version always give the same decision. The customer's message doesn't have to reach LexQ, which needs only the numbers and the model name.

The store's six rules sit in one policy group, the set of rules your application calls by ID, and in one mutex group, where exactly one rule decides each call. If several rules match, the highest-priority one decides. The table lists all six, highest priority first:

When STANDARD_RETURN is wrong, the customer wastes a trip and the store loses no refund money, so rule 5 uses TypeSafe's starting value of 0.3 instead of a threshold per amount.

Rules 1 and 2 don't read the probability, for two reasons. Arithmetic and counting aren't the model's job, as TypeSafe's jev-1.13 notes say: "Jev is not a calculator." The same page says adversarial content "can move the answer." A message written to steer the model can push the number up, but it can't change the amount or the refund count. So 0.96 on a $180 refund still goes to human review, and so does 0.95 on $60 from a customer with 4 refunds in 90 days.

Rules 3 and 4 use the thresholds calculated above: 0.9 at $60 and 0.6 at $15. Each band of amounts uses the threshold of its largest amount, so refunds over $15 and up to $60 need 0.9, and refunds of $15 or less need 0.6. A $30 refund needs 0.9 even though the formula would accept 0.8. More cases go to human review that way, but no refund goes out automatically below the formula's threshold.

For the $60 blender, the store's application sends this request, where {groupId} is the policy group's ID:

curl -X POST https://api.lexq.io/api/v1/execution/groups/{groupId} \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "facts": {
      "defectProbability": 0.62,
      "probabilityModel": "gpt-6-luna",
      "refundAmountUsd": 60,
      "refundsLast90Days": 0
    }
  }'

LexQ responds with this, shown here with inputs and per-rule details collapsed:

{
  "result": "SUCCESS",
  "data": {
    "traceId": "...",
    "inputFacts": { ... },
    "mutatedFacts": {},
    "generatedVariables": {
      "refundDecision": "HUMAN_REVIEW"
    },
    "executionTraces": [ ... ],
    "decisionTraces": [ ... ]
  }
}

generatedVariables holds what the rules wrote, so generatedVariables.refundDecision is the answer. mutatedFacts is empty because no rule changed an input. traceId identifies the record of this decision. decisionTraces says which rule decided, and executionTraces shows the condition each rule checked.

Rules 1 to 5 did not match here. The customer has no recent refunds, and $60 isn't over the limit. At 0.62 the probability is too low for rule 3 and too high for rule 5, and the amount is too large for rule 4. Rule 6 matches every call, so it decided.

Change only the amount and a different rule can decide. At $15, rule 4 returns AUTO_REFUND. At $180, rule 2 returns HUMAN_REVIEW whatever the probability is.

TypeSafe says models like Jev do not "generate explanations of their reasoning," and OpenAI's answer objects have no explanation field. So when support asks why this refund went to human review, the answer comes from LexQ's record: inputFacts keeps 0.62 and the model name, and the traces give the deciding rule, the version it ran in, and the 0.9 condition that rule 3 checked. Support can tell the customer that the model put the chance of a defect at 0.62 and that a $60 refund needs 0.9 to go out automatically.

The application can ask either provider and send LexQ the same two facts:

const QUESTION =
  "Does the message describe a manufacturing defect " +
  "rather than normal wear or misuse?";

async function askOpenAI(message: string, order: Order) {
  const decision = await openai.decisions.create({
    model: "gpt-6-luna",
    input: [
      {
        role: "user",
        content: [
          { type: "input_text", text: message },
          { type: "input_json", data: { order } },
        ],
      },
    ],
    questions: [{ type: "predicate", name: "defect", instructions: QUESTION }],
  });
  const answer = decision.answers[0];
  return {
    // A refusal has no probability, so it becomes null.
    defectProbability: answer.type === "predicate" ? answer.probability : null,
    probabilityModel: decision.model,
  };
}

async function askJev(message: string, order: Order) {
  const { answers, model } = await typesafe.systemOne({
    state: { message, order },
    questions: { defect: noul(QUESTION) },
    model: "jev-1.14.0",
  });
  return { defectProbability: answers.defect.noul, probabilityModel: model };
}

Both functions return the same shape, so the code that calls LexQ doesn't change when the model does.

You still need a decision when the model gives no number. OpenAI's reference says, "Any question can return a refusal instead of an answer," and a refusal has type: "refusal". The model call can also fail, and you may not be able to read the amount or the refund count. Here is what the application does in each case:

Null means no value, and a condition on a null value doesn't match. That is what "present" means in the rule table. Rules 3, 4, and 5 also test the amount and the refund count (>= 0 is enough), so a null keeps them from matching and the call ends in HUMAN_REVIEW. Without those tests, a null amount would also slip past rule 2's refundAmountUsd > 60, and a 0.96 could reach an automatic refund. Keep the key in the request, though. If a call omits a key that a rule reads, it fails with the error P-015.

Feeding a model's number into a rule as a fact also works for agents. The AI agent approval post shows it with a separate model that scores tool calls.

When a threshold or a model changes

Changing a threshold

Say a month of review decisions shows that refund requests at 0.85 to 0.90 involved real defects 92% of the time. Put 0.92 in the expected-loss formula and a $60 refund has an expected loss of 0.08 × $60 = $4.80, under the $6 review cost. Those cases cost less to refund automatically. The store lowers rule 3's threshold in five steps:

1. Clone the live version, v1, into a draft. A draft is an editable copy that takes no traffic, so customers see no change.
2. In the draft, change rule 3 from 0.9 to 0.85.
3. Run Decision Replay, the LexQ feature that reruns stored calls from a date range through the draft. It compares each result with the stored decision and counts the changes.
4. Check that the changes are the ones you expected.
5. Publish the draft to lock its rules, then deploy it so live calls go to it. If something goes wrong after the deploy, roll back to v1 and its 0.9.

Each rerun reads the stored probability, so nothing calls OpenAI or TypeSafe. Model noise stays out of the count, so it can't be why a decision changed. A stored 0.87 on $60, for example, goes to HUMAN_REVIEW under v1 and AUTO_REFUND under the draft.

If rule 3 is the only edit, every changed decision fits one description: defect probability from 0.85 to under 0.90, amount over $15 and up to $60, fewer than 3 refunds in 90 days, moving from HUMAN_REVIEW to AUTO_REFUND. The limits come from the other rules: rule 4 already refunds $15 or less at that probability, and rules 1 and 2 still send the rest to human review. Anything else may come from a second edit in the draft, decisions made by another version, or failed LexQ calls or batch calls in the range. A failed call has no stored decision to compare, and a batch call, which sends many requests at once, stores no input for the rerun to read.

The result lists up to 100 changed calls by traceId, and you can see each call's numbers by opening its trace. A replay reads only the newest 1,000 calls in the range by default, so raise maxRecords if there are more. The Decision Replay pattern has the full procedure.

Changing the model

The 0.85 came from counting real defects among one model's numbers, so a new model needs its own count. TypeSafe's models page warns about aliases such as jev-latest: "An alias moves when a new release ships, so its answers can change without any change to your code." It also says, "If you have tuned confidence thresholds against one version, send that version's ID instead of the alias." OpenAI's model page says snapshots "let you lock in a specific version of the model," and the only one it lists for GPT-6 Luna is gpt-6-luna.

That is why probabilityModel is a fact. The model value from the provider's response is stored with each decision, so you can tell which model version produced which number. If a model changes under the same name, this field won't show it, so also watch the distribution of the stored numbers over time. If the share of refunds at 0.9 or higher suddenly changes, check whether the model did.

Decision Replay reads stored numbers and never calls a model, so it can't test a new model on calls that hold only the old model's numbers. Instead, use Impact Simulation, the LexQ feature that runs a file of inputs through a version and counts the decisions each rule made:

1. Send last month's messages to the new model and collect the probabilities.
2. Build a CSV whose columns are the fact names. Where the model gave no answer, leave the cell empty. LexQ reads an empty cell as null.
3. Upload the file and run it through the live version.
4. Run the same file through a draft with new thresholds, and compare the per-rule counts.

The file looks like this, and the last row is a case with no answer:

defectProbability,probabilityModel,refundAmountUsd,refundsLast90Days
0.91,jev-1.14.0,42.50,0
0.62,jev-1.14.0,60,0
,jev-1.14.0,25,1

Per-rule counts are on by default only in LexQ's web console, so set includeRuleStats to true if you call the API. Raise maxRecords if the file has more than 1,000 rows. The Decision Replay and Impact Simulation guide explains how the two checks differ.

To run two models at once, put probabilityModel in the conditions of the rules that read the probability, so each model has its own threshold, such as 0.85 for gpt-6-luna and 0.9 for jev-1.14.0.

Comparing the Decisions API and Jev

A 0.62 from one model need not mean what 0.62 means from the other. To compare them on your own refunds, call both models for a month.

Send the number from the model you use today as defectProbability and the number from the model you're testing as shadowDefectProbability, together in one call rather than a batch. The live rules don't read the second one, so customers get the same decisions as before. LexQ still stores every fact in a non-batch call, so the test model's numbers build up next to the decisions.

When the month ends, clone the live version, point rules 3, 4, and 5 at shadowDefectProbability, and replay the month's calls through that draft. The replay counts the decisions that would have changed with the test model's number.

Check those cases against your reviewers' decisions. Reviewers saw only the cases that went to them, so this covers only part of your traffic. Calls where one model gave no number can also show up as changes, so leave them out.

The comparison costs two model calls per request while it runs. To try the other model on live traffic, publish that draft and start an A/B test. Send a customer ID in context.trafficKey, next to facts. LexQ splits traffic by that value, so each customer stays on one version, and requests without it never reach the test version.

When you don't need LexQ

If a person still decides every refund and the number only sorts the queue, you have no threshold to manage.

If you get a few dozen refund requests a month, too few calls fall between 0.85 and 0.90 for a replay to tell you anything. Start with a constant and a log, and move to LexQ when volume grows.

→ Start free at lexq.io