Typed decisions, the API contract and token economics
A decision AI model answers (by design) narrower questions than a generative LLM: given an input “state” and explicitly described choice options, it produces a decision in the form of a strictly typed result. It is a useful component when an application needs routing, classification, scoring or a gate, rather than a natural-language justification of the decision (which probably annoys you about LLMs as often as it annoys me). The architectural consequence is twofold: the output contract is easier to validate, and the cost is, in most cases, significantly lower. The savings do not come from “free AI”, though, but from a smaller task, a shorter context and limited text generation – that is what models of this type are trained for, and efficiency is one of their core design goals (on both the vendor’s and the customer’s side).
Below, I break down a concrete example with three decisions for a single message about two failed payments (it is a generic example from Microsoft Foundry that you can test yourself, which I of course encourage). I try to separate, or rather distinguish, the behaviour Microsoft describes in its documentation from my interpretation of the response, architectural hypotheses and cost assumptions, so you will find a bit more here than in the model documentation. The Microsoft-Decision-1 model is currently marked as Preview in the catalog, so we should not treat the Foundry implementation as final (those of you who use Foundry know…)
The decision as a typed result
A typical generative AI workflow looks like this: the application gives the model instructions, asks for JSON, receives text, parses it and checks that no fields are missing and that values fall within the allowed set. A constrained response format can improve reliability, but it is still token generation. A decision-type model takes a state (state) and one or more questions with explicit types (statically typed, in dev terms, although it is really more about declaring the type). It returns results under the same names (keys) instead of producing free-form output. The documentation describes three type definitions: noul for a binary question, choice for a closed choice and score for an ordered scale.
noul means “yes/no” expressed as a probability for a condition, in the range from 0 to 1. choice returns a single label together with the probabilities of the options. score returns the probabilities of the scale levels and a number calculated as their average weighted by level indices. For score, indices start at zero; three levels form an index range of 0–2.
A single request can contain multiple questions about the same state. This lets the model in the example below assess urgency, team and priority within one input, and the application does not have to make three independent calls. The documentation points to combining questions as a way to reduce the number of requests and (above all – author’s note) to keep a shared context. This does not guarantee identical quality or independence of results: each question still needs separate validation on production data – but that is a topic for another post.
Example: two failed payments
The input state is a customer message:
My account payment has failed twice today. I need help before the end of the day.
The contract defines three questions. The first checks whether urgent help is needed; the second routes the case to either the billing or the technical team; the third rates priority on three ordered levels.
{
"model": "<deployment-name>",
"state": "My account payment has failed twice today. I need help before the end of the day.",
"questions": {
"urgency": {
"type": "noul",
"instructions": "Does the customer need urgent help?"
},
"team": {
"type": "choice",
"instructions": "Which team should handle the request?",
"criteria": {
"billing": "Payment and invoice issues",
"technical": "Software and integration issues"
}
},
"priority": {
"type": "score",
"instructions": "How high is the priority of this request?",
"criteria": [
"Routine, no time pressure",
"Important, some time pressure",
"Urgent, immediate attention needed"
]
}
}
} model in the call is the deployment name in Foundry. It does not have to match the model identifier returned in the response. state can be text or JSON; in a larger system it is worth passing only the information needed for this decision, e.g. the message body, the number of previous contacts and relevant transaction attributes. questions is a map in which the key becomes the result identifier. In a question definition, type selects the primitive, instructions describes the problem being decided, and criteria gives meaning to categories or levels. For noul, the true/false criteria are optional; for a task with operational consequences it is worth adding them to distinguish, for example, “requires a same-day response” from “requires immediate intervention”.
The response and its fields
In this example, the API returned the following results. Numbers are shown with six decimal places for readability; the original response contained longer floating-point values.{
"model": "microsoft-decision-1",
"answers": {
"urgency": {
"noul": 0.995930,
"type": "noul"
},
"team": {
"choice": "billing",
"confidence": 0.999408,
"probabilities": {
"billing": 0.999704,
"technical": 0.000296
},
"type": "choice"
},
"priority": {
"confidence": 0.696294,
"legend": {
"0": "Routine, no time pressure",
"1": "Important, some time pressure",
"2": "Urgent, immediate attention needed"
},
"probabilities": {
"0": 0.000824,
"1": 0.797529,
"2": 0.201647
},
"score": 1.200823,
"type": "score"
}
},
"usage": {
"input_tokens": 115,
"output_tokens": 3
}
} At the response object level, model identifies the actual model that handled the request; it may differ from the deployment name used in the request. answers contains a map of decisions under the keys from questions. The application should check for the presence of every expected key, its type, the type of the result value, and whether the choice label belongs to the locally allowed list. usage.input_tokens and usage.output_tokens are counters returned by the service. This is billing and operational metadata – a topic very close to my heart and probably a future topic for posts on this blog.
The noul and choice values are easy to consume, but they should not drive a side effect directly. For example, urgency: 0.995930 can be compared against a threshold chosen after validation, and choice: billing can be mapped to the billing queue. In turn, probabilities for team shows the distribution across options, which makes it possible to tell a clear choice from an ambiguous situation.
How to calculate score
For levels with indices 0, 1 and 2, the response gives approximately 0.000824, 0.797529 and 0.201647 respectively. The returned result can be reproduced as the expected value of the index:
score = 0 × P(0) + 1 × P(1) + 2 × P(2)
score ≈ 0 × 0.000824 + 1 × 0.797529 + 2 × 0.201647 = 1.200823
This explains why score is not a label from the list. It is a continuous average along the level axis, so it can be used for sorting or thresholding within the same, stably defined rating scale. If the levels have unequal semantic spacing, indices 0, 1, 2 still treat them as equidistant. In that case the result must not be interpreted as an interval scale where differences have a natural meaning; it should be treated as an ordering or an approximate ranking. Changing the order of the criteria changes the meaning of the indices and may change the score. The documentation recommends using score for relative ordering and for thresholds validated on your own examples, not as an absolute, calibrated rating.
What can be said about confidence
For choice, the API returns the selected label, the probabilities distribution and confidence; for score it also returns confidence, the distribution over levels and the score. Microsoft describes the model as returning calibrated option probabilities and talks about confidence signals. In the public endpoint documentation I did not find a mathematical definition of the confidence field that would allow it to be tied to a specific formula. So you should not claim that it is the maximum of the distribution, the margin between the top two options, entropy or the probability of being correct. In the example, team.confidence (0.999408) is not equal to P(billing) (0.999704), which in itself shows that these fields cannot be treated as the same thing.
In practice: log both fields separately, check how they behave on labelled data and build your own quality curve for the specific task. If a business rule needs a calibrated class probability, use probabilities and verify the calibration. Treat confidence as an additional signal provided by the service until the API version documentation defines its semantics more precisely.
There is a noticeable difference between urgency (about 0.996) and priority, where level 1 has a probability of about 0.798. This is not necessarily a contradiction: the binary question asks about the need for urgent help, while the top level of the scale requires immediate attention. If an organisation defines urgency as an “end of day” SLA and the highest priority as “immediately”, these results may be consistent. The model, however, does not know the local policy unless it has been expressed in the state or the criteria.
Tokens and cost accounting
The example response reports input_tokens = 115 and output_tokens = 3. That is a useful fact for this request, but not a promise that every variant of the question, a longer message or a different version of the service will cost the same. Input tokens cover the input representation as counted by the service, so in practice they grow with the content of the state, the questions, the instructions and the criteria. It is worth measuring the actual distribution of input tokens on your own traffic. Short output_tokens does not mean the JSON content takes up three tokens: JSON with fields, probabilities and metadata is much longer in bytes. Token usage reflects the service’s billing convention, not HTTP transport.
Token cost is not the cost of the whole solution. TCO includes calls, a possible minimum charge or reserved throughput, the gateway layer, logging, retries, validation, data storage, human review and the consequences of wrong decisions. Combining three related questions into one request can reduce the number of calls; however, you should not assume that adding questions costs nothing — the prompt grows, and the exact billing method must be read from usage and the pricing for your deployment.
Scenario comparison
The calculation below is illustrative and is not an actual quote, for a simple reason: at the time of writing, the model catalog marks Decision-1 as Preview. The publicly available pricing page lists a cost of USD 0.042 per million input tokens and free output tokens. Will these prices stay the same once the model leaves Preview? I don’t know. The parameters of the standard generative LLM are also a comparative assumption rather than a specific vendor offer: USD 0.40/M input tokens and USD 1.60/M output tokens.
The formula for cost per million events is N × (Tin × Pin + Tout × Pout) / 1,000,000, where N is the number of requests, Tin and Tout are the average tokens per request, and Pin and Pout are the prices per million tokens.
| Scenario per request | Input tokens | Output tokens | D1 assuming USD 0.042/0 per M | LLM assuming USD 0.40/1.60 per M |
|---|---|---|---|---|
| Example from the conversation, D1 | 115 | 3 | USD 0.00000483 | ___ |
| Generative JSON, short | 500 | 100 | ___ | USD 0.000360 |
| Generative JSON, extended | 1,200 | 250 | ___ | USD 0.000880 |
For one million decisions, the first D1 configuration would cost USD 4.83 in this scenario. The short generative variant would cost USD 360, and the extended one USD 880. For 10 million: USD 48.30, USD 3,600 and USD 8,800 respectively; for 100 million: USD 483, USD 36,000 and USD 88,000. The short variant has a token cost about 74.5 times higher than the assumed D1; the extended variant about 182 times. The difference comes mainly from the assumed rate and the number of tokens, not from any universal price advantage of the Decision model.
This table does not compare quality, latency or the total system cost. An LLM can be given an additional task of explaining, summarising or extracting; Decision-1 will not deliver such results. On the other hand, forcing an LLM to generate JSON, validating it and retrying a failed call can push the cost beyond the simple example. A fair comparison should use the same set of events, the same input information, the quality required by the business and current prices – all in the context of your environments, not my demo.
Architectural pattern
Decision-1 fits most easily as a narrow component between event normalisation and the execution of an action. The application or AI gateway builds the state, attaches a versioned question contract and sends the request. On the integration side, a validator checks the JSON shape, required keys, ranges, allowed value lists and the completeness of distributions. Only then does the decision policy choose an automatic action, a fallback or a human review queue. This split keeps execution rules in the system, while the model provides a signal for them.
Routing and document classification. choice suits a closed catalogue of teams, document types, queues or models. For a document, the input should be the result of OCR/parsing rather than the file itself, because the model is text-only. Long documents need to be split or summarised by another component; that extra stage carries its own risk of losing context, as well as its own cost. The catalog lists a limit of up to 32K tokens, but the limit does not mean it is always worth sending the full document.
Events and alerts. noul can filter entry conditions for a process, and score can order a queue of incidents or tickets. The result works well for prioritisation if the levels are consistent and teams understand how the indices map to SLAs. For critical alerts, it is safer to treat the model as an additional signal than as the only suppression mechanism.
Agent orchestration. Before a tool is executed, the system can assess whether the proposed action is allowed, whether it requires approval, or which agent should handle it. Microsoft lists evaluating agent actions and tool/model selection as use cases. This is not authorisation, however. A decision cannot replace permission checks, tool argument validation, transaction limits or deterministic policy rules.
AI gateway and model cascade. A gateway can route a simple, structured case to Decision-1 and send only out-of-scope cases to a generative LLM. Such routing can lower the average cost in mixed traffic. The benefit is an architectural hypothesis that has to be measured: there is an extra request, and an incorrect initial classification can degrade the quality of the whole cascade. It is worth logging the version of the criteria sets, token usage, class distribution, latency, fallbacks and error against reference labels.
Scoring and guardrails. You can evaluate model responses for compliance with a rubric or check for risky conditions. The result is neither proof of truthfulness nor a formal safety guarantee. Microsoft points to the risk of errors in classifying harmful content and to sensitivity to how questions/options are phrased. That is why the threshold should be chosen based on the cost of false positives and false negatives, with borderline cases routed to a further check or to a human.
Performance, reliability and security boundaries
Microsoft describes single-pass decision scoring and up to 32K tokens of context. However, a single model pass should not be equated with a specific SLA. The real advantage over a generative interface is a response with a predictable shape. You still need to handle transport errors, missing fields, unknown types, invalid labels, timeouts, repeated requests and the deployment version. It is worth having contract tests for the input/output schema and idempotent retries on the application side. An automated action should also require domain rules, e.g. verifying the payment status in the source system, instead of treating the customer’s message as fact.
Within security boundaries, the input is still untrusted text. Prompt injection in a ticket does not disappear just because the model does not generate a response. If Decision-1 assesses risk, its result is one of several signals and can be bypassed. Do not pass it secrets the classification does not need; apply data minimisation, access control for logs, retention in line with policy, and protection against mass abuse of the endpoint. For decisions that affect people, the model should not be the sole decision-maker (I hope that is obvious to you). Microsoft recommends validation on representative data, thresholds based on the consequences of errors, the option to abstain with “cannot tell”, monitoring for disparities and meaningful human oversight in high-risk applications – and I recommend the same.
The most important operational limitation stems from the very construction of the question: a closed set of answers forces a choice. If a possible case does not fit any category, you need to add an “other” or “cannot tell” option and check whether the model can actually use it. Without it, high confidence can mask an ontology error, not just a model error. Version your instructions and criteria like code: run regression tests on a reference set for any change to descriptions, level order, thresholds and queue mappings.
Takeaway for the architect
Decision-1 is a sensible candidate for a stage that makes small, closed decisions in a flow whose subsequent steps are already defined by the application. It is worth considering for routing, triage, classification, ranking and selecting the next component, especially with a high volume of events and a small required output. It will not replace a generative model where the product is meant to be text, nor a rules engine where deterministic policy compliance is required.
Before deployment, four questions need answering: does the rubric cover real cases; what is the quality at the chosen thresholds and abstention options; what is the cost after plugging in current pricing and the actual token distribution; and what does the system do under uncertainty, an API error or a version change. Only then do a short output and a low token cost become an architectural argument rather than just an attractive number from a single sample.
