Most conversations about AI in capital-intensive decisions optimize for the wrong thing: accuracy. For example, a healthcare operator deciding where to open their next facility, and at what capacity, can now get a modeled answer to that question in minutes.

The challenge is what happens next.

That answer always travels outside the room it was produced in: to a lender, an investment committee, a Board. And the first question is never how accurate the model is but, “Can you explain this number to me?”

That gap between approval and governance is already visible at the Board level. Grant Thornton’s 2026 AI Impact Survey Report found three out of four Boards have approved major AI investments, but fewer than 50% have set governance expectations for those systems. And, among companies still piloting AI, just 7% have confidence that they could pass an independent AI governance audit within 90 days.

A deployed AI model only has value if a human can defend its output to whoever has to approve or reject the decision it feeds. A number with no visible derivation cannot be underwritten, no matter how accurate it is. Accuracy answers whether the model is right; defensibility answers whether the person accountable for the number can take it apart, and stand behind it in the room where the decision gets approved. That second answer is what most AI models were never built to survive.

Why one model doesn’t work

Let’s return to our original question: where should this operator open their next facility, and at what capacity?

The fastest thing to build is a single model that outputs one score per location, ranked. It’s also the wrong architecture for this problem.

An end-to-end model — even a highly accurate one — collapses distinct, sometimes competing, signals into a single number.  Even with a blended score, you can recover which raw features drove it: post-hoc attribution methods are capable of that much.  What you’ll miss is the mapping between that attribution and the decision planes that an operator reasons in.

For example, demographic fit, revenue per unit of capacity, or demand saturation in the market. Instead of an explainability tool added after the fact, the architecture needs this embedded.

At Teragonia, we think about this question as training three separate models, each scoped to a decision plane an operator reasons in:

•  A demographic-fit model, trained on population and payer-mix data

•  A revenue-per-capacity model, trained on comparable-site performance

•  A demand-saturation model, netting addressable demand against competing capacity already in the market

Each model stays a distinct artifact with its own inputs, its own error profile, and its own audit trail. Nothing gets weighted into a single score before a human sees it. That’s a deliberate constraint on architecture, not a limitation: keeping the planes separate helps an operator interrogate any one of them without touching the others.

Every prediction carries its own derivation

Accuracy and explainability trade off in most AI model families. A gradient-boosted ensemble (many decision trees layered to correct each other’s errors) or a neural net (layers of interconnected units learning complex patterns) can outperform a linear or additive model on raw accuracy, but it also makes the answer to “why this response?” harder to produce.

For the revenue-per-capacity model, we constrain the architecture to preserve that answer: ridge regression on standardized inputs with per-feature contributions rolled into business relevant driver groups, so every prediction decomposes into the marginal dollar contribution of each driver: comparable-site density, population growth, payer mix, drive-time coverage.

That decomposition is not a report generated after the fact; it’s stored alongside the prediction as a first-class output, and it’s what renders on the operator’s screen the moment they open that number: which of the three planes moved it, and by how much.

2% more accuracy gets you nowhere if you can’t show your work.

We accept a small accuracy cost for that trade. For example, a fully unconstrained ensemble might edge out the linear model on backtested error, but we haven’t found that to be worth it. The constraint functions as a form of regularization, shrinking the hypothesis so that the attribution can be consistent across similar inputs. This is a better alternative to SHAP values that shift for reasons that don’t correspond well to market realities.

Human overrides are training signals, not exceptions

When an operator overrides the model, such as reclassifying a competitor the dataset missed, or redrawing where a market actually begins, that override is written to an append-only decision ledger: original value, new value, who changed it, when, and why, with a pointer back to the prediction it’s correcting.

This serves two purposes: it’s an audit trail and it’s a source of labeled training data. When someone overrides the model, that correction doesn’t just get blended back into the training data as one more example. Instead, it’s flagged as a human-corrected instance, treated as ground truth, and up-weighted so it counts for more than a regular data point when the model is retrained.

A market an experienced operator has corrected twice carries more signal than one no one has touched. The result is a model that’s harder to be wrong about in the same way, twice.

Every defended decision doesn’t just get logged … it changes what the next prediction is built on.

Not every override is a signal

An override is only a training signal if it’s correct. An operator who overrides a demand-saturation call because a competitor announced a new site is correcting the model with real information the dataset didn’t have.

An operator who overrides the same plane repeatedly because they’re anchored to how a market used to look is teaching the model to defer to a stale prior.

The ledger tags each override with a reason category: competitive landscape change, data quality issue, or judgment call. Where enough time has passed for the decision to have played out, the override is checked against the outcome it predicted before it gets up-weighted. An override that beats the model earns more weight in the next training cycle. One that didn’t stay in the ledger is an audit record but doesn’t get reinforced.

Without that check, the model isn’t learning to be more accurate; it’s learning to agree with whoever overrides it most often. And, that concept is harder to defend when you’re in front of the people approving the decision.

From one-off study to standing capability

This is where the economics change: defensibility stops being a one-off purchase and becomes a standing capability.

Before, an operator bought defensibility one market at a time. Commissioning a new feasibility study meant a fresh engagement.

Healthcare-specific feasibility studies typically run $50,000 to $500,000, and take six to eighteen weeks to return (based on typical costs for investor-grade market feasibility work) for a single market, one time.

Now, that same operator holds dozens of markets inside a single framework: three live models per market, a derivation attached to every prediction, and an override ledger that gets sharper every time someone uses it.

Accuracy answers one question, once. Defensibility compounds. Every market you enter makes the next one cheaper to underwrite, because the ledger and the derivations are already built.

The Defensibility Standard: Questions to Ask

If you’re being asked to approve a number you didn’t build, the defensibility standard in this piece translates into four concrete questions.

1) Can you see the derivation (not just the score)?

· Every prediction should decompose into the specific drivers that produced it.

2) Which of the underlying models drove this, and by how much?

· A number that can’t be traced back to a specific decision plane can’t be selectively challenged.

3) Where’s the override ledger?

· If no one has corrected the model yet, that’s worth knowing. If someone has made corrections, then you should be able to see what changed, and why.

4) Has a correction been checked against what actually happened?

· An override no one has validated against an outcome is an opinion, not evidence that the model is learning.

Note that none of the above are questions about accuracy, but instead about whether the person answering the number can take it apart in front of you, which is the standard this piece has been arguing for all along.

Accuracy is something a model earns in backtesting; defensibility is something a person earns (and defends) in the room; that’s the gap defensibility closes that accuracy never could.