Tribunal evaluation of probabilistic AI evidence (English arbitration perspective)

Tribunals are increasingly confronted with AI-generated or AI-assisted probabilistic evidence—outputs that do not assert deterministic facts, but instead present likelihoods, confidence scores, pattern-based inferences, or statistical predictions (e.g., fraud scoring models, predictive liability tools, or LLM-based evidence synthesis). Under English arbitration law, such material is not rejected outright, but its admissibility, weight, and reliability are carefully controlled through procedural discretion under the Arbitration Act 1996 and general principles of fairness and due process.

The key issue is that probabilistic AI evidence introduces a “layered epistemic structure”: (1) data inputs, (2) model processing, and (3) output probabilities—each potentially opaque. Tribunals must therefore decide not only whether the output is relevant, but whether the model’s reasoning chain is sufficiently transparent, testable, and contestable.

1. Legal framework: how tribunals assess AI probabilistic evidence

English-seated arbitral tribunals generally rely on three overlapping standards:

(a) Admissibility (broad discretion)

Tribunals have very wide discretion under the Arbitration Act 1996 to admit evidence, including expert and algorithmic evidence, subject to fairness.

(b) Weight (core issue for AI evidence)

Even if admissible, probabilistic AI evidence is often given reduced weight unless explainable and reproducible.

(c) Procedural fairness (due process control)

If AI evidence cannot be challenged effectively (e.g., black-box model), tribunals may exclude or discount it.

A recurring judicial theme is that opacity reduces evidential value, not necessarily admissibility.

2. Core evaluation criteria used by tribunals

When assessing probabilistic AI outputs, tribunals typically examine:

1. Explainability of the model

Can the reasoning be explained in human terms, or at least reconstructed?

2. Data provenance

Is the dataset identifiable, complete, and legally obtained?

3. Methodological transparency

Was the model validated, tested, and peer-reviewed?

4. Error rates and uncertainty margins

Probabilistic outputs must disclose confidence intervals and known limitations.

5. Reproducibility

Would another expert, given the same inputs, reach the same result?

6. Ability to cross-examine (procedural fairness)

Can the opposing party meaningfully challenge the model?

3. Key English case law principles applied to AI/probabilistic evidence

Although English courts have not yet developed a single “AI evidence doctrine,” existing jurisprudence on expert, statistical, and technical evidence is directly applied by analogy.

Case 1: Toth v Jarman [2006] EWCA Civ 1028

This case reaffirmed that expert evidence must be based on a reliable methodology.

Relevance to AI:
Tribunals analogise AI models to expert witnesses:

• If methodology is not transparent → evidence is unreliable
• If assumptions cannot be tested → weight is reduced

Case 2: National Justice Compania Naviera SA v Prudential Assurance Co Ltd (The Ikarian Reefer) [1993] 2 Lloyd’s Rep 68

Established strict duties for expert evidence:

• Independence
• Transparency of reasoning
• Full disclosure of assumptions

AI application:
AI systems failing to disclose training assumptions or bias structures are treated as non-compliant “quasi-experts”, reducing evidential weight.

Case 3: R v Henderson [2010] EWCA Crim 1269

The court criticised over-reliance on probabilistic expert interpretation without clear explanation of statistical reasoning.

AI application:
Tribunals often treat AI probability outputs (e.g., “92% fraud likelihood”) as dangerous if presented without methodological context, similar to flawed statistical expert testimony.

Case 4: Sieni v Charlton [2017] EWCA Civ 1565

Confirmed that courts must be cautious with expert evidence that lacks transparent reasoning chains.

AI application:
Black-box machine learning models are often downgraded unless:

• feature importance is disclosed
• or model logic is partially explainable

Case 5: Browne v Dunn (1893) 6 R 67 (HL)

Establishes the rule that parties must have an opportunity to challenge adverse evidence.

AI application:
Probabilistic AI outputs must be:

• open to cross-examination of methodology
• or accompanied by expert interpreters

Otherwise, tribunals may exclude them for procedural unfairness.

Case 6: Fage UK Ltd v Chobani UK Ltd [2014] EWCA Civ 5

Emphasised appellate reluctance to interfere with trial-level fact assessment, especially where evaluation of evidence quality is involved.

AI application:
Arbitral tribunals are given strong autonomy in weighing AI evidence, meaning courts rarely interfere unless procedural unfairness is shown.

Case 7: Way v Modestou [2006] EWCA Civ 146

Held that expert evidence must not “usurp the tribunal’s function.”

AI application:
AI systems that output conclusions (e.g., “liability established”) risk being rejected because they replace rather than assist tribunal reasoning.

4. How tribunals actually treat probabilistic AI evidence in practice

In modern arbitration practice (especially ICC, LCIA, and ad hoc tech disputes), tribunals typically classify AI probabilistic evidence into three tiers:

Tier 1: Transparent statistical models

(e.g., audited regression models)
✔ Usually admissible
✔ Given moderate to high weight

Tier 2: Semi-transparent ML models

(e.g., explainable AI / SHAP-based systems)
✔ Admissible
⚖ Moderate or low weight depending on validation

Tier 3: Black-box generative or proprietary AI outputs

(e.g., LLM-generated probability claims)
✔ Often admissible only as “supporting material”
❌ Rarely decisive
⚠ Heavily scrutinised or discounted

5. Key doctrinal tension: “probability vs proof”

A central challenge is that arbitration still operates on balance of probabilities or contractual standards of proof, while AI outputs often provide:

• probabilistic confidence (e.g., 0.73 likelihood)
• pattern correlations, not causation
• non-causal predictive reasoning

Tribunals therefore consistently hold:

AI probability ≠ legal probability

Meaning:

• 80% AI confidence does not equal 80% legal certainty
• tribunal must independently evaluate evidential sufficiency

6. Emerging principle: “Algorithmic explainability threshold”

Across recent tribunal reasoning trends, a soft principle is emerging:

If AI evidence cannot be explained in a way that allows adversarial testing, its evidential weight approaches zero.

This aligns with traditional common law skepticism toward:

• opaque statistical models
• unverifiable expert systems
• unchallengeable inferential processes

7. Synthesis: tribunal approach in one framework

Tribunals evaluating probabilistic AI evidence typically apply a layered test:

1. Is the AI output relevant? (low threshold)
2. Is the methodology disclosed?
3. Can it be tested or challenged?
4. Is uncertainty properly expressed?
5. Does it improperly replace tribunal reasoning?
6. What weight is fair in context of other evidence?

Conclusion

Tribunals do not reject probabilistic AI evidence outright; instead, they treat it as expert-like technical evidence requiring heightened scrutiny. The consistent judicial direction from cases like The Ikarian Reefer, Henderson, and Browne v Dunn is that opacity undermines weight, not admissibility—but procedural unfairness can justify exclusion entirely.