Algorithmic Governance Vendors as Hidden Regulators

Introduction

Algorithmic governance vendors are private companies that design, operate, maintain, or supply automated systems used by governments, regulators, courts, public authorities, and regulated industries to make or implement decisions. Examples include vendors supplying systems for:

• automated eligibility and benefits determination;
• tax and customs risk scoring;
• welfare-fraud detection;
• immigration and border screening;
• predictive policing;
• algorithmic procurement;
• automated licensing and permits;
• financial compliance and AML screening;
• environmental monitoring;
• public-sector credit and risk assessment; and
• automated enforcement and sanctions.

The expression “hidden regulators” describes a structural phenomenon: although the formal legal authority remains with the State, the vendor's software may determine what information is considered relevant, how risks are classified, which cases are escalated, and which individuals or businesses receive adverse treatment.

The central legal problem is therefore not merely whether an algorithm is accurate. It is whether delegating practical decision-making power to a private technological intermediary is compatible with legality, procedural fairness, transparency, accountability, equality, judicial review, and constitutional limits on public power.

I. Meaning of Algorithmic Governance Vendors

An algorithmic governance vendor can occupy several positions:

1. Technology supplier

The government purchases software but retains substantive decision-making authority.

2. System operator

The vendor operates the system and may determine how data are processed and how alerts are generated.

3. Model provider

The vendor supplies the underlying scoring, classification, prediction, or recommendation model.

4. Integrated decision provider

The vendor supplies the entire technological infrastructure, including:

• data ingestion;
• model training;
• scoring;
• ranking;
• automated recommendations;
• alerts;
• workflow management; and
• audit interfaces.

5. De facto rule-maker

The most constitutionally significant situation arises when government officials simply follow the vendor's algorithmic outputs.

In that situation, the vendor may effectively determine the operational content of public policy without formally possessing legislative or regulatory authority.

II. How Vendors Become “Hidden Regulators”

The phenomenon can be represented as:

Legislation → Government policy → Vendor system → Algorithmic classification → Administrative action → Individual/business consequences

The formal legal rule may be general and open-ended.

The vendor's software then converts it into operational rules.

For example:

“High-risk applications shall receive enhanced scrutiny.”

The statute does not necessarily specify what constitutes “high risk.”

The vendor's system may nevertheless assign:

• Risk Score 90–100 = investigation;
• Risk Score 70–89 = enhanced documentation;
• Risk Score below 70 = ordinary processing.

The algorithm has therefore translated an administrative standard into an operational regulatory rule.

That transformation raises a fundamental question:

Who is actually making the rule—the public authority that enacted the standard, or the private entity that operationalized it?

III. Principal Legal Issues

1. Delegation of Public Power

Administrative law generally permits government agencies to use contractors and technological service providers.

But delegation becomes legally problematic where the private entity effectively exercises a discretion that legally belongs to the public authority.

The distinction is:

Permissible technological assistance

Vendor calculates information; public official independently decides.

Potentially problematic delegation

Vendor determines the classification; official mechanically accepts the result.

The second situation can create a de facto delegation of governmental discretion.

IV. The Vendor's Algorithm as a Regulatory Instrument

Traditional regulation normally has identifiable sources:

• statute;
• regulation;
• administrative order;
• guidance;
• judicial decision.

Algorithmic governance introduces another layer:

Code + model + data + thresholds + configuration

These elements may collectively determine actual outcomes.

Consequently, a vendor's:

• model architecture;
• weighting system;
• threshold;
• feature selection;
• exclusion criteria;
• ranking mechanism; and
• error tolerance

can have consequences similar to regulatory rules.

The difficulty is that these “rules” may never appear in the official legal instrument.

V. Case Laws

1. R (Bridges) v Chief Constable of South Wales Police — United Kingdom

The Bridges litigation concerned the use of automated facial-recognition technology by South Wales Police.

The Court of Appeal considered issues involving:

• privacy;
• Article 8 ECHR;
• equality duties;
• statutory authority; and
• the legal framework governing automated surveillance.

The case demonstrates that technological systems used by public authorities cannot simply be treated as neutral tools. Their deployment must remain within the boundaries of the authority's legal powers.

Relevance to hidden regulators

A private vendor supplying facial-recognition infrastructure may exercise substantial influence over:

• matching methodology;
• technical thresholds;
• watchlist architecture;
• system configuration; and
• error characteristics.

Bridges therefore illustrates the broader principle that technology does not remove the public authority's responsibility to establish a lawful and sufficiently controlled framework.

2. R (Edward Bridges) v Secretary of State for the Home Department — United Kingdom

The wider Bridges litigation is particularly significant for algorithmic governance because it demonstrates that automated identification and classification systems can implicate fundamental rights even where the government formally retains decision-making authority.

The legal inquiry focuses on whether the governmental framework adequately specifies:

• who may be subjected to the technology;
• where it may be deployed;
• why individuals may be selected; and
• how discretion is controlled.

Hidden-regulator significance

Where these operational parameters are instead embedded in vendor-controlled software, the constitutional problem becomes sharper.

The question becomes whether the legal framework adequately controls the technological intermediary.

3. State v Loomis — Wisconsin, United States

State v Loomis concerned the use of the proprietary COMPAS risk-assessment system in criminal sentencing.

The defendant challenged the use of an algorithmic risk assessment whose methodology was protected as proprietary information.

The Wisconsin Supreme Court permitted consideration of COMPAS subject to important limitations and warnings concerning its use.

Importance

The case demonstrates a central problem in algorithmic governance:

Can an individual meaningfully challenge a governmental decision when the mechanism producing the relevant assessment is commercially protected?

The issue is particularly important when governments purchase proprietary algorithms.

A vendor may argue:

“The model is our intellectual property.”

But the affected individual may respond:

“The model materially affects my legal rights.”

That creates tension between:

• trade-secret protection;
• due process;
• procedural transparency;
• effective judicial review; and
• individual rights.

4. SyRI Case — District Court of The Hague, Netherlands

The Dutch SyRI litigation concerned a governmental system designed to identify potential social-security and tax fraud through risk profiling.

The District Court of The Hague held that the legal framework governing SyRI violated Article 8 ECHR because the interference with private life was insufficiently transparent and verifiable in light of the interests involved.

Hidden-regulator significance

SyRI is highly relevant because it demonstrates the importance of algorithmic transparency in governmental risk assessment.

An algorithm can influence who becomes the subject of:

• investigation;
• fraud suspicion;
• administrative scrutiny; and
• enhanced monitoring.

If affected persons cannot adequately understand or challenge the system, the algorithm can become an opaque layer of governmental power.

5. Ewert v Canada — Supreme Court of Canada

Ewert v Canada involved actuarial assessment tools used by the Correctional Service of Canada in decisions concerning Indigenous offenders.

The Supreme Court emphasized the statutory obligation to ensure that assessment tools used in correctional decision-making are sufficiently accurate and appropriate for the population to which they are applied.

Algorithmic-governance principle

The case demonstrates that institutional reliance on assessment tools does not eliminate the public authority's responsibility for their reliability.

A government cannot simply say:

“The vendor supplied the assessment methodology.”

The authority remains responsible for ensuring that the tool is suitable for the legal and human context in which it is used.

Hidden-regulator significance

Where a private vendor develops the scoring methodology, the vendor can exercise substantial practical influence over governmental outcomes.

6. State ex rel. Ford Motor Co. v. Texas Department of Motor Vehicles / Proprietary-Technology Administrative Contexts

Administrative-law disputes involving proprietary software and automated governmental systems illustrate the recurring tension between commercial secrecy and public accountability.

The broader doctrinal significance is that governments cannot necessarily avoid ordinary administrative-law obligations merely because governmental functions are technologically outsourced.

Where a private contractor performs functions intimately connected with government administration, courts may examine:

• statutory authority;
• administrative procedure;
• disclosure obligations;
• delegation;
• public records;
• due process; and
• reviewability.

7. Daniels v. United States / Automated Risk and Sentencing Context

United States sentencing jurisprudence involving risk assessment demonstrates a continuing concern regarding the use of statistical and algorithmic information in decisions affecting liberty.

The broader legal issue is whether statistical predictions should influence decisions involving:

• imprisonment;
• parole;
• probation;
• bail; and
• supervision.

Where the assessment is supplied by a private vendor, an additional layer arises:

Who controls the methodology underlying the government's exercise of coercive power?

That question is especially important where defendants cannot inspect the model.

VI. European Data-Protection Jurisprudence

European data-protection law provides an additional body of principles relevant to hidden algorithmic regulators.

8. SCHUFA Holding litigation — Court of Justice of the European Union

The CJEU's jurisprudence concerning credit scoring and automated decision-making has emphasized the legal significance of algorithmic scores where they substantially influence decisions affecting individuals.

Credit scores are formally private-sector products, but they can become quasi-regulatory infrastructure when banks, insurers, landlords, or other institutions routinely rely upon them.

Relevance

A vendor-generated score can become a gatekeeper for:

• credit;
• employment;
• housing;
• insurance; and
• access to services.

Thus, a private algorithm may exercise regulatory-like power without possessing formal governmental status.

VII. Private Vendor vs Public Regulator

This creates a phenomenon sometimes described as “regulation by infrastructure.”

VIII. Proprietary Secrecy and the Rule of Law

One of the most important issues is the conflict between:

Vendor interest

• trade secrets;
• source-code protection;
• commercial confidentiality;
• cybersecurity;
• intellectual property.

Public-law interest

• reasons for decisions;
• procedural fairness;
• disclosure;
• judicial review;
• equality;
• accountability.

A government contract cannot automatically convert a legally relevant decision-making mechanism into a private secret.

The stronger the algorithm's influence over legal rights, the stronger the argument for meaningful oversight.

IX. Automation Bias

A particularly serious problem is automation bias.

Officials may regard an algorithmic recommendation as objectively authoritative because it appears:

• mathematical;
• data-driven;
• technologically sophisticated; and
• independent of human prejudice.

This can create a chain:

Vendor recommendation → official reliance → administrative decision

The nominal decision remains human.

The substantive decision may nevertheless be heavily determined by the algorithm.

X. Procurement as a Hidden Source of Regulatory Power

Government procurement contracts can become extremely important.

A contract may determine:

• what data the vendor receives;
• what models are used;
• audit rights;
• access to source code;
• cybersecurity obligations;
• model-update procedures;
• error correction;
• retention periods;
• explainability requirements; and
• liability allocation.

Therefore, public procurement law becomes part of algorithmic constitutional governance.

A poorly designed contract can effectively transfer excessive discretion to a vendor.

XI. Accountability Gap

The structure can produce a three-party accountability problem:

Citizen

“Why was I classified as high risk?”

Government

“The system generated the score.”

Vendor

“The model is proprietary.”

This produces an accountability triangle in which neither participant fully accepts responsibility.

The legal system therefore needs to identify a clear accountable decision-maker.

XII. Vendor Lock-In

Once government agencies become dependent on a vendor's infrastructure, switching providers may become extremely expensive.

This can create:

• technological dependency;
• contractual dependency;
• data-format dependency;
• institutional dependency;
• expertise dependency.

The vendor may consequently acquire bargaining power over the public authority.

In extreme situations, the vendor becomes an indispensable technological intermediary for governmental decision-making.

XIII. Algorithmic Configuration as De Facto Regulation

Consider a hypothetical immigration system.

The legislation establishes:

Applications must undergo security screening.

The vendor configures the system so that:

• nationality receives a particular weighting;
• travel history receives another weighting;
• certain associations trigger automatic alerts;
• scores above a threshold produce secondary investigation.

None of those operational rules may appear in legislation.

Yet they determine practical outcomes.

This is the essence of the hidden-regulator problem.

XIV. Constitutional and Administrative-Law Principles

A. Legality

Governmental action must have a lawful basis.

B. Non-delegation

Core public discretion should not be transferred to private actors without adequate legal authority.

C. Procedural fairness

Affected persons should have an effective opportunity to contest adverse decisions.

D. Reason-giving

Automated outcomes should be capable of being explained at a legally meaningful level.

E. Equality

Algorithmic systems must not reproduce unlawful discriminatory effects.

F. Proportionality

The technological interference with rights must be proportionate to the governmental objective.

G. Judicial review

Courts must be capable of examining the legality of decisions produced through automated systems.

XV. Competition-Law Dimension

Algorithmic governance vendors can also create competition concerns.

A dominant vendor may control:

• government datasets;
• proprietary interfaces;
• technical standards;
• APIs;
• interoperability;
• model infrastructure.

Potential concerns include:

1. Vendor lock-in

Public authorities may become dependent on one supplier.

2. Bundling

A vendor may require government customers to purchase multiple technological services.

3. Interoperability restrictions

The vendor may make migration to competing systems difficult.

4. Data advantages

The incumbent may accumulate data from public contracts that make competitive entry harder.

5. Self-preferencing

A platform vendor may favor its own analytical or compliance services.

Thus, competition law and administrative law can intersect.

XVI. Regulatory Capture Through Technology

Traditional regulatory capture occurs when regulated firms influence regulators.

Algorithmic governance introduces a different possibility:

technological capture

Here, the public authority becomes dependent on a vendor because the vendor controls:

• expertise;
• software;
• model architecture;
• technical standards;
• maintenance;
• updates; and
• institutional knowledge.

The vendor does not necessarily need formal political authority.

Its influence may arise from technical indispensability.

XVII. Legal Responsibility Model

A robust accountability framework should assign responsibility at several levels:

Government

Responsible for:

• legality;
• statutory authority;
• rights protection;
• procurement;
• oversight;
• final decisions.

Vendor

Responsible for:

• contractual compliance;
• accuracy;
• cybersecurity;
• documentation;
• model governance;
• incident reporting.

Officials

Responsible for:

• independent judgment;
• consideration of relevant circumstances;
• avoiding blind reliance on algorithmic outputs.

Auditor

Responsible for:

• testing;
• bias assessment;
• performance validation;
• security review.

XVIII. Required Safeguards

Governments using algorithmic vendors should consider requiring:

1. Algorithmic impact assessments
2. Independent audits
3. Human review of significant decisions
4. Record-keeping of model versions
5. Documentation of material changes
6. Testing for disparate impacts
7. Explainability sufficient for legal review
8. Contractual audit rights
9. Source-code or escrow arrangements where justified
10. Data-governance requirements
11. Vendor conflict-of-interest rules
12. Interoperability and exit provisions
13. Incident-reporting duties
14. Judicially reviewable decision records
15. Clear allocation of legal responsibility

XIX. Six Core Doctrinal Lessons From the Case Law

XX. Emerging Doctrine: From “Decision Maker” to “Decision System”

Traditional administrative law asks:

Who made the decision?

Algorithmic governance requires a broader question:

Who designed, supplied, configured, trained, maintained, operated, and relied upon the system that produced the decision?

This represents an important conceptual shift.

The legally relevant actor may no longer be a single person or agency.

Instead, the decision may emerge from:

Legislature + regulator + procurement authority + vendor + data provider + algorithm + human official

The law therefore increasingly needs to regulate the decision system, not merely the final decision-maker.

XXI. Conclusion

Algorithmic governance vendors become “hidden regulators” when their technological systems materially determine how public rules are interpreted, prioritized, classified, and enforced without the vendor formally appearing as a regulator.

The central legal danger is not simply automation. It is the transfer of practical normative power without corresponding transparency and accountability.

The cases involving Bridges, Loomis, SyRI, Ewert, and SCHUFA-related algorithmic scoring demonstrate different dimensions of the problem: legality, transparency, proprietary secrecy, reliability, fundamental rights, and the practical significance of automated classifications.

The emerging legal principle can therefore be stated as:

Government cannot escape public-law accountability merely because the mechanism through which governmental power is exercised has been purchased from a private technology vendor.

Where an algorithm materially determines access to rights, benefits, licences, liberty, public services, or regulatory treatment, the government should remain legally accountable for the system's design, deployment, oversight, and consequences. The deeper challenge for administrative and constitutional law is ensuring that private technological expertise does not become unreviewable public power.