In this week’s Data Diaries, we follow last week’s data-boundary thread into the harder question in Enterprise AI part 4: what happens when the model remembers a person it should forget, and who answers for the decisions that model then makes? Privacy and automated decisions sit on the same fault line, and the law now treats them that way.
Start with the legal floor. GDPR Article 22 — and equivalents under the UK Data (Use and Access) Act 2025 — give people the right to a human reviewer for consequential AI decisions, and the reviewer must hold genuine authority to override the model. Colorado SB 26-189, signed May 14, 2026 and effective January 1, 2027, layers a U.S. version on top. The EU AI Act Article 27 Fundamental Rights Impact Assessment (FRIA) reaches Annex III deployers in the public sector and certain private-sector deployers in banking and insurance — not every high-risk system, so scope the obligation before you scope the budget.
On employee monitoring, the Hamburg ruling (case 24 BVGa 1/24) addressed staff using ChatGPT on personal accounts, and the court rested its decision on the employer holding no access to the data. Read it narrowly, because any enterprise-controlled deployment likely flips that outcome and triggers German Works Constitution Act §87 BetrVG co-determination.
So what does this stack mean operationally? Your organization owns the data-controller role for AI outputs under GDPR, and that includes hallucinated facts your model invents about real people — California AB 1008 carries the same logic into U.S. consumer privacy. A customer files a deletion request on Monday; your CRM purges the row by Friday; your fine-tuned model still carries fragments inside its parameters, and retraining costs are non-trivial in both time and money.
What you don’t want is these tools intentionally or accidentally exfiltrating personal data to a third party — that is a disaster waiting to happen, and the legal exposure attaches to you, not the vendor. Courts have ruled that handing your data to a third-party company breaks attorney-client privilege, while running that same data on your own hardware preserves the privilege because the data never exfiltrates.
Now what does a defensible architecture look like? For HIPAA-class data, regulated financial records, and privileged legal material, the operational definition is simple: data never leaves your control. Cloud LLMs like ChatGPT, Claude, or Gemini do not meet that bar — not by default, not on a business-tier contract, not with a Data Processing Agreement bolted on. The architectural answer is an inference hub: vLLM serving Qwen, Gemma, or MedGemma on hardware sitting inside your own WAN or LAN, ideally with no outbound internet access at all.
There is zero excuse for a clinical enterprise to not have your own AI inference hub. Healthcare, legal, and financial-services enterprises should build that hub now and wire every regulated workflow to it before the next audit cycle. That’s all there is to it.
Mid-market leaders should not skip this conversation just because the hub itself sits beyond this quarter’s budget. Start now with the paperwork — Health Insurance Portability and Accountability Act (HIPAA) Business Associate Agreements (BAAs), a Data Protection Impact Assessment (DPIA) merged with the Article 27 FRIA into one document, and a named human reviewer with override authority — and put the inference hub on a 12 to 18 month capital plan. If you run a smaller agency or lean team, the lever moves to procurement: require BAAs and FRIA-equivalent assessments from every AI vendor, refuse contracts that withhold either, and add one paragraph to your privacy notice disclosing AI use, lawful basis, and the human-review path.
Next week, we move from where your data lives to where your work lives — the workforce side of enterprise AI, and what changes when the templated tasks no longer need a human to do them.
Newsletter
Podcast
