Short answer: yes — if no one outside your organization creates, receives, maintains, or transmits patient data. An open-weight model running on hospital-controlled hardware, operated by your own staff, involves no business associate for the model itself. The moment PHI goes to a cloud AI service, or a vendor’s staff can reach PHI to install, support, or tune the system, a Business Associate Agreement is required for that party.
When HIPAA requires a BAA
Under HIPAA, a business associate is a person or organization that creates, receives, maintains, or transmits protected health information on behalf of a covered entity (45 CFR §160.103). Before a business associate handles PHI, the covered entity needs a written Business Associate Agreement that limits how the data can be used and requires it to be safeguarded.
So the BAA question for any AI deployment is not “is this AI?” but “who outside our organization touches the data?”
Six common LLM setups: which need a BAA?
| Setup | Who outside your organization touches PHI? | BAA needed? |
|---|---|---|
| Cloud LLM API or chat tool, PHI in prompts | The AI provider receives, processes, and may store it | Yes, with the provider |
| Cloud LLM with encrypted storage the provider cannot read | The provider still stores and processes ePHI | Yes — encryption does not remove business-associate status |
| Open-weight model on your servers, run by your staff | No one | No, for the model itself |
| On-premise model with vendor installation, support, or tuning | Vendor staff who can access PHI on the system | Yes, with that vendor |
| Software licensed from a vendor that never accesses your system or data | No one | Usually no |
| Properly de-identified data sent to a cloud tool | No one — de-identified data is not PHI | No, if de-identification is done correctly |
Encryption and the “conduit” exception don’t get cloud AI off the hook
Two arguments come up often. Neither holds for AI services:
- “The data is encrypted, so the provider can’t see it.” HHS’s guidance on cloud computing says a provider that stores or processes ePHI is a business associate even if the data is encrypted and the provider has no key. An LLM also has to read the text to work, so the data is decrypted for processing anyway.
- “They’re just a conduit.” The conduit exception covers transmission-only services, like a phone company or internet provider, that pass data along with only transient storage. A service that processes prompts and keeps logs or history is not a conduit.
No BAA doesn’t mean no HIPAA
Running a model in-house removes the outside party, not your obligations. The Security Rule applies to every system that holds ePHI, including an AI server:
- Include the AI system in your risk analysis.
- Restrict who and what can query it, with role-based access.
- Log every request with actor, timestamp, and data scope.
- Encrypt stored prompts, outputs, and logs.
- Keep the server behind a default-deny egress rule so it cannot send data out. How firewall policy applies across locations.
Questions to settle before you deploy
- Will any PHI leave our network, including telemetry, crash reports, or logs?
- Can anyone outside our organization access the system, on-site or remotely, for support or updates?
- Who performs fine-tuning, and on whose hardware does the training data live?
- Who owns the trained model weights or adapters, and does any vendor keep copies?
- If we plan to de-identify data instead, which method do we use, and who validates it?
How GofarAI approaches it
GofarAI’s platform is designed so that, in normal operation, patient data, prompts, and outputs stay on the hospital’s infrastructure and are not sent to GofarAI — there is no external model API in the data path. Some engagements, such as on-site fine-tuning, involve our people working inside a hospital’s environment; that work is covered by a written agreement with the hospital, including a Business Associate Agreement where HIPAA requires one. Adapters trained on hospital data belong to the hospital and are not retained by us.
Compare on-premise and cloud AI vendors · See the security model
Frequently asked questions
Do I need a BAA to use ChatGPT or another cloud LLM with patient data?
Yes. If prompts or files containing PHI are sent to a cloud AI service, the provider receives and processes PHI on your behalf, which makes it a business associate. Use it with PHI only under a signed Business Associate Agreement covering that service and only if your organization has approved it.
Does encryption remove the need for a BAA with a cloud AI provider?
No. HHS guidance on cloud computing says a provider that stores or processes ePHI is a business associate even when the data is encrypted and the provider does not hold the key.
If we run an open-weight model on our own servers, do we need a BAA?
Not for the model itself: no outside party receives the data. You may still need a BAA with any vendor whose staff or systems can access PHI on that server, such as for installation, support, or fine-tuning. And the HIPAA Security Rule still applies to the system you run.
Can we avoid a BAA by de-identifying data first?
Properly de-identified data is not PHI, so a BAA is not required to process it. The hard part is de-identification itself: clinical free text often contains names, dates, and other identifiers that are easy to miss, so it needs a reliable, documented method.
This article is general information, not legal advice. Confirm your organization’s obligations with your privacy officer or counsel.
Want AI without a new business associate in the data path? GofarAI Labs deploys open-weight models on hospital hardware. Plan a pilot →
