The Platform

The On-Premise AI Stack, Layer by Layer

Five layers. One deployment. Every layer designed to keep patient data inside your building. Hospitals don’t buy models — they buy working solutions. We ship the full solution.

The core platform, including local inference, model management, audit logging, fine-tuning, and the MCP gateway, is a working prototype available for hospital pilot evaluation.

What does an on-premise Gofar Labs deployment include?

An on-premise Gofar Labs deployment combines hospital hardware, an installable platform, open-weight models, workflow tools, and an MCP gateway. The stack is designed to run inside your building so your team controls the environment and the data.

Gofar Labs on-premise architecture Inside the hospital network, clinical documents and existing systems feed the Gofar Labs server, which runs open-weight models on hospital GPUs. Staff review every draft in a review interface, and every request is written to an encrypted audit log. Outside the network, cloud AI services are not part of the data path; outbound traffic can be blocked, and updates arrive as signed offline packages. YOUR HOSPITAL NETWORK Clinical documents PDFs, faxes, scans, referral packets Existing systems EHR exports, scheduling, other MCP endpoints Gofar Labs server Open-weight models on your GPU · document processing · MCP gateway GPU Staff review Approve · edit · reject every AI draft before it is used Audit log Every request: actor, time, data scope · encrypted at rest · streams to your SIEM Cloud AI APIs & external model providers Not in the data path No outbound dependency Egress can be blocked at your firewall Updates Signed offline packages, on your schedule
How a Gofar Labs deployment fits inside your network: documents and systems feed a server on your hardware, staff approve every draft, and every request is logged.

Layer 01

Hardware

We specify, source, and install the server and GPU on premise — or certify hardware you already own. A single 24GB card runs a quantized 8–14B model well for document workflows. Larger deployments get multi-GPU boxes with department isolation between models.

Speed is not the selling point. Privacy and local control are. Hospitals accept slightly slower processing in exchange for data never leaving the premises.

Assembling a multi-GPU server for an on-premise AI deployment
Assembling a multi-GPU server for an on-premise deployment.

Layer 02

Platform

The installable Gofar Labs package sits on top of your hardware. This is where HIPAA-friendly architecture lives.

  • Inference server (vLLM or llama.cpp)
  • Model manager — swap models as better open weights ship
  • Audit logging of every inference, prompt, and completion
  • Role-based access controls
  • Encryption at rest
  • Zero external calls — the platform never phones home

Layer 03

Models

Tuned open-weight small models in the 7B–14B range. Swappable as better open weights ship — you’re never locked to a single vendor’s roadmap.

Multi-model deployments for larger hospitals

  • Specialization by role — one model for revenue-cycle language, another for clinical documentation
  • Department isolation — separate fine-tunes so weights never mix information domains (important for 42 CFR Part 2 behavioral-health data)
  • A small router-classifier model in front decides which LLM handles each request

Layer 04

Tools

Workflow tools are modular plugins on the platform. You start with one, add modules over time. Same server, same models, more value every quarter.

Every tool follows the same shape: read → extract → classify → draft, with a human approving. Same engine, different prompts and output templates.

See the full tool catalog →

Layer 05

MCP Gateway

The Gofar Labs box exposes its tools as MCP endpoints, so authorized agents — EHR-adjacent systems, other vendors’ agents, future AI purchases — can call them via a standard protocol. It also acts as an MCP client, letting the local LLM call other hospital systems (scheduling, labs, bed management) that expose MCP endpoints.

The gateway enforces policy: which agent can call which tool, with what data scope, all logged. As hospitals accumulate more vendors’ AI agents, this becomes the trusted control point — every agent-to-agent interaction is inspectable.

Common questions

What hardware does an on-premise hospital LLM need?

For document workflows, a single 24GB GPU runs a quantized 8–14B open-weight model well. Larger deployments use multi-GPU servers with department isolation between models. We can specify and install new hardware or certify a server you already own.

Does patient data leave the hospital?

Not during normal operation. Inference, logging, and model storage run on the Gofar Labs server inside your network, and the platform is designed to work behind a strict egress firewall. Our security approach covers the controls in detail.

Which AI models does Gofar Labs deploy?

Open-weight small models in the 7B–14B range from families such as Llama, Qwen, Gemma, Phi, and Mistral. Each is tuned for a specific hospital workflow, and models can be swapped as better open weights ship, so you are not locked to one vendor.

Where does a hospital usually start?

With one workflow tool, most often document intake and routing. Additional workflow tools are added over time on the same server and the same models.

Planning a private LLM? Read our private LLM deployment guide for healthcare: model choice, hardware sizing, and governance.

Comparing vendors? Read our buyer’s guide: On-premise vs cloud AI for hospitals — which vendors deploy behind your firewall.

See the stack in your environment.

A short call. We map your workflows, walk through the deployment plan, and answer your security team’s questions.