Search⌘ K
AI Features
Home
Courses
Forward Deployed Engineer for AI Systems

System Design: Fine-Tuning the Model

Explore how to fine-tune an open-weight AI model using a hospital's coded encounter history to enhance billing accuracy. Understand the full system design, from data extraction and preparation to model adaptation, evaluation, deployment, and integration into Epic workflows for coder review. This lesson covers practical considerations like training pipelines, model selection, fine-tuning techniques such as QLoRA, evaluation metrics, and on-premises deployment tailored to healthcare constraints.

We'll cover the following...

The decomposition established that the right approach is to adapt an open-weight model on the hospital's own coded encounter history and integrate the result into the coder's Epic workflow. This lesson designs that system. Every major decision connects back to a constraint or finding the case study surfaced. We start with an overview of the full system before designing each component.

Architecture overview

The two pipelines share the adapted model as their common point. The training pipeline produces it. The inference pipeline uses it.

  1. The training pipeline runs on a schedule and moves through four stages. The hospital’s Epic system is the source of both physician notes and confirmed code assignments. Those records are extracted, cleaned, and formatted as training examples. An open-weight base model is then adapted on those examples. The resulting adapter is evaluated against a held-out test set before it is promoted to the inference server.

  2. The inference pipeline runs each time a coder opens a new encounter. The physician note is retrieved from Epic and passed to the adapted model on the hospital’s inference server. The model returns a structured code set. The result is formatted, attributed back to the relevant phrases in the note, and surfaced inside the coder’s Epic coding screen for review and confirmation.

Architectural diagram of training and inference pipeline
Architectural diagram of training and inference pipeline

The sections below design each component in pipeline order, starting with the training data.

Training data

The training pipeline begins with the hospital’s existing coded encounter records. Before any model training starts, we establish what data is available and prepare it into a form the model can learn from.

Extracting records from Epic

We query Epic for completed encounters from the past year where a certified coder confirmed the final code assignments. Each record contains two things: the full ...