1. Contextual Introduction

In the last five years, the adoption of conversational artificial intelligence in the health sector has grown 215 % according to the IDC Health AI 2024 report. Patients demand immediate, accurate answers that are available 24 h/7 d, while providers look to reduce the operational load of their call‑centers. The result is a clear opportunity: a conversational agent that can handle symptom inquiries, appointment scheduling, laboratory results, and preventive education, all integrated with existing clinical systems (EHR, SIS, etc.).

However, implementation is not trivial. The challenges include:

• Regulation and privacy (HIPAA, GDPR, Health Data Protection Laws in each country).
• Latency: a delayed response erodes patient trust and impacts Core Web Vitals (CLS, LCP, FID).
• Scalability: seasonal peaks (influenza, COVID‑19) can multiply traffic by 5‑10×.

Solving these problems today is critical because competitors are already deploying LLM‑based bots that deliver a “near‑human” experience. Those who do not evolve will lose both user satisfaction and operational efficiency. In this article, I describe a robust architecture, step by step, from a naive prototype to a production‑ready solution, with an emphasis on Next.js, Node.js, Python, PostgreSQL, Supabase, and Docker.

2. Fundamentals and Architecture

2.1. Design Principles

2.2. Conceptual Diagram (described in text)

1. Web client (Next.js SSR + React SPA) → requests /api/chat via HTTP/2.
2. API Gateway (NGINX + Cloudflare) → routes the request to Node.js (Express) or FastAPI (Python) depending on the path (/api/chat vs /api/rag).
3. Orchestrator (Node.js) validates the JWT token, checks usage limits, and delegates to:
   • External LLM (OpenAI, Anthropic) or local model (llama.cpp) through a Factory.
   • RAG service (Supabase + pgvector) for retrieval of clinical documents.
4. Persistence: PostgreSQL with schemas conversations, patients, documents.
5. Cache: Redis (TTL 5 min) for identical responses and vector‑search results.
6. Observability: Prometheus + Grafana; logs in Loki.

2.3. Critical Trade‑offs

3. Practical Implementation

Below, I show the evolution of five critical components. Each block exceeds 15 lines and is ready to copy into a real project.

3.1. Basic Endpoint in Node.js (Express)

Naive version: receives the patient’s message and forwards it directly to the OpenAI API.

// src/api/chatNaive.ts
import express, { Request, Response } from "express";
import fetch from "node-fetch";

const router = express.Router();

/**
 * POST /api/chat-naive
 * body: { mensaje: string, pacienteId: string }
 */
router.post("/chat-naive", async (req: Request, res: Response) => {
  const { mensaje, pacienteId } = req.body;

  // No validation or error handling
  const openAiResponse = await fetch("https://api.openai.com/v1/chat/completions", {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
    },
    body: JSON.stringify({
      model: "gpt-4o-mini",
      messages: [{ role: "user", content: mensaje }],
    }),
  });

  const data = await openAiResponse.json();
  res.json({ respuesta: data.choices[0].message.content });
});

export default router;

Problems: no input validation, token count is not controlled, nothing is logged, and the response is delivered without caching.

3.2. Production‑grade Endpoint with Factory, Validation, and Cache

// src/api/chat.ts
import express, { Request, Response, NextFunction } from "express";
import { z } from "zod";
import { LLMFactory } from "../services/LLMFactory";
import redisClient from "../utils/redisClient";
import logger from "../utils/logger";

const router = express.Router();

// Zod schema to validate the payload
const chatSchema = z.object({
  mensaje: z.string().min(1).max(1000),
  pacienteId: z.string().uuid(),
});

/**
 * Validation middleware
 */
function validateBody(req: Request, res: Response, next: NextFunction) {
  const result = chatSchema.safeParse(req.body);
  if (!result.success) {
    logger.warn("Validation failed", result.error.format());
    return res.status(400).json({ error: "Invalid input data" });
  }
  // Attach parsed data
  req.body = result.data;
  next();
}

/**
 * POST /api/chat
 */
router.post("/chat", validateBody, async (req: Request, res: Response) => {
  const { mensaje, pacienteId } = req.body as { mensaje: string; pacienteId: string };
  const cacheKey = `chat:${pacienteId}:${hashString(mensaje)}`;

  // 1️⃣ Try to retrieve from Redis (TTL 300 s)
  const cached = await redisClient.get(cacheKey);
  if (cached) {
    logger.info("Cache hit", { cacheKey });
    return res.json(JSON.parse(cached));
  }

  // 2️⃣ Select LLM via Factory (could be SaaS or local)
  const llm = LLMFactory.getInstance(process.env.LLM_STRATEGY ?? "openai");

  try {
    const respuesta = await llm.generate({
      prompt: mensaje,
      userId: pacienteId,
    });

    const payload = { respuesta };
    // 3️⃣ Store in cache
    await redisClient.setEx(cacheKey, 300, JSON.stringify(payload));

    // 4️⃣ Persist conversation (Repository pattern)
    await llm.conversationRepository.save({
      pacienteId,
      mensaje,
      respuesta,
      timestamp: new Date(),
    });

    res.json(payload);
  } catch (error) {
    logger.error("LLM error", { error });
    res.status(502).json({ error: "AI service unavailable" });
  }
});

/**
 * Simple hash function (SHA‑256) to generate a cache key
 */
import crypto from "crypto";
function hashString(input: string): string {
  return crypto.createHash("sha256").update(input).digest("hex");
}

export default router;

Improvements introduced:

• Validation with Zod (prevents injection).
• Factory (LLMFactory) lets you switch providers without touching the endpoint.
• Cache in Redis with TTL to reduce token costs.
• Repository to persist the conversation (shown below).
• Structured logging (e.g., Loki).

3.3. LLM Service with Factory Pattern (Node)

// src/services/LLMFactory.ts
import { OpenAIService } from "./OpenAIService";
import { LocalLlamaService } from "./LocalLlamaService";
import { ILlmService } from "./ILlmService";

/**
 * Factory that returns a concrete implementation of ILlmService.
 * The strategy is defined via environment variable.
 */
export class LLMFactory {
  private static instance: ILlmService | null = null;

  static getInstance(strategy: string): ILlmService {
    if (this.instance) return this.instance;

    switch (strategy) {
      case "local":
        this.instance = new LocalLlamaService();
        break;
      case "openai":
      default:
        this.instance = new OpenAIService();
        break;
    }
    return this.instance;
  }
}

Common interface:

// src/services/ILlmService.ts
export interface ILlmService {
  generate(params: { prompt: string; userId: string }): Promise<string>;
  conversationRepository: ConversationRepository;
}

3.4. Conversation Repository Implementation (PostgreSQL + Supabase)

// src/repositories/ConversationRepository.ts
import { createClient, SupabaseClient } from "@supabase/supabase-js";

export interface Conversation {
  id?: number;
  pacienteId: string;
  mensaje: string;
  respuesta: string;
  timestamp: Date;
}

/**
 * Repository that abstracts access to the `conversations` table.
 * Uses Supabase as the data layer (PostgreSQL + Row Level Security).
 */
export class ConversationRepository {
  private client: SupabaseClient;

  constructor() {
    this.client = createClient(
      process.env.SUPABASE_URL!,
      process.env.SUPABASE_ANON_KEY!
    );
  }

  async save(conv: Conversation): Promise<void> {
    const { data, error } = await this.client
      .from<Conversation>("conversations")
      .insert([conv]);

    if (error) {
      throw new Error(`Error saving conversation: ${error.message}`);
    }
  }

  async getByPaciente(pacienteId: string, limit = 20) {
    const { data, error } = await this.client
      .from<Conversation>("conversations")
      .select("*")
      .eq("pacienteId", pacienteId)
      .order("timestamp", { ascending: false })
      .limit(limit);

    if (error) {
      throw new Error(`Error retrieving conversations: ${error.message}`);
    }
    return data;
  }
}

3.5. React Frontend (Next.js) – Naive Chat Component

// components/ChatBoxNaive.tsx
import { useState } from "react";

export default function ChatBoxNaive() {
  const [mensaje, setMensaje] = useState("");
  const [historial, setHistorial] = useState<string[]>([]);

  const enviar = async () => {
    const res = await fetch("/api/chat-naive", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ mensaje, pacienteId: "11111111-2222-3333-4444-555555555555" }),
    });
    const data = await res.json();
    setHistorial([...historial, `Me: ${mensaje}`, `Bot: ${data.respuesta}`]);
    setMensaje("");
  };

  return (
    <div className="p-4 border rounded">
      <div className="h-64 overflow-y-auto mb-2">
        {historial.map((line, i) => (
          <p key={i}>{line}</p>
        ))}
      </div>
      <input
        value={mensaje}
        onChange={(e) => setMensaje(e.target.value)}
        placeholder="Write your query..."
        className="border p-2 w-full"
      />
      <button onClick={enviar} className="mt-2 bg-blue-600 text-white p-2 w-full">
        Send
      </button>
    </div>
  );
}

3.6. Production‑grade Frontend with Lazy Loading, Memo, and Error Handling

// components/ChatBox.tsx
import { useState, useCallback, useEffect, memo } from "react";
import useSWR from "swr";

/**
 * SWR hook for fetch with exponential retries and browser cache.
 */
const fetcher = (url: string, body: any) =>
  fetch(url, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(body),
  }).then((r) => {
    if (!r.ok) throw new Error(`HTTP ${r.status}`);
    return r.json();
  });

interface Mensaje {
  rol: "user" | "bot";
  contenido: string;
}

/**
 * Memoized ChatBox to avoid unnecessary renders.
 */
const ChatBox = memo(function ChatBox() {
  const [input, setInput] = useState("");
  const [historial, setHistorial] = useState<Mensaje[]>([]);
  const pacienteId = "11111111-2222-3333-4444-555555555555"; // In production comes from JWT

  const { mutate, error, isValidating, data } = useSWR(
    // The key includes the message so SWR can cache it
    input ? ["/api/chat", { mensaje: input, pacienteId }] : null,
    fetcher,
    { revalidateOnFocus: false, shouldRetryOnError: true }
  );

  // When mutate returns data, add it to the history
  useEffect(() => {
    if (data?.respuesta) {
      setHistorial((prev) => [
        ...prev,
        { rol: "user", contenido: input },
        { rol: "bot", contenido: data.respuesta },
      ]);
      setInput("");
    }
  }, [data]);

  const enviar = useCallback(() => {
    if (!input.trim()) return;
    mutate(); // trigger SWR
  }, [input, mutate]);

  return (
    <section className="p-4 border rounded-lg shadow-md max-w-lg mx-auto">
      <div className="h-72 overflow-y-auto mb-3 bg-gray-50 p-2 rounded">
        {historial.map((msg, idx) => (
          <p key={idx} className={msg.rol === "user" ? "text-right" : "text-left"}>
            <strong>{msg.rol === "user" ? "You" : "Bot"}:</strong> {msg.contenido}
          </p>
        ))}
        {isValidating && <p className="text-gray-500">Typing…</p>}
        {error && <p className="text-red-600">Error: {error.message}</p>}
      </div>
      <div className="flex gap-2">
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Write your query..."
          className="border p-2 flex-1"
        />
        <button
          onClick={enviar}
          disabled={isValidating}
          className="bg-green-600 text-white px-4 py-2 disabled:opacity-50"
        >
          Send
        </button>
      </div>
    </section>
  );
});

export default ChatBox;

4. Deploying with Docker & CI/CD

Below is a minimal docker‑compose.yml that spins up the API, Redis, and PostgreSQL (via Supabase).

version: "3.9"

services:
  api:
    build: ./api
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - LLM_STRATEGY=openai
      - SUPABASE_URL=${SUPABASE_URL}
      - SUPABASE_ANON_KEY=${SUPABASE_ANON_KEY}
    depends_on:
      - redis
      - db

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru"]

  db:
    image: supabase/postgres:15
    ports:
      - "5432:5432"
    environment:
      - POSTGRES_PASSWORD=supersecret
      - POSTGRES_DB=healthbot
    volumes: