Unified Chat — Bedrock + MCP Tools + Claude Code Dispatch

The unified chat module powers the AI Assistant in OutpostAI and Compass. It uses AWS Bedrock (Claude) for inference, 16 MCP servers (283 tools) for agentic tool calling, and Claude Code dispatch for autonomous code generation and deployment.

Unified Chat — Bedrock + MCP Tools + Claude Code Dispatch

The unified chat module (unified_chat.py) is the backend that powers the Federal Frontier AI Assistant — the conversational interface available in both OutpostAI and Compass. It replaced two divergent chat implementations (one vLLM-only, one Bedrock-only) with a single agentic backend that provides:

• AWS Bedrock inference (Claude Sonnet 4.6 / Opus 4.6) with Ollama fallback for edge
• 283 MCP tools from 16 servers, loaded via JSON-RPC
• Agentic loop — up to 10 iterations of LLM reasoning and tool calling per request
• Claude Code dispatch — spawn Kubernetes Jobs that write code, create PRs, and deploy via GitOps
• Runtime model switching — operators select Sonnet, Opus, or Ollama from the UI dropdown

Why It Exists

Before April 2026, OutpostAI and Compass each had their own chat backend:

The OutpostAI chatbot was broken — vLLM was unavailable, causing every request to return “Connection error.” The model selector dropdown showed “AWS Bedrock — Claude Opus 4.6” but the backend ignored it. The two implementations were diverging with no shared code.

The unified chat module replaces both with a single implementation that both frontends use.

Architecture

Request Flow

1. Operator types a message in the ChatBot.vue component (shared by OutpostAI and Compass)
2. Frontend sends POST /api/v1/chat with {message, image, context, conversation_history}
3. Trailboss API receives the request and delegates to unified_chat.chat()
4. Unified chat builds the conversation with a system prompt, loads MCP tools, and enters the agentic loop
5. The LLM (Bedrock or Ollama) reasons about which tools to call
6. Tool calls are executed against MCP servers via JSON-RPC
7. Results are fed back to the LLM for the next iteration
8. After up to 10 iterations, the final response is returned to the frontend

Agentic Loop

The core of the unified chat is a multi-iteration agentic loop. The LLM does not simply answer questions — it reasons about what tools to call, executes them, interprets results, and decides whether to call more tools or respond.

Iteration 1: User asks "What's the Ceph health?"
  → LLM decides to call ceph_get_cluster_health
  → Tool returns: {health: "HEALTH_WARN", checks: [...]}
  → LLM decides more data needed, calls ceph_get_storage_summary

Iteration 2:
  → Tool returns: {total: "24TB", used: "18TB", pools: [...]}
  → LLM has enough data, generates final response with formatted table

Total: 2 iterations, 2 tool calls, 1 response

Configuration

Deduplication

The agentic loop tracks every tool call by (tool_name, arguments) hash. If the LLM attempts to call the same tool with the same arguments twice, the cached result is returned instead of re-executing. This prevents infinite loops and redundant API calls.

MCP Tool Loading

On the first chat request, the unified chat module loads tool definitions from all 16 MCP servers via JSON-RPC tools/list calls. Results are cached for subsequent requests.

Server                    → Endpoint (JSON-RPC)                                    → Tools
ffo-mcp-server            → http://ffo-mcp-server.f3iai:50060/jsonrpc              → 7
keycloak-mcp-server       → http://keycloak-mcp-server.f3iai:50057/jsonrpc         → 12
openstack-mcp-server      → http://openstack-mcp-server.f3iai:8080/jsonrpc         → 31
ceph-mcp-server           → http://ceph-mcp-server.f3iai:50054/jsonrpc             → 12
atlassian-mcp-server      → http://atlassian-mcp-server.f3iai:50052/jsonrpc        → 23
grafana-mcp-server        → http://grafana-mcp-server.f3iai:50056/jsonrpc          → 28
argocd-mcp-server         → http://argocd-mcp-server.f3iai:50051/jsonrpc           → 11
gitea-mcp-server          → http://gitea-mcp-server.f3iai:50055/jsonrpc            → 22
kolla-mcp-server          → http://kolla-mcp-server.f3iai:50061/jsonrpc            → 10
k8s-mcp-server            → http://k8s-mcp-server.f3iai:50062/jsonrpc              → 20
web-mcp-server            → http://web-mcp-server.f3iai:50063/jsonrpc              → 4
federal-compliance-mcp    → http://federal-compliance-mcp-server.f3iai:50064/jsonrpc → 3
trailboss-mcp-server      → http://trailboss-mcp-server.f3iai:50059/jsonrpc        → 1

Tool names are prefixed with the server name to avoid collisions (e.g., keycloak_list_users, openstack_list_vms). The MCP tool format is converted to OpenAI function-calling format, then to Bedrock toolSpec format for the converse API.

Each MCP server URL is configurable via environment variables (e.g., KEYCLOAK_MCP_URL, CEPH_MCP_URL) to support both in-cluster service DNS and local development with port-forwarding.

Tool Categories

Infrastructure Query (Read)

Infrastructure Action (Write)

Claude Code Dispatch (Code + Deploy)

The claude_code_dispatch tool is unique — it spawns a Claude Code agent as a Kubernetes Job that can write code, create merge requests, and deploy changes through the GitOps pipeline.

Claude Code Dispatch

When an operator asks the chatbot to write code, fix a bug, or deploy a change, the LLM calls claude_code_dispatch with:

Severity Levels

How It Works

1. The unified chat module receives claude_code_dispatch from the LLM’s tool call
2. A unique session ID is generated (cc-<12-hex-chars>)
3. A Kubernetes Job manifest is built with the task prompt, repo, severity, and branch
4. The Job is created in the f3iai namespace via the K8s MCP server
5. The Claude Code agent runs with access to the full MCP tool surface
6. For MEDIUM severity: agent creates a branch, commits changes, opens a Gitea MR
7. The MR flows through the normal GitOps pipeline — human reviews, merges, ArgoCD syncs

Example Interaction

Operator: "Fix the CORS header bug in the openstack-mcp-server and open a PR"

LLM calls: claude_code_dispatch({
  task: "Fix CORS header handling in the OpenStack MCP server. The server
        returns 'Access-Control-Allow-Origin: *' for preflight requests but
        not for actual requests. Fix the middleware in src/main.py.",
  repo: "admin/federal-frontier-platform",
  severity: "medium"
})

Response: "Claude Code agent dispatched as K8s Job 'claude-code-cc-a3f8b1c2d4e5'.
The agent will:
1. Clone the repo
2. Fix the CORS middleware in openstack-mcp-server/src/main.py
3. Create a Gitea merge request for your review

Track progress in the OutpostAI Dispatches panel."

LLM Configuration

Runtime Model Switching

The chatbot UI includes a model selector dropdown. When the operator changes it, the frontend calls:

PUT /api/v1/llm/config?preset=bedrock-opus

This switches the active LLM at runtime — no restart required. Available presets:

Bedrock Integration

The unified chat uses the AWS Bedrock converse API (not invoke_model). This provides:

• Native tool calling — Bedrock handles tool use/result cycling natively
• Cross-region inference profiles — model IDs prefixed with us. for cross-region routing
• Automatic message formatting — OpenAI-format messages are converted to Bedrock’s alternating user/assistant format with toolUse and toolResult blocks

Credentials are provided via the aws-bedrock-credentials Kubernetes secret:

env:
  - name: AWS_ACCESS_KEY_ID
    valueFrom:
      secretKeyRef:
        name: aws-bedrock-credentials
        key: aws-access-key-id
  - name: AWS_SECRET_ACCESS_KEY
    valueFrom:
      secretKeyRef:
        name: aws-bedrock-credentials
        key: aws-secret-access-key

Sovereign Inference

For IL6 and tactical edge deployments where AWS Bedrock is unavailable, the dev preset routes to a local Ollama instance running Qwen 35B. The agentic loop works the same way — only the LLM endpoint changes. See Sovereign Inference for details on IL2-IL6 inference backends.

System Prompt

The unified chat includes a system prompt that instructs the LLM to:

1. Always use tools — never suggest CLI commands or code blocks for the user to run
2. Select the right tool — a categorized guide maps query types to specific tools
3. Use Claude Code dispatch for code/deploy tasks — don’t try to generate code inline
4. Confirm destructive operations — delete, restart, drain require operator confirmation
5. Use real data — never fabricate infrastructure state

The system prompt is ~30 lines and focuses on tool selection guidance rather than personality.

Deployment

Components

Environment Variables

Build and Deploy

# On build server (texas-dell-04):
cd ~/federal-frontier-platform
git pull
docker build -t harbor.vitro.lan/ffp/trailboss:v5.5.0 .
docker push harbor.vitro.lan/ffp/trailboss:v5.5.0

# Update deployment:
kubectl -n f3iai set image deployment/frontier-cluster-api \
  frontier-cluster-api=harbor.vitro.lan/ffp/trailboss:v5.5.0

File Structure

common/api/
  unified_chat.py       # Bedrock client, MCP tools, agentic loop, Claude Code dispatch
  trailboss_api.py      # FastAPI app — /api/v1/chat delegates to unified_chat.chat()

frontend/src/components/
  ChatBot.vue            # Shared chat UI component (model selector, message history, send)

The unified_chat.py module is ~600 lines and has no dependencies beyond httpx, boto3, and pydantic — all included in the Trailboss image.

Validated April 3, 2026

The unified chat was validated with real queries against live infrastructure:

• “What MCP tools are available?” — returned formatted tables of all 283 tools across 16 servers
• “Hello, what can you do?” — returned capability overview with tool-backed examples
• Model switching — runtime switch from Sonnet to Opus and back via UI dropdown
• MCP tool calling — agentic loop successfully queried Ceph, OpenStack, Keycloak, and Kubernetes
• Claude Code dispatch — tool definition present and callable by the LLM
• End-to-end — OutpostAI at outpostai.vitro.lan serving Bedrock-powered responses through frontier-cluster-api → unified_chat.chat()