AI Workstation chat routing

The Frontier llm-router sends a user’s chat from the OutpostAI AI Assistant and CompassAI Ask the Ontology to that user’s own AI Workstation, at no token cost, and falls back to the cloud tier when the device cannot serve.

User chats from the OutpostAI AI Assistant and CompassAI Ask the Ontology can be answered by the user’s own AI Workstation instead of the cloud tier. The Frontier llm-router sends the user’s chat to that user’s own enrolled device. Answers served on the device have no token cost. If the device cannot serve the request, the router falls back to the cloud tier automatically and says so.

AI Workstations, the router and the cloud tier all run in customer-controlled environments: customer hardware or the customer’s own hyperscaler account. Federal Frontier does not host any of them.

Enrolled AI Workstations make up the customer’s own Federal Frontier AI Compute Grid.

How it works

1. The AI Workstation enrolls through Keycloak, using PKCE or a device code.
2. The device keeps one outbound HTTPS WebSocket open to the router on port 443, with a heartbeat about every 30 seconds.
3. Over that connection the device reports its hardware, its models, each model’s origin, and a self-test result.
4. When the user chats in the OutpostAI AI Assistant or CompassAI Ask the Ontology, the router sends the chat to that user’s own device. The device runs the model and returns the answer, including tool calls.
5. The router writes an audit line for each request. Requests served on the device show tier: device.

The device agent is tool-capable; the current version is 0.3.1.

Measured examples

These figures were measured on one AI Workstation, an Apple M4 Max with 36 GB of memory running ggml-org/gemma-4-26B-A4B-it-GGUF:Q4_0 (about 83 tokens/s in its self-test). They come from router audit lines on October 30, 2026. They are examples, not guarantees.

Each of these requests was logged as tier: device, at $0 token cost.

Requirements

• One of these supported platforms:
  • macOS on Apple Silicon
  • Linux with an NVIDIA GPU
  • Windows with an NVIDIA GPU
• A model whose origin is on the allowed list. The allowed model origins are the United States, France, Canada and the United Arab Emirates.
• Outbound HTTPS on port 443 from the device to the router.
• A Keycloak account for the user.

On Linux and Windows, self-tests passed on an NVIDIA Tesla T4 with llama3.2:3b, at about 86 tokens/s on Linux and about 82 tokens/s on Windows.

Enrolling an AI Workstation

Install the Frontier CLI first: see Install the Frontier CLI.

Enroll the device, then start the agent:

frontier workstation enroll
frontier workstation run

frontier workstation enroll signs the device in through Keycloak (PKCE or device code). frontier workstation run opens the outbound connection to the router, reports the device’s hardware, models, model origins and self-test, and serves the user’s chats.

The AI Workstations tab

The AI Workstations tab shows the user’s enrolled device with the details it reports: hardware, models, each model’s origin, and its self-test result.

Privacy

The user’s prompts go from the router to the user’s own device.

Fallback to the cloud tier

The router falls back to the cloud tier automatically, and says so, when the device cannot serve the request. This happens when:

• the device is offline,
• the model is outside the allowed origin list,
• the model’s context is too small for the request, or
• the model returns a malformed tool call.

Troubleshooting

If chats are answered by the cloud tier instead of the AI Workstation, check the causes above:

In each case the request is still answered, by the cloud tier.