Model Selection Guide
Open-weight local catalog and Bedrock options for Federal Frontier — pick by UMA, tools, and provenance; tags churn.
Last updated: September 2026
Choosing models for Federal Frontier agentic tool calling. Model names and tags churn — treat every tag below as an illustration; verify current Ollama and Hugging Face names before you pull or deploy. Prefer the most capable open-weight, no-cost model that fits your unified memory (local), or a Bedrock model your account has enabled (connected). The catalogs below are the models these docs cover.
Developer workstation vs production — Local Ollama on Apple Silicon is for development, operator tooling, and CompassAI on a boundary-approved AI workstation. Production air-gapped paths use vLLM; connected CUI paths often use Bedrock over VPC PrivateLink. See Sovereign Inference.
Provenance & policy
| Rule | Guidance |
|---|---|
| Local / workstation / air-gap weights | Open-weight, no-cost models from the catalog below |
| Connected cloud (Bedrock) | Models from the Bedrock families listed below (account/region enablement required) |
| IL6 / air-gap | Prefer documented, vetted open-weight instruct models (e.g. Meta Llama) |
Operators should not treat any single Ollama tag or HF id as “the FFP model.” Pick per workload: tool-calling quality, context size, latency, and memory.
Local open-weight catalog (Ollama illustrations)
Strong open-weight families commonly used with Ollama (illustrative tags — verify before pull):
| Family | Example tags | Notes |
|---|---|---|
| Meta Llama 4 | llama4:scout / llama4:17b-scout-16e-instruct-q4_K_M, llama4:maverick / llama4:17b-maverick-128e-instruct-q4_K_M |
MoE-style Scout/Maverick variants; good workstation candidates when UMA allows |
| Meta Llama 3.3 / 3.1 | llama3.3:70b-instruct-q4_K_M, llama3.1:8b, llama3.1:405b-instruct-q4_K_M (and q8_0 on very high UMA) |
Broad ecosystem; 8B for edge/dev; 70B-class for serious tool calling; 405B needs large UMA |
| Microsoft Phi | phi4:14b |
Compact option for lighter workstations |
| Mistral / Mixtral / Devstral | current Mistral instruct / Mixtral / Devstral tags | Strong code and fast inference options |
| Google Gemma | current Gemma instruct tags | When available as open weights via Ollama |
# Illustrative pulls — substitute any allowed open-weight tag that fits your UMA
ollama pull llama4:scout
ollama pull llama3.3:70b-instruct-q4_K_M
ollama pull llama3.1:8b
ollama pull phi4:14b
Env samples may show LLM_MODEL=llama4:scout (or another Llama illustration). That is not exclusive — pick any open-weight tag from families in this catalog that fits your UMA and tool-calling needs.
Apple Silicon UMA ladder (through 512GB)
Prefer large unified memory. Floor for serious multi-tool agent work is typically ~64GB; CompassAI-class workstations benefit from ≥96GB. Mac Studio-class machines with 128–512GB can hold much larger open-weight models.
| Unified memory | Class of model that typically fits (Q4-ish unless noted) |
|---|---|
| ~36–48GB | Small/dev: 8B–14B class (llama3.1:8b, phi4:14b, similar) |
| ~64GB | 70B Q4-class fallback (e.g. llama3.3:70b-instruct-q4_K_M) with KV headroom for tool payloads |
| ~96GB | Recommended CompassAI workstation band — Llama 4 Scout Q4-class or 70B Q4/Q5 with comfortable KV |
| ~128GB | Larger 70B quantizations or Scout at higher quants |
| ~192GB | Scout Q8-class or 405B at aggressive quants (e.g. Q2_K) |
| ~256GB | Llama 4 Maverick Q4-class or 405B mid quants |
| ~512GB | Maverick Q8-class or llama3.1:405b Q4/Q8 — Mac Studio-class |
Exact GB used = weights + KV cache + runtime. Measure with ollama ps under your real tool payload; do not size from weight files alone.
Tool calling (local)
Large MCP tool lists need models that return structured tool_calls reliably. Validate on your tool schema — quality varies by family, size, and quantization. Prefer larger instruct models on ≥64–96GB UMA for 100+ tools. Keep system prompts short (one to two lines) when tool JSON is large.
vLLM (air-gapped / Kubernetes)
Illustrative Hugging Face ids (Meta Llama open weights; verify revisions):
| Role | HF id | Notes |
|---|---|---|
| Primary example | meta-llama/Llama-3.3-70B-Instruct |
Production-style tool calling |
| Lightweight | meta-llama/Llama-3.1-8B-Instruct |
Fast / smaller GPU |
| Optional | meta-llama/Llama-3.1-70B-Instruct |
When 3.3 is unavailable |
For vLLM, enable structured tool calling with --enable-auto-tool-choice --tool-call-parser hermes (or llama3_json when validated for Llama 3.x).
Amazon Bedrock (connected)
These docs cover the Bedrock provider families below (region and account model access required). FFP often uses Anthropic Claude (Sonnet/Opus/Haiku families, including current 4.x lines already documented such as Sonnet/Opus 4.6) via VPC PrivateLink for connected agent work — that path is common, not exclusive.
| Provider family (examples) | Notes |
|---|---|
| Anthropic Claude (Sonnet / Opus / Haiku) | Common connected/CUI path via PrivateLink |
| Meta Llama on Bedrock (e.g. Llama 4 Scout/Maverick, Llama 3.3 70B) | Open-weight peer when enabled |
| Amazon Nova (Pro / Premier family) | AWS-native alternative |
| Mistral AI (e.g. Large / Devstral when offered) | Optional |
| AI21, Cohere, Amazon Titan, Google Gemma (when present on Bedrock) | Optional when enabled |
IL5: use Bedrock GovCloud region-available models only — do not invent GovCloud model IDs; confirm in the target region console.
Example commercial Bedrock model IDs (illustrative; confirm in-account):
- Claude Sonnet 4.6:
us.anthropic.claude-sonnet-4-6-v1:0 - Claude Opus 4.7:
us.anthropic.claude-opus-4-7-v1:0 - Llama 4 Scout:
meta.llama4-scout-17b-instruct-v1:0 - Llama 4 Maverick:
meta.llama4-maverick-17b-instruct-v1:0 - Llama 3.3 70B:
meta.llama3-3-70b-instruct-v1:0