Handbook
301-02 — Model routing and gateways
A routing matrix (table) mapping CI jobs → transport profile → gateway availability.
Updated
What you will build
A routing matrix (table) mapping CI jobs → transport profile → gateway availability.
Prerequisites
- 301-01
- Guides: Model routing, Prompt caching
Files you will touch
Optional local env files (never commit) — reference only live Granite tests.
Step 1 — Document offline default
Default pytest excludes granite markers (pyproject.toml addopts). Record that CI stays network-free unless a dedicated job passes -m granite.
Step 2 — Profile checklist
For each environment (dev laptop, staging CI, prod):
| Column | Fill |
|---|---|
LlmEnvProfile.kind |
e.g. certificator |
| Timeouts | Align with gateway SLA |
| JSON mode preference | Matches provider capability |
Step 3 — Failure rehearsal
Read transport retry logs from transport helpers—plan backoff / circuit-breaking outside LCDL if needed.
Expected output
Operators know which job is allowed to dial live gateways.
Common failures
| Symptom | Mitigation |
|---|---|
| CI flakes on live endpoints | Split job; keep default offline |
Verify
pytest -q --collect-only | head
Confirm granite tests deselected unless opted in.
Optional live callout
Only after credentials exist:
pytest -q -m granite