Back to Insights
AI & Machine Learning•The End of Per-Provider Agent Lock-In: Designing a BYOK Coding-Agent Control Plane for Claude, Codex, and Serverless Local Models in 2026•deep dive•November 20, 2026•17 min read

The End of Per-Provider Agent Lock-In: Designing a BYOK Coding-Agent Control Plane for Claude, Codex, and Serverless Local Models in 2026

Design a self-hosted BYOK control plane to make Claude Code, OpenAI Codex, and serverless local models interchangeable — with routing, governance, and fallback.

T
Tamiz UddinFull-Stack Engineer

The past two years of AI coding agents have been defined by a paradox. The models are commoditizing — Claude, GPT, and open-weight models increasingly converge on the same chat-completion core — but the agents aren't. Claude Code, OpenAI Codex, Cursor, and a dozen open-source clones each ship their own CLI, their own session model, their own prompt templates, their own MCP client, and, most painfully, their own provider contract. The result is lock-in not to a model, but to a vendor's implementation of an agent.

That's changing. In 2026, the winning pattern for platform teams is a BYOK (Bring Your Own Key) control plane: a thin, self-hosted layer that owns the provider keys, the routing, the cost governance, and the session state, while the agents themselves become interchangeable executors behind a unified interface. You bring the keys; the control plane handles the rest.

This is a deep-dive into designing that control plane. We'll walk through the architecture, the provider abstraction, capability-aware routing, session normalization, secret handling, fallback strategies, and observability — with runnable TypeScript you can adapt to your own stack.


1. Why the Agent Layer Is the New Lock-In

In 2023, the lock-in problem was simple: you couldn't swap models without rewriting prompts. That's largely solved. Anthropic and OpenAI both expose OpenAI-compatible endpoints, and open-weight models run on the same protocol. A chat.completions call is a chat.completions call.

But the agent layer is a different story. An agent isn't a model call — it's a system: a prompt template, a tool schema, a loop that calls the model, parses tool calls, executes them, and feeds results back. Each vendor implements that loop differently, with different conventions for:

  • Session format — how conversation history is stored and rehydrated.
  • Tool-calling protocol — the JSON schema for tool definitions and the parsing of tool_calls.
  • Prompt caching — how context is marked for cache reuse, and how cache invalidation works.
  • Configuration — ~/.claude/, .codex/, environment variables, and CI hooks.
  • Hooks and lifecycle — pre-tool, post-tool, and error hooks, each with their own contract.

Switch from Claude Code to Codex and you're not just changing a model — you're rewriting your CI pipeline, your hooks, your prompt cache, your session recovery, and your team's muscle memory. That's the lock-in.

The BYOK control plane inverts the dependency. Instead of your workflow depending on a specific agent, the agent depends on a generic provider contract. Your CI calls the control plane; the control plane decides which provider to hit, with whose keys, under what budget, and with what fallback. Swap providers and your workflow doesn't change.

2. What a BYOK Control Plane Actually Is

A BYOK control plane is a self-hosted service (or embedded library) that sits between your developers/CI and the actual model providers. It exposes a single unified agent API and translates requests to provider-specific implementations. "BYOK" means the user supplies their own API keys, which the control plane stores securely and uses on their behalf — never exposing them to the developer's machine or to CI logs.

Why build one? Four reasons:

ReasonWhat it buys you
PortabilitySwap Claude for Codex or a local model without touching workflows
GovernanceCentralized cost tracking, quotas, and audit trails
SecurityKeys never live in developer configs, .env files, or CI artifacts
IntelligenceCapability-aware routing, prompt caching, and automatic fallback

The control plane isn't an agent. It doesn't write code or run tools. It's a dispatcher and governor: it knows which providers exist, what each can do, what each costs, and how much budget each team has left. The actual agent logic (the loop, the tools, the file edits) stays in the client — the CLI, the IDE extension, or the CI job.

3. Architecture Overview

Here's the high-level topology:

text
+-------------------+       +--------------------------+       +----------------------+
|  Developer / CI   |       |      Control Plane       |       |      Providers        |
|                   |       |                          |       |                      |
|  - Agent CLI      |  -->  |  - Auth & Key Vault      |  -->  |  - Claude (Anthropic) |
|  - Agent SDK      |       |  - Routing Engine        |       |  - Codex (OpenAI)     |
|  - MCP Client     |       |  - Session Store         |       |  - Local / Serverless |
|                   |       |  - Cost Governor         |       |  - Open-weight models |
|                   |       |  - Circuit Breaker       |       |                      |
+-------------------+       +--------------------------+       +----------------------+

The control plane is composed of five core components:

  1. Auth & Key Vault — authenticates the client, retrieves the user's provider keys, and injects them into provider calls without exposing them to the client.
  2. Routing Engine — selects a provider and model based on the request's requirements and the provider's capabilities.
  3. Session Store — normalizes conversation state across providers so a session can be rehydrated by any agent.
  4. Cost Governor — enforces budgets, quotas, and rate limits before a call is dispatched.
  5. Circuit Breaker — tracks provider health and routes around failing providers.

Everything is observable: every request, every provider call, every token, every dollar is traced and attributed.

4. The Provider Abstraction

The heart of the control plane is a single interface. Every provider — Anthropic, OpenAI, local, serverless — implements it. This is the contract that makes providers interchangeable.

ts
// types.ts
export type ProviderId = "anthropic" | "openai" | "local" | "serverless";

export interface FileContext {
  path: string;
  content: string;
  language: string;
}

export interface ToolSpec {
  name: string;
  description: string;
  parameters: Record<string, unknown>;
}

export interface AgentRequest {
  sessionId: string;
  prompt: string;
  files?: FileContext[];
  tools?: ToolSpec[];
  model?: string;
  maxTokens?: number;
  temperature?: number;
  metadata?: Record<string, unknown>;
}

export interface TokenUsage {
  inputTokens: number;
  outputTokens: number;
  cacheReadTokens?: number;
  cacheWriteTokens?: number;
}

export interface AgentResponse {
  sessionId: string;
  provider: ProviderId;
  model: string;
  content: string;
  toolCalls?: ToolCall[];
  usage: TokenUsage;
  latencyMs: number;
  raw: unknown;
}

export interface AgentProvider {
  readonly id: ProviderId;
  readonly capabilities: ProviderCapabilities;
  execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse>;
  stream?(req: AgentRequest, ctx: ProviderContext): AsyncIterable<AgentChunk>;
}

The ProviderContext carries the decrypted API key, the request ID, and the trace span — everything a provider needs to make the call without knowing why it's being called.

ts
export interface ProviderContext {
  apiKey: string;
  requestId: string;
  traceParent?: string;
  signal?: AbortSignal;
}

Each provider implementation is thin. It translates an AgentRequest into the provider's native API shape and maps the response back into the unified AgentResponse. For example, an Anthropic implementation maps max_tokens, system, and tools to Anthropic's API, while an OpenAI implementation maps to chat.completions.

The key insight: the interface is the product. Once you have this contract, adding a new provider is a small, well-defined task — implement execute, declare capabilities, register it in the registry. It's the same pattern as a database driver or a cloud provider SDK.

5. Capability-Aware Routing

Not all providers are equal. A local Llama 3.3 model might be free but have a 128K context window and no prompt caching. Claude Opus 4.5 might be expensive but excellent at complex refactors. Codex might be best for multi-file edits but lack long-context support. The routing engine needs to know each provider's capabilities to make a smart choice.

ts
export interface ProviderCapabilities {
  maxContextTokens: number;
  supportsTools: boolean;
  supportsStreaming: boolean;
  supportsImages: boolean;
  supportsPromptCaching: boolean;
  costPerMillionInput: number;
  costPerMillionOutput: number;
  latencyP50Ms: number;
  reliabilityScore: number; // 0-1, updated by circuit breaker
}

Routing is a two-phase process: filter, then score.

Phase 1 — Filter by hard requirements. A provider is eligible only if it satisfies the non-negotiable constraints of the request:

ts
function isEligible(p: ProviderCapabilities, req: AgentRequest): boolean {
  const needsTools = req.tools && req.tools.length > 0;
  const contextTokens = estimateContextTokens(req);
  return (
    (!needsTools || p.supportsTools) &&
    p.maxContextTokens >= contextTokens &&
    (!req.metadata?.requiresImages || p.supportsImages)
  );
}

Phase 2 — Score by policy. Among eligible providers, compute a weighted score based on cost, latency, and capability fit. The weights come from a routing policy, which can be per-team, per-project, or per-request.

ts
export interface RoutingPolicy {
  weightCost: number;
  weightLatency: number;
  weightReliability: number;
  weightCapabilityFit: number;
  maxCostPerRequest?: number;
  preferProvider?: ProviderId;
}

function scoreProvider(
  p: ProviderCapabilities,
  req: AgentRequest,
  policy: RoutingPolicy
): number {
  const estTokens = estimateContextTokens(req);
  const estCost =
    (estTokens / 1_000_000) * p.costPerMillionInput +
    (req.maxTokens ?? 4096) / 1_000_000 * p.costPerMillionOutput;

  const costScore = 1 / (1 + estCost * 100); // normalize
  const latencyScore = 1 / (1 + p.latencyP50Ms / 1000);
  const reliabilityScore = p.reliabilityScore;
  const fitScore = p.supportsPromptCaching ? 1.2 : 1.0;

  return (
    policy.weightCost * costScore +
    policy.weightLatency * latencyScore +
    policy.weightReliability * reliabilityScore +
    policy.weightCapabilityFit * fitScore
  );
}

The engine then picks the highest-scoring provider, optionally respecting a preferProvider pin for testing or compliance. The result is that a cheap, fast local model wins for simple edits, while a premium model wins for complex, multi-file refactors — automatically, with no workflow changes.

6. Session and Context Normalization

The hardest part of cross-provider portability isn't the API — it's the session state. Claude Code stores sessions as a directory of JSON files with a specific prompt-cache layout. Codex uses a SQLite database. A local agent might use a simple transcript. If your control plane stores sessions in a vendor-specific format, you're back to lock-in.

The solution is a normalized session model that all providers can hydrate from and serialize to.

ts
export interface SessionMessage {
  role: "system" | "user" | "assistant" | "tool";
  content: string;
  toolCalls?: ToolCall[];
  toolResult?: ToolResult;
  timestamp: number;
  provider?: ProviderId; // which provider produced this message
}

export interface SessionState {
  sessionId: string;
  projectPath: string;
  messages: SessionMessage[];
  fileSnapshots: Map<string, string>; // path -> content hash
  createdAt: number;
  updatedAt: number;
  metadata: Record<string, unknown>;
}

When a request comes in, the control plane loads the SessionState, converts it to the target provider's expected format (e.g., Anthropic's system + messages array, or OpenAI's messages array), and executes. When the response comes back, it converts the provider's output back into SessionMessage entries and persists them.

This normalization is also where prompt caching gets handled. Different providers have different cache mechanisms — Anthropic uses cache_control blocks, OpenAI uses automatic prefix caching. The control plane decides which messages are cacheable and marks them appropriately, so you get cache hits even when you switch providers.

ts
function toAnthropicMessages(session: SessionState): AnthropicMessage[] {
  return session.messages.map((m) => ({
    role: m.role,
    content: m.content,
    ...(m.toolCalls && { tool_calls: m.toolCalls }),
    ...(isCacheable(m) && { cache_control: { type: "ephemeral" } }),
  }));
}

The key benefit: a developer can start a session with Claude, hit a rate limit, and the control plane can transparently rehydrate the session with Codex — without losing context, tool results, or file state.

7. Cost, Quota, and Rate-Limit Governance

BYOK means the user pays, but the platform still needs to govern how they pay. The cost governor is a pre-flight gate: before a request is dispatched, it checks whether the caller has budget, whether the provider's rate limit is respected, and whether the request fits within the team's quota.

ts
export interface BudgetLedger {
  teamId: string;
  userId: string;
  projectId: string;
  monthlyBudgetUsd: number;
  spentUsd: number;
  requestsThisMinute: number;
  tokensThisHour: number;
}

export class CostGovernor {
  constructor(private store: LedgerStore) {}

  async authorize(req: AgentRequest, provider: ProviderCapabilities): Promise<Decision> {
    const ledger = await this.store.getLedger(req.metadata?.teamId, req.metadata?.userId);
    const estCost = this.estimateCost(req, provider);

    if (ledger.spentUsd + estCost > ledger.monthlyBudgetUsd) {
      return { allowed: false, reason: "BUDGET_EXCEEDED" };
    }
    if (ledger.requestsThisMinute >= 60) {
      return { allowed: false, reason: "RATE_LIMITED" };
    }
    return { allowed: true, estimatedCostUsd: estCost };
  }

  async recordUsage(req: AgentRequest, usage: TokenUsage, provider: ProviderCapabilities) {
    const cost = this.calculateCost(usage, provider);
    await this.store.incrementSpend(req.metadata?.teamId, req.metadata?.userId, cost);
  }
}

This is where the control plane earns its keep in an enterprise: centralized cost attribution. Every request is tagged with team, user, project, and provider, so finance gets a clean ledger instead of a pile of individual provider invoices. You can enforce "no more than $500/month per developer" or "local models first for CI" — and the governor enforces it automatically.

8. Secret Handling and Key Rotation

The "BYOK" part is the security-critical piece. Keys must never appear in:

  • Developer .env files or shell history.
  • CI logs, build artifacts, or error traces.
  • The agent's own config files.

Instead, the control plane holds keys in a secret store — Vault, AWS Secrets Manager, or a KMS-backed encrypted database — and injects them per-request into the ProviderContext.

ts
export interface SecretStore {
  getKey(userId: string, provider: ProviderId): Promise<string>;
  rotateKey(userId: string, provider: ProviderId, newKey: string): Promise<void>;
  revokeKey(userId: string, provider: ProviderId): Promise<void>;
}

export class KeyVault {
  constructor(private secrets: SecretStore) {}

  async resolve(userId: string, provider: ProviderId): Promise<string> {
    // Decrypt on read, never log, never cache in plaintext
    const encrypted = await this.secrets.getKey(userId, provider);
    return decrypt(encrypted);
  }
}

Key rotation is a first-class concern. When a user rotates their Anthropic key, the control plane needs to update its store and invalidate any cached sessions that depend on the old key. The pattern is:

  1. User submits a new key via the control plane's admin API.
  2. The control plane encrypts it and writes it to the secret store.
  3. The old key is marked deprecated but retained for 24 hours to allow in-flight requests to complete.
  4. After the grace period, the old key is deleted.

This gives you a clean rotation path without breaking active sessions.

9. Serverless Local Models as a First-Class Provider

The most interesting 2026 trend is the rise of serverless local models: open-weight models (Qwen, DeepSeek, Llama) running on ephemeral GPU workers via Modal, RunPod, or Cloudflare Workers AI, or on a developer's own machine via Ollama. These aren't just "cheaper" — they change the economics of the control plane.

A local provider has a different cost profile:

MetricClaude OpusServerless Local
Cost per 1M input tokens~$15~$0 (compute already paid)
Cost per 1M output tokens~$75~$0
Context window200K128K–256K
Prompt cachingYesSometimes
Latency P50~2s~4–8s
PrivacyData leaves VPCData stays in VPC

The control plane treats "local" as just another provider, but with two special rules:

  1. Privacy-first routing: if the request's metadata includes sensitive: true (e.g., proprietary code), the router must prefer the local provider, regardless of cost.
  2. Capability ceiling: local models may not support complex tool-calling or long-context reasoning, so the router only sends tasks within their capability envelope.
ts
export class LocalProvider implements AgentProvider {
  readonly id = "local" as const;
  readonly capabilities: ProviderCapabilities = {
    maxContextTokens: 131_072,
    supportsTools: true,
    supportsStreaming: true,
    supportsImages: false,
    supportsPromptCaching: false,
    costPerMillionInput: 0,
    costPerMillionOutput: 0,
    latencyP50Ms: 6000,
    reliabilityScore: 0.9,
  };

  constructor(private endpoint: string) {}

  async execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse> {
    // Call Ollama / vLLM / Modal endpoint with the unified request
    const res = await fetch(`${this.endpoint}/v1/chat/completions`, {
      method: "POST",
      headers: { Authorization: `Bearer ${ctx.apiKey}` },
      body: JSON.stringify(toOpenAIShape(req)),
    });
    return fromOpenAIShape(await res.json(), this.id);
  }
}

The strategic value: for CI workloads, code review bots, and sensitive-code tasks, the local provider becomes the default, and premium providers become the exception — a dramatic cost reduction with zero workflow changes.

10. Fallback, Retries, and Degraded Modes

Providers fail. Rate limits hit. Models get deprecated. A control plane without resilience is just a single point of failure. The fallback chain is the mechanism that keeps the system alive.

ts
export class FallbackChain {
  constructor(
    private providers: AgentProvider[],
    private router: RoutingEngine,
    private breaker: CircuitBreaker
  ) {}

  async execute(req: AgentRequest, ctx: ProviderContext): Promise<AgentResponse> {
    const candidates = this.router.rank(req, this.providers);
    let lastError: Error | undefined;

    for (const provider of candidates) {
      if (this.breaker.isOpen(provider.id)) continue;

      try {
        const res = await this.withRetry(provider, req, ctx);
        this.breaker.recordSuccess(provider.id);
        return res;
      } catch (err) {
        lastError = err as Error;
        this.breaker.recordFailure(provider.id);
        // Log the fallback for observability
        logFallback(req, provider.id, lastError);
      }
    }

    throw new AgentUnavailableError(lastError);
  }

  private async withRetry(provider: AgentProvider, req: AgentRequest, ctx: ProviderContext) {
    const maxRetries = 2;
    for (let attempt = 0; attempt <= maxRetries; attempt++) {
      try {
        return await provider.execute(req, ctx);
      } catch (err) {
        if (attempt === maxRetries || !isRetryable(err)) throw err;
        await sleep(backoffMs(attempt)); // 500ms, 2s, 8s
      }
    }
  }
}

The circuit breaker tracks provider health. After N consecutive failures, it opens the circuit and the router skips that provider for a cooldown period. When the cooldown expires, the circuit goes "half-open" — one probe request is allowed through to test recovery.

ts
export class CircuitBreaker {
  private failures = new Map<ProviderId, number>();
  private lastFailureAt = new Map<ProviderId, number>();

  isOpen(provider: ProviderId): boolean {
    const failures = this.failures.get(provider) ?? 0;
    const lastFailure = this.lastFailureAt.get(provider) ?? 0;
    if (failures >= 5 && Date.now() - lastFailure < 30_000) return true;
    return false;
  }

  recordFailure(provider: ProviderId) {
    this.failures.set(provider, (this.failures.get(provider) ?? 0) + 1);
    this.lastFailureAt.set(provider, Date.now());
  }

  recordSuccess(provider: ProviderId) {
    this.failures.delete(provider);
  }
}

This is what "degraded mode" means in practice: if Anthropic is down, the control plane routes to Codex; if both are down, it routes to a local model. The developer never sees an error — they just see a slightly different model in the output.

11. Observability and Tracing

A control plane without observability is a black box. You need to know: which provider handled this request? How much did it cost? How long did it take? Did it fall back? Every request should emit an OpenTelemetry span with provider, model, token usage, and cost attributes.

ts
import { trace } from "@opentelemetry/api";

export function traceAgentCall(req: AgentRequest, provider: AgentProvider) {
  return trace.getTracer("byok-control-plane").startSpan("agent.execute", {
    attributes: {
      "agent.session_id": req.sessionId,
      "agent.provider": provider.id,
      "agent.model": req.model ?? "auto",
      "agent.request_id": crypto.randomUUID(),
    },
  });
}

export function recordUsage(
  span: Span,
  usage: TokenUsage,
  costUsd: number
) {
  span.setAttributes({
    "agent.tokens.input": usage.inputTokens,
    "agent.tokens.output": usage.outputTokens,
    "agent.tokens.cache_read": usage.cacheReadTokens ?? 0,
    "agent.cost.usd": costUsd,
  });
  span.end();
}

Attribution is the killer feature. Because every span carries team_id, user_id, and project_id (from request metadata), you can answer questions like:

  • "What did team X spend on Claude this month?"
  • "Which provider has the highest fallback rate?"
  • "What's the P95 latency for local models in CI?"

This turns the control plane from a convenience into a governance instrument.

12. Production Best Practices

Building a control plane is straightforward; operating one is where the discipline lives. Here's the checklist I'd hand to any team taking this on:

  • Start with one provider. Don't build the abstraction for five providers you don't have. Implement the interface for Claude, then add Codex once the contract is stable.
  • Define the capability contract first. The ProviderCapabilities interface is the spec. Document what each field means and how the router interprets it before you write any provider code.
  • Use config-as-code. Routing policies, budgets, and fallback chains should be declarative YAML or JSON, versioned in git, and reviewed like code.
  • Audit every call. Log the provider, model, tokens, cost, and decision reason for every request. You'll need this for cost disputes and incident analysis.
  • Cache aggressively. Prompt caching is the cheapest latency and cost win. Make the control plane responsible for marking cacheable messages, not the individual providers.
  • Treat keys as secrets, not config. Never log a key, never put it in a URL, never store it in plaintext. The KeyVault should be the only component that touches decrypted keys.
  • Test the fallback chain. Kill a provider in staging and verify the router degrades gracefully. This is a drill, not a one-time check.
  • Version the API. The unified agent API will evolve. Version it from day one so you can add providers without breaking existing clients.

13. Conclusion

The model layer is commoditized. The agent layer is not — and for the last two years, that's been the real lock-in. A BYOK control plane breaks it by moving the provider contract out of the agent and into a thin, self-hosted layer that you own.

The architecture is simple: one interface, five components (vault, router, session store, governor, breaker), and a lot of observability. The payoff is real: your developers can use Claude, Codex, or a local model interchangeably; your finance team gets a clean cost ledger; your security team gets keys out of the repo; and your platform team gets a single place to enforce policy.

The agents will keep evolving. The models will keep commoditizing. But if you own the control plane, you

...you control the future of your developer experience.

The agents will keep evolving. The models will keep commoditizing. But if you own the control plane, you control the future of your developer experience. That is the thesis of this article: not that you should build your own agent, but that you should build the interface through which all agents pass. The interface is the moat. The interface is the policy. The interface is the ledger. And the interface is yours.


The Reference Architecture: A Control Plane in Three Planes

Before we dive into code, let's establish a common mental model. A BYOK control plane is not a single service; it's a composition of three logical planes, each with a distinct responsibility. You can implement them as separate microservices, as modules in a single binary, or even as a serverless function bundle—but the separation of concerns must remain.

1. The Policy Plane

This is the brain. It decides what an agent is allowed to do, which model it may use, and when it may use it. It is the only component that holds the master keys—or rather, the references to the keys. The policy plane never exposes a secret to an agent directly; it injects it into the agent's runtime environment at the moment of invocation, and only after validating the request against a set of rules.

The policy plane exposes a single gRPC or REST endpoint: Authorize(Request) -> Session. The Session is a short-lived token that carries the resolved permissions, the model endpoint, and a reference to the key material. The key material itself is not in the token; it's in a secure vault that the runtime plane fetches on the agent's behalf.

2. The Runtime Plane

This is the executor. It takes an AuthorizedSession, fetches the key material from the vault, and launches the agent process (or container) with the key injected as an environment variable or a mounted secret file. The runtime plane is deliberately dumb: it does not know why a request is allowed, only that it is allowed. It enforces the session's time-to-live, network egress rules, and resource limits.

In a serverless context, the runtime plane is often a sidecar or a wrapper script that runs inside the agent's execution environment. In a Kubernetes context, it might be a mutating admission webhook that injects the secret into a pod spec.

3. The Ledger Plane

This is the accountant. It records every agent invocation: who asked, which model was used, how many tokens were consumed, how much time elapsed, and what the estimated cost was. The ledger plane is the source of truth for chargeback, budgeting, and anomaly detection. It writes to an append-only store (e.g., a Postgres table, a Kafka topic, or an S3 bucket of JSONL files).

The ledger plane is also the audit plane. Every policy decision is logged, including the reason for a denial. This is what makes the control plane defensible in a security review: you can always answer the question "why did this agent see this key?"


Tutorial: Building the Control Plane

Now let's build a minimal but functional control plane. We'll use Go for the service (because it's the language of choice for infrastructure tools), but the patterns translate directly to TypeScript, Rust, or Python. We'll assume you have a Kubernetes cluster, a Vault instance, and a Postgres database. If you don't, you can run all of this locally with docker compose.

Step 1: The Policy Definition

First, we need a way to express policy. We'll use a YAML file that defines which agents can use which models, and under what conditions. Here's an example:

yaml
# policies/team-default.yaml
apiVersion: controlplane.io/v1
kind: AgentPolicy
metadata:
  name: team-default
spec:
  agents:
    - id: "claude-code"
      allowed_models:
        - "anthropic/claude-3-5-sonnet"
        - "anthropic/claude-3-5-haiku"
      max_tokens_per_run: 200000
      max_runs_per_day: 50
      allowed_tools:
        - "read_file"
        - "write_file"
        - "bash"
      key_ref: "vault:secret/data/claude#api_key"
    - id: "codex"
      allowed_models:
        - "openai/gpt-4o"
        - "openai/gpt-4o-mini"
      max_tokens_per_run: 150000
      max_runs_per_day: 30
      allowed_tools:
        - "read_file"
        - "write_file"
      key_ref: "vault:secret/data/codex#api_key"
    - id: "local-serverless"
      allowed_models:
        - "local/llama-3.1-8b"
      max_tokens_per_run: 50000
      max_runs_per_day: 100
      allowed_tools:
        - "read_file"
        - "write_file"
        - "bash"
      key_ref: "none"  # no external key needed

The key_ref is a pointer to a secret in Vault. The runtime plane will resolve this pointer at invocation time. For local models, there is no external key, so we mark it as none.

Step 2: The Policy Engine

Now we write the Go code that loads this policy and evaluates requests. We'll keep it simple but production-ready.

go
// policy/policy.go
package policy

import (
	"context"
	"fmt"
	"os"
	"time"

	"gopkg.in/yaml.v3"
)

type AgentPolicy struct {
	APIVersion string `yaml:"apiVersion"`
	Kind       string `yaml:"kind"`
	Metadata   struct {
		Name string `yaml:"name"`
	} `yaml:"metadata"`
	Spec AgentPolicySpec `yaml:"spec"`
}

type AgentPolicySpec struct {
	Agents []AgentRule `yaml:"agents"`
}

type AgentRule struct {
	ID            string   `yaml:"id"`
	AllowedModels []string `yaml:"allowed_models"`
	MaxTokensPerRun int    `yaml:"max_tokens_per_run"`
	MaxRunsPerDay  int     `yaml:"max_runs_per_day"`
	AllowedTools   []string `yaml:"allowed_tools"`
	KeyRef         string   `yaml:"key_ref"`
}

type PolicyEngine struct {
	rules map[string]AgentRule
}

func NewPolicyEngine(path string) (*PolicyEngine, error) {
	data, err := os.ReadFile(path)
	if err != nil {
		return nil, fmt.Errorf("read policy file: %w", err)
	}

	var policy AgentPolicy
	if err := yaml.Unmarshal(data, &policy); err != nil {
		return nil, fmt.Errorf("parse policy file: %w", err)
	}

	rules := make(map[string]AgentRule)
	for _, rule := range policy.Spec.Agents {
		rules[rule.ID] = rule
	}

	return &PolicyEngine{rules: rules}, nil
}

type AuthorizeRequest struct {
	AgentID    string   `json:"agent_id"`
	Model      string   `json:"model"`
	Tools      []string `json:"tools"`
	UserID     string   `json:"user_id"`
	RequestedTokens int `json:"requested_tokens"`
}

type AuthorizeResponse struct {
	Allowed     bool   `json:"allowed"`
	Reason      string `json:"reason,omitempty"`
	SessionID   string `json:"session_id,omitempty"`
	KeyRef      string `json:"key_ref,omitempty"`
	ExpiresAt   time.Time `json:"expires_at"`
}

func (e *PolicyEngine) Authorize(ctx context.Context, req *AuthorizeRequest) (*AuthorizeResponse, error) {
	rule, ok := e.rules[req.AgentID]
	if !ok {
		return &AuthorizeResponse{Allowed: false, Reason: "agent not found"}, nil
	}

	// Check model
	modelAllowed := false
	for _, m := range rule.AllowedModels {
		if m == req.Model {
			modelAllowed = true
			break
		}
	}
	if !modelAllowed {
		return &AuthorizeResponse{Allowed: false, Reason: fmt.Sprintf("model %s not allowed for agent %s", req.Model, req.AgentID)}, nil
	}

	// Check tools
	for _, tool := range req.Tools {
		toolAllowed := false
		for _, t := range rule.AllowedTools {
			if t == tool {
				toolAllowed = true
				break
			}
		}
		if !toolAllowed {
			return &AuthorizeResponse{Allowed: false, Reason: fmt.Sprintf("tool %s not allowed", tool)}, nil
		}
	}

	// Check token budget
	if req.RequestedTokens > rule.MaxTokensPerRun {
		return &AuthorizeResponse{Allowed: false, Reason: fmt.Sprintf("requested tokens %d exceed max %d", req.RequestedTokens, rule.MaxTokensPerRun)}, nil
	}

	// Check daily run limit (simplified - in real impl, query ledger)
	// We'll assume a ledger client is injected here.
	// For now, just pass.

	// Generate a session ID and expiry
	sessionID := generateSessionID()
	expiresAt := time.Now().Add(15 * time.Minute)

	return &AuthorizeResponse{
		Allowed:   true,
		SessionID: sessionID,
		KeyRef:    rule.KeyRef,
		ExpiresAt: expiresAt,
	}, nil
}

func generateSessionID() string {
	// Use crypto/rand in production
	return fmt.Sprintf("sess_%d", time.Now().UnixNano())
}

Step 3: The Runtime Plane (Agent Launcher)

Now we need something that actually launches the agent. We'll create a simple CLI that takes a session token and the agent ID, fetches the key from Vault, and runs the agent with the key injected as an environment variable.

go
// runtime/launcher.go
package runtime

import (
	"context"
	"fmt"
	"os"
	"os/exec"
	"time"

	"github.com/hashicorp/vault/api"
)

type Launcher struct {
	vaultClient *api.Client
}

func NewLauncher(vaultAddr, vaultToken string) (*Launcher, error) {
	config := api.DefaultConfig()
	config.Address = vaultAddr

	client, err := api.NewClient(config)
	if err != nil {
		return nil, fmt.Errorf("create vault client: %w", err)
	}
	client.SetToken(vaultToken)

	return &Launcher{vaultClient: client}, nil
}

func (l *Launcher) Launch(ctx context.Context, agentID, sessionID, keyRef string, env map[string]string) error {
	// Resolve key from Vault
	key, err := l.resolveKey(keyRef)
	if err != nil {
		return fmt.Errorf("resolve key: %w", err)
	}

	// Build the command
	cmd := exec.CommandContext(ctx, "agent", "--session", sessionID)
	cmd.Env = os.Environ()
	cmd.Env = append(cmd.Env, fmt.Sprintf("AGENT_API_KEY=%s", key))

	// Add additional env vars from the policy
	for k, v := range env {
		cmd.Env = append(cmd.Env, fmt.Sprintf("%s=%s", k, v))
	}

	// Set a hard timeout
	cmd.WaitDelay = 30 * time.Second

	// Capture output for logging
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr

	if err := cmd.Run(); err != nil {
		return fmt.Errorf("agent execution failed: %w", err)
	}
	return nil
}

func (l *Launcher) resolveKey(keyRef string) (string, error) {
	if keyRef == "none" {
		return "", nil
	}
	// Parse "vault:secret/data/claude#api_key"
	// In production, use proper parsing. Here we assume a simple format.
	// We'll just read the whole secret and return the first value.
	secret, err := l.vaultClient.Logical().Read("secret/data/claude")
	if err != nil {
		return "", err
	}
	if secret == nil || secret.Data == nil {
		return "", fmt.Errorf("secret not found")
	}
	data := secret.Data["data"].(map[string]interface{})
	key, ok := data["api_key"].(string)
	if !ok {
		return "", fmt.Errorf("api_key not found in secret")
	}
	return key, nil
}

Step 4: The Ledger Plane

Finally, we need to record everything. We'll write a simple ledger that appends JSONL to a file or a Postgres table. For the tutorial, we'll use a file-based append-only log.

go
// ledger/ledger.go
package ledger

import (
	"encoding/json"
	"fmt"
	"os"
	"sync"
	"time"
)

type LedgerEntry struct {
	Timestamp   time.Time `json:"timestamp"`
	UserID      string    `json:"user_id"`
	AgentID     string    `json:"agent_id"`
	Model       string    `json:"model"`
	TokensIn    int       `json:"tokens_in"`
	TokensOut   int       `json:"tokens_out"`
	CostUSD     float64   `json:"cost_usd"`
	SessionID   string    `json:"session_id"`
	Decision    string    `json:"decision"` // "allowed" or "denied"
	Reason      string    `json:"reason,omitempty"`
}

type Ledger struct {
	mu     sync.Mutex
	file   *os.File
	encoder *json.Encoder
}

func NewLedger(path string) (*Ledger, error) {
	f, err := os.OpenFile(path, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
	if err != nil {
		return nil, fmt.Errorf("open ledger file: %w", err)
	}
	return &Ledger{
		file:    f,
		encoder: json.NewEncoder(f),
	}, nil
}

func (l *Ledger) Record(entry LedgerEntry) error {
	l.mu.Lock()
	defer l.mu.Unlock()
	return l.encoder.Encode(entry)
}

func (l *Ledger) Close() error {
	return l.file.Close()
}

Step 5: Wiring It All Together

Now we need a small HTTP server that exposes the Authorize endpoint and calls the launcher. We'll use the standard library to keep dependencies minimal.

go
// main.go
package main

import (
	"context"
	"encoding/json"
	"log"
	"net/http"
	"os"
	"time"

	"yourmodule/ledger"
	"yourmodule/policy"
	"yourmodule/runtime"
)

func main() {
	policyPath := os.Getenv("POLICY_PATH")
	if policyPath == "" {
		policyPath = "policies/team-default.yaml"
	}

	engine, err := policy.NewPolicyEngine(policyPath)
	if err != nil {
		log.Fatalf("failed to load policy: %v", err)
	}

	ledgerPath := os.Getenv("LEDGER_PATH")
	if ledgerPath == "" {
		ledgerPath = "/var/log/controlplane/ledger.jsonl"
	}
	ldg, err := ledger.NewLedger(ledgerPath)
	if err != nil {
		log.Fatalf("failed to open ledger: %v", err)
	}
	defer ldg.Close()

	vaultAddr := os.Getenv("VAULT_ADDR")
	vaultToken := os.Getenv("VAULT_TOKEN")
	launcher, err := runtime.NewLauncher(vaultAddr, vaultToken)
	if err != nil {
		log.Fatalf("failed to create launcher: %v", err)
	}

	http.HandleFunc("/authorize", func(w http.ResponseWriter, r *http.Request) {
		var req policy.AuthorizeRequest
		if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
			http.Error(w, "invalid request", http.StatusBadRequest)
			return
		}

		resp, err := engine.Authorize(r.Context(), &req)
		if err != nil {
			http.Error(w, "internal error", http.StatusInternalServerError)
			return
		}

		// Record the decision
		entry := ledger.LedgerEntry{
			Timestamp: time.Now(),
			UserID:    req.UserID,
			AgentID:   req.AgentID,
			Model:     req.Model,
			TokensIn:  req.RequestedTokens,
			Decision:  "denied",
		}
		if resp.Allowed {
			entry.Decision = "allowed"
			entry.SessionID = resp.SessionID
			entry.KeyRef = resp.KeyRef
			entry.CostUSD = estimateCost(req.Model, req.RequestedTokens) // placeholder
		} else {
			entry.Reason = resp.Reason
		}
		if err := ldg.Record(entry); err != nil {
			log.Printf("ledger record error: %v", err)
		}

		w.Header().Set("Content-Type", "application/json")
		json.NewEncoder(w).Encode(resp)
	})

	http.HandleFunc("/launch", func(w http.ResponseWriter, r *http.Request) {
		// In a real system, this would be called by the agent itself after receiving a session token.
		// For simplicity, we just launch the agent with the session ID.
		var req struct {
			AgentID   string            `json:"agent_id"`
			SessionID string            `json:"session_id"`
			KeyRef    string            `json:"key_ref"`
			Env       map[string]string `json:"env"`
		}
		if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
			http.Error(w, "invalid request", http.StatusBadRequest)
			return
		}

		ctx, cancel := context.WithTimeout(r.Context(), 10*time.Minute)
		defer cancel()

		if err := launcher.Launch(ctx, req.AgentID, req.SessionID, req.KeyRef, req.Env); err != nil {
			http.Error(w, err.Error(), http.StatusInternalServerError)
			return
		}

		w.WriteHeader(http.StatusOK)
		w.Write([]byte(`{"status":"launched"}`))
	})

	log.Println("control plane listening on :8080")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func estimateCost(model string, tokens int) float64 {
	// A very rough cost model. In production, you'd query a pricing table.
	switch model {
	case "anthropic/claude-3-5-sonnet":
		return float64(tokens) * 0.000003 // $3 per million tokens
	case "openai/gpt-4o":
		return float64(tokens) * 0.0000025 // $2.5 per million tokens
	default:
		return 0.0
	}
}

Step 6: The Client Side

The final piece is the client. This is what your developers actually run. It's a thin wrapper around the agent CLI that talks to the control plane first.

bash
#!/bin/bash
# byok-agent - a wrapper for Claude Code, Codex, or any other agent

set -euo pipefail

CONTROL_PLANE_URL="${CONTROL_PLANE_URL:-http://localhost:8080}"
AGENT_ID="${AGENT_ID:-claude-code}"
MODEL="${MODEL:-anthropic/claude-3-5-sonnet}"
USER_ID="${USER_ID:-$(whoami)}"

# Step 1: Request authorization
AUTH_RESPONSE=$(curl -s -X POST "$CONTROL_PLANE_URL/authorize" \
  -H "Content-Type: application/json" \
  -d "{\"agent_id\":\"$AGENT_ID\",\"model\":\"$MODEL\",\"tools\":[\"read_file\",\"write_file\"],\"user_id\":\"$USER_ID\",\"requested_tokens\":100000}")

ALLOWED=$(echo "$AUTH_RESPONSE" | jq -r '.allowed')
if [ "$ALLOWED" != "true" ]; then
  echo "Authorization denied: $(echo "$AUTH_RESPONSE" | jq -r '.reason')" >&2
  exit 1
fi

SESSION_ID=$(echo "$AUTH_RESPONSE" | jq -r '.session_id')
KEY_REF=$(echo "$AUTH_RESPONSE" | jq -r '.key_ref')

# Step 2: Launch the agent via the runtime plane
curl -s -X POST "$CONTROL_PLANE_URL/launch" \
  -H "Content-Type: application/json" \
  -d "{\"agent_id\":\"$AGENT_ID\",\"session_id\":\"$SESSION_ID\",\"key_ref\":\"$KEY_REF\"}"

# Step 3: The agent runs with the key injected by the runtime plane
# The runtime plane sets AGENT_API_KEY in the environment.

This is the complete flow. The developer never sees the key. The policy engine never sees the key. The runtime plane fetches it from Vault only at the moment of execution.


Deep Dive: Policy Enforcement and Secure Key Injection

The previous tutorial showed a basic implementation. Now let's examine the security-critical aspects in more detail.

Key Rotation and Versioning

One of the biggest advantages of a control plane is that you can rotate keys without touching the developer's machine. When you rotate a key in Vault, you simply update the secret. The next time an agent runs, the runtime plane fetches the new version. There's no need to redistribute keys to hundreds of developers.

But there's a subtlety: if an agent is already running, it holds the old key. The runtime plane should support graceful revocation. We can do this by having the runtime plane monitor a "key version" in Vault. If the version changes mid-run, the runtime plane can send a SIGTERM to the agent process, forcing it to restart with the new key. This is a simple but effective pattern.

Session Tokens and Short-Lived Credentials

Instead of passing the actual API key to the agent, you can pass a short-lived token that the agent exchanges for the real key at the provider's side. This is what AWS STS does for IAM roles. For Claude and Codex, this is harder because they expect a static API key. However, you can use a proxy pattern:

  • The control plane issues a session token.
  • The agent calls the control plane's /exchange endpoint with the session token.
  • The control plane returns a one-time API key that is valid for 15 minutes.
  • The agent uses that key for its API calls.

This is more secure because the key is never stored on the developer's machine. It exists only in memory for the duration of the session. The trade-off is that you need to implement a proxy for the model API calls, or the agent must support a custom auth flow. Some agents (like Claude Code) support custom API endpoints, which makes this feasible.

Policy as Code

We used YAML for policy definitions. But in 2026, the best practice is to express policy as code—specifically, as a set of functions that are evaluated against the request. This allows for complex logic like "allow this model only if the user is in the platform group and the repository is critical-infra."

Here's a TypeScript example using a policy-as-code approach:

typescript
// policies/team-policy.ts
import { PolicyContext, PolicyDecision } from "@controlplane/sdk";

export default function evaluate(ctx: PolicyContext): PolicyDecision {
  const { user, repo, agent, model, tools } = ctx;

  // Deny if the user is not in the platform group
  if (!user.groups.includes("platform")) {
    return { allowed: false, reason: "user not in platform group" };
  }

  // Deny if the repo is critical-infra and the agent is not codex
  if (repo === "critical-infra" && agent.id !== "codex") {
    return { allowed: false, reason: "only codex allowed on critical-infra" };
  }

  // Allow only specific models
  if (!["anthropic/claude-3-5-sonnet", "openai/gpt-4o"].includes(model)) {
    return { allowed: false, reason: "model not allowed" };
  }

  // Allow
  return {
    allowed: true,
    keyRef: `vault:secret/data/${agent.id}#api_key`,
    sessionTtl: 15 * 60,
  };
}

This is much more flexible than YAML. You can unit test it, version it, and review it like any other code. The control plane loads these functions via a plugin system (e.g., WASM or a sandboxed JavaScript runtime).

The Zero-Trust Agent Gateway

In a zero-trust model, the control plane is the only component that can talk to the model providers. The agents themselves never have direct network access to the providers. Instead, the control plane acts as a gateway:

  1. The agent sends a request to the control plane's /proxy endpoint.
  2. The control plane validates the session, checks the policy, and forwards the request to the model provider with the API key.
  3. The provider's response is streamed back to the agent.

This gives you complete visibility into every API call, the ability to rate-limit, and the ability to redact sensitive data in prompts. The downside is that you add latency and a single point of failure. In 2026, with the proliferation of local models, this is less of a concern because many requests can be served locally.


Deep Dive: Cost Attribution and Metering

The ledger plane is where the money is. In a BYOK world, the developer brings their own key, so you might think you don't need to track costs. But you do—because the whole point of a control plane is to provide a unified cost ledger across all providers, including local models.

The Cost Model

Every model provider has a different pricing structure. Some charge per token, some per request, some per second of compute. The ledger plane needs a normalized cost model. We'll define a simple schema:

json
{
  "timestamp": "2026-03-15T10:30:00Z",
  "user_id": "alice",
  "agent_id": "claude-code",
  "model": "anthropic/claude-3-5-sonnet",
  "tokens_in": 12000,
  "tokens_out": 4500,
  "cost_usd": 0.0495,
  "session_id": "sess_123",
  "metadata": {
    "repo": "acme/backend",
    "branch": "feature/foo",
    "duration_ms": 8500
  }
}

The cost_usd is calculated by the runtime plane after the agent finishes, using the actual token counts returned by the provider. For local models, the cost is the opportunity cost of the compute time—you can estimate it based on the GPU hours consumed.

The Metering Pipeline

The ledger plane should not be a single Postgres table. In a high-throughput environment, you want an append-only log that is asynchronously aggregated. Here's a typical pipeline:

  1. The runtime plane writes a RunStarted event to a Kafka topic.
  2. The runtime plane writes a RunFinished event with the final token counts.
  3. A consumer joins the two events, calculates the cost, and writes to a time-series database (e.g., TimescaleDB or ClickHouse).
  4. A daily job aggregates the data into a summary table for dashboards.

For our tutorial, we'll keep it simple with a JSONL file, but the pattern is the same.

Chargeback and Budgets

Once you have the ledger, you can implement budgets. The policy engine can query the ledger to check if a user or team has exceeded their monthly budget. If they have, the policy engine denies new runs.

go
// In the policy engine's Authorize method:
func (e *PolicyEngine) checkBudget(ctx context.Context, userID string, agentID string) (bool, error) {
	// Query the ledger for the current month's spend
	// If spend > budget, return false
	// This is a placeholder - in production, you'd use a SQL query.
	return true, nil
}

Anomaly Detection

The ledger is also your security alarm. If a user suddenly starts requesting 10x their normal token usage, or if a user starts using a model they've never used before, the ledger plane can trigger an alert. This is the "canary in the coal mine" for compromised credentials.


Deep Dive: Serverless Local Models

The title mentions "Serverless Local Models." This is the most interesting and forward-looking part of the 2026 landscape. The idea is that you don't need to call an external API at all—you can run a model locally on your own hardware, but you want it to be serverless in the sense that it scales to zero and spins up on demand.

The Local Model Runtime

A serverless local model is a containerized inference server (e.g., vLLM, Ollama, or llama.cpp) that is started on demand. The control plane treats it like any other model provider, but instead of an API key, it needs a compute budget.

yaml
agents:
  - id: "local-serverless"
    allowed_models:
      - "local/llama-3.1-8b"
    max_tokens_per_run: 50000
    max_runs_per_day: 100
    allowed_tools:
      - "read_file"
      - "write_file"
      - "bash"
    key_ref: "none"
    runtime:
      type: "serverless"
      image: "ghcr.io/acme/llama-server:latest"
      gpu: "a10g"
      max_idle_seconds: 60
      min_instances: 0
      max_instances: 4

When a request comes in for local/llama-3.1-8b, the runtime plane checks if an instance is already running. If not, it starts one (using a Kubernetes Deployment or a Knative Service). The agent then connects to the local endpoint (e.g., http://localhost:8080/v1/chat/completions). After the run, the instance stays alive for max_idle_seconds and then scales to zero.

The Cost of Local Models

Local models have a different cost structure. The cost is not per token; it's per GPU-second. The ledger plane needs to account for this. We'll define a cost model that estimates the GPU cost based on the instance type and the duration of the run.

go
func estimateLocalCost(instanceType string, durationSeconds float64) float64 {
	// Example: $1.50 per hour for an A10G
	ratePerHour := map[string]float64{
		"a10g": 1.50,
		"t4":   0.80,
		"l4":   1.20,
	}
	rate, ok := ratePerHour[instanceType]
	if !ok {
		rate = 1.00
	}
	return rate * (durationSeconds / 3600.0)
}

The Hybrid Model

The real power of a BYOK control plane is that it can choose between a local model and a cloud model based on policy. For example:

  • If the request is for a simple code completion, use the local model (cheap, fast, private).
  • If the request is for a complex refactoring, use Claude (expensive, but smarter).

This is a routing decision that the policy engine makes. The developer doesn't choose; the control plane does. This is the ultimate form of lock-in prevention: you're not locked into any single provider because the control plane can switch providers transparently.


Conclusion: The End of Lock-In

We've covered a lot of ground. Let's recap the key architectural decisions:

  1. The control plane is the only component that holds keys. Agents never see them directly.
  2. Policy is code, not config. You can test it, version it, and review it.
  3. The ledger is the source of truth for cost and audit. Every decision is recorded.
  4. Local models are first-class citizens. They have the same policy and ledger treatment as cloud models.
  5. Routing is a policy decision. The control plane chooses the best model for the job, not the developer.

The future of coding agents is not about which model wins. It's about which control plane wins. If you build a control plane that is provider-agnostic, policy-driven, and cost-aware, you will never be locked into a single agent again. Your developers will use whatever tool is best for the job, and your platform team will have a single place to enforce security, budget, and compliance.

The agents will keep evolving. The models will keep commoditizing. But if you own the control plane, you control the future of your developer experience.


This article is part of a series on infrastructure for AI development. The code examples are simplified for readability; production implementations should use proper error handling, observability, and security best practices.