Disclosure: Control Plane publishes this guide.

TL;DR

• What doubles and what does not. Infrastructure definitions gain a second provider module. Deployments, identity, observability, on-call, and compliance processes stay single if you standardize the operational layer. Kubernetes upgrades and networking double unless a platform takes them over.
• Warm standby vs. active-active. Warm standby is cheaper at rest but slower to recover, and a rarely used standby can fail when you need it. Active-active keeps serving capacity on both providers, with autoscaling holding it close to real traffic, and makes failover a routing change for stateless tiers with no cold start.
• The data layer. Async replication of a single primary is simplest. Distributed SQL databases such as CockroachDB or YugabyteDB can span providers when self-hosted, and need replicas in at least three failure domains to survive losing one provider. Object storage needs a sync tool, and backups restored on the other provider recover the slowest.
• Traffic routing. A global routing layer outside either provider, or health-checked DNS with low TTLs and client retries, steers traffic to healthy locations without manual intervention.
• The phased plan. Pick one critical service, containerize it, unify your pipeline and identity, replicate data, deploy to the second provider, send it traffic, drill failover and failback, and expand service by service.
• How Control Plane fits. Control Plane’s GVC lets you add a location on another provider and run the same workload definition active-active in every location from one deployment, with multi-location database templates for the data tier.

How do I add a second cloud provider for failover without doubling my ops work?

Standardize the operational layer and isolate provider differences in a thin substrate. Your deployment pipeline, identity model, secrets management, observability stack, alerting rules, compliance evidence process, and on-call rotation should all be provider-agnostic. The parts that change when you add a second provider are the infrastructure modules that provision resources there and the networking that connects them.

When teams skip this step, they end up running two parallel stacks with two sets of dashboards, two alerting configurations, two identity systems, and two on-call rotations. That is what doubles ops work, and it is avoidable.

This ops work ledger compares each operational task across three approaches: two fully native stacks, shared tooling (Terraform or OpenTofu plus Kubernetes plus GitOps), and a platform that abstracts providers, described here as Control Plane implements it.

Even with Terraform and a GitOps controller such as Argo CD or Flux deploying to both providers, you still manage version upgrades for two Kubernetes control planes, two sets of node pools with their own upgrade cadences, and two networking setups that need separate configuration and troubleshooting. Shared tooling removes much of the duplication and leaves the infrastructure lifecycle work in place. Run the GitOps controller per cluster, or outside both providers, so deployments still work when one provider is down.

How to fail over an application across cloud providers automatically

Automatic cross-provider failover needs four components working together.

Health checks that test real user journeys. Run synthetic checks from several vantage points outside both providers against the same endpoints your users hit, and fail over only when most of them agree for a sustained window. A checker inside the failing provider can go down with the workload, and a check that touches a shared database can fail both providers at once.

A global routing layer independent of either provider. This can be a global load balancer, a DNS service with health-checked records and low TTLs, or both. The routing layer cannot live only inside the provider you are failing away from, and it is a single point of failure in its own right, so run authoritative DNS on two providers or use a routing service with its own multi-provider footprint.

Stateless services deployed to both providers from one pipeline. When a service carries no local state, failover is a traffic routing decision. Containerized workloads with externalized configuration are the natural building block.

A data strategy aligned to your RPO. Each approach has a different recovery point.

Automated promotion needs a tiebreaker outside both providers, such as a witness or consensus store in a third location, so a partition between providers does not produce two primaries. The source provider bills cross-provider replication as egress, at lower per-GB rates over a dedicated or cross-cloud interconnect, so price your write and change volume before choosing a design. Replication also copies corruption and accidental deletes to the other provider, so keep point-in-time backups alongside any replication method.

Many teams combine approaches: a replicated primary or distributed SQL for transactional data, and object sync for media and assets. For routing and database patterns in more depth, see running an app across multiple regions with routing, databases, and failover.

Warm standby or active-active across providers?

The choice depends on your RTO and RPO targets, budget, team size, and data model.

Warm standby keeps a scaled-down copy of the application on the second provider. At failover you scale up compute, promote database replicas, and switch traffic. It costs less in normal operation, but recovery takes minutes to tens of minutes, and the scale-up can fail if the second provider lacks capacity or the environment has drifted from production.

Active-active across providers serves real traffic from both providers continuously. Every deployment exercises both environments, failover for stateless services is a routing change, and the second provider is never cold. It needs continuous serving capacity, headroom and raised quotas on each provider to absorb the other’s full load, and a data strategy that supports reads or writes in more than one location.

The underrated risk of warm standby is that a rarely used environment fails when you need it. Configuration drifts, capacity reservations lapse, and certificates renew on the primary but not on the standby. Teams that choose warm standby have to drill often to keep it viable. For a cost comparison, see active-active vs. active-passive cost.

A middle path is to run staging, batch jobs, or internal tools on the second provider, so it carries real load without serving production users and stays tested at limited cost.

How do I reduce the blast radius of a cloud provider outage?

Failover is only as strong as its weakest shared dependency.

• Map every dependency. Include DNS, the CDN and WAF, your identity provider, CI/CD, the secrets manager, the container registry, the observability backend, the status page, and third-party APIs such as payments, email, and authentication. Any of them living only on the failing provider stalls failover.
• Keep single-provider global control planes out of the failover path. A deployment tool, secrets vault, or service mesh control plane that runs on one provider becomes a bottleneck during that provider’s outage.
• Push container images to registries on both providers as part of CI, so you can deploy and scale on the second provider while the first is down.
• Replicate secrets and certificates to both providers, and rotate them in both. A rotation that reaches only one side breaks the other at failover, and if you publish CAA records, they must list every CA that issues your certificates.
• Isolate by cell or tenant. Partitioning the application keeps a failure in one cell from reaching every user.
• Use feature flags to degrade non-critical features when a dependency is unavailable, so core functions keep working.
• Monitor from outside the affected provider, so alerting still works when that provider is down.
• Keep DNS TTLs short and pair them with client retries. Failover still takes detection time plus TTL, some resolvers and runtimes cache past the TTL, and keep-alive, gRPC, and WebSocket connections stay on the old endpoint until they fail. Anycast global load balancing moves traffic faster than DNS changes.
• Run game days that simulate provider-level failures, then fix what breaks. See how to test failover without taking production down and, for AWS-specific steps, keeping your app online when an AWS region goes down.

Is multi-cloud a real solution to vendor lock-in?

Multi-cloud for failover reduces your dependence on one provider’s availability. Whether it reduces lock-in depends on how you build.

Workloads in containers that use open-source databases and standard protocols such as HTTP, gRPC, and SQL are portable. Moving their compute between providers is mostly configuration, while moving their data still takes migration time and egress cost. Workloads built deeply on provider-managed services, such as proprietary serverless functions, managed queues, or specialized AI APIs, stay locked in on each provider, and running on two providers can mean two sets of lock-in to maintain.

The practical approach is to use provider-specific services where they deliver clear value, isolate them behind application interfaces, and build the failover path on the portable layer. For a deeper analysis, see managed services vs. portability and cloud lock-in.

A phased plan for adding a second cloud provider

1. Pick one critical service and set RTO and RPO. Choose a service that matters enough to justify the work and is simple enough to succeed with first.
2. Containerize it and make it stateless. Externalize session state, configuration, and secrets.
3. Unify the pipeline, identity, secrets, and observability. One CI/CD pipeline deploys to both providers, one identity model governs access, one replicated secrets store provides credentials, and one backend, hosted outside either provider or on both, collects logs, metrics, and traces.
4. Replicate data. Pick an approach from the data table, set up the replica on the second provider, and confirm replication lag meets your RPO.
5. Deploy to the second provider and send it traffic. Real traffic is better than an idle standby because it validates the environment continuously.
6. Drill failover and failback. Simulate a provider outage, measure the RTO and RPO you actually achieve, and fix every gap. Make failback a planned manual step: rebuild the old primary as a replica of the promoted one, let it catch up, then move traffic back in a scheduled window.
7. Expand service by service, in order of business criticality. Each iteration is faster because the shared operational layer is in place.

How do I know ops work did not double?

Track these before and after adding the second provider:

• Engineer hours per release. A big jump means the pipeline is not unified.
• Number of pipelines. One per service, regardless of provider count.
• Alert volume per provider. It should follow traffic. A spike from the second provider signals drift.
• On-call pages per provider. A new class of incidents from the second provider means the environments are diverging.
• Drill RTO and RPO achieved. Stable or improving numbers show the failover design is maturing.
• Configuration drift. Growing differences between the two environments predict future ops burden.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes (MK8s), which runs Kubernetes nodes in your own cloud account or on your own servers, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

For example, a GVC can hold a GCP region next to an AWS region. A workload deployed to a GVC runs active-active in every location of that GVC from one definition, and a global endpoint routes clients to the nearest healthy location, so failover is a routing change with no standby to start. For a walkthrough, see how to run the same app on AWS, GCP, and Azure.

On Control Plane-managed locations in AWS, GCP, Azure, and Oracle Cloud regions, there is no Kubernetes cluster to operate, which removes the cluster-upgrade row of the ledger for those locations.

Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security. The CLI, console, Terraform and Pulumi providers, and the Control Plane MCP server all work through the same Control Plane API and cover every location. Control Plane ships its own Terraform and Pulumi providers, so it complements the infrastructure as code you already run. Control Plane’s Universal Cloud Identity (UCI) lets workloads in any location call AWS, GCP, and Azure services with no long-lived cloud keys in your code.

Control Plane covers the data layer too. Control Plane’s Template Catalog, a library of preconfigured stacks you install into a GVC, includes multi-location PostgreSQL and CockroachDB templates whose members span the locations of a GVC, so the database spans providers along with the application. Multi-location PostgreSQL keeps one primary with asynchronous replicas. With three or more locations, it promotes a replica in a surviving location automatically. With two locations, the surviving replica stays read-only until you promote it. CockroachDB with three or more locations keeps serving reads and writes through the loss of one location. To survive the loss of a whole provider with either template, spread at least three locations across three providers, so no provider holds a majority. You pick the template that matches your RPO and drill failover on a schedule.

During the October 20, 2025 AWS us-east-1 outage, Control Plane customers with us-east-1 in their GVC had workloads active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane’s internal incident telemetry (how that failover worked). For outage trends, see the 2026 cloud outage report.

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations, and those locations can be on different providers. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. You pay for the CPU and memory a workload reserves, with no contracts or minimums. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

FAQ

Answers to the questions engineering leaders ask most before adding a second provider.

Do I need Kubernetes on both clouds to fail over between them?

No. Kubernetes on each provider gives a consistent runtime at the cost of operating two clusters. On Control Plane-managed locations, the same workload runs on AWS, GCP, Azure, and Oracle Cloud with no cluster to manage. Teams that want Kubernetes can still use Control Plane’s Managed Kubernetes or add clusters they already run as locations.

What is the hardest part of multi-cloud failover?

The data layer. Stateless services move easily between providers, while stateful services need replication, fencing, and tested promotion. Most teams underestimate how hard it is to prove the replication strategy meets the RPO under real failure conditions.

How much does adding a second cloud for failover cost?

Warm standby costs the least at rest, active-active costs more but recovers fastest, and cross-provider data transfer adds cost to both. Warm standby keeps a scaled-down copy, so recovery is slower. Active-active runs both providers at full readiness, so recovery is a routing change. On Control Plane, active-active runs without paying for idle standby capacity, because every location serves real traffic and scales with it.

Can I fail over between clouds without a second operations team?

Yes. One pipeline, one identity model, one observability stack, and one on-call rotation can cover both providers. If the second provider needs its own team, the operational layer has not been standardized.

How do I test multi-cloud failover without affecting production users?

Send a small share of traffic to the second provider, validate it with synthetic checks, and simulate failures by blocking the primary provider’s endpoints in a controlled window. On Control Plane, a workload can be suspended in one location while others keep serving, which turns a drill into a configuration change.

How long does it take to add a second cloud provider?

For a stateless service with a unified pipeline already in place, days to weeks. For a stateful service with demanding replication requirements, weeks to months. The phased plan delivers value one service at a time.

What happens to compliance when I add a second provider?

Each provider adds its own audit surface, configuration baselines, and evidence. Standardizing policies, logging, and access across both keeps the work from doubling, but both providers stay in scope for your audits.

Is multi-cloud failover worth it if my primary provider has multi-region redundancy?

Yes, if you need to stay up through a provider-wide incident or a failure in a provider’s global services. Multi-region within one provider covers regional failures only.

To add a second cloud without a second operations team, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Every input in the model is stated so you can swap in your own numbers.

• Cost formula. Tasks × calls per task × tokens per call × token price, plus infrastructure (compute, networking, storage, observability).
• Worked result. 300,000 tasks a month at 8 model calls each costs about $21,919, or $0.073 per task, at list prices.
• Biggest levers. Prompt caching cut the token bill 45 percent. Routing simple steps to a smaller model, fewer calls per task, and shorter outputs come next.
• Infrastructure. Sandboxes, always-on orchestration replicas, NAT gateways, and trace storage came to about $799 a month, or $0.0027 per task.
• Control Plane runs agent services and their code-execution steps on AWS, GCP, and Azure with autoscaling and scale to zero, and, where enabled, Control Plane’s Capacity AI adjusts a workload’s reserved CPU and memory between set minimums and maximums based on historical usage.

What does it cost to run AI agent workloads in production?

Monthly cost splits into token spend and infrastructure spend.

Token spend depends on four variables: tasks, model calls per task, tokens per call (input and output), and price per token. Agents run multi-step loops that resend accumulated context, so they use far more tokens per task than a single-turn chatbot.

Infrastructure spend covers the orchestration services that run the agent loop, sandboxes that execute model-generated code, NAT gateways that carry traffic to model APIs, and storage for traces and logs.

Monthly cost = token cost across all calls + sandbox compute + orchestration compute + networking + storage

Worked model: 300,000 tasks per month

This is an illustrative scenario at list prices checked in November 2026. Check current pricing before you budget.

Assumptions

• 10,000 agent tasks per day, 300,000 per month.
• Each task makes 8 model calls.
• Each call sends 6,000 input tokens, of which 4,500 are a repeated prefix (system prompt, tool definitions, context) that is cached, and returns 400 output tokens. Input is held flat for simplicity; real loops grow context with each step, which raises cost, and token counts differ by model tokenizer.
• The first call in each task writes the cache and the other 7 read it, assuming calls are less than 5 minutes apart (each hit refreshes the 5-minute cache lifetime). A long sandbox run between calls can force a new write. If the prefix is identical across tasks and traffic is steady, most first calls also hit the cache, so this assumption is conservative.
• Model: Claude Sonnet 5.5 at list prices of $2 per million input tokens, $10 per million output tokens, $2.50 per million tokens for 5-minute cache writes, and $0.10 per million for cache reads (Claude pricing, prompt caching).

Without caching, all 14.4 billion input tokens bill at $2 per million ($28,800), plus $9,600 of output, for $38,400. Caching cuts the token bill 45 percent.

Prices are AWS us-east-1 on-demand list prices, applied to both regions in the model: AWS Fargate (Linux/x86) at $0.04048 per vCPU-hour and $0.004445 per GB-hour, NAT gateways at $0.045 per hour and $0.045 per GB processed, and Amazon S3 Standard at $0.023 per GB-month.

Sandbox hours count execution time only. Fargate bills per second with a one-minute minimum from the start of the image pull, so startup time, warm pools held for latency, and peak concurrency above the average all add cost.

The model deliberately leaves out databases, vector stores and embedding calls, observability or tracing subscriptions, load balancers, internet egress to end users, engineering time, and taxes, because they vary too much by architecture to price generically. The storage line prices S3 at rest only; log and trace ingestion and indexing are billed separately and often cost more than storage.

What the model shows

• Tokens dominate at this scale.
• Caching the repeated prefix is the single largest saving.
• Output tokens cost 5 times input tokens, so verbose outputs are expensive.
• Each model call removed from a task cuts cost roughly in proportion.
• Sandbox cost scales with execution minutes, so idle sandboxes should sleep.
• Always-on replicas and NAT gateways are fixed costs, so at much lower task volumes their share of the bill rises.

How do I estimate costs for AI agent infrastructure including sandbox execution, storage, and networking at scale?

Build the estimate from measured traces wherever you can, in seven steps:

1. Measure tokens per call and calls per task. Pull input and output token counts from your provider’s usage data or trace spans, then multiply by projected task volume. Re-count tokens with the target model before switching, because tokenizers differ.
2. Separate cached from uncached input. Identify the stable prefix (system prompt, tool schemas, shared context) and count how many calls per task read the cache and how many write it.
3. Size sandboxes by execution minutes. Estimate the share of tasks that run code, how long each run lasts, and the CPU and memory each sandbox needs.
4. Count always-on replicas and regions. Multiply replicas by their CPU and memory, by hours per month, and by the number of regions.
5. Add networking. NAT gateways charge per hour and per GB processed; add internet egress and cross-zone traffic between orchestration, sandboxes, and databases.
6. Add storage retention. Multiply per-task trace and output size by monthly volume and by the retention period to get steady-state storage.
7. Add a margin for retries and failed tasks. Size it from your observed failure and retry rates, since agents can retry steps or loop, and plan for provider rate limits, which can force higher usage tiers or queueing at peak.

How do I cut AI agent costs?

Roughly ordered by typical impact; items 1 and 3 are calculated in the worked model.

1. Prompt caching. Keep the cacheable prefix stable across calls within a task so later calls hit the cache.
2. Route simple steps to a smaller model. Parsing, extraction, and formatting steps rarely need a frontier model; reserve the larger model for planning and hard reasoning.
3. Make fewer calls per task. In the worked model, going from 8 calls to 6 cuts the token bill about 21 percent. Combine steps and detect completion early.
4. Shorten outputs. At $10 per million output tokens against $2 for input, concise and structured outputs save money on every call.
5. Batch non-interactive work. Anthropic’s Message Batches API costs 50 percent less than standard API calls, which suits nightly processing, evaluations, and bulk classification.
6. Let idle sandboxes sleep. Stop or sleep environments between tasks.
7. Right-size reservations. Match always-on replicas to measured load with autoscaling and right-sizing.
8. Avoid NAT processing charges where possible. For model APIs reachable through private endpoints, such as Amazon Bedrock over AWS PrivateLink interface endpoints, traffic skips NAT gateway processing, though the endpoints carry their own hourly and per-GB charges. The charge was small in the worked model and grows with data volume.

How much does setting up a production grade LangChain stack cost?

The open-source LangChain and LangGraph libraries are free under the MIT license, so production cost comes from what surrounds them: model tokens, hosting, a database or vector store, and observability.

• Model tokens remain the largest line item; the framework does not change per-token prices.
• Hosting for the agent service and its workers, with autoscaling, availability, and security, drives compute cost.
• Databases and vector stores depend on corpus size, embedding model, and query volume.
• Observability and evaluation. LangSmith, LangChain’s tracing and evaluation product, has a free tier and paid plans; see LangSmith pricing for current plans and limits.

At the volume in the worked model, a LangChain stack faces the same token bill as any other framework, plus hosting and tooling costs set by your architecture.

What infrastructure do I need to run AI agents securely at scale?

A secure production setup includes:

• Isolated sandboxes for model-generated code, ideally with microVM isolation, so generated code cannot escape to the host or other workloads. See the best AI agent sandboxes in 2026.
• Per-agent identity and least privilege, scoped to the services each agent needs.
• Short-lived credentials through identity federation instead of static API keys.
• Network egress controls that restrict which endpoints an agent can reach.
• Audit logs of every action an agent takes.
• Human approval for destructive or irreversible actions.
• Autoscaling and multiple regions for availability under spikes and regional outages.

For the full checklist, see how to let AI agents operate production infrastructure safely, and for why agent work should leave local machines, stop running AI agents on developer laptops.

Which platform gives the lowest cost for always-on production workloads?

The cheapest platform depends on your load pattern. Compare platforms on these factors:

• Billing granularity. Platforms bill provisioned instances, reserved CPU and memory, or per-request consumption at different granularities; unused reservation is waste unless right-sizing closes the gap.
• Right-sizing. Automatic adjustment of CPU and memory reservations to observed usage prevents chronic over-provisioning.
• Commitments vs. on-demand. Reserved instances and savings plans lower unit prices in exchange for a term commitment; on-demand keeps flexibility at a higher rate.
• Scale to zero. Services idle for hours, including staging and low-traffic agent endpoints, should cost little or nothing in compute while idle.
• Networking and egress fees. NAT, cross-region transfer, and internet egress pricing differ by provider and grow with data volume.
• Multi-region duplication. Running in two regions roughly doubles always-on capacity, so per-unit price and right-sizing matter more.

Compare total cost across all of these factors.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. Each workload runs across a Control Plane Global Virtual Cloud (GVC), a named set of locations across one or more clouds.

• Billing. You pay for the CPU and memory a workload reserves, with no contracts or minimums, and, where enabled, Capacity AI adjusts the reservation between set minimums and maximums based on historical usage. Egress, storage, observability, dedicated load balancers, and GPUs are billed separately, and Capacity AI does not apply to GPU workloads or to workloads that autoscale on CPU utilization or multiple metrics.
• Scale to zero. Serverless workloads scale to zero natively; standard and stateful workloads can scale to zero through KEDA, enabled per GVC.
• Sandboxes. Control Plane Sandboxes give coding agents dedicated environments, sized from 2 to 16 or more CPUs, that sleep when idle and spin back up in under one second. Sandboxes are for building and running agent code during development; production agent services, including their code-execution steps, run as Control Plane workloads. Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Sandboxes run in dedicated Kata Containers microVMs.
• Cloud access without stored keys. Control Plane’s Universal Cloud Identity (UCI) lets a workload call AWS, GCP, and Azure services, such as Amazon Bedrock or Google’s Gemini Enterprise Agent Platform (formerly Vertex AI), through an IAM role or service-account grant instead of long-lived keys in your code, with short-lived, least-privilege credentials at runtime.
• Operations. Built-in logs, metrics, and traces, and Control Plane’s MCP server, which lets AI assistants operate Control Plane resources with the signed-in user’s permissions. A Cost & Usage view, a billing_viewer role, and a monthly spend alert for billing admins track spend.
• Availability. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Agent services spread across locations keep running when a zone, region, or provider fails.

Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure; that applies to the compute share of agent costs, while model tokens are billed by the model provider. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. For how training, inference, and agents fit together, see AI infrastructure in 2026.

FAQ

Short answers on AI agent production costs.

What does it cost to run AI agent workloads in production?

It depends on task volume, calls per task, tokens per call, and model price. In the worked model, 300,000 tasks a month on Claude Sonnet 5.5 cost about $21,919: $21,120 in tokens and $799 in infrastructure.

How do I estimate costs for AI agent infrastructure including sandbox execution, storage, and networking at scale?

Measure tokens and calls from traces, split cached from uncached input, size sandboxes by execution minutes, count always-on replicas per region, then add NAT, egress, storage retention, and a retry margin.

How much does setting up a production grade LangChain stack cost?

The LangChain and LangGraph libraries are free under the MIT license; you pay for model tokens, hosting, a database or vector store, and observability such as LangSmith’s paid plans.

What infrastructure do I need to run AI agents securely at scale?

Isolated sandboxes for generated code, per-agent identity with least privilege, short-lived credentials, egress controls, audit logs, human approval for destructive actions, and autoscaling across regions.

Which platform gives the lowest cost for always-on production workloads?

It depends on load pattern. Compare billing granularity, right-sizing, commitment discounts, scale to zero, networking fees, and multi-region duplication.

What share of AI agent costs is tokens versus infrastructure?

Tokens dominate at moderate scale: 96.4 percent in the worked model. At low volume, fixed costs such as always-on replicas and NAT gateways take a larger share.

How much does prompt caching save?

In the worked model, caching the 4,500-token prefix cut the token bill from $38,400 to $21,120, or 45 percent.

Why are output tokens so expensive for agents?

Output costs 5 times input in the worked model ($10 versus $2 per million), and every call in every task pays it, so concise outputs save money quickly.

Does batch processing reduce agent costs?

Yes, for work that can wait. Anthropic’s Message Batches API costs 50 percent less than standard calls.

How do always-on costs affect agent economics at low volume?

They dominate it. The $347.64 of orchestration and NAT gateway hours in the worked model stays roughly the same at 1,000 tasks a month, so per-task infrastructure cost rises sharply unless idle orchestration services scale to zero and gateway count shrinks.

Talk to the Control Plane team or sign up free to run agent services with autoscaling and scale to zero.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Five deployment options:

• Managed model APIs: Amazon Bedrock, Google Gemini Enterprise Agent Platform (formerly Vertex AI), Microsoft Foundry (formerly Azure AI Foundry), and model providers’ own APIs
• Managed ML endpoints (Amazon SageMaker AI endpoints, Gemini Enterprise Agent Platform endpoints, Azure Machine Learning online endpoints)
• Serverless GPU inference services
• Self-hosted serving on containers or Kubernetes: vLLM, NVIDIA Dynamo-Triton (formerly Triton Inference Server), or KServe as the Kubernetes serving layer
• A platform layer that runs your model containers across clouds, such as Control Plane

Decision factors:

• Model type: a proprietary API model or an open-weight model
• Traffic shape: steady, bursty, or batch
• Latency target: interactive or offline
• Model features: structured output, tool calling, context length, and fine-tuned adapters
• Data control and residency: where prompts and completions are processed
• Cost model: per token, per GPU-hour, or reserved capacity
• Team skills: the operations experience you have to run what you choose

Top inference challenges: latency, GPU cost from low utilization, cold starts, autoscaling on the wrong signal, provider rate limits and outages, GPU capacity, data privacy, observability gaps, and multi-region availability.

Containers vs managed platforms: Running model servers in your own containers gives you full control over the runtime, weights, and optimizations. A platform layer removes the work of operating Kubernetes clusters, autoscaling, and multi-cloud networking while keeping that container-level control.

How Control Plane fits: Control Plane runs containerized model servers such as vLLM as workloads, on Control Plane-managed locations in AWS, GCP, and Azure regions or on GPU clusters you add as locations, and handles Day 2 operations for you, including autoscaling, built-in observability, and security.

How do I choose an AI model inference service for my application?

Start with these seven factors.

Model type

If you need a proprietary model from a provider such as OpenAI, Anthropic, or Google, you call it through a managed model API. If you run an open-weight model such as Llama, Mistral, or Qwen, you can call it per token through a managed API that hosts it, deploy it to a managed ML endpoint, or serve it yourself. Open-weight models give you portability and the ability to fine-tune, quantize, and change versions on your schedule.

Traffic shape

Steady, predictable traffic favors reserved or dedicated compute where GPU utilization stays high. Bursty traffic benefits from services that scale elastically and avoid paying for idle GPUs. Batch workloads, such as nightly embedding jobs, can run on spot or preemptible GPUs and tolerate queuing.

Latency target

Interactive applications such as chat, code completion, and retrieval-augmented search need a short time to first token and steady token output. Dedicated endpoints and self-hosted serving give the most predictable p99 latency. Offline workloads such as summarization pipelines or evaluation runs can tolerate cold starts and queuing.

Model features

List the features your application depends on before you pick a service. Structured output and tool calling vary by API and model, and self-hosted servers such as vLLM enforce JSON schemas through structured outputs (guided decoding). Context length sets KV cache memory, so long contexts need much more GPU memory per request than short ones. Serving many fine-tuned variants is cheapest as LoRA adapters on one base model, which vLLM can serve together on one GPU. Applications that call several models need a routing layer, such as an LLM gateway, for fallback and cost-based routing.

Data control and residency

Regulated industries and privacy-sensitive applications often require that prompts and completions stay in a specific region or in the organization’s own environment. Self-hosted serving and managed ML endpoints with private networking can meet these requirements. Managed model APIs process data on the provider’s infrastructure. Hyperscaler APIs such as Bedrock, Gemini Enterprise Agent Platform, and Microsoft Foundry offer regional processing and private connectivity, and retention terms vary by provider and contract. Check whether cross-region inference routing is on, because it can move requests out of your chosen region.

Cost model

Per-token pricing is simple to budget at low volume but grows with usage. Per-GPU-hour pricing rewards high utilization. Reserved capacity lowers the hourly rate but requires a volume forecast. Include idle GPU time, egress fees, and engineering time to get a true total cost of ownership. For the build-or-rent decision, see the build vs rent GPUs break-even model.

Team skills

Managed APIs need almost no MLOps experience. Managed ML endpoints need moderate DevOps and ML engineering effort. Self-hosted serving needs experience with GPU drivers, container runtimes, model optimization, autoscaling, and incident response. A platform layer needs container and configuration skills without cluster operations.

Decision table

Can you compare different AI model deployment solutions?

This table compares five inference deployment options across six dimensions, based on each option’s published documentation.

Many production architectures combine more than one option. A common pattern uses a managed model API for proprietary models and self-hosted serving for open-weight models where cost, data control, or customization matter.

What are common challenges in AI model inference and how can I overcome them?

This table maps each common problem to its cause and a fix.

For the infrastructure components behind these fixes, see what infrastructure you need to run an inference layer.

Where can I find a cloud friendly deployment platform for AI models?

A cloud friendly inference platform meets five criteria:

• Runs standard containers. You package your model server as a standard container image and deploy it without rewriting it for a proprietary runtime.
• Works on more than one cloud. You can run the same workload definition on AWS, GCP, and Azure regions and on your own GPU clusters.
• Supports GPUs. Your serving framework gets GPU access without manual driver work.
• Scales on demand signals. Autoscaling follows request concurrency or queue depth instead of generic CPU metrics.
• Keeps data where you choose. Inference traffic and model weights stay in the regions and clusters you select.

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. For a wider comparison of hosting platforms, see the best PaaS providers in 2026.

How do containers compare with a managed PaaS for scaling AI infrastructure with predictable costs?

Running model servers in containers yourself gives you full control over the serving framework, weights, quantization, and GPU choice. The cost is operational work, including Kubernetes clusters, GPU drivers, networking, autoscaling policies, certificate rotation, and incident response.

A managed platform takes over that layer. You still bring your container image, and the platform runs compute, scaling, load balancing, and Day 2 operations. The cost shifts from engineering hours to platform fees.

How Control Plane fits

Control Plane runs containerized model servers such as vLLM as workloads. On Control Plane-managed locations in AWS, GCP, and Azure regions there is no Kubernetes cluster to operate, and teams can also add GPU clusters they already run, including on a neocloud, as locations.

The vLLM template serves one Hugging Face model through vLLM’s OpenAI-compatible API on a built-in NVIDIA A10G or T4, or on your own GPU cluster registered as a location, and caches weights on a persistent volume so restarts skip the download. It runs in a GVC with exactly one location. For redundancy across regions, install it once per region, each in its own single-location GVC, and fail over between their endpoints in your client or gateway.

On Control Plane, serverless workloads scale on request concurrency and can scale to zero, and standard and stateful workloads can scale on custom metrics such as queue length through KEDA, enabled per GVC. A workload deployed to a GVC runs active-active in every location of that GVC from one definition, and the workload’s global endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes.

Control Plane’s Universal Cloud Identity (UCI) lets workloads call AWS, GCP, and Azure services, such as Amazon Bedrock or Google’s Gemini Enterprise Agent Platform, through an IAM role or service-account grant instead of long-lived keys in your code, with short-lived, least-privilege credentials at runtime, so an application can combine self-hosted open-weight models with managed model APIs.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

FAQ

These questions come up often when teams compare inference deployment options.

Should I use a model API or host my own model?

Use a model API when you need a proprietary model, want to start quickly, or have low to moderate traffic. Host your own model when you need control over data, want to fine-tune or quantize, or when per-token cost at your volume exceeds the cost of dedicated GPU compute.

What is the cheapest way to serve an open-weight model?

It depends on utilization. At high, steady traffic, self-hosted serving on reserved GPUs usually gives the lowest cost per token, and spot GPUs suit batch jobs that tolerate preemption. At low or bursty traffic, a per-token API that hosts the model, or a GPU service that scales down when idle, avoids paying for unused GPUs. Include engineering and operations time in the comparison.

How do I reduce inference latency?

Cut time to first token by reusing cached prompt prefixes and splitting long prompts into chunks. Cut the time between tokens with quantized weights and speculative decoding. Keep warm replicas near users so requests never wait on a cold start.

How do I keep inference running when a model provider has an outage?

Keep a fallback. For managed model APIs, route to an alternative provider or model when calls fail. For self-hosted models, run replicas in at least two regions. On Control Plane, a workload in a GVC with several locations runs active-active, and the workload’s global endpoint uses DNS to route clients to the nearest location where it passes its readiness probes. Every location in that GVC must offer the GPU model the workload requests.

What GPU should I use for LLM inference?

Match GPU memory to model size, context length, and concurrency. Weight memory is the parameter count times bytes per parameter: 2 at FP16 or BF16, 1 at FP8, and about 0.5 at 4-bit, plus headroom for the KV cache, which grows with context length and concurrent requests. A 7 or 8 billion parameter model needs about 14 to 16 GB at FP16 and fits on a 24 GB GPU such as the A10G with moderate context, or on a 16 GB T4 with 4-bit quantization. A 70 billion parameter model needs about 140 GB at FP16, so it runs on four 80 GB GPUs such as the H100 with tensor parallelism, on two 80 GB GPUs at FP8, on one 80 GB GPU only at 4-bit, or on one H200 or B200 at FP8. Control Plane’s pricing page lists NVIDIA GPUs from T4 to B200, subject to availability.

How do I autoscale AI inference workloads correctly?

Scale on request concurrency, waiting requests, or KV cache usage instead of CPU or GPU utilization, which do not reflect inference load. On Control Plane, a vLLM server running as a standard or stateful workload can scale on vLLM’s waiting-request count through KEDA.

Can I run inference across multiple clouds?

Yes. Package the model server as a container and run it in more than one cloud. On Control Plane, one workload definition runs across AWS, GCP, and Azure regions and your own clusters. A GPU workload uses one GPU source per GVC: built-in NVIDIA T4 or A10G on Control Plane-managed regions, or the GPUs on your own clusters added as locations.

How do I handle data privacy requirements for AI inference?

Host models in your own environment or use managed endpoints with private networking and regional processing. Send sensitive data to a third-party model API only when its retention and usage terms meet your requirements.

What infrastructure do I need to run an inference layer?

You need GPU compute, a model server such as vLLM or Dynamo-Triton, a container runtime or a serving layer such as KServe, autoscaling, load balancing, observability, and storage for model weights. For the full breakdown, see what infrastructure you need to run an inference layer.

Is a platform layer worth it compared to managing Kubernetes myself?

For most teams, yes, when engineering time is the constraint. A platform layer removes cluster operations and custom autoscaling setup while you keep control of the model server and weights. The tradeoff is a platform fee in place of engineering hours, and Control Plane customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.

To run inference workloads across clouds without operating clusters, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

• Twelve criteria. They cover isolation, lifecycle, developer experience, cost, and compliance. Use them as a vendor questionnaire and to find claims that need technical proof.
• Managed sandbox vs self-hosted sandbox vs production workload. The trade-off is who owns isolation, patching, capacity, and monitoring. Managed services cut operations work, self-hosting gives control for custom isolation and compliance, and production workloads add replicas, health checks, autoscaling, and SLAs.
• Alternatives exist. Allow-listed tool calls, WebAssembly runtimes, single-use CI runners, per-tenant isolated workloads, and human-approved actions reduce risk when general-purpose code execution is unnecessary.
• Portability. Use standard container images, open APIs and CLIs, workload identity for cloud access, and secrets kept outside the sandbox so you can change or mix providers.
• How Control Plane fits. Control Plane Sandboxes are for development and untrusted, agent-generated code in dedicated Kata Containers microVMs. Production agents run as Control Plane workloads, active-active across regions and clouds from one definition.

What should I look for in an AI agent sandbox?

Evaluate these 12 criteria before you select a sandbox for AI agents.

How do I choose an AI agent sandbox for my project?

Different projects weight the criteria differently. Find the row closest to your project and weight the checklist accordingly.

Many teams combine approaches: microVM sandboxes for customer-submitted code, and the agent’s API and orchestration on a production platform.

Can I use a self-hosted AI sandbox for production applications?

Yes, as the execution layer behind an agent product, if your team can staff host operations, upgrades, and on-call. You might run Kata Containers on your own Kubernetes cluster to give customer-submitted code microVM isolation, while the agent’s API, orchestration, and user-facing endpoints run as production workloads. In its standard setup, Kata Containers needs hardware virtualization, so nodes must be bare metal, or VMs with nested virtualization, which adds CPU and I/O overhead. The peer pods option launches each pod VM as a separate cloud instance instead. You also own the RuntimeClass setup, guest kernel and hypervisor upgrades, capacity planning for per-pod memory overhead, and any GPU passthrough.

| Managed sandbox service| Self-hosted sandbox| Production workload
---|---|---|---
What you operate| Your agent code and orchestration. The provider runs the sandbox runtime, hosts, and networking.| Hosts, the container or microVM runtime, networking, patching, capacity, and compliance controls.| Your application code and configuration. The platform runs compute, networking, scaling, and routing.
Isolation| Provider-managed, often microVMs or user-space kernels.| Your choice and configuration, such as Kata Containers on your Kubernetes.| Platform-enforced isolation with cgroups, namespaces, restricted syscalls and capabilities, network policy, and access controls.
Best for| Validating agent concepts and running untrusted code without deep infrastructure work.| Strict customization, data residency, or compliance needs with the operations capacity to match.| User-facing APIs, orchestration, and services that need replicas, health checks, autoscaling, and SLAs.
Watch out for| Lock-in, limited customization, and cost at sustained scale.| KVM-capable nodes, patching, scaling, monitoring, incident response, and security testing are all yours.| Arbitrary untrusted code belongs in a sandbox.

Are there alternatives to traditional AI agent sandboxes?

Yes. When an agent does not need general-purpose code execution, these options reduce attack surface and cost:

• Restricted tools. If the agent performs a fixed set of actions, give it an allow-list of tool calls or curated APIs instead of a shell.
• WebAssembly runtimes. WebAssembly runs code in a sandboxed linear memory that cannot reach host memory, and the host grants file, network, and system access explicitly through capability-based interfaces such as WASI. It suits small functions such as transformations, plugins, and user-defined functions, with lower overhead than a VM. It cannot run native binaries, and WASI support for networking, threads, and packages with C extensions is limited, so most agent toolchains will not run unchanged.
• Single-use CI runners. When agent tasks resemble build or test jobs, single-use runners give predictable cleanup and reuse existing controls. Use one runner per job with no repository secrets attached, since persistent self-hosted runners carry files and credentials between jobs.
• Per-tenant isolated workloads. When tenants run code your team wrote and reviewed, per-tenant workloads with resource limits, network policies, and service identities are often enough and cost less at scale.
• Human-approved actions. For sensitive operations, the agent proposes a concrete plan, a person approves it, and the action runs with logging.

For arbitrary, unpredictable model-generated code, a full sandbox remains the right choice.

How do I evaluate open ecosystems that let third-party tools plug into an AI agent infrastructure platform?

Check whether the platform exposes standard interfaces at every layer:

• MCP support. A Model Context Protocol server lets agents discover and call tools through a standard contract.
• Standard container images. You can move images between providers and run the same image locally and in CI.
• Open APIs and CLIs. Everything in the console is scriptable through a REST API with stable schemas and a CLI.
• Terraform or Pulumi providers. You can version, review, and automate configuration with the rest of your infrastructure.
• OpenTelemetry export. Logs, metrics, and traces flow to your existing observability stack.
• No proprietary runtime. No platform-only binaries or SDKs stand between you and another provider.

Ask each vendor for a concrete export workflow covering images, state, and telemetry. A vendor that cannot show one will be harder to leave.

How can I build a portable AI agent SDK that works across different sandbox providers and clouds?

Put a thin interface in front of every provider and keep responsibilities separate:

• Adapter interface. Make create, exec, upload, download, and destroy required, and treat snapshot, fork, suspend, and port exposure as optional capabilities each adapter declares. Define exec semantics (argument list or shell, streaming output, exit codes, timeouts) explicitly, since providers differ most there. Changing providers then means writing a new adapter.
• Standard images. Bake runtimes, packages, and tools into small, cacheable container images built from templates, so the same image runs on any provider that accepts standard container images.
• Secrets at runtime. Inject credentials when a task starts, scoped to that task, and use workload identity or short-lived tokens for cloud access.
• Configuration as code. Keep image definitions, network policies, and quotas in Terraform or Pulumi so environments are reproducible.

For guardrails on agents that touch production, see how to let AI agents operate production infrastructure safely.

How Control Plane fits

Control Plane Sandboxes are cloud development environments for building and for running untrusted, agent-generated code. Each sandbox provides a browser IDE (code-server), a browser terminal, SSH access for desktop IDEs through cpln sandbox connect in the Control Plane CLI, and a shareable public URL that serves only the app port you set when you create the sandbox. The IDE, terminal, and every other detected port sit behind the sandbox password, SSH is reachable only through a port-forward, and callback paths for OAuth and webhooks are open so external services can reach them. Treat anything served on the app port as public. Sandboxes start from toolbox images that you build from a template, from scratch, or from your own container image, with runtimes, packages, and AI coding tools such as Claude Code, Codex CLI, and Gemini CLI; templates and scratch builds include the Claude Code CLI. When an agent or developer installs packages in a running sandbox, Fork captures them into a new toolbox image.

The home directory (/root) sits on a persistent volume, so files there survive container restarts and suspend/resume. Suspend scales the sandbox to zero and keeps the volume, so you pay for storage only, and deleting the sandbox removes the volume.

Production agents run as Control Plane workloads. Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Sandboxes run in dedicated Kata Containers microVMs.

A workload deployed to Control Plane’s Global Virtual Cloud (GVC), a named set of locations across one or more clouds, runs active-active in every location of that GVC from one definition, and its canonical endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations.

Control Plane’s Universal Cloud Identity (UCI) lets workloads call AWS, GCP, and Azure services through an IAM role or service-account grant instead of long-lived keys in your code, with short-lived, least-privilege credentials at runtime. The Control Plane MCP server lets AI agents operate Control Plane through the same API as the console and CLI, with OAuth and per-org consent, and a readonly profile that has no write tools. Control Plane records platform actions in a tamper-proof audit trail and includes built-in logs, metrics, and traces for workloads. Control Plane ships its own Terraform and Pulumi providers, so it complements the infrastructure as code you already run.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. The Control Plane platform that runs these workloads is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. For more context, see why you should stop running AI agents on developer laptops and the best AI agent infrastructure platforms.

FAQ

These questions come up often when teams evaluate AI agent sandboxes.

How do I choose an AI agent sandbox?

Score candidates against the 12 criteria, weighted by your project type and risk profile, and pilot the top two with a representative workload before you commit.

Is a container enough isolation for AI agents?

For untrusted model-generated code, usually no. Containers share the host kernel, so microVMs or user-space kernels give stronger separation. Hardened containers can be enough for trusted internal code.

What is a microVM sandbox?

A microVM sandbox runs each environment inside a lightweight virtual machine with its own kernel, so a hypervisor using hardware virtualization, such as KVM, separates sandboxes from each other and from the host.

Should coding agents run on developer laptops?

No. Laptops hold broad access to credentials, source code, internal networks, and cloud accounts. Run coding agents in isolated sandboxes with controlled network access and scoped credentials.

How long should an AI agent sandbox live?

As long as the task needs. Ephemeral sandboxes should end with the task. Persistent sandboxes for active development can run for hours or days, with suspend and explicit lifecycle controls.

What is the difference between a sandbox and a production workload for AI agents?

A sandbox isolates untrusted code and developer work. A production workload serves users and orchestration with replicas, monitoring, autoscaling, routing, and SLAs.

How do I control network access in an AI agent sandbox?

Start with deny-by-default egress enforced outside the sandbox, allow-list only what the agent needs, block cloud metadata endpoints, keep exposed ports private unless you choose otherwise, and log outbound connections for audit.

Can I run AI agent sandboxes across multiple clouds?

Yes. Run one provider’s sandboxes in several clouds or regions, or put a common adapter in front of several providers. Standard images, open APIs, and workload identity keep the setup portable, and data residency rules decide where sandbox volumes may live.

What should I look for in AI agent sandbox security?

Strong isolation, controlled egress, short-lived scoped credentials injected at runtime, audit logs stored outside the sandbox, and patched, signed base images, plus human approval for high-risk actions.

How do managed and self-hosted AI sandboxes compare on cost?

Managed sandboxes usually cost more per unit of compute but remove operations work. Self-hosted sandboxes can cost less per unit at high, steady utilization, but microVM isolation usually needs bare-metal or nested-virtualization nodes and engineering time for provisioning, patching, monitoring, and capacity. Compare the cost per completed task, including retries, storage, and engineering hours.

To run agent sandboxes in dedicated microVMs and production agents active-active across regions and clouds on one platform, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Each layer covers a different failure. Most hybrid estates need a tool from every layer.

• Layer 1, keep services running (application availability): Control Plane, Red Hat OpenShift with Advanced Cluster Management, InfoScale, Linux Pacemaker
• Layer 2, recover data (backup, replication, immutability): Veeam Data Platform, HPE Zerto, Commvault Cloud, Rubrik Security Cloud, Cohesity, Veeam Kasten for Kubernetes, Google Cloud Backup and DR Service
• Layer 3, orchestrate recovery (DRaaS and cloud-native DR): AWS Elastic Disaster Recovery, Azure Site Recovery
• Layer 4, plan and govern (BC management software): Fusion Risk Management, Archer, Riskonnect

How do high availability, disaster recovery, and business continuity differ?

Three terms get used interchangeably but mean different things. High availability keeps a service running through component or site failures using redundancy and automatic failover, and its target is expressed in nines of uptime. Disaster recovery restores a service after a failure that exceeds the HA design, such as a full data center loss or a ransomware attack. Business continuity is the broadest discipline, covering people, process, and technology so the organization can sustain critical operations under any disruption.

Two metrics drive every design decision. Recovery Time Objective (RTO) defines how quickly a service must be back. Recovery Point Objective (RPO) defines how much data loss is tolerable. Teams set both per service from a business impact analysis. Mission-critical services commonly target near-zero values for both. Less critical workloads often accept an RTO and RPO of hours, which allows scheduled backups instead of continuous replication.

Replication without point-in-time history does not protect against corruption or ransomware. Corrupted or encrypted data replicates to the secondary site with the next replication cycle, within seconds when replication is continuous, so both copies hold the damage. Immutable backups stored in a separate account or air-gapped environment remain necessary however strong the HA layer is.

What are the four layers of hybrid business continuity?

Layer 1: Keep services running. Application availability tools run workloads across multiple sites, either active-active or as clusters with automatic failover, so losing one location does not take the service down. Active-active designs can shift traffic in seconds. Cluster failover typically takes seconds to minutes.

Layer 2: Recover data. Backup, replication, and immutability tools capture point-in-time copies of data and replicate state between sites. They protect against data loss, corruption, and ransomware. RPO depends on the method: zero with synchronous replication, seconds with continuous data protection, and hours to a day with scheduled snapshots.

Layer 3: Orchestrate recovery. Disaster Recovery as a Service (DRaaS) and cloud-native DR products automate the failover of servers and VMs to a secondary environment. They handle network remapping, boot order, and test drills. They matter most when whole environments must come up in a specific order.

Layer 4: Plan and govern. Business continuity management software documents plans, tracks risks and business impact analyses, manages compliance, coordinates communication during incidents, and schedules and tracks readiness tests.

Layer 1: Keep services running (application availability)

Application availability tools keep a service running when a node, site, or region fails. They differ in whether sites run active-active or fail over as active-passive clusters, and in how they route traffic.

Control Plane

What it is. Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes (MK8s), and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. A workload deployed to Control Plane’s Global Virtual Cloud (GVC), a named set of locations across one or more clouds, runs active-active in every location of that GVC from one definition. Each workload’s global endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes, and moves traffic automatically when a location fails.

Best for. Production services that must stay up through a site or region loss, with traffic rerouted automatically and Day 2 operations handled by the platform.

Strengths. Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security, with no Kubernetes cluster to operate in Control Plane-managed regions. Locations can be Control Plane-managed regions on AWS, GCP, Azure, and Oracle Cloud, Kubernetes clusters you already run, joined as locations through Control Plane’s Bring Your Own Kubernetes (BYOK), or your own servers, including in an on premises data center, through MK8s. One GVC can mix an on premises location with cloud regions. Control Plane’s Cloud Wormhole connects workloads to TCP endpoints in private networks, including on premises data centers, through an agent inside the network that holds an outbound connection, so no inbound ports open to the internet. The Template Catalog, Control Plane’s library of ready-to-install apps, includes multi-location databases such as CockroachDB and pgEdge so the data layer can also run across locations. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations, and those locations can be on different providers. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.

During the October 20, 2025 AWS us-east-1 outage, Control Plane customers with us-east-1 in their GVC had workloads active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane’s internal incident telemetry.

Scope. Control Plane keeps services running and routes traffic, and its volume sets take scheduled snapshots for point-in-time restore. Immutable, isolated backups and BC planning come from Layers 2 and 4.

On premises and cloud coverage. On premises through MK8s or BYOK. AWS, GCP, and Azure through Control Plane-managed regions, and other providers such as Hetzner through MK8s or BYOK.

Red Hat OpenShift with Advanced Cluster Management

What it is. An enterprise Kubernetes platform paired with a management layer that deploys, governs, and monitors workloads across many clusters in the data center, the cloud, and the edge.

Best for. Organizations already standardized on OpenShift that need multi-cluster governance and policy-based placement.

Strengths. Deep integration with Red Hat’s ecosystem. Policy-driven workload placement across clusters. Multisite, multicluster disaster recovery capabilities. Supports on premises, bare metal, and major cloud providers.

Limitations. Global traffic routing across clusters requires external load balancing or DNS configuration. Operational work grows with cluster count.

On premises and cloud coverage. On premises bare metal and virtualized, plus AWS, GCP, and Azure.

InfoScale

What it is. A clustering and replication platform, now part of Cloud Software Group, that provides application-level HA and DR for physical, virtual, and cloud workloads.

Best for. Enterprise applications on physical or virtual infrastructure that need automated failover between on premises sites or to the cloud.

Strengths. Supports a wide range of operating systems and applications. Provides volume replication and cluster management in one product, with support for Kubernetes and OpenShift Virtualization.

Limitations. Strongest with traditional enterprise applications. It does not route global traffic across sites.

On premises and cloud coverage. On premises physical and virtual. Cloud support for AWS, Azure, Google Cloud, and Oracle Cloud.

Linux Pacemaker clustering (open source)

What it is. An open source high availability cluster resource manager, commonly used with Corosync for Linux workloads.

Best for. Teams with strong Linux expertise that need HA for specific services such as databases or file systems without commercial licensing costs.

Strengths. Free and well documented. Long proven in production. Supported by major Linux distributions.

Limitations. Requires significant manual configuration. No global traffic routing across sites. Large clusters add operational complexity.

On premises and cloud coverage. On premises Linux servers, plus cloud VMs on AWS, Azure, and Google Cloud through cloud-specific fence and resource agents.

Layer 2: Recover data (backup, replication, immutability)

Data protection tools set the RPO. They differ in workload coverage, replication method (scheduled snapshots or continuous journaling), and ransomware features. They form the core of most cloud disaster recovery solutions.

Veeam Data Platform

What it is. A unified backup, recovery, and data security platform for virtual, physical, cloud, and SaaS workloads.

Best for. Broad backup coverage across heterogeneous environments with strong ransomware protection.

Strengths. Supports VMware, Hyper-V, Nutanix, AWS, Azure, Google Cloud, and physical servers. Immutable backup repositories. Instant VM recovery. Monitoring and reporting.

Limitations. Licensing can be complex for large, diverse environments. Recovery orchestration is included in the Premium edition and sold as an add-on for Foundation and Advanced.

On premises and cloud coverage. On premises hypervisors and physical servers. Native cloud backup for AWS, Azure, and Google Cloud.

HPE Zerto

What it is. A continuous data protection and disaster recovery platform built for near-zero RPO replication of virtualized and cloud workloads.

Best for. Near-zero RPO for VMware vSphere workloads replicating between on premises sites or to the cloud.

Strengths. Journal-based continuous data protection, with restore points seconds apart, allows recovery to a point just before corruption. Automated failover orchestration with non-disruptive testing. Strong VMware integration.

Limitations. Focused on virtualized workloads. Kubernetes protection requires a complementary tool.

On premises and cloud coverage. On premises VMware vSphere. Cloud replication to AWS, Azure, Google Cloud, IBM Cloud, and Oracle Cloud.

Commvault Cloud

What it is. An enterprise data management platform for backup, recovery, and data governance across on premises, cloud, and SaaS environments.

Best for. Large enterprises that want one platform for data protection, compliance, and cyber resilience across diverse workloads.

Strengths. Broad workload coverage including databases, VMs, containers, and SaaS applications. Built-in anomaly detection, cyber deception through ThreatWise, and on-demand cleanrooms for recovery testing.

Limitations. Initial deployment and configuration can be complex. Smaller organizations may find the feature set more than they need.

On premises and cloud coverage. On premises physical and virtual. AWS, Azure, and Google Cloud.

Rubrik Security Cloud

What it is. A data security platform that combines backup, ransomware recovery, and data observability.

Best for. Ransomware recovery and data security posture management across hybrid environments.

Strengths. Immutable backups. Threat monitoring and anomaly detection in the backup workflow. Clean recovery after ransomware. A SaaS management plane reduces infrastructure to run.

Limitations. Focused on data protection. Application-level HA and traffic routing are outside its scope.

On premises and cloud coverage. On premises data centers and major public clouds.

Cohesity

What it is. A data security and management platform that consolidates backup, recovery, file services, and analytics, and that took over Veritas’s data protection business, including NetBackup, in December 2024.

Best for. Consolidating secondary storage and adding ransomware resilience to hybrid environments.

Strengths. Reduces data fragmentation by unifying backup targets, file shares, and test and dev copies. Threat detection and anomaly scanning. Isolated recovery options.

Limitations. Primarily a data platform. DR orchestration and application availability require separate solutions.

On premises and cloud coverage. On premises appliances and software-defined deployments. Major public clouds.

Veeam Kasten for Kubernetes

What it is. Kubernetes-native data protection for backup, DR, and mobility of containerized applications, formerly named Kasten K10.

Best for. Kubernetes data protection across on premises and cloud clusters.

Strengths. Application-aware snapshots and backups for Kubernetes workloads. Policy-based automation. Supports major Kubernetes distributions and cloud-managed Kubernetes services. Integrates with Veeam’s broader platform.

Limitations. Scope is Kubernetes environments, including KubeVirt and OpenShift Virtualization VMs. Workloads outside Kubernetes need a separate backup product such as Veeam Data Platform.

On premises and cloud coverage. Any supported Kubernetes cluster, on premises or cloud.

Google Cloud Backup and DR Service

What it is. A Google Cloud managed service for centralized backup and recovery of workloads running in Google Cloud and hybrid environments.

Best for. Organizations running workloads on Google Cloud, including Compute Engine, Google Cloud VMware Engine, Cloud SQL, AlloyDB, and self-managed databases.

Strengths. Integrated with the Google Cloud console. Incremental-forever capture with instant volume mounting. Application-consistent protection for databases and VMs.

Limitations. Backup and restore within Google Cloud; it does not orchestrate failover of a whole environment.

On premises and cloud coverage. Google Cloud workloads, with limited hybrid support.

Layer 3: Orchestrate recovery (DRaaS and cloud-native DR)

Recovery orchestration tools restart replicated servers in a secondary environment in a defined order. AWS Elastic Disaster Recovery recovers only into AWS, and Azure Site Recovery recovers into Azure or a secondary on premises site, which is the main trade-off in cloud vs. on premises disaster recovery.

AWS Elastic Disaster Recovery

What it is. A cloud-native DR service that replicates on premises or cloud servers into AWS for automated failover and recovery.

Best for. Organizations replicating on premises workloads to AWS as a DR target with minimal standby infrastructure cost.

Strengths. Continuous block-level replication with RPOs of seconds and RTOs of minutes. A low-cost staging area holds replicated data until you launch recovery instances. Automated launch and recovery into AWS.

Limitations. Recovery target is AWS only. Multi-cloud DR requires additional tooling.

On premises and cloud coverage. Source: on premises physical servers, VMware vSphere, Microsoft Hyper-V, and other cloud providers. Target: AWS regions.

Azure Site Recovery

What it is. Microsoft’s DR orchestration service for replicating and failing over VMs and physical servers.

Best for. Organizations with Azure investments that need DR for VMware, Hyper-V, or physical servers.

Strengths. Native integration with Azure services. Replication from on premises to Azure and between Azure regions. Recovery plans with customizable sequencing.

Limitations. Recovery target is Azure, or a secondary on premises site for VMware, VMM-managed Hyper-V, and physical servers. Multi-cloud DR requires additional tooling.

On premises and cloud coverage. Source: on premises VMware, Hyper-V, physical Windows and Linux servers, Azure VMs, and AWS Windows instances. Target: Azure regions or a secondary site.

Layer 4: Plan and govern (BC management software)

BC management software centralizes continuity plans, tracks business impact analyses, manages compliance obligations, and coordinates communication during incidents.

Fusion Risk Management provides a cloud platform for business continuity, IT disaster recovery, crisis management, operational resilience, and third-party risk.

Archer offers an integrated risk management platform with operational resilience modules for impact analysis, plan management, and incident response, deployable on premises or as SaaS.

Riskonnect delivers risk management and business continuity planning tools focused on operational resilience, third-party risk, and regulatory compliance.

How do the options compare?

What does a reference architecture for HA across on premises and cloud look like?

A hybrid continuity design uses all four layers. Steps 1 to 3 follow the active-active high availability across regions and clouds pattern on top of a hybrid cloud deployment.

1. Run the service active-active in the data center and at least one cloud region. Deploy workloads to a Control Plane GVC that includes an on premises location (through MK8s or BYOK) and one or more cloud regions. The application runs in every location of the GVC at once.
2. Route users to healthy sites. On Control Plane, each workload’s global endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes, so users stop reaching a location that fails.
3. Replicate data with a database built for multiple sites. Deploy a multi-location database from the Template Catalog so the data layer spans the same locations. CockroachDB replicates by consensus, and surviving a region loss needs three locations. pgEdge runs multi-master across locations with asynchronous replication and last-update-wins conflict resolution, which keeps writes local and fast, though writes not yet replicated when a location fails may need reconciliation. Synchronous replication between an on premises site and a distant cloud region adds a round trip to every write, so pick based on how strict consistency must be versus write latency.
4. Keep immutable backups in a separate account. Use a Layer 2 tool such as Veeam, Rubrik, or Cohesity to keep immutable backups in a cloud account or air-gapped environment isolated from production credentials. Use object lock or WORM storage, separate credentials and MFA for the backup account, and a retention period longer than an attacker’s likely dwell time. Restore from these backups during tests. This protects against ransomware and data corruption that replication would spread.
5. Orchestrate and test failover. For workloads outside the active-active footprint, such as legacy VMs, use AWS Elastic Disaster Recovery or Azure Site Recovery to automate failover. Many teams run non-disruptive DR tests quarterly and after major infrastructure changes; regulated industries may have their own required cadence.
6. Document the plan. Keep runbooks, communication trees, and compliance records in a BC management platform, and update them after every test and every significant infrastructure change.

How do you choose a tool for each gap?

Each row covers one gap, so most hybrid estates combine one tool per layer. Provider-level failover tools are compared in best multi-cloud failover solutions for 2026.

FAQ

Answers to common questions about hybrid business continuity.

What are the best business continuity solutions for scaling high availability services on premises and cloud?

Control Plane for application availability. Veeam, HPE Zerto, Commvault, Rubrik, Cohesity, or Veeam Kasten for data protection. AWS Elastic Disaster Recovery or Azure Site Recovery for recovery orchestration. Fusion, Archer, or Riskonnect for BC planning. No single product covers all four layers.

What are the best cloud operations solutions for addressing business continuity and disaster recovery?

Pick by gap. Control Plane covers uptime across clouds and data centers. Backup platforms such as Veeam or Rubrik cover recoverability. AWS Elastic Disaster Recovery or Azure Site Recovery cover server failover into one cloud, and Azure Site Recovery can also fail over to a secondary on premises site. BC software such as Fusion covers governance.

What is the difference between high availability and disaster recovery?

High availability keeps a service running through component or site failures, typically with an RTO of seconds to a few minutes. Disaster recovery restores a service after events that exceed the HA design, such as a full site loss or ransomware, typically with an RTO of minutes to hours. Business continuity covers both, plus people and process.

Do I still need backups if I run active-active?

Yes. Active-active protects against infrastructure failures but not against data corruption or ransomware, so immutable backups in a separate account or air-gapped environment remain necessary.

How do I fail over between an on premises data center and the cloud?

At the application layer, run workloads in a Control Plane Global Virtual Cloud (GVC) that includes on premises and cloud locations, so traffic shifts automatically. For VM-level failover, replicate to a cloud DR target, trigger the DR plan when needed, and plan failback to the data center.

What RTO and RPO should critical services target?

Set RTO and RPO per service from a business impact analysis. Mission-critical customer-facing services commonly target near-zero RTO with an active-active application layer, and an RPO of zero to seconds with synchronous replication or continuous data protection.

Is DRaaS enough for business continuity?

No. DRaaS handles recovery orchestration but does not replace an HA layer, immutable backups, or BC governance, so it works alongside the other three layers.

How does Control Plane support business continuity on premises and in the cloud?

Control Plane runs each workload active-active across every location in its Global Virtual Cloud (GVC), which can mix on premises data centers, joined through Control Plane’s Managed Kubernetes (MK8s) or Bring Your Own Kubernetes (BYOK), with cloud regions on AWS, GCP, Azure, and Oracle Cloud. Each workload’s global endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes. Control Plane’s Cloud Wormhole connects workloads to TCP endpoints in private networks without opening inbound ports. Control Plane also handles Day 2 operations such as autoscaling, observability, and security. The 99.999% SLA applies to workloads running at least two replicas in at least two locations.

Can I use Control Plane with my existing Kubernetes clusters?

Yes. Control Plane’s Bring Your Own Kubernetes (BYOK) installs a few components into an existing cluster, which then registers as a Control Plane location. Add that location to a Global Virtual Cloud (GVC), and workloads run active-active across it alongside Control Plane-managed regions or Managed Kubernetes (MK8s) locations.

How often should I test my business continuity plan?

Quarterly is a common cadence for non-disruptive DR tests, with extra tests after significant infrastructure changes. Validate failover and restore separately, and update runbooks after each test.

To add the application availability layer to your business continuity stack, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

The Kubernetes outcomes are the same across deployment models. What changes is who does the operational work.

• Benefits teams want: declarative deployments and rollbacks, autoscaling, self-healing, service discovery and load balancing, bin packing, portability across clouds, and a standard API with a large ecosystem.
• What automated managed tiers still leave to you: testing workloads against each Kubernetes version, third-party add-ons such as cert-manager and external-dns, pod-level autoscaling, secrets and identity bindings, tracing and alerting, multi-region failover, resource requests that drive cost, and on-call for workload configuration and add-ons.
• When serverless containers fit: stateless services on one cloud that need no custom controllers, CRDs, or Helm-based third-party software. Google Cloud Run, Amazon ECS with AWS Fargate, Azure Container Apps, and Control Plane serverless workloads all fit here.
• How Control Plane fits: Control Plane runs serverless, standard, stateful, and cron workloads on AWS, GCP, Azure, and Oracle Cloud, in one region or active-active across regions and clouds, and handles the Day 2 operations a platform team would own.

What are the benefits of Kubernetes?

Teams commonly adopt Kubernetes for these seven outcomes.

• Declarative deployments and rollbacks. You describe the desired state, and the system converges to it. Rolling back a Deployment is a single command.
• Autoscaling. The built-in Horizontal Pod Autoscaler adjusts replica counts on metrics you choose, and the Vertical Pod Autoscaler add-on adjusts resource requests.
• Self-healing. Failed containers restart automatically, and unhealthy pods are replaced.
• Service discovery and load balancing. Each Service gets a DNS name, ClusterIP Services get a stable virtual IP, and traffic is spread across healthy pods.
• Bin packing. The scheduler places pods to use CPU and memory across nodes efficiently.
• Portability across clouds. The same manifests run on any conformant cluster, in AWS, GCP, Azure, or on premises, with changes limited to cloud-specific pieces such as storage classes and load balancer settings.
• A large ecosystem and standard API. Operators, tools, and integrations speak the Kubernetes API, which helps with hiring and documentation.

Who does the work?

Responsibility for each operational task, by deployment model:

Each tier moves more rows from you to the provider. Even the most automated managed tiers leave workload-level operations, multi-region failover, and cost tuning with you. Production-grade infrastructure without a platform team lists the full set of production requirements.

What do GKE Autopilot, EKS Auto Mode, and AKS Automatic handle?

These are the most automated managed Kubernetes tiers from Google Cloud, AWS, and Azure.

What they handle:

• Node provisioning, patching, OS updates, and scaling
• Core cluster components such as pod networking, cluster DNS, load balancer integration, and storage drivers
• Version upgrades on GKE Autopilot and AKS Automatic, and node replacement after you start an upgrade on EKS Auto Mode
• Managed logs and Prometheus metrics by default on GKE Autopilot and AKS Automatic

What still stays with you:

• Version compatibility. You test workloads, deprecated APIs, and third-party add-ons against each new Kubernetes version.
• Ingress and DNS beyond the defaults. You configure routes, TLS certificates, and DNS records, and on GKE Autopilot and EKS Auto Mode you install external-dns if you want automatic DNS records.
• Third-party add-ons. Helm charts and operators you install are yours to upgrade.
• Pod autoscaling. You configure the Horizontal Pod Autoscaler and Vertical Pod Autoscaler for each workload.
• Secrets and cloud access. You manage secrets and bind workload identities to cloud IAM roles.
• Tracing and alerting. You set up tracing and alerts, and on EKS Auto Mode, monitoring as a whole.
• Multi-region failover. Each cluster sits in one region. Multi-region takes several clusters, a global load balancer, and a data replication plan.
• Cost. You set resource requests on every pod. Overprovisioning raises cost, and underprovisioning causes throttling.
• On-call. The provider covers nodes. You are on call for workload configuration and add-ons.

A small team can run a single-region production workload on an automated managed tier. The other items above still add work that a platform team would otherwise own, even in one region, and multi-region availability adds more.

Kubernetes or serverless containers for a small team?

Small teams with stateless services on one cloud should evaluate serverless containers before committing to Kubernetes.

Google Cloud Run runs containers with scaling to zero, built-in TLS, and request-based billing by default, with instance-based billing as an option. Amazon ECS with AWS Fargate runs containers without EC2 instances to manage, with Elastic Load Balancing, IAM, and CloudWatch integration. Azure Container Apps offers event-driven scaling, managed Dapr APIs, and managed ingress. Each gives revision-based or rolling deployments, health checks, autoscaling, and load balancing with no cluster to operate. Control Plane covers the same ground with serverless workloads and runs one workload definition active-active across regions and across AWS, GCP, Azure, and Oracle Cloud, with automatic failover between locations.

When you need the Kubernetes API:

• You depend on operators or custom resource definitions.
• You deploy third-party software distributed as Helm charts that needs Kubernetes.
• You need scheduling controls such as pod affinity, tolerations, or priority classes.
• You want portability through the standard Kubernetes API.

If any of these apply, you need the Kubernetes API. Control Plane’s Managed Kubernetes (MK8s) provides it, and Bring Your Own Kubernetes (BYOK) connects clusters you already run. Kubernetes alternatives for different team profiles compares the options.

A minimal stack for a team without platform engineers

A team without platform engineers can run production containers on a stack that removes the tasks a platform team would handle.

1. Pick a managed runtime. Choose a platform such as Control Plane that handles infrastructure, scaling, and availability, an automated managed Kubernetes tier, or a serverless container service.
2. Deploy with GitOps or CI pipelines. Argo CD or Flux reconciles desired state from Git, or your CI system pushes deployments. Either gives you auditability and rollback. Argo CD vs Flux for managing environments and clusters compares the two GitOps tools.
3. Use managed databases. Amazon RDS, Cloud SQL, or Azure Database for PostgreSQL keep backups and failover off your plate instead of running databases on a cluster yourself.
4. Use managed observability. Datadog, Grafana Cloud, or your cloud provider’s monitoring give you logs, metrics, and alerting without running Prometheus, Loki, or Jaeger yourself.
5. Centralize secrets. Use AWS Secrets Manager, Google Secret Manager, Azure Key Vault, or your platform’s built-in secrets, and keep credentials out of environment files and ConfigMaps.
6. Assign one person to own upgrades on a calendar. A monthly or quarterly review of version upgrades, add-on updates, and dependency patches covers routine maintenance, with critical security patches applied when they are released. Any engineer with a checklist and a maintenance window can own this.

Platform engineering without a platform team covers this stack in practice.

How Control Plane fits

Control Plane runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, Azure, and Oracle Cloud Infrastructure (OCI), VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. On Control Plane-provided compute, there is no Kubernetes cluster you have to operate.

You deploy a workload, a container image with CPU, memory, and ports, to Control Plane’s Global Virtual Cloud (GVC), a named set of locations across one or more clouds. The workload runs in every location of that GVC from one definition.

Day 2 operations. Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security. Autoscaling works on requests per second, CPU, or memory for serverless, standard, and stateful workloads, on concurrency for serverless workloads, and on latency or KEDA event sources for standard and stateful workloads. Capacity AI, Control Plane’s right-sizing feature, adjusts a workload’s reserved CPU and memory between set minimums and maximums based on historical usage, where enabled. Capacity AI is not available with CPU-based or multi-metric autoscaling, or for GPU workloads.

Networking and security. Internal traffic between workloads uses mTLS with a unique client certificate per workload, rotated hourly, and a new workload has inbound, outbound, and internal traffic closed until you allow it. Each workload’s built-in global endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes, and traffic moves automatically when a location fails. TLS certificates for your domains are created and renewed automatically.

Secrets and cloud identity. Built-in secrets are available to every workload. Universal Cloud Identity gives workloads access to 600+ AWS, GCP, and Azure services through an IAM role or service-account grant instead of long-lived keys in your code, with short-lived, least-privilege credentials at runtime.

Observability. Built-in logs and metrics run in every location. Tracing is built in and turned on per GVC or org.

When you need the Kubernetes API. Control Plane’s Managed Kubernetes (MK8s) creates clusters as Control Plane locations, with Control Plane operating the Kubernetes control plane while worker nodes run in your AWS account, your GCP project, another supported provider’s account, or on servers you provide. Control Plane’s Bring Your Own Kubernetes (BYOK) connects clusters you already run. The Control Plane CLI command cpln helm install installs Helm charts whose templates define Control Plane resources, and the Template Catalog, Control Plane’s library of ready-to-install apps, packages common applications that way.

Availability and compliance. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be on different providers. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Pricing. You pay for the CPU and memory a workload reserves, with no contracts or minimums. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.

Best Kubernetes management platforms compares management platforms. Kubernetes production architecture covers upgrades, autoscaling, and databases.

FAQ

Teams without platform engineers usually ask about automated managed tiers, serverless containers, Helm, multi-region failover, cost, and compliance.

How do I get Kubernetes benefits without hiring a platform team?

Buy the layers a platform team would build. Choose an automated managed Kubernetes tier that runs nodes for you, or a platform such as Control Plane that delivers declarative deployments, autoscaling, self-healing, service discovery, mTLS, and multi-region failover with no cluster you have to operate. Add managed databases, managed observability, and a secrets manager, and give one engineer ownership of upgrades on a calendar.

Do I need a platform team to run Kubernetes?

No. Automated managed tiers and platforms that remove cluster operations let small teams run production workloads. You need one engineer accountable for upgrades, security patches, and cost reviews, working from a checklist.

Is GKE Autopilot or EKS Auto Mode enough without a platform team?

For a single-region deployment with modest requirements, yes. They provision, patch, and scale nodes, and GKE Autopilot also upgrades the cluster and turns on managed logs and metrics. You still own version compatibility testing, ingress routes and DNS, third-party add-ons, pod autoscaling, secrets, tracing and alerting, multi-region failover, resource requests, and workload on-call. A platform such as Control Plane takes those Day 2 operations off your team in a single region too, and adds multi-region failover when you need it.

What parts of Kubernetes still need operating on managed Kubernetes?

Even on the most automated tiers, you own third-party add-ons such as cert-manager and external-dns, workload autoscaling configuration, secrets and identity bindings, tracing and alerting, multi-region failover, resource requests that drive cost, Kubernetes version compatibility testing, and on-call for workload configuration and add-ons.

Should a small team use Kubernetes or serverless containers?

Serverless containers, if you run stateless services on one cloud and need no operators, CRDs, or Helm-based third-party software. Cloud Run, ECS with Fargate, Azure Container Apps, and Control Plane serverless workloads are simpler to operate than a cluster. Choose Kubernetes when you need its API, scheduling controls, or ecosystem.

Can I use Helm charts without operating a cluster?

Yes, for charts built from Control Plane resources. The Control Plane CLI command cpln helm install deploys charts whose templates define Control Plane workloads, GVCs, identities, and volume sets, with no cluster for you to run, and the Template Catalog packages common applications that way. Standard Kubernetes charts need the Kubernetes API. For software that needs a full Kubernetes cluster, Control Plane’s Managed Kubernetes (MK8s) gives you the Kubernetes API with Control Plane operating the Kubernetes control plane, and Bring Your Own Kubernetes (BYOK) connects a cluster you already run so Control Plane can deploy and manage workloads on it.

How does Control Plane give Kubernetes benefits without a cluster to operate?

You deploy a workload to a Control Plane Global Virtual Cloud (GVC), and it runs in every location of that GVC from one definition on Control Plane-provided compute, with no Kubernetes cluster you have to operate. Autoscaling, self-healing, internal service discovery, mTLS, built-in observability, secrets, and global traffic routing come with the platform. For the Kubernetes API itself, MK8s and BYOK provide it.

What is the cheapest way to run containers in production without a platform team?

Run on a platform that bills for what workloads reserve and includes the layers you would otherwise pay engineers to build. On Control Plane, you pay for the CPU and memory a workload reserves, with no contracts or minimums, and Capacity AI, where enabled, adjusts reserved CPU and memory between set minimums and maximums based on historical usage. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Autoscaling, observability, secrets, and global failover are built in, in one region or many. Keep state in a managed database or a multi-location database from the Template Catalog.

How do I handle multi-region failover without a platform team?

Use a platform with built-in multi-region routing. On managed Kubernetes, failover takes multiple clusters, a global load balancer, data replication, and custom health checks. On Control Plane, each workload’s endpoint uses DNS to route clients to the nearest location where the workload passes its readiness probes, so traffic fails over automatically, and workloads with at least two replicas in at least two locations are covered by the 99.999% SLA.

What security and compliance standards matter for a team without platform engineers?

The ones your customers and regulators require, such as PCI DSS, SOC 2, and HIPAA. Control Plane’s platform is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. On managed Kubernetes, the provider secures the infrastructure, and you own RBAC, network policies, secrets, and application security.

Talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

• Versioned, reversible deploys come first. Store every release as an immutable artifact. Rollback should take minutes.
• Roll back data last, and carefully. Destructive schema migrations and data changes cannot be undone with a traffic switch. Use expand-and-contract migrations so the old code still works while the new schema is live.
• Health-gated progressive delivery limits damage. Rolling updates, canary releases, blue/green switches, and feature flags each trade cost and complexity for faster rollback and smaller blast radius.
• Self-healing handles routine failures automatically. Health checks, automatic restarts, autoscaling, circuit breakers, and automatic traffic failover handle the failures that do not need a person.
• Blast radius shrinks with failure domains. Running across multiple availability zones, regions, or providers removes the single points of failure that turn a provider incident into your outage.
• Control Plane ties these together. Readiness-gated rolling updates, rollback through its MCP server, blue/green on domain routes, a tamper-proof audit trail, and active-active across every location in a GVC.

How do I roll back a bad AI-generated change in production?

Many outages start with a change: a bad deploy, a config push, a migration, or an AI-generated edit nobody fully reviewed. These steps apply whether a human or an AI agent wrote the change.

Step 1: Detect

Tie alerts to the signals that matter: error rate, request latency, CPU or memory saturation, and failed health checks. Synthetic monitors and real-user metrics catch what log-based alerts miss. The goal is to know something is wrong within one to two minutes of the change reaching production.

Step 2: Decide whether to roll back or fix forward

If the root cause is obvious and the fix is a one-line, well-tested change, fix forward. In every other case, roll back first, then investigate.

Step 3: Roll back

The mechanism depends on the change type:

• Application image: Redeploy the last known-good image digest through the same CI/CD pipeline or GitOps reconciliation loop.
• Config or environment variable: Revert the commit in the config repo or redeploy the previous config version.
• Feature flag: Toggle the flag off. This is the fastest rollback path because it requires no deploy.
• Infrastructure as code: Revert the commit and let the reconciliation loop or pipeline apply the previous state.
• Domain route (blue/green): Point the route back to the previous workload.

On Control Plane, an AI assistant can roll a workload back to its previous image (environment and config stay as they are) in one call through Control Plane’s MCP server, which makes the change, waits for the workload to report ready, and reports the result.

Step 4: Verify

Rollback is not complete until error rates, latency, and health probes return to baseline. Check business-critical user journeys, not just infrastructure metrics.

Step 5: Follow up

Preserve logs and traces. Run a blameless postmortem. Add a test that would have caught the failure before production.

Change types and how each is undone

The hard part: destructive schema and data changes

A column drop, a table rename, or a data backfill that overwrites values cannot be undone by redeploying an image. The solution is expand-and-contract (parallel change) migrations:

1. Expand: Add the new column or table alongside the old one. Deploy code that writes to both and still reads the old one.
2. Migrate: Backfill existing data into the new structure.
3. Switch reads: Deploy code that reads the new structure.
4. Contract: Drop the old column only after rollback to the old version is no longer needed.

This pattern keeps the previous application version compatible with the current schema at every step.

Guardrails for AI-generated changes

AI coding assistants and infrastructure agents accelerate development, but they also produce changes that humans may not fully read before merging. Add these guardrails:

• Require human review for any change that touches a production path.
• Keep diffs small. A 2,000-line AI-generated PR is effectively unreviewed.
• Run the full test suite in CI before merge.
• Keep an audit trail that shows whether a person or an agent made a change, and who approved it.

On Control Plane, a tamper-proof audit trail records who changed which workload and when, including service accounts and AI agents using the MCP server (cpln workload audit), and the MCP server offers a read-only profile for agents that should observe without changing anything. For more on agent permissions, see how to let AI agents touch production infrastructure safely.

Deploy patterns that limit damage

The right pattern depends on how fast you need to roll back, your infrastructure budget, and how mature your observability is.

Rolling updates

Replace instances of the old version with the new version incrementally. If new replicas fail their readiness checks, the rollout stalls while old replicas keep serving. Kubernetes does not roll back automatically (you run kubectl rollout undo), and tools like Argo Rollouts add automated rollback. Rolling updates are also the default on Control Plane, where rollout options such as minReadySeconds, maxUnavailableReplicas, and maxSurgeReplicas control how many replicas turn over at once.

Blue/green deployments

Run two full environments. Route traffic to one while deploying to the other, cut over when the new environment is healthy, and roll back by switching the route back. On Control Plane, blue/green uses domain routes, and traffic returns to the previous workload within seconds of pointing the route back.

Canary releases

Send a small share of traffic to the new version, watch error rates and latency, and promote or roll back based on the results. Argo Rollouts and Flagger automate canary analysis in Kubernetes.

Feature flags

Decouple deployment from release. Ship code dark, then enable it for specific users or percentages, and roll back by toggling the flag. OpenFeature provides a vendor-neutral API for flag evaluation.

The hyperscalers offer managed versions of these patterns. AWS CodeDeploy supports in-place (rolling) deployments for EC2 and on-premises servers, blue/green deployments for EC2, and canary, linear, or all-at-once traffic shifting for Lambda and ECS. Google Cloud Deploy provides managed continuous delivery to GKE, Cloud Run, and GKE attached clusters, with approvals, canary deployments, and rollback. Azure Pipelines supports multi-stage pipelines with approvals and checks, and deployment strategies whose on-failure hooks can run rollback steps.

Which CI/CD and IaC solutions offer the best disaster recovery and self-healing patterns for cloud native applications?

The patterns matter more than the vendor. Together, they recover from failed deploys, infrastructure drift, and the loss of an environment.

GitOps reconciliation

Argo CD and Flux continuously compare a cluster with the desired state declared in Git. When someone edits a resource by hand or a deploy goes wrong, reconciliation restores the declared state (Flux does this by default; Argo CD requires automated sync with self-heal enabled). Rolling back means reverting a commit.

Immutable images and infrastructure as code

Reference every release by its content-addressable digest, not a mutable tag like latest, so a rollback redeploys exactly the same binary. Terraform, OpenTofu, and Pulumi support remote state backends, most with state locking, to prevent concurrent changes. Back up state files, because without valid state these tools cannot tell what exists. Control Plane ships its own Terraform and Pulumi providers, so workload definitions can go through the same review process as the rest of your infrastructure, and workloads can be promoted from a staging GVC to a production GVC.

GitOps restores compute and configuration. It does not restore databases, third-party API state, or DNS records managed outside the reconciliation loop. Those need their own backup and recovery plan.

Self-healing infrastructure

Self-healing means the system detects an unhealthy condition and takes a bounded corrective action without waiting for a human. Each mechanism handles a specific failure mode.

Health checks

• Liveness probes answer “is this process stuck?” If a liveness check fails repeatedly, the kubelet or platform restarts the container.
• Readiness probes answer “can this replica handle traffic?” A replica that fails readiness stops receiving traffic but is not restarted.
• Startup probes give slow-starting applications time to initialize before liveness and readiness checks begin.

On Control Plane, readiness and liveness probes can be HTTP, gRPC, TCP, or a command, and they decide when a replica takes traffic and when it is restarted.

Restarts, rescheduling, and autoscaling

When a container crashes or a node fails, the orchestrator restarts the container or reschedules it onto a healthy node. Horizontal autoscaling adds replicas as demand rises, and right-sizing adjusts CPU and memory to actual usage. On Control Plane, built-in autoscaling and Capacity AI, Control Plane’s right-sizing of CPU and memory based on historical usage, handle both.

Circuit breakers, timeouts, and retries with backoff

When a downstream dependency fails, circuit breakers stop sending it traffic, which prevents cascading failures. Timeouts bound how long a request waits. Retries with exponential backoff and jitter recover from transient errors without overwhelming the failing service. Retry only idempotent operations, or use idempotency keys, to avoid duplicate side effects.

Automatic traffic failover between locations

Restarting a container fixes a process-level failure on one node. Failing traffic over to another location fixes a zone-level or region-level failure, and that requires workloads already running in more than one location plus a global load balancer that detects location health. On Control Plane, a workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes clients to the nearest healthy location. Priority routing tiers (a lower-priority tier receives traffic only when every location in the higher-priority tier is unavailable) and latency offsets let you shape where traffic goes.

How do I reduce the blast radius of a cloud provider outage?

Blast radius applies to your own changes and to your provider’s failures.

Staged rollouts by location

Deploy to one location first, watch it, then promote to the rest. A bad change then reaches only a fraction of your traffic.

Cell-based architecture

Partition users or tenants into independent cells, each with its own compute, data, and dependencies, so a failure in one cell does not reach the others.

Availability zones, regions, and providers

• Multiple availability zones protect against single-datacenter failures. Many regional managed services, such as S3 and DynamoDB, replicate across zones automatically, but EC2 instances, EBS volumes, and single-AZ RDS databases are zonal unless you configure Multi-AZ.
• Multiple regions protect against region-wide outages, which are rarer but have much higher impact.
• Multiple providers protect against provider-wide failures and reduce dependence on any single vendor.

Each step adds complexity and cost; see active-active vs active-passive cost for the numbers.

Hidden single-region dependencies

Your compute may be multi-region, but if your DNS provider, identity provider, CI/CD system, secrets manager, container registry, or a provider management API is pinned to one region, that region is still a single point of failure. Audit every dependency in the request path.

Rate limiting, load shedding, and kill switches

When a failure reduces capacity, rate limiting and load shedding protect the remaining healthy capacity; shed low-priority traffic first. A kill switch is a pre-built way to disable a feature, a dependency call, or a traffic path instantly, and it should be tested regularly.

For the region-failure playbook, see how to keep your app online when an AWS region goes down. Control Plane’s cloud outage tracker records how often these events happen.

What is the best cloud architecture model that delivers the highest level of resilience and availability?

The answer depends on how much downtime and data loss your business can tolerate, so start from your recovery targets.

How do I design cloud resilience and failover to meet a specific RTO and RPO?

Recovery time objective (RTO) is how fast you need to be back. Recovery point objective (RPO) is how much data you can afford to lose. Your data replication strategy sets RPO; compute architecture does not.

Near-zero RPO requires synchronous or near-synchronous data replication. Active-active gives the lowest RTO because failover takes only as long as health checks need to mark a location unhealthy and reroute traffic, typically seconds, with no standby to start or scale up. A multi-region compute layer paired with a single-region database does not give you multi-region resilience, so databases need their own replication and rollback design.

How Control Plane approaches change safety and recovery

Control Plane is the platform for running unbreakable apps and agents. It runs serverless, standard, stateful, and cron workloads on AWS, GCP, and Azure, on Kubernetes clusters you already run, and on your own machines turned into managed locations, including on premises. Many teams run it in one cloud as well.

Every workload update is a rolling update. The previous version keeps serving while the new one starts, and the update completes only when the new version reports ready. If the new version never becomes ready, the rollout stops and the previous version keeps handling traffic until you fix the workload or roll it back.

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers.

During the October 20, 2025 AWS us-east-1 outage, which lasted about 15 hours, Control Plane customers with us-east-1 in their GVC had no downtime, and their workloads were “active and running in healthy locations within 10 seconds,” according to Control Plane internal incident telemetry.

Control Plane handles Day 2 operations for you, including autoscaling, Capacity AI right-sizing, built-in observability, and security, with no Kubernetes cluster you have to operate. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

For the broader checklist, see how to prevent website downtime.

FAQ

These answers cover the questions teams ask most about rollback, self-healing, and blast radius.

How do I roll back a bad deployment quickly?

Redeploy the last known-good image or config through the same pipeline that shipped the broken version. That requires every release to be an immutable, versioned artifact and a pipeline that can deploy a specific version, not just the latest.

What is the difference between rolling back and fixing forward?

Rolling back restores the previous version. Fixing forward ships a new version that corrects the problem. Roll back when the root cause is unclear or the fix is not trivial, and fix forward only when the fix is small, tested, and faster than a rollback.

How do I roll back a database migration?

Use expand-and-contract migrations. Add the new columns or tables alongside the old ones, deploy code that writes to both and reads the old one, backfill, switch reads to the new structure, then drop the old structure once rollback is no longer needed. Destructive changes like column drops or renames cannot be reversed by redeploying application code.

Which deploy pattern has the fastest rollback?

Feature flags are the fastest because toggling a flag is instant and needs no deploy. Blue/green and canary rollbacks take seconds because only traffic moves. Rolling updates take longer because replicas must be replaced.

What is self-healing infrastructure?

Self-healing infrastructure detects unhealthy conditions and takes bounded corrective actions: restarting a failed container, removing an unhealthy replica from traffic, scaling up to meet demand, or failing traffic over to a healthy location. It does not replace human judgment for complex or ambiguous failures.

How do I reduce the blast radius of a cloud provider outage?

Run workloads across more than one availability zone, region, or provider, and remove hidden single-region dependencies like DNS, identity, CI/CD, secrets, and container registries. On Control Plane, workloads run active-active across every location in a GVC, and traffic fails over to healthy locations automatically.

What RTO and RPO can I achieve with multi-region active-active?

Multi-region active-active delivers an RTO measured in seconds. RPO near zero requires synchronous replication, which adds cross-region write latency; asynchronous replication gives an RPO equal to the replication lag, usually seconds.

How do I make AI-generated changes safer in production?

Require human review on production paths, keep diffs small, run tests before merge, deploy in health-gated stages, and keep an audit trail of what each agent changed. On Control Plane, agents using the MCP server act through the same API as everyone else, so every change lands in the audit trail, and a read-only profile is available for agents that should only observe.

To run workloads active-active with readiness-gated rollouts, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

This is general guidance, not legal or audit advice.

TL;DR

• What platform compliance covers. SOC 2 Type II, HIPAA eligibility, and PCI DSS attestations apply to the platform’s services, not your application logic or data handling.
• How to verify a platform ’s claims. Request the SOC 2 Type II report under NDA, the PCI DSS Attestation of Compliance, and a signed BAA. Check which services and regions are in scope.
• Data residency with failover. Choose two or more locations inside the same country, run active-active or active-passive between them, and pin replicas, backups, logs, and keys to those locations.
• The compliance checklist. Classify data, enforce region allow-lists with policy as code, manage secrets outside your codebase, isolate tenants, log everything immutably, and document AI data flows.
• How Control Plane fits. On Control Plane, a workload runs only in the locations of Control Plane’s Global Virtual Cloud (GVC), a set of locations a workload runs across, so you control data residency by choosing locations. Workloads deploy active-active across every location in a GVC, and a global endpoint routes clients to the nearest healthy location.

Which application platforms are SOC 2 Type II, HIPAA, and PCI DSS compliant?

AWS, Google Cloud, and Azure each hold SOC 2 Type II reports and PCI DSS Level 1 service provider attestations, and each signs a BAA covering a published list of HIPAA-eligible services. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. Its SOC 2 Type II audit was completed by Prescient Assurance, and its SOC 2 Type II report and PCI DSS Attestation of Compliance are available from support@controlplane.com, and its docs publish a PCI Responsibility Matrix.

What each framework means for a platform buyer

SOC 2 Type II is an attestation report from an independent CPA firm. The auditor tests whether a platform’s controls operated effectively over a review period, commonly three to twelve months. A Type I report covers only control design at a single point in time. Platforms renew the report each year, so check the period end date and ask for a bridge letter if it ended months ago. A badge on a marketing page shows none of the scope or exceptions, so ask for the report itself.

HIPAA has no official certification program. The U.S. Department of Health and Human Services does not certify software or cloud platforms as “HIPAA compliant.” Instead, look for two things: a signed Business Associate Agreement (BAA), which sets out how the platform must protect electronic protected health information (ePHI), and a clear list of HIPAA-eligible services. Under the HITECH Act, a platform that handles ePHI as a business associate is also directly liable for complying with the HIPAA Security Rule. Any service not on that list should not touch ePHI.

PCI DSS compliance is documented in an Attestation of Compliance (AoC), the official PCI Security Standards Council form summarizing the results of an assessment. Platforms that process, store, or transmit cardholder data are assessed as service providers. Card brands such as Visa and Mastercard classify a service provider as Level 1 when it handles more than 300,000 card transactions a year, and Level 1 requires an annual assessment by a Qualified Security Assessor (QSA) and a Report on Compliance. The AoC states which services and environments are in scope.

How to verify a platform’s compliance

1. Request the SOC 2 Type II report under NDA. Read the auditor’s opinion, the description of the system boundary, and any exceptions noted.
2. Request the PCI DSS Attestation of Compliance. Confirm the service provider level and check which services and data center regions are covered.
3. Request a signed BAA before any ePHI touches the platform. Confirm the list of HIPAA-eligible services.
4. Check scope : which regions, which services, and which deployment models are included. A platform may be compliant in some regions but not others.

AWS, Google Cloud, and Azure each publish compliance programs with lists of in-scope services. AWS maintains its compliance programs page, Google Cloud publishes its compliance offerings, and Azure documents its compliance offerings.

Who owns which compliance controls, the platform or the customer?

The table below maps common control areas to who typically owns them.

A platform’s audit reports cover the rows marked Platform and the platform’s part of the Shared rows. Rows marked Customer or Shared need your own controls and evidence.

How do I keep customer data in a specific country while still having multi-region failover?

Run the application in two or more locations inside the country, replicate data only between those locations, and keep every replica, backup, log, and key in-country. Traffic fails over between the in-country locations, never to a region abroad.

Architecture options:

• Active-active between two in-country locations removes failover delay, because each location already serves traffic, and a global load balancer routes requests to the nearest healthy one.
• Active-passive keeps a standby location in-country that takes over during an outage. It costs less but introduces failover time.

Databases in active-active. The application tier can run active-active, but most relational databases accept writes on one primary. Put the primary in one in-country location and a replica in the other, and send writes from both locations to the primary. Synchronous replication gives a recovery point objective (RPO) of zero, meaning no committed write is lost, at the cost of added write latency. In-country locations are usually close enough to keep that cost low. Asynchronous replication adds no write latency but can lose writes not yet copied when the primary fails. On failure, promote the replica and repoint the application. Use a quorum or fencing so two primaries never accept writes at once. To accept writes in every location, use a distributed SQL database such as CockroachDB, YugabyteDB, or Google Cloud Spanner, and keep all of its nodes in-country.

For a deeper comparison of these patterns, see active-active vs. active-passive cost and architecture trade-offs.

What to pin in-country:

• Primary database and all replicas
• Backups and snapshots
• Object storage buckets
• Application logs and audit logs
• Encryption keys
• Cache layers

What to check:

• Where does the platform’s support staff access data from?
• Where do telemetry and monitoring data flow?
• Where does TLS terminate? Global load balancers and CDNs can decrypt requests at edge sites outside the country.
• Where do managed AI APIs send prompts and responses for processing?
• Does the platform offer contractual commitments on data location?

Where can I find a cloud provider that keeps my data in one country to reduce risk?

AWS, Google Cloud, and Azure each run two or more regions in the United States, Canada, Australia, Japan, and India, so one provider can cover in-country failover in those countries. Not every country has two cloud regions from a single provider. In those cases, combining regions from multiple providers, or adding your own data center, may be the only viable in-country option. A detailed breakdown of provider coverage by country is available in best cloud providers for data sovereignty in 2026. For architectural patterns, see sovereign cloud architecture for data residency compliance.

What practical controls ensure data residency and sovereignty in a multi-cloud infrastructure?

• Region allow-lists enforced by policy. Each major cloud has a native mechanism: AWS Service Control Policies with aws:RequestedRegion conditions (with exemptions for global services such as IAM), Azure Policy’s built-in Allowed locations policy, and the Google Cloud resource locations organization policy constraint.
• Data classification and tagging. Tag every resource with its data classification (e.g., data-class: phi, data-class: pci). Use tags to trigger policy enforcement and to generate audit reports.
• Encryption with keys held in-region. Store encryption keys in the same jurisdiction as the data they protect. Cross-border key storage can undermine sovereignty even if the ciphertext stays in-country.
• Egress controls. Block or alert on data transfers to regions outside the approved list. Use VPC flow logs, firewall rules, and DLP inspection at egress points.
• Residency-aware backups. Explicitly configure backup destinations. Some services copy backups or geo-redundant replicas to a second region by default or with a single setting, and that region is not always in the same country. For example, Azure geo-redundant storage replicates to the paired region, and Brazil South is paired with South Central US. Multi-region storage locations such as Google Cloud’s EU or US buckets span several countries.
• Logging location. Route logs to a logging service or bucket pinned to an approved region. If you use a third-party logging provider, confirm where they store and process your data.
• Regular audits. Periodically scan for resources created outside approved regions. Automated compliance scanners can flag drift before it becomes an audit finding.

What cloud compliance checklist should I follow when migrating a regulated application to AI infrastructure?

AI workloads introduce new compliance surfaces. Prompts, model outputs, training data, and vector store contents can all contain regulated data. Follow this checklist:

1. Classify data before it reaches a model. Determine whether inputs contain ePHI, cardholder data, PII, or other regulated categories. Apply classification before inference, not after.
2. Decide what can go to third-party model APIs. If you use external APIs for inference, confirm the provider’s compliance posture, sign the appropriate agreements (BAA, DPA), and understand where prompts and responses are processed and stored. Confirm in writing that prompts are not used for training and how long the provider keeps them.
3. Prefer self-hosted or in-region inference for regulated data. Running models on infrastructure you control, within approved regions, keeps regulated data off third-party networks.
4. Log prompts and outputs with redaction. Maintain audit logs of what was sent to a model and what it returned. Redact or tokenize regulated fields before logging.
5. Apply access control to models and vector stores. Treat model endpoints and vector databases as regulated data stores. Enforce RBAC, authentication, and network segmentation.
6. Keep training and fine-tuning data in-region. If you fine-tune models on regulated data, the training data, checkpoints, and resulting model weights must stay within approved jurisdictions.
7. Document data flows for auditors. Create a data flow diagram that shows how regulated data moves through your AI pipeline: ingestion, preprocessing, inference, post-processing, storage, and deletion. Auditors will ask for this.
8. Plan for deletion. Embeddings and fine-tuned weights derived from personal data must be deletable when a data subject asks for erasure, so record which data each one came from.

What are the best practices for secrets management in IaC pipelines to meet cloud compliance requirements?

Secrets in infrastructure-as-code pipelines are a frequent audit finding and a common attack vector.

• No secrets in code, state files, or CI logs. Credentials leaked in repositories are a common way attackers get cloud access. Terraform writes values read through data sources into state; Terraform 1.10 added ephemeral resources and 1.11 added write-only arguments to keep secrets out of state and plan files.
• Use a secrets manager, referenced by name. Store secrets in a dedicated service such as AWS Secrets Manager, Google Cloud Secret Manager, or Azure Key Vault, or in HashiCorp Vault. IaC templates should reference secret names, never values.
• Short-lived credentials through workload identity federation. Replace static API keys with short-lived federated tokens that expire automatically, typically within an hour. AWS, GCP, and Azure all support workload identity federation natively.
• Least-privilege pipeline roles. The CI/CD service account should have only the permissions needed for the current deployment. Separate roles for plan and apply stages.
• Rotation. Automate rotation on a defined schedule, and rotate immediately after a suspected exposure or when someone with access leaves. Log every rotation.
• Secret scanning in CI. Run tools like Gitleaks or TruffleHog in your CI pipeline to catch secrets before they merge.
• Encrypted and access-controlled IaC state. Terraform and Pulumi state files can contain sensitive values. Encrypt state at rest, restrict access to the state backend, and audit who reads it.

How should I structure governance and policy as code for compliant multi cloud infrastructure using IaC?

Policy as code turns compliance rules into enforceable, version-controlled, auditable artifacts. Structure it in layers:

Preventive guardrails at the account or organization level. These are the broadest controls. Service Control Policies in AWS, Organization Policies in Google Cloud, and Azure Policy assignments at the management group level prevent non-compliant resources from being created in the first place.

Policy checks in CI before apply. Before any IaC change reaches production, validate it against your compliance rules. Open Policy Agent (OPA) is a widely adopted open-source engine for this purpose. Evaluate the JSON output of terraform plan with OPA through a tool such as Conftest, or use Pulumi CrossGuard policy packs with pulumi preview, and fail the pipeline on violations. In Kubernetes, OPA Gatekeeper or Kyverno applies the same rules at admission time.

Runtime detection. Even with preventive controls, configuration drift happens. Use runtime scanners to detect resources that have drifted from their compliant state and alert or auto-remediate.

Evidence collection for audits. Every policy evaluation, pass or fail, should produce a log entry. These logs become audit evidence showing that controls were in place and operating continuously.

Examples of rules to codify:

• Allowed deployment regions (for example, only the regions inside the country your data must stay in)
• Required encryption on all storage resources
• Required classification tags on every resource
• No public access on storage buckets or object stores
• MFA required for privileged roles

What cloud solutions offer multi tenant protection against cross tenant data breaches?

Cloud platforms separate tenants at three layers: separate accounts or projects (AWS accounts, Google Cloud projects, Azure subscriptions), compute isolation (separate clusters or microVM sandboxes such as Kata Containers), and data isolation (separate databases and per-tenant encryption keys).

Tenant isolation models

Choose a compute model and a data model separately. A common pattern is separate accounts for the most sensitive tenants and shared clusters with separate databases for the rest.

How do I design AI infrastructure for cloud compliance using BYOK and tenant data segregation?

In this context, BYOK (bring your own key) means encrypting data with keys you control, generated or imported into a cloud key management service, instead of provider-managed keys. Tenant isolation and per-tenant encryption are the two pillars of preventing cross-tenant data exposure.

Per-tenant encryption keys

Store each tenant’s encryption key in a cloud KMS (such as AWS KMS, Google Cloud KMS, or Azure Key Vault). The application references the tenant’s key ID when encrypting or decrypting that tenant’s data. If one tenant’s key is compromised, only that tenant’s data is exposed. Disabling or deleting a tenant’s key makes their data unreadable, which supports offboarding and deletion requests, and KMS logs show every use of each tenant’s key. Scope the application’s decrypt permission per tenant, or one compromised service can still use every key.

Tenant separation in AI workloads

Vector stores and model caches introduce new cross-tenant risks. If multiple tenants share a vector database, queries from one tenant could surface embeddings from another. Mitigations include:

• Namespace or collection-level separation within the vector store
• Tenant-scoped query filters enforced at the application layer
• Separate model inference endpoints per tenant for the most sensitive workloads
• Scoping prompt caches and response caches to one tenant. Inference servers that reuse the KV cache for shared prompt prefixes, and semantic caches that return a stored answer for a similar prompt, can leak data or timing signals across tenants. Add the tenant ID to the cache key or disable cross-tenant cache sharing.

How Control Plane approaches regulated workloads

Control Plane is the unbreakable platform for production workloads. It combines any cloud, neocloud, bare metal, and your own servers into what Control Plane calls a Global Virtual Cloud (GVC), and teams run it in a single region, a single cloud, or across clouds. Control Plane handles Day 2 operations for you, including autoscaling, right-sizing, built-in observability, and security, with no Kubernetes cluster you have to operate.

Compliance posture. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. These controls and the evidence they produce support your compliance, though they do not make you compliant on their own.

Data residency through location selection. A workload runs only in the locations of its GVC, which can include AWS, GCP, and Azure regions, Kubernetes clusters you already run, and servers in your own data center or from bare-metal and GPU providers. Control Plane Managed Kubernetes (MK8s) joins those servers into a cluster that Control Plane runs for you as a location. You can keep two or more locations inside one country, including combinations of providers, so the GVC’s location list acts as the region allow-list for that workload. Two in-country locations can also meet the 99.999% SLA condition below. Platform data is handled separately: org secrets are encrypted with envelope encryption and held in cloud HSMs across multiple providers, and logs are aggregated by Control Plane (30-day default retention) and can also be shipped to destinations you control, such as an Amazon S3 bucket or CloudWatch in your chosen region.

Active-active failover. A workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes clients to the nearest healthy location. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers. For strategies on surviving a single-region outage, see how to keep your app online during an AWS region outage.

Security defaults. Workloads are closed by default behind a deny-by-default internal firewall, and traffic between them uses mTLS, with each workload’s client certificate rotated every hour. All data is encrypted at rest and in transit. Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Control Plane Sandboxes, isolated environments for AI coding agents and untrusted code, run in dedicated Kata Containers microVMs.

Secrets and identity. Control Plane’s built-in secrets management lets workloads reference a secret as cpln://secret/NAME once their identity is granted the reveal permission, so secret values stay out of code and images. Control Plane’s Universal Cloud Identity gives workloads short-lived, least-privilege access to 600+ AWS, GCP, and Azure services, with no long-lived cloud keys in your code.

Audit trail. Every action, whether made through the console, CLI, Terraform, Pulumi, or AI agents using the Control Plane MCP server, is recorded in a tamper-proof audit trail. Your own workloads can write application events to the same audit trail through a custom audit context.

Policy and IaC. Control Plane policies are org-scoped and grant fine-grained permissions on every resource type to users, groups, service accounts, and workload identities. Control Plane ships its own Terraform and Pulumi providers, so it complements the infrastructure as code you already run.

Cost. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. SAFE Health, a connected-diagnostics platform working with Mayo Clinic, migrated 8 environments from AWS to Control Plane and cut its AWS spend 75 percent.

For teams looking to simplify multi-cloud operations or consolidate existing infrastructure, see how to consolidate multiple clouds into one platform. For a comparison of managed platform services versus building your own portability layer, see managed services vs. portability and cloud lock-in.

FAQ

Does a SOC 2 Type II report mean a platform is secure?

No. A SOC 2 Type II report shows that an independent auditor tested specific controls and found they operated effectively over a defined period. Read the report’s scope, exceptions, and complementary user entity controls to understand what the platform covers and what you must handle.

Is there an official HIPAA certification for cloud platforms?

No. The U.S. Department of Health and Human Services does not certify cloud platforms as HIPAA compliant. Look for a signed Business Associate Agreement and a published list of HIPAA-eligible services. The BAA sets the platform’s contractual obligations for safeguarding ePHI, and under HIPAA a business associate is also directly liable for certain requirements, including the Security Rule.

What is the difference between PCI DSS Level 1 and other levels?

Level 1 is the top tier for service providers. Visa and Mastercard assign it to service providers that handle more than 300,000 card transactions a year, and it requires an annual assessment by a Qualified Security Assessor, who issues a Report on Compliance (ROC) and an Attestation of Compliance (AoC). Level 2 service providers can self-assess with Self-Assessment Questionnaire D. Ask any provider for its responsibility matrix, which PCI DSS v4.0 requirement 12.9.2 obliges it to provide on request.

Can I achieve multi-region failover while keeping all data in one country?

Yes, if the country has two or more available cloud regions or data center locations. Run active-active or active-passive between them and pin all replicas, backups, logs, and encryption keys to those locations. If a single provider lacks two in-country regions, combine providers or add your own data center.

How do I prevent secrets from leaking in IaC pipelines?

Store secrets in a dedicated secrets manager and reference them by name in your IaC templates. Use short-lived credentials through workload identity federation, run secret-scanning tools in CI, encrypt your IaC state files, and restrict access to the state backend.

What is the shared responsibility model in cloud compliance?

The platform is responsible for infrastructure-level controls such as physical security, hypervisor patching, and network fabric. You are responsible for identity and access management, secrets, application code security, and data classification. Network segmentation, encryption, logging, backups, and incident response are shared.

How should I handle regulated data in AI inference pipelines?

Classify data before it reaches a model. Use self-hosted or in-region inference for regulated data. Log prompts and outputs with redaction of sensitive fields. Apply access control to model endpoints and vector stores. Document the full data flow for auditors.

Does Control Plane make my application compliant?

No. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant and provides audited controls and evidence that support your compliance, but your application still needs its own controls for data classification, identity, secrets, application security, and logging.

Can Control Plane keep my workload in a single cloud?

Yes. Many teams run Control Plane in a single cloud for its Day 2 operations and security defaults. A GVC can contain locations from a single provider, and two regions of one cloud qualify for the 99.999% SLA when a workload runs at least two replicas in those two locations.

How does Control Plane handle workload isolation?

Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Control Plane Sandboxes run in dedicated Kata Containers microVMs. Workloads are closed by default behind a deny-by-default internal firewall, and traffic between them uses mTLS.

See how Control Plane runs regulated workloads.

Talk to the Control Plane team or sign up free

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Policy as code expresses governance, security, and compliance rules as machine-readable code that runs automatically at every stage of your delivery pipeline. Keep policies versioned, tested, and enforced at several points.

• Four enforcement points. Editor and pull request checks, CI plan validation, deploy-time admission control, and continuous runtime scanning. Each catches a different class of issue, and none is sufficient alone.
• Repository structure. Keep a dedicated policy library separate from infrastructure modules. Map policies to control IDs (CIS Benchmarks, SOC 2 criteria), test every policy, and publish versioned releases that pipelines pin.
• Tool categories. Use IaC provisioning tools, policy engines and static scanners, and cloud-native controls together.
• Patching and upgrades. Keep cluster versions within the supported window, automate node image patching through rolling node pool replacement, and use maintenance windows and pod disruption budgets.
• Evidence for audits. Store plan outputs, policy decisions, approvals, and exception records as immutable artifacts.
• Control Plane. One access model and a tamper-proof audit trail across every cloud and cluster Control Plane runs on, including AWS, GCP, Azure, and your own Kubernetes clusters, plus Terraform and Pulumi providers and a Kubernetes operator that fit into GitOps pipelines.

How should I structure governance and policy as code for compliant multi cloud infrastructure using IaC?

The reference structure covers policy tiers, enforcement points, repository layout, severity levels, drift detection, state security, and audit evidence.

Policy tiers

Organize policies into three tiers that mirror organizational responsibility.

Organization guardrails are the broadest controls. AWS Service Control Policies, Azure Policy assignments at management group scope, and Google Cloud Organization Policy constraints set boundaries that no team can override. They block whole categories of risk, such as deploying resources in unapproved regions or enabling public access to storage. SCPs do not restrict the AWS management account, so keep workloads out of it, and AWS resource control policies (RCPs) add org-wide limits on resource access.

Platform policies apply to IaC plans and Kubernetes admission. They are written in a policy engine language (Rego, Sentinel, or Kyverno YAML) and evaluated in CI pipelines and at admission time. Examples include requiring encryption at rest, enforcing tagging standards, and restricting instance types.

Team-level rules are scoped to a single service or namespace. A machine learning team might limit its namespace to two GPU instance types from the platform-approved list. Team rules never widen what the tiers above allow; widening needs a platform-level exception.

The four enforcement points

Each enforcement point catches a different class of problem.

Admission controllers can validate (reject) or mutate (rewrite) a request. Kyverno, Gatekeeper, and Azure Policy modify and deployIfNotExists effects can mutate. Mutation changes resources silently, so the live object differs from Git. Prefer validation for security controls, and keep mutation to defaults such as labels and resource requests.

Repository layout

A clean layout separates concerns and makes policies consumable by any pipeline.

• policy-library/ holds all policy source files organized by control family (identity, encryption, networking, tagging, compute). Each policy references one or more control IDs, such as a CIS AWS Foundations Benchmark recommendation or SOC 2 CC6.1.
• policy-library/tests/ holds unit tests for every policy, run with opa test, conftest verify, kyverno test, or sentinel test. Fixtures include a passing and a failing input for each rule, and CI runs the tests on every change to the library.
• infrastructure-modules/ holds reusable Terraform, OpenTofu, or Pulumi modules.
• environments/ holds environment-specific variable files and backend configuration.

Tag the policy library with semantic versions. Pipelines pin a version and upgrade deliberately, like any other dependency.

Protect the policy library with CODEOWNERS and branch protection so policy changes need security or platform approval. Make the policy step a required status check that infrastructure repositories cannot disable. Split CI identities so the plan role is read-only and only the apply role can change resources.

Severity levels and exceptions

Give every policy a severity level.

• Advisory. Logs a warning and does not block deployment. Useful for new policies under evaluation.
• Soft-mandatory. Blocks deployment unless an authorized approver overrides it. The override is recorded in the run log or in Git with a justification, an owner, and an expiration date.
• Hard-mandatory. Cannot be overridden. Reserved for critical controls, such as blocking public databases or wildcard IAM permissions.

Exceptions follow a time-boxed approval process. The request is a pull request in the policy repository with the business justification, compensating controls, the approver’s identity, and an expiration date. An automated job revokes expired exceptions and opens a ticket if the underlying issue remains.

Keep a break-glass path for incidents. An emergency role bypasses pipeline gates, requires two-person approval, alerts the security team on use, expires automatically within hours, and triggers a post-incident review. Every break-glass session is logged and later reconciled back into code.

Roll out new admission policies in audit mode first (Gatekeeper enforcementAction: dryrun or warn, Kyverno Audit), review the violations, then switch to enforcement.

Drift detection and remediation

Drift happens when someone changes a resource outside the IaC pipeline, through a console click or an ad hoc API call. Detect it by running terraform plan -refresh-only -detailed-exitcode or tofu plan -refresh-only -detailed-exitcode on a schedule; exit code 2 means the plan found drift. HCP Terraform health assessments run drift detection automatically. GitOps controllers such as Argo CD and Flux reconcile continuously and can self-heal Kubernetes resources, and Crossplane reconciles cloud resources the same way. AWS Config rules and Azure Policy with the audit effect give continuous drift signals.

Automate remediation for low-risk corrections (re-enabling logging, reapplying a tag) and require human approval for higher-impact changes (modifying network rules, resizing databases). Before reverting, decide whether to revert the change or import it into code; blindly reverting an emergency fix can reopen an incident.

State file security

State files and saved plans hold resource attributes and secrets in plaintext. Store state in an encrypted backend with versioning, restrict read access to the pipeline identity and a small admin group, and log every access. OpenTofu 1.7 and later supports client-side state encryption. Never commit state or plan files to Git, and apply the saved plan file (for example terraform apply tfplan) so the plan that passed policy is the plan that runs.

Evidence for audits

Auditors need proof that each control ran. Store these artifacts automatically:

• Plan output for every deployment, with sensitive values redacted before storage, because plan JSON and state files contain secrets in plaintext.
• Policy evaluation results with pass, fail, and advisory outcomes.
• Approval records from pull request reviews and exception overrides.
• Drift detection reports with timestamps and remediation actions.
• Each artifact tagged with the commit SHA, the policy library version, and the approver identity.

Keep artifacts in a write-once store, such as S3 Object Lock in compliance mode or Azure immutable blob storage, with retention set by your compliance framework. PCI DSS requires at least 12 months of audit log history, with the most recent 3 months immediately available, and SOC 2 Type II observation periods usually run 3 to 12 months. Export the Azure Activity Log, which keeps only 90 days by default.

What are the best infrastructure-as-code tools?

The IaC landscape includes cross-cloud provisioning tools, cloud-specific frameworks, Kubernetes-native resource managers, and configuration management systems. For a broader list, see top 10 infrastructure as code tools.

Standardize on one cross-cloud tool for most of your infrastructure. Use a cloud-specific tool only where a single cloud owns the workload end to end and the native tool offers a clear advantage, such as faster support for new services.

Which tools automate AI infrastructure provisioning and policy enforcement for multi cloud environments?

Policy engines and static scanners complement IaC tools by evaluating what those tools produce.

AI-specific guardrails

AI and ML workloads add resource and data governance requirements that fit the policy as code model. Write them as ordinary policies in whichever engine you use.

• GPU instance types and quotas. Allow only approved GPU types per team and cap concurrent GPU instances through cloud quotas such as AWS Service Quotas and Azure quotas, because plan-time policy cannot count across deployments.
• Approved regions for model and data residency. Restrict training jobs and model storage to regions that meet data residency requirements.
• Required tags for cost attribution. Enforce tags such as project, cost-center, and model-name on every GPU instance and storage bucket.
• Encryption for model artifacts and training data. Require encryption at rest and in transit for buckets and Azure Blob containers holding datasets or model weights.
• Network egress limits for inference services. Restrict outbound traffic from inference endpoints to known destinations.
• Approval gates for large GPU reservations. Require human approval when a deployment requests GPU capacity above a set threshold.

How can AI operable infrastructure automate patching, upgrades, and policy enforcement for clusters?

Keeping clusters current is continuous work. The Kubernetes project maintains release branches for the three most recent minor versions, and the Kubernetes version skew policy defines how far control plane and node components can drift apart. Falling outside the supported window means losing security patches and risking incompatibility.

Patching and upgrade practices

Automate node image patching through rolling node pool replacement instead of in-place updates. Create a node pool with the patched image, cordon and drain old nodes while respecting pod disruption budgets, and remove the old pool once workloads have moved. Schedule upgrades in maintenance windows that match your lowest-traffic periods.

Upgrade the control plane first, one minor version at a time, then the nodes. Scan manifests for removed APIs before each upgrade with tools such as kubent or Pluto. Managed services add their own windows: Amazon EKS gives 14 months of standard support per version plus paid extended support, AKS and GKE offer auto-upgrade or release channels, and Karpenter can replace nodes automatically when their image changes. Set surge and max-unavailable values, and check that no pod disruption budget allows zero disruptions, because that blocks drains.

Run policy checks at admission time and continuously. A cluster that was compliant at deploy time can drift as resources are created, images change, or configurations are edited.

AI assistants and infrastructure changes

AI assistants can propose infrastructure changes as pull requests or run read-only diagnostics against live clusters. Agent-initiated changes must pass the same policy checks, approval gates, and audit trail as human changes, and a read-only profile must not be able to modify resources. For agent guardrails in depth, see how to let AI agents operate production infrastructure safely.

Which tools provide an easy installation and setup process for cloud infrastructure orchestration across AWS and Azure?

Judge setup effort against these criteria before committing to a tool or platform.

Setup checklist for a first cross-cloud pipeline

• One CLI. A single CLI that works against both AWS and Azure without manual context switching.
• One state backend. A managed backend (HCP Terraform, Pulumi Cloud, or an S3 or Azure Blob backend with locking) to avoid state corruption. A managed backend reduces the work of locking, encryption, and access control; a self-hosted backend gives more control but needs maintenance.
• Workload identity federation for provider authentication. Configure OIDC trust between your CI system and each cloud, so CI variables hold no static access keys. An AWS IAM OIDC identity provider with a role trust policy and Azure federated credentials both support this pattern.
• Reusable modules. Publish shared modules to an internal registry so teams consume versioned modules instead of copying code.
• Policy checks from day one. Wire Checkov or Conftest into the pipeline before the first production deployment. Retrofitting policy checks is harder than starting with them.

On Control Plane, the Terraform and Pulumi providers and the Kubernetes operator plug into this kind of pipeline; details are in the How Control Plane fits section below.

A starter policy set for multi-cloud IaC

These policies cover the most common compliance and security gaps. Adapt severity and enforcement points to your risk tolerance.

Admission for cloud resources means deploy-time hooks such as the Azure Policy deny effect or AWS CloudFormation Hooks.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes (MK8s), a CNCF-certified Kubernetes service, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

IaC integration

Control Plane ships its own Terraform and Pulumi providers, so it complements the infrastructure as code you already run. The Terraform provider is published at registry.terraform.io/providers/controlplane-com/cpln. The Control Plane Kubernetes Operator manages Control Plane resources from Kubernetes manifests and supports Argo CD and GitOps workflows. On Control Plane, workloads run in a Global Virtual Cloud (GVC), Control Plane’s term for a set of locations a workload runs across. The CLI exports existing resources as Terraform HCL, for example cpln gvc get my-gvc --output tf, and as Kubernetes manifests for the operator with --output crd. The CLI, console, Terraform and Pulumi providers, and the Control Plane MCP server all work through the same Control Plane API.

Access control and guardrails

Beyond the built-in superusers and viewers policies, nothing is permitted inside a Control Plane org until a policy grants it. Policies can target resources by name or by a query on tags, properties, or relations, so resources created later that match are covered automatically. Every resource has an Access Report showing who holds each permission and which policy granted it. Control Plane prevents deletion of any resource tagged cpln/protected=true. Org quotas cap resource counts, such as workloads per GVC, and only an approved increase request raises one. Built-in metrics report quota_utilization_percent once a quota passes half its max, so you can alert before creates fail.

Several defaults line up with the starter policy set. Workload-to-workload traffic is denied by default, and external inbound and outbound traffic stay closed until a firewall rule opens them. Default workload endpoints accept TLS 1.2 or higher. Control Plane’s Universal Cloud Identity (UCI) covers the no-long-lived-keys control for cloud access, because workloads receive credentials at runtime with no long-lived cloud keys in your code.

Control Plane works alongside OPA, Sentinel, Checkov, Kyverno, and cloud-native policy services. It narrows what those tools must govern by applying one access model across every cloud it runs on.

Audit trail and compliance

A tamper-proof audit trail records platform actions from every interface; it does not collect logs from downstream services. Workloads can write their own events to the same audit trail through an audit context, and org logs can ship to external providers such as Amazon S3, CloudWatch, and Datadog for off-site retention. Control Plane publishes a PCI responsibility matrix showing which controls it covers and which stay with the customer. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant, and the customer’s own controls remain the customer’s responsibility.

Day 2 operations

Control Plane handles Day 2 operations for you: autoscaling, built-in observability, security, and right-sizing with Control Plane’s Capacity AI, which adjusts reserved CPU and memory based on historical usage, with no Kubernetes cluster you have to operate. On MK8s locations, Control Plane maintains the clusters for you, including upgrades to the Kubernetes version you select and node autoscaling. Capacity AI does not apply to GPU workloads or to workloads that autoscale on CPU utilization or on multiple metrics. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations.

AI assistants

AI assistants connect to Control Plane’s MCP server through OAuth with per-org consent, reach only the orgs the user selects, and call the same API as every other interface, so their actions land in the same audit trail. A readonly profile exposes no tool that creates, updates, deletes, or executes, so an assistant on it cannot change a live resource.

For running one app across clouds, see how to run the same app on AWS, GCP, and Azure at the same time. For teams without a dedicated platform team, see production-grade infrastructure without a platform team.

FAQ

Common questions from teams adopting policy as code for multi-cloud infrastructure.

What is the difference between policy as code and infrastructure as code?

Infrastructure as code defines what resources to create, such as a virtual machine, a database, or a network. Policy as code defines the rules those resources must satisfy, such as “encryption at rest is required” or “only approved regions are allowed.” IaC provisions and policy as code governs, and both run in the same pipeline.

Should I use OPA or Sentinel?

Use OPA if your governance scope extends beyond Terraform, because it is a general-purpose, CNCF-graduated engine that works across IaC, Kubernetes, and APIs. Sentinel fits teams standardized on HCP Terraform or Terraform Enterprise that want its built-in integration. Many organizations use both: Sentinel for Terraform plan checks and OPA for Kubernetes admission and other policy decisions.

Is Terraform still open source?

No. HashiCorp moved Terraform from MPL 2.0 to the Business Source License in August 2023, which makes it source-available but not open source under the Open Source Initiative’s definition. OpenTofu is a community fork, now a CNCF Sandbox project, that keeps the MPL 2.0 license.

How do I handle exceptions to a policy?

Open a pull request in the policy repository that names the rule, the reason, a compensating control, an approver, and an expiry date. Expired exceptions are revoked automatically, and each approval and revocation is kept as evidence.

How do I structure governance and policy as code for compliant multi-cloud infrastructure using IaC?

Use three policy tiers: organization guardrails in cloud-native controls, platform policies evaluated in CI and at admission, and team-level rules scoped to specific services. Keep all policies in a versioned, tested library repository, enforce them at four points from editor to runtime, run a documented exception process, and store evidence automatically.

Which tools automate AI infrastructure provisioning and policy enforcement for multi-cloud environments?

Combine a cross-cloud IaC tool (Terraform, OpenTofu, or Pulumi) with a policy engine (OPA, Sentinel, or Kyverno) and a static scanner such as Checkov, and layer cloud-native controls (AWS SCPs, Azure Policy, Google Cloud Organization Policy) underneath. For AI workloads, add policies for GPU types and quotas, approved regions, required tags, and encryption of model artifacts.

What is drift detection and why does it matter?

Drift detection compares a resource’s live state with what your code declares and flags differences, which usually come from console or API changes made outside the pipeline. Without it, your IaC stops being a reliable record of what exists, and compliance evidence loses its value.

How can AI operable infrastructure automate patching, upgrades, and policy enforcement for clusters?

Replace node pools with patched images on a schedule, keep cluster versions inside the supported Kubernetes release window, and run admission and runtime policy checks continuously. Let AI assistants propose changes as pull requests with human approval for production, and route agent changes through the same policy pipeline and audit trail as human changes.

Which tools provide an easy installation and setup process for cloud infrastructure orchestration across AWS and Azure?

Look for a single CLI, a managed state backend, OIDC-based workload identity federation for provider authentication, a module registry, and built-in policy checks. Terraform, OpenTofu, and Pulumi all support these patterns. Policy checks come from HCP Terraform (Sentinel or OPA), Pulumi CrossGuard, or an external scanner such as Checkov or Conftest. Control Plane adds its own Terraform and Pulumi providers and a Kubernetes operator that fit into the same pipeline.

Do I need a separate policy repository or can policies live with my infrastructure code?

Use a separate policy library repository. It lets a security or platform team review policies centrally, version them independently, and serve many infrastructure repositories, each pinned to a policy version so policy releases do not couple to infrastructure changes.

Ready to run workloads across clouds with one access model and one audit trail? Talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

OpenClaw calls itself “The AI that really does things”, and the project has more than 390,000 GitHub stars. One Gateway connects the model of your choice to chat apps such as Telegram, Slack, Discord and WhatsApp, and gives it a shell, files, a headless browser, scheduled jobs, memory and MCP servers. The hard part is running an agent with a shell safely: long-lived state, bot tokens that must stay secret, and an endpoint you do not want on the internet by accident. On Control Plane, OpenClaw is one install from the Template Catalog, Control Plane’s library of ready-to-install apps, from the Console, CLI, Terraform or Pulumi. It has no public endpoint by default, reads its keys from a secret you create, and comes with the Control Plane MCP server registered, so after you sign in the agent can manage your Control Plane resources from chat.

Why self-host OpenClaw on Control Plane

• One install. No manifests and no wiring. The Gateway, its storage, its identity and its browser come up together with secure defaults.
• Secrets kept out of config. Your gateway token, model key and bot tokens live in one Control Plane secret you create. The workload reads them by reference, so none of them appear in Helm values or your install command.
• Least privilege, created for you. The workload’s Control Plane identity can read your secret and its own first-boot config (the Gateway’s initial settings file), and nothing else. Signing in to the Control Plane MCP server is a separate, explicit step.
• Private by default. No public endpoint until you ask for one. Telegram, Slack, Discord and WhatsApp all connect outbound, so they work on a private install.
• Control Plane from chat. The Control Plane MCP server exposes the Control Plane API as tools an AI agent can call. It is pre-registered and does nothing until someone signs in, and after sign-in the agent can do what that account can do.
• Snapshots included. State sits on a volume with daily snapshots, with no bucket or backup job to set up.

Included out of the box

• Any model you bring: Anthropic, OpenAI, or any OpenAI-compatible endpoint such as OpenRouter or a LiteLLM gateway
• Headless Chromium for the agent’s browser tool (can be turned off)
• The built-in Control UI for configuring and watching the agent
• A 10 GiB volume for config, sessions, plugins, workspace and memory, which can grow automatically (off by default)
• Daily snapshots at 03:00 UTC, kept for 7 days (schedule and retention are configurable)
• An optional OpenAI-compatible HTTP API
• Built-in Control Plane logs and metrics, plus Control Plane’s tamper-proof audit trail of platform actions

How to run OpenClaw securely without a public URL

The Gateway accepts no traffic from the internet until you turn it on, and by default only workloads in the same Global Virtual Cloud (GVC), Control Plane’s term for a set of locations a workload runs across, can reach it. Traffic between workloads uses mTLS. For setup you reach the Control UI with cpln port-forward from the Control Plane CLI, and you turn on public access, optionally on a custom domain, only when you want it. Apps in the same GVC reach the optional HTTP API over internal DNS by sending the gateway token, which is a full-admin credential, as a bearer token, plus an X-Forwarded-For header with a non-loopback address. Because the chat channels dial out, you can run a Slack or Telegram assistant with no public URL at all.

If you turn on public access, the gateway token guards the agent from the internet. Each new browser also needs a one-time approval from the CLI, but the HTTP API, if you enable it, accepts the token alone, so keep the token long and random.

Control Plane isolates each workload with cgroups, namespaces, and restricted syscalls and capabilities. The agent’s shell runs inside the Gateway’s own container, with the Gateway’s access: its volume, its keys and the workload’s Control Plane token, which can only reveal the two secrets the template grants. Outbound traffic is open so the Gateway can reach your model and chat apps. Like any agent with a shell, OpenClaw can be steered by an approved contact, a web page it reads or a third-party skill, so give it only the access you would give those sources.

Where OpenClaw’s API keys and bot tokens live

You create one Control Plane dictionary secret, a secret that holds named key-value pairs, with a gateway-token (your Control UI login, at least 24 characters), an llm-api-key, and the bot tokens for each channel you enable (WhatsApp needs none). The template creates a policy that lets the workload’s identity reveal exactly that secret and the first-boot config, which holds no credentials. The Gateway, and so the agent’s shell, can read these values at runtime, so use a model key with a spend limit and bot tokens you can revoke. OpenClaw also runs a heartbeat turn every 30 minutes by default, so it spends some model tokens even when nobody messages it.

To rotate a key, export the secret with cpln secret reveal SECRET_NAME -o yaml, edit it, re-apply it with cpln apply -f, and delete the exported file, which holds your keys in plain text. Then run cpln workload force-redeployment to load the new values. Because OpenClaw runs a single Gateway, the redeployment takes the assistant offline for about 4 minutes, so rotate during a quiet window. If a token leaks, revoke it with the provider too, because changing the secret does not invalidate the old token.

Manage Control Plane from your chat apps

With cplnMcp.enabled (the default), the Control Plane MCP server is registered as cpln. In the default shared mode, you sign in once from a terminal and approved chat contacts act through that account, so sign in with a dedicated Control Plane user whose policies grant only what the assistant needs, or set cplnMcp.enabled to false. Control Plane actions taken through the MCP server are recorded in the audit trail under that user. In per-requester mode, which needs public access, each person signs in with their own Control Plane account from a chat link. Once signed in, you can ask the assistant to list your GVCs, inspect a workload or check a deployment.

State that survives

Pairings, the WhatsApp link, MCP sign-ins, sessions and memory live on the volume, and the platform snapshots it on a schedule. The Gateway runs as one replica by design, because OpenClaw supports one Gateway per state directory and each bot token holds only one connection. For a separate assistant per person or team, install the template again under a different release name, with its own secret and bot tokens. Control Plane encrypts volumes by default; snapshots still hold channel sessions and tokens, so limit who can restore them. Uninstalling deletes the volume and all state, so copy out anything you need first.

How to deploy OpenClaw on Control Plane

Sign up for Control Plane, then work in this order. First, create the dictionary secret described above, in the Console or with cpln secret create-dictionary, using openssl rand -hex 32 for the gateway token. It must exist before you install: an install that points at a missing secret reports success but never starts. Second, pick an existing GVC with a single location, because the template does not create one and each extra location would run its own assistant competing for the same bot tokens. Third, install OpenClaw from the Template Catalog in the Console, or with cpln helm install, setting secret.name to your secret’s name. Open the Control UI with cpln port-forward and sign in with the gateway-token value.

To connect a chat app, add its bot token to the secret first (WhatsApp needs none), then enable the channel with cpln helm upgrade, repeating every value you installed with, including secret.name, because an upgrade does not keep values you leave out. DM your bot, approve the pairing code it replies with using cpln workload exec, then DM it again, and the reply comes from your model. Approve only contacts you would trust as admins, because an approved contact can have the agent run shell commands in the Gateway’s container. To hold each command for your approval, set tools.exec.ask to always.

The OpenClaw template documentation has the exact commands, bot setup for each channel, the custom domain, the HTTP API and troubleshooting. Running agents for a team or across your company? Request a demo.

FAQ

These answers cover running OpenClaw with the Control Plane template.

What is OpenClaw?

OpenClaw is an open source, self-hosted AI assistant you chat with in apps such as WhatsApp, Telegram, Slack, Discord, iMessage and Teams. It can run shell commands, browse the web, edit files and run scheduled jobs with the model you choose. The Control Plane template connects Telegram, Slack, Discord and WhatsApp.

How do I self-host OpenClaw?

Create a dictionary secret with a gateway token and your model key, then install OpenClaw from the Control Plane Template Catalog into an existing single-location GVC, from the Console or with cpln helm install, setting secret.name to that secret. The template creates the Gateway workload, its volume, identity and access policy, and keeps it private by default.

What resources does OpenClaw need on Control Plane?

The template reserves 500 millicores of CPU and 1 GiB of memory by default, with limits of 2 cores and 3 GiB, plus a 10 GiB volume. With the browser tool turned off, a 2 GiB memory limit is enough. You pay for the CPU and memory a workload reserves, with no contracts or minimums.

How do I run OpenClaw securely?

Keep the Gateway off the public internet and keep its keys in a secret the workload reads by reference, which the Control Plane template does by default. Then approve pairings only for trusted contacts, vet third-party skills before installing them, and sign in the Control Plane MCP server with a limited-permission user or disable it. To approve each shell command yourself, set tools.exec.ask to always. Like any agent with a shell, OpenClaw can be steered by an approved contact or by a page it reads, so these limits matter.

Does OpenClaw on Control Plane need a public URL?

No. The install is private by default, you reach the Control UI with cpln port-forward, and all four chat channels in the template, including WhatsApp, connect outbound. Turn on public access, optionally on a custom domain, when you want the Control UI on the web for daily use, the HTTP API for callers outside Control Plane, or each chat user signed in to Control Plane with their own account.

Where are OpenClaw’s API keys and bot tokens stored?

In one Control Plane dictionary secret that you create before installing. The workload reads the values by reference at startup, through a policy created by the template that grants its identity access to that secret and the first-boot config only.

Can I run OpenClaw in more than one location?

Run it in one location. A workload runs in every location of its GVC, so each extra location would start its own assistant, and they would compete for the same bot tokens. Pick a GVC with a single location, and install the template again under a different release name, with its own secret and bot tokens, for more assistants.

How is OpenClaw’s state backed up?

All state, including pairings, the WhatsApp link, MCP sign-ins, sessions and memory, lives on a volume (10 GiB to start) that the platform snapshots daily at 03:00 UTC and keeps for 7 days by default. No bucket or cloud account is needed. Restoring a snapshot reverts the volume in place, so anything written after it, including newer pairings, is lost, and uninstalling deletes the volume.

Which models can OpenClaw use on Control Plane?

Any model you bring a key for: Anthropic, OpenAI, or any OpenAI-compatible endpoint such as OpenRouter or a LiteLLM gateway. You set the provider and bare model ID as template values, plus a base URL for an OpenAI-compatible endpoint. The default is Anthropic’s Claude Opus 5.5.

What can someone who messages my OpenClaw bot do?

Nothing until you approve them: a new contact gets a pairing code, and the agent does not process their messages until you approve it. An approved contact can use everything the agent can, including shell commands in its container, the browser, the keys it can read, and, if the Control Plane MCP server is signed in, that account’s permissions. Approve only people you would trust as admins.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

• One definition everywhere. Deploy the same workload definition to every region so regions cannot drift apart.
• Global routing with health checks. DNS routing, anycast load balancers, or both send users to the nearest healthy region and away from failures.
• A data topology per service. Single primary with replicas, multi-primary, or distributed SQL, each with its own trade-off between write latency, data loss on failover, and complexity.
• Failover differs by layer. Stateless services fail over through routing. Databases need fencing, promotion, and repointed connections, automated and rehearsed.
• Quorum sets the location count. Automatic failover needs a majority, so two locations survive no location loss, three survive one, and five survive two.
• On Control Plane, one definition runs active-active in every location of a GVC behind a global endpoint, with database templates for the data layer.

How do I run my app across multiple regions?

The goal is to deploy the same application to every region without hand-built per-region setup, so adding or removing a region is an operational change.

How do I deploy one application to multiple regions with a single configuration?

1. Make services stateless. Move sessions, caches, and mutable state into external stores so any instance can be replaced or moved.
2. Externalize configuration and secrets. Connection strings, keys, and feature flags come from a secrets manager or configuration service, so one container image runs in any region.
3. Use infrastructure as code or a platform with multi-region placement. Terraform or Pulumi can target many regions from one module, and some platforms place a workload in several regions from a single definition.
4. Override per region only where you must. Limit overrides to region-specific needs such as instance types or data residency labels; keep the image, resources, and health checks identical.
5. Choose a rollout order. Region-by-region rollouts catch problems before they spread; all-at-once rollouts are faster but risk a bad release hitting every region together.
6. Check regional dependencies first. Quotas, instance availability, and data residency rules differ by region, so request capacity before you need it.

For one application on AWS, GCP, and Azure at once, see running the same app across AWS, GCP, and Azure.

How do I route users to the nearest healthy region automatically?

Two routing mechanisms, both driven by health checks, cover most designs.

• DNS latency and geo routing. Amazon Route 53 offers latency, geolocation, and failover routing with health checks. Azure Traffic Manager routes at the DNS level with endpoint monitoring. Google Cloud DNS routing policies support geolocation, weighted round robin, and failover policies with health checks. DNS routing is simple, but TTLs and client caching slow the switch.
• Anycast global load balancers. Google Cloud’s global external Application Load Balancer uses anycast IPs for HTTP(S) traffic. AWS Global Accelerator gives static anycast IPs that route TCP and UDP to the nearest healthy endpoint. Azure Front Door combines global routing with WAF and caching. These route each connection without waiting for DNS caches to expire.
• Health checks. Check application health endpoints in addition to TCP ports, and require several failures before removing a region so traffic does not flap.

How do I run stateful workloads like databases across multiple regions?

Every multi-region database design trades write latency, data loss on failure, consistency, and operational effort against each other.

Replication modes

• Synchronous replication makes the primary wait for at least one remote replica to confirm each write. Committed data survives a failover to that replica, but every write pays a cross-region round trip, which grows with distance. Writes stall if no synchronous replica is reachable, so fail over only to a replica confirmed in sync.
• Asynchronous replication confirms writes locally and ships changes in the background. Writes stay fast, but anything not yet replicated when the primary fails is lost, so the data at risk equals the replication lag at that moment.

Database topologies

• Single primary with read replicas. One region takes writes; others serve reads and can promote a replica. Amazon Aurora Global Database works this way, and PostgreSQL with Patroni provides the same pattern in open source with automatic leader election.
• Multi-primary (active-active writes). Every region accepts writes, which removes the write-latency penalty for remote users but requires conflict resolution. Last-writer-wins is simple and can silently discard concurrent updates; ownership rules or conflict-free data types reduce that risk. Azure Cosmos DB supports multi-region writes.
• Distributed SQL with consensus. Each write commits once a majority of replicas agree, giving strong consistency without a single primary at the cost of cross-region write latency. Google Cloud Spanner, CockroachDB, and YugabyteDB take this approach.
• Partitioning by tenant or region. Each tenant or partition has a home region that takes its writes, which avoids conflicts and helps with data residency.

Quorum math and location count

Consensus systems need a majority to agree, so a cluster across N locations survives the loss of floor((N-1)/2) locations.

• Two locations survive no location loss for automatic failover, because the survivor cannot form a majority alone.
• Three locations survive the loss of one, the minimum for safe automatic promotion.
• Five locations survive the loss of two.

With two locations and no witness, a system cannot tell a network partition from a failure, so automatic promotion risks split brain. Run consensus-based databases in at least three locations, or add a witness or arbiter in an independent third location.

Caches, queues, and object storage

• Caches. Redis replication with Sentinel or Redis Cluster spans regions, and Sentinel needs a majority across three locations to fail over automatically. Redis replication is asynchronous, so treat caches as rebuildable, and expect a cold cache in the surviving region to shift load onto the database.
• Queues and streams. RabbitMQ supports federation and shovels between clusters, and Kafka uses MirrorMaker 2 for cross-cluster replication, which translates consumer offsets approximately, so consumers must tolerate duplicates. Most designs keep ordering per partition within a region and accept eventual consistency across regions.
• Object storage. S3 Cross-Region Replication, Cloud Storage dual-region and multi-region buckets, and Azure geo-redundant storage replicate asynchronously, so the newest objects may be missing or unreadable in the surviving region right after a failure. Cloud Storage’s default replication targets a 12-hour RPO, and turbo replication on dual-region buckets targets 15 minutes.

Consistency pitfalls

• Read-your-writes. Users moved to another region, or reading from asynchronous replicas, can see stale data; route reads to the primary right after a write, or use session tokens.
• Unique IDs. Multi-primary designs need globally unique IDs, such as UUIDv7, instead of per-node sequences.
• Schema changes. Use expand-and-contract migrations, because old and new code run side by side during region-by-region rollouts, and check whether your replication carries DDL.

How do I fail over a database-backed application between cloud regions?

Run database failover as an ordered sequence.

1. Detect. Application-level health checks confirm the primary region is down rather than slow.
2. Fence the old primary. It must lose its leader lease and demote itself; use STONITH or a fencing token where available, and revoke its write credentials if it is still reachable, so two primaries never write at once.
3. Promote. Promote a replica in a surviving region. With asynchronous replication, writes not yet replicated are lost, so track replication lag continuously to know your real RPO.
4. Repoint applications. A stable endpoint or proxy that follows the current leader makes this automatic; otherwise applications need new connection strings or DNS changes. Drain or kill pooled connections to the old primary.
5. Verify. Confirm writes succeed, replicas stream from the new primary, and application health checks pass.
6. Rebuild the old region as a replica. Export any writes it holds that never replicated, resync it with pg_rewind or a fresh base backup, and only then let it rejoin as a replica. Moving the primary back is a separate, planned operation.

Clients should retry with idempotency keys, because in-flight transactions may have committed just before the failure.

Alert when replication lag exceeds your RPO, because lag grows under heavy writes and network trouble. Serving reads while blocking writes is a useful degraded mode when promotion is unsafe. Rehearse the whole sequence in regular game days to measure real RTO and expose hidden dependencies; for testing approaches and costs, see active-active vs. active-passive costs.

How can my company avoid being taken down by the next major cloud outage?

Each item removes a common cause of regional downtime.

• Run in more than one region, or on more than one provider; availability zones alone share a region’s fate.
• Audit hidden single-region dependencies: DNS, secrets, container registries, CI/CD, monitoring, and identity.
• Choose a data topology per service instead of one pattern for everything.
• Test failover regularly in production-like conditions and fix what breaks.
• Keep one runbook for drills and real events, and update it after each drill.
• Keep enough headroom or reserved capacity for surviving regions to absorb a failed region’s load; autoscaling can hit quota or capacity limits during a regional outage.
• Plan who is paged, what the status page says, and how customers hear about it.

For a step-by-step regional plan, see keeping your app online during an AWS region outage, and for incident history, the cloud outage tracker.

How Control Plane fits

Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into a GVC, the set of locations a workload runs across, and runs serverless, standard, stateful, and cron workloads on AWS, GCP, and Azure, on Kubernetes clusters you already run, and on your own machines turned into managed locations, including on premises.

One definition, every location. Each location of a GVC serves traffic at once, and the global endpoint sends each request to the nearest healthy one. Disabling a location removes its replicas and routes traffic to the locations that remain. Location routing options include latency offsets that make a location appear closer or farther, and priority routing tiers, where a lower-priority tier receives traffic only when every location in the higher-priority tier is unavailable. Domain route changes apply to every location of a GVC at once. Workload updates roll out behind readiness checks, each location keeps serving the last healthy version until the new one reports ready, and rolling back is an explicit action.

Database templates. Postgres Multi-Location deploys one PostgreSQL cluster managed by Patroni whose members span several locations, with a single primary that takes all writes, asynchronous streaming replicas in the other locations, automatic promotion of a replica in a surviving location when the primary’s location is lost, and an HAProxy tier in every location that routes connections to the current leader, so applications connect to a stable name in their own region. Automatic promotion needs a quorum, so plan for at least three locations. Because replication is asynchronous, the most recent writes can be lost when a location fails. pgEdge deploys active-active PostgreSQL with Spock multi-master replication across the locations you list, with a per-location HAProxy failover tier and PgBouncer pooler. Redis multi-location runs one Sentinel per location, so two locations give no automatic failover and three survive the loss of one; etcd multi-location tolerates one loss with three locations and two with five. A CockroachDB template is also available. Template backups are optional and off by default, restore support varies by template (pgEdge’s backup job also restores, while others document manual procedures), and Control Plane does not make arbitrary databases multi-region automatically.

Proof. During the October 20, 2025 AWS us-east-1 outage, which lasted about 15 hours, Control Plane customers with us-east-1 in their GVC had no downtime, and their workloads were “active and running in healthy locations within 10 seconds,” according to Control Plane internal incident telemetry (details).

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers. The SLA covers workloads; database failover follows the quorum rules above. Control Plane handles Day 2 operations for you, including autoscaling, Capacity AI right-sizing, built-in observability, and security, with no Kubernetes cluster you have to operate, and it ships Terraform and Pulumi providers. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

FAQ

Short answers to common multi-region questions.

How do I run my app across multiple regions?

Deploy stateless services from one definition to every region, put health-checked global routing in front, and pick a replication topology for each database based on its write-latency and data-loss tolerance.

How do I deploy one application to multiple regions with a single configuration?

Make services stateless, externalize configuration and secrets, and use infrastructure as code or a platform with multi-region placement, keeping per-region overrides to the minimum.

How do I route users to the nearest healthy region automatically?

Use DNS latency or geo routing with health checks, an anycast global load balancer, or both. Anycast avoids waiting on DNS caches; DNS is simpler to adopt.

How do I run stateful workloads like databases across multiple regions?

Choose single primary with replicas, multi-primary with conflict handling, distributed SQL with quorum writes, or partitioning by tenant or region, and run consensus-based systems in at least three locations, or two plus a witness in a third.

How do I fail over a database-backed application between cloud regions?

Detect the failure, fence the old primary, promote a replica in a surviving region, repoint applications through a stable endpoint, verify, and rebuild the failed region as a replica.

What happens to writes during a failover with asynchronous replication?

Writes committed on the old primary but not yet replicated are lost; the amount equals the replication lag at the moment of failure.

Why do I need three locations for automatic database failover?

Automatic failover needs a majority. With two locations the survivor cannot form one, while three locations let the remaining two elect a new primary.

What is the difference between active-active and active-passive?

Active-active serves traffic from every region at once, though writes may still go to one primary; active-passive sends all traffic to one region and switches only on failure.

How can my company avoid being taken down by the next major cloud outage?

Run in more than one region or provider, remove single-region dependencies, choose data topologies per service, and rehearse failover until it is routine.

Can Control Plane run my database across multiple regions?

Yes, through templates: Postgres Multi-Location (single primary with automatic promotion), pgEdge (multi-master PostgreSQL), Redis with Sentinel, etcd, and CockroachDB, each spanning the GVC locations you choose.

Talk to the Control Plane team or sign up free to run one definition across every region you need.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Preventing cross-tenant data breaches in multi-tenant SaaS and AI systems requires isolation at every architectural layer and the right tenancy model for each tier.

• Tenancy models. Silo (dedicated stack per tenant), pool (shared infrastructure with tenant ID enforced everywhere), and bridge (shared tiers plus dedicated tiers for regulated or large tenants) each carry different cost, complexity, and isolation trade-offs.
• Isolation layers. Identity and authorization, network segmentation, compute boundaries, storage and databases, encryption keys, AI data (indexes, caches, memory, fine-tuning), logs and observability, and CI/CD each need their own tenant-aware controls.
• Customer-managed keys. BYOK (bring your own key), meaning customer-managed encryption keys held in the customer’s key management service, gives tenants a cryptographic kill switch over their data. Per-tenant data keys wrapped by each tenant’s own customer-managed key through envelope encryption limit blast radius.
• AI-specific risks. Shared retrieval indexes, prompt caches, conversation memory, and fine-tuning datasets can leak data across tenants if the tenant filter is missing or applied after retrieval.
• Evidence for auditors. Automated cross-tenant tests in CI, access reports, penetration tests, and immutable audit trails demonstrate isolation to compliance reviewers.
• How Control Plane fits. Control Plane provides deny-by-default internal firewalls, workload identities, policy-based access control, and a separate Global Virtual Cloud (GVC), Control Plane’s term for a set of locations a workload runs across, or a separate org per tenant, on AWS, GCP, Azure, Kubernetes clusters you already run, or on premises.

What causes cross-tenant data breaches?

Cross-tenant breaches stem from a small set of recurring failure classes.

Missing tenant filter in a query. A database query that omits the WHERE tenant_id = ? predicate returns rows belonging to other tenants.

Broken object-level authorization. When an API endpoint checks authentication but does not verify that the caller may access the specific object requested, one tenant can read or modify another tenant’s resources. OWASP lists this as API1: Broken Object Level Authorization.

Shared caches keyed without tenant ID. A cache entry keyed only by object ID or query hash can serve one tenant’s data to another. Every cache key must include the tenant identifier.

Shared credentials or service accounts across tenants. When several tenants share one service account or database credential, a compromise in one tenant’s context grants access to every tenant’s data.

Overly broad network paths between services. If every service can reach every other service, a vulnerability in one can be used to reach tenant data held by another.

Shared retrieval indexes or vector stores without tenant filtering. In AI systems using retrieval-augmented generation (RAG), a vector search that does not enforce a tenant metadata filter before returning documents can surface another tenant’s content to the model.

Logs and support tooling exposing data across tenants. Centralized logging or support dashboards that do not filter by tenant let operators or attackers view data from tenants they should not see.

Container escape on shared hosts. A kernel exploit or container runtime vulnerability on a shared host can expose the credentials, environment variables, and storage volumes of co-located tenants. Shared GPUs can leak data between co-located jobs when memory is not cleared between them.

Tenant context lost in async paths. Queue messages, background jobs, and event handlers that drop the tenant ID run with no tenant scope or the wrong one.

Server-side request forgery. Tenant-supplied URLs (webhooks, imports, agent browsing) can reach internal services or cloud metadata endpoints.

Noisy neighbor and resource exhaustion. One tenant’s traffic, jobs, or GPU use can starve shared compute, connection pools, or model provider quotas, which degrades every other tenant.

What cloud solutions offer multi tenant protection against cross tenant data breaches?

Silo, pool, and bridge tenancy models compared

The AWS Well-Architected SaaS Lens describes three tenancy models: silo, pool, and bridge. Each makes a different trade-off between isolation strength, cost, and operational effort.

Capabilities to look for in a platform

Whatever model you choose, the underlying platform should provide:

• Deny-by-default network policy between services, so workloads cannot communicate unless explicitly permitted.
• Per-service identity, so each service authenticates on its own and no shared static keys exist.
• Fine-grained access policy that can target resources by name, tag, or query, and that applies automatically to resources created later.
• Separate environments per tenant, with the option to place a tenant’s workloads in a dedicated account or region.
• A dedicated infrastructure option for tenants that need physical separation, including in a tenant’s own cloud account or on premises.
• Encryption with customer-managed keys, so tenants keep control over their data’s cryptographic lifecycle.
• A tamper-proof audit trail of platform actions, queryable for compliance evidence.

On Control Plane, deny-by-default firewalls, workload identities, query-based policies, a separate GVC or org per tenant, dedicated clusters in a tenant’s own cloud account, and a tamper-proof audit trail are built in; the details are in How Control Plane fits.

How do I design AI infrastructure for cloud compliance using BYOK and tenant data segregation?

Customer-managed keys, BYOK, and HYOK

BYOK (bring your own key) means the tenant creates and manages an encryption key in its own cloud key management service (such as AWS KMS, Azure Key Vault, or Google Cloud KMS) and grants the SaaS provider limited permission to use that key to encrypt and decrypt the tenant’s data. The provider never holds the customer’s key material; it holds plaintext data keys in memory only while it processes that tenant’s data.

HYOK (hold your own key) is a stricter variant in which the key stays in an external key manager or hardware security module the tenant controls, often on premises. The provider calls the tenant’s key manager to wrap or unwrap each data key, which adds latency and availability risk.

Envelope encryption with per-tenant keys

A practical pattern is envelope encryption. Each tenant’s data is encrypted with a unique data encryption key (DEK). That DEK is wrapped by the tenant’s customer-managed key (the key encryption key, or KEK) in the cloud KMS, and the wrapped DEK is stored alongside the ciphertext. To decrypt, the provider calls the tenant’s KMS to unwrap the DEK, then uses the DEK locally.

Bind each wrap and unwrap call to the tenant with a KMS encryption context (additional authenticated data) such as tenant_id. A wrapped DEK then fails to unwrap under any other tenant’s context, and on AWS KMS each CloudTrail entry records the tenant. Rotating the KEK does not require re-encrypting data. The KMS keeps prior key versions for unwrapping, and you can rewrap DEKs under the new version.

Key revocation works as a kill switch for data encrypted under that tenant’s DEKs. It takes effect once cached plaintext DEKs expire, so keep the DEK cache short-lived. Vector embeddings, caches, logs, and copies held by third parties are covered only if they are also encrypted under the tenant’s keys. Revocation breaks reads, writes, backups, search, and AI inference for that tenant, so document the blast radius before launch.

AI-specific data segregation

AI workloads add data surfaces that need tenant isolation:

• Retrieval indexes and vector namespaces. Use separate vector collections or namespaces per tenant when feasible. When a shared index is necessary, apply a mandatory server-side tenant metadata filter on every query. Never let the model choose or infer the tenant ID.
• Prompt and response caches. Key every cache entry by tenant ID, and add user ID when the response depends on the user’s permissions. A shared semantic cache without tenant scoping will serve one tenant’s cached answer to another.
• Conversation memory and logs. Scope conversation history, uploaded files, and agent state to tenant, user, and session. Purge or archive them per the tenant’s retention policy.
• Fine-tuning data. Train per-tenant models or adapters on that tenant’s data only, and route inference to the requesting tenant’s adapter. Pool data across tenants only with explicit contractual consent, because fine-tuned models can reproduce training data. Keep provenance records for every training dataset.
• Document-level permissions. Filter retrieval by the requesting user’s permissions as well as the tenant ID. Without this, a shared index leaks documents across roles inside one tenant.
• Agent tool calls. Tools and connectors an agent invokes must run with credentials scoped to the requesting tenant and user, never a shared service account.
• Model provider data retention. Configure data retention and training opt-out settings with every model provider that processes tenant data, and recheck them periodically.
• Per-tenant region pinning. For data residency, pin a tenant’s data, including AI indexes and caches, to locations in the required jurisdiction.

Prompt injection can try to make the model retrieve or reveal another tenant’s data. Authorization must sit outside the model, enforced by the application or platform before any data reaches the model context. The OWASP Top 10 for LLM Applications catalogs these risks.

What are the best identity and access management solutions for multi tenant environments with provenance in mind?

Evaluate IAM options for a multi-tenant environment against these criteria.

Tenant context in signed tokens. Every request should carry a tenant claim in a signed token (for example, a JWT) that is validated on every API call. Derive the tenant ID from the authentication system, never from user input or URL parameters. Reject requests without a valid tenant claim before business logic runs, and reject a request when a tenant ID in the URL or body differs from the token claim.

Separate identities for each service and environment. Each service and each environment (development, staging, production) should have its own identity. A service account shared across services or environments turns a compromise in one context into access to all of them.

Workload identity federation instead of shared static keys. Cloud providers support workload identity federation, where a workload presents a token from a trusted identity provider and exchanges it for short-lived cloud credentials. That removes long-lived API keys and service account files that could leak.

Least privilege scoped to tenant resources. Scope permissions to the specific tenant’s resources rather than every resource of a type. A database role that can read every tenant’s rows is too broad even if it cannot write.

Provenance. Every action, whether by a human operator or an automated workload, must be attributable to a specific identity in an immutable audit trail. Provenance also covers software artifacts: signed builds and a verifiable build process, as defined by the SLSA framework, show that the code running in production is the code that was reviewed and approved.

Tenant isolation checklist by layer

Each layer below lists a control and a concrete way to verify it.

What cloud compliance checklist should I follow when migrating a regulated application to AI infrastructure?

This checklist is not legal advice. Consult qualified counsel for your specific obligations.

1. Classify data and decide what may reach a model. Identify which data elements are regulated (PHI, personal data, cardholder data) and decide whether they may go to a third-party model or must stay within your own infrastructure.
2. Confirm data residency and region pinning. Verify that processing, storage, and AI inference for regulated data happen in the required jurisdiction, and pin workloads and data stores to those regions.
3. Sign a BAA or DPA with every provider that touches regulated data. HIPAA requires a business associate agreement with each business associate that creates, receives, maintains, or transmits PHI on your behalf, and each business associate must sign one with its subcontractors. GDPR Article 28 requires a contract or other binding legal act, usually called a data processing agreement, with each processor, and each processor must bind its sub-processors to the same obligations. That includes model providers, infrastructure providers, and observability vendors.
4. Set model provider data retention and training opt-outs. Confirm in writing how long the model provider retains prompts and completions and that it does not train on your data.
5. Encrypt with customer-managed keys where required. Use envelope encryption with per-tenant keys, and grant the provider only the key permissions it needs.
6. Isolate tenants per the checklist above. Apply every layer of the tenant isolation checklist, with particular attention to AI data surfaces.
7. Log access to regulated data. Keep immutable logs of who accessed what data, when, and why, including model inference calls that process regulated data. Confirm breach notification timelines (under GDPR, 72 hours to the supervisory authority where feasible).
8. Run a DPIA for AI processing of personal data. GDPR Article 35 requires a data protection impact assessment for high-risk processing; update the subprocessor list and customer notices too.
9. Update the risk assessment and SOC 2 system description. Adding AI components changes the system boundary. Document new risks, controls, and third-party dependencies.
10. Test incident response for AI-specific failures. Simulate a model returning another tenant’s data, a prompt injection attack, and a model provider outage, and verify detection, containment, and notification.
11. Document human review where decisions affect people. If AI outputs influence decisions about individuals (credit, insurance, employment), document the human review process and the criteria for overriding model outputs.

For a deeper walkthrough, see our guide to running regulated apps in the cloud.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant; the customer’s own controls remain the customer’s responsibility.

Resource model and tenant boundaries

An org is Control Plane’s top-level isolation boundary. Secrets, policies, domains, and images are org-scoped. Workloads, identities, and volume sets are scoped to a GVC, and a workload can only use volume sets and identities from its own GVC. Teams use a separate GVC per tenant or per tier for siloed tenants, or a separate org where they want a harder boundary. For Kubernetes-specific pitfalls, see Kubernetes multi-tenancy: 5 things to avoid.

Network isolation

The internal firewall between workloads is deny by default. Each workload’s inbound rule allows nothing, the same GVC, the same org, or a named list of workloads. External inbound and outbound traffic are disabled by default, and outbound rules can allow specific CIDRs or hostnames. All traffic between workloads uses mTLS with a unique client certificate per workload.

Access control and audit

Beyond the built-in superusers and viewers policies, nothing is permitted inside an org until a policy grants it. A policy targets one resource kind and covers every item, named items, or items matched by a query on tags, properties, or relations, so resources created later that match are covered automatically. It binds permissions to users, groups, service accounts, or workload identities. Every resource has an Access Report showing who holds each permission and which policy granted it. Edit on a secret also grants reveal, so grant use when a principal only needs to attach a secret. A tamper-proof audit trail records platform actions from every interface and is queryable for evidence. Workloads can also write their own events, such as tenant data access, to it through an audit context, though it does not collect logs from downstream services.

Workload identity and key management

Attach an identity to a workload and credentials arrive at runtime, stored nowhere in the container. Control Plane’s Universal Cloud Identity (UCI) lets a workload call AWS, GCP, and Azure services, including each cloud’s key management service, with no long-lived cloud keys in your code. Each identity connects to one cloud account per provider. Your application code can therefore call AWS KMS, Azure Key Vault, or Google Cloud KMS at runtime to unwrap per-tenant data encryption keys, with no cloud key stored in the workload.

Compute isolation and secrets

Each workload is isolated with cgroups, namespaces, and restricted syscalls and capabilities, and Control Plane’s Sandboxes, isolated environments for building and testing code and AI agents, run in dedicated Kata Containers microVMs. Org secrets are encrypted with envelope encryption and stored on multiple cloud providers using cloud HSMs, and a workload’s identity needs the reveal permission to read a secret. Control Plane encrypts all volumes by default, and on AWS locations a volume set can use your own AWS KMS key in each region instead, so a tenant’s dedicated volume set can be encrypted with a key you control in your own AWS KMS. Custom volume encryption is currently available only on AWS.

Dedicated infrastructure and data residency

A GVC can run on Control Plane-managed locations in AWS, GCP, and Azure regions, on Kubernetes clusters you already run, or on your own machines, including on premises. A regulated tenant can get dedicated infrastructure, including a Kubernetes cluster in its own cloud account joined as a location, either one it already runs or one created through Control Plane’s Managed Kubernetes, operated the same way as shared tiers. Pin a tenant’s GVC to locations in one country for data residency. For more, see data residency vs data sovereignty.

Endpoint authentication and AI assistants

Control Plane can validate JWTs in a workload’s Envoy sidecar, configured per workload or once for every workload in a GVC, so a request that a rule requires to carry a valid token is rejected before it reaches application code. Claims such as a tenant ID can be passed to the application as request headers. AI assistants act through Control Plane’s MCP server with OAuth sign-in, only in the orgs the user authorizes, and with that user’s permissions; a readonly profile cannot change resources, and every action an assistant takes lands in the same audit trail as actions from the Console, CLI, and Terraform. For more on agent guardrails, see how to let AI agents operate production infrastructure safely.

Availability, operations, and cost

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Control Plane handles Day 2 operations for you, including autoscaling, right-sizing with Control Plane’s Capacity AI, which adjusts reserved CPU and memory based on historical usage, built-in observability, and security, with no Kubernetes cluster you have to operate. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.

Control Plane enforces the boundary at the infrastructure and platform layers. The application must still enforce the tenant ID in its own queries, caches, and AI retrieval logic.

FAQ

Common questions about multi-tenant isolation for SaaS and AI platforms.

What is the difference between silo, pool, and bridge tenancy?

Silo gives each tenant a dedicated stack or cloud account. Pool runs all tenants on shared infrastructure with tenant ID enforced in every layer. Bridge shares some tiers (such as compute and the API gateway) and dedicates others (such as databases or encryption keys) for regulated or large tenants. Many enterprise SaaS platforms use a bridge model: a shared default tier, with dedicated tiers for tenants whose contract or compliance needs require them.

How do I stop one tenant’s data appearing in another tenant’s AI answers?

Enforce a server-side tenant metadata filter on every vector query and retrieval call before documents reach the model context. The tenant ID comes from the authenticated session, never from the model. Scope prompt caches by tenant, and by user where permissions differ; scope conversation memory and uploaded files by tenant, user, and session. Test with automated cross-tenant retrieval queries that must return zero results.

Does a dedicated environment per tenant cost more?

Yes. A silo model duplicates infrastructure per tenant, which raises cost and operational overhead. A bridge model dedicates only the tiers that need it, such as the database or encryption keys, while sharing compute and networking, so the extra cost falls only on tenants that require dedicated resources.

What is BYOK and how does it help with tenant isolation?

BYOK (bring your own key) means the tenant manages an encryption key in its own cloud KMS and grants the SaaS provider limited permission to use it. If the tenant revokes that permission, data encrypted under its keys becomes unreadable once cached data keys expire. BYOK does not replace authorization or access controls, but it gives the tenant a cryptographic kill switch and limits blast radius.

How do I prove tenant isolation to auditors?

Run automated cross-tenant access tests in CI covering reads, writes, deletes, exports, and AI retrieval. Use access reports to show which identities hold which permissions, query the platform’s audit trail for evidence, run periodic penetration tests that attempt lateral movement between tenant workloads, and document the isolation architecture and its limits.

Can prompt injection bypass tenant isolation in AI systems?

Yes, if authorization depends on the model. A malicious prompt can instruct a model to retrieve another tenant’s data or repeat data from its context window. Enforce authorization at the application or platform layer before any data enters the model context. The OWASP Top 10 for LLM Applications documents this and related risks.

Should I use row-level security or separate databases for tenant data?

It depends on the tenant’s risk profile and your compliance requirements. Row-level security in a shared database is cost-efficient for many tenants but needs rigorous configuration, testing, and per-transaction tenant context in connection pools. Separate databases give stronger isolation and suit regulated or high-value tenants. A bridge model lets you use both.

How does workload identity federation reduce cross-tenant risk?

Workload identity federation lets a service prove its identity to a cloud provider and receive short-lived, scoped credentials at runtime instead of storing long-lived API keys or service account files. Without stored keys, there is no static secret whose leak could open many tenants’ cloud resources.

What happens if a tenant revokes its customer-managed encryption key?

Data whose DEKs are wrapped by that key becomes unreadable once cached DEKs expire, and every service that decrypts that tenant’s data fails. Plan for this before offering customer-managed keys, including error handling, tenant notification, and documented recovery procedures.

How does Control Plane enforce deny-by-default networking?

Control Plane blocks workload-to-workload traffic until a rule opens it to the same GVC, the same org, or listed workloads. Inbound and outbound internet traffic also start blocked, and outbound access can be opened to listed CIDRs or hostnames. Traffic between workloads uses mTLS.

Ready to enforce tenant isolation at every layer? Talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

1. Control Plane: best for applications that must stay available in one region or across regions and clouds, with no gateways, tunnels, or route tables to run.
2. Google Cloud networking: best for global applications anchored in Google Cloud.
3. AWS networking: best for large AWS estates spanning many regions and accounts.
4. Azure networking: best for hub-and-spoke topologies in Azure-centric environments.
5. Aviatrix: best for a multicloud transit layer with deep visibility and troubleshooting.
6. Alkira: best for multicloud networking consumed as a service.
7. Cisco Multicloud Defense: best for unified cloud firewall policy across clouds.
8. Equinix Fabric: best for private, high-bandwidth links between clouds, data centers, and partners.
9. Megaport: best for on-demand virtual cross-connects to multiple clouds.
10. Cloudflare Load Balancing: best for traffic steering in front of existing deployments.

What makes multi-region networking flexible?

Seven factors determine flexibility:

• Regions and clouds covered. Can you reach every region you need today and add more without re-architecting, across AWS, GCP, Azure, and on premises?
• Network joins or application routing. Joining VPCs and VNets at layer 3 differs from routing requests at layer 4 or layer 7. Many applications need only the second.
• Overlapping IP ranges. Mergers and multi-cloud growth make CIDR conflicts common. Check how each option handles them.
• Failover and health-checked routing. Automatic failover needs health checks, a routing layer that shifts traffic, and capacity already running in the target location.
• Security model. Compare encryption in transit, identity-based access, segmentation, and firewall defaults.
• Operations staffing. Some options need a network engineering team. Others run the routing, upgrades, and monitoring for you.
• Cost model. Data transfer, gateway hours, attachments, ports, licenses, and staff time all add up.

The ranked list

1. Control Plane

What it is. Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

Best for. Applications that must stay available, whether they run in one region today or across regions and clouds. Teams on a single cloud or region get the same built-in firewall, mTLS, and global endpoint, and add locations later as a configuration change.

Key strengths. A workload deployed to Control Plane’s Global Virtual Cloud (GVC), a named set of locations across one or more clouds, runs active-active in every location of that GVC from one definition. A built-in global endpoint uses DNS geo-routing to send each request to the nearest healthy location. Adding a region or another cloud means adding a location to the GVC.

Every workload has a deny-by-default internal firewall. Internal traffic between workloads is encrypted with mTLS on a built-in service mesh, with certificates Control Plane manages. Workloads in different locations and clouds reach each other without VPC peering or VPN tunnels you manage.

Control Plane’s Cloud Wormhole connects workloads to any TCP endpoint in VPCs and other private networks, including on premises data centers. A Cloud Wormhole agent inside the private network holds a secure, persistent outbound connection to Control Plane, so no inbound ports open to the internet. Access is limited to the IP addresses or domain names and ports listed in the workload’s identity. Version 2 agents run active-active, so running two or more keeps the connection up if one fails. For a step-by-step walkthrough, see cross-cloud private networking.

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be on different providers. During the October 20, 2025 AWS us-east-1 outage, Control Plane customers with us-east-1 in their GVC had workloads active and running in healthy locations within 10 seconds, according to Control Plane’s internal incident telemetry. See how to build active-active high availability across regions and clouds.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Limitations. Control Plane networks the workloads it runs and the private endpoints they reach through Cloud Wormhole. To join whole corporate networks or branch offices at layer 3, pair it with multicloud networking software or a transit service.

Pricing model. You pay for the CPU and memory a workload reserves, with no contracts or minimums.

2. Google Cloud networking

What it is. Google Cloud’s networking includes global VPC networks, Cloud Load Balancing with a single anycast IP, Cross-Cloud Interconnect (dedicated links to AWS, Azure, Oracle Cloud Infrastructure, and Alibaba Cloud), and Network Connectivity Center.

Best for. Global applications anchored in Google Cloud.

Key strengths. A VPC network is a global resource, so regions share one network without peering. Cloud Load Balancing serves health-checked backends in any region behind one IP. Network Connectivity Center connects VPC networks and hybrid spokes (HA VPN, Interconnect, and router appliances for SD-WAN) through a central hub.

Limitations. Cross-Cloud Interconnect requires dedicated ports at locations where Google and the other provider are both present. Partner Cross-Cloud Interconnect for AWS removes the physical setup but covers AWS only.

Pricing model. Per-hour forwarding-rule charges and per-GB data processing for load balancers, interconnect port fees, and data transfer charges.

3. AWS networking

What it is. AWS offers Transit Gateway (a regional hub for VPCs and on premises networks), Cloud WAN (a managed global core network), Global Accelerator (anycast entry to AWS endpoints), Route 53 (DNS with health-checked routing), and AWS Interconnect – multicloud.

Best for. Large AWS estates with many VPCs across regions and accounts.

Key strengths. Transit Gateway connects VPCs and on premises networks in a region. Cloud WAN extends this globally with central policy and segments. Global Accelerator routes users over the AWS backbone to healthy endpoints. Route 53 supports latency-based, geolocation, geoproximity, weighted, and failover routing.

Limitations. Transit Gateway peering between regions uses static routes, and each hop adds attachment and data processing charges. Managed cross-cloud links through AWS Interconnect – multicloud are generally available for Google Cloud and in preview for Azure. Other clouds need Site-to-Site VPN or Direct Connect through a colocation partner.

Pricing model. Per-hour attachment and per-GB processing for Transit Gateway, core network edge and attachment hours for Cloud WAN, hourly and data transfer fees for Global Accelerator, and hosted zone, query, and health check pricing for Route 53.

4. Azure networking

What it is. Azure provides Virtual WAN (a managed hub-and-spoke service), global VNet peering, Azure Front Door (global HTTP load balancing), and Traffic Manager (DNS-based routing).

Best for. Hub-and-spoke topologies in Azure-centric environments with branch connectivity.

Key strengths. Virtual WAN combines VPN, ExpressRoute, and SD-WAN connectivity in managed hubs with automated routing. Global VNet peering connects VNets in any region with no gateways. Front Door adds health-probed global load balancing with an integrated WAF.

Limitations. VNet peering is non-transitive, so spoke-to-spoke traffic needs a hub (Virtual WAN or a network virtual appliance). Connecting to other clouds uses ExpressRoute through a colocation provider or VPN.

Pricing model. Hub and connection unit hours for Virtual WAN, per-GB charges for VNet peering, and a monthly base fee plus request and data transfer pricing for Front Door.

5. Aviatrix

What it is. Aviatrix builds encrypted transit networks across AWS, Azure, Google Cloud, and OCI, with central visibility and security. Gartner tracks this category as multicloud networking software.

Best for. Network teams that want one transit layer with deep operational visibility across clouds.

Key strengths. Gateways in each cloud run in HA pairs and handle encrypted tunnels, BGP routing, segmentation, and NAT for overlapping CIDRs. CoPilot provides topology views, flow analytics, and latency monitoring. Distributed Cloud Firewall adds east-west and egress policy.

Limitations. Gateway instances run in your cloud accounts and add compute cost. The controller and CoPilot are either self-hosted or operated by Aviatrix with Aviatrix PaaS. Designing the transit network takes networking expertise.

Pricing model. Subscription through the AWS or Azure Marketplace or a direct contract, plus the cloud compute cost of gateway instances.

6. Alkira

What it is. Alkira offers multicloud networking as a service, with cloud exchange points (CXPs) that act as virtual hubs in cloud regions.

Best for. Organizations that want multicloud networking without deploying gateway appliances.

Key strengths. Alkira runs the CXPs. It connects on premises sites over IPsec or SD-WAN, inserts third-party firewalls inline, applies NAT for overlapping ranges, and provides intent-based policy and end-to-end visibility.

Limitations. Traffic passes through Alkira’s CXPs, which adds a hop and depends on CXP locations.

Pricing model. Subscription; contact Alkira for pricing.

7. Cisco Multicloud Defense

What it is. Cisco Multicloud Defense provides cloud firewall and security policy across AWS, Azure, Google Cloud, and OCI from a SaaS controller.

Best for. Enterprises with Cisco security investments that need one cloud firewall policy across clouds.

Key strengths. Gateways in each cloud inspect ingress, egress, and east-west traffic, with IDS/IPS, WAF, and FQDN filtering. Policy is managed centrally with tag-based rules, and gateways scale automatically.

Limitations. It is a security and policy layer. Routing between clouds comes from cloud networking or a transit product.

Pricing model. SaaS subscription (contact Cisco for pricing), plus cloud compute costs for gateway instances.

8. Equinix Fabric

What it is. Equinix Fabric is a software-defined interconnection service for private connections between clouds, data centers, networks, and service providers at Equinix facilities.

Best for. Private, high-bandwidth links between multiple clouds, on premises data centers, and partners.

Key strengths. Private connections to AWS, Azure, Google Cloud, and many other providers, with redundant connections for high availability. Fabric Cloud Router adds layer 3 routing between clouds.

Limitations. Requires presence at or connectivity to an Equinix facility. It provides links; application routing, health checks, and failover logic sit elsewhere.

Pricing model. Port fees, bandwidth-based connection charges, and colocation costs.

9. Megaport

What it is. Megaport is a network-as-a-service platform for on-demand virtual cross-connects to cloud providers, data centers, and internet exchanges.

Best for. On-demand virtual cross-connects to several clouds.

Key strengths. Quick provisioning of connections to AWS, Azure, Google Cloud, and others. Megaport Cloud Router routes between clouds, and Megaport Virtual Edge hosts SD-WAN, virtual routers, and virtual firewalls.

Limitations. Megaport provides connectivity. Application routing and failover sit elsewhere.

Pricing model. Port and virtual cross-connect fees based on bandwidth, with term options set at order time.

10. Cloudflare Load Balancing

What it is. Cloudflare Load Balancing steers traffic across origins in any cloud or data center, with health checks, failover, and geographic routing.

Best for. Traffic steering in front of existing deployments without changing the underlying network.

Key strengths. Health-checked failover with geo and latency steering at Cloudflare’s edge, in DNS-only or proxied mode. Integrates with Cloudflare’s CDN, WAF, and DDoS protection, and reaches private origins through Cloudflare Tunnel.

Limitations. Steers traffic at Cloudflare’s edge (DNS, HTTP, and layer 4 through Spectrum). Backend-to-backend traffic between clouds needs separate networking.

Pricing model. Paid add-on. Cost rises with the number of endpoints, health-check regions, and steering options.

Comparison table

Pick by situation

What does multi-region networking cost?

• Data transfer between regions and clouds. Egress applies when traffic leaves a region or provider. Direct Connect, ExpressRoute, and Cloud Interconnect carry lower per-GB rates than internet egress.
• Gateway and attachment hours. Transit Gateway attachments, Cloud WAN core network edges, Virtual WAN hubs, and multicloud gateways charge hourly, plus per-GB processing.
• NAT and load balancer processing. NAT gateways and load balancers charge per GB processed. AWS and Google Cloud also bill traffic between zones in the same region.
• Interconnect ports and cross-connects. Fabrics and dedicated interconnects charge for port capacity and bandwidth.
• Licenses. Multicloud networking software adds subscriptions on top of gateway compute.
• Staff time. Operating transit networks, BGP sessions, firewall rules, and routing incidents takes specialized engineers. A platform that runs these for you moves that cost into its fee.

Also plan for

• Inter-region latency. Distance limits synchronous calls and synchronous database replication between continents.
• Data replication. Networking moves requests. Replicating state is a separate database decision.
• Private DNS across clouds. Use Route 53 Resolver endpoints, Azure DNS Private Resolver, and Cloud DNS forwarding.
• BGP design. Plan ASNs, route table limits, and BFD for fast link failover.
• IPv6 and MTU. Dual-stack support varies by service, and IPsec tunnels lower the effective MTU.

FAQ

Answers to common questions about multi-region networking.

Where can I get the most flexible multi-region networking for my cloud platform?

For applications, Control Plane is the most flexible option. Workloads run active-active in a Global Virtual Cloud that can be a single cloud region or span AWS, GCP, Azure, on premises, and your own clusters. A global endpoint, mTLS, and Cloud Wormhole are built in. To join entire networks across clouds, use multicloud networking software such as Aviatrix or Alkira. Within one cloud, that cloud’s transit service is the most integrated choice. For private physical links, use Equinix Fabric or Megaport.

Is Google Cloud’s network better for multi-region than AWS or Azure?

For private networking inside one cloud, Google Cloud is simpler: a VPC network is global, so regions share one network. Google Cloud Load Balancing, AWS Global Accelerator, and Azure Front Door each offer a single anycast entry point. The difference is the private network: AWS and Azure connect regions with inter-region peering, Transit Gateway or Cloud WAN, or Virtual WAN hubs. Across clouds, you need cross-cloud connectivity either way.

Do I need a transit hub or a service mesh?

A transit hub (Transit Gateway, Virtual WAN, Aviatrix, Alkira) routes between networks at layer 3. A service mesh (Istio, Linkerd) secures and controls service-to-service traffic at layer 7. Join VPCs, VNets, and on premises networks with a transit hub; control traffic between services with a mesh. On Control Plane, every workload runs on a built-in service mesh with mTLS, a deny-by-default internal firewall, and global routing, so you do not deploy or operate a separate mesh.

How do I fail over traffic between regions automatically?

Automatic failover needs health checks, a routing layer that shifts traffic, and capacity already running in the target region. DNS-based failover (Route 53, Cloudflare in DNS-only mode) updates records when health checks fail, and how fast clients move depends on record TTLs and resolver caching. Failover time is roughly the health-check interval times the failure threshold, plus TTL. Anycast load balancers (Google Cloud Load Balancing, AWS Global Accelerator, Azure Front Door) and Cloudflare in proxied mode keep one entry point and stop sending traffic to unhealthy backends, so client DNS caching has no effect. Control Plane’s global endpoint uses DNS geo-routing to the nearest healthy location, and because the workload already runs in every location, there is no standby to start. The target region needs spare capacity and replicated data. See the best multi-cloud failover solutions.

How do I connect AWS and Azure privately?

Use Equinix Fabric or Megaport to cross-connect AWS Direct Connect and Azure ExpressRoute at a shared facility, AWS Interconnect – multicloud to Azure (in preview), or multicloud networking software with encrypted tunnels between gateways in both clouds. For application traffic, Control Plane’s Cloud Wormhole connects workloads on Control Plane to private endpoints in AWS VPCs and Azure VNets with no VPN tunnels or peering to manage. See the cross-cloud private networking guide.

Can I use overlapping IP ranges across clouds?

Yes, with NAT or endpoint-level connectivity. AWS Cloud WAN offers 1:1 NAT on attachments, AWS documents private NAT gateways with Transit Gateway for overlapping VPCs, and Azure Virtual WAN applies NAT rules on its site-to-site VPN gateway. Google Cloud Private NAT does not translate to overlapping subnets, so those need re-addressing or third-party NAT. Aviatrix and Alkira apply NAT in their transit networks. Endpoint-level connectivity, such as Control Plane’s Cloud Wormhole, avoids joining networks at the IP layer: a workload reaches a specific private endpoint through an agent inside that network, so the networks never exchange routes.

How does Control Plane handle multi-region networking?

On Control Plane, multi-region networking is part of the Global Virtual Cloud (GVC). Deploy a workload to a GVC with locations in, for example, AWS us-east-1, GCP europe-west1, and Azure eastus2, and it runs in all three from one definition behind one global endpoint. Traffic between locations stays on Control Plane’s internal mesh. Cloud Wormhole reaches private resources in your VPCs, VNets, and data centers. Adding a location, in the same cloud or a different one, is a change to the GVC.

Which hyperscaler connects to other clouds most directly?

Google Cloud’s Cross-Cloud Interconnect offers dedicated links to AWS, Azure, Oracle Cloud Infrastructure, and Alibaba Cloud, and AWS Interconnect – multicloud is generally available for Google Cloud and in preview for Azure. Azure reaches other clouds through ExpressRoute at colocation facilities or VPN.

To run workloads active-active across regions and clouds with networking built in, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

A successful cloud migration picks the right strategy per application, executes in controlled waves, and uses tooling that matches each workload.

• Pick a strategy per app. The 7 Rs give you seven options, from retire (turn it off) to refactor (rebuild for the cloud). Most portfolios mix several of them.
• Migrate in waves. Start with a pilot, prove the rollback plan, then move workloads in groups organized by dependency and business priority.
• Use the providers ’ tools. Every hyperscaler ships discovery, server migration, and database migration services. Terraform and OpenTofu make landing zones repeatable.
• Watch the hidden costs. Dual running, data egress, refactoring labor, and licensing are where budgets overrun. Retiring unused apps first shrinks every other cost line.
• Containerize once. On Control Plane, a workload is deployed to Control Plane’s Global Virtual Cloud (GVC), a set of locations a workload runs across, so a later provider change is a location change in the GVC plus moving any stateful data.

How do I move an app from on premises to the cloud?

Cloud migration works best as a phased, wave-based process. Skipping discovery and landing-zone design is a common source of cost overruns and outages.

1. Discover and assess. Build an inventory of servers, applications, databases, and their dependencies. Map data gravity (where the largest data sets live and which services read them), licensing entitlements that may not transfer to the cloud, and compliance requirements such as data residency. Use automated discovery tools rather than spreadsheets.
2. Choose a strategy per application. Apply the 7 Rs (next section) to every application in the portfolio.
3. Design the landing zone. Prepare the target environment before any workload moves: account or project structure, network topology (VPCs, subnets, peering), identity and access management, encryption, logging, monitoring, and cost controls such as budgets and tagging.
4. Build a pilot. Migrate one representative workload end to end. Compare performance with on-premises baselines, confirm security controls, test the rollback procedure, and write down what you learn.
5. Migrate in waves. Group the remaining workloads by dependency and business priority. Watch for data integrity issues, latency changes, and authentication failures, and run old and new environments in parallel during each wave. Encrypt data in transit, rotate credentials after cutover, and re-validate compliance controls on the target.
6. Cut over. Lower DNS TTLs days ahead, run the final data synchronization, switch DNS, and keep the rollback plan active for a defined acceptance period. Once writes land on the target, rolling back requires reverse replication to the source, so set it up before cutover.
7. Optimize after the move. Rightsize instances, enable autoscaling, buy reserved capacity once usage is known, and set up cost governance. Decommission legacy systems only after the acceptance period closes.

Pick a data migration method per database. Offline migrations use dump and restore or bulk transfer during a downtime window. Online migrations do an initial load, then change data capture (CDC) or native replication keeps the target in sync until cutover. Validate row counts and checksums before switching writes. For very large datasets, transfer devices such as AWS Snowball, Azure Data Box, or Google Transfer Appliance can beat network transfer.

What are the 7 Rs of cloud migration?

The 7 Rs, described in AWS’s migration strategy guidance, give you a decision model for every application in the portfolio.

Rehosting is the fastest path for applications that must leave a data center on a deadline. Refactoring costs the most engineering time and can pay back through lower operating cost and portability. A typical plan retires what it can, rehosts what must move quickly, and refactors the applications that justify the effort.

How do I migrate from on-premises data centers to the cloud and modernize legacy IT infrastructure?

Migration gets workloads off your hardware. Modernization reshapes them to use cloud capabilities. Most programs do both, in that order or in parallel.

Containerize stateless services first. Stateless web servers, APIs, and microservices are the easiest candidates. Package them as container images, externalize configuration, and deploy them to a managed container platform. Early wins build skills for harder workloads.

Move databases to managed services or replicated clusters. Managed database services handle patching, backups, and failover. For databases that cannot use a managed offering (unusual engines, strict version needs), run replicated clusters and use the provider’s database migration service for the initial load.

Use the strangler-fig pattern for monoliths. Route new features and extracted components through a new service layer while the monolith handles the rest. The monolith shrinks until it can be retired.

Keep some workloads on premises. Ultra-low-latency processing next to physical equipment, data sovereignty requirements no cloud region meets, and hardware with years of depreciation left can stay. A hybrid architecture lets them work with cloud-hosted services. For the cost comparison, see cloud vs on-premises vs hybrid cost.

How do I modernize a VM based application to Kubernetes on multi cloud infrastructure?

Moving from VMs to Kubernetes on several clouds is an engineering project with six steps.

1. Package the app as a container image. Write a Dockerfile or use a buildpack to produce a reproducible image, and store it in a private registry.
2. Externalize config and secrets. Environment variables, ConfigMaps, and a secrets manager replace baked-in configuration files.
3. Separate state from compute. Databases, file stores, and caches run as external services, outside the application containers. This step matters most for portability.
4. Add health checks. Liveness and readiness probes let Kubernetes restart unhealthy containers and stop routing traffic to them. Set CPU and memory requests and limits, log to stdout, and handle SIGTERM for graceful shutdown.
5. Write manifests or Helm charts. Keep deployments, services, and routing in version-controlled YAML.
6. Deploy to one cloud, then add a second. Validate on one cluster first. A second cloud exposes assumptions about DNS, identity, and storage that you then fix.

For VMs that cannot be containerized yet, KubeVirt runs VM workloads inside a Kubernetes cluster, so you get one orchestration layer while you work toward containers.

Running Kubernetes yourself on several clouds means operating a cluster per provider and handling upgrades, networking, and security policy on each one. That operational load grows with every cluster you add.

Best cloud migration services and platforms

The hyperscalers each provide migration services built to bring workloads into their own cloud, and open-source tools cover infrastructure automation and Kubernetes portability.

AWS

• AWS Transform MGN (formerly AWS Application Migration Service) replicates source servers to AWS continuously at the block level and orchestrates test and cutover launches.
• AWS Database Migration Service supports homogeneous and heterogeneous database migrations with continuous replication.
• AWS Transform handles discovery, migration planning, and tracking. It replaced AWS Migration Hub, which closed to new customers in November 2025.

Azure

• Azure Migrate discovers, assesses, and migrates servers, databases, and web apps.
• Azure Database Migration Service runs online and offline migrations to Azure database services.

Google Cloud

• Migration Center provides discovery, assessment, and planning.
• Migrate to Virtual Machines moves VM workloads from on premises, AWS, or Azure into Google Cloud.
• Database Migration Service migrates databases to Cloud SQL and AlloyDB.

Open-source tooling

• Terraform and OpenTofu define landing zones and infrastructure as code, so environments are repeatable across providers.
• Velero backs up and restores Kubernetes resources and persistent volumes, which makes it useful for moving workloads between clusters or clouds.

What are the best cloud migration tools?

The right tool depends on what you are moving and where it is going. Check each provider’s pricing page for current terms.

How do I migrate workloads between clouds?

Four things cause most of the difficulty in cloud-to-cloud migration: proprietary managed services with no direct equivalent on the target, egress charges on large transfers, identity systems that differ (IAM roles on AWS, service accounts on Google Cloud, managed identities on Azure), and networking constructs that do not translate one to one.

These patterns make cloud-to-cloud moves repeatable and reduce lock-in:

• Containers as the deployment unit, so compute is portable.
• Infrastructure as code with provider-agnostic tools such as Terraform, OpenTofu, or Pulumi.
• Portable databases on open-source engines such as PostgreSQL, MySQL, and Redis.
• A platform layer that hides provider differences, so application teams do not learn every cloud’s APIs.

For a walkthrough of one application on all three hyperscalers, see run the same app on AWS, GCP, and Azure. For the architecture that keeps future moves cheap, see how to avoid cloud vendor lock-in.

Which vendors have experience migrating from one cloud platform to another?

Three kinds of vendor do this work: the cloud providers’ own professional services and partner programs, systems integrators with multi-cloud practices, and platforms built for multi-cloud deployment. Ask any of them:

• How do you handle services on the source cloud with no equivalent on the target?
• How do you plan data migration and control egress cost?
• Can you show a completed cloud-to-cloud migration of similar scale?
• Do we end up with portable artifacts (containers, infrastructure as code) or your proprietary tooling?
• What support do we get after cutover?

What are the most cost efficient cloud migration options for enterprises?

Costs fall when you shrink scope first, leave paid facilities quickly, and optimize once usage data exists.

Cost drivers:

• Dual running. You pay for both environments during each wave’s parallel run. Longer waves cost more.
• Data transfer and egress. Moving terabytes out of a cloud or across regions incurs egress charges, and moving data out of a data center needs bandwidth too.
• Refactoring labor. Engineering time is often the largest line item in a modernization-heavy migration.
• Licensing. Some on-premises licenses, especially Windows Server and SQL Server, need new cloud licenses or bring-your-own-license terms. Microsoft restricts bring-your-own-license on AWS and Google Cloud, and Oracle licensing varies by cloud.
• Training. Teams new to the target cloud need ramp-up time, which slows early waves.

Cost levers:

• Rehost first, then optimize. Leaving the data center quickly stops facility and hardware spend. Rightsizing comes after stabilization.
• Migration funding programs. AWS, Microsoft, and Google each run migration incentive programs; terms and eligibility vary, so ask each provider for current details.
• Reserved capacity after stabilization. Commit to one- or three-year terms only once you have production usage data.
• Retire unused apps. Every application you retire before migrating is labor, licensing, and cloud spend you never incur.

Worked example (illustrative numbers only, not real prices). An enterprise migrates 45 applications over six months: 30 rehosted, 12 replatformed, and 3 refactored, after retiring 5 more.

In this example, refactoring three apps costs more than rehosting thirty, which is why refactoring is reserved for the applications that justify it. The ongoing bill afterwards depends on rightsizing. For tactics, see cut your AWS bill 30 percent without re-architecting.

How Control Plane fits

Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into one GVC, and it runs serverless, standard, stateful, and cron workloads on AWS, GCP, and Azure; on your existing Kubernetes clusters, through Bring Your Own Kubernetes (BYOK), Control Plane’s option for connecting a cluster you already run; and on your own servers, including on premises, through Managed Kubernetes (MK8s), Control Plane’s way of turning your own servers into managed locations.

Bring what you have. Workloads run from container images in each org’s free private registry or in Docker Hub, Amazon ECR, Google Artifact Registry, Azure Container Registry, GitHub Container Registry, or any registry you already use, and cpln image build builds and pushes an image. cpln convert converts existing Kubernetes manifests, including workloads, secrets, and persistent volume claims (which become Control Plane volume sets), so teams reuse their YAML instead of rewriting it. cpln stack deploy deploys a Docker Compose project directly. cpln apply and cpln helm manage Control Plane resources from YAML and Helm charts kept in Git, and Control Plane ships Terraform and Pulumi providers.

Migrate once. A workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes each request to the nearest healthy location. For stateless workloads, moving to another provider later means adding its location to the GVC and removing the old one. Stateful workloads also need their data copied or replicated to the new location first.

Skip the cluster work. Control Plane handles Day 2 operations for you, including autoscaling, right-sizing with Capacity AI (Control Plane’s automatic CPU and memory sizing based on historical usage), built-in observability, and security, with no Kubernetes cluster you have to operate. With MK8s, Control Plane operates the Kubernetes layer on servers you join; with BYOK, you keep running the cluster you connected.

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Control Plane does not migrate databases or data for you. Data moves with the providers’ database migration services, replication, or backup and restore.

FAQ

These answers cover the questions teams ask most when planning a cloud migration.

What are the 7 Rs of cloud migration?

Retire, retain, rehost, relocate, replatform, repurchase, and refactor. They are a framework for choosing a migration strategy per application, from decommissioning to a cloud-native redesign, described in AWS’s prescriptive guidance.

How do I move an app from on premises to the cloud?

Inventory the app and its dependencies, choose a strategy from the 7 Rs, prepare a landing zone, migrate a pilot, then move in waves with a tested rollback and a parallel run before the DNS cutover.

How long does a cloud migration take?

It depends on application count, dependencies, and strategy mix. Rehost-heavy programs finish fastest; refactor-heavy programs take longer and leave more portable, cheaper-to-run applications.

What is the cheapest way to migrate to the cloud?

Retire unused applications to shrink scope, rehost the rest to stop data-center costs quickly, and optimize after stabilization. Ask your target provider about its migration incentive programs.

What are the best cloud migration tools?

The providers’ own services: AWS Transform MGN and Database Migration Service; Azure Migrate and Azure Database Migration Service; Google Cloud’s Migration Center, Migrate to Virtual Machines, and Database Migration Service. Add Terraform or OpenTofu for infrastructure as code and Velero for Kubernetes workloads.

Can I migrate from one cloud provider to another?

Yes. Cloud-to-cloud migration follows the same phases, plus work on proprietary managed services, egress costs, identity differences, and networking. Containerized workloads defined in infrastructure as code move most easily.

How do I modernize a VM-based application for Kubernetes?

Package it as a container image, externalize configuration and secrets, separate state from compute, add health probes, write manifests or Helm charts, and deploy to one cluster before adding a second. KubeVirt runs VMs that cannot be containerized yet inside Kubernetes.

Which vendors have experience migrating from one cloud platform to another?

Cloud providers’ professional services and partners, systems integrators with multi-cloud practices, and multi-cloud platforms. Ask each for a completed cloud-to-cloud migration of similar scale and whether you keep portable artifacts afterwards.

How does Control Plane help with cloud migration?

Containerize an application once and deploy it to a GVC spanning AWS, GCP, Azure, or your own servers, so changing providers is mostly a location change in the GVC, plus moving any stateful data. Control Plane converts Kubernetes manifests, deploys Docker Compose projects, and handles Day 2 operations.

Is it worth migrating if I have years left on my hardware?

Not for every workload. Compare facility, staffing, and opportunity costs with the remaining depreciation, and keep workloads on hardware that still pays for itself in a hybrid setup. For the full comparison, see cloud vs on-premises vs hybrid cost.

Talk to the Control Plane team or sign up free to containerize once and run on any cloud.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

One application can call services across AWS, GCP, and Azure once it solves four problems: identity, networking, data gravity and egress, and operations.

• Identity. Your workload needs credentials each provider accepts. Options range from static access keys to workload identity federation to a platform-provided identity on all three clouds.
• Networking. Every cross-cloud call needs a secure path: public endpoints with TLS, private connectivity, or application-level tunnels to specific endpoints.
• Data gravity and egress. Providers charge for data leaving their network. Run compute close to the data it reads most, and send small requests and results across clouds.
• Operations. Each provider has its own SDKs, quotas, failure modes, and billing. Centralize observability, policy, and cost tracking.
• Platforms that help. Infrastructure as code provisions resources, Kubernetes distributions standardize deployment, management platforms govern policy, and secrets brokers issue credentials. Control Plane runs the application across clouds with identity on AWS, GCP, and Azure.

Can an application on one cloud use services from another?

Yes. Common patterns:

• An application on AWS calls a model through Azure OpenAI in Microsoft Foundry because the model it needs is offered there.
• A product analytics pipeline writes to BigQuery on GCP, while the application generating the events runs on AWS.
• A managed PostgreSQL database lives on one provider, and services on other providers read from it.
• An acquisition brings a team running on Azure into an organization standardized on AWS, and both systems call each other’s APIs.

Teams do this for best-of-breed services, model availability, existing data, contracts, and regulation. Every cross-cloud call must solve authentication, traverse a network path, and possibly move data that incurs egress charges.

How do I call AWS, GCP, and Azure services from one application without managing cloud keys?

Use workload identity federation or a platform-provided identity, so each cloud issues short-lived credentials at runtime and no long-lived keys sit in code. Four approaches compare as follows.

Federation removes static secrets, but its setup differs between AWS, GCP, and Azure: each has its own trust policy syntax, token audience rules, and role bindings. Teams running many services across two or three clouds maintain a matrix of trust configurations. For how federation works on each provider, see Control Plane’s guide to workload identity federation.

How should the network path between clouds work?

Public endpoints with TLS and provider firewalls. Most cloud services expose HTTPS endpoints. You restrict access with IP allowlists and identity-based policies. This path suits low-volume, latency-tolerant calls; traffic crosses the public internet, and access control rests on provider-side rules. Many services, including Azure OpenAI and BigQuery, also offer private endpoints (Azure Private Link, Google Private Service Connect), which a workload on another cloud can reach only over private connectivity.

Private connectivity. Site-to-site VPNs, dedicated interconnects such as AWS Direct Connect or Azure ExpressRoute (cross-connected at a colocation facility or through a network partner for cloud-to-cloud use), and cross-cloud products such as Google Cloud Cross-Cloud Interconnect provide private, dedicated paths that bypass the public internet. Dedicated links do not encrypt traffic by default, so add MACsec or IPsec where required. Costs include circuit fees, provisioning lead time for dedicated links, and link maintenance.

Application-level tunnels to specific endpoints. You can tunnel traffic to individual TCP endpoints in a remote VPC or data center, which limits exposure and avoids full network peering. Control Plane’s Cloud Wormhole is one example.

For a comparison of private networking options across AWS, GCP, and Azure, see cross-cloud private networking for VPCs and VNets.

Data gravity and egress: where should each component run?

Cloud providers charge for data leaving their network, with rates that vary by provider, region, and volume tier. Ingress from the internet is usually free. Egress also applies between regions inside one cloud, and private interconnects usually carry lower per-GB rates than internet egress.

Data attracts the compute that reads it. If an application reads large volumes from a database on GCP and sends small inference requests to Azure, run the database-reading workload on GCP and keep the Azure call cross-cloud.

• Run compute close to the data it reads most. A service whose primary data source is BigQuery belongs on GCP.
• Send small requests and results across clouds. A model call sends a prompt and receives a response, both small.
• Keep large datasets in their home cloud. Analyze a large dataset where it lives and send the summary to the other cloud.

Every cross-cloud call also adds network round-trip time, so component placement matters for latency as much as for cost.

Can an application running on AWS use Google BigQuery or Azure OpenAI?

Yes. The application authenticates to each provider through federated identity, calls their public or private endpoints, and keeps heavy data reads on AWS. Example: an e-commerce company runs order processing on AWS, generates product descriptions with Azure OpenAI in Microsoft Foundry, and writes analytics events to BigQuery.

1. Identity for Azure OpenAI. The application gets a short-lived token from AWS STS through IAM outbound identity federation. In Microsoft Entra, the team adds a federated identity credential to a user-assigned managed identity (or app registration) that trusts that issuer, subject, and audience. The application exchanges the token for a Microsoft Entra access token. The managed identity holds the Cognitive Services OpenAI User role on the Azure OpenAI resource, and no API key is stored in the application.
2. Identity for BigQuery. The team creates a GCP Workload Identity Federation pool with an AWS provider for the AWS account. Google verifies the application’s IAM role credentials through a signed GetCallerIdentity request and exchanges them for a short-lived Google credential. The federated principal, or a service account it impersonates, holds a BigQuery role on the dataset. No service account key file is involved.
3. Network path to Azure OpenAI. The team calls the public HTTPS endpoint over TLS, sends from fixed egress IPs on AWS (for example through a NAT gateway), and allowlists them in the Azure resource’s firewall rules. Prompts and responses are a few kilobytes of text.
4. Network path to BigQuery. The application calls the BigQuery API over TLS, with access limited by the IAM grant from step 2.
5. Data flow. The application reads order data from a PostgreSQL database on AWS. Analytics events are batched and compressed before each write to BigQuery. All heavy reads stay on AWS.
6. Failure handling. Each Azure OpenAI call has a timeout sized to the user-facing latency budget and retries with exponential backoff and jitter, honoring Retry-After on rate-limit responses. If retries fail, the application serves a cached description or a template. Events go to a durable queue on AWS, such as Amazon SQS, and flush when BigQuery recovers; writes carry insert IDs so retries do not duplicate rows. The application caches federated credentials until shortly before expiry, so a token-exchange outage does not block every call.
7. Observability. The team instruments the application with OpenTelemetry. Each request produces one trace with client spans for the Azure OpenAI call. Batched BigQuery writes get their own spans, linked to the requests that produced the events. Managed services do not continue your trace, so the application records provider request IDs, latency, and status codes as span attributes, and the team sees every cross-cloud dependency’s latency and errors in one backend.

On Control Plane, steps 1 and 2 become one identity with Azure and GCP access rules, and the workload receives short-lived credentials for both at runtime.

Which platforms let one application use services from multiple clouds?

Five categories of platforms address this. Control Plane covers identity, networking, and deployment in one platform; the others each cover one layer.

For a broader comparison, see the best multi-cloud management platforms in 2026.

What are the challenges of using services from multiple cloud providers in one application?

• Identity sprawl. Each cloud has its own IAM model, role syntax, and trust policy format.
• Network path design. Public endpoints, private interconnects, and tunnels each carry different cost, latency, and security trade-offs.
• Egress cost. Charges for data leaving a provider’s network grow with high-volume traffic.
• Latency. Every cross-cloud call adds round-trip time, which constrains placement for latency-sensitive features.
• Different SDKs and quotas per provider. Client libraries, rate limits, error codes, and retry semantics differ.
• Compliance and data residency. Rules on where data is stored and processed apply to every provider in the path, and audits must trace data flows across all of them.
• Failure handling when one provider ’s service is down. Each cross-cloud call needs timeouts, retries, and a fallback.
• Observability across providers. Traces, metrics, and logs from different clouds must be correlated in one view.
• Cost attribution. Allocating spend to teams, features, or customers is harder across several billing systems.

Which platform unifies identity, networking, and deployment across AWS, GCP, and Azure?

Control Plane covers all three in one platform. Control Plane’s Universal Cloud Identity handles identity, Cloud Wormhole and mTLS handle networking, and Control Plane’s Global Virtual Cloud (GVC) handles deployment.

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises. Workloads run active-active across regions and clouds.

Identity. Control Plane’s Universal Cloud Identity lets a workload call 600+ AWS, GCP, and Azure services with no long-lived cloud keys in your code, from any location, including locations on a different cloud. You connect each cloud account once (an IAM role on AWS, a service-account grant on GCP, a connector function deployed to your Azure subscription), instead of maintaining a separate federation trust per cloud, and workloads receive short-lived, least-privilege credentials at runtime.

Deployment. On Control Plane, a workload deployed to a Global Virtual Cloud (GVC), Control Plane’s named set of locations across one or more clouds, runs in every location of that GVC from one definition. Locations can be regions of one cloud or regions on AWS, GCP, and Azure together, and Control Plane’s built-in global endpoint routes each request to the nearest healthy location. For more, see how to run the same app on AWS, GCP, and Azure.

Networking. Control Plane’s Cloud Wormhole connects workloads to specific TCP endpoints in VPCs, data centers, and other private networks through an agent you install in that network. Every workload has a deny-by-default internal firewall with mTLS between services.

Operations. Control Plane handles Day 2 operations for you, including autoscaling, security, and built-in logs, metrics, and tracing for every workload. On Control Plane-managed locations there is no Kubernetes cluster to operate. The CLI, console, Terraform and Pulumi providers, and the Control Plane MCP server all work through the same Control Plane API, and Control Plane complements the infrastructure as code you already run. Teams can consolidate multiple clouds into one platform incrementally and keep using provider-specific services where they help; see managed services vs portability.

Reliability, compliance, and cost. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. You pay for the CPU and memory a workload reserves, with no contracts or minimums.

FAQ

Short answers for teams combining services from AWS, GCP, and Azure in one application.

Is it safe to use services from different clouds in one app?

Yes, when identity and networking are handled correctly. Use short-lived, least-privilege credentials through workload identity federation or a platform-provided identity, encrypt all cross-cloud traffic with TLS, and apply deny-by-default rules between services. Control Plane provides mTLS between workloads and a deny-by-default internal firewall for every workload.

How do I avoid egress fees when using multiple clouds?

You can reduce them. Run compute close to the data it reads most, send small requests and results across clouds, keep large datasets in their home cloud, batch and compress payloads, and cache frequently used data locally.

Do I need Kubernetes to use services from multiple clouds?

No. Any compute environment can call another cloud’s services through its APIs. Control Plane-managed locations need no cluster of your own, and teams that want Kubernetes can run workloads on their own clusters or on Control Plane’s Managed Kubernetes.

What is workload identity federation?

Workload identity federation lets a workload authenticate to a cloud provider by presenting a token from a trusted identity issuer. The provider verifies the token and issues short-lived credentials. AWS, GCP, and Azure all support it. See Control Plane’s guide to workload identity federation.

Can an application on AWS call Azure OpenAI without storing an API key?

Yes. Microsoft Entra workload identity federation exchanges a trusted token, such as one from AWS IAM outbound identity federation, for a short-lived Microsoft Entra access token for an identity that holds the Cognitive Services OpenAI User role. On Control Plane, Universal Cloud Identity issues that access at runtime.

How do I handle failures when one cloud provider’s service is down?

Give every cross-cloud call a timeout, retries with exponential backoff and jitter, and a fallback. Buffer writes to external services in a durable queue so they flush when the service recovers, and for critical features keep a fallback on a different provider or a degraded mode.

How do I get one view of calls across AWS, GCP, and Azure?

Instrument the application with OpenTelemetry and propagate trace context between your own services. Record provider request IDs, latency, and status codes on the client spans for managed-service calls, and send traces, metrics, and logs to one backend. On Control Plane, workload telemetry is built in and can feed your existing observability stack.

Can I start with one cloud and add another later?

Yes. Most teams start on one cloud and add cross-cloud calls as they adopt services from other providers. Set up identity federation and networking patterns early so each new service follows the same pattern. On Control Plane, you add a location on another cloud to the workload’s GVC, and workloads in that GVC deploy to the new location from their existing definitions.

What is the biggest hidden cost of using services from multiple clouds?

Operational complexity. Egress fees get attention, but the ongoing work of managing different IAM models, SDKs, error handling, billing systems, and compliance requirements across providers can exceed them. Centralizing identity, observability, and policy reduces it.

To run one application across AWS, GCP, and Azure with identity on all three, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Multi-cloud deployment is a three-layer problem, and a typical stack takes one tool from each layer.

• Runtime (runs your app across clouds): 1. Control Plane, 2. Red Hat OpenShift, 3. Qovery, 4. Kubernetes on each cloud (EKS, GKE, AKS).
• Delivery and release (gets your code deployed): 1. Argo CD, 2. Flux, 3. Spinnaker, 4. Harness, 5. GitHub Actions and GitLab CI/CD.
• Infrastructure provisioning (creates the cloud resources): 1. Terraform and OpenTofu, 2. Pulumi, 3. Crossplane, 4. Spacelift.
• Main decision: operate a Kubernetes cluster on every cloud and engineer failover yourself, or let a platform run the app active-active across clouds.

What do you need to deploy one application to AWS, GCP, and Azure?

Three layers must be in place before a request reaches your application on all three clouds.

Provisioning creates the cloud resources your application depends on, such as networks, databases, object storage, IAM roles, and DNS zones. These differ in API, naming, and behavior across AWS, GCP, and Azure, so you need a tool that expresses all of them declaratively.

Build and delivery compiles code, runs tests, builds container images, and triggers the deployment. This layer is largely cloud-agnostic, apart from CI authentication to each cloud (usually OIDC federation) and image pulls, where cross-cloud egress and pull credentials push teams to replicate images into Amazon ECR, Google Artifact Registry, and Azure Container Registry.

Runtime is where the application runs and serves traffic. You can run Kubernetes on each cloud, use each cloud’s serverless container service, or use a platform that runs your workload across clouds from a single definition.

Identity, secrets, observability, and global traffic routing cut across all three layers and differ on every cloud, so any multi-cloud toolchain must either unify them or let you configure them per cloud.

How we evaluated

Each tool was assessed against seven criteria for deploying one application to AWS, GCP, and Azure at the same time.

• Works natively on all three clouds, with documented, maintained integrations for each.
• What you still operate, such as Kubernetes clusters, CI runners, or the tool’s own control plane.
• One definition or per-cloud configuration, since fewer definitions mean less drift.
• Identity and secrets across clouds, for example a workload on GCP calling Amazon S3.
• Observability across clouds, with logs, metrics, and traces in one place.
• Failover between clouds, whether built in, possible with configuration, or left to you.
• Lock-in, through proprietary formats, vendor APIs, or operational dependencies.

Platforms that run your application across clouds

These tools sit at the runtime layer. They run your containers and route traffic on AWS, GCP, and Azure.

Control Plane

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes (MK8s), which runs worker nodes in your own cloud account or on your own servers while Control Plane operates the Kubernetes control plane, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

A workload deployed to Control Plane’s Global Virtual Cloud (GVC), a set of locations across regions and clouds where each workload runs, runs active-active in every location of that GVC from one definition, and a global endpoint routes each request to the nearest healthy location. On Control Plane-managed locations in AWS, GCP, and Azure regions there is no Kubernetes cluster to operate.

Control Plane’s Universal Cloud Identity (UCI) lets workloads call AWS, GCP, and Azure services with no long-lived cloud keys in your code. Control Plane handles Day 2 operations for you, including autoscaling, built-in observability, and security.

The CLI, console, Terraform and Pulumi providers, and the Control Plane MCP server all work through the same Control Plane API, so Control Plane complements the infrastructure as code you already run. CI examples exist for GitHub Actions, GitLab CI, Bitbucket Pipelines, CircleCI, and Google Cloud Build, and the Control Plane Kubernetes Operator lets Argo CD manage Control Plane resources from Git.

During the October 20, 2025 AWS us-east-1 outage, Control Plane customers with us-east-1 in their GVC had workloads active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane’s internal incident telemetry. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. You pay for the CPU and memory a workload reserves, with no contracts or minimums.

Best for: teams that want to run the same application across AWS, GCP, and Azure, or across regions of one cloud, without operating Kubernetes clusters, with built-in failover, identity, and observability.

Limitations: Control Plane is the runtime. It does not build code or provision cloud infrastructure. Hyperscaler-managed services such as Amazon RDS or Amazon S3 are still provisioned with your IaC, while databases such as PostgreSQL and CockroachDB can run on Control Plane from Control Plane’s Template Catalog, its library of installable templates for databases and other open-source software.

Red Hat OpenShift

Red Hat OpenShift is an enterprise Kubernetes platform with managed offerings on all three major clouds (Red Hat OpenShift Service on AWS, Microsoft Azure Red Hat OpenShift, and Red Hat OpenShift Dedicated on Google Cloud) plus IBM Cloud, and self-managed installations on other infrastructure. It provides a consistent Kubernetes distribution with developer tooling, pipelines, service mesh, and security policies across clusters.

What it handles: a standardized cluster platform, developer tooling, an operator framework, and enterprise support.

What you still operate: an OpenShift cluster per cloud (operated with Red Hat’s site reliability engineers on the managed offerings, or by your team when self-managed), plus networking, identity federation between clouds, cross-cloud traffic routing, and failover.

Best for: enterprises that want a supported, standardized Kubernetes distribution across clouds.

Limitations: Red Hat Advanced Cluster Management covers multicluster management, but cross-cloud traffic routing and failover still need additional tooling.

Qovery

Qovery is an internal developer platform that deploys applications into your own AWS, GCP, Azure, or Scaleway accounts on managed Kubernetes clusters it provisions, or onto existing Kubernetes clusters you connect. It provides self-service environments, preview environments, and deployment automation.

What it handles: environment provisioning, deployment automation, developer self-service, and preview environments.

What you still operate: the cloud accounts, their cloud costs and governance, and any existing clusters you connect.

Best for: teams that want a developer-friendly deployment experience on Kubernetes in their own cloud accounts.

Limitations: each cloud runs its own cluster, so global traffic routing and cross-cloud failover are separate concerns to plan.

Kubernetes on each cloud (EKS, GKE, AKS)

Running managed Kubernetes directly on each cloud is the do-it-yourself baseline. Amazon EKS, Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS) each manage the control plane, and their automatic modes (EKS Auto Mode, GKE Autopilot, AKS Automatic) also manage nodes. You still handle networking, ingress, identity, observability, and keeping configurations consistent across clouds.

What it handles: container orchestration, scheduling, and the Kubernetes API on each cloud.

What you still operate: version upgrades, ingress controllers, certificates, identity federation, secrets, observability, and cross-cloud traffic routing. Cluster API, Rancher, GKE fleets, and Azure Arc-enabled Kubernetes create or manage clusters across clouds from one place, but each cluster still runs separately. For tools that help, see the best Kubernetes management platforms.

Best for: teams with deep Kubernetes expertise and platform engineers who want control over every layer.

Limitations: the highest operational burden, common configuration drift between clouds, and failover that needs custom engineering. Each cloud’s serverless container service (Amazon ECS on Fargate, Google Cloud Run, Azure Container Apps) avoids cluster work on that cloud but has its own configuration model.

Delivery and release tools

These tools take code from your repository and deploy it to your runtime.

Argo CD

Argo CD is a declarative GitOps continuous delivery tool for Kubernetes. It reconciles the state in Git with the live state of your clusters and can manage many clusters, which suits teams running Kubernetes on several clouds.

What it handles: reconciliation of Helm charts, Kustomize overlays, and plain manifests from Git, multi-cluster sync through ApplicationSets, health status, and rollback.

What you still operate: the Argo CD installation, its access to target clusters, and the Git repositories that define desired state.

Best for: teams already on Kubernetes across clouds that want pull-based GitOps with drift correction.

Limitations: Kubernetes only. It does not provision infrastructure or build images, canary or blue-green releases need Argo Rollouts, and promotion between environments is usually handled with Git changes or a tool such as Kargo.

Flux

Flux is a GitOps toolkit for Kubernetes that reconciles cluster state from Git, Helm, and OCI artifact sources, with multi-tenancy support.

What it handles: GitOps reconciliation, Helm release management, image automation, and multi-tenant deployments.

What you still operate: Flux controllers in each cluster and their source configuration.

Best for: teams that prefer a lightweight, composable GitOps operator in each cluster.

Limitations: Kubernetes only. The core controllers are driven by the CLI and Kubernetes API, with web UIs available from ecosystem projects, and canary releases need Flagger.

Spinnaker

Spinnaker is an open-source multi-cloud continuous delivery platform, created at Netflix, that deploys to AWS, GCP, Azure, Kubernetes, and other targets with pipeline orchestration and progressive delivery such as canary analysis and blue-green deployments.

What it handles: multi-cloud deployment pipelines, progressive delivery, and approval workflows.

What you still operate: Spinnaker itself, a multi-service system that needs its own infrastructure and engineers to upgrade and maintain it.

Best for: large organizations with complex release processes across clouds and clusters.

Limitations: high operational overhead for smaller teams.

Harness

Harness is a commercial software delivery platform covering CI/CD, Argo CD-based GitOps, feature management, and other delivery modules, with deployments to AWS, GCP, Azure, Kubernetes, and VMs.

What it handles: pipelines, deployment verification, feature flags, and governance policies.

What you still operate: pipeline configuration and integrations, plus the Harness installation itself if you choose the Self-Managed Enterprise Edition instead of SaaS.

Best for: teams that want a commercial, full-featured delivery platform with built-in governance.

Limitations: pipeline definitions are Harness-specific.

GitHub Actions and GitLab CI/CD

GitHub Actions and GitLab CI/CD are CI/CD systems built into their source control platforms. They build images, run tests, and trigger deployments to any target their runners can reach. Jenkins and Azure Pipelines (part of Azure DevOps) fill the same role in many enterprises.

What they handle: building, testing, packaging, and triggering deployments.

What you still operate: workflow files, plus self-hosted runners when you need private network access.

Best for: teams whose code lives on GitHub or GitLab.

Limitations: they provide environments and deployment approvals, but multi-cloud rollout logic must be scripted or handed to a delivery tool or runtime platform.

Infrastructure provisioning tools

These tools create and manage the cloud resources your applications depend on.

Terraform and OpenTofu

Terraform is a widely used declarative provisioning tool, with providers for AWS, GCP, Azure, and many other services. OpenTofu is an open-source fork of Terraform under the Linux Foundation that works as a drop-in replacement for existing Terraform configurations.

What they handle: declarative provisioning with a plan-and-apply workflow, state management, and a large provider ecosystem.

What you still operate: state storage and the pipelines that run plans and applies (or HCP Terraform, which hosts both), and modules for each cloud.

Best for: any team provisioning infrastructure across clouds; this is the default starting point.

Limitations: Kubernetes and Helm providers can apply manifests, but Terraform is not designed for ongoing application rollouts, health checks, or traffic routing.

Pulumi

Pulumi provisions infrastructure with general-purpose languages such as TypeScript, Python, Go, C#, and Java, as well as YAML and HCL, across AWS, GCP, Azure, and Kubernetes, and can use any Terraform or OpenTofu provider and Terraform modules.

What it handles: infrastructure in familiar programming languages, with testing, abstraction, and reuse.

What you still operate: pipelines and shared components, plus state storage if you use a self-managed backend.

Best for: developer-centric teams that want to write infrastructure in a general-purpose programming language.

Limitations: like Terraform, it provisions infrastructure and does not run your application, and state is stored in Pulumi Cloud by default unless you configure a self-managed backend.

Crossplane

Crossplane exposes cloud resources as Kubernetes custom resources and continuously reconciles them, so teams provision a database or bucket with YAML through the Kubernetes API.

What it handles: Kubernetes-native provisioning with continuous reconciliation, and composition of resources into higher-level abstractions.

What you still operate: a Kubernetes cluster running Crossplane and its providers for each cloud, or a hosted control plane from Upbound.

Best for: platform teams building Kubernetes-based internal developer platforms.

Limitations: it requires a Kubernetes cluster and familiarity with operators and custom resources.

Spacelift

Spacelift orchestrates runs for Terraform, OpenTofu, Terragrunt, Pulumi, AWS CloudFormation, Kubernetes, and Ansible, with policy enforcement, drift detection, approvals, and audit trails.

What it handles: governance and collaboration around IaC runs.

What you still operate: stack and policy configuration.

Best for: teams that need governance on top of existing IaC. HCP Terraform, env0, Scalr, and Atlantis cover similar ground.

Limitations: it orchestrates IaC runs and depends on Terraform, OpenTofu, Pulumi, or another supported tool to provision resources.

For a full reference stack across these layers, see how to automate app deployment with IaC, CI/CD, and GitOps.

Comparison table

Which stack fits your team?

Teams already on Kubernetes everywhere

Add Argo CD or Flux for GitOps delivery and Terraform or OpenTofu for infrastructure. This is the most common pattern and gives full control, with all the operational work that comes with it.

Teams that do not want to operate Kubernetes

Use Control Plane as the runtime, provision cloud-managed services with Terraform or Pulumi, and build images with GitHub Actions or GitLab CI/CD. You skip cluster operations and run compute active-active across clouds; for the data tier, Control Plane’s Template Catalog includes multi-location PostgreSQL and CockroachDB templates that replicate data across the locations of a GVC. For consolidating clouds onto one platform, see how to consolidate multiple clouds into one platform.

Enterprises standardizing on a supported Kubernetes distribution

OpenShift gives a consistent platform with enterprise support. You still run a cluster per cloud; pair it with Terraform and Argo CD.

Teams that need failover between clouds

Control Plane fails over between clouds by default because every GVC location serves traffic. With Kubernetes on each cloud, equivalent failover needs your own global load balancing, health checks, and DNS automation. Stateless failover is the easier part; databases need a replication design that tolerates losing a cloud.

Small platform teams

Reduce what you operate. A runtime platform removes cluster operations, a managed IaC runner such as Spacelift or HCP Terraform adds governance, and GitHub Actions handles build and test, so a small team can run a multi-cloud pipeline without dedicated Kubernetes engineers.

How do I run the same app across AWS, GCP, and Azure without rewriting it?

Package the application as a container that reads configuration from environment variables or mounted files, so the same image runs on any cloud. Connect to data services through connection strings injected at deploy time, and keep cloud-specific service calls behind interfaces you can swap per environment.

Build one image in CI, replicate it to a registry each cloud can pull from, and deploy it through one pipeline. On Control Plane, one workload definition deploys that image to every location in a GVC; on Kubernetes, you maintain per-cluster Helm values or Kustomize overlays. For the full walkthrough, see how to run the same app on AWS, GCP, and Azure, and for the trade-offs of provider-specific services, see managed services vs. portability.

FAQ

These answers assume containerized applications.

What are the best tools for deploying applications across AWS, GCP, and Azure?

Terraform or OpenTofu, a CI system with Argo CD or Flux, and a runtime. Control Plane runs one workload definition on all three clouds with no clusters to operate; OpenShift, Qovery, and per-cloud Kubernetes run a cluster on each cloud.

Is Terraform enough to deploy apps across clouds?

No. Terraform provisions networks, databases, and clusters and can apply a Helm release, but it does not keep the application running, roll it out safely, or route traffic between clouds.

Do I need Kubernetes to deploy to AWS, GCP, and Azure?

No. Each cloud has a serverless container service (ECS on Fargate, Cloud Run, Azure Container Apps), but each one has its own configuration. On Control Plane-managed locations, one workload definition runs on all three clouds with no Kubernetes cluster to operate.

What is the difference between multi-cloud deployment and multi-cloud management?

Multi-cloud deployment is running an application on more than one provider. Multi-cloud management is the broader work of governing infrastructure, cost, security, and compliance across those providers.

How do I handle identity and secrets across three clouds?

Use workload identity federation or short-lived credentials instead of stored keys, and sync secrets where each cloud needs them with tools such as External Secrets Operator or Vault. On Control Plane, a Universal Cloud Identity (UCI) identity attached to a workload grants it scoped access to AWS, GCP, and Azure services.

Can I use Argo CD with Control Plane?

Yes. The Control Plane Kubernetes Operator exposes Control Plane resources as Kubernetes custom resources, so Argo CD can manage workloads, GVCs, and identities from Git.

How fast can traffic fail over between clouds?

Seconds when health-checked global load balancing shifts traffic, and minutes when DNS failover waits on health check intervals and TTLs. Data replication between clouds usually sets the real limit.

Does Control Plane replace my CI/CD pipeline?

No. Control Plane is the runtime. Your CI system still builds and tests code, then deploys to Control Plane with the CLI, the Terraform or Pulumi provider, or the Kubernetes Operator.

Can I run stateful workloads across multiple clouds?

Yes, with a replication design that fits your consistency needs. Control Plane runs stateful workloads with persistent storage, and its Postgres Multi-Location template runs a Patroni primary with asynchronous replicas in other GVC locations, while the CockroachDB template survives the loss of a location when deployed to three or more.

What compliance certifications does Control Plane hold?

Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. Your own applications and processes still need their own compliance work.

To deploy one application across AWS, GCP, and Azure from a single definition, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Lock-in becomes a problem when it is accidental, unmeasured, or applied to the wrong layer.

• When lock-in is worth it: A managed service materially reduces staffing, improves reliability, or accelerates delivery, and you have documented the exit cost.
• When it is not: The service is your core compute runtime or your identity plane, where switching costs compound across every workload. Managed Kubernetes is fine when workload definitions stay in standard manifests.
• The portable layer to protect: Application code, container images, deployment pipelines, data formats, and the runtime itself.
• The exit plan: Maintain a service-dependency register, test data exports quarterly, keep infrastructure as code, and budget for egress.
• How Control Plane fits: Control Plane runs workloads on AWS, GCP, Azure, your own Kubernetes clusters through Control Plane’s Bring Your Own Kubernetes (BYOK), and your own machines, from bare metal to on-premises servers, joined as locations with Control Plane Managed Kubernetes (MK8s), all mixed in one environment. Universal Cloud Identity lets those workloads call AWS, GCP, and Azure managed services without long-lived cloud keys in your code. You get managed-service benefits without surrendering runtime portability.

For a deeper look at the full lock-in landscape, see our companion piece on how to avoid cloud vendor lock-in.

When Do Managed Platform Services Increase Vendor Lock-In?

Managed services create lock-in in five ways, and they differ in how hard they are to undo.

Proprietary APIs

When your application code calls a vendor-specific API, every call is a line of migration debt. The more deeply integrated the API, the more refactoring a move requires. This is the most visible form of lock-in and the one engineers think of first.

Data Gravity and Egress Costs

Large datasets are expensive to move. Cloud providers charge for data leaving their network, and the sheer time to transfer terabytes or petabytes creates inertia. Transfer time and egress fees grow with data volume, so the cost of leaving rises the longer you stay.

Proprietary Identity and Permissions

IAM systems differ significantly between AWS, GCP, and Azure. Policies, roles, and service accounts written for one cloud do not translate to another. Migrating identity is tedious, error-prone, and security-sensitive.

Proprietary Serverless Runtimes

Functions tied to a specific serverless platform carry implicit dependencies on the platform’s event model, scaling behavior, timeout limits, and deployment tooling. These are difficult to replicate elsewhere.

Contract Terms

Committed-use discounts, enterprise agreements, and reserved capacity create financial lock-in. Breaking a contract early can mean forfeiting discounts or paying penalties.

Data gravity and proprietary identity are the hardest to undo. Moving data is a large but bounded transfer project, while identity must be redesigned and re-audited for every workload. Proprietary APIs come next, then serverless runtimes, then contract terms, which have a defined end date.

Decision Framework: When Is the Managed Option Worth It?

Use this table to evaluate each service category.

Which Open-Source Alternatives to Managed Databases Best Support Portability?

When a managed service’s lock-in level is too high for your risk tolerance, open-source alternatives can restore portability. The table below maps common managed services to their portable counterparts.

The right choice depends on your team’s operational capacity and workload requirements, not on a blanket recommendation.

When Is It Worth Accepting Platform Lock-In for Managed AI Services?

AI services are the newest and fastest-moving category of managed cloud offerings. The lock-in calculus here is different because the field changes so quickly that today’s best model may not be tomorrow’s.

Evaluate Five Factors

1. Speed to market. Managed AI APIs remove the work of hosting, scaling, and updating models, so you can ship a first version without building GPU infrastructure. If time-to-market is critical, the lock-in trade-off is often favorable.
2. Model quality. Some proprietary models outperform open alternatives on specific tasks. Benchmark on your data before committing.
3. Data sensitivity. Sending data to a third-party model API may conflict with compliance requirements. On-premises or private inference may be necessary.
4. Cost at scale. API pricing that looks cheap at prototype volume can become prohibitive at production scale. Model the cost curve before you commit.
5. Ability to swap models behind an abstraction. If your application calls models through a thin abstraction layer, switching providers is mostly a configuration change. You still need to re-test prompts and tool calls, re-run evaluations, and re-embed stored vectors if the embedding model changes.

Use managed AI APIs for experimentation and initial deployment, and build an abstraction layer for model calls from day one. For production inference at scale, portable containers running on GPU infrastructure give you the flexibility to move between clouds or neoclouds as pricing and performance shift. On Control Plane, workloads can attach NVIDIA GPUs, and MK8s lets you join GPU servers from other providers as locations. See our guide on running AI inference across multiple GPU clouds.

What Multi-Cloud Infrastructure Patterns Reduce Dependence on Proprietary Managed Services?

Six engineering patterns reduce dependence on proprietary managed services when applied across the stack.

Containerize the Runtime

Package every workload as a standard container image that follows the OCI image spec. An image packages application code and dependencies so any runtime that supports the OCI image spec can run it. Portability still has limits. Build multi-architecture images if you may run on both x86 and Arm nodes, keep configuration in environment variables or secrets rather than the image, and treat kernel features and GPU drivers as host dependencies.

Infrastructure as Code

Define all infrastructure in code with tools such as Terraform, OpenTofu, or Pulumi. Infrastructure as code makes environments reproducible and auditable. On Control Plane, the CLI, console, and Control Plane’s own Terraform and Pulumi providers all work through the same Control Plane API, so your definitions stay consistent.

Open Protocols and Formats

Prefer open standards: OpenTelemetry for observability, S3-compatible APIs for object storage, SQL for relational data, AMQP or NATS for messaging. Open protocols reduce the surface area of proprietary dependency.

Isolate Provider-Specific Code Behind Interfaces

When you do use a managed service, wrap it behind a well-defined interface in your application code. A storage interface, a queue interface, or a model-call interface means you can swap implementations without touching business logic.

Scheduled Data Replication

For critical datasets, replicate data to a second cloud or to your own infrastructure on a schedule. This keeps a current copy outside the primary provider for migration or disaster recovery. Replication adds ongoing egress and storage costs, so limit it to data you cannot rebuild.

Workload Identity Instead of Long-Lived Keys

Long-lived API keys and service account credentials are both a security risk and a portability obstacle. Use workload identity federation to grant short-lived credentials scoped to the current task. On Control Plane, Universal Cloud Identity does this for AWS, GCP, and Azure services.

For a broader look at cloud-native design principles that support portability, see our guide on cloud-native architecture principles, patterns, and building portable systems.

Should I Choose PaaS or SaaS for Vendor Lock-In and Portability?

Choose a PaaS when portability matters. A PaaS runs your application, so you keep the code, images, and data. A SaaS product replaces the application, so only your data can move.

The platform provides the runtime, networking, and operational tooling. If the PaaS uses standard container images and open APIs, your application is portable even if the platform itself is not.

With SaaS, the contract usually says you own your data, but the export format and completeness depend on the vendor. You do not control the application logic, the runtime, or the operational model. Portability depends entirely on the vendor’s data export capabilities and API compatibility. Switching SaaS providers often means rebuilding workflows from scratch.

A PaaS keeps these advantages only if it runs standard containers and does not impose a proprietary build or deployment format. For a detailed comparison of PaaS options, see our guide to the best PaaS providers in 2026.

What Contract Clauses Should I Require for Data Portability and Service Exit?

Before signing a cloud or managed-service agreement, negotiate terms that protect your ability to leave. The following checklist covers the essentials.

This is general guidance, not legal advice. Consult qualified counsel for your specific situation.

• Data export format and timeline. The provider must export your data in a standard, documented format within a defined number of days after termination.
• Exit assistance. The contract should include a transition period during which the provider supports migration activities.
• Egress fee terms. Cap or waive egress fees during the transition period. At minimum, get clarity on per-GB costs.
• Notice periods. Ensure notice periods for termination are reasonable and symmetric. You should not need 12 months’ notice to leave.
• Termination for convenience. You should be able to terminate without cause, subject to reasonable notice and payment for services consumed.
• Price change caps. Limit annual price increases to a defined percentage, or require advance notice that gives you time to evaluate alternatives.
• Data deletion certification. After migration, the provider must certify in writing that your data has been deleted from their systems, including backups, within a specified timeframe.

Maintain a service-dependency register that tracks each provider-specific component, its owner, data location, export procedure, portable alternative, and estimated exit cost. Review it quarterly.

How Control Plane Approaches Portability

Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into what Control Plane calls a Global Virtual Cloud (GVC), a set of locations a workload runs across.

On Control Plane, a workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes each request to the nearest healthy location, so you get failover without paying for idle standby capacity. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations, and those locations can be on different providers.

During the October 2025 AWS us-east-1 outage, which lasted about 15 hours, Control Plane customers with us-east-1 in their GVC had no downtime, and their workloads were “active and running in healthy locations within 10 seconds,” according to Control Plane internal incident telemetry. See designing systems that survive the inevitable.

Control Plane runs serverless, standard, stateful, cron, and VM workloads on AWS, GCP, and Azure, on clusters you already run through Control Plane’s Bring Your Own Kubernetes (BYOK), on your own machines (bare metal or VMs) turned into managed locations with Control Plane Managed Kubernetes (MK8s), and on premises. On Control Plane-provided compute, there is no Kubernetes cluster you have to operate, and Control Plane handles Day 2 operations for you, including autoscaling, Capacity AI (Control Plane’s right-sizing of CPU and memory based on historical usage), built-in observability, and security. You pay for the CPU and memory you reserve, by the millicore and the megabyte, with no contracts or minimums. Egress, storage, observability, dedicated load balancers, and GPUs are billed separately.

On Control Plane, the template catalog deploys open-source data services such as PostgreSQL, MySQL, MariaDB, Cassandra, Kafka, NATS, RabbitMQ, and Redis or Valkey into a GVC, so the portable alternatives above run on the same platform as your workloads.

Control Plane’s Universal Cloud Identity gives workloads short-lived, least-privilege access to 600+ AWS, GCP, and Azure services, with no long-lived cloud keys in your code. The Control Plane MCP server uses the same API as the CLI, console, and Terraform and Pulumi providers, so Control Plane complements the infrastructure as code you already run.

Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.

You use each cloud’s managed services where they earn their place, and your runtime stays free to move. For a walkthrough of deploying the same application across AWS, GCP, and Azure, see how to run the same app on AWS, GCP, and Azure.

FAQ

When do managed platform services increase vendor lock-in?

Managed services increase lock-in when they expose proprietary APIs, store large volumes of data subject to egress fees, impose proprietary identity models, or use non-standard event systems. The more deeply your application integrates with these surfaces, the higher the switching cost.

Which open-source alternatives to managed databases best support portability?

PostgreSQL, MySQL, MariaDB, Apache Cassandra, Redis, and Valkey are widely adopted portable alternatives. ScyllaDB is a source-available option with a DynamoDB-compatible API. For queues and streaming, Apache Kafka, NATS, and RabbitMQ replace proprietary offerings.

When is it worth accepting platform lock-in for managed AI services?

It is worth it when speed to market is critical and you build an abstraction layer over model calls from day one. Avoid deep lock-in for production inference at scale; portable containers on GPU infrastructure give you pricing and performance flexibility.

What multi-cloud infrastructure patterns reduce dependence on proprietary managed services?

Containerize workloads, define infrastructure as code, use open protocols (OpenTelemetry, S3-compatible APIs, SQL), isolate provider-specific code behind interfaces, replicate critical data on a schedule, and use workload identity instead of long-lived keys.

Should I choose PaaS or SaaS for vendor lock-in and portability?

Choose PaaS when portability is a priority, because you keep control of your application code, container images, and data. SaaS portability depends almost entirely on the vendor’s data export capabilities.

What contract clauses should I require for data portability and service exit?

Require data export in standard formats within a defined timeline, exit assistance during a transition period, capped or waived egress fees, reasonable notice periods, termination for convenience, price change caps, and written data deletion certification.

How does Control Plane keep the runtime portable while using managed services?

On Control Plane, workloads (containers or VMs) run on a Global Virtual Cloud (GVC), Control Plane’s set of locations that can span AWS, GCP, Azure, Kubernetes clusters you bring through Control Plane’s Bring Your Own Kubernetes (BYOK), and your own servers on premises. Control Plane’s Universal Cloud Identity lets those workloads use AWS, GCP, and Azure managed services without long-lived cloud keys in your code.

Is multi-cloud the same as cloud-agnostic?

No. Multi-cloud means using more than one cloud provider. Cloud-agnostic means your workloads can move between providers with minimal effort. You can be multi-cloud and still deeply locked in to each provider’s proprietary services. True cloud-agnostic architecture requires portable runtimes, open standards, and abstraction layers.

Does containerization solve cloud portability?

Only partly. Containers make application code and dependencies portable, as long as images are built for the CPU architectures you target. They do not solve portability for databases, storage, identity, networking, or observability, which often remain cloud-specific. You need additional patterns, such as open protocols and interface abstraction, to address those layers.

How often should I test my cloud exit plan?

Test your cloud exit plan on a fixed schedule set by how critical the system is, such as quarterly for revenue-critical workloads, and after any major architecture change. Each test should export real data and deploy a representative workload on an alternative cloud.

Talk to the Control Plane team or sign up free to try it.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

Treat every model as a replaceable dependency behind a routing layer you control.

• What fails: full outages, elevated error rates, rate limits, latency spikes, regional capacity limits, broken streams, and model deprecations.
• Gateway: one OpenAI-compatible endpoint in front of several providers, with routing, fallbacks, retries, budgets, caching, and central logging.
• Fallback chain: the same model in another region first, then an equivalent model from another provider, then a self-hosted open-weight model.
• Self-hosted fallback: an open-weight model on an OpenAI-compatible server such as vLLM, kept warm and validated against your evaluation set.
• Client handling: timeouts, backoff with jitter, circuit breakers, idempotent tool calls, and a policy for broken streams.
• Multi-region: run the gateway, agent services, and their state in at least two regions behind a health-checked global endpoint.
• Testing: block providers in staging, inject 429s and timeouts, and measure fallback time and quality.
• On Control Plane, the LiteLLM template deploys the gateway, the vLLM template serves a self-hosted fallback on GPUs, and agent services run active-active across the locations of Control Plane’s Global Virtual Cloud (GVC).

What actually fails when a model provider has an outage?

• Full API outages. Requests fail with connection errors, HTTP 503, or provider-specific overload codes such as 529.
• Elevated 5xx error rates. The API is up, but a share of requests return 500 or 502, which is harder to detect because some calls still succeed.
• 429 rate limiting and quota exhaustion. The provider throttles your traffic after you exceed token, request, or spend limits, even while it serves other customers.
• Latency spikes and timeouts. Responses slow down until they exceed your client’s timeout, sometimes without a declared incident.
• Regional capacity limits. A model is available in some regions but has no capacity in the one you call.
• Streams that break mid-answer. A streamed response starts and then drops, after partial output has reached the user.
• Deprecations and behavior changes. A model version is retired or updated, which changes output format, quality, or tool-calling behavior.

Each provider’s official status page and incident history is the authoritative record of past and current incidents; check it when you choose providers and set expectations.

How do I make an AI inference API highly available across providers?

Availability across providers depends on a gateway, an ordered fallback chain, portable prompts, and approved data terms for every provider.

Use a gateway

Put a single OpenAI-compatible endpoint between your application and every provider. The application calls one URL with one credential, and the gateway handles provider selection, provider keys, retries, fallbacks, per-key rate limits and budgets, response caching, and logging. Adding a provider or reordering fallbacks becomes a configuration change.

LiteLLM is an open-source gateway that implements this pattern. It exposes one endpoint for 100+ providers, issues virtual keys with per-key and per-team budgets, enforces rate limits, and supports ordered fallbacks with configurable retries and timeouts.

A gateway adds a network hop and some latency. It holds every provider key, so treat it as a security boundary: restrict network access, rotate keys, and audit admin access. Key response caches by tenant and user, and exclude personalized output from them.

Design the fallback chain

Order fallbacks from most to least similar to the primary model:

1. Same model, another region or deployment. The model stays the same if the same version is deployed in that region. Amazon Bedrock cross-region inference uses an inference profile to route requests across regions in a geography, and Azure deployment types include global and data zone deployments that draw on capacity in more than one region; Microsoft notes these do not fail over if the deployment’s primary region is interrupted, so keep a second deployment in another region as the real fallback.
2. Equivalent model, another provider. This survives a full provider outage, with some differences in output.
3. Self-hosted open-weight model. This removes the external dependency, with the largest quality gap for demanding tasks.

Keep prompts portable

Write prompts that do not depend on one provider’s quirks, and use structured outputs and tool schemas that every model in the chain supports. Keep an evaluation set of representative prompts, and require each fallback model to pass it on JSON validity, tool-call correctness, accuracy, tone, and latency before it enters the chain.

Know what changes on failover

Fallback models have different context windows and tokenizers, so a prompt that fits the primary can exceed a fallback’s limit. Provider prompt caches do not carry over, which raises latency and cost after a switch. Safety filters and refusal behavior differ by provider. The secondary provider needs quota and rate-limit tiers provisioned ahead of time for full production load. Failing over to a pricier model can spike spend, so set budgets per tier. Providers update models, so rerun the evaluation set on a schedule.

Check contracts and data residency

Confirm that every fallback provider is allowed for the data you send: data processing terms, retention settings, and regions. Features that handle regulated data may need to fail closed instead of routing to an unapproved provider.

How do I handle provider failures in the client?

Keep client safeguards, but retry in one layer only; if both the client and the gateway retry, attempts multiply during an outage.

• Timeouts on every call. Set an explicit timeout sized to the task so a hung connection cannot stall a user request or an agent step, and set a separate time-to-first-token timeout for streams.
• Retries with exponential backoff and jitter. Retry 5xx errors and timeouts a small number of times within a strict total deadline, with random jitter to avoid synchronized retry storms.
• Retry-After on 429s. Wait the time the provider indicates before retrying. A 429 that reports exhausted quota or spend limits will not clear on retry (OpenAI marks these as insufficient_quota in the error body, and Anthropic’s spend-limit 429s carry no retry-after header); route it to the next tier instead, and do not retry other 4xx errors. See the OpenAI rate limits guide and Anthropic API errors for how each signals rate limiting and overload.
• Circuit breakers. After repeated failures, stop sending traffic to a provider for a cooldown period, route to the next tier, and send a probe request before restoring traffic.
• Idempotent tool calls. Give side-effecting agent actions (emails, writes, payments) idempotency keys so a retry does not repeat them.
• Broken streams. Decide per feature whether to discard partial output and retry, or keep it and mark it incomplete; failing over before a stream starts is simpler.
• Queues for non-interactive work. Put batch and background jobs on a queue so they wait out an outage instead of consuming fallback capacity.

Hedged requests send a second request to another provider when the first exceeds a latency threshold. Hedging raises token cost and needs idempotent tool calls, so reserve it for latency-critical paths.

What should a degraded mode look like?

Prepare degraded mode before an outage:

• A smaller or cheaper fallback model, configured and tested for basic functions.
• Cached answers for repeated, safe-to-reuse questions, with expiry and privacy controls.
• Feature flags that turn off non-essential AI features.
• Queues that accept long tasks and notify users when they finish.
• User messaging, written and approved in advance, that says what is limited and when to expect recovery.
• Logging of which model served each request.
• Fail-closed behavior for compliance-sensitive features.

Should I keep a self-hosted fallback model?

Keep one when you have strict uptime targets, data that cannot go to third-party APIs, or enough volume that predictable cost outweighs the operating effort.

A self-hosted fallback needs GPU capacity, an open-weight model that passes your evaluation set, and an OpenAI-compatible server so the gateway can call it like any other provider. Confirm the server supports tool calling and structured output for the chosen model. vLLM serves open-weight models through an OpenAI-compatible API with continuous batching and efficient memory management.

Large models take time to load into GPU memory, and longer if the weights must download first. A fallback that has to answer within seconds of a primary failure needs at least one warm replica; caching weights on a persistent volume removes the download but not the load time. Size the warm tier for the traffic it must absorb, since smaller GPUs such as T4 and A10G limit model size.

For failover between GPU clouds, see running AI inference across multiple GPU clouds, and for the full serving stack, what infrastructure you need to run an inference layer.

Where should I run AI agents in production so they don’t go down?

Run the gateway, the agent services, and their state (queues, databases, vector stores, caches) in at least two regions, behind a health-checked global endpoint that sends each request to the nearest healthy region.

• Remove hidden single-region dependencies. The gateway’s own database, the secrets store, the vector store, and the queue broker each become a single point of failure if they live in one region.
• Rehearse failures. Disable a region and block a provider in staging on a schedule, and confirm traffic and fallbacks shift as designed.

For region outages, see how to keep your app online when an AWS region goes down and the cloud outage tracker; for operating agents safely, see how to let AI agents operate production infrastructure safely.

How do I test LLM failover?

1. Block a provider. Mark the primary unavailable in the gateway and confirm requests reach the next tier; record how long the switch takes.
2. Inject 429s and timeouts. Confirm retries, backoff, and circuit breakers behave as designed.
3. Break a stream mid-response. Confirm the client applies your partial-output policy.
4. Load-test the secondary provider. Send production volume to confirm its quotas hold.
5. Score the fallback. Run the evaluation set on each fallback model and compare results with the primary; a fallback that returns unusable output does not count.
6. Run a game day. Walk through detection, alerting, fallback, user messaging, and recovery with the team.

Monitor per-provider error rates (5xx, 429, timeouts), per-provider latency percentiles, token spend by provider and tier, and how often each fallback tier serves traffic.

How Control Plane fits

Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into a GVC, the set of locations a workload runs across, and runs serverless, standard, stateful, and cron workloads on AWS, GCP, and Azure, on Kubernetes clusters you already run, and on your own machines turned into managed locations, including on premises.

• Gateway. The LiteLLM template deploys LiteLLM’s OpenAI-compatible gateway (100+ providers, routing with fallbacks) with virtual API keys, per-key and per-team spend tracking and budgets, and rate limiting. It runs a stateless proxy tier backed by a bundled PostgreSQL for keys, spend, teams, and budgets, and a bundled Sentinel-managed Redis for rate-limit counters and the response cache shared across proxy replicas. Run two or more proxy replicas for an always-on tier, and keep Redis enabled whenever there is more than one replica so limits are shared. The template’s PostgreSQL and Redis run inside one GVC, so for a multi-region gateway, deploy a release per region or point it at a multi-location database. Separate per-region releases keep separate virtual keys, budgets, and spend records unless they share that database.
• Self-hosted fallback. The vLLM template serves one Hugging Face open-weight model through vLLM’s OpenAI-compatible API on an NVIDIA GPU, either Control Plane’s built-in A10G or T4 GPUs or your own GPU cluster registered as a Control Plane bring-your-own-Kubernetes (BYOK) location. The API is protected by an API key, and the model is cached on a persistent volume so restarts do not download it again. The template runs in a GVC with one location, so deploy one per region for regional redundancy. An Ollama template is also available, and Control Plane’s pricing page lists NVIDIA GPUs from T4 to B200.
• Multi-location services. A workload deployed to a GVC runs active-active in every location of that GVC from one definition, and a global endpoint routes each request to the nearest healthy location. Disabling a location removes its replicas and routes traffic to the locations that remain.
• Keyless cloud model access. Control Plane’s Universal Cloud Identity (UCI) lets workloads call AWS, GCP, and Azure services such as Amazon Bedrock, Vertex AI, or Azure OpenAI with no long-lived cloud keys in your code.

During the October 20, 2025 AWS us-east-1 outage, which lasted about 15 hours, Control Plane customers with us-east-1 in their GVC had no downtime, and their workloads were “active and running in healthy locations within 10 seconds,” according to Control Plane internal incident telemetry (details). That was a cloud region outage; model provider outages are handled by the gateway and fallback design in this guide.

The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. It covers workloads running on Control Plane, and it does not cover third-party model APIs. Control Plane handles Day 2 operations for you, including autoscaling, Control Plane’s Capacity AI right-sizing, built-in observability, and security, with no Kubernetes cluster you have to operate; Capacity AI does not apply to GPU workloads. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure, which applies to compute and not to model tokens. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

FAQ

Short answers on keeping AI products online through provider outages.

How do I keep an AI product online when my model provider or GPU cloud has an outage?

Route every model call through a gateway with a fallback chain across regions, providers, and a self-hosted model, handle timeouts and rate limits in the client, and run the gateway and agents in at least two regions. For GPU cloud outages, fail over self-hosted inference to another GPU provider.

How do I make an AI inference API highly available across providers?

Put an OpenAI-compatible gateway in front of several providers, configure ordered fallbacks triggered by 5xx errors, timeouts, and 429s, and keep every fallback model validated against your evaluation set.

Where should I run AI agents in production so they don’t go down?

In at least two regions, behind a health-checked global endpoint, with the gateway, queues, databases, and vector stores also spread across regions so none is a single-region dependency.

What is an LLM gateway and do I need one?

A gateway is a proxy that handles routing, fallbacks, retries, provider keys, budgets, caching, and logging behind one endpoint. You need one once you use more than one provider or need fallbacks and central cost tracking.

What order should a fallback chain follow?

Same model in another region or deployment first, then an equivalent model from another provider, then a self-hosted open-weight model.

How should my app handle 429 rate-limit errors?

Wait the time the provider indicates, retry with backoff and jitter, open a circuit breaker after repeated failures, and let the gateway route to the next provider if limits persist.

Should I self-host a fallback LLM?

Yes, if you have strict uptime targets, data that cannot leave your control, or high volume. Serve it with an OpenAI-compatible server such as vLLM and keep at least one replica warm.

How do I test that LLM failover works?

Block the primary provider in staging, inject 429s and timeouts, time the switch, score fallback output with your evaluation set, and run periodic game days.

What should users see during a model outage?

Normal responses if the fallback meets your quality bar; otherwise a clear degraded-mode notice, queued processing for long tasks, and fail-closed behavior for compliance-sensitive features.

Can an LLM gateway itself be a single point of failure?

Yes. Run at least two stateless proxy replicas, place them in more than one region, and make sure the gateway’s database and cache are not confined to a single region.

Talk to the Control Plane team or sign up free to run your gateway and agent services across regions.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

• Architecture. Use a managed control plane (EKS, GKE, or AKS), separate system and workload node pools, an ingress controller or Gateway API implementation behind a cloud load balancer, cert-manager for TLS, and network policies for isolation.
• Upgrades. Read release notes, check for deprecated APIs, upgrade staging first, upgrade the control plane, then roll through node pools one at a time with surge nodes and PodDisruptionBudgets.
• Autoscaling. Layer the Horizontal Pod Autoscaler (HPA) for replica count, the Vertical Pod Autoscaler (VPA) for right-sizing, and a node autoscaler (Cluster Autoscaler or Karpenter) for capacity. KEDA adds event-driven triggers.
• Stateful data. Managed databases are the simplest path. When you need Postgres in Kubernetes, use an operator such as CloudNativePG with replicas across zones, automated failover, and rehearsed backup and restore.
• When not to run it yourself. If your team spends more time operating Kubernetes than shipping product, a platform that handles Day 2 for you is the faster path.

What is a simple Kubernetes cluster architecture for production, including ingress and load balancer traffic flow?

A production cluster has three layers: the control plane, the node pools, and the networking stack that carries traffic from the internet to your pods.

Reference architecture

Managed control plane. Use a managed Kubernetes service so the cloud provider runs etcd, the API server, and control plane upgrades. The three major options are Amazon EKS, Google Kubernetes Engine (GKE), and Azure Kubernetes Service (AKS).

Node pools. Create at least two node pools. A small system node pool runs cluster add-ons (CoreDNS, metrics-server, the ingress controller). A workload node pool runs your application pods. Spread both pools across three availability zones. Use topology spread constraints or pod anti-affinity rules so replicas land in different zones. Set resource requests and limits on every pod so the scheduler can place pods accurately.

Ingress and gateway layer. An ingress controller or a Gateway API implementation sits behind a cloud load balancer and routes external HTTP and HTTPS traffic to Kubernetes Services. Use cert-manager to issue and renew TLS certificates from Let’s Encrypt or an internal CA. Use external-dns to keep DNS records in sync with your routes. Apply network policies to restrict traffic between namespaces.

Secrets management. Store secrets in an external secrets manager (AWS Secrets Manager, Google Secret Manager, Azure Key Vault, or HashiCorp Vault) and sync them into the cluster with the External Secrets Operator or the Secrets Store CSI Driver.

Request path from DNS to pod

1. DNS resolution. The client resolves your domain. The record, kept current by external-dns, points to the cloud load balancer.
2. Cloud load balancer. The load balancer accepts the connection, optionally terminates TLS, and forwards traffic to the ingress controller’s pods, either through node ports or directly to pod IPs (IP target mode on the AWS Load Balancer Controller, container-native load balancing on GKE). The load balancer’s health checks decide which ingress targets receive traffic.
3. Ingress controller or gateway. The controller terminates TLS if the load balancer did not, matches the host and path, and forwards the request to the right Kubernetes Service. Most ingress controllers read the Service’s endpoints and send traffic directly to pod IPs.
4. Service. The Service selects ready pods by label. kube-proxy or a CNI plugin such as Cilium balances connections across pod IPs.
5. Pod. The container handles the request. Readiness probes decide whether a pod is listed as a Service endpoint. Liveness probes restart a container that has stopped responding.

Component table

The community ingress-nginx controller was retired in March 2026 and receives no further releases or security fixes. The Kubernetes project recommends migrating to the Gateway API.

How do I plan zero downtime Kubernetes upgrades and migrations while keeping AI infrastructure compliant and stable?

Kubernetes ships about three minor releases a year, and the project maintains release branches for the three most recent minor versions; the Kubernetes releases page lists the current versions and their end-of-life dates. Each release can remove APIs and change defaults, so a repeatable upgrade process keeps routine upgrades from turning into incidents.

Upgrade runbook

1. Read the release notes. The release notes and the deprecated API migration guide list every removal and behavior change.
2. Check API deprecations. The API server exposes the apiserver_requested_deprecated_apis metric, and managed services such as EKS and GKE flag deprecated API usage before an upgrade. Fix manifests and Helm charts before you upgrade.
3. Respect version skew. Upgrade one minor version at a time; the control plane cannot skip minor versions, and kubelets may lag the API server by up to three minor versions. Upgrade add-ons (CNI, CoreDNS, CSI drivers, kube-proxy) to versions that support the target.
4. Upgrade a staging cluster first. Apply the new version to a non-production cluster that mirrors production. Run integration and smoke tests, and confirm every controller, operator, and custom resource still works.
5. Upgrade the control plane. On a managed service this is a console action, a CLI command, or a Terraform change. Running pods keep serving while the control plane updates.
6. Upgrade node pools one at a time. Add surge nodes on the new version, then cordon and drain old nodes. PodDisruptionBudgets limit how many replicas of an application can be evicted at once. Readiness probes keep traffic off replacement pods until they can serve. Give pods a preStop delay and a termination grace period so endpoint removal finishes before shutdown. Single-replica applications still drop traffic, and a PodDisruptionBudget that allows zero disruptions blocks the drain.
7. Validate after each pool. Check pod health, error rates, and latency before moving to the next pool.

Blue/green cluster upgrades

For large version jumps, or when an in-place rollback is unacceptable, build a new cluster on the target version, deploy workloads to it, shift traffic at the load balancer or DNS layer, and retire the old cluster. This costs more while both clusters run and removes in-place upgrade risk. Stateful data must be replicated or restored into the new cluster, and DNS TTLs set how fast traffic moves.

GPU and AI workload considerations

The NVIDIA driver on the node, the CUDA version in the container image, and the framework version must be compatible. Validate the combination in staging before you roll GPU node pools.

Compliance and change management

Keep a change record for every upgrade: what changed, who approved it, what was tested, and how to roll it back. Store cluster configuration in version control so auditors can see the history of every change.

Which managed Kubernetes solution is best for autoscaling EC2 nodes and automatically scaling pods?

Kubernetes autoscaling works at two layers. Pod autoscaling changes how many pods run or how large they are. Node autoscaling changes the compute underneath them. On AWS, the usual answer is EKS with the HPA for pods and either Karpenter or EKS Auto Mode, which runs Karpenter for you, for EC2 nodes. GKE and AKS offer managed node provisioning of their own.

Pod autoscaling

Horizontal Pod Autoscaler (HPA). The HPA adjusts replica count from CPU, memory, or custom metrics. HPA computes CPU utilization as a percentage of the pod’s request, so accurate requests matter. Stabilization windows prevent flapping during short spikes.

Vertical Pod Autoscaler (VPA). The VPA recommends or sets resource requests from observed usage. Do not let VPA and HPA act on the same CPU or memory metric; running VPA in recommendation mode alongside HPA avoids the conflict.

KEDA. KEDA scales pods from event sources such as queue depth, stream lag, cron schedules, or Prometheus queries. It manages HPA objects for you and can scale a workload to zero.

Node autoscaling

Cluster Autoscaler. The Cluster Autoscaler adds nodes when pods cannot be scheduled and removes underused nodes. It works with node groups and managed node pools on all three major clouds.

Karpenter. Karpenter is an open-source node provisioner first built for AWS. It launches right-sized instances directly from the cloud API instead of scaling fixed node groups, and it consolidates workloads onto fewer or cheaper nodes when demand drops.

GKE Autopilot and node auto-provisioning. GKE Autopilot manages nodes for you and bills general-purpose pods by the resources they request; pods that select specific hardware, such as GPUs or a machine series, are billed by node. On GKE Standard, node auto-provisioning creates and deletes node pools as needed.

AKS node autoprovisioning. AKS node autoprovisioning, built on Karpenter, picks VM sizes from pending pod requests so you don’t predefine a node pool for each VM size.

Autoscaling comparison

On Control Plane, each workload autoscales per location between a minimum and maximum, targeting concurrent requests, requests per second, CPU, memory, or latency depending on workload type, and KEDA is available for event-driven scaling. Capacity AI, Control Plane’s right-sizing of CPU and memory based on historical usage, adjusts resource allocation over time. Capacity AI is not used for GPU workloads or for workloads that autoscale on CPU or on multiple metrics.

Best platforms for running stateful workloads like Postgres in containers

Running Postgres in production needs durable storage, replication with automated failover, rehearsed backups, and a plan for major version upgrades. There are three broad options.

Managed database services

The cloud provider handles replication, failover, backups, and minor-version patching, and provides tooling for major version upgrades that you schedule.

• AWS: Amazon RDS for PostgreSQL and Amazon Aurora PostgreSQL
• GCP: Cloud SQL for PostgreSQL and AlloyDB
• Azure: Azure Database for PostgreSQL

You give up some control and portability in exchange for a lighter operational load.

Kubernetes operators

Operators turn Postgres lifecycle tasks (provisioning, replication, failover, backup, restore) into Kubernetes resources. The established open-source operators are:

• CloudNativePG
• Zalando’s Postgres Operator
• Crunchy Data’s PGO

They run Postgres pods with persistent volumes, configure streaming replication, promote a replica when the primary fails, and archive WAL to object storage for point-in-time recovery.

Platforms with packaged Postgres templates

On Control Plane, the template catalog includes highly available Postgres (a Patroni cluster of at least three members with one leader and streaming replicas) and Postgres Multi-Location (one Patroni cluster stretched across locations with a single primary, streaming replicas in other locations, automatic promotion when a location is lost, and a leader-routing proxy per location). pgEdge and CockroachDB templates are also available. Stateful workloads on Control Plane use volume sets: persistent volumes with a performance class, snapshots, and restore from snapshot.

Key operational considerations

Synchronous vs. asynchronous replication. With synchronous replication, the primary confirms a commit only after a replica has it, so failover to that synchronous replica loses no committed transactions. The cost is added write latency, and writes stall when no synchronous replica is available unless the operator falls back to asynchronous mode, which drops the guarantee. Asynchronous replication is faster and can lose the most recent transactions during failover. Most operators let you choose.

Backups and point-in-time recovery. Take base backups to object storage and archive WAL continuously. Rehearse restores on a schedule.

Storage classes and volume performance. Use SSD-backed storage classes sized for your IOPS and throughput. Block volumes such as Amazon EBS and Persistent Disk are zonal, so a pod cannot follow its volume to another zone; replica failover is what survives a zone loss. Test what happens when a volume detaches, a node dies, or a zone goes down.

Major version upgrades. pg_upgrade needs a short outage. Copy mode scales with data size; link mode takes minutes but leaves no fallback once the new cluster starts. Run ANALYZE afterward. Logical replication to a new cluster shortens the cutover, but it does not carry DDL or sequence values, and tables need primary keys or a replica identity.

Do not autoscale database replicas with HPA. Replication topology and consistency rules make reactive scaling unsafe. Change replica counts deliberately through the operator.

Decision table

Why is Kubernetes considered complex for AI infrastructure teams and what are the biggest operational pain points?

AI and ML teams face every standard Kubernetes challenge plus problems specific to GPUs, large models, and fast-moving frameworks.

1. GPU scheduling and node pools. Kubernetes treats GPUs as extended resources, and a pod gets whole GPUs unless you add sharing such as MIG, time-slicing, or MPS, or allocate devices through Dynamic Resource Allocation. Mitigation: isolate GPU node pools with labels and taints, and use the NVIDIA GPU Operator to manage drivers and the device plugin.
2. Driver and CUDA version management. A mismatch between the node driver, the CUDA version in the image, and the framework causes crashes or silent failures. Mitigation: pin driver versions in node images and test the full stack in CI.
3. Image sizes and pull times. AI images often run to several gigabytes, and pulling one onto a cold node can take minutes. Mitigation: pre-pull or cache images on GPU nodes and trim images with multi-stage builds.
4. Autoscaling GPU nodes with slow cold starts. GPU instances take longer to launch than CPU instances and are often capacity-constrained, so a scale-up can leave pods pending. Mitigation: keep a warm GPU node or reserved capacity for latency-sensitive inference.
5. Multi-tenant isolation. Teams sharing a GPU cluster can starve each other. Mitigation: use namespaces, ResourceQuotas, LimitRanges, and priority classes.
6. Networking and egress costs. Moving large datasets between zones or out of a cloud adds transfer charges. Mitigation: keep data and compute in the same zone and use topology-aware scheduling.
7. Storage for models and datasets. Model checkpoints and datasets need high-throughput storage that many pods can read at once. Mitigation: use shared file storage through CSI drivers (Amazon FSx for Lustre, Filestore, Azure Files) or object storage, and cache hot models on local disks.
8. Skills gap. Most ML engineers are not Kubernetes operators. Mitigation: give them a platform layer that hides cluster details.

On Control Plane, workloads can attach NVIDIA GPUs, and the pricing page lists models from T4 to B200. Control Plane handles scheduling, autoscaling, and rolling updates for those workloads.

Where can I find the best in class container management and networking tools?

The CNCF Landscape is the most complete directory of cloud native projects.

Service mesh. Istio and Linkerd add mutual TLS, traffic management, and service-to-service telemetry.

CNI plugins. Cilium is eBPF-based and covers network policy, observability, and service mesh features. Calico is widely deployed and supports network policy and BGP.

Ingress and Gateway API. The Gateway API is the successor to the Ingress resource, with richer routing, traffic splitting, and role-based configuration. Implementations include Cilium, Istio, Contour, and Envoy Gateway; cloud options include the AWS Load Balancer Controller and GKE Ingress.

Observability. Prometheus for metrics, OpenTelemetry for traces, metrics, and logs, and Grafana with Alertmanager for dashboards and alerting.

Policy engines. OPA Gatekeeper and Kyverno enforce admission policies such as required labels, no privileged containers, and mandatory resource limits.

For a comparison of platforms that manage Kubernetes for you, see the best Kubernetes management platforms roundup.

When not to run Kubernetes yourself, and how Control Plane fits

If your engineers spend more hours on cluster upgrades, node patching, autoscaler tuning, and incident response than on your product, operating Kubernetes is costing you delivery time.

Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into what Control Plane calls a Global Virtual Cloud (GVC), a set of locations a workload runs across. It runs serverless, standard, stateful, and cron workloads on AWS, GCP, and Azure; on existing Kubernetes clusters, through Bring Your Own Kubernetes (BYOK), Control Plane’s option for connecting a cluster you already run; and on your own servers, including on premises, through Managed Kubernetes (MK8s), Control Plane’s way of turning your own servers into managed locations. Many teams run it in one cloud as well.

Control Plane handles Day 2 operations for you, including autoscaling, Capacity AI right-sizing, built-in observability, and security, with no Kubernetes cluster you have to operate. With MK8s, Control Plane operates the Kubernetes layer on the servers you join; with BYOK, you keep running the cluster you connected.

Every workload update is a rolling update: the previous version keeps serving until the new version reports ready. Readiness and liveness probes decide when a replica takes traffic and when it restarts. A workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes each request to the nearest healthy location. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers. For the cost side of that design, see active-active vs. active-passive costs.

Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant. Control Plane ships Terraform and Pulumi providers, so workloads fit the same infrastructure-as-code review process as the rest of your stack, plus an MCP server for AI assistants.

Teams weighing whether to build a platform team at all can read the guide to production-grade infrastructure without a platform team.

FAQ

Short answers to common production Kubernetes questions.

What is a simple Kubernetes cluster architecture for production, including ingress and load balancer traffic flow?

A managed control plane (EKS, GKE, or AKS), worker nodes across three availability zones, and an ingress controller or Gateway API implementation behind a cloud load balancer. Traffic flows from DNS to the load balancer, through the ingress or gateway where TLS usually terminates, to a Kubernetes Service, and then to pods that pass their readiness probes.

How do I plan zero downtime Kubernetes upgrades and migrations while keeping AI infrastructure compliant and stable?

Read the release notes and fix deprecated APIs, upgrade a staging cluster, upgrade the production control plane, then roll node pools one at a time with surge nodes and PodDisruptionBudgets. For AI workloads, validate GPU driver and CUDA compatibility before touching GPU node pools, and keep a change record for every upgrade.

Which managed Kubernetes solution is best for autoscaling EC2 nodes and automatically scaling pods?

On AWS, EKS with the Horizontal Pod Autoscaler for pods and Karpenter, or EKS Auto Mode, which runs Karpenter for you, for EC2 nodes. Karpenter launches right-sized instances directly instead of scaling fixed node groups. Add KEDA if workloads scale on queue depth or event streams.

What are the best platforms for running stateful workloads like Postgres in containers?

Managed database services (RDS, Aurora, Cloud SQL, AlloyDB, Azure Database for PostgreSQL) carry the lightest operational load. Inside Kubernetes, operators such as CloudNativePG, Zalando’s Postgres Operator, and Crunchy Data’s PGO handle replication, failover, and backup. Control Plane’s template catalog includes Patroni-based highly available and multi-location Postgres.

Why is Kubernetes considered complex for AI infrastructure teams?

GPU scheduling, driver and CUDA compatibility, large image pulls, slow GPU node launches, multi-tenant contention, and the gap between ML engineering and cluster operations are the main sources. Many AI teams adopt a platform layer to absorb them.

How does pod autoscaling work with node autoscaling in Kubernetes?

The HPA adds or removes pod replicas based on metrics. When new pods cannot be scheduled, the node autoscaler (Cluster Autoscaler or Karpenter) adds nodes, and it removes nodes that become underused after pods scale down. Configure both layers together.

What are the biggest operational pain points of running Kubernetes in production?

Upgrade churn across Kubernetes versions, node images, and add-ons is the most persistent. Autoscaler tuning, secrets management, network policy, observability, and incident response for stateful workloads follow. Teams without platform engineers often find Day 2 work takes more time than the initial setup.

Can I run production Kubernetes workloads without managing a cluster myself?

Yes. On Control Plane-managed compute in AWS, GCP, and Azure regions there is no cluster to operate, and autoscaling, rolling updates, observability, and security are built in. You can also connect existing clusters with BYOK, which you keep operating, or join your own servers with MK8s, where Control Plane operates the Kubernetes layer.

How do I run Postgres with high availability across multiple cloud regions?

Run a Patroni cluster with a single primary, streaming replicas in other regions, and leader-aware routing in each location, and decide whether replication to the remote regions is synchronous or asynchronous; synchronous commits across regions add a network round trip to every write. Control Plane’s Postgres Multi-Location template packages this design.

Where can I find the best container management and networking tools for Kubernetes?

Start with the CNCF Landscape. For CNI, look at Cilium and Calico; for service mesh, Istio and Linkerd; for observability, Prometheus and OpenTelemetry; for policy, OPA Gatekeeper and Kyverno.

Talk to the Control Plane team or sign up free to run your containers without operating Kubernetes.

Originally published on the Control Plane blog.

Disclosure: Control Plane publishes this guide.

TL;DR

• Where lock-in hides: cloud-specific CNIs, load balancer controllers, IAM bindings, secret managers, monitoring agents, managed databases, and cloud consoles.
• The portable stack: Terraform or OpenTofu for provisioning, Argo CD or Flux for GitOps, Cilium where you can run it, Gateway API for ingress, Rook-Ceph, Longhorn, or application-layer replication for data, External Secrets Operator or HashiCorp Vault for secrets, workload identity federation for cloud access, OpenTelemetry and Prometheus for observability, and PostgreSQL or MySQL you can run anywhere.
• The cost of portability: more clusters and components to operate, cross-cloud egress, and sometimes lowest-common-denominator features. A managed service is worth its lock-in when you document an exit plan.
• How Control Plane fits: Control Plane’s Global Virtual Cloud (GVC), a named set of locations across one or more clouds where each workload runs, deploys one workload definition active-active across AWS, GCP, Azure, your own Kubernetes clusters, and on-premises servers, with networking, identity, and observability built in.

Which parts of Kubernetes cause cloud lock-in?

Each layer has a cloud-specific default, a portable alternative, and a trade-off.

Each cloud-specific default lowers setup effort and raises switching cost. Decide and document where you accept lock-in, and keep an exit plan for each case.

How do I manage Kubernetes across multiple clouds without vendor lock-in?

Manage every cluster from Git and infrastructure as code, run the same open-source component at each layer on every cloud, route traffic globally, and test moves to a second cloud on a schedule. The ten steps below build that setup.

1. Git and a GitOps controller for every cluster. Keep manifests, Helm charts, and Kustomize overlays in Git, with one base per application and an overlay per cloud for StorageClass names, load balancer annotations, and identity bindings. Run Argo CD or Flux so each cluster reconciles from Git. See Argo CD vs Flux.
2. Infrastructure as code for clusters and cloud resources. Define clusters, networks, IAM roles, and DNS zones in Terraform, OpenTofu, or Pulumi, with shared modules and provider-specific parameters. Crossplane manages cloud resources as Kubernetes custom resources.
3. The same networking on every cluster where you can. Run Cilium where the provider allows it: GKE Dataplane V2 and Azure CNI Powered by Cilium are Cilium-based, and EKS supports Cilium in chaining or replacement mode. Write network rules as standard Kubernetes NetworkPolicy so they apply whichever CNI enforces them. For ingress, adopt Gateway API so routing definitions are the same on every cloud; the community ingress-nginx project was retired in March 2026.
4. Secrets from one source. Run External Secrets Operator on every cluster against one backend, such as HashiCorp Vault, so every workload fetches secrets the same way.
5. Workload identity per cloud. Configure workload identity federation so pods get cloud credentials without long-lived keys. Each cloud wires it differently: IRSA uses a service account annotation, EKS Pod Identity uses an association in the EKS API, Workload Identity Federation for GKE can grant IAM roles directly to the Kubernetes service account as a principal, with no annotation, and Microsoft Entra Workload ID on AKS uses a service account annotation plus a pod label, backed by a federated identity credential in Entra ID. Keep these bindings in per-cloud overlays; application code that uses the default cloud SDK credential chain does not change.
6. One observability backend. Instrument workloads with OpenTelemetry and run Collectors on each cluster that ship metrics, logs, and traces to one backend, with Prometheus and Grafana for metrics and dashboards. Telemetry from every cloud lands in one place, with no dependency on CloudWatch, Cloud Monitoring, or Azure Monitor.
7. A global traffic layer. Use external-dns with DNS-based global load balancing, or another global traffic manager, to route users to the nearest healthy cluster, with health checks per cluster and an active-active or active-passive policy per application.
8. A data strategy per service. Stateless services need no cross-cloud data plan beyond the stores they call. For stateful services, run a portable database on Kubernetes with application-layer replication, use a managed database on each cloud with asynchronous replication between them, or keep the service on one cloud and document the lock-in. Weigh cross-cloud transfer cost and latency.
9. Policy as code across clusters. Run OPA Gatekeeper or Kyverno on every cluster with a shared policy library in Git, enforcing Pod Security Standards, resource limits, image provenance, and network policy.
10. Regular portability drills. Deploy a service to the second cloud on a schedule and verify it works, starting with non-critical workloads. Drills expose hidden dependencies on cloud-specific features.

What tools help with cloud portability and avoiding vendor lock-in?

The main open-source tools are Terraform or OpenTofu, Argo CD or Flux, Cilium, Gateway API, External Secrets Operator or Vault, OpenTelemetry and Prometheus, Velero, and a fleet manager such as Rancher.

GKE fleet management can also manage clusters on other clouds, but it is tied to Google Cloud. For a broader comparison, see the best Kubernetes management platforms.

Is multi-cloud a real solution to vendor lock-in?

Partly. Running on two or more clouds reduces dependence on one provider’s availability and pricing, but it removes lock-in only when each layer uses portable components. Workloads that depend on Aurora on AWS and Spanner on GCP are locked in to both providers.

Multi-cloud also adds operational cost: more credentials, networking models, billing systems, and provider quirks. Teams that adopt it without the portable stack often carry the operating complexity of two clouds and the lock-in of each.

The practical approach is selective portability. Run each workload where it fits best, keep its layers portable so it can move, and accept managed-service lock-in only with a documented exit plan. See managed services vs portability and how to avoid cloud vendor lock-in.

What does portability cost?

Budget for these costs when you commit to a portable stack on every cloud.

More clusters to upgrade. Each Kubernetes upgrade is tested and rolled out to every cluster, and CNI, ingress, mesh, and policy components each have their own upgrade cycle. The work grows with the number of clusters.

More components to operate. Running Cilium, a mesh, Vault, External Secrets Operator, OpenTelemetry Collectors, Prometheus, Grafana, and a policy engine on every cluster means monitoring, capacity planning, and incident response for each.

Cross-cloud egress. Providers bill data leaving their network, so cross-cloud database replication, service calls, and log shipping add recurring transfer charges. Keep chatty services and their data in one cloud and track egress spend per service.

Lowest-common-denominator features. Portable alternatives can lack the integration of managed services. A self-managed PostgreSQL cluster needs engineering effort to match a managed database’s automated backups, point-in-time recovery, and read replicas.

When managed services are worth it. A small team may be better served by a managed database despite the dependency. Document the lock-in and the migration path (most managed PostgreSQL and MySQL services support logical or binlog replication to an external target), and revisit the decision as the team grows.

Operating the clusters. The largest recurring labor cost is operating Kubernetes itself on every cloud: upgrades, node pools, autoscaler tuning, and patching. A platform that handles Day 2 operations across clouds removes much of it, which is the approach Control Plane takes.

How Control Plane fits

Control Plane is the unbreakable platform for running production workloads. It runs serverless, standard, stateful, and cron workloads natively on AWS, GCP, and Azure, VM workloads on locations joined through Control Plane’s Managed Kubernetes, and workloads on Kubernetes clusters you already run or your own machines, including on premises, active-active across regions and clouds.

A workload deployed to a GVC runs active-active in every location of that GVC from one definition. Locations can be Control Plane-managed regions on AWS, GCP, and Azure, Kubernetes clusters you already run joined through Control Plane’s Bring Your Own Kubernetes (BYOK), or clusters created through Control Plane’s Managed Kubernetes (MK8s). On Control Plane-managed locations there is no Kubernetes cluster to operate. MK8s runs worker nodes, load balancers, and storage in your AWS account or GCP project, or on servers you provide (documented providers include Generic, AWS, GCP, Hetzner, and Triton), while Control Plane operates the Kubernetes control plane.

Networking. Control Plane’s built-in global endpoint routes each request to the nearest healthy location, and every workload has a deny-by-default internal firewall with mTLS between services. Control Plane runs the service mesh for you, so there is no separate mesh to install or operate.

Identity. Control Plane’s Universal Cloud Identity gives workloads access to 600+ AWS, GCP, and Azure services with no long-lived cloud keys in your code. You connect each cloud account once through an IAM role or service-account grant, and workloads receive short-lived, least-privilege credentials at runtime, in place of the per-cluster federation setup in step 5.

Day 2 operations. Control Plane handles autoscaling, built-in observability, and security for every workload.

Tooling. The CLI, console, Terraform and Pulumi providers, and the Control Plane MCP server all work through the same Control Plane API, so Control Plane complements the infrastructure as code tools you already run.

Availability, cost, and compliance. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations, and those locations can be on different providers. You pay for the CPU and memory a workload reserves, with no contracts or minimums. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.

Portability. Workloads on Control Plane stay standard container images, and you can export Control Plane resources to Terraform with the CLI. Your containers, Git repository, and infrastructure as code remain yours, which keeps the exit path short compared with a cloud-specific managed service.

See how to run the same app on AWS, GCP, and Azure, and for teams reconsidering Kubernetes itself, the best Kubernetes alternatives.

FAQ

Short answers for teams planning multi-cloud Kubernetes.

How do I run Kubernetes on EKS, GKE, and AKS with one set of tools?

Provision all three clusters from one Terraform or OpenTofu codebase. Register them with one Argo CD instance or install Flux on each, and keep one base of manifests with a Kustomize overlay per cloud for StorageClass names, load balancer annotations, and identity bindings. Use Gateway API for ingress, External Secrets Operator for secrets, one OpenTelemetry Collector configuration, one Kyverno or Gatekeeper policy library, and Rancher or Red Hat Advanced Cluster Management for one access and inventory layer. With Control Plane, existing EKS, GKE, and AKS clusters join one GVC through BYOK, and one workload definition deploys to all three.

Is Kubernetes itself a form of lock-in?

No. Kubernetes is an open-source CNCF project with a consistent API across providers. The lock-in risk comes from the cloud-specific services attached to it.

Can I use managed Kubernetes (EKS, GKE, AKS) and still avoid lock-in?

Yes. The API server, etcd, and scheduler are largely interchangeable across EKS, GKE, and AKS. Keep the add-ons you attach portable, using the layer table above.

How do I move a Kubernetes workload from one cloud to another?

Create the target cluster from your infrastructure as code, point your GitOps controller at it with that cloud’s overlay, and move data with Velero file-system backups or database-native replication; native cloud volume snapshots do not transfer between clouds. Run the workload in both places, shift traffic gradually through the global traffic layer, then retire the old deployment.

Do I need a service mesh for multi-cloud Kubernetes?

Not always. Cilium Cluster Mesh provides cross-cluster connectivity and network policy without a full mesh. If you need mTLS and L7 traffic policy between services, Istio or Linkerd provides both.

Which multi-cloud Kubernetes cost do teams most often underestimate?

Cross-cloud egress. Replication, service calls, and log shipping between clouds add transfer charges, so design data placement to minimize cross-cloud traffic and track egress per service.

Does CSI make storage portable?

No. CSI is the interface that EBS, Persistent Disk, and Azure Disk already use, so data stays on each cloud’s disks. Use portable storage such as Rook-Ceph or Longhorn, or replicate at the application layer, and move data between clouds with Velero or database replication.

Can Control Plane replace my multi-cluster management tooling?

For running workloads, yes. Control Plane deploys, networks, secures, and observes workloads across clouds from one API and console. Control Plane-managed locations and MK8s clusters need no control plane work from your team. Clusters joined through BYOK run Control Plane workloads in the same GVC, and your team keeps operating those clusters.

What is the minimum multi-cloud setup to reduce lock-in risk?

Run primary workloads on one cloud and keep a validated environment on a second cloud where you can deploy the same workloads on demand. Keep manifests in GitOps, keep the second cloud’s infrastructure as code current, and run portability drills on a fixed schedule. Active-active can come later.

To run the same workloads across AWS, GCP, Azure, and your own infrastructure without having to operate the clusters yourself, talk to the Control Plane team or sign up free.

Originally published on the Control Plane blog.