If you’ve only ever run Kubernetes in the public cloud, you’ve probably been spoiled without realizing it. You click a button (or apply a bit of Terraform), and you get a managed control plane, a load balancer, block and object storage, DNS, and certificates all provisioned for you.
Recently, our team worked with a client in the mining sector under a strict requirement: sensitive operational data had to stay on-premises. The platform also needed to operate with controlled external connectivity and remain usable at remote sites with limited or unreliable network access—so managed Kubernetes, cloud load balancers, and public object storage were not viable options. With only a set of virtual machines to start from, we had to build the surrounding infrastructure that a public cloud would normally provide.
Running Kubernetes outside the public cloud means every capability the cloud quietly handles must be designed, deployed, and operated by your team. For this client, that full ownership was the cost of meeting data residency expectations. If you're evaluating or running Kubernetes where data can't leave your walls, this is the guide I wish we'd had going in.
What’s in this article:
- How to architect load balancing and object storage locally to meet strict data residency requirements
- Designing a resilient control plane and persistent storage layer for stateful workloads without relying on managed infrastructure
- Standardizing deployments using GitOps for a self-service, cloud-like developer experience on local hardware
On top of rebuilding managed services ourselves, we had to integrate the cluster with the organization's existing network and firewall, control outbound connectivity to only approved public services, guarantee persistent storage for stateful workloads, and land on a deployment model the operations team could actually maintain.
The Solution Architecture
Here's how we put it together.
Kubernetes Infrastructure
We were given a set of virtual machines to work with, and we used them as three control-plane (master) nodes and three worker nodes, plus a dedicated bastion host for administrative access.
For provisioning, we used Kubespray, an Ansible-based installer. Manually configuring a cluster so that it comes out identical across every node is tedious and error-prone, and it doesn't reproduce reliably. Kubespray (and Ansible underneath it) gave us idempotency, ensuring nodes didn't drift apart. Because the Ansible playbooks act as version-controlled runbooks, adding a worker or rebuilding a failed node became a matter of rerunning a script rather than executing a day's worth of manual steps.
High Availability and Load Balancing
In a managed cloud service, creating a Service of type LoadBalancer provisions an external load balancer. On bare metal, that same request sits in a "pending" state forever because there is no underlying infrastructure to fulfill it.
To bridge this, we reserved a range of IP addresses on the on-premises network and deployed MetalLB to hand those addresses out to LoadBalancer services, effectively rebuilding the cloud's "free" load balancer by hand. For control-plane access, we ran three control-plane nodes and used Kube-VIP to expose a single highly available virtual IP in front of them, so clients connect to one stable endpoint and lose nothing if an individual master goes down.
MetalLB was not as plug-and-play as we hoped. We experimented with both its Layer 2 (ARP) and BGP modes while trying to get traffic flowing correctly. Even after MetalLB was configured and advertising its addresses, the services still weren't reachable. The load-balancer IPs simply weren't listening as expected from outside the cluster. It turned out the traffic wasn't being routed to the ports MetalLB had opened. We had to engage the data center (DC) networking team to open up and route traffic to those IPs and ports. Only after that coordination did MetalLB start serving traffic as intended, a good reminder that on-prem load balancing is as much a networking conversation as a Kubernetes one.
Persistent Storage
For stateful workloads needing Persistent Volumes (PVs) and Persistent Volume Claims (PVCs), we deployed Rook-Ceph. This gave the cluster a shared, distributed, Kubernetes-native storage layer (the on-prem equivalent of managed block/file storage) without depending on any cloud storage service.
This is where we hit our biggest snags.
Firstly, correctly identifying the storage device names on each VM (the disks Ceph should claim as Object Storage Daemons, or OSDs) was time-consuming. Device naming required careful manual mapping before the storage cluster could form cleanly. The second snag was far more complex. While bringing up the Rook-Ceph operator, the OSDs failed to form a healthy cluster, and traffic between nodes seemed to vanish. The root cause turned out to be a VMware bug with the VMXNET3 network adapter. Packets encapsulated in VXLAN—which our CNI (Flannel) uses for pod-to-pod networking—were being generated with invalid checksums. As a result, the receiving node silently dropped them.
Because the drops were silent, nothing in the logs indicated a network issue. The symptoms were that of storage components failing to communicate. To fix it, we disabled TX checksum offloading on the VMs' network interfaces so the checksums would compute correctly:
1sudo ethtool -K ens160 tx-checksum-ip-generic offOnce we applied this on the affected nodes, VXLAN traffic flowed correctly, and the OSDs came up cleanly. After that, Rook-Ceph settled into a reliable, self-managing storage layer. This issue illustrates the reality of running Kubernetes on your own infrastructure: you own the entire stack, right down to NIC-level quirks of the underlying hypervisor.
Core Infrastructure Services
DNS and Domain Management
With the foundational infrastructure in place, we integrated a local DNS server to resolve domains for services hosted on the platform, so internal clients could reach cluster services by name.
Ingress and TLS
We configured Ingress as the single entry point for applications in the cluster, with TLS/SSL terminated at the ingress layer to provide secure HTTPS access to exposed services.
S3-Compatible Object Storage
For applications built to talk to S3, we deployed MinIO as an on-prem, S3-compatible object store. Workloads that expected an S3 API kept working unchanged, while the data stayed inside the on-premises environment.
How We Secured and Managed the Environment
External Connectivity and Firewall Integration
We identified the applications that needed to reach approved cloud or public services and allowlisted only the required external domains through the on-premises firewall. This gave us controlled outbound connectivity without loosening the organization's network security posture.
Container Registry
We stood up a local container registry to host and distribute images internally. This cut our dependency on external registries and gave us controlled, internal image distribution.
GitOps-Based Deployment
For deployment, we implemented Flux CD as our GitOps engine, with applications packaged as Helm releases. This let the operations team keep deployments declarative and version-controlled, with the cluster continuously reconciled to the desired state defined in Git.
The Outcome: Fast Deployments, Secure Data
By strategically building the required capabilities locally, we delivered a platform that satisfied the organization’s strictest data residency and security requirements for copper mine operations—keeping sensitive operational data on-premises—while giving the operations team the seamless, self-service experience they expect.
The infrastructure operations team gained a highly available, self-service environment. With automated load balancing, dynamic persistent storage, and GitOps pipelines fully integrated into their local data center, they achieved strong deployment velocity even at remote sites with unreliable connectivity, while ensuring sensitive workload data never crossed the corporate firewall.
The Verdict: Complete Control Means Total Ownership
Building a production-ready Kubernetes platform on-premises is ultimately an investment in absolute control. Yes, stepping away from managed cloud services means your engineering team takes full ownership of the underlying infrastructure, from distributed storage devices to internal network routing.
However, for organizations bound by strict data residency laws, this architectural approach delivers the best of both worlds. They can secure sensitive workloads entirely behind their own firewall, while still empowering operations teams with declarative deployments, automated load balancing, and dynamic storage. Owning the entire stack is the cost of true data sovereignty, but with the right engineering practices in place, it creates a highly resilient and sustainable platform.
If your enterprise is looking to modernize its infrastructure without giving up data sovereignty, explore our Cloud Services to see how we help you design, build, and support production-ready platforms on any infrastructure.