Kubernetes Multitenancy and Tenant Isolation

k As organizations adopt Kubernetes for large-scale applications, multitenancy becomes a critical concern. This article explores different approaches to making a Kubernetes cluster multitenant while ensuring proper tenant isolation.

Why Multitenancy in Kubernetes?

Multitenancy allows multiple teams or applications to share a Kubernetes cluster while maintaining:

• Security: Preventing unauthorized access between tenants.
• Resource Efficiency: Maximizing infrastructure usage.
• Operational Simplicity: Managing fewer clusters while maintaining isolation.

Kubernetes does not provide built-in multitenancy features, so implementing isolation requires a combination of namespaces, RBAC, network policies, and resource constraints.

Types of Multitenancy Approaches

1. Logical Isolation (Soft Multitenancy)

Logical isolation is achieved within a shared cluster using namespaces, RBAC, and policies.

a) Namespace-Based Isolation

Each tenant gets a dedicated namespace, restricting workloads to that namespace.

apiVersion: v1
kind: Namespace
metadata:
  name: tenant1

• Use RBAC (Role-Based Access Control) to restrict access:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tenant1
  name: tenant1-role
rules:
  - apiGroups: [""]
    resources: ["pods", "services"]
    verbs: ["create", "delete", "get", "list", "watch"]

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: tenant1
  name: tenant1-binding
subjects:
  - kind: User
    name: tenant1-user
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tenant1-role
  apiGroup: rbac.authorization.k8s.io

b) Network Isolation

By default, all pods in a cluster can communicate with each other. To isolate tenants:

• Use Network Policies to limit communication.
• Example: Deny all ingress and egress traffic.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  namespace: tenant1
  name: deny-all
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress
  ingress: []
  egress: []

c) Resource Quotas & Limits

Prevent tenants from consuming excessive resources.

apiVersion: v1
kind: ResourceQuota
metadata:
  namespace: tenant1
  name: tenant1-quota
spec:
  hard:
    pods: "10"
    requests.cpu: "4"
    requests.memory: "8Gi"
    limits.cpu: "8"
    limits.memory: "16Gi"

d) Pod Security Admission

• Enforce security rules (e.g., no privileged containers, running as non-root).
• Replace deprecated PodSecurityPolicy with Pod Security Admission in Kubernetes 1.23+.

2. Hard Isolation (Strong Multitenancy)

For stronger security, tenants should have dedicated infrastructure or virtual clusters.

a) Dedicated Node Pools (Taints & Tolerations)

• Assign specific nodes to tenants.
• Use taints & tolerations to ensure workloads are scheduled only on designated nodes.

kubectl taint nodes node1 tenant=tenant1:NoSchedule

tolerations:
  - key: "tenant"
    operator: "Equal"
    value: "tenant1"
    effect: "NoSchedule"

b) Virtual Clusters (vCluster)

Loft Labs' vCluster allows running isolated virtual clusters inside a physical cluster.

vcluster create tenant1 --namespace tenant1

Each tenant gets:

• A separate Kubernetes API server.
• Full control over resources.
• Stronger isolation than namespace-based multitenancy.

c) Separate Kubernetes Clusters

For maximum security, each tenant can get its own Kubernetes cluster.

• Managed via Cluster API or Kubernetes Federation.
• Higher cost, but ensures strict tenant isolation.

3. Comparing Multitenancy Approaches

Recommendation:

• Use namespaces + RBAC for lightweight multitenancy.
• Use vClusters for stronger isolation while sharing infrastructure.
• Use dedicated clusters for maximum security.

4. Final Thoughts

Multitenancy in Kubernetes requires a combination of security, access control, and resource isolation. The right approach depends on your needs:

• Small teams: Namespace-based multitenancy is sufficient.
• Enterprise: Virtual clusters or node isolation offer better control.
• Highly regulated environments: Dedicated clusters ensure compliance.

By implementing the right strategy, Kubernetes can support multi-tenant workloads efficiently while maintaining security and performance.