Kubernetes►Cloud

Kubernetes Multitenancy and Tenant Isolation

k
As organizations adopt Kubernetes for large-scale applications, multitenancy becomes a critical concern. This article explores different approaches to making a Kubernetes cluster multitenant while ensuring proper tenant isolation.

Why Multitenancy in Kubernetes?

Multitenancy allows multiple teams or applications to share a Kubernetes cluster while maintaining:

  • Security: Preventing unauthorized access between tenants.
  • Resource Efficiency: Maximizing infrastructure usage.
  • Operational Simplicity: Managing fewer clusters while maintaining isolation.

Kubernetes does not provide built-in multitenancy features, so implementing isolation requires a combination of namespaces, RBAC, network policies, and resource constraints.

Types of Multitenancy Approaches

1. Logical Isolation (Soft Multitenancy)

Logical isolation is achieved within a shared cluster using namespaces, RBAC, and policies.

a) Namespace-Based Isolation

Each tenant gets a dedicated namespace, restricting workloads to that namespace.

1
2
3
4
apiVersion: v1
kind: Namespace
metadata:
  name: tenant1
  • Use RBAC (Role-Based Access Control) to restrict access:
1
2
3
4
5
6
7
8
9
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: tenant1
  name: tenant1-role
rules:
  - apiGroups: [""]
    resources: ["pods", "services"]
    verbs: ["create", "delete", "get", "list", "watch"]
1
2
3
4
5
6
7
8
9
10
11
12
13
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  namespace: tenant1
  name: tenant1-binding
subjects:
  - kind: User
    name: tenant1-user
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: tenant1-role
  apiGroup: rbac.authorization.k8s.io

b) Network Isolation

By default, all pods in a cluster can communicate with each other. To isolate tenants:

  • Use Network Policies to limit communication.
  • Example: Deny all ingress and egress traffic.
1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  namespace: tenant1
  name: deny-all
spec:
  podSelector: {}
  policyTypes:
    - Ingress
    - Egress
  ingress: []
  egress: []

c) Resource Quotas & Limits

Prevent tenants from consuming excessive resources.

1
2
3
4
5
6
7
8
9
10
11
12
apiVersion: v1
kind: ResourceQuota
metadata:
  namespace: tenant1
  name: tenant1-quota
spec:
  hard:
    pods: "10"
    requests.cpu: "4"
    requests.memory: "8Gi"
    limits.cpu: "8"
    limits.memory: "16Gi"

d) Pod Security Admission

  • Enforce security rules (e.g., no privileged containers, running as non-root).
  • Replace deprecated PodSecurityPolicy with Pod Security Admission in Kubernetes 1.23+.

2. Hard Isolation (Strong Multitenancy)

For stronger security, tenants should have dedicated infrastructure or virtual clusters.

a) Dedicated Node Pools (Taints & Tolerations)

  • Assign specific nodes to tenants.
  • Use taints & tolerations to ensure workloads are scheduled only on designated nodes.
1
kubectl taint nodes node1 tenant=tenant1:NoSchedule
1
2
3
4
5
tolerations:
  - key: "tenant"
    operator: "Equal"
    value: "tenant1"
    effect: "NoSchedule"

b) Virtual Clusters (vCluster)

Loft Labs’ vCluster allows running isolated virtual clusters inside a physical cluster.

1
vcluster create tenant1 --namespace tenant1

Each tenant gets:

  • A separate Kubernetes API server.
  • Full control over resources.
  • Stronger isolation than namespace-based multitenancy.

c) Separate Kubernetes Clusters

For maximum security, each tenant can get its own Kubernetes cluster.

  • Managed via Cluster API or Kubernetes Federation.
  • Higher cost, but ensures strict tenant isolation.

3. Comparing Multitenancy Approaches

Approach Cost Security Isolation Complexity
Namespace-Based Low Medium Simple
Virtual Clusters Medium High Moderate
Dedicated Nodes Medium High Complex
Separate Clusters High Strong Complex

Recommendation:

  • Use namespaces + RBAC for lightweight multitenancy.
  • Use vClusters for stronger isolation while sharing infrastructure.
  • Use dedicated clusters for maximum security.

4. Final Thoughts

Multitenancy in Kubernetes requires a combination of security, access control, and resource isolation. The right approach depends on your needs:

  • Small teams: Namespace-based multitenancy is sufficient.
  • Enterprise: Virtual clusters or node isolation offer better control.
  • Highly regulated environments: Dedicated clusters ensure compliance.

By implementing the right strategy, Kubernetes can support multi-tenant workloads efficiently while maintaining security and performance.