Kubernetes CPU/Memory limits and requests

12/01/2024 - KUBERNETES

Kubernetes limits defines the maximum amount of CPU/memory a container can use. This is the hard limit and it won't go beyond the defined limits. Requests defines the minimum guaranteed amount of CPU/memory a container can use.


Total CPU capacity of a K8S cluster is the maximum amount of CPU cores used by all available nodes within the cluster.


Assume there is a cluster with 3 nodes. The first node has 2 cores, the second node 2 cores too and the third node has 1 core. Based on this, total CPU capacity of a K8S cluster is the maximum 5 CPU cores (2+2+1). If there is a pod that requires 1.7 cores, it won't be scheduled to the third one because its limit is 1 core. Instead, it will be scheduled to the first or second node.


For a good visual explanation, please visit sysdig. For more details, visit Kubernetes (namespace) and Kubernetes (pod/container) docs.


Example


$ minikube start --memory 4000 --cpus=2

- Using the hyperkit driver based on user configuration
- Starting "minikube" primary control-plane node in "minikube" cluster
- Creating hyperkit VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
- Preparing Kubernetes v1.30.0 on Docker 26.0.1 ...

$ kubectl describe node

...
Capacity:
  cpu:                2
  memory:             3912944Ki
Allocatable:
  cpu:                2
  memory:             3912944Ki
...

$ kubectl top node
NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
minikube   295m         14%    1495Mi          39%

CPU(cores) - 295m means 295 millicpu. If 1000m equals to 1 CPU, thus 295m means 29.5% of 1 CPU.
CPU% - The total CPU usage percentage of the node which is 14% in this case.
MEMORY(bytes) - The total memory usage of the node which 1495MB (1.495GB) in this case.
MEMORY% - The total memory usage percentage of the node which is 39% in this case.

Dummy app with settings


apiVersion: apps/v1
kind: Deployment

metadata:
  name: gomax-deployment
  namespace: default
  labels:
    app: gomax

spec:
  replicas: 1
  selector:
    matchLabels:
      app: gomax
  template:
    metadata:
      labels:
        app: gomax
    spec:
      containers:
        - name: golang
          image: me/gomax:latest
          ports:
            - containerPort: 8000
          resources:
            limits:
              cpu: 1000m
              memory: 1000Mi
            requests:
              cpu: 750m
              memory: 750Mi

$ kubectl describe node

...
Non-terminated Pods:  (9 in total)
  Namespace   Name                CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------   ----                ------------  ----------  ---------------  -------------  ---
  default     gomax-deployment    750m (37%)    1 (50%)     750Mi (19%)      1000Mi (26%)   16s
...