Container Runtimes
You need to install a container runtime into each node in the cluster so that Pods can run there. This page outlines what is involved and describes related tasks for setting up nodes.
Kubernetes 1.23 requires that you use a runtime that conforms with the Container Runtime Interface (CRI).
See CRI version support for more information.
This page provides an outline of how to use several common container runtimes with Kubernetes.
Kubernetes releases before v1.24 included a direct integration with Docker Engine, using a component named dockershim. That special direct integration is no longer part of Kubernetes (this removal was announced as part of the v1.20 release). You can read Check whether Dockershim deprecation affects you to understand how this removal might affect you. To learn about migrating from using dockershim, see Migrating from dockershim.
If you are running a version of Kubernetes other than v1.23, check the documentation for that version.
Cgroup drivers
On Linux, control groups are used to constrain resources that are allocated to processes.
A single cgroup manager simplifies the view of what resources are being allocated
and will by default have a more consistent view of the available and in-use resources.
When there are two cgroup managers on a system, you end up with two views of those resources.
In the field, people have reported cases where nodes that are configured to use cgroupfs
for the kubelet and Docker, but systemd
for the rest of the processes, become unstable under
resource pressure.
Changing the settings such that your container runtime and kubelet use systemd
as the cgroup driver
stabilized the system. To configure this for Docker, set native.cgroupdriver=systemd
.
Changing the cgroup driver of a Node that has joined a cluster is a sensitive operation. If the kubelet has created Pods using the semantics of one cgroup driver, changing the container runtime to another cgroup driver can cause errors when trying to re-create the Pod sandbox for such existing Pods. Restarting the kubelet may not solve such errors.
If you have automation that makes it feasible, replace the node with another using the updated configuration, or reinstall it using automation.
Cgroup version 2
Cgroup v2 is the next version of the cgroup Linux API. Differently than cgroup v1, there is a single hierarchy instead of a different one for each controller.
The new version offers several improvements over cgroup v1, some of these improvements are:
- cleaner and easier to use API
- safe sub-tree delegation to containers
- newer features like Pressure Stall Information
Even if the kernel supports a hybrid configuration where some controllers are managed by cgroup v1 and some others by cgroup v2, Kubernetes supports only the same cgroup version to manage all the controllers.
If systemd doesn't use cgroup v2 by default, you can configure the system to use it by adding
systemd.unified_cgroup_hierarchy=1
to the kernel command line.
# This example is for a Linux OS that uses the DNF package manager
# Your system might use a different method for setting the command line
# that the Linux kernel uses.
sudo dnf install -y grubby && \
sudo grubby \
--update-kernel=ALL \
--args="systemd.unified_cgroup_hierarchy=1"
If you change the command line for the kernel, you must reboot the node before your change takes effect.
There should not be any noticeable difference in the user experience when switching to cgroup v2, unless users are accessing the cgroup file system directly, either on the node or from within the containers.
In order to use it, cgroup v2 must be supported by the CRI runtime as well.
Migrating to the systemd
driver in kubeadm managed clusters
If you wish to migrate to the systemd
cgroup driver in existing kubeadm managed clusters,
follow configuring a cgroup driver.
CRI version support
Your container runtime must support at least v1alpha2 of the container runtime interface.
Kubernetes 1.23 defaults to using v1 of the CRI API. If a container runtime does not support the v1 API, the kubelet falls back to using the (deprecated) v1alpha2 API instead.
Container runtimes
containerd
This section outlines the necessary steps to use containerd as CRI runtime.
Use the following commands to install Containerd on your system:
-
Install and configure prerequisites:
(these instructions apply to Linux nodes only)
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf overlay br_netfilter EOF sudo modprobe overlay sudo modprobe br_netfilter # Setup required sysctl params, these persist across reboots. cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 net.bridge.bridge-nf-call-ip6tables = 1 EOF # Apply sysctl params without reboot sudo sysctl --system
-
Install containerd:
On Linux the default CRI socket for containerd is /run/containerd/containerd.sock
.
On Windows the default CRI endpoint is npipe://./pipe/containerd-containerd
.
Configuring the systemd
cgroup driver
To use the systemd
cgroup driver in /etc/containerd/config.toml
with runc
, set
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
...
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
SystemdCgroup = true
If you apply this change, make sure to restart containerd:
sudo systemctl restart containerd
When using kubeadm, manually configure the cgroup driver for kubelet.
CRI-O
This section contains the necessary steps to install CRI-O as a container runtime.
To install CRI-O, follow .
cgroup driver
CRI-O uses the systemd cgroup driver per default, which is likely to work fine
for you. To switch to the cgroupfs
cgroup driver, either edit
/etc/crio/crio.conf
or place a drop-in configuration in
/etc/crio/crio.conf.d/02-cgroup-manager.conf
, for example:
[crio.runtime]
conmon_cgroup = "pod"
cgroup_manager = "cgroupfs"
You should also note the changed conmon_cgroup
, which has to be set to the value
pod
when using CRI-O with cgroupfs
. It is generally necessary to keep the
cgroup driver configuration of the kubelet (usually done via kubeadm) and CRI-O
in sync.
For CRI-O, the CRI socket is /var/run/crio/crio.sock
by default.
Docker Engine
-
On each of your nodes, install Docker for your Linux distribution as per .
For cri-dockerd
, the CRI socket is /run/cri-dockerd.sock
by default.
Mirantis Container Runtime
You can use Mirantis Container Runtime with Kubernetes using the open source
To learn more about how to install Mirantis Container Runtime,
visit . Check the systemd unit named As well as a container runtime, your cluster will need a working
network plugin.cri-docker.socket
to find out the path to the CRI
socket.What's next
Items on this page refer to third party products or projects that provide functionality required by Kubernetes. The Kubernetes project authors aren't responsible for those third-party products or projects. See the