instruction: Explain Set up a High Availability etcd Cluster with kubeadm
output: You are viewing documentation for Kubernetes version: v1.32
  Kubernetes v1.32 documentation is no longer actively maintained. The version you are currently viewing is a static snapshot. For up-to-date information, see the
  latest version.
  Set up a High Availability etcd Cluster with kubeadm
  By default, kubeadm runs a local etcd instance on each control plane node.
  It is also possible to treat the etcd cluster as external and provision
  etcd instances on separate hosts. The differences between the two approaches are covered in the
  Options for Highly Available topology
  page.
  This task walks through the process of creating a high availability external
  etcd cluster of three members that can be used by kubeadm during cluster creation.
  Before you begin
  Three hosts that can talk to each other over TCP ports 2379 and 2380. This
  document assumes these default ports. However, they are configurable through
  the kubeadm config file.
  Each host must have systemd and a bash compatible shell installed.
  Each host must
  have a container runtime, kubelet, and kubeadm installed
  Each host should have access to the Kubernetes container image registry (
  registry.k8s.io
  ) or list/pull the required etcd image using
  kubeadm config images list/pull
  . This guide will set up etcd instances as
  static pods
  managed by a kubelet.
  Some infrastructure to copy files between hosts. For example
  can satisfy this requirement.
  Setting up the cluster
  The general approach is to generate all certs on one node and only distribute
  necessary
  files to the other nodes.
  Note:
  kubeadm contains all the necessary cryptographic machinery to generate
  the certificates described below; no other cryptographic tooling is required for
  this example.
  Note:
  The examples below use IPv4 addresses but you can also configure kubeadm, the kubelet and etcd
  to use IPv6 addresses. Dual-stack is supported by some Kubernetes options, but not by etcd. For more details
  on Kubernetes dual-stack support see
  Dual-stack support with kubeadm
  Configure the kubelet to be a service manager for etcd.
  Note:
  You must do this on every host where etcd should be running.
  Since etcd was created first, you must override the service priority by creating a new unit file
  that has higher precedence than the kubeadm-provided kubelet unit file.
  << EOF > /etc/systemd/system/kubelet.service.d/kubelet.conf
  # Replace "systemd" with the cgroup driver of your container runtime. The default value in the kubelet is "cgroupfs".
  # Replace the value of "containerRuntimeEndpoint" for a different container runtime if needed.
  apiVersion: kubelet.config.k8s.io/v1beta1
  kind: KubeletConfiguration
  authentication:
  anonymous:
  enabled: false
  webhook:
  enabled: false
  authorization:
  mode: AlwaysAllow
  cgroupDriver: systemd
  address: 127.0.0.1
  containerRuntimeEndpoint: unix:///var/run/containerd/containerd.sock
  staticPodPath: /etc/kubernetes/manifests
  << EOF > /etc/systemd/system/kubelet.service.d/20-etcd-service-manager.conf
  [Service]
  ExecStart=
  ExecStart=/usr/bin/kubelet --config=/etc/systemd/system/kubelet.service.d/kubelet.conf
  Restart=always
  systemctl daemon-reload
  systemctl restart kubelet
  Check the kubelet status to ensure it is running.
  systemctl status kubelet
  Create configuration files for kubeadm.
  Generate one kubeadm configuration file for each host that will have an etcd
  member running on it using the following script.
  # Update HOST0, HOST1 and HOST2 with the IPs of your hosts
  export
  HOST0
  export
  HOST1
  export
  HOST2
  # Update NAME0, NAME1 and NAME2 with the hostnames of your hosts
  export
  NAME0
  "infra0"
  export
  NAME1
  "infra1"
  export
  NAME2
  "infra2"
  # Create temp directories to store files that will end up on other hosts
  mkdir -p /tmp/
  HOST0
  / /tmp/
  HOST1
  / /tmp/
  HOST2
  HOSTS
  HOST0
  HOST1
  HOST2
  NAMES
  NAME0
  NAME1
  NAME2
  i in
  !HOSTS[@]
  HOST
  HOSTS
  NAME
  NAMES
  << EOF > /tmp/${HOST}/kubeadmcfg.yaml
  apiVersion: "kubeadm.k8s.io/v1beta4"
  kind: InitConfiguration
  nodeRegistration:
  name: ${NAME}
  localAPIEndpoint:
  advertiseAddress: ${HOST}
  apiVersion: "kubeadm.k8s.io/v1beta4"
  kind: ClusterConfiguration
  etcd:
  local:
  serverCertSANs:
  - "${HOST}"
  peerCertSANs:
  - "${HOST}"
  extraArgs:
  - name: initial-cluster
  value: ${NAMES[0]}=https://${HOSTS[0]}:2380,${NAMES[1]}=https://${HOSTS[1]}:2380,${NAMES[2]}=https://${HOSTS[2]}:2380
  - name: initial-cluster-state
  value: new
  - name: name
  value: ${NAME}
  - name: listen-peer-urls
  value: https://${HOST}:2380
  - name: listen-client-urls
  value: https://${HOST}:2379
  - name: advertise-client-urls
  value: https://${HOST}:2379
  - name: initial-advertise-peer-urls
  value: https://${HOST}:2380
  done
  Generate the certificate authority.
  If you already have a CA then the only action that is copying the CA's
  file to
  /etc/kubernetes/pki/etcd/ca.crt
  /etc/kubernetes/pki/etcd/ca.key
  . After those files have been copied,
  proceed to the next step, "Create certificates for each member".
  If you do not already have a CA then run this command on
  $HOST0
  (where you
  generated the configuration files for kubeadm).
  kubeadm init phase certs etcd-ca
  This creates two files:
  /etc/kubernetes/pki/etcd/ca.crt
  /etc/kubernetes/pki/etcd/ca.key
  Create certificates for each member.
  kubeadm init phase certs etcd-server --config
  /tmp/
  HOST2
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-peer --config
  /tmp/
  HOST2
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-healthcheck-client --config
  /tmp/
  HOST2
  /kubeadmcfg.yaml
  kubeadm init phase certs apiserver-etcd-client --config
  /tmp/
  HOST2
  /kubeadmcfg.yaml
  cp -R /etc/kubernetes/pki /tmp/
  HOST2
  # cleanup non-reusable certificates
  find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
  kubeadm init phase certs etcd-server --config
  /tmp/
  HOST1
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-peer --config
  /tmp/
  HOST1
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-healthcheck-client --config
  /tmp/
  HOST1
  /kubeadmcfg.yaml
  kubeadm init phase certs apiserver-etcd-client --config
  /tmp/
  HOST1
  /kubeadmcfg.yaml
  cp -R /etc/kubernetes/pki /tmp/
  HOST1
  find /etc/kubernetes/pki -not -name ca.crt -not -name ca.key -type f -delete
  kubeadm init phase certs etcd-server --config
  /tmp/
  HOST0
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-peer --config
  /tmp/
  HOST0
  /kubeadmcfg.yaml
  kubeadm init phase certs etcd-healthcheck-client --config
  /tmp/
  HOST0
  /kubeadmcfg.yaml
  kubeadm init phase certs apiserver-etcd-client --config
  /tmp/
  HOST0
  /kubeadmcfg.yaml
  # No need to move the certs because they are for HOST0
  # clean up certs that should not be copied off this host
  find /tmp/
  HOST2
  -name ca.key -type f -delete
  find /tmp/
  HOST1
  -name ca.key -type f -delete
  Copy certificates and kubeadm configs.
  The certificates have been generated and now they must be moved to their
  respective hosts.
  USER
  ubuntu
  HOST
  HOST1
  scp -r /tmp/
  HOST
  USER
  HOST
  USER
  HOST
  USER@HOST $ sudo -Es
  root@HOST $ chown -R root:root pki
  root@HOST $ mv pki /etc/kubernetes/
  Ensure all expected files exist.
  The complete list of required files on
  $HOST0
  /tmp/${HOST0}
  └── kubeadmcfg.yaml
  /etc/kubernetes/pki
  ├── apiserver-etcd-client.crt
  ├── apiserver-etcd-client.key
  └── etcd
  ├── ca.crt
  ├── ca.key
  ├── healthcheck-client.crt
  ├── healthcheck-client.key
  ├── peer.crt
  ├── peer.key
  ├── server.crt
  └── server.key
  $HOST1
  $HOME
  └── kubeadmcfg.yaml
  /etc/kubernetes/pki
  ├── apiserver-etcd-client.crt
  ├── apiserver-etcd-client.key
  └── etcd
  ├── ca.crt
  ├── healthcheck-client.crt
  ├── healthcheck-client.key
  ├── peer.crt
  ├── peer.key
  ├── server.crt
  └── server.key
  $HOST2
  $HOME
  └── kubeadmcfg.yaml
  /etc/kubernetes/pki
  ├── apiserver-etcd-client.crt
  ├── apiserver-etcd-client.key
  └── etcd
  ├── ca.crt
  ├── healthcheck-client.crt
  ├── healthcheck-client.key
  ├── peer.crt
  ├── peer.key
  ├── server.crt
  └── server.key
  Create the static pod manifests.
  Now that the certificates and configs are in place it's time to create the
  manifests. On each host run the
  kubeadm
  command to generate a static manifest
  for etcd.
  root@HOST0 $ kubeadm init phase etcd
  local
  --config
  /tmp/
  HOST0
  /kubeadmcfg.yaml
  root@HOST1 $ kubeadm init phase etcd
  local
  --config
  $HOME
  /kubeadmcfg.yaml
  root@HOST2 $ kubeadm init phase etcd
  local
  --config
  $HOME
  /kubeadmcfg.yaml
  Optional: Check the cluster health.
  etcdctl
  isn't available, you can run this tool inside a container image.
  You would do that directly with your container runtime using a tool such as
  crictl run
  and not through Kubernetes
  ETCDCTL_API
  etcdctl
  --cert /etc/kubernetes/pki/etcd/peer.crt
  --key /etc/kubernetes/pki/etcd/peer.key
  --cacert /etc/kubernetes/pki/etcd/ca.crt
  --endpoints https://
  HOST0
  :2379 endpoint health
  https://
  HOST0 IP
  :2379 is healthy: successfully committed proposal:
  took
  16.283339ms
  https://
  HOST1 IP
  :2379 is healthy: successfully committed proposal:
  took
  19.44402ms
  https://
  HOST2 IP
  :2379 is healthy: successfully committed proposal:
  took
  35.926451ms
  ${HOST0}
  to the IP address of the host you are testing.
  What's next
  Once you have an etcd cluster with 3 working members, you can continue setting up a
  highly available control plane using the
  external etcd method with kubeadm
  Was this page helpful?
  Thanks for the feedback. If you have a specific, answerable question about how to use Kubernetes, ask it on
  Stack Overflow
  Open an issue in the
  GitHub Repository
  if you want to
  report a problem
  suggest an improvement
  Last modified October 02, 2024 at 12:45 AM PST:
  Remove deprecated note on "Set up a High Availability etcd Cluster with kubeadm" page (#48120) (40419a5ef5)