Kubernetes1.25.4版本安装.md · 胡桃夹子/markdown

# Kubernetes1.25.4版本安装
## 1 配置

###  1.1 环境介绍
OS：CentOS Linux release 8.5.2111

机器:

|      **IP**       |  **hostname**  |
| :-----------: | :--------: |
| **10.104.10.201** | **k8s-master** |
| **10.104.10.202** | **k8s-node1**   |
| **10.104.10.203** | **k8s-node2**   |

所有机器，都将yum源改为国内阿里云开源镜像源

```shell
cd /etc/yum.repos.d/
mkdir backup
mv CentOS-* backup
wget -O /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-vault-8.5.2111.repo
yum clean all && yum makecache
```

### 1.2 时间同步

从CentOS8.x开始，改为chrony同步时间
`所有机器`都安装chrony，设置为开机启动，并立即启动服务

```shell
yum install chrony -y
systemctl enable chronyd --now
```
修改配置，改为使用阿里云地址
`vim /etc/chrony.conf`

```shell
# 注释掉 pool 2.centos.pool.ntp.org.iburst
# 添加阿里云地址
server ntp.aliyun.com iburst
server cn.ntp.org.cn iburst
```
```shell
# 用东八区，北京，上海的时间
cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
# 重启服务
systemctl restart chronyd
# 立即同步时间
chronyc sources && chronyc -a makestep
```

### 1.3 升级内核

`所有机器`统一升级内核

```shell
# 导入elrepo gpg key
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org
# 安装 elrepo YUM源仓库
dnf install https://www.elrepo.org/elrepo-release-8.el8.elrepo.noarch.rpm -y
# 安装kernel-ml版本，ml为长期稳定版本，lt为长期维护版本，建议安装lt版本
dnf --enablerepo=elrepo-kernel install kernel-lt -y
# 设置grub2默认引导为0
grub2-set-default 0
# 重新生成grub2引导文件
grub2-mkconfig -o /boot/grub2/grub.cfg
# 更新后，需要重启，使升级的内核生效
reboot
```

`所有机器`重启后执行 `uname -ar`查看kernel 版本

### 1.4 关闭防火墙
`所有机器`关闭防火墙

```shell
systemctl stop firewalld && systemctl disable firewalld
systemctl stop iptables && systemctl disable iptables
```
### 1.5 关闭swap

`所有机器`关闭swap

```shell
# 临时关闭swap
swapoff -a
# 永久关闭swap
sed -ri 's/.*swap.*/#&/' /etc/fstab
```
`free -m`命令查看swap状态

若swap行都显示 0 则表示关闭成功

### 1.6 禁用SELuinux

`所有机器`禁用

```shell
# 临时禁用SELuinux
setenforce 0
# 永久禁用SELuinux
sed -i 's/^SELINUX=enforcing$/SELINUX=disabled/' /etc/selinux/config
```

`sestatus -v`命令查看SELuinux状态，若显示为`disabled`，则代表已经禁用

### 1.7 修改hostname

`所有机器`依次修改hostname

```shell
# 在10.104.10.201执行
hostnamectl set-hostname k8s-master
# 在10.104.10.202执行
hostnamectl set-hostname k8s-node1
# 在10.104.10.203执行
hostnamectl set-hostname k8s-node2
```

### 1.8 配置 hosts

`所有机器`配置下 hosts

```shell
cat >> /etc/hosts<<EOF
10.104.10.201 k8s-master
10.104.10.202 k8s-node1
10.104.10.203 k8s-node2
EOF
```

### 1.9 配置 ssh 互信

`所有机器`输入以下命令，创建一组公钥和私钥

```shell
# 直接一直回车就行
ssh-keygen
```
`所有机器`上执行公钥拷贝命令

```shell
ssh-copy-id -i ~/.ssh/id_rsa.pub root@k8s-master
ssh-copy-id -i ~/.ssh/id_rsa.pub root@k8s-node1
ssh-copy-id -i ~/.ssh/id_rsa.pub root@k8s-node2
```

### 1.10 桥接流量

允许 iptables 检查桥接流量

`所有机器`上执行

```shell
cat > /etc/sysctl.d/k8s_better.conf << EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

modprobe br_netfilter
lsmod |grep conntrack
modprobe ip_conntrack
sysctl -p /etc/sysctl.d/k8s_better.conf
```

`所有机器`上安装系统依赖包

```shell
dnf install -y lrzsz conntrack ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git
```

`所有机器`上执行，开启ipvs 转发

```shell
modprobe br_netfilter
```

```shell
cat > /etc/sysconfig/modules/ipvs.modules << EOF 
#!/bin/bash 
modprobe -- ip_vs 
modprobe -- ip_vs_rr 
modprobe -- ip_vs_wrr 
modprobe -- ip_vs_sh 
modprobe -- nf_conntrack
EOF

chmod 755 /etc/sysconfig/modules/ipvs.modules 
bash /etc/sysconfig/modules/ipvs.modules 
lsmod | grep -e ip_vs -e nf_conntrack
```

## 2.安装containerd

`所有机器`上都安装containerd

```shell
cat << EOF > /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

modprobe overlay
modprobe br_netfilter

# 获取阿里云YUM源
wget -O /etc/yum.repos.d/docker-ce.repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
# centos8默认安装有podman等组件，它与containerd安装发生冲突，需要卸载
dnf erase podman buildah -y
# 查看containerd.io所有版本
# 下载安装最新版本
# yum install -y containerd.io
# 安装指定版本
dnf install -y containerd.io-1.6.9

# 生成containerd的配置文件
mkdir /etc/containerd -p
# 生成配置文件
containerd config default > /etc/containerd/config.toml
```

编辑配置文件

> SystemdCgroup = **false** 改为 SystemdCgroup = **true**
> sandbox_image = "**k8s.gcr.io/pause:3.6**" 改为 sandbox_image = "**registry.aliyuncs.com/google_containers/pause:3.6**"

```shell
sed -i 's#SystemdCgroup = false#SystemdCgroup = true#g' /etc/containerd/config.toml
sed -i "s#registry.k8s.io/pause#registry.aliyuncs.com/google_containers/pause#g" /etc/containerd/config.toml
# 查看编辑情况
grep SystemdCgroup /etc/containerd/config.toml
grep sandbox_image /etc/containerd/config.toml
```

修改完config.tomal后，设置开机启动并立即启动服务

```shell
systemctl enable --now containerd
```

添加阿里云YUM软件源

```shell
cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

yum makecache
```

## 3.安装kubeadm，kubelet和kubectl

`所有机器`上安装kubeadm、kubelet、kubectl

默认是安装最新版本，这里指定安装1.25.4版本

```shell
# 查看所有版本
dnf list kubeadm --showduplicates
# 安装最新版本
# dnf install -y kubectl kubelet kubeadm
# 安装指定版本
dnf install -y kubectl-1.25.4 kubelet-1.25.4 kubeadm-1.25.4
```

为了实现docker使用的cgroupdriver与kubelet使用的cgroup的一致性，建议修改如下文件内容

```shell
# KUBELET_EXTRA_ARGS= 改为 KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
sed -i 's/^KUBELET_EXTRA_ARGS=$/KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"/' /etc/sysconfig/kubelet
# 查看修改内容
cat /etc/sysconfig/kubelet
```

kubelet服务设置为开机启动

```shell
systemctl enable kubelet
```

准备Kubernetes 1.25.4 所需要的镜像

```shell
kubeadm config images list --kubernetes-version=v1.25.4
```

## 4.使用kubeadm初始化

注意：kubeadm仅在`master机器`上执行初始化

```shell
kubeadm init \
    --kubernetes-version=v1.25.4 \
    --pod-network-cidr=10.224.0.0/16 \
    --apiserver-advertise-address=10.104.10.201 \
    --image-repository registry.aliyuncs.com/google_containers
```

> --apiserver-advertise-address 集群通告地址
 > --image-repository 由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址
 > --kubernetes-version K8s版本，与上面安装的一致
 > --service-cidr 集群内部虚拟网络，Pod统一访问入口
 > --pod-network-cidr Pod网络，，与下面部署的CNI网络组件yaml中保持一致

初始化正常情况下，执行下面脚本

```shell
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
```

所有`node节点`加入，token是初始化成功生成的

```shell
kubeadm join 10.104.10.201:6443 --token tvui3d.486osvnjdcbbj2gl \
	--discovery-token-ca-cert-hash sha256:ce643d557a53291d9f7b0b1b5164537bd68fc4b49624fa134577dbed9b6b9712
```

`master机器`执行

```shell
kubectl get nodes
```
正常情况下，是可以看到所有的节点信息
`node节点`执行

```shell
kubectl get nodes
```

发现报错：`The connection to the server localhost:8080 was refused - did you specify the right host or port?`

什么原因呢？主要是`node节点`没有配置环境变量

从`master `节点copy配置文件至`node`节点

```shell
scp /etc/kubernetes/admin.conf k8s-node1:/etc/kubernetes/
scp /etc/kubernetes/admin.conf k8s-node2:/etc/kubernetes/
```

在所有`node节点`上执行环境变量配置

```shell
echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile
source /etc/profile
```

此时再执行`kubectl get nodes`命令时，就能正常看到节点信息。

## 5.部署网络插件

***注意：仅在master机器上执行下面操作***

> 集群部署网络插件 master节点IP
> 网络组件有很多种，只需要部署其中一个即可，推荐Calico。
> Calico是一个纯三层的数据中心网络方案，Calico支持广泛的平台，包括Kubernetes、OpenStack等。
> Calico 在每一个计算节点利用 Linux Kernel 实现了一个高效的虚拟路由器（ vRouter） 来负责数据转发，而每个 vRouter 通过 BGP 协议负责把自己上运行的 workload 的路由信息向整个 Calico 网络内传播。
> 此外，Calico 项目还实现了 Kubernetes 网络策略，提供ACL功能。

```shell
# 1.下载Calico
wget https://docs.projectcalico.org/manifests/calico.yaml --no-check-certificate

vim calico.yaml

# 修改网段
...
- name: CALICO_IPV4POOL_CIDR
  value: "10.244.0.0/16"
...
```

```
kubectl apply -f calico.yaml
```

查看节点

```shell
kubectl get pod -n kube-system -o wide
kubectl get node
```

查看calico状态

```
kubectl get pods --all-namespaces
```

##  6.部署dashborad
注意：仅在`master机器`上执行下面操作

准备dashboard配置文件

注意，版本要和k8s版本匹配，具体参考：https://github.com/kubernetes/dashboard/releases

```shell
wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
```
```shell 
# 编辑配置文件
cp recommended.yaml recommended-secret.yaml
vim recommended-secret.yaml
```
指定 type，及 nodePort
```yaml
----
spec:
   type: NodePort   # 增加type=NodePort，为了外部可以访问dashboard应用
   ports:
     - port: 443
       targetPort: 8443
       nodePort: 32732 # 指定端口，若此处不指定，可通过命令查看：kubectl get svc -n kubernetes-dashboard
   selector:
     k8s-app: kubernetes-dashboardspec:
----
```
删除dashboard-certs，为后面使用自签证书作准备，即recommended-secret.yaml文件中找到下面内容进行删除
```yaml
---

apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

---
```

创建用户的配置文件

```shell
cat > dashboard-adminuser.yaml << EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard
EOF
```

```shell
# 初始化dashboard
kubectl apply -f recommended-secret.yaml
# 创建用户
kubectl apply -f dashboard-adminuser.yaml
```

如若要删除dashboard pod，命令如 下：

```shell
kubectl delete -f recommended-secret.yaml
```

查看dashboard状态，确保所有节点是`Running`状态：

```shell
kubectl get pods,svc -n kubernetes-dashboard
kubectl get pods --all-namespaces
```

**Kubernetes Dashboard，目前像Edge/Chrome等都无法访问，只有Firefox可以访问，这种问题是浏览器自带安全机制决定的，但给签发证书就可以访问**

签发证书在指定目录下进行：

```shell
mkdir tls && cd tls
```

**创建自签名CA**:

```shell
# 生成私钥
openssl genrsa -out ca.key 2048
# 生成自签名证书
# subj 
openssl req -new -x509 \
  -key ca.key \
  -out ca.crt \
  -days 3650 \
  -subj "/C=CN/ST=HB/L=WH/O=DM/OU=YPT/CN=CA"
# 查看CA内容
openssl x509 -in ca.crt -noout -text
```

**签发Dashboard证书:**

```shell
# 生成私钥
openssl genrsa -out dashboard.key 2048
# 申请签名请求，注意，IP为安装Dashboard服务器IP
openssl req -new -sha256 \
  -key dashboard.key \
  -out dashboard.csr \
  -subj "/C=CN/ST=Shanghai/L=Shanghai/O=k8s/OU=System/CN=10.104.10.201"
# 配置文件，注意，IP为安装Dashboard服务器IP
cat > dashboard.cnf << EOF
extensions = san
[san]
keyUsage = digitalSignature
extendedKeyUsage = clientAuth,serverAuth
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
subjectAltName = IP:10.104.10.201,IP:127.0.0.1,DNS:10.104.10.201,DNS:localhost
EOF

# 签发证书
openssl x509 -req -sha256 \
  -days 3650 \
  -in dashboard.csr \
  -out dashboard.crt \
  -CA ca.crt \
  -CAkey ca.key \
  -CAcreateserial \
  -extfile dashboard.cnf
# 查看证书
openssl x509 -in dashboard.crt -noout -text
```

> **申请签名subj参数，发证书单位标识信息，如:  C=CN,ST=Hunan,L=Changsha,O=k8s,OU=System,CN=Kubernetes**

```shell
kubectl create secret generic kubernetes-dashboard-certs \
    --from-file=tls/dashboard.key \
    --from-file=tls/dashboard.crt \
    -n kubernetes-dashboard
```

如若删除证书，命令如下：

```shell 
kubectl delete secret kubernetes-dashboard-certs -n kubernetes-dashboard
```

创建token

```shell
kubectl -n kubernetes-dashboard create token admin-user
```

记得将token保存，浏览器访问 dashboard时，需要用到 token

```
eyJhbGciOiJSUzI1NiIsImtpZCI6IkxuUVpjazNvWHE1SGZMMllGWWg3czRvRHpES3FvWkl0aVZhTjhzeGhpQVUifQ.eyJhdWQiOlsiaHR0cHM6Ly9rdWJlcm5ldGVzLmRlZmF1bHQuc3ZjLmNsdXN0ZXIubG9jYWwiXSwiZXhwIjoxNjcyNjU0ODI5LCJpYXQiOjE2NzI2NTEyMjksImlzcyI6Imh0dHBzOi8va3ViZXJuZXRlcy5kZWZhdWx0LnN2Yy5jbHVzdGVyLmxvY2FsIiwia3ViZXJuZXRlcy5pbyI6eyJuYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsInNlcnZpY2VhY2NvdW50Ijp7Im5hbWUiOiJhZG1pbi11c2VyIiwidWlkIjoiM2ZmZjM3NDItNjQ5Zi00N2ZlLWEzYjgtOTA2MGY5OWIwN2E0In19LCJuYmYiOjE2NzI2NTEyMjksInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlcm5ldGVzLWRhc2hib2FyZDphZG1pbi11c2VyIn0.iYUHgwOx1ho8VKvlxnMjEBZJJf87O95El5mYDj8cVL3AgA2-kp67BaGQ4SSuT6KbwqiK6dAJuQeHp5TV_-dNxkyHBGXW7yWrx14ZBu42ydaXS7Ku1K-GFekMLsz7Q8OoaFP8uCrP_6o14IvYqdXHkw18GOeOJ6D_KCRyK_uDcnvNNmavM97BzfApV37bnC7MDx1zPkvx9WiM8NTijf91iUKyHY_Q4JSSNpgVKGovy2RYK11SHQGjUF6rZ5pTTbT-zwvcwX7wRX1Vck6fy2L4hJmWbHJrk_95mHS5mb9u9WPhoXBqwBaPDlcfIRMSaJ3CVyPOSc1nfciLW1GO1BTq_g
```

找到dashboard  pod  service端口

```shell
kubectl get svc -n kubernetes-dashboard
```

https://10.104.10.201:32732 ，效果大致如下：

[![k8s_dashboard.png](https://gitee.com/infowangxin/markdown-images/raw/master/k8s/k8s_dashboard.png)](https://gitee.com/infowangxin/markdown-images/raw/master/k8s/k8s_dashboard.png)

## 7. 引用Reference
- [CentOS8搭建nfs服务](https://blog.csdn.net/wangxin_wangxin/article/details/128723006)
- [Kubernetes1.25.4版本安装](https://blog.csdn.net/wangxin_wangxin/article/details/128718696)
- [kubeasz安装kubernetes1.25.5](https://blog.csdn.net/wangxin_wangxin/article/details/128723301)
- [k8s一键安装redis单机版](https://blog.csdn.net/wangxin_wangxin/article/details/128724186)
- [k8s一键安装mysql8单机版](https://blog.csdn.net/wangxin_wangxin/article/details/128723587)
- [k8s部署springboot应用](https://blog.csdn.net/wangxin_wangxin/article/details/128736157)
- [Docker安装及学习](https://blog.csdn.net/wangxin_wangxin/article/details/128919721)
- [Docker制作springboot运行应用镜像](https://blog.csdn.net/wangxin_wangxin/article/details/128724395)
- [Docker制作Java8环境镜像](https://blog.csdn.net/wangxin_wangxin/article/details/131914351)
- [Docker安装Mysql5.7.31](https://blog.csdn.net/wangxin_wangxin/article/details/131225961)
- [Docker安装Mysql8.1.0](https://blog.csdn.net/wangxin_wangxin/article/details/132295867)
- [Elasticsearch单机版本安装](https://blog.csdn.net/wangxin_wangxin/article/details/128871261)
- [Elasticsearch集群安装](https://blog.csdn.net/wangxin_wangxin/article/details/128869697)
- [ELK安装](https://blog.csdn.net/wangxin_wangxin/article/details/131636303)
- [Docker安装ELK](https://blog.csdn.net/wangxin_wangxin/article/details/131636262)
- [zookeeper集群安装](https://blog.csdn.net/wangxin_wangxin/article/details/128725036)
- [Nginx日志切割](https://blog.csdn.net/wangxin_wangxin/article/details/128725060)
- [RabbitMQ集群安装](https://blog.csdn.net/wangxin_wangxin/article/details/130768429)
- [Docker安装RabbitMQ单机版](https://blog.csdn.net/wangxin_wangxin/article/details/132297116)
- [springboot集成prometheus+grafana](https://blog.csdn.net/wangxin_wangxin/article/details/128905651)
- [windows11安装android应用](https://blog.csdn.net/wangxin_wangxin/article/details/131836379)

胡桃夹子/markdown

简介

发行版

贡献者

近期动态

胡桃夹子/markdown .gitee-modal { width: 500px !important; }

简介

发行版

贡献者

近期动态

搜索帮助

胡桃夹子/markdown