aws生产实践-19：在aws的eks上部署kubesphere3.2.0并修改其分布

目录：

(1).版本相关

(2).部署kubesphere3.2.0

(3).规划kubesphere的pod分布

1.eks创建运维nodegroup

2.kubectl edit修改kubesphere组建分布

2.1.获取kubesphere的所有deployment和statefulset

2.2.依次修改kubesphere的所有deployment和statefulset的nodeselector

(1).版本相关

eks中的kubernetes版本：1.21

kubesphere版本：v3.2.0

(2).部署kubesphere

使用 kubectl 安装 KubeSphere，以下命令仅用于默认的最小安装：

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/kubesphere-installer.yaml

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.0/cluster-configuration.yaml

检查安装日志：

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

(3).规划kubesphere的pod分布

我们只想让他分布在运维的worknode分组上。

1.eks创建运维nodegroup

创建nodegroup：devops，这个节点分组专门存放运维相关的pods。

并且要给这个nodegroup打一个标记：devops=yes

2.kubectl edit修改kubesphere组建分布

2.1.获取kubesphere的所有deployment和statefulset

一共是这些组件，我们要找到对应的deployment和statefulset，然后使用kubectl edit修改nodeselector使他们都分布在devops这个节点分组上。

使用命令找到所有的deployment和statefulset：

kubectl get all -A | grep -i -E "deployment|statefulset" | grep -i kubesphere | awk '{print

找到：第一列是namespace，第二列是deployment/statefulset。

kubesphere-controls-system deployment.apps/default-http-backend

kubesphere-controls-system deployment.apps/kubectl-admin

kubesphere-monitoring-system deployment.apps/kube-state-metrics

kubesphere-monitoring-system deployment.apps/notification-manager-deployment

kubesphere-monitoring-system deployment.apps/notification-manager-operator

kubesphere-monitoring-system deployment.apps/prometheus-operator

kubesphere-system deployment.apps/ks-apiserver

kubesphere-system deployment.apps/ks-console

kubesphere-system deployment.apps/ks-controller-manager

kubesphere-system deployment.apps/ks-installer

kubesphere-monitoring-system statefulset.apps/alertmanager-main

kubesphere-monitoring-system statefulset.apps/prometheus-k8s

2.2.依次修改kubesphere的所有deployment和statefulset的nodeselector

nodeSelector:

devops: "yes"

注意，下边这三个的replica都大于1，要改成1，仅仅是为了省钱。可用性交给k8s保证。

deployment.apps/notification-manager-deployment

statefulset.apps/alertmanager-main

statefulset.apps/prometheus-k8s

alertmanager特殊，它是由Alertmanager组件控制的，所以先要修改它的nodeSelector和replica(改为1)：

kubectl edit -n kubesphere-monitoring-system Alertmanager main

notification-manager特殊，它是由NotificationManager组件控制，所以要先修改它的nodeSelector和replica(改为1)：

kubectl edit NotificationManager notification-manager

prometheus特殊，它是由Prometheus组件控制，所以要先修改它的nodeSelector和replica(改为1)：

kubectl edit prometheus k8s -n kubesphere-monitoring-system

使用命令输出完整的kubectl edit命令：

kubectl get all -A | grep -i -E "deployment|statefulset" | grep -i kubesphere | awk '{print "kubectl edit -n "

获得：

kubectl edit -n kubesphere-controls-system deployment.apps/default-http-backend

kubectl edit -n kubesphere-controls-system deployment.apps/kubectl-admin

kubectl edit -n kubesphere-monitoring-system deployment.apps/kube-state-metrics

kubectl edit -n kubesphere-monitoring-system deployment.apps/notification-manager-deployment

kubectl edit -n kubesphere-monitoring-system deployment.apps/notification-manager-operator

kubectl edit -n kubesphere-monitoring-system deployment.apps/prometheus-operator

kubectl edit -n kubesphere-system deployment.apps/ks-apiserver

kubectl edit -n kubesphere-system deployment.apps/ks-console

kubectl edit -n kubesphere-system deployment.apps/ks-controller-manager

kubectl edit -n kubesphere-system deployment.apps/ks-installer

kubectl edit -n kubesphere-monitoring-system statefulset.apps/alertmanager-main

kubectl edit -n kubesphere-monitoring-system statefulset.apps/prometheus-k8s

这样edit后可能还不会生效，因为删除老的（没有nodeSelector）replicaset还在，pod可能会根据老的replicaset进行创建；所以直接将kubesphere的所有replicaset都删除，让他根据新的deployment和statefulset进行replicaset的创建。

最终，只有kubesphere的node-exporter组件是所有node都有一个pod，这也是显然的。

最终pod：

kubectl get po -A | grep -i kubesphere