提供了编程的基础技术教程

zl程序教程

您现在的位置是：首页 >  其他

当前栏目

持续集成Harbor+Helm

2023-03-20 14:57:27 时间

1 环境准备

在原来Harbor 2.1.2安装 的基础上，继续集成Helm仓库

  • Helm 是一个命令行下的客户端工具。主要用于 Kubernetes 应用程序 Chart 的创建、打包、发布以及创建和管理本地和远程的 Chart 仓库。
  • Tiller 是 Helm 的服务端，部署在 Kubernetes 集群中。Tiller 用于接收 Helm 的请求，并根据 Chart 生成 Kubernetes 的部署文件（ Helm 称为 Release ），然后提交给 Kubernetes 创建应用。Tiller 还提供了 Release 的升级、删除、回滚等一系列功能。
  • Chart Helm 的软件包，采用 TAR 格式。类似于 APT 的 DEB 包或者 YUM 的 RPM 包，其包含了一组定义 Kubernetes 资源相关的 YAML 文件
  • Repoistory Helm 的软件仓库，Repository 本质上是一个 Web 服务器，该服务器保存了一系列的 Chart 软件包以供用户下载，并且提供了一个该 Repository 的 Chart 包的清单文件以供查询。Helm 可以同时管理多个不同的 Repository。-->Harbor
  • Release 使用 helm install 命令在 Kubernetes 集群中部署的 Chart 称为 Release

注意点：

helm3移除了tiller这个组件，默认通过~/.kube/config与集群进行交互，也就是说使用了与kubctl相同的上下文访问权限，若不在默认位置可通过–kubeconfig参数进行指定，按照官方安装文档安装即可直接使用

**-------------------------------Harbor服务器

相关操作如下----------------------------**

2 启用 Harbor的chart repository服务

默认新版 harbor不会启用 chart repository service ，如果需要管理 helm ，我们需要在安装时添加额外的参数

## 默认安装
$ cd /usr/local/harbor
$ ./install.sh
## 启动 chart repository service 服务
$ cd /usr/local/harbor
$ ./install.sh --with-chartmuseum

等待安装完成，完成后会有如下提示：

[Step 0]: checking if docker is installed ...
Note: docker version: 20.10.1
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.27.4
[Step 2]: loading Harbor images ...
Loaded image: goharbor/chartmuseum-photon:v2.1.2
Loaded image: goharbor/prepare:v2.1.2
Loaded image: goharbor/harbor-log:v2.1.2
Loaded image: goharbor/harbor-registryctl:v2.1.2
Loaded image: goharbor/clair-adapter-photon:v2.1.2
Loaded image: goharbor/harbor-db:v2.1.2
Loaded image: goharbor/harbor-jobservice:v2.1.2
Loaded image: goharbor/clair-photon:v2.1.2
Loaded image: goharbor/notary-signer-photon:v2.1.2
Loaded image: goharbor/harbor-portal:v2.1.2
Loaded image: goharbor/redis-photon:v2.1.2
Loaded image: goharbor/nginx-photon:v2.1.2
Loaded image: goharbor/trivy-adapter-photon:v2.1.2
Loaded image: goharbor/harbor-core:v2.1.2
Loaded image: goharbor/registry-photon:v2.1.2
Loaded image: goharbor/notary-server-photon:v2.1.2
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /usr/local/harbor
Clearing the configuration file: /config/portal/nginx.conf
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/passwd
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/root.crt
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Generated configuration file: /config/portal/nginx.conf
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /data/secret/keys/secretkey
Generated configuration file: /config/chartserver/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Note: stopping existing Harbor instance ...
Stopping harbor-jobservice ... done
Stopping nginx             ... done
Stopping harbor-core       ... done
Stopping harbor-portal     ... done
Stopping redis             ... done
Stopping registryctl       ... done
Stopping registry          ... done
Stopping harbor-db         ... done
Stopping harbor-log        ... done
Removing harbor-jobservice ... done
Removing nginx             ... done
Removing harbor-core       ... done
Removing harbor-portal     ... done
Removing redis             ... done
Removing registryctl       ... done
Removing registry          ... done
Removing harbor-db         ... done
Removing harbor-log        ... done
Removing network harbor_harbor
Removing network harbor_harbor-chartmuseum
WARNING: Network harbor_harbor-chartmuseum not found.
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-chartmuseum" with the default driver
Creating harbor-log ... done
Creating harbor-db     ... done
Creating registry      ... done
Creating registryctl   ... done
Creating redis         ... done
Creating harbor-portal ... done
Creating chartmuseum   ... done
Creating harbor-core   ... done
Creating nginx             ... done
Creating harbor-jobservice ... done
✔ ----Harbor has been installed and started successfully.----

这之后，就可以用上述 harbor 来管理 helm charts。

3 图形界面操作

3.1 创建项目

首先，需要在 harbor 上创建一个名为 helm-repo 的项目，如图所示：

3.2 创建test用户

**-------------------------------kmaster服务器

相关操作如下----------------------------**

4 k8s集群一侧，安装helm等

4.1 安装helm

$ wget https://get.helm.sh/helm-v3.4.2-linux-amd64.tar.gz
$ tar -zxvf helm-v3.4.2-linux-amd64.tar.gz
$ cd linux-amd64/
# 拷贝helm到 /usr/local/bin 
$ cp helm /usr/local/bin

4.2 验证helm

$ helm 
The Kubernetes package manager
Common actions for Helm:
- helm search:    search for charts
- helm pull:      download a chart to your local directory to view
- helm install:   upload the chart to Kubernetes
- helm list:      list releases of charts
$ helm version
version.BuildInfo{Version:"v3.4.2", GitCommit:"23dd3af5e19a02d4f4baa5b2f242645a1a3af629", GitTre                                                                                                  eState:"clean", GoVersion:"go1.14.13"}

5 chart仓库

5.1 添加 hostname

$ vi /etc/hosts
192.168.8.131 reg.local.com

5.1 添加chart仓库

5.1.1 添加认证

cat /etc/docker/daemon.json
{"exec-opts":["native.cgroupdriver=systemd"],"log-driver":"json-file","log-opts":{"max-size":"100m"},"insecure-registries": ["reg.local.com"]}

**-------------------------------harbor服务的认证信息

拷贝到kmaster服务器----------------------------**

5.1.2 把harbor的认证信息拷贝到kmaster服务器的相同位置

[root@harbor cert]# ll
total 32
-rw-r--r--. 1 root root 2033 Dec 31 04:25 ca.crt
-rw-r--r--. 1 root root 3243 Dec 31 04:25 ca.key
-rw-r--r--. 1 root root   17 Dec 31 04:25 ca.srl
-rw-r--r--. 1 root root 2094 Dec 31 04:25 reg.local.com.cert
-rw-r--r--. 1 root root 2094 Dec 31 04:25 reg.local.com.crt
-rw-r--r--. 1 root root 1708 Dec 31 04:25 reg.local.com.csr
-rw-r--r--. 1 root root 3247 Dec 31 04:25 reg.local.com.key
-rw-r--r--. 1 root root  263 Dec 31 04:25 v3.ext
[root@kmaster cert]# ll
total 32
-rw-r--r-- 1 root root 2033 Dec 31 04:25 ca.crt
-rw-r--r-- 1 root root 3243 Dec 31 04:25 ca.key
-rw-r--r-- 1 root root   17 Dec 31 04:25 ca.srl
-rw-r--r-- 1 root root 2094 Dec 31 04:25 reg.local.com.cert
-rw-r--r-- 1 root root 2094 Dec 31 04:25 reg.local.com.crt
-rw-r--r-- 1 root root 1708 Dec 31 04:25 reg.local.com.csr
-rw-r--r-- 1 root root 3247 Dec 31 04:25 reg.local.com.key
-rw-r--r-- 1 root root  263 Dec 31 04:25 v3.ext

-------------------------------kmaster服务器相关操作如下----------------------------

5.1.3 追加私有仓库

$ helm repo add stable https://charts.helm.sh/stable
$ helm repo add --ca-file /data/cert/ca.crt --cert-file /data/cert/reg.local.com.cert --key-file /data/cert/reg.local.com.key test https://reg.local.com/chartrepo/helm-repo
"test" has been added to your repositories

5.2 更新chart列表

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "test" chart repository
...Successfully got an update from the "stable" chart repository

5.3 使用helm安装mysql

$ helm install stable/mysql --generate-name

5.4 查看mysql是否成功安装

$ helm ls

5.5 推送应用

helm3 现在默认不支持推送到charts库，需要安装插件helm-push

5.5.1 安装插件

$ helm plugin install https://github.com/chartmuseum/helm-push
Downloading and installing helm-push v0.9.0 ...
https://github.com/chartmuseum/helm-push/releases/download/v0.9.0/helm-push_0.9.0_linux_amd64.tar.gz
Installed plugin: push

5.5.2 查看追加的chart仓库

$ helm repo list
NAME    URL
stable  https://charts.helm.sh/stable
test    https://reg.local.com/chartrepo/helm-repo

5.5.3 创建chart

$ helm create mychart
Creating mychart
$ ll
total 8
drwxr-xr-x 2 root root    6 Jan  6 15:15 charts
-rw-r--r-- 1 root root 1098 Jan  6 15:15 Chart.yaml
drwxr-xr-x 3 root root  162 Jan  6 15:15 templates
-rw-r--r-- 1 root root 1800 Jan  6 15:15 values.yaml

5.5.4 检查chart

$ helm lint ./mychart
==> Linting ./mychart
Error unable to check Chart.yaml file in chart: stat mychart/Chart.yaml: no such file or directory
Error: 1 chart(s) linted, 1 chart(s) failed
[root@kmaster mychart]# cd ..
[root@kmaster ~]# helm lint ./mychart
==> Linting ./mychart
[INFO] Chart.yaml: icon is recommended
1 chart(s) linted, 0 chart(s) failed

5.5.5 推送chart

$ helm push mychart test --ca-file /data/cert/ca.crt -u admin -p Harbor12345
Pushing mychart-0.1.0.tgz to test...
Done.

-------------------------------harbor服务器相关操作如下----------------------------

6 验证刚推送成功的chart

版本0.1.0

-------------------------------kmaster服务器相关操作如下----------------------------

7 从chart仓库安装

7.1 现有安装好的chart

$ helm list
NAME                    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION

7.2 安装mychart 0.1.0

$ helm install my1  mychart
NAME: my1
LAST DEPLOYED: Wed Jan  6 15:57:47 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=mychart,app.kubernetes.io/instance=my1" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT

7.3 卸载mychart 0.1.0

$ helm list
NAME    NAMESPACE       REVISION        UPDATED                                 STATUS          CHART           APP VERSION
my1     default         1               2021-01-06 15:57:47.410255239 +0900 JST deployed        mychart-0.1.0   1.16.0
$ helm uninstall my1
release "my1" uninstalled

8 从Harbor拉web-demo镜像执行，看一下结果

8.1 镜像拉下来

docker pull reg.local.com/dev/web-demo:v1.0
v1.0: Pulling from dev/web-demo
7448db3b31eb: Pull complete
c36604fa7939: Pull complete
29e8ef0e3340: Pull complete
a0c934d2565d: Pull complete
a360a17c9cab: Pull complete
cfcc996af805: Pull complete
2cf014724202: Pull complete
4bc402a00dfe: Pull complete
7f506d2e6327: Pull complete
577ef7ca6761: Pull complete
Digest: sha256:80530996e66df715a51f3407423802a106271b1bc4267114d5a6cc5f9a0a87ef
Status: Downloaded newer image for reg.local.com/dev/web-demo:v1.0
reg.local.com/dev/web-demo:v1.0

8.2 查看一下镜像

$ docker images
REPOSITORY                           TAG           IMAGE ID       CREATED         SIZE
reg.local.com/dev/web-demo           v1.0          e03f3e0f181f   41 hours ago    677MB

8.3 执行镜像 并把端口8080 外放成8888

docker run --publish 8888:8080 reg.local.com/dev/web-demo:v1.0
  .   ____          _            __ _ _
 /\ / ___'_ __ _ _(_)_ __  __ _    
( ( )\___ | '_ | '_| | '_ / _` |    
 \/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.1)
2021-01-06 17:41:12.950  INFO 1 --- [           main] com.local.demo.DemoApplication           : Starting DemoApplication v0.0.1-SNAPSHOT using Java 1.8.0_111 on 8f2fb5b3f985 with PID 1 (/app.jar started by root in /)
2021-01-06 17:41:12.957  INFO 1 --- [           main] com.local.demo.DemoApplication           : No active profile set, falling back to default profiles: default
2021-01-06 17:41:14.150  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port(s): 8080 (http)
2021-01-06 17:41:14.160  INFO 1 --- [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2021-01-06 17:41:14.160  INFO 1 --- [           main] org.apache.catalina.core.StandardEngine  : Starting Servlet engine: [Apache Tomcat/9.0.41]
2021-01-06 17:41:14.212  INFO 1 --- [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2021-01-06 17:41:14.212  INFO 1 --- [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1177 ms
2021-01-06 17:41:14.476  INFO 1 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2021-01-06 17:41:14.641  INFO 1 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2021-01-06 17:41:14.651  INFO 1 --- [           main] com.local.demo.DemoApplication           : Started DemoApplication in 2.136 seconds (JVM running for 2.486)
2021-01-06 17:41:30.169  INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2021-01-06 17:41:30.169  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2021-01-06 17:41:30.170  INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms

8.4 浏览器查看一下画面

其他方式部署可以参考：k8s+jenkins实现自动化部署应用至k8s集群

相关文章