通过 GitLab 开始部署到 Kubernetes-未完纪

本教程介绍 GitLab 支持的 Kubernetes 部署方式。完成后，将理解如何使用 Flux 部署，如何从 GitLab CI/CD 向集群部署或运行命令，以及如何结合两者。

开始之前

教程基于“将 Kubernetes 集群连接到 GitLab”中创建的项目，也可使用任意已连接 Kubernetes、并完成 Flux 初始化的项目。

从 CI/CD 向集群运行命令

Kubernetes 代理与 GitLab CI/CD 集成，可安全、可扩展地执行 kubectl apply、helm upgrade 等命令。

本节使用流水线创建集群 Secret，用于访问 GitLab 容器注册表，后续步骤会继续使用它。

创建带 read_registry 作用域的 deploy token。
将令牌与用户名分别保存为 CI/CD 变量 CONTAINER_REGISTRY_ACCESS_TOKEN、CONTAINER_REGISTRY_ACCESS_USERNAME。
两个变量的环境都设为 container-registry-secret*。
对令牌变量启用 Mask 和 Protect。
将以下片段加入 .gitlab-ci.yml，把两处 AGENT_KUBECONTEXT 改为自己的项目路径：

stages:
- setup
- deploy
- stop

create-registry-secret:
  stage: setup
  image: "portainer/kubectl-shell:latest"
  variables:
    AGENT_KUBECONTEXT: my-group/optional-subgroup/my-repository:testing
  before_script:
    # The available agents are automatically injected into the runner environment
    # You need to select the agent to use
    - kubectl config use-context $AGENT_KUBECONTEXT
  script:
    - kubectl delete secret gitlab-registry-auth -n flux-system --ignore-not-found
    - kubectl create secret docker-registry gitlab-registry-auth -n flux-system
      --docker-password="${CONTAINER_REGISTRY_ACCESS_TOKEN}" --docker-username="${CONTAINER_REGISTRY_ACCESS_USERNAME}" --docker-server="${CI_REGISTRY}"
  environment:
    name: container-registry-secret
    on_stop: delete-registry-secret

delete-registry-secret:
  stage: stop
  image: ""
  variables:
    AGENT_KUBECONTEXT: my-group/optional-subgroup/my-repository:testing
  before_script:
    # The available agents are automatically injected into the runner environment
    # You need to select the agent to use
    - kubectl config use-context $AGENT_KUBECONTEXT
  script:
    - kubectl delete secret -n flux-system gitlab-registry-auth
  environment:
    name: container-registry-secret
    action: stop
  when: manual

继续之前，可以思考如何用相同方式运行其他命令。

将清单构建为 OCI 镜像并部署

生产环境的最佳实践，是在 Git 仓库与 FluxCD 之间使用 OCI 仓库作为缓存层。GitLab 流水线构建符合 Flux 要求的 OCI 镜像，FluxCD 检查仓库中的新镜像。更多企业实践见相关企业注意事项。

本节将简单 Kubernetes 清单打包为 OCI 制品，并部署到集群。

运行以下 Flux CLI 命令，告诉 Flux 从哪里获取镜像，以及如何部署内容。根据 GitLab 实例调整 --url，容器注册表地址可在“Deploy > Container registry”找到。查看生成的 clusters/testing/nginx.yaml，可以理解 Flux 如何寻找清单。

flux create source oci nginx-example \
 --url oci://registry.gitlab.example.org/my-group/optional-subgroup/my-repository/nginx-example \
 --tag latest \
 --secret-ref gitlab-registry-auth \
 --interval 1m \
 --namespace flux-system \
 --export > clusters/testing/nginx.yaml
 flux create kustomization nginx-example \
 --source OCIRepository/nginx-example \
 --path "." \
 --prune true \
 --target-namespace default \
 --interval 1m \
 --namespace flux-system \
 --export >> clusters/testing/nginx.yaml

以 NGINX 为例，在 clusters/applications/nginx/nginx.yaml 写入：

 apiVersion: apps/v1
 kind: Deployment
 metadata:
   name: nginx-example
   namespace: default
 spec:
   replicas: 1
   selector:
     matchLabels:
       app: nginx-example
   template:
     metadata:
       labels:
         app: nginx-example
     spec:
       containers:
         - name: nginx
           image: nginx:1.25
           ports:
             - containerPort: 80
               protocol: TCP
 ---
 apiVersion: v1
 kind: Service
 metadata:
   name: nginx-example
   namespace: default
 spec:
   ports:
     - port: 80
       targetPort: 80
       protocol: TCP
   selector:
     app: nginx-example

接着打包为 OCI 镜像。在 .gitlab-ci.yml 加入以下内容，并再次调整 AGENT_KUBECONTEXT：

 nginx-deployment:
     stage: deploy
     variables:
         IMAGE_NAME: nginx-example   # Image name to push
         IMAGE_TAG: latest
         MANIFEST_PATH: "./clusters/applications/nginx"
         IMAGE_TITLE: NGINX example   # Image title to use in OCI annotation
         AGENT_KUBECONTEXT: my-group/optional-subgroup/my-repository:testing
         FLUX_OCI_REPO_NAME: nginx-example  # Flux OCIRepository to reconcile
         NAMESPACE: flux-system  # Namespace for the OCIRepository resource
     # This section configures a GitLab environment for the nginx deployment specifically
     environment:
         name: applications/nginx
         kubernetes:
             agent: $AGENT_KUBECONTEXT
             dashboard:
               namespace: default
               flux_resource_path: kustomize.toolkit.fluxcd.io/v1/namespaces/flux-system/kustomizations/nginx-example  # You will deploy this resource in the next step
     image:
         name: "fluxcd/flux-cli:v2.4.0"
         entrypoint: [""]
     before_script:
         - kubectl config use-context $AGENT_KUBECONTEXT
     script:
         # This line builds and pushes the OCI container to the GitLab container registry.
         # You can read more about this command in https://fluxcd.io/flux/cmd/flux_push_artifact/
         - flux push artifact oci://${CI_REGISTRY_IMAGE}/${IMAGE_NAME}:${IMAGE_TAG}
             --source="${CI_REPOSITORY_URL}"
             --path="${MANIFEST_PATH}"
             --revision="${CI_COMMIT_SHORT_SHA}"
             --creds="${CI_REGISTRY_USER}:${CI_REGISTRY_PASSWORD}"
             --annotations="org.opencontainers.image.url=${CI_PROJECT_URL}"
             --annotations="org.opencontainers.image.title=${IMAGE_TITLE}"
             --annotations="com.gitlab.job.id=${CI_JOB_ID}"
             --annotations="com.gitlab.job.url=${CI_JOB_URL}"
         # This line triggers an immediate reconciliation of the resource. Otherwise Flux would reconcile following its configured reconciliation period.
         # You can read more about the various reconcile commands in https://fluxcd.io/flux/cmd/flux_reconcile/
         - flux reconcile source oci -n ${NAMESPACE} ${FLUX_OCI_REPO_NAME}

提交并推送更改，等待构建流水线完成。左侧进入“Operate > Environments”，查看 Kubernetes 仪表盘，applications/nginx 应处于健康状态。

保护流水线的集群访问

本节适用于 Premium、Ultimate，以及 GitLab.com、自管理和 Dedicated 部署。

代理由 .gitlab/agents/testing/config.yaml 配置。默认允许运行流水线的项目访问其配置集群，并使用代理服务账户执行命令。

可以将访问限定到静态服务账户，也可以把 CI/CD 作业作为集群中的身份，再使用 Kubernetes RBAC 限制权限。本节给每个作业加入身份，并在集群中模拟该作业。

编辑代理配置，加入以下内容并替换项目路径：

ci_access:
   projects:
      - id: my-group/optional-subgroup/my-repository
        access_as:
           ci_job: {}

由于作业尚无集群权限绑定，此时不能从 CI/CD 执行 Kubernetes 命令。

接下来，按原教程为作业添加 Secret 创建与删除权限。创建 clusters/testing/gitlab-ci-job-secret-write.yaml：

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
   name: secret-manager
   namespace: default
rules:
   - apiGroups: [""]
     resources: ["secrets"]
     verbs: ["create", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
   name: gitlab-ci-secrets-binding
   namespace: default
subjects:
   - kind: Group
     name: gitlab:ci_job
     apiGroup: rbac.authorization.k8s.io
roleRef:
   kind: Role
   name: secret-manager
   apiGroup: rbac.authorization.k8s.io

再允许作业触发 FluxCD 协调。创建 clusters/testing/gitlab-ci-job-flux-reconciler.yaml：

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    name: ci-job-admin
roleRef:
    name: flux-edit-flux-system
    kind: ClusterRole
    apiGroup: rbac.authorization.k8s.io
subjects:
    - name: gitlab:ci_job
      kind: Group
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
    name: ci-job-view
roleRef:
    name: flux-view-flux-system
    kind: ClusterRole
    apiGroup: rbac.authorization.k8s.io
subjects:
    - name: gitlab:ci_job
      kind: Group

更详细的 CI/CD 访问配置，见 GitLab CI/CD 与 Kubernetes 集群集成文档。