使用kind创建一个k8s集群

kind是一个用于创建k8s集群的工具, 它使用docker容器作为节点, 可以快速创建一个k8s集群, 用于测试或者开发.
官方提供了相应的文档Ingress, 在使用该文档的时候出现了一些问题.
本文基于官方文档并作出了一定的修改介绍如何使用kind创建一个k8s集群, 并在集群中部署kong-ingress.最后部署一个测试应用并通过 kong-ingress访问该应用.

安装kind

执行以下命令安装kind

go install sigs.k8s.io/kind@v0.18.0

创建一个k8s集群

执行以下命令创建一个k8s集群, 集群中包含一个master节和一个worker节点.
同时将master节点的80&443端口映射到宿主机的80&443端口, 用于后续将ingress映射到宿主机的80&443端口.

cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    kubeadmConfigPatches:
      - |
        kind: InitConfiguration
        nodeRegistration:
          kubeletExtraArgs:
            node-labels: "ingress-ready=true"
    extraPortMappings:
      - containerPort: 80
        hostPort: 80
        protocol: TCP
      - containerPort: 443
        hostPort: 443
        protocol: TCP
  - role: worker
EOF

执行docker ps命令查看kind创建的docker容器

CONTAINER ID   IMAGE                               COMMAND                  CREATED       STATUS       PORTS                                                                 NAMES
b4368e7fbfb1   kindest/node:v1.26.3                "/usr/local/bin/entr…"   2 hours ago   Up 2 hours   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 127.0.0.1:45145->6443/tcp   kind-control-plane
c0638805438a   kindest/node:v1.26.3                "/usr/local/bin/entr…"   2 hours ago   Up 2 hours                                                                         kind-worker

安装kong

执行以下命令安装kong-ingress

kubectl apply -f https://raw.githubusercontent.com/Kong/kubernetes-ingress-controller/main/deploy/single/all-in-one-dbless.yaml

配置kong

执行以下命令配置kong.
第一个命令用于将deployment为proxy-kong下面的pod的replicas设置为1同时将端口映射到host上( 这里的host就是docker容器的host, 同时docker容器对应的port又映射到了本地的端口).

第二个命令用于将kong-proxy的service的类型设置为NodePort, 这样就可以通过host的port访问kong了.

kubectl patch deployment -n kong proxy-kong -p '{"spec":{"replicas":1, "template":{"spec":{"containers":[{"name":"proxy", "ports":[{"containerPort":8000,"hostPort":80,"name":"proxy","protocol":"TCP"},{"containerPort":8443,"hostPort":443,"name":"proxy-ssl","protocol":"TCP"}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

kubectl patch service -n kong kong-proxy -p '{"spec":{"type":"NodePort"}}'

启动测试应用

创建如下文件用于后续执行kubectl apply命令

usage.yml

kind: Pod
apiVersion: v1
metadata:
  name: foo-app
  labels:
    app: foo
spec:
  containers:
    - command:
        - /agnhost
        - netexec
        - --http-port
        - "8080"
      image: opsdockerimage/e2e-test-images-agnhost:2.26
      name: foo-app
---
kind: Service
apiVersion: v1
metadata:
  name: foo-service
spec:
  selector:
    app: foo
  ports:
    # Default port used by the image
    - port: 8080
---
kind: Pod
apiVersion: v1
metadata:
  name: bar-app
  labels:
    app: bar
spec:
  containers:
    - command:
        - /agnhost
        - netexec
        - --http-port
        - "8080"
      image: opsdockerimage/e2e-test-images-agnhost:2.26
      name: bar-app
---
kind: Service
apiVersion: v1
metadata:
  name: bar-service
spec:
  selector:
    app: bar
  ports:
    # Default port used by the image
    - port: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - http:
        paths:
          - pathType: Prefix
            path: /foo(/|$)(.*)
            backend:
              service:
                name: foo-service
                port:
                  number: 8080
          - pathType: Prefix
            path: /bar(/|$)(.*)
            backend:
              service:
                name: bar-service
                port:
                  number: 8080
---

docker puull kind load docker-image opsdockerimage/e2e-test-images-agnhost:2.26s
kind load docker-image opsdockerimage/e2e-test-images-agnhost:2.26
kubectl apply -f usage.yml

测试

执行以下命令测试

curl localhost/foo/hostname
curl localhost/bar/hostname

遇到的问题

nginx和contour等ingress安装失败

这个问题的原因是因为国内无法下载对应的镜像导致无法安装成功.
解决方案是采用kong作为ingress.

官方文档中的关于kong的ingress的配置无法生效

执行官方文档中的如下命令失败

kubectl patch deployment -n kong ingress-kong -p '{"spec":{"template":{"spec":{"containers":[{"name":"proxy","ports":[{"containerPort":8000,"hostPort":80,"name":"proxy","protocol":"TCP"},{"containerPort":8443,"hostPort":443,"name":"proxy-ssl","protocol":"TCP"}]}],"nodeSelector":{"ingress-ready":"true"},"tolerations":[{"key":"node-role.kubernetes.io/control-plane","operator":"Equal","effect":"NoSchedule"},{"key":"node-role.kubernetes.io/master","operator":"Equal","effect":"NoSchedule"}]}}}}'

报错如下

The Deployment "ingress-kong" is invalid: spec.template.spec.containers[0].image: Required value

这个问题的原因是因为deployment给错了, 正确的deployment应该是proxy-kong.

将命令中的ingress-kong替换为proxy-kong后, 执行成功.

对于这个问题, 我已经提交了pr, 但是还没有被合并, 可以参考
doc: fix wrong implementation of creation of kong-ingress #3183

参考资料

• kind