aks微软容器paas平台

主要是使用azure 容器平台的一些基本问题

aks 节点升级操作过程

增加新节点
cordon
drain
delete

kubeconfig 文件

对比 aws 的容器平台 aws 提供的config 文件为证书认证 区别于token 认证。不依赖于云平台指令 如 aws cli ，az cli。而且能更好的兼容第三方配置。这点和阿里云容器平台类似

点击查看代码

aws config 示例
kind: Config
preferences: {}
users:
- name: arn:aws:region-1:110222010012:cluster/aws-xproject-prod
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1alpha1
      args:
      - --region
      - ap-southeast-1
      - eks
      - get-token
      - --cluster-name
      - iambigb
      command: aws

打开api防火墙 防止暴力破环

使用HTTP 应用程序路由

annotations:
kubernetes.io/ingress.class: addon-http-application-routing

限制

HTTP 应用程序路由当前不适用于 AKS 版本 1.22.6+ （太垃圾了真心地），目前测试版本1.23x，虽然提示不兼容但是我还是要强力一试

开启插件

搞了半天就是用的 开源nginx-ingress 垃圾玩意，感觉智商受到了极大的侮辱mlgbdzzwy

应用程序网关

还是这个靠谱
看看官方文档 https://azure.github.io/application-gateway-kubernetes-ingress/annotations/

注： 亲测控制台关闭时会直接删除网关服务

virtual node 的概念

启动集群时开启了virtual node的话 可以得到以下节点，virtual kubelet 的一种实现场景。包括阿里云 ECI等都在这个项目中有贡献https://virtual-kubelet.io/docs/

点击查看代码

Name:               virtual-node-aci-linux
Roles:              agent
Labels:             alpha.service-controller.kubernetes.io/exclude-balancer=true
                    beta.kubernetes.io/os=linux
                    kubernetes.azure.com/managed=false
                    kubernetes.azure.com/role=agent
                    kubernetes.io/hostname=virtual-node-aci-linux
                    kubernetes.io/role=agent
                    node-role.kubernetes.io/agent=
                    node.kubernetes.io/exclude-from-external-load-balancers=true
                    type=virtual-kubelet
Annotations:        node.alpha.kubernetes.io/ttl: 0
                    virtual-kubelet.io/last-applied-node-status:
                      {"capacity":{"cpu":"10k","memory":"4Ti","nvidia.com/gpu":"100","pods":"5k"},"allocatable":{"cpu":"10k","memory":"4Ti","nvidia.com/gpu":"10...
                    virtual-kubelet.io/last-applied-object-meta:
                      {"name":"virtual-node-aci-linux","uid":"bef0bb03-f65f-430f-aa7d-2f86232b5882","creationTimestamp":null,"labels":{"alpha.service-controller...
CreationTimestamp:  Sun, 26 Jun 2022 01:12:03 +0800
Taints:             virtual-kubelet.io/provider=azure:NoSchedule
Unschedulable:      false
Lease:
  HolderIdentity:  virtual-node-aci-linux
  AcquireTime:     <unset>
  RenewTime:       Sun, 26 Jun 2022 01:15:10 +0800
Conditions:
  Type                 Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----                 ------  -----------------                 ------------------                ------                       -------
  Ready                True    Sun, 26 Jun 2022 01:15:03 +0800   Wed, 01 Jun 2022 11:55:35 +0800   KubeletReady                 kubelet is ready.
  OutOfDisk            False   Sun, 26 Jun 2022 01:15:03 +0800   Wed, 01 Jun 2022 11:55:35 +0800   KubeletHasSufficientDisk     kubelet has sufficient disk space available
  MemoryPressure       False   Sun, 26 Jun 2022 01:15:03 +0800   Wed, 01 Jun 2022 11:55:35 +0800   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure         False   Sun, 26 Jun 2022 01:15:03 +0800   Wed, 01 Jun 2022 11:55:35 +0800   KubeletHasNoDiskPressure     kubelet has no disk pressure
  NetworkUnavailable   False   Sun, 26 Jun 2022 01:15:03 +0800   Wed, 01 Jun 2022 11:55:35 +0800   RouteCreated                 RouteController created a route
Addresses:
  InternalIP:  10.2.1.239
Capacity:
  cpu:             10k
  memory:          4Ti
  nvidia.com/gpu:  100
  pods:            5k
Allocatable:
  cpu:             10k
  memory:          4Ti
  nvidia.com/gpu:  100
  pods:            5k
System Info:
  Machine ID:
  System UUID:
  Boot ID:
  Kernel Version:
  OS Image:
  Operating System:           Linux
  Architecture:               amd64
  Container Runtime Version:
  Kubelet Version:            v1.19.10-vk-azure-aci-v1.4.1
  Kube-Proxy Version:
Non-terminated Pods:          (0 in total)
  Namespace                   Name    CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----    ------------  ----------  ---------------  -------------  ---
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests  Limits
  --------           --------  ------
  cpu                0 (0%)    0 (0%)
  memory             0 (0%)    0 (0%)
  ephemeral-storage  0 (0%)    0 (0%)
  nvidia.com/gpu     0         0
Events:
  Type    Reason          Age    From             Message
  ----    ------          ----   ----             -------
  Normal  RegisteredNode  3m10s  node-controller  Node virtual-node-aci-linux event: Registered Node virtual-node-aci-linux in Controller

如果想使用的话

点击查看代码

apiVersion: apps/v1
kind: Deployment
metadata:
  name: aci-helloworld
spec:
  replicas: 1
  selector:
    matchLabels:
      app: aci-helloworld
  template:
    metadata:
      labels:
        app: aci-helloworld
    spec:
      containers:
      - name: aci-helloworld
        image: mcr.microsoft.com/azuredocs/aci-helloworld
        ports:
        - containerPort: 80
      nodeSelector:
        kubernetes.io/role: agent
        beta.kubernetes.io/os: linux
        type: virtual-kubelet
      tolerations:
      - key: virtual-kubelet.io/provider
        operator: Exists

network policy

注意 不是policy for pod security 来限制pod是否能拥有privilege 权限等，是network policy 。
文档
aks 有两种网络插件 kubenet 和 azure cni。本次测试集群采用 azure cni + azure network policy 的方式进行，文档中描述了azure network policy 和 calico network policy 的区别。