Knative - Pub/Sub - Message Broker【十一】

Message Broker

#Broker
    ◼ 承载消息队列的组件，它从生产者接收消息，并根据消息交换规则将其交换至相应的队列（或Topic）
        ◆生产者通过特定的协议将Message投递至Broker
    ◼ 然后，通过队列（或Topic），将消息传递给消费者
    ◼ Kafka、RabbitMQ、ActiveMQ和RocketMQ是较为常见的代表产品

#消息代理模式
    ◼ Point-to-point messaging
        ◆消息的发送者与接收者之间存在“一对一”的关系，队列中的每条消息只发送一个接收者，并且只能被消费一次
        ◆适合消息仅能被处理一次的场景，例工资单处理、金融交易处理等
    ◼ Publish/subscribe messaging
        ◆即“发布/订阅”模式，每条消息的生产者将消息发布到一个主题（Topic），多个消费者可以访问他们希望从中接收
#消息的Topic
    ◆发布到Topic的所有消息，都会分发给订阅该Topic的消费者
        ⚫ Kafka的Topic内部由一到多个队列（Queue）组成，这些内部队列称为Partition
        ⚫ 消费者也可以在Partition级别订阅
    ◆广播式分发机制，消息的发布者与消息的消费者之间存在“一对多”的关系

Knative的Broker/Trigger 消息传递框架

#Broker
    ◼ Knative Eventing提供的CRD，负责收集CloudEvents类型的事件
    ◼ Broker对象会提供一个用于事件传入的入口端点，各生产者可以调用该入口将事件发往Broker
    ◼ 将事件投递至目的地的任务则由Trigger资源负责
    ◼ Trigger基于属性过滤事件，并将筛选出的的事件投递给订阅该Trigger的Subscriber
    ◼ Subscriber还可生成响应事件，并将这些新生成的事件传入Broker
    
#事件投递机制的具体实现方式，则依赖于Broker Class的配置

 Broker类型

#Knative Eventing支持以下几种类型的Broker
    ◼ 基于Channel的多租户Broker （Multi-tenant channel-based broker，简称为MT-Channel-based Broker）
        ◆基于Channel进行事件路由
        ◆需要部署至少一种Channel的实现
            ⚫ InMemoryChannel：可用于开发和测试目的，但不为生产环境提供适当的事件交付保证
            ⚫ KafkaChannel：提供生产环境所需的事件交付保证
    ◼ 其它的可用的Broker类型
        ◆Apache Kafka Broker
        ◆RabbitMQ Broker
        ◆GCP Broker

使用默认的Broker

#Knative Serving在名称空间级别提供了一个名为default的默认Broker，但使用前需要通过某种方式先行完成创建
#创建默认Broker的方法
    ◼ 命令式命令，或使用配置文件
        ◆kn broker create default --namespace NS_NAME
    ◼ 在Trigger资源上使用特定的Annotation自动创建
        ◆eventing.knative.dev/injection=enabled
    ◼ 在名称空间上添加特定的Label自动创建
        ◆eventing.knative.dev/injection=enabled
#删除默认的Broker
◼ 第一种方法创建的默认Broker可直接进行删除
◼ 后面两种是通过Injection的方式进行的资源创建，这类资源需要由管理员手动才能完成删除

Broker/Trigger 实践

#示例环境说明
    ◼ 基于MT-Channel-based的Broker
    ◼ Trigger1过滤“type=sayhi”类的事件
        ◆Sink为ksvc/event-display-hi
    ◼ Triiger2过滤“type=saybye”类的事件
        ◆Sink为ksvc/event-display-bye
    ◼ curl命令作为event source
    ◆基于HTTP协议推送消息至broker
    ◆Trigger基于类型过滤事件并完成分发
#命令式命令
    ◼ 两个KService
        ◆~$ kn service create event-display-hi --image ikubernetes/event_display --port 8080 --scale-min 1 -n event-demo
        ◆~$ kn service create event-display-bye --image ikubernetes/event_display --port 8080 --scale-min 1 -n event-demo
        ◆列出：kn service list -n event-demo

1.查看默认集群 Broker

#查看 集群默认 Broker 
[root@xianchaomaster1 ~]# kubectl get cm -n knative-eventing
NAME                          DATA   AGE
config-br-default-channel     1      19h
config-br-defaults            1      19h
config-features               6      19h
config-imc-event-dispatcher   2      18h
config-kreference-mapping     1      19h
config-leader-election        1      19h
config-logging                3      19h
config-observability          1      19h
config-ping-defaults          1      19h
config-sugar                  1      19h
config-tracing                1      19h
default-ch-webhook            1      19h
istio-ca-root-cert            1      19h
kube-root-ca.crt              1      19h
[root@xianchaomaster1 ~]# kubectl get cm config-br-defaults -n knative-eventing
NAME                 DATA   AGE
config-br-defaults   1      19h
#集群默认 Broker 为 MTChannelBasedBroker
[root@xianchaomaster1 ~]# kubectl get cm config-br-defaults -n knative-eventing -o yaml
apiVersion: v1
data:
  default-br-config: |
    clusterDefault:
      brokerClass: MTChannelBasedBroker
      apiVersion: v1
      kind: ConfigMap
      name: config-br-default-channel
      namespace: knative-eventing
      delivery:
        retry: 10
        backoffPolicy: exponential
        backoffDelay: PT0.2S
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"default-br-config":"clusterDefault:\n  brokerClass: MTChannelBasedBroker\n  apiVersion: v1\n  kind: ConfigMap\n  name: config-br-default-channel\n  namespace: knative-eventing\n  delivery:\n    retry: 10\n    backoffPolicy: exponential\n    backoffDelay: PT0.2S\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"knative-eventing","app.kubernetes.io/version":"1.7.1","eventing.knative.dev/release":"v1.7.1"},"name":"config-br-defaults","namespace":"knative-eventing"}}
  creationTimestamp: "2023-07-06T08:50:35Z"
  labels:
    app.kubernetes.io/name: knative-eventing
    app.kubernetes.io/version: 1.7.1
    eventing.knative.dev/release: v1.7.1
  name: config-br-defaults
  namespace: knative-eventing
  resourceVersion: "466564"
  uid: 5df83f4e-55a5-4209-875a-afd391376b7f

2.创建Broker

[root@xianchaomaster1 ~]# kn broker create br01
Broker 'br01' successfully created in namespace 'default'.

[root@xianchaomaster1 ~]# kn broker list
NAME   URL                                                                     AGE   CONDITIONS   READY   REASON
br01   http://broker-ingress.knative-eventing.svc.cluster.local/default/br01   8s    6 OK / 6     True

3.创建Trigger

#◼ Trigger1过滤“type=sayhi”类的事件
    ◆Sink为ksvc/event-display-hi
#◼ Triiger2过滤“type=saybye”类的事件
     ◆Sink为ksvc/event-display-bye

#创建Trigger tr01 指向 ksvc:event-display-001 过滤 com.xks.sayhi001、broker：br01
[root@xianchaomaster1 ~]# kn trigger create tr01 --broker br01 --sink ksvc:event-display-001 --filter type=com.xks.sayhi001
Trigger 'tr01' successfully created in namespace 'default'.


#创建Trigger tr02 指向 ksvc:event-display-002 过滤 com.xks.saybye002 、broker：br01
[root@xianchaomaster1 ~]# kn trigger create tr02 --broker br01 --sink ksvc:event-display-002 --filter type=com.xks.saybye002
Trigger 'tr02' successfully created in namespace 'default'.

[root@xianchaomaster1 ~]# kn trigger list
NAME   BROKER   SINK                     AGE   CONDITIONS   READY   REASON
tr01   br01     ksvc:event-display-001   89s   6 OK / 6     True
tr02   br01     ksvc:event-display-002   7s    6 OK / 6     True

#查看配置Yaml格式
[root@xianchaomaster1 ~]# kubectl get trigger tr01 -o yaml
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  annotations:
    eventing.knative.dev/creator: kubernetes-admin
    eventing.knative.dev/lastModifier: kubernetes-admin
  creationTimestamp: "2023-07-07T04:04:49Z"
  generation: 1
  labels:
    eventing.knative.dev/broker: br01
  name: tr01
  namespace: default
  resourceVersion: "970666"
  uid: 5fd1942a-7420-4a7f-a9a7-da76737a0e36
spec:
  broker: br01
  filter:
    attributes:
      type: com.xks.sayhi001
  subscriber:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display-001
      namespace: default
status:
  conditions:
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    status: "True"
    type: BrokerReady
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    message: No dead letter sink is configured.
    reason: DeadLetterSinkNotConfigured
    status: "True"
    type: DeadLetterSinkResolved
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    status: "True"
    type: DependencyReady
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    status: "True"
    type: SubscriberResolved
  - lastTransitionTime: "2023-07-07T04:04:50Z"
    status: "True"
    type: SubscriptionReady
  observedGeneration: 1
  subscriberUri: http://event-display-001.default.svc.cluster.local

 4.Curl 测试发送消息并且验证

[root@xianchaomaster1 ~]# kn broker list
NAME   URL                                                                     AGE   CONDITIONS   READY   REASON
br01   http://broker-ingress.knative-eventing.svc.cluster.local/default/br01   10m   6 OK / 6     True


#1.【Curl请求 Hello Knative default Broker Say HI】
[root@xianchaomaster1 ~]# kubectl run client-$RANDOM --image=ikubernetes/admin-box:v1.2 --restart=Never -it --command -- /bin/sh
curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/default/br01" -X POST -H "Content-Type: application/cloudevents+json" \
-d '{"id": "say-hi", "specversion": "1.0", "type": "com.xks.sayhi001", "source": "sendoff", "data": {"msg":"Hello Knative default Broker Say HI"}}'#测试验证 event-display-001-00001-deployment-68cdddddcb-hj8pg 接收到 Hello Knative default Broker Say HI

#测试验证  event-display-001-00001-deployment-68cdddddcb-hj8pg 收到 Hello Knative default Broker Say HI、不会收到 Hello Knative default Broker Say BYE
[root@xianchaomaster1 ~]# kubectl get pods
NAME                                                  READY   STATUS    RESTARTS   AGE
event-display-001-00001-deployment-68cdddddcb-hj8pg   2/2     Running   0          98m
event-display-002-00001-deployment-8769bf8bd-nfl9q    2/2     Running   0          98m

[root@xianchaomaster1 ~]# kubectl logs -f event-display-001-00001-deployment-68cdddddcb-hj8pg
☁️  cloudevents.Event
Context Attributes,
  specversion: 1.0
  type: com.xks.sayhi001
  source: sendoff
  id: say-hi
Extensions,
  knativearrivaltime: 2023-07-07T04:22:23.984876655Z
Data,
   {"msg":"Hello Knative default Broker Say HI"}

#2.【Curl请求 Hello Knative default Broker Say BY】
[root@xianchaomaster1 ~]# kubectl run client-$RANDOM --image=ikubernetes/admin-box:v1.2 --restart=Never -it --command -- /bin/sh
curl -v "http://broker-ingress.knative-eventing.svc.cluster.local/default/br01" -X POST -H "Content-Type: application/cloudevents+json" \
-d '{"id": "say-bye", "specversion": "1.0", "type": "com.xks.saybye002", "source": "sendoff", "data": {"msg":"Hello Knative default Broker Say BYE"}}'Note: Unnecessary use of -X or --request, POST is already inferred.

#测试验证  event-display-002-00001-deployment-8769bf8bd-nfl9q 收到 Hello Knative default Broker Say BYE 、不会收到 Hello Knative default Broker Say HI
[root@xianchaomaster1 ~]# kubectl get pods
event-display-001-00001-deployment-68cdddddcb-hj8pg   2/2     Running   0          98m
event-display-002-00001-deployment-8769bf8bd-nfl9q    2/2     Running   0          98m

[root@xianchaomaster1 ~]# kubectl logs -f event-display-002-00001-deployment-8769bf8bd-nfl9q
☁️  cloudevents.Event
Context Attributes,
  specversion: 1.0
  type: com.xks.saybye002
  source: sendoff
  id: say-bye
Extensions,
  knativearrivaltime: 2023-07-07T04:23:32.512395477Z
Data,
   {"msg":"Hello Knative default Broker Say BYE"}

另：Yaml资源

#Broker资源配置文件
piVersion: eventing.knative.dev/v1
kind: Broker
metadata:
  name: default
  namespace: event-demo

#Trigger资源配置文件
#trigger1
apiVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: trigger1
  namespace: event-demo
spec:
  broker: default
  filter:
    attributes:
      type: com.magedu.com.sayhi
  subscriber:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display-hi

#trigger2
piVersion: eventing.knative.dev/v1
kind: Trigger
metadata:
  name: trigger2
  namespace: event-demo
spec:
  broker: default
  filter:
    attributes:
      type: com.magedu.com.saybye
  subscriber:
    ref:
      apiVersion: serving.knative.dev/v1
      kind: Service
      name: event-display-bye

总结

消息传递模式的总结：
    一对多（不过滤）：
        Channel/Subscription CRD 
            messaging.knative.dev/v1 

            生产者要把事件传递给Channel，由Channel实现“一对多”的分发； 

        sources --> Channel <-- Subscription --> KSVC/SVC 

    一对多（过滤）：
        Broker/Trigger CRD 
            eventing.knative.dev/v1 

            生产者要把事件传递给Broker，由Trigger基于过滤条件从Broker获取消息集合中的一个满足过滤条件子集，并将过滤后的消息发给该Trigger的Subscriber；

        sources --> Broker <-- Trigger --> KSVC/SVC 

        补充：
            定义Trigger时，filter为可选；即不对消息进行过滤，而是将所有事件都发给该Trigger的订阅者； 

    分布式的EDA应用：
        A --> B 
        A --> C