Envoy-可观测性之日志采集和存储【十八】

示例：

1、Readme

[root@k8s-node02 accesslog-with-efk]# cat README.md
## 使用EFK收集Envoy的访问日志
### 环境说明
##### Envoy Mesh使用的网络: 172.31.76.0/24
##### 7个Service:
- front-envoy：Front Proxy,地址为172.31.76.10
- 3个后端服务，仅是用于提供测试用的上游服务器
  - service_blue
  - service_red
  - service_green
- 三个日志服务
  - elasticsearch，地址为172.31.76.15，绑定宿主机的9200端口
  - kibana，地址为172.31.76.16，绑定宿主机的5601端口
  - filebeat
##### 特殊要求
目录logs/envoy/下的日志文件front-envoy-access.log的属主需要修改为envoy容器中运行envoy进程的用户envoy，其UID和GID默认分别为100和101，否则，front-envoy进程将日志写入到该文件时，将显示为“Permission Denied.”

```
chown 100.101 logs/envoy/front-envoy-access.log
```
### 运行并测试
1.  启动服务

   ```
   docker-compose up
   ```
2. 文本日志
   先使用类似如下命令向Front-Envoy发起请求，以便持续生成访问日志；

   ```
   while true; do curl 172.31.76.10/service/colors; sleep 0.$RANDOM; done
   ```
3. 确认ElasticSearch服务正常工作，且Filebeat已经输出日志信息到指定的索引中

   ```
   curl 172.31.76.15:9200
   # 正常运行的ElasticSearch将返回类似如下内容
   {
     "name" : "myes01",
     "cluster_name" : "myes",
     "cluster_uuid" : "QSAkdrV-QziRgGuZUgbCyg",
     "version" : {
       "number" : "7.14.2",
       "build_flavor" : "default",
       "build_type" : "docker",
       "build_hash" : "6bc13727ce758c0e943c3c21653b3da82f627f75",
       "build_date" : "2021-09-15T10:18:09.722761972Z",
       "build_snapshot" : false,
       "lucene_version" : "8.9.0",
       "minimum_wire_compatibility_version" : "6.8.0",
       "minimum_index_compatibility_version" : "6.0.0-beta1"
     },
     "tagline" : "You Know, for Search"
   }
   ```
   查看是否已经存在由filebeat生成的索引；

   ```
   curl 172.31.76.15:9200/_cat/indices
   # 命令返回的索引中包含类似如下内容，即表示filebeat已经生成相应的索引
   ……
   yellow open filebeat-2021.11.03               VTqwVr_8RD2k8aGal-YCHg 1 1 44   0 80.6kb 80.6kb
   ……
   ```
4. 访问Kibana
   在宿主机可以访问Kibana，端口为5601
   ![kb01](images/kb-create-index-pattern-001.png)
   第二步：
   ![kb02](images/kb-create-index-pattern-002.png)
   第三步：
   ![kb03](images/kb-create-index-pattern-003.png)
   第四步：
   ![kb04](images/kb-create-index-pattern-004.png)
   第五步：再次访问Discover，即可看到由filebeat收集的envoy的日志
   ![kb05](images/kb-discover-001.png)
5.  进一步操作
    可采用类似于Front-Envoy的方式，将service_blue、service_red和service_green的日志都收集到ElasticSearch之中。
6. 停止后清理

测试

#1、文本日志
[root@k8s-node02 statsd-sink-and-prometheus]#    while true; do curl 172.31.76.10/service/colors; sleep 0.$RANDOM; done
Hello from App behind Envoy (service blue)! hostname: 23db84a90ff3 resolved hostname: 172.31.76.2
Hello from App behind Envoy (service green)! hostname: 67d02b32a362 resolved hostname: 172.31.76.4
Hello from App behind Envoy (service green)! hostname: 67d02b32a362 resolved hostname: 172.31.76.4
Hello from App behind Envoy (service blue)! hostname: 23db84a90ff3 resolved hostname: 172.31.76.2
Hello from App behind Envoy (service red)! hostname: 5a57c53b5bcf resolved hostname: 172.31.76.3
Hello from App behind Envoy (service green)! hostname: 67d02b32a362 resolved hostname: 172.31.76.4
Hello from App behind Envoy (service blue)! hostname: 23db84a90ff3 resolved hostname: 172.31.76.2
Hello from App behind Envoy (service red)! hostname: 5a57c53b5bcf resolved hostname: 172.31.76.3
Hello from App behind Envoy (service red)! hostname: 5a57c53b5bcf resolved hostname: 172.31.76.3

#2、确认ElasticSearch服务正常工作，且Filebeat已经输出日志信息到指定的索引中
[root@k8s-node02 statsd-sink-and-prometheus]#  curl 172.31.76.15:9200
{
  "name" : "myes01",
  "cluster_name" : "myes",
  "cluster_uuid" : "Maksqg3ETxKF2s6R5ibWDw",
  "version" : {
    "number" : "7.17.5",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "8d61b4f7ddf931f219e3745f295ed2bbc50c8e84",
    "build_date" : "2022-06-23T21:57:28.736740635Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}
   
#3、查看是否已经存在由filebeat生成的索引；
[root@k8s-node02 statsd-sink-and-prometheus]# curl 172.31.76.15:9200/_cat/indices
green  open .geoip_databases                  b0TaWLy4TQC-sprEs2_uzw 1 0   39  0  36.3mb  36.3mb
yellow open filebeat-2024.02.26               2rz-e-IMSDymvMHV5Nw5Tg 1 1 4117  0   2.5mb   2.5mb
green  open .kibana_7.17.5_001                4CFYOLjsSfa_DHq483WNrQ 1 0   15  1   4.7mb   4.7mb
green  open .apm-custom-link                  HwlNUC5kSOS5caGJJSP7OA 1 0    0  0    226b    226b
green  open .apm-agent-configuration          y8PwuHwTSeuN79M8O25X1g 1 0    0  0    226b    226b
green  open .kibana_task_manager_7.17.5_001   v7RYlumVTZSU6IcXS-_EsQ 1 0   17 93 280.5kb 280.5kb
yellow open filebeat-7.17.5-2024.02.26-000001 KSOqld5uQyefzCiO97ygZA 1 1    0  0    226b    226b
green  open .tasks                            K5TVg72ZRaW2_Ij5vH7c6w 1 0    2  0   7.7kb   7.7kb

#4、访问Kibana

   在宿主机可以访问Kibana，端口为5601
   ![kb01](images/kb-create-index-pattern-001.png)
   第二步：
   ![kb02](images/kb-create-index-pattern-002.png)
   第三步：
   ![kb03](images/kb-create-index-pattern-003.png)
   第四步：
   ![kb04](images/kb-create-index-pattern-004.png)
   第五步：再次访问Discover，即可看到由filebeat收集的envoy的日志
   ![kb05](images/kb-discover-001.png)
   
#5、进一步操作
    可采用类似于Front-Envoy的方式，将service_blue、service_red和service_green的日志都收集到ElasticSearch之中。