ceph集群状态 pgs not scrubbed in time

检查集群状态

# ceph -s
  cluster:
    id:     83738b81-56e4-4d34-bdc2-3a60d789d224
    health: HEALTH_WARN
            75 pgs not scrubbed in time
 
  services:
    mon: 3 daemons, quorum master,node01,node02 (age 4m)
    mgr: master(active, since 4m), standbys: node01, node02
    mds:  3 up:standby
    osd: 3 osds: 3 up (since 3m), 3 in (since 11d)
    rgw: 3 daemons active (master, node01, node02)
 
  task status:
 
  data:
    pools:   5 pools, 256 pgs
    objects: 1.58k objects, 3.5 GiB
    usage:   12 GiB used, 288 GiB / 300 GiB avail
    pgs:     256 active+clean
 
  io:
    client:   11 KiB/s wr, 0 op/s rd, 1 op/s wr

scrubbed 功能用于 PG，是为了检测 PG 分布到各 osds 上的数据是否一致，deep-scrubbed 的检测模式除了检测数据的元数据信息外，还会检测数据的内容是否一致，速度较慢且消耗磁盘读取，一般设置在晚上进行scrubbed ，如果超过了设定时间，若有PG未进行deep-scrubbed，就会报警了

解决方法

先查看哪些PG需要deep-scrubbed

# ceph health detail
HEALTH_WARN 7 pgs not scrubbed in time
PG_NOT_SCRUBBED 7 pgs not scrubbed in time
    pg 5.4e not scrubbed since 2023-08-20 15:32:38.610888
    pg 2.10 not scrubbed since 2023-08-20 12:28:51.567203
    pg 4.9 not scrubbed since 2023-08-20 13:25:02.277965
    pg 3.9 not scrubbed since 2023-08-20 15:34:04.989330
    pg 2.b not scrubbed since 2023-08-20 11:37:12.511534
    pg 4.1d not scrubbed since 2023-08-20 15:00:29.771250
    pg 5.20 not scrubbed since 2023-08-20 13:19:41.921986

手动 deep-scrubbed

[root@master ~]# ceph pg deep-scrub 5.4e
instructing pg 5.4e on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 2.10
instructing pg 2.10 on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 4.9
instructing pg 4.9 on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 3.9
instructing pg 3.9 on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 2.b
instructing pg 2.b on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 4.1d
instructing pg 4.1d on osd.2 to deep-scrub
[root@master ~]# ceph pg deep-scrub 5.20
instructing pg 5.20 on osd.2 to deep-scrub

再检查集群状态

[root@master ~]# ceph health detail
HEALTH_OK