ceph之ceph-mgr管理员指南

设置一个ceph-mgr

创建一个身份验证密钥

ceph --cluster ceph auth get-or-create mgr.ceph4 mon 'allow profile mgr' osd 'allow *' mds 'allow *'
[mgr.ceph4]
    key = AQBHFcxciXa/LRAAyuJBnVr8M2ICIwxNPw5XcQ==
 
mkdir /var/lib/ceph/mgr/ceph-ceph4/
 
ceph --cluster ceph auth get-or-create mgr.ceph4 -o /var/lib/ceph/mgr/ceph-ceph4/keyring

　　

启动守护进程

ceph-mgr -i ceph4

　　

 

 

ceph mgr -h

ceph mgr -h

 General usage: 
 ==============
usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE]
            [--id CLIENT_ID] [--name CLIENT_NAME] [--cluster CLUSTER]
            [--admin-daemon ADMIN_SOCKET] [-s] [-w] [--watch-debug]
            [--watch-info] [--watch-sec] [--watch-warn] [--watch-error]
            [--watch-channel WATCH_CHANNEL] [--version] [--verbose]
            [--concise] [-f {json,json-pretty,xml,xml-pretty,plain}]
            [--connect-timeout CLUSTER_TIMEOUT]

Ceph administration tool

optional arguments:
  -h, --help            request mon help
  -c CEPHCONF, --conf CEPHCONF
                        ceph configuration file
  -i INPUT_FILE, --in-file INPUT_FILE
                        input file, or "-" for stdin
  -o OUTPUT_FILE, --out-file OUTPUT_FILE
                        output file, or "-" for stdout
  --id CLIENT_ID, --user CLIENT_ID
                        client id for authentication
  --name CLIENT_NAME, -n CLIENT_NAME
                        client name for authentication
  --cluster CLUSTER     cluster name
  --admin-daemon ADMIN_SOCKET
                        submit admin-socket commands ("help" for help
  -s, --status          show cluster status
  -w, --watch           watch live cluster changes
  --watch-debug         watch debug events
  --watch-info          watch info events
  --watch-sec           watch security events
  --watch-warn          watch warn events
  --watch-error         watch error events
  --watch-channel WATCH_CHANNEL
                        which log channel to follow when using -w/--watch. One
                        of ['cluster', 'audit', '*'
  --version, -v         display version
  --verbose             make verbose
  --concise             make less verbose
  -f {json,json-pretty,xml,xml-pretty,plain}, --format {json,json-pretty,xml,xml-pretty,plain}
  --connect-timeout CLUSTER_TIMEOUT
                        set a timeout for connecting to the cluster

 Local commands: 
 ===============

ping <mon.id>           Send simple presence/life test to a mon
                        <mon.id> may be 'mon.*' for all mons
daemon {type.id|path} <cmd>
                        Same as --admin-daemon, but auto-find admin socket
daemonperf {type.id | path} [stat-pats] [priority] [<interval>] [<count>]
daemonperf {type.id | path} list|ls [stat-pats] [priority]
                        Get selected perf stats from daemon/admin socket
                        Optional shell-glob comma-delim match string stat-pats
                        Optional selection priority (can abbreviate name):
                         critical, interesting, useful, noninteresting, debug
                        List shows a table of all available stats
                        Run <count> times (default forever),
                         once per <interval> seconds (default 1)
    

 Monitor commands: 
 =================
mgr count-metadata <property>                                                                   count ceph-mgr daemons by metadata field property
mgr dump {<int[0-]>}                                                                            dump the latest MgrMap
mgr fail <who>                                                                                  treat the named manager daemon as failed
mgr metadata {<id>}                                                                             dump metadata for all daemons or a specific daemon
mgr module disable <module>                                                                     disable mgr module
mgr module enable <module> {--force}                                                            enable mgr module
mgr module ls                                                                                   list active mgr modules
mgr services                                                                                    list service endpoints provided by mgr modules
mgr versions                                                                                    check running versions of ceph-mgr daemons

　　

根据元数据字段属性计算ceph-mgr守护进程

ceph mgr count-metadata os
{
    "Linux": 1
}

　　

获取mgrmap，默认最新

ceph mgr dump 22	#22可省略，
{
    "epoch": 22,
    "active_gid": 14103,
    "active_name": "ceph4",
    "active_addr": "10.1.1.24:6804/7416",
    "available": true,
    "standbys": [],
    "modules": [
        "balancer",
        "dashboard",
        "restful",
        "status"
    ],
    "available_modules": [
        "balancer",
        "dashboard",
        "influx",
        "localpool",
        "prometheus",
        "restful",
        "selftest",
        "status",
        "zabbix"
    ],
    "services": {
        "dashboard": "http://ceph4:7000/"
    }
}

　　

将mgr.ceph4视为失败，模拟失败，后自动拉起啦

ceph mgr fail ceph4
[root@ceph4 ~]# ceph -s
  cluster:
    id:     1f0490cd-f938-4e20-8ea5-d817d941a6e6
    health: HEALTH_OK
 
  services:
    mon: 1 daemons, quorum ceph4
    mgr: ceph4(active, starting)
    osd: 1 osds: 1 up, 1 in
 
  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0B
    usage:   1.00GiB used, 3.90GiB / 4.90GiB avail
    pgs:

　　

ceph mgr metadata ceph4
[
    {
        "id": "ceph4",
        "arch": "x86_64",
        "ceph_version": "ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable)",
        "cpu": "Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz",
        "distro": "centos",
        "distro_description": "CentOS Linux 7 (Core)",
        "distro_version": "7",
        "hostname": "ceph4",
        "kernel_description": "#1 SMP Thu Nov 8 23:39:32 UTC 2018",
        "kernel_version": "3.10.0-957.el7.x86_64",
        "mem_swap_kb": "2097148",
        "mem_total_kb": "995896",
        "os": "Linux"
    }
]

　　

查看哪些模块可用

ceph mgr module ls
{
    "enabled_modules": [
        "balancer",
        "restful",
        "status"
    ],
    "disabled_modules": [
        "dashboard",
        "influx",
        "localpool",
        "prometheus",
        "selftest",
        "zabbix"
    ]
}

　　

启用模块dashboard

ceph mgr module enable dashboard

　　

ceph mgr module ls
{
    "enabled_modules": [
        "balancer",
        "dashboard",
        "restful",
        "status"
    ],
    "disabled_modules": [
        "influx",
        "localpool",
        "prometheus",
        "selftest",
        "zabbix"
    ]
}

　　

 配置文件修改

/etc/ceph/ceph.conf

[mgr]
mgr modules = dashboard

　　

 

ceph mgr services
{
    "dashboard": "http://ceph4:7000/"
}

　　

 

ceph mgr versions
{
    "ceph version 12.2.11 (26dc3775efc7bb286a1d6d66faee0ba30ea23eee) luminous (stable)": 1
}

　　

 修改mgr监听地址及端口

修改前确认：

ceph config-key  ls
[]

ceph config-key  dump
{}

　　

修改

ceph config-key set mgr/dashboard/ceph4/server_addr 10.1.1.24
set mgr/dashboard/ceph4/server_addr

ceph config-key set mgr/dashboard/ceph4/server_port 8000
set mgr/dashboard/ceph4/server_port

　　

修改后确认

ceph config-key  ls
[
    "mgr/dashboard/ceph4/server_addr",
    "mgr/dashboard/ceph4/server_port"
]
ceph config-key  dump
{
    "mgr/dashboard/ceph4/server_addr": "10.1.1.24",
    "mgr/dashboard/ceph4/server_port": "8000"
}

　　

服务管理

cp /usr/lib/systemd/system/ceph-mgr@.service  /usr/lib/systemcd/system/ceph-mgr@ceph4.service

　　注意：mgr服务命名需要跟ceph -s中mgr定义的命名匹配

 

查看ceph-mgr状态，并重启生效

systemctl status ceph-mgr@ceph4
systemctl restart ceph-mgr@ceph4

　　

简述：

dashboard，luminous版本新特色，由mgr提供的一个web界面，只能够用于监控显示，无法进行信息更改

 

关于高可用

ceph-mgr对于当前ceph版本十分重要：

主要用于管理pg map作用

当ceph-mgr发生故障，相当于整个ceph集群都会出现严重问题

建议在每个mon节点中都创建独立的ceph-mgr（至少3个ceph-mon节点 ）

只需要在每个mon节点进行创建（每个mgr需要不同的独立的命名）

ceph -s

其中一个为active，另外的均为standby

dashboard只可用于active ceph-mgr中

访问dashboard

默认7000