ceph提供rgw
ceph对象网关
1、文件系统和对象存储的区别
-
层次化和扁平化：
-
文件系统允许用户进行增删改查，有一个mds服务器，检索元数据。是一个层次化的存储结构，受限与mds服务器(单个元数据服务器能力，也可以加多个mds服务器)
-
对象存储扁平化存储结构，只有增删查，不允许修改(修改的话，上传一份新的对象)，具有高扩展性，去掉了复杂目录结构，所有数据都分散存储在上千架服务器上，无限扩展(没有中心化设备，不够就加多个服务器)，通过分布式键值索引
-
-
对象包含数据本身，元数据，全局唯一id，放弃了传统的目录结构，将数据和元数据放在一起了
1、对象存储使用场景和接口标准
-
图片
-
视频
-
镜像文件
-
大文件，修改少(基本不修改)
-
接口标准
-
s3 aws提出来的，行业标准
-
swift openstack提出来的
-
2、桶(bucket)
-
桶是对象存储的一个逻辑概念，所有对象都是存储存放在桶里面的，用户可见的
-
之前说过，ceph上所有的数据都是存储在pg上的，那么到底是存储在pg，还是桶呢？
-
pg是ceph上面的概念
-
桶是rgw上面的概念
-
其实最后的数据还是存储在osd上
-
对象存储不管你底层是把数据往哪放，反正我，往桶里面放
-
2、rgw(rados gateway)
-
负责接收http请求
-
实现桶的逻辑(创建等)
-
用户认证，桶的认证，ak,sk
rgw不属于这个ceph服务端的组件，属于ceph客户端组件，这样做是因为对象存储是提供的http地址，然后这个ceph服务端是没有提供http地址的，所以将rgw放在客户端上面了，当用户请求这个http地址时，用户此时并不需要安装ceph客户端，只需要发起http请求，客户端再去请求ceph集群即可
1、对象存储认证
-
这个是非常重要到，因为这个认证，有权限才能操作对象存储，生产环境中也是这样到
-
对象存储认证是基于桶的认证，与这个cephx认证是不一样的
-
需要用户提供ak,sk,这个是用户与rgw的认证的关系
-
用户需要存放一个文件到ceph到对象存储桶里面，需要去请求ceph客户端，并与ceph客户端rgw完成认证，此时ceph客户端再去与ceph服务端完成cephx认证
2、对象网关所涉及到概念
-
realm 领域
- 一个realm可以有多个区域，但是只有一个master zonegroup,同一个realm下到同一个zonegroup的多个zone之间的数据是自动同步的
-
zonegroup区域组
- 一个zonegroup中有一个或者多个zone,但是只有一个主zone,同一个zonegrouo下面的多个zone会自动同步数据
-
zone区域
-
一个zone有多个rgw ，就是提供访问的地址
-
最小的数据单元，有自己的数据池和元数据存储池
-
-
rgw
- 对外提供服务的窗口
3、关于可用区和ceph集群的关系
-
一个zone一定只属于一个ceph集群
-
一个ceph集群当中可以有多个zone
3、部署对象网关
-
流程
-
在服务端创建realm,zonegroup,zone
-
上面的操作就会有很多的存储池，然后更新这个realm即可
-
创建rgw服务即可，部署一个即可
-
测试访问对象存储即可
-
使用对象存储
-
需要创建用户，这个用户是s3类型的，因此的话，还需要创建这个swift类型的，可以创建这个子用户即可
-
桶的创建是需要这个rgw客户端来创建的，也就是使用swift
-
桶的创建是需要这个s3或者这个swift来创建的，这2个都是协议，只是访问形式的不同而已，rgw支持这2个的协议
-
-
realm (域)
zonegroup(区域组)
zone(区域)
rgw(实例)
1、创建realm域
[root@ceph01 ~]# radosgw-admin realm create --rgw-realm=for_swift
{
"id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4", # 唯一id，标识不同的realm
"name": "for_swift", # realm的名称
"current_period": "e96be4aa-b502-4407-a0a5-c6e9bb737d38", # 配置文件的id 当前period的id，配置文件
"epoch": 1 # 版本号。变更时，是递增的
}
# radosgw-admin 是ceph对象网关管理工具
# realm create 创建一个新的realm
# --rgw-realm 指定realm的名称为for_swift
- 创建了这个realm,里面会有这个zonegroup,zone这些，这些信息都存储在配置文件里面，会有一个配置文件记录这些的，peri记录，会有一个id,后面创建了zonegroup,zone,rgw,这个配置文件就需要更新
1、列出所有的realm
[root@ceph01 ~]# radosgw-admin realm list
{
"default_info": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4", # realm的id
"realms": [
"for_swift" # realm为for_swift
]
}
2、列出某个realm的详细信息
[root@ceph01 ~]# radosgw-admin realm get --rgw-realm for_swift
{
"id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"name": "for_swift",
"current_period": "e96be4aa-b502-4407-a0a5-c6e9bb737d38",
"epoch": 1
}
3、删除某个realm
# 创建一个realm
[root@ceph01 ~]# radosgw-admin realm create --rgw-realm test
{
"id": "905d5f14-40df-4e29-8f46-fd584dea3100",
"name": "test",
"current_period": "9c566df3-116d-434c-b226-0d3bd792e240",
"epoch": 1
}
# 删除realm
[root@ceph01 ~]# radosgw-admin realm delete --rgw-realm test
2、创建一个zonegroup
[root@ceph01 ~]# radosgw-admin zonegroup create --rgw-realm for_swift --rgw-zonegroup for_swift
{
"id": "67001b3c-cc81-4051-a449-f4bea5238226", # zonegroup的id
"name": "for_swift",
"api_name": "for_swift",
"is_master": "false", # 不是主zonegroup
"endpoints": [],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "",
"zones": [],
"placement_targets": [],
"default_placement": "",
"realm_id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4", # realm的id
"sync_policy": {
"groups": []
}
}
1、修改zonegroup为主
[root@ceph01 ~]# radosgw-admin zonegroup modify --rgw-realm for_swift --rgw-zonegroup for_swift --default --master
{
"id": "67001b3c-cc81-4051-a449-f4bea5238226",
"name": "for_swift",
"api_name": "for_swift",
"is_master": "true", # 是主master了
"endpoints": [],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "",
"zones": [],
"placement_targets": [],
"default_placement": "",
"realm_id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"sync_policy": {
"groups": []
}
}
2、查看所有zonegroup
[root@ceph01 ~]# radosgw-admin zonegroup list
{
"default_info": "67001b3c-cc81-4051-a449-f4bea5238226",
"zonegroups": [
"for_swift"
]
}
3、创建zone
# 创建主zone
# 可以发现创建了很多的存储池
[root@ceph01 ~]# radosgw-admin zone create --rgw-realm for_swift --rgw-zonegroup for_swift --rgw-zone for_swift --master --default
{
"id": "e5e86566-9261-4b07-b830-9cb622a8a8de",
"name": "for_swift",
"domain_root": "for_swift.rgw.meta:root",
"control_pool": "for_swift.rgw.control",
"gc_pool": "for_swift.rgw.log:gc",
"lc_pool": "for_swift.rgw.log:lc",
"log_pool": "for_swift.rgw.log",
"intent_log_pool": "for_swift.rgw.log:intent",
"usage_log_pool": "for_swift.rgw.log:usage",
"roles_pool": "for_swift.rgw.meta:roles",
"reshard_pool": "for_swift.rgw.log:reshard",
"user_keys_pool": "for_swift.rgw.meta:users.keys",
"user_email_pool": "for_swift.rgw.meta:users.email",
"user_swift_pool": "for_swift.rgw.meta:users.swift",
"user_uid_pool": "for_swift.rgw.meta:users.uid",
"otp_pool": "for_swift.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "for_swift.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "for_swift.rgw.buckets.data"
}
},
"data_extra_pool": "for_swift.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"notif_pool": "for_swift.rgw.log:notif"
}
1、查看存储池
[root@ceph01 ~]# ceph osd pool ls
device_health_metrics
.rgw.root
2、更新配置
# 指定默认的realm
[root@ceph01 ~]# radosgw-admin period update --commit --rgw-realm for_swift
3、验证id是否变更
# 发现变更了
[root@ceph01 ~]# radosgw-admin realm get
{
"id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"name": "for_swift",
"current_period": "9d2718bc-6906-4c73-8c72-4906a0cfed68", # 当前配置文件的id
"epoch": 2 # 版本号为2
}
4、再次查看存储池
[root@ceph01 ~]# ceph osd pool ls
device_health_metrics
.rgw.root
for_swift.rgw.log
for_swift.rgw.control
for_swift.rgw.meta
4、创建rgw
# rgw的实例为 for_swift
[root@ceph01 ~]# ceph orch apply rgw for_swift --placement="1"
Scheduled rgw.for_swift update...
# placement 指的是部署几个rgw
# 默认监听在80端口，如果想要修改的话，就加上--port
1、访问端口
# 发现rgw服务部署在ceph2上面
[root@ceph01 ~]# ceph orch ps | grep rgw
rgw.for_swift.ceph02.efwkys ceph02 *:80 running (3m) 3m ago 3m 46.0M - 16.2.13 e08a45948779 d44f256e2b5c
# 访问ceph2
[root@ceph01 ~]# curl 192.168.50.21
<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>[root@ceph01 ~]#
# 有正常的返回值
# 过了一段时间，创建了许多的存储池
[root@ceph01 ~]# ceph osd pool ls
device_health_metrics
.rgw.root
for_swift.rgw.log
for_swift.rgw.control
for_swift.rgw.meta
default.rgw.log
default.rgw.control
default.rgw.meta
4、使用对象存储
1、用户管理
1、创建用户
[root@ceph01 ~]# radosgw-admin user create --uid=testuser --display-name=testuser
{
"user_id": "testuser", # 唯一标识
"display_name": "testuser", # 展示的用户名，描述信息
"email": "",
"suspended": 0, # 0 表示正常的状态，1表示被暂停，无法访问
"max_buckets": 1000, # 用户可以创建最大的bucket数量
"subusers": [],
"keys": [ # 3c认证
{
"user": "testuser",
"access_key": "DT5VSYMD7M08E1O34R3H", # 用户名
"secret_key": "Pk1Vy88xiE2EjGwqGFsCqU1XVheVBfg8KLMjmHrD" # 密码
}
],
# 返回了ak,sk 这个是s3标准的，swift使用不了，这个用户是s3用户
# 使用swift的话，需要创建子用户
# --access-key= 手动创建ak
# --secret-key= 手动创建sk
1、给用户生成一个key
- 上面是自动的生成一个key,这次增加一个key即可
[root@ceph01 ~]# radosgw-admin key create --uid=testuser --key-type s3 --gen-access-key --gen-secret
{
"user_id": "testuser",
"display_name": "testuser",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [],
"keys": [
{
"user": "testuser",
"access_key": "DT5VSYMD7M08E1O34R3H",
"secret_key": "Pk1Vy88xiE2EjGwqGFsCqU1XVheVBfg8KLMjmHrD"
},
{
"user": "testuser", # 第二个ak,sk
"access_key": "J40RSQQR36M60G5EU0HV",
"secret_key": "JYuRJbXgFCgfCtiwCuKDXIf6mSOOeDbFgDeP8dC7"
}
],
2、查看用户列表
# 2个用户，一个是图形化的用户
[root@ceph01 ~]# radosgw-admin user list
[
"dashboard",
"testuser"
]
3、创建子用户
# --uid 主用户(已经存在的s3用户)，必须是已经创建了用户
# --subuser 主用户:子用户
[root@ceph01 ~]# radosgw-admin subuser create --uid=testuser --subuser testuser:swift
{
"user_id": "testuser",
"display_name": "testuser",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [
{
"id": "testuser:swift",
"permissions": "<none>"
}
],
"keys": [
{
"user": "testuser",
"access_key": "DT5VSYMD7M08E1O34R3H",
"secret_key": "Pk1Vy88xiE2EjGwqGFsCqU1XVheVBfg8KLMjmHrD"
},
{
"user": "testuser",
"access_key": "J40RSQQR36M60G5EU0HV",
"secret_key": "JYuRJbXgFCgfCtiwCuKDXIf6mSOOeDbFgDeP8dC7"
}
],
"swift_keys": [ # swift的密钥信息
{
"user": "testuser:swift",
"secret_key": "OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW"
}
],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"default_storage_class": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw",
"mfa_ids": []
}
# 包含在主用户里面了，查看主用户，就相当于是查看子用户
4、子用户权限
root@ceph01 ~]# radosgw-admin subuser modify --uid=testuser --subuser testuser:swift --access full
{
"user_id": "testuser",
"display_name": "testuser",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [
{
"id": "testuser:swift",
"permissions": "full-control" # 设置完全控制了
}
],
"keys": [
{
"user": "testuser",
"access_key": "DT5VSYMD7M08E1O34R3H",
"secret_key": "Pk1Vy88xiE2EjGwqGFsCqU1XVheVBfg8KLMjmHrD"
},
{
"user": "testuser",
"access_key": "J40RSQQR36M60G5EU0HV",
"secret_key": "JYuRJbXgFCgfCtiwCuKDXIf6mSOOeDbFgDeP8dC7"
}
],
"swift_keys": [
{
"user": "testuser:swift",
"secret_key": "OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW"
}
],
"caps": [],
"op_mask": "read, write, delete",
"default_placement": "",
"default_storage_class": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw",
"mfa_ids": []
}
2、bucket管理
- bucket创建必须使用rgw客户端来创建，必须创建客户端
[root@ceph01 ~]# yum -y install python3-pip
[root@ceph01 ~]# pip install python-swiftclient -i https://pypi.tuna.tsinghua.edu.cn/simple
1、测试子用户权限
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW stat -v --insecure
StorageURL: http://192.168.50.21/swift/v1
Auth Token: AUTH_rgwtk0e00000074657374757365723a737769667485cb671f7d2d9563a386be6934b7b235169082d2acd577d167e69349c6b06e9cf14cb50d
Account: v1
Containers: 0
Objects: 0
Bytes: 0
Objects in policy "default-placement-bytes": 0
Bytes in policy "default-placement-bytes": 0
Containers in policy "default-placement": 0
Objects in policy "default-placement": 0
Bytes in policy "default-placement": 0
X-Timestamp: 1774007587.90565
X-Account-Bytes-Used-Actual: 0
X-Trans-Id: tx00000a99f792023171e6d-0069bd3523-2d0b7-for_swift
X-Openstack-Request-Id: tx00000a99f792023171e6d-0069bd3523-2d0b7-for_swift
Accept-Ranges: bytes
Content-Type: text/plain; charset=utf-8
Connection: Keep-Alive
# -A auth url 认证地址
# -U 用户名(主用户名：子用户格式)
# -K swift密钥
# stat -v 查看账户状态(详细模式)
# --insecure 允许不安全的ssl连接，http连接的
2、列出桶
# 发现是空的，没有桶
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW list
3、创建桶
# post 桶名 创建桶
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW post hosts
# 查看桶，test桶
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW list
hosts
3、数据操作
1、上传对象
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW upload hosts /etc/hosts
etc/hosts
2、查看容器内容
[root@ceph01 ~]# swift -A http://192.168.50.21/auth/1.0 -U 'testuser:swift' -K OjNACUklB1ddEb6xjocDGnqVilQOjm4EnrUA0tqW list hosts -v
etc/hosts
3、使用s3协议
-
下载s3 browser客户端，直接连接
-
https://s3browser.com/s3browser-first-run.aspx?v=13.1.1&fam=x64
-
因为默认文件都是私有的，然后将文件的权限设置为所有人可以查看
-
浏览器就能访问，下载了
5、配置多区域网关
-
上面的实验实现一个realm，zonegroup,一个zone，一个rgw，创建一个桶，实现了对象存储的方式
-
后面这个多区域网关的话，就是实现了不同的zone的数据同步(在同一个realm，zonegroup里面)，所有zone的数据都保持一致
-
这个是不同的ceph集群，还有一个单节点的ceph集群，
-
应用场景,异地容灾
-
流程
-
ceph2集群需要与ceph1集群同步信息，就需要cep2集群先同步ceph1的realm和zonegrpup
-
然后自己创建一个zone,在创建rgw
-
创建完成后，2个集群会自动的同步，第一次是完全同步，后续是增量同步
-
1、原集群操作
-
主zone段配置
-
添加一个系统用户
-
修改主zone,将系统用户和zone绑定
-
修改rgw的配置，明确其realm，zonegroup，zone
-
更新period
-
重启rgw服务/进程
-
1、创建系统账户
-
需要创建用户的原因是为了安全，不然的话别人只要知道集群的IP就能够同步的话数据就会泄露，所以需要认证，并且用户还必须是系统用户
-
负责在不同集群间执行数据同步任务，确保同步安全
[root@ceph01 ~]# radosgw-admin user create --uid=system-user --display-name system-user --system
"keys": [
{
"user": "system-user",
"access_key": "0LSRKSDIF5ADS8TVLT2Q",
"secret_key": "yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz"
}
],
2、绑定用户与zone
-
配置主zone
-
系统用户绑定这个zone
# endpoints 指定rgw地址，有几个写几个
# ak 系统用户的ak, sk 系统用户的sk
[root@ceph01 ~]# radosgw-admin zone modify --rgw-realm for_swift --rgw-zonegroup for_swift --rgw-zone for_swift --master --default --access-key "0LSRKSDIF5ADS8TVLT2Q" --secret "yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz" --endpoints "http://192.168.50.21"
{
"id": "e5e86566-9261-4b07-b830-9cb622a8a8de",
"name": "for_swift",
"domain_root": "for_swift.rgw.meta:root",
"control_pool": "for_swift.rgw.control",
"gc_pool": "for_swift.rgw.log:gc",
"lc_pool": "for_swift.rgw.log:lc",
"log_pool": "for_swift.rgw.log",
"intent_log_pool": "for_swift.rgw.log:intent",
"usage_log_pool": "for_swift.rgw.log:usage",
"roles_pool": "for_swift.rgw.meta:roles",
"reshard_pool": "for_swift.rgw.log:reshard",
"user_keys_pool": "for_swift.rgw.meta:users.keys",
"user_email_pool": "for_swift.rgw.meta:users.email",
"user_swift_pool": "for_swift.rgw.meta:users.swift",
"user_uid_pool": "for_swift.rgw.meta:users.uid",
"otp_pool": "for_swift.rgw.otp",
"system_key": {
"access_key": "0LSRKSDIF5ADS8TVLT2Q",
"secret_key": "yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz"
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "for_swift.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "for_swift.rgw.buckets.data"
}
},
"data_extra_pool": "for_swift.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"notif_pool": "for_swift.rgw.log:notif"
}
[root@ceph01 ~]#
3、声明realm,zonegroup,zone
- 有几个rgw就需要操作几次
# rgw部署在ceph02上面，所以修改ceph02上面的rgw即可
# 主站点加入进去
[root@ceph01 ~]# ceph config set client.rgw.for_swift.ceph02 rgw_realm for_swift
[root@ceph01 ~]# ceph config set client.rgw.for_swift.ceph02 rgw_zonegroup for_swift
[root@ceph01 ~]# ceph config set client.rgw.for_swift.ceph02 rgw_zone for_swift
4、更新集群的period
[root@ceph01 ~]# radosgw-admin period update --commit
5、重启rgw
[root@ceph01 ~]# ceph orch restart rgw.for_swift
Scheduled to restart rgw.for_swift.ceph02.efwkys on host 'ceph02'
2、secondary配置
-
secondary zone配置
-
拉取realm信息
-
拉取最新的period信息创建自己的zone
-
创建rgw
-
将rgw加入secondary zone
-
更新period
-
重启rgw服务/进程
-
检测是否有数据同步
-
1、拉取realm
[root@client ~]# radosgw-admin realm pull --url http://192.168.50.21 --access-key 0LSRKSDIF5ADS8TVLT2Q --secret yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz
2026-03-20T21:48:49.257+0800 7f3b8a64b880 1 error read_lastest_epoch .rgw.root:periods.9d2718bc-6906-4c73-8c72-4906a0cfed68.latest_epoch
2026-03-20T21:48:49.267+0800 7f3b8a64b880 1 Set the period's master zonegroup 67001b3c-cc81-4051-a449-f4bea5238226 as the default
{
"id": "a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4",
"name": "for_swift",
"current_period": "9d2718bc-6906-4c73-8c72-4906a0cfed68",
"epoch": 2
}
# 拉取完成之后在本地可以使用 radosgw-admin realm list 去查看
2、同步period
[root@client ~]# radosgw-admin period pull --url http://192.168.50.21 --access-key 0LSRKSDIF5ADS8TVLT2Q --secret yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz
3、创建zone
[root@client ~]# radosgw-admin zone create --rgw-realm for_swift --rgw-zonegroup for_swift --rgw-zone for_swift_backup --access-key 0LSRKSDIF5ADS8TVLT2Q --secret yxnMaf3VVsQUAQhu2SCStgHeyeShsYI5EvaYxhkz --endpoints "http://192.168.50.30"
4、创建rgw网关
[root@client ~]# ceph orch apply rgw for_swift_backup --placement="1"
Scheduled rgw.for_swift_backup update...
5、将网关加入zone
# 从站点 zone=for_swift_backup
# 将rgw加入到realm,zonegroup里面
# # rgw加入到zone里面去
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_realm for_swift
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_zonegroup for_swift
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_zone for_swift_backup
6、更新period
radosgw-admin period update --commit
7、重启服务
[root@client ~]# ceph orch restart rgw.for_swift_backup
Scheduled to restart rgw.for_swift_backup.client.yymetk on host 'client'
8、验证同步
[root@client ~]# radosgw-admin user list
[
"system-user",
"dashboard",
"testuser"
]
# 桶都同步过了
[root@client ~]# radosgw-admin bucket list
[
"hosts"
]
[root@client ~]# radosgw-admin sync status
realm a651323c-4c81-4dc3-bd9d-d4ef9e4e29c4 (for_swift)
zonegroup 67001b3c-cc81-4051-a449-f4bea5238226 (for_swift)
zone 9a820661-aced-4590-9751-0732c94ddaaa (for_swift_backup)
metadata sync syncing
full sync: 0/64 shards
incremental sync: 64/64 shards
metadata is caught up with master
data sync source: e5e86566-9261-4b07-b830-9cb622a8a8de (for_swift)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
data is caught up with source
6、命令查看
1、查看rgw配置
# 第一个ceph集群
[root@ceph01 ~]# ceph config dump | grep rgw.for_swift
client.rgw.for_swift.ceph02 advanced rgw_realm for_swift *
client.rgw.for_swift.ceph02 advanced rgw_zone for_swift *
client.rgw.for_swift.ceph02 advanced rgw_zonegroup for_swift *
client.rgw.for_swift.ceph02.efwkys basic rgw_frontends beast port=80
# 第二个ceph集群
[root@client ~]# ceph config dump | grep rgw.for_swift_backup
client.rgw.for_swift_backup.client advanced rgw_realm for_swift *
client.rgw.for_swift_backup.client advanced rgw_zone for_swift_backup *
client.rgw.for_swift_backup.client advanced rgw_zonegroup for_swift *
client.rgw.for_swift_backup.client.yymetk basic rgw_frontends beast port=80
2、将rgw加入到zone里面
-
没有指定的话，这个rgw会自动的关联default zonegroup, default zone
-
之前设置的zone，zonegroup,为默认的，是针对radosgw-admin 这个命令的
1、在部署rgw的时候，就指定
ceph orch apply rgw for_swift_backup \
--realm=for_swift \
--zonegroup=for_swift \
--zone=for_swift_backup \
--placement="1"
2、部署后指定
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_realm for_swift
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_zonegroup for_swift
[root@client ~]# ceph config set client.rgw.for_swift_backup.client rgw_zone for_swift_backup
浙公网安备 33010602011771号