【大咖专栏】如何配置CEPH RGW对象存储与公有云同步
新钛云服已为您服务1280天
容灾 (Disaster Recovery),即容灾备份或灾备,是业务连续性系统的一个子集,用于保障 IT 系统在遭受自然灾害、人为操作失误或蓄意破坏后的数据还原和业务恢复。
Ceph RGW 多数据中心(multisite)功能旨在实现异地双活,提供了备份容灾的能力。并且具有多个数据中心供用户选择,存放资源。
主节点在对外提供服务时,用户数据在主节点落盘后即向用户回应“写成功”应答,然后实时记录数据变化的相关日志信息。备节点则实时比较主备数据差异,并及时将差异化数据拉回备节点。异步复制技术适用于远距离的容灾方案,对系统性能影响较小。
以往的容灾都是在单该或者多该ceph集群之间的,本次主要由于模拟 ceph 的 RGW 可以与其他所有的S3云供应商(https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/)进行数据同步,从而实现Ceph的RGW对象存储的灾备。
内容
要求:
Ceph 集群 2 个 RGW 守护进程正在运行 S3 目标
我们将使用三个端点:
http://192.168.112.5:80 RGW 为我们现有的集群管理的端点 http://192.168.112.6:80 我们将创建一个新端点,它将依赖 RGW 将数据同步到另一个 S3 提供者 http://192.168.105.5:80 我们将用于推送/同步数据的 S3 目标
检查我们现有的存储池:
所有 rgw 都有一个区域和区域组,它可能是
default
.您可以检查池的名称:
(docker-croit)@mycephcluster / $ ceph osd lspools
1 device_health_metrics
2 .rgw.root
3 default.rgw.control
4 default.rgw.log
5 default.rgw.meta
6 default.rgw.buckets.non-ec
7 default.rgw.buckets.index
8 default.rgw.buckets.data
1 device_health_metrics
2 .rgw.root
3 default.rgw.control
4 default.rgw.log
5 default.rgw.meta
6 default.rgw.buckets.non-ec
7 default.rgw.buckets.index
8 default.rgw.buckets.data
或者直接检查区域或区域组:
(docker-croit)@mycephcluster / $ radosgw-admin zone get --rgw-zone=default
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
...
}
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
...
}
(docker-croit)@mycephcluster / $ radosgw-admin zonegroup get --rgw-zonegroup=default
{
"id": "881cf806-f6d2-47a0-b7dc-d65ee87f8ef4",
"name": "default",
"api_name": "default",
"is_master": "true",
...
"zones": [
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
{
"id": "881cf806-f6d2-47a0-b7dc-d65ee87f8ef4",
"name": "default",
"api_name": "default",
"is_master": "true",
...
"zones": [
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
准备存储池:
我们的新区将需要一些池。我们将手动创建它们以确保没有问题(例如每个 osd 有太多 PG)阻止它们的创建。
(docker-croit)@mycephcluster / $ for pool in sync.rgw.meta sync.rgw.log sync.rgw.control sync.rgw.buckets.non-ec sync.rgw.buckets.index sync.rgw.buckets.data; do ceph osd pool create $pool 16 16 replicated; done
pool 'sync.rgw.meta' created
pool 'sync.rgw.log' created
pool 'sync.rgw.control' created
pool 'sync.rgw.buckets.non-ec' created
pool 'sync.rgw.buckets.index' created
pool 'sync.rgw.buckets.data' created
pool 'sync.rgw.meta' created
pool 'sync.rgw.log' created
pool 'sync.rgw.control' created
pool 'sync.rgw.buckets.non-ec' created
pool 'sync.rgw.buckets.index' created
pool 'sync.rgw.buckets.data' created
创建新区域:
我们的新区域将命名为
sync
:(docker-croit)@mycephcluster / $ radosgw-admin zone create --rgw-zonegroup=default --rgw-zone=sync --endpoints=http://192.168.112.6/ --tier-type=cloud
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"domain_root": "sync.rgw.meta:root",
"control_pool": "sync.rgw.control",
"gc_pool": "sync.rgw.log:gc",
"lc_pool": "sync.rgw.log:lc",
"log_pool": "sync.rgw.log",
"intent_log_pool": "sync.rgw.log:intent",
"usage_log_pool": "sync.rgw.log:usage",
"roles_pool": "sync.rgw.meta:roles",
"reshard_pool": "sync.rgw.log:reshard",
"user_keys_pool": "sync.rgw.meta:users.keys",
"user_email_pool": "sync.rgw.meta:users.email",
"user_swift_pool": "sync.rgw.meta:users.swift",
"user_uid_pool": "sync.rgw.meta:users.uid",
"otp_pool": "sync.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "sync.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "sync.rgw.buckets.data"
}
},
"data_extra_pool": "sync.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"domain_root": "sync.rgw.meta:root",
"control_pool": "sync.rgw.control",
"gc_pool": "sync.rgw.log:gc",
"lc_pool": "sync.rgw.log:lc",
"log_pool": "sync.rgw.log",
"intent_log_pool": "sync.rgw.log:intent",
"usage_log_pool": "sync.rgw.log:usage",
"roles_pool": "sync.rgw.meta:roles",
"reshard_pool": "sync.rgw.log:reshard",
"user_keys_pool": "sync.rgw.meta:users.keys",
"user_email_pool": "sync.rgw.meta:users.email",
"user_swift_pool": "sync.rgw.meta:users.swift",
"user_uid_pool": "sync.rgw.meta:users.uid",
"otp_pool": "sync.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "sync.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "sync.rgw.buckets.data"
}
},
"data_extra_pool": "sync.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
rgw-zonegroup:我们的新区域将成为默认区域组的一部分。 端点:我们的新区域需要它自己的 RGW,因此它使用新的端点。 tier-type :我们使用 cloud
层类型,请参阅文档(https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/)了解更多设置
修改现有区域:
我们需要添加现有
default
区域的端点。(docker-croit)@mycephcluster / $ radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=default --endpoints=http://192.168.112.5:80
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
"domain_root": "default.rgw.meta:root",
"control_pool": "default.rgw.control",
"gc_pool": "default.rgw.log:gc",
"lc_pool": "default.rgw.log:lc",
"log_pool": "default.rgw.log",
"intent_log_pool": "default.rgw.log:intent",
"usage_log_pool": "default.rgw.log:usage",
"roles_pool": "default.rgw.meta:roles",
"reshard_pool": "default.rgw.log:reshard",
"user_keys_pool": "default.rgw.meta:users.keys",
"user_email_pool": "default.rgw.meta:users.email",
"user_swift_pool": "default.rgw.meta:users.swift",
"user_uid_pool": "default.rgw.meta:users.uid",
"otp_pool": "default.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "default.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "default.rgw.buckets.data"
}
},
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
"domain_root": "default.rgw.meta:root",
"control_pool": "default.rgw.control",
"gc_pool": "default.rgw.log:gc",
"lc_pool": "default.rgw.log:lc",
"log_pool": "default.rgw.log",
"intent_log_pool": "default.rgw.log:intent",
"usage_log_pool": "default.rgw.log:usage",
"roles_pool": "default.rgw.meta:roles",
"reshard_pool": "default.rgw.log:reshard",
"user_keys_pool": "default.rgw.meta:users.keys",
"user_email_pool": "default.rgw.meta:users.email",
"user_swift_pool": "default.rgw.meta:users.swift",
"user_uid_pool": "default.rgw.meta:users.uid",
"otp_pool": "default.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "default.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "default.rgw.buckets.data"
}
},
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
创建系统用户:
系统用户将用于同步数据,可以通过 CLI 创建此用户。
(docker-croit)@mycephcluster / $ radosgw-admin user create --uid=syncuser --display-name=syncuser --system
{
"user_id": "syncuser",
"display_name": "syncuser",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [],
"keys": [
{
"user": "syncuser",
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
}
],
"swift_keys": [],
"caps": [],
"op_mask": "read, write, delete",
"system": "true",
"default_placement": "",
"default_storage_class": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw",
"mfa_ids": []
}
{
"user_id": "syncuser",
"display_name": "syncuser",
"email": "",
"suspended": 0,
"max_buckets": 1000,
"subusers": [],
"keys": [
{
"user": "syncuser",
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
}
],
"swift_keys": [],
"caps": [],
"op_mask": "read, write, delete",
"system": "true",
"default_placement": "",
"default_storage_class": "",
"placement_tags": [],
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"temp_url_keys": [],
"type": "rgw",
"mfa_ids": []
}
配置同步区域以使用此系统用户:
我们将更改两个区域以使用我们的新系统用户。
(docker-croit)@mycephcluster / $ radosgw-admin user info --uid syncuser| jq '.keys'
[
{
"user": "syncuser",
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
}
]
[
{
"user": "syncuser",
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
}
]
(docker-croit)@mycephcluster / $ radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=default --access-key=VGIF31FGOHZ0Q6MQRBQR --secret=1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
"domain_root": "default.rgw.meta:root",
"control_pool": "default.rgw.control",
"gc_pool": "default.rgw.log:gc",
"lc_pool": "default.rgw.log:lc",
"log_pool": "default.rgw.log",
"intent_log_pool": "default.rgw.log:intent",
"usage_log_pool": "default.rgw.log:usage",
"roles_pool": "default.rgw.meta:roles",
"reshard_pool": "default.rgw.log:reshard",
"user_keys_pool": "default.rgw.meta:users.keys",
"user_email_pool": "default.rgw.meta:users.email",
"user_swift_pool": "default.rgw.meta:users.swift",
"user_uid_pool": "default.rgw.meta:users.uid",
"otp_pool": "default.rgw.otp",
"system_key": {
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "default.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "default.rgw.buckets.data"
}
},
"data_extra_pool": "default.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
(docker-croit)@mycephcluster / $ radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=sync --access-key=VGIF31FGOHZ0Q6MQRBQR --secret=1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"domain_root": "sync.rgw.meta:root",
"control_pool": "sync.rgw.control",
"gc_pool": "sync.rgw.log:gc",
"lc_pool": "sync.rgw.log:lc",
"log_pool": "sync.rgw.log",
"intent_log_pool": "sync.rgw.log:intent",
"usage_log_pool": "sync.rgw.log:usage",
"roles_pool": "sync.rgw.meta:roles",
"reshard_pool": "sync.rgw.log:reshard",
"user_keys_pool": "sync.rgw.meta:users.keys",
"user_email_pool": "sync.rgw.meta:users.email",
"user_swift_pool": "sync.rgw.meta:users.swift",
"user_uid_pool": "sync.rgw.meta:users.uid",
"otp_pool": "sync.rgw.otp",
"system_key": {
"access_key": "VGIF31FGOHZ0Q6MQRBQR",
"secret_key": "1FwPZH0ICfV1e1zi8okXApJJJEB0XHfiOxe1mmTr"
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "sync.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "sync.rgw.buckets.data"
}
},
"data_extra_pool": "sync.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"tier_config": {
"connection": {
"access_key": "JO4RQ1787A6OGI6XMFDW",
"endpoint": "http://192.168.105.5:80",
"secret": "Dx5kKGUUeR0DaSRYueBWhV6oDRvJ9oXH2gPcVJ6s"
}
},
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
确保默认区域是主区域:
(docker-croit)@mycephcluster / $ radosgw-admin zonegroup get
{
"id": "881cf806-f6d2-47a0-b7dc-d65ee87f8ef4",
"name": "default",
"api_name": "default",
"is_master": "true",
"endpoints": [],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "303a00f5-f50d-43fd-afee-aa0503926952",
"zones": [
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
如果
default
zone 不是 master,则可以通过执行强制它radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=default --master --default
提交更改并验证配置:
(docker-croit)@mycephcluster / $ radosgw-admin period update --commit
{
"id": "1861622f-b748-410d-b4a9-7338f4b6842b",
"epoch": 3,
"predecessor_uuid": "b6cd42db-6567-4a4b-9433-aee238da0c9d",
"sync_status": [],
"period_map": {
"id": "1861622f-b748-410d-b4a9-7338f4b6842b",
"zonegroups": [
{
"id": "881cf806-f6d2-47a0-b7dc-d65ee87f8ef4",
"name": "default",
"api_name": "default",
"is_master": "true",
"endpoints": [],
"hostnames": [],
"hostnames_s3website": [],
"master_zone": "303a00f5-f50d-43fd-afee-aa0503926952",
"zones": [
{
"id": "303a00f5-f50d-43fd-afee-aa0503926952",
"name": "default",
"endpoints": [
"http://192.168.112.5:80"
],
"log_meta": "false",
"log_data": "true",
"bucket_index_max_shards": 11,
"read_only": "false",
"tier_type": "",
"sync_from_all": "true",
"sync_from": [],
"redirect_zone": ""
},
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"endpoints": [
"http://192.168.112.6/"
],
"log_meta": "false",
"log_data": "true",
"bucket_index_max_shards": 11,
"read_only": "false",
"tier_type": "cloud",
"sync_from_all": "true",
"sync_from": [],
"redirect_zone": ""
}
],
"placement_targets": [
{
"name": "default-placement",
"tags": [],
"storage_classes": [
"STANDARD"
]
}
],
"default_placement": "default-placement",
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109",
"sync_policy": {
"groups": []
}
}
],
"short_zone_ids": [
{
"key": "303a00f5-f50d-43fd-afee-aa0503926952",
"val": 2796720163
},
{
"key": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"val": 2175446857
}
]
},
"master_zonegroup": "881cf806-f6d2-47a0-b7dc-d65ee87f8ef4",
"master_zone": "303a00f5-f50d-43fd-afee-aa0503926952",
"period_config": {
"bucket_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
},
"user_quota": {
"enabled": false,
"check_on_raw": false,
"max_size": -1,
"max_size_kb": 0,
"max_objects": -1
}
},
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109",
"realm_name": "default",
"realm_epoch": 2
}
配置新区域:
我们的云同步模块需要一些配置(https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/#cloud-sync-tier-type-configuration)。
我们将定义用于同步数据的端点和 S3 用户凭据。注意:如果您的密钥以 0 开头,您将无法配置它。例如,使用
05XXXXXXXX
时 ,访问密钥将被错误地存储:(docker-croit)@mycephcluster / $ radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=sync --tier-config=connection.endpoint=http://192.168.105.5:80,connection.access_key=05XXXXXXXX,connection.secret=56NwS1p7krU0IMYaXXXXXXXXXXXXX
(docker-croit)@mycephcluster / $ radosgw-admin zone get --rgw-zone=sync | jq '.tier_config'
{
"connection": {
"access_key": 5,
"endpoint": "http://192.168.105.5:80",
"secret": 56NwS1p7krU0IMYaXXXXXXXXXXXXX
}
}
(docker-croit)@mycephcluster / $ radosgw-admin zone modify --rgw-zonegroup=default --rgw-zone=sync --tier-config=connection.endpoint=http://192.168.105.5:80,connection.access_key=JO4RQ1787A6OGI6XMFDW,connection.secret=Dx5kKGUUeR0DaSRYueBWhV6oDRvJ9oXH2gPcVJ6s
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"domain_root": "sync.rgw.meta:root",
"control_pool": "sync.rgw.control",
"gc_pool": "sync.rgw.log:gc",
"lc_pool": "sync.rgw.log:lc",
"log_pool": "sync.rgw.log",
"intent_log_pool": "sync.rgw.log:intent",
"usage_log_pool": "sync.rgw.log:usage",
"roles_pool": "sync.rgw.meta:roles",
"reshard_pool": "sync.rgw.log:reshard",
"user_keys_pool": "sync.rgw.meta:users.keys",
"user_email_pool": "sync.rgw.meta:users.email",
"user_swift_pool": "sync.rgw.meta:users.swift",
"user_uid_pool": "sync.rgw.meta:users.uid",
"otp_pool": "sync.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "sync.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "sync.rgw.buckets.data"
}
},
"data_extra_pool": "sync.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"tier_config": {
"connection": {
"access_key": "JO4RQ1787A6OGI6XMFDW",
"endpoint": "http://192.168.105.5:80",
"secret": "Dx5kKGUUeR0DaSRYueBWhV6oDRvJ9oXH2gPcVJ6s"
}
},
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
检查配置是否已正确应用。
(docker-croit)@mycephcluster / $ radosgw-admin zone get --rgw-zone=sync | jq '.tier_config'
{
"connection": {
"access_key": "JO4RQ1787A6OGI6XMFDW",
"endpoint": "http://192.168.105.5:80",
"secret": "Dx5kKGUUeR0DaSRYueBWhV6oDRvJ9oXH2gPcVJ6s"
}
}
提交更改:
(docker-croit)@mycephcluster / $ radosgw-admin zone get --rgw-zone=sync
{
"id": "7ead9532-0938-4698-9b4a-2d84d0d00869",
"name": "sync",
"domain_root": "sync.rgw.meta:root",
"control_pool": "sync.rgw.control",
"gc_pool": "sync.rgw.log:gc",
"lc_pool": "sync.rgw.log:lc",
"log_pool": "sync.rgw.log",
"intent_log_pool": "sync.rgw.log:intent",
"usage_log_pool": "sync.rgw.log:usage",
"roles_pool": "sync.rgw.meta:roles",
"reshard_pool": "sync.rgw.log:reshard",
"user_keys_pool": "sync.rgw.meta:users.keys",
"user_email_pool": "sync.rgw.meta:users.email",
"user_swift_pool": "sync.rgw.meta:users.swift",
"user_uid_pool": "sync.rgw.meta:users.uid",
"otp_pool": "sync.rgw.otp",
"system_key": {
"access_key": "",
"secret_key": ""
},
"placement_pools": [
{
"key": "default-placement",
"val": {
"index_pool": "sync.rgw.buckets.index",
"storage_classes": {
"STANDARD": {
"data_pool": "sync.rgw.buckets.data"
}
},
"data_extra_pool": "sync.rgw.buckets.non-ec",
"index_type": 0
}
}
],
"tier_config": {
"connection": {
"access_key": "JO4RQ1787A6OGI6XMFDW",
"endpoint": "http://192.168.105.5:80",
"secret": "Dx5kKGUUeR0DaSRYueBWhV6oDRvJ9oXH2gPcVJ6s"
}
},
"realm_id": "46669d35-f7ed-4374-8247-2b8f41218109"
}
配置 RGW:
我们需要修改每个 radosgw 配置以通过删除
rgw zone = default
和添加来管理正确的区域[client.rgw.$(hostname)]
host = $(hostname)
rgw zone = default
[client.rgw.$(hostname)]
host = $(hostname)
rgw zone = sync
在我们的基础设施上,我们
/etc/ceph/ceph.conf
通过添加以下内容进行编辑:[client.rgw.new-croit-host-C0DE01]
host = new-croit-host-C0DE01
rgw zone = default
[client.rgw.new-croit-host-C0DE02]
host = new-croit-host-C0DE02
rgw zone = sync
如果你使用 croit,你可以简单地
ceph.conf
用这个内容替换模板。[global]
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
auth supported = cephx
mon client hunt interval = {{huntInterval}}
mon client hunt parallel = {{huntParallel}}
fsid = {{fsid}}
mon host = {{commaSeparatedList mons}}
{{~#if full}}
{{~#unless server.managementHost}}
crush location = host={{server.hostname}}
{{~/unless}}
{{~#if publicNets}}
public network = {{publicNets}}
{{~/if}}
{{~#if privateNets}}
cluster network = {{privateNets}}
{{~/if}}
log file = /dev/null
mon cluster log file = /dev/null
mon cluster log to syslog = true
log to syslog = true
err to syslog = true
{{~#replaceAll "rgw zone = default" ""~}}
{{~#options}}
{{key}} = {{value}}
{{~/options}}
{{~/replaceAll}}
[client.rgw.new-croit-host-C0DE01]
host = new-croit-host-C0DE01
rgw zone = default
[client.rgw.new-croit-host-C0DE02]
host = new-croit-host-C0DE02
rgw zone = sync
{{~/if}}
要应用更改,您必须重新启动 RGW。
root@new-croit-host-C0DE01 ~ $ systemctl restart [email protected]
root@new-croit-host-C0DE02 ~ $ systemctl restart [email protected]
测试 S3 同步
我们将使用 s3cmd 进行测试。
在源和目标上生成用户并配置他们的 s3cmd 配置文件。检查 API 访问是否有效。
~ s3cmd -c .s3cfg_source ls
~ s3cmd -c .s3cfg_target ls
~
如果您有 API 错误错误,请检查您的凭据、端点。
在源上创建一个存储桶并添加对象:
我们首先创建一个bucket:
~ s3cmd -c .s3cfg_source mb s3://mystetbucket
Bucket 's3://mystetbucket/' created
~ s3cmd -c .s3cfg_source ls
2021-06-24 15:06 s3://mystetbucket
现在我们在源上添加一个对象:
~ s3cmd -c .s3cfg_source put /tmp/myobject s3://mystetbucket/synctest 64
WARNING: Module python-magic is not available. Guessing MIME types based on file extensions.
upload: '/tmp/myobject' -> 's3://mystetbucket/synctest' [1 of 1]
13 of 13 100% in 0s 325.90 B/s done
并检查它是否已同步到目标:
~ s3cmd -c .s3cfg_target ls s3://
2021-06-24 15:30 s3://rgw-default-271b93c16a9565d8
~ s3cmd -c .s3cfg_target ls s3://rgw-default-271b93c16a9565d8
DIR s3://rgw-default-271b93c16a9565d8/mystetbucket/
~ s3cmd -c .s3cfg_target ls s3://rgw-default-271b93c16a9565d8/mystetbucket/
2021-06-24 15:36 13 s3://rgw-default-271b93c16a9565d8/mystetbucket/synctest
技巧和窍门
您可以随时增加调试日志记录以便于调试:
root@new-croit-host-C0DE02 ~ $ ceph --admin-daemon /var/run/ceph/ceph-client.rgw.new-croit-host-C0DE02.96866.94534872347832.asok config set debug_rgw_sync 5
root@new-croit-host-C0DE02 ~ $ ceph --admin-daemon /var/run/ceph/ceph-client.rgw.new-croit-host-C0DE02.96866.94534872347832.asok config set debug_rgw 5
检查同步状态:
(docker-croit)@mycephcluster / $ radosgw-admin sync status --rgw-zone=sync
realm 46669d35-f7ed-4374-8247-2b8f41218109 (default)
zonegroup 881cf806-f6d2-47a0-b7dc-d65ee87f8ef4 (default)
zone 7ead9532-0938-4698-9b4a-2d84d0d00869 (sync)
metadata sync syncing
full sync: 0/64 shards
incremental sync: 64/64 shards
metadata is caught up with master
data sync source: 303a00f5-f50d-43fd-afee-aa0503926952 (default)
syncing
full sync: 0/128 shards
incremental sync: 128/128 shards
data is caught up with source
参考文章:
- https://docs.ceph.com/en/latest/radosgw/multisite/
- https://docs.ceph.com/en/latest/radosgw/cloud-sync-module/#cloud-sync-tier-type-configuration
- https://croit.io/blog/setting-up-ceph-cloud-sync-module
每月第1周 祝祥专栏
祝祥
资深云计算架构师
OpenStack官方特邀讲师
上万台云主机和几十PB分布式存储的
建设管理经验
了解新钛云服
往期技术干货
点👇分享
戳👇在看
关键词
数据
用户
对象
架构
新钛云服
最新评论
推荐文章
作者最新文章
你可能感兴趣的文章
Copyright Disclaimer: The copyright of contents (including texts, images, videos and audios) posted above belong to the User who shared or the third-party website which the User shared from. If you found your copyright have been infringed, please send a DMCA takedown notice to [email protected]. For more detail of the source, please click on the button "Read Original Post" below. For other communications, please send to [email protected].
版权声明:以上内容为用户推荐收藏至CareerEngine平台,其内容(含文字、图片、视频、音频等)及知识版权均属用户或用户转发自的第三方网站,如涉嫌侵权,请通知[email protected]进行信息删除。如需查看信息来源,请点击“查看原文”。如需洽谈其它事宜,请联系[email protected]。
版权声明:以上内容为用户推荐收藏至CareerEngine平台,其内容(含文字、图片、视频、音频等)及知识版权均属用户或用户转发自的第三方网站,如涉嫌侵权,请通知[email protected]进行信息删除。如需查看信息来源,请点击“查看原文”。如需洽谈其它事宜,请联系[email protected]。