안녕하세요. Openstack Keystone을 Ceph 인증 메커니즘으로 통합하려고 합니다. 그래서 ceph 개체 저장소를 openstack Swift 백엔드로 사용할 수 있습니다.
환경:
Kernel : Ubunutu Server LTS 22.04 (minimal)
Openstack : Zed (Manual Installation)
Ceph : quiny (Cephadm Installation)
//컨트롤러 노드
openstack service create --name swift object-store
openstack user create --domain default --password-prompt swift
openstack user create --domain default --password-prompt rgw
openstack role add --user swift --project service admin
openstack role add --user swift --project service swiftoperator
openstack role add --user rgw --project service admin
openstack role add --user rgw --project service swiftoperator
openstack endpoint create --region Tehran object-store public http://<rados_gatway>:8080/swift/v1
openstack endpoint create --region Tehran object-store internal http://<rados_gatway>:8080/swift/v1
openstack endpoint create --region Tehran object-store admin http://<rados_gatway>:8080/swift/v1
//세프 클러스터
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_api_version 3
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_url http://<keystone_url>:5000
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_accepted_roles admin,member,swiftoperator,Member,_member_
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_token_cache_size 500
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_admin_user rgw
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_admin_password rgw
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_admin_domain default
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_keystone_admin_project service
ceph config set client.rgw.default.ceph-2.ncsnqh rgw_s3_auth_use_keystone true
이제 실행하면 swift list이런 오류가 발생합니다 ;(
Account GET failed: http://<rados_gatway>:8080/swift/v1?format=json 401 Unauthorized [first 60 chars of response] b'{"Code":"AccessDenied","RequestId":"tx00000ff92593343f6fbac-'
Failed Transaction ID: tx00000ff92593343f6fbac-0063b3dcd8-455e0-default
여기에 뭔가 빠진 것 같은 느낌이 듭니다. 많은 문서를 읽었지만 그 중 하나만 해결책을 찾았고 그것은 OpenStack에서 radosgw 사용자를 생성하고 이에 Swift 운영자 역할을 할당하는 것이었습니다. Swift 사용자에도 불구하고 문제는 여전히 존재합니다. 서비스 프로젝트에 관리자 역할이 있습니다. 나는 그것에 신속한 운영자 역할을 할당했습니다! 난 아직도 문제가 있어
curl -v http://<keystone_url>:5000 (on ceph-2 returns no error)
여기에 완전한 것이 있습니다swift list --debug
DEBUG:keystoneclient.auth.identity.v3.base:Making authentication request to http://<keystone_url>:5000/v3/auth/tokens
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): <keystone_url>:5000
DEBUG:urllib3.connectionpool:http://<keystone_url>:5000 "POST /v3/auth/tokens HTTP/1.1" 201 4678
DEBUG:keystoneclient.auth.identity.v3.base:{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6622244113204a689e3a367847291166", "name": "hoodad", "password_expires_at": null}, "audit_ids": ["zNYqN-lESbCt8U1MA3tl5Q"], "expires_at": "2023-01-03T08:41:55.000000Z", "issued_at": "2023-01-03T07:41:55.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "36905f5fbaa64feaa0a47dcc3d8f5455", "name": "admin"}, "is_domain": false, "roles": [{"id": "5365f6dcb2fc4577a3c31693e671e5ee", "name": "reader"}, {"id": "7d90492c8771403b93d5bf8e1d33e40b", "name": "admin"}, {"id": "514cde82919e436aaec7568ad1ba4bee", "name": "member"}], "catalog": [{"endpoints": [{"id": "349bda8b61cc4bee932887f213de41c7", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "8c981d7f64f74174ba1a0bc3eaf4aa91", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "9c94c3bdc0394abea5f3646f8986022f", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}], "id": "061f492117d74190bc0084986feb377a", "type": "volumev3", "name": "cinder"}, {"endpoints": [{"id": "3fd3b010a41b4a3a86fa76b308f3a053", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}, {"id": "455c040c8f304f4e99eae8104a57ec17", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}, {"id": "86e24eb7164d449da0a8bf56af1d56b7", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}], "id": "1c97054f81db4fcc8ed16d3aa42869a9", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "16be2834ab4d4fdb9c4c293b550d4980", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}, {"id": "b58356f6e25747a7bce5e9c9c4a0bd7e", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}, {"id": "cd7430601c6d4e928e3ea279aa75d63d", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}], "id": "6408ac009be64d93b82c6803aad17607", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "a21330d45c8f4530a06c99a62c187e14", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}, {"id": "dbc4d63dfbb94ae7afcf20458b428319", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}, {"id": "de26e420bc994cb9b9332922f088a670", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}], "id": "694e0790a22d41c29585b786bc263009", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "85434ec1ef2e4d2ca52f0467df6a9001", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}, {"id": "98773c3f67b640f18f53885b569e4d73", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}, {"id": "c1e80464da124b7eaa0279e28c1f25d2", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}], "id": "c8fe90a32cfc417fa5369b60092c0dfc", "type": "image", "name": "glance"}, {"endpoints": [{"id": "26464c37332e4c0da96ca4e8f7b82ae9", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "623dc6a74b634117b3edcc5892cc1bbb", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "8314196253d2446aaeec6e9e6e45fd47", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}], "id": "ccc120553ee14e3f8b3157f698190492", "type": "compute", "name": "nova"}, {"endpoints": [{"id": "8751532bb2ca4f81a0f51ecb67df6eb4", "interface": "public", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}, {"id": "9f31cf8882554f199ab9ead345e05825", "interface": "internal", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}, {"id": "e98011666e844f13aac4e423a316fde6", "interface": "admin", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}], "id": "e38f897497d547c6a06bb6a52be1be13", "type": "object-store", "name": "swift"}]}}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): <rados_gatway>:8080
DEBUG:urllib3.connectionpool:http://<rados_gatway>:8080 "GET /swift/v1?format=json HTTP/1.1" 401 119
INFO:swiftclient:REQ: curl -i http://<rados_gatway>:8080/swift/v1?format=json -X GET -H "X-Auth-Token: gAAAAABjs9xDrD6NgcD6Uyatc0QH4q74_SiztiLkYPpoHKK0b8yGWwyXfAw-V4klq7x6nCekqmHwa2ELQVHI_Cj5AzygU98Hdr6rrrpL3Wihl1CdqMyoXnw_GdNWh4dNQPGxOQatYXR2XwU5U7r9Juv-G4cJjFYFh5RRKyPNCzN6z_vhI-xm5sc" -H "Accept-Encoding: gzip"
INFO:swiftclient:RESP STATUS: 401 Unauthorized
INFO:swiftclient:RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx00000a7c6f54a4f0a7eac-0063b3dc43-455e0-default', 'X-Openstack-Request-Id': 'tx00000a7c6f54a4f0a7eac-0063b3dc43-455e0-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Tue, 03 Jan 2023 07:41:55 GMT', 'Connection': 'Keep-Alive'}
INFO:swiftclient:RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx00000a7c6f54a4f0a7eac-0063b3dc43-455e0-default","HostId":"455e0-default-default"}'
DEBUG:keystoneclient.auth.identity.v3.base:Making authentication request to http://<keystone_url>:5000/v3/auth/tokens
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): <keystone_url>:5000
DEBUG:urllib3.connectionpool:http://<keystone_url>:5000 "POST /v3/auth/tokens HTTP/1.1" 201 4678
DEBUG:keystoneclient.auth.identity.v3.base:{"token": {"methods": ["password"], "user": {"domain": {"id": "default", "name": "Default"}, "id": "6622244113204a689e3a367847291166", "name": "hoodad", "password_expires_at": null}, "audit_ids": ["B3g606MNTUqZS6tUgEHyHQ"], "expires_at": "2023-01-03T08:41:56.000000Z", "issued_at": "2023-01-03T07:41:56.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "36905f5fbaa64feaa0a47dcc3d8f5455", "name": "admin"}, "is_domain": false, "roles": [{"id": "5365f6dcb2fc4577a3c31693e671e5ee", "name": "reader"}, {"id": "7d90492c8771403b93d5bf8e1d33e40b", "name": "admin"}, {"id": "514cde82919e436aaec7568ad1ba4bee", "name": "member"}], "catalog": [{"endpoints": [{"id": "349bda8b61cc4bee932887f213de41c7", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "8c981d7f64f74174ba1a0bc3eaf4aa91", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "9c94c3bdc0394abea5f3646f8986022f", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8776/v3/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}], "id": "061f492117d74190bc0084986feb377a", "type": "volumev3", "name": "cinder"}, {"endpoints": [{"id": "3fd3b010a41b4a3a86fa76b308f3a053", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}, {"id": "455c040c8f304f4e99eae8104a57ec17", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}, {"id": "86e24eb7164d449da0a8bf56af1d56b7", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:5000/v3/", "region": "Tehran"}], "id": "1c97054f81db4fcc8ed16d3aa42869a9", "type": "identity", "name": "keystone"}, {"endpoints": [{"id": "16be2834ab4d4fdb9c4c293b550d4980", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}, {"id": "b58356f6e25747a7bce5e9c9c4a0bd7e", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}, {"id": "cd7430601c6d4e928e3ea279aa75d63d", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8778", "region": "Tehran"}], "id": "6408ac009be64d93b82c6803aad17607", "type": "placement", "name": "placement"}, {"endpoints": [{"id": "a21330d45c8f4530a06c99a62c187e14", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}, {"id": "dbc4d63dfbb94ae7afcf20458b428319", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}, {"id": "de26e420bc994cb9b9332922f088a670", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:9696", "region": "Tehran"}], "id": "694e0790a22d41c29585b786bc263009", "type": "network", "name": "neutron"}, {"endpoints": [{"id": "85434ec1ef2e4d2ca52f0467df6a9001", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}, {"id": "98773c3f67b640f18f53885b569e4d73", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}, {"id": "c1e80464da124b7eaa0279e28c1f25d2", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:9292", "region": "Tehran"}], "id": "c8fe90a32cfc417fa5369b60092c0dfc", "type": "image", "name": "glance"}, {"endpoints": [{"id": "26464c37332e4c0da96ca4e8f7b82ae9", "interface": "admin", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "623dc6a74b634117b3edcc5892cc1bbb", "interface": "public", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}, {"id": "8314196253d2446aaeec6e9e6e45fd47", "interface": "internal", "region_id": "Tehran", "url": "http://<keystone_url>:8774/v2.1/36905f5fbaa64feaa0a47dcc3d8f5455", "region": "Tehran"}], "id": "ccc120553ee14e3f8b3157f698190492", "type": "compute", "name": "nova"}, {"endpoints": [{"id": "8751532bb2ca4f81a0f51ecb67df6eb4", "interface": "public", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}, {"id": "9f31cf8882554f199ab9ead345e05825", "interface": "internal", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}, {"id": "e98011666e844f13aac4e423a316fde6", "interface": "admin", "region_id": "Tehran", "url": "http://<rados_gatway>:8080/swift/v1", "region": "Tehran"}], "id": "e38f897497d547c6a06bb6a52be1be13", "type": "object-store", "name": "swift"}]}}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): <rados_gatway>:8080
DEBUG:urllib3.connectionpool:http://<rados_gatway>:8080 "GET /swift/v1?format=json HTTP/1.1" 401 119
INFO:swiftclient:REQ: curl -i http://<rados_gatway>:8080/swift/v1?format=json -X GET -H "X-Auth-Token: gAAAAABjs9xEgshf9a7GAexTEQ27dZFkFSP7TaC-o-2Bba_WbaH7WeMS9ohHrJhlU_tFdcWsd-71UEE4e33bOEtA8vM6yA6Nu2IAm8SU2QN6Ox5tuhps5Dc0E_inQfqxg-9cAgpjwsm8czG06SsCku6Cgxt-UqSdyCGn9CcShRgH0u7Mb1eyEvw" -H "Accept-Encoding: gzip"
INFO:swiftclient:RESP STATUS: 401 Unauthorized
INFO:swiftclient:RESP HEADERS: {'Content-Length': '119', 'X-Trans-Id': 'tx0000081618694ce1134ad-0063b3dc44-455e0-default', 'X-Openstack-Request-Id': 'tx0000081618694ce1134ad-0063b3dc44-455e0-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/json; charset=utf-8', 'Date': 'Tue, 03 Jan 2023 07:41:56 GMT', 'Connection': 'Keep-Alive'}
INFO:swiftclient:RESP BODY: b'{"Code":"AccessDenied","RequestId":"tx0000081618694ce1134ad-0063b3dc44-455e0-default","HostId":"455e0-default-default"}'
ERROR:swiftclient.service:Account GET failed: http://<rados_gatway>:8080/swift/v1?format=json 401 Unauthorized [first 60 chars of response] b'{"Code":"AccessDenied","RequestId":"tx0000081618694ce1134ad-' (txn: tx0000081618694ce1134ad-0063b3dc44-455e0-default)
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/swiftclient/service.py", line 949, in _list_account_job
_, items = conn.get_account(
File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1911, in get_account
return self._retry(None, get_account, marker=marker, limit=limit,
File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1856, in _retry
rv = func(self.url, self.token, *args,
File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 883, in get_account
raise ClientException.from_response(resp, 'Account GET failed', body)
swiftclient.exceptions.ClientException: Account GET failed: http://<rados_gatway>:8080/swift/v1?format=json 401 Unauthorized [first 60 chars of response] b'{"Code":"AccessDenied","RequestId":"tx0000081618694ce1134ad-' (txn: tx0000081618694ce1134ad-0063b3dc44-455e0-default)
Account GET failed: http://<rados_gatway>:8080/swift/v1?format=json 401 Unauthorized [first 60 chars of response] b'{"Code":"AccessDenied","RequestId":"tx0000081618694ce1134ad-'
Failed Transaction ID: tx0000081618694ce1134ad-0063b3dc44-455e0-default
답변1
[불완전] Ceph Octopus(약간 수정된 구성 변경)와 작동하도록 만들 수 있었습니다. 아래 세부 정보를 참조하세요. 나는 여전히 작동하는 Pacific RGW 구성을 얻으려고 노력하고 있으며 Quincy도 현재 나에게 작동하지 않습니다.
최근 크로스 프로젝트 노트를 보니문서. 완전히 만족스러운 답변은 아닐 수도 있지만, 제 메모를 붙여넣으려면 댓글만으로는 충분하지 않습니다. Ceph Octopus와 OpenStack Victoria를 사용했을 때 효과가 있었던 것은 다음과 같습니다.
# ceph.conf
[client.rgw.keystone.storage01.vtakeh]
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_api_version 3
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_accepted_roles "admin,Member,_member_,member"
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_admin_user rgw
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_admin_password ****
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_admin_domain default
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_admin_project service
ceph config set client.rgw.keystone.storage01.vtakeh rgw_s3_auth_use_keystone true
ceph config set client.rgw.keystone.storage01.vtakeh rgw_keystone_url https://control.fqdn:5000
ceph config set client.rgw.keystone.storage01.vtakeh rgw_swift_account_in_url true
---
# create swift service
$ openstack service create --name=swift --description="Swift Service" object-store
$ openstack user create rgw --password *****
# add role to user
$ openstack role add --user rgw --project service admin
# create keystone endpoints
$ openstack endpoint create --region RegionOne swift admin "http://ses6-mon1.fqdn:80/swift/v1/AUTH_$(project_id)s"
$ openstack endpoint create --region RegionOne swift internal "http://ses6-mon1.fqdn:80/swift/v1/AUTH_$(project_id)s"
$ openstack endpoint create --region RegionOne swift public "http://ses6-mon1.fqdn:80/swift/v1/AUTH_$(project_id)s"
이러한 옵션을 구성한 후 성공적으로 실행할 수 있었습니다 openstack container create swift1. 위의 명령은 새로운 rgw를 설정하고 openstack을 통해 액세스하는 데 효과적이었습니다.
control01:~ # openstack container create swift1
+---------+-----------+---------------------------------------------------+
| account | container | x-trans-id |
+---------+-----------+---------------------------------------------------+
| v1 | swift1 | tx0000095214f842753ecaa-00639b24cd-606d44-default |
+---------+-----------+---------------------------------------------------+
control01:~ # openstack container list
+--------+
| Name |
+--------+
| swift1 |
+--------+
사소한 변경으로 기본적으로 ceph nautilus 및 openstack rocky에서도 동일하게 작동했습니다.