Я пытаюсь запустить керберизованный NFSv4 в нашей сети. Сервер — CentOS 7. Я могу смонтировать общий ресурс на моей рабочей станции Arch Linux, но не на наших серверах баз данных Oracle Linux 7.
Я пытаюсь смонтировать из testdb с помощью команды:
mount -v -t nfs4 -o rw,proto=tcp,vers=4.1,port=2049,sec=krb5 gs-storage.ad.goldblattsystems.com:/dba_work /dba_work
Что возвращает (stdout и /var/log/messages):
mount.nfs4: timeout set for Mon Sep 17 12:11:40 2018
mount.nfs4: trying text-based options 'proto=tcp,vers=4.1,port=2049,sec=krb5,addr=192.168.5.30,clientaddr=192.168.5.32'
Sep 17 12:09:40 testdb kernel: NFS: nfs mount opts='proto=tcp,vers=4.1,port=2049,sec=krb5,addr=192.168.5.30,clientaddr=192.168.5.32'
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'proto=tcp'
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'vers=4.1'
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'port=2049'
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'sec=krb5'
Sep 17 12:09:40 testdb kernel: NFS: parsing sec=krb5 option
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'addr=192.168.5.30'
Sep 17 12:09:40 testdb kernel: NFS: parsing nfs mount option 'clientaddr=192.168.5.32'
Sep 17 12:09:40 testdb kernel: NFS: MNTPATH: '/dba_work'
Sep 17 12:09:40 testdb kernel: --> nfs4_try_mount()
Sep 17 12:09:40 testdb kernel: --> nfs4_create_server()
Sep 17 12:09:40 testdb kernel: --> nfs4_init_server()
Sep 17 12:09:40 testdb kernel: --> nfs_get_client(gs-storage.ad.goldblattsystems.com,v4)
Sep 17 12:09:40 testdb kernel: NFS: get client cookie (0xffff88061f3c3800/0xffff8801166af528)
Sep 17 12:09:40 testdb kernel: nfs_callback_create_svc: service created
Sep 17 12:09:40 testdb kernel: NFS: create per-net callback data; net=ffffffff81c60000
Sep 17 12:09:40 testdb kernel: NFS: svc_create_xprt(tcp-bc) returned 0
Sep 17 12:09:40 testdb kernel: --> nfs41_callback_up return 0
Sep 17 12:09:40 testdb kernel: nfs_callback_up: service started
Sep 17 12:09:40 testdb kernel: NFS: nfs4_discover_server_trunking: testing 'gs-storage.ad.goldblattsystems.com'
Sep 17 12:09:40 testdb kernel: NFS call exchange_id auth=RPCSEC_GSS, 'Linux NFSv4.1 testdb'
Sep 17 12:09:40 testdb kernel: NFS reply exchange_id: -13
Sep 17 12:09:40 testdb kernel: NFS call exchange_id auth=RPCSEC_GSS, 'Linux NFSv4.1 testdb'
Sep 17 12:09:40 testdb kernel: NFS reply exchange_id: -13
Sep 17 12:09:40 testdb kernel: NFS call exchange_id auth=RPCSEC_GSS, 'Linux NFSv4.1 testdb'
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: -13
Sep 17 12:09:41 testdb kernel: NFS call exchange_id auth=RPCSEC_GSS, 'Linux NFSv4.1 testdb'
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: -13
Sep 17 12:09:41 testdb kernel: NFS call exchange_id auth=UNIX, 'Linux NFSv4.1 testdb'
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: Server Implementation ID: domain: , name: , date: 0,0
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: 0
Sep 17 12:09:41 testdb kernel: NFS: <-- nfs41_walk_client_list using nfs_client = ffff88061f3c3800 ({2})
Sep 17 12:09:41 testdb kernel: NFS: <-- nfs41_walk_client_list status = 0
Sep 17 12:09:41 testdb kernel: NFS call exchange_id auth=UNIX, 'Linux NFSv4.1 testdb'
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: Server Implementation ID: domain: , name: , date: 0,0
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: 0
Sep 17 12:09:41 testdb kernel: --> nfs4_proc_create_session clp=ffff88061f3c3800 session=ffff880622919400
Sep 17 12:09:41 testdb kernel: nfs4_init_channel_attrs: Fore Channel : max_rqst_sz=1049620 max_resp_sz=1049480 max_ops=8 max_reqs=64
Sep 17 12:09:41 testdb kernel: nfs4_init_channel_attrs: Back Channel : max_rqst_sz=4096 max_resp_sz=4096 max_resp_sz_cached=0 max_ops=2 max_reqs=1
Sep 17 12:09:41 testdb kernel: --> nfs4_setup_session_slot_tables
Sep 17 12:09:41 testdb kernel: --> nfs4_realloc_slot_table: max_reqs=10, tbl->max_slots 0
Sep 17 12:09:41 testdb kernel: nfs4_realloc_slot_table: tbl=ffff880622919440 slots=ffff88013a4a5380 max_slots=10
Sep 17 12:09:41 testdb kernel: <-- nfs4_realloc_slot_table: return 0
Sep 17 12:09:41 testdb kernel: --> nfs4_realloc_slot_table: max_reqs=1, tbl->max_slots 0
Sep 17 12:09:41 testdb kernel: nfs4_realloc_slot_table: tbl=ffff880622919608 slots=ffff88013a4a5040 max_slots=1
Sep 17 12:09:41 testdb kernel: <-- nfs4_realloc_slot_table: return 0
Sep 17 12:09:41 testdb kernel: slot table setup returned 0
Sep 17 12:09:41 testdb kernel: nfs4_proc_create_session client>seqid 2 sessionid 1534807995:93024321:52:0
Sep 17 12:09:41 testdb kernel: <-- nfs4_proc_create_session
Sep 17 12:09:41 testdb kernel: nfs4_schedule_state_renewal: requeueing work. Lease period = 5
Sep 17 12:09:41 testdb kernel: --> nfs41_proc_reclaim_complete
Sep 17 12:09:41 testdb kernel: --> nfs41_setup_sequence
Sep 17 12:09:41 testdb kernel: --> nfs4_alloc_slot used_slots=0000 highest_used=4294967295 max_slots=10
Sep 17 12:09:41 testdb kernel: <-- nfs4_alloc_slot used_slots=0001 highest_used=0 slotid=0
Sep 17 12:09:41 testdb kernel: <-- nfs41_setup_sequence slotid=0 seqid=1
Sep 17 12:09:41 testdb kernel: NFS: nfs4_discover_server_trunking: status = 0
Sep 17 12:09:41 testdb kernel: encode_sequence: sessionid=1534807995:93024321:52:0 seqid=1 slotid=0 max_slotid=0 cache_this=0
Sep 17 12:09:41 testdb kernel: --> nfs_put_client({3})
Sep 17 12:09:41 testdb kernel: <-- nfs4_init_server() = 0
Sep 17 12:09:41 testdb kernel: --> nfs4_get_rootfh()
Sep 17 12:09:41 testdb kernel: --> nfs41_call_sync_prepare data->seq_server ffff88061eabf000
Sep 17 12:09:41 testdb kernel: --> nfs41_setup_sequence
Sep 17 12:09:41 testdb kernel: nfs41_setup_sequence session is draining
Sep 17 12:09:41 testdb kernel: --> nfs4_reclaim_complete_done
Sep 17 12:09:41 testdb kernel: --> nfs4_alloc_slot used_slots=0001 highest_used=0 max_slots=10
Sep 17 12:09:41 testdb kernel: <-- nfs4_alloc_slot used_slots=0003 highest_used=1 slotid=1
Sep 17 12:09:41 testdb kernel: nfs4_free_slot: slotid 1 highest_used_slotid 0
Sep 17 12:09:41 testdb kernel: nfs41_sequence_done: Error 0 free the slot
Sep 17 12:09:41 testdb kernel: nfs4_free_slot: slotid 0 highest_used_slotid 4294967295
Sep 17 12:09:41 testdb kernel: <-- nfs4_reclaim_complete_done
Sep 17 12:09:41 testdb kernel: --> nfs4_alloc_slot used_slots=0000 highest_used=4294967295 max_slots=1
Sep 17 12:09:41 testdb kernel: <-- nfs4_alloc_slot used_slots=0001 highest_used=0 slotid=0
Sep 17 12:09:41 testdb kernel: nfs4_free_slot: slotid 0 highest_used_slotid 4294967295
Sep 17 12:09:41 testdb kernel: --> nfs4_alloc_slot used_slots=0000 highest_used=4294967295 max_slots=10
Sep 17 12:09:41 testdb kernel: <-- nfs4_alloc_slot used_slots=0001 highest_used=0 slotid=0
Sep 17 12:09:41 testdb kernel: --> nfs4_alloc_slot used_slots=0001 highest_used=0 max_slots=10
Sep 17 12:09:41 testdb kernel: <-- nfs4_alloc_slot used_slots=0003 highest_used=1 slotid=1
Sep 17 12:09:41 testdb kernel: nfs4_free_slot: slotid 1 highest_used_slotid 0
Sep 17 12:09:41 testdb kernel: --> nfs_put_client({2})
Sep 17 12:09:41 testdb kernel: --> nfs41_call_sync_prepare data->seq_server ffff88061eabf000
Sep 17 12:09:41 testdb kernel: --> nfs41_setup_sequence
Sep 17 12:09:41 testdb kernel: nfs41_sequence_done: Error 1 free the slot
Sep 17 12:09:41 testdb kernel: nfs4_free_slot: slotid 0 highest_used_slotid 4294967295
Sep 17 12:09:41 testdb kernel: nfs4_get_rootfh: getroot error = 13
Sep 17 12:09:41 testdb kernel: <-- nfs4_get_rootfh() = -13
Sep 17 12:09:41 testdb kernel: --> nfs_free_server()
Sep 17 12:09:41 testdb kernel: --> nfs_put_client({1})
Sep 17 12:09:41 testdb kernel: --> nfs4_proc_destroy_session
Sep 17 12:09:41 testdb kernel: <-- nfs4_proc_destroy_session
Sep 17 12:09:41 testdb kernel: nfs4_destroy_session Destroy backchannel for xprt ffff88061eabb000
Sep 17 12:09:41 testdb kernel: NFS: destroy per-net callback data; net=ffffffff81c60000
mount.nfs4: mount(2): Permission denied
mount.nfs4: access denied by server while mounting gs-storage.ad.goldblattsystems.com:/dba_work
Sep 17 12:09:41 testdb kernel: nfs_callback_down: service stopped
Sep 17 12:09:41 testdb kernel: nfs_callback_down: service destroyed
Sep 17 12:09:41 testdb kernel: --> nfs_free_client(4)
Sep 17 12:09:41 testdb kernel: NFS: releasing client cookie (0xffff88061f3c3800/0xffff8801166af528)
Sep 17 12:09:41 testdb kernel: <-- nfs_free_client()
Sep 17 12:09:41 testdb kernel: <-- nfs_free_server()
Sep 17 12:09:41 testdb kernel: <-- nfs4_create_server() = error -13
Sep 17 12:09:41 testdb kernel: <-- nfs4_try_mount() = -13 [error]
Я использовал net ads keytab add nfsдля создания необходимого keytab с обеих сторон. Что я делаю не так? Дополнительная информация по отладке приведена ниже:
/data 192.168.0.0/16(rw,no_subtree_check,sync,no_root_squash,insecure,sec=krb5:krb5i:krb5p,fsid=0)
/data/dba_work 192.168.0.0/16(rw,no_subtree_check,sync,no_root_squash,insecure,sec=krb5:krb5i:krb5p,fsid=1)
Обзор окружающей среды:
Server Client
============================
Name: gs-storage testdb
IP: 192.168.5.30 192.168.5.32
OS: CentOS 7 Oracle Linux 7
Вот /var/log/messagesвывод сервера при монтировании, активированного с помощью rpcdebug -m nfsd all:
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 0
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #1/1: 42 (OP_EXCHANGE_ID)
Sep 17 12:08:32 gs-storage kernel: nfsd4_exchange_id rqstp=ffff9484f90ec000 exid=ffff94886989e0a0 clname.len=20 clname.data=ffff94886636c060 ip_addr=192.168.5.32 flags 103, spa_how 0
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7040)
Sep 17 12:08:32 gs-storage kernel: nfsd4_exchange_id seqid 0 flags 20001
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 1 #1: 42: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound returned 0
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #1/1: 42 (OP_EXCHANGE_ID)
Sep 17 12:08:32 gs-storage kernel: nfsd4_exchange_id rqstp=ffff9484f90ec000 exid=ffff94886989e0a0 clname.len=20 clname.data=ffff94886636c060 ip_addr=192.168.5.32 flags 103, spa_how 0
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: nfsd4_exchange_id seqid 0 flags 20001
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 1 #1: 42: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound returned 0
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #1/1: 43 (OP_CREATE_SESSION)
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: check_slot_seqid enter. seqid 1 slot_seqid 0
Sep 17 12:08:32 gs-storage kernel: NFSD: move_to_confirm nfs4_client ffff948844b8f800
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 1 #1: 43: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound returned 0
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #1/2: 53 (OP_SEQUENCE)
Sep 17 12:08:32 gs-storage kernel: __find_in_sessionid_hashtbl: 1534807995:93024321:52:0
Sep 17 12:08:32 gs-storage kernel: nfsd4_sequence: slotid 0
Sep 17 12:08:32 gs-storage kernel: check_slot_seqid enter. seqid 1 slot_seqid 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 2 #1: 53: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #2/2: 58 (OP_RECLAIM_COMPLETE)
Sep 17 12:08:32 gs-storage kernel: nfsd4_umh_cltrack_upcall: cmd: create
Sep 17 12:08:32 gs-storage kernel: nfsd4_umh_cltrack_upcall: arg: 4c696e7578204e465376342e3120746573746462
Sep 17 12:08:32 gs-storage kernel: nfsd4_umh_cltrack_upcall: env0: NFSDCLTRACK_CLIENT_HAS_SESSION=Y
Sep 17 12:08:32 gs-storage kernel: nfsd4_umh_cltrack_upcall: env1: NFSDCLTRACK_GRACE_START=1534807995
Sep 17 12:08:32 gs-storage kernel: nfsd4_umh_cltrack_upcall: /sbin/nfsdcltrack return value: 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 2 #2: 58: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound returned 0
Sep 17 12:08:32 gs-storage kernel: --> nfsd4_store_cache_entry slot ffff9488443ca000
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op #1/1: 44 (OP_DESTROY_SESSION)
Sep 17 12:08:32 gs-storage kernel: nfsd4_destroy_session: 1534807995:93024321:52:0
Sep 17 12:08:32 gs-storage kernel: __find_in_sessionid_hashtbl: 1534807995:93024321:52:0
Sep 17 12:08:32 gs-storage kernel: NFSD: warning: no callback path to client Linux NFSv4.1 testdb: error -22
Sep 17 12:08:32 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 1 #1: 44: status 0
Sep 17 12:08:32 gs-storage kernel: nfsv4 compound returned 0
Sep 17 12:08:33 gs-storage kernel: nfsd_dispatch: vers 4 proc 1
Sep 17 12:08:33 gs-storage kernel: nfsv4 compound op #1/1: 57 (OP_DESTROY_CLIENTID)
Sep 17 12:08:33 gs-storage kernel: renewing client (clientid 5b7b4fbb/058b7041)
Sep 17 12:08:33 gs-storage kernel: nfsd4_umh_cltrack_upcall: cmd: remove
Sep 17 12:08:33 gs-storage kernel: nfsd4_umh_cltrack_upcall: arg: 4c696e7578204e465376342e3120746573746462
Sep 17 12:08:33 gs-storage kernel: nfsd4_umh_cltrack_upcall: env0: (null)
Sep 17 12:08:33 gs-storage kernel: nfsd4_umh_cltrack_upcall: env1: (null)
Sep 17 12:08:33 gs-storage kernel: nfsd4_umh_cltrack_upcall: /sbin/nfsdcltrack return value: 0
Sep 17 12:08:33 gs-storage kernel: nfsv4 compound op ffff94886989e080 opcnt 1 #1: 57: status 0
Sep 17 12:08:33 gs-storage kernel: nfsv4 compound returned 0
Редактировать:Вопрос, отмеченный как дубликат, не соответствует данному обстоятельству.krb5 являетсявключенный механизм аутентификации/транспорта. Кроме того, ошибка в другом вопросе полностью отличается от этой.
решение1
Этот:
Sep 17 12:09:40 testdb kernel: NFS call exchange_id auth=RPCSEC_GSS, 'Linux NFSv4.1 testdb'
Sep 17 12:09:40 testdb kernel: NFS reply exchange_id: -13
наводит меня на мысль, что rpc.gssd, возможно, не запущен и сразу после него есть откат к sys security:
Sep 17 12:09:41 testdb kernel: NFS call exchange_id auth=UNIX, 'Linux NFSv4.1 testdb'
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: Server Implementation ID: domain: , name: , date: 0,0
Sep 17 12:09:41 testdb kernel: NFS reply exchange_id: 0
Работает ли служба nfs-secure?
Кроме того, проверьте, правильно ли настроен id_resolver. Проверьте /etc/request-key.conf и /etc/request-key.d/*.conf на наличие такой строки:
create id_resolver * * /usr/sbin/nfsidmap %k %d