24x 16TB 드라이브 droid2 어레이에서 zfs 리실버 속도를 어떻게 높일 수 있습니까?: "총 117T... 47일 02:31:23 남았습니다."

24x 16TB 드라이브 droid2 어레이에서 zfs 리실버 속도를 어떻게 높일 수 있습니까?: "총 117T... 47일 02:31:23 남았습니다."

(형식을 명확히 하기 위해 여러 번 편집한 점에 대해 사과드립니다)

위와 같이. 드라이브에 SMART 오류가 표시되지 않습니다.

HBA <--> 드라이브 통신 오류를 확인하는 방법이 있습니까?

이는 AMD 64코어 Threadripper Pro CPU, 256GB ECC RAM, LSI HBA 및 24개의 16TB Seagate X18 드라이브를 갖춘 Supermicro 플랫폼의 Ubuntu 22.04에서 실행됩니다. 전원은 OneAC 절연 변압기를 통해 필터링됩니다.

출력 은 다음과 같습니다 zpool status.

  pool: lake_24x16TB  
 state: ONLINE  
status: One or more devices is currently being resilvered.  The pool will  
    continue to function, possibly in a degraded state.  
action: Wait for the resilver to complete.  
  scan: resilver in progress since Thu Sep  7 19:06:13 2023  
    4.21T scanned at 135M/s, 959G issued at 30.0M/s, 117T total  
    177G resilvered, 0.80% done, 46 days 22:15:51 to go  
config:  
  
NAME                                     STATE     READ WRITE CKSUM  
lake_24x16TB                             ONLINE       0     0     0  
  draid2:21d:24c:1s-0                    ONLINE       0     0     0  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM001G-2KK103_ZL______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    spare-20                             ONLINE       0     0     0  
      ata-ST16000NM000J-2TW103_ZR______  ONLINE       0     0     0  (resilvering)  
      draid2-0-0                         ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
    ata-ST16000NM000J-2TW103_ZR______    ONLINE       0     0     0  (resilvering)  
spares  
  draid2-0-0                             INUSE     currently in use  

errors: No known data errors 

출력 의 첫 번째 단계는 다음과 같습니다 iostat -x -k 1.

Linux 6.2.0-32-generic (Kaala)  09/09/2023  _x86_64_    (128 CPU)    
  
avg-cpu:  %user   %nice %system %iowait  %steal   %idle  
           0.11    0.00    3.10   16.29    0.00   80.49  
  
Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util  
loop0            0.00      0.00     0.00   0.00    0.00     1.21    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop1            0.01      0.03     0.00   0.00    0.02     2.90    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop10           0.04      0.12     0.00   0.00    0.02     2.91    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop11           0.05      0.53     0.00   0.00    0.06    11.01    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop12           0.01      0.03     0.00   0.00    0.02     2.59    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop13           0.00      0.01     0.00   0.00    0.10     4.80    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop14           0.00      0.01     0.00   0.00    0.18     7.22    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop15           0.02      0.67     0.00   0.00    0.25    36.36    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop16           0.00      0.00     0.00   0.00    0.05     2.70    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop17           0.00      0.01     0.00   0.00    0.12     9.81    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop18           0.00      0.00     0.00   0.00    0.00     1.27    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop2            0.01      0.03     0.00   0.00    0.02     2.88    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop3            0.01      0.06     0.00   0.00    0.03     4.35    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop4            0.02      0.17     0.00   0.00    0.15     8.17    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop5            0.00      0.03     0.00   0.00    0.10    16.14    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop6            0.00      0.03     0.00   0.00    0.14    15.12    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop7            0.02      0.07     0.00   0.00    0.02     3.91    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop8            0.02      0.07     0.00   0.00    0.02     3.89    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
loop9            0.02      0.08     0.00   0.00    0.03     3.78    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00    0.00    0.00   0.00  
nvme0n1          1.95     61.41     0.53  21.39    0.16    31.42    3.71     55.30     3.15  45.95    1.64    14.91    0.00      0.00     0.00   0.00    0.00     0.00    0.24    0.90    0.01   0.34  
sda            657.15  17662.73     0.00   0.00    2.46    26.88  674.99   7473.83    15.98   2.31    0.59    11.07    0.00      0.00     0.00   0.00    0.00     0.00    0.18   61.75    2.02  80.00  
sdb            654.70  17626.14     0.00   0.00    2.47    26.92  673.86   7469.65    16.29   2.36    0.59    11.08    0.00      0.00     0.00   0.00    0.00     0.00    0.18   60.57    2.02  80.01  
sdc            657.38  17589.65     0.00   0.00    2.64    26.76  671.59   7476.97    15.74   2.29    0.60    11.13    0.00      0.00     0.00   0.00    0.00     0.00    0.18   66.51    2.15  80.11  
sdd            659.12  17616.74     0.00   0.00    2.50    26.73  670.35   7474.24    15.95   2.32    0.60    11.15    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.60    2.06  79.98  
sde            651.63  17480.21     0.00   0.00    2.34    26.83  554.62   7472.05    16.00   2.80    0.94    13.47    0.00      0.00     0.00   0.00    0.00     0.00    0.18   65.38    2.06  79.99  
sdf            625.72  16634.25     0.00   0.00    1.83    26.58  671.25   7719.82    15.92   2.32    0.61    11.50    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.77    1.57  77.70  
sdg            653.95  17600.39     0.00   0.00    2.61    26.91  666.46   7468.19    15.91   2.33    0.61    11.21    0.00      0.00     0.00   0.00    0.00     0.00    0.18   64.79    2.13  80.09  
sdh            655.76  17574.57     0.00   0.00    2.69    26.80  670.95   7467.93    15.91   2.32    0.60    11.13    0.00      0.00     0.00   0.00    0.00     0.00    0.18   67.54    2.18  80.10  
sdi            655.44  17568.31     0.00   0.00    2.45    26.80  671.10   7471.13    15.91   2.32    0.60    11.13    0.00      0.00     0.00   0.00    0.00     0.00    0.18   62.33    2.02  79.93  
sdj            647.18  17203.93     0.00   0.00    2.35    26.58  677.54   7540.63    15.81   2.28    0.58    11.13    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.77    1.93  79.73  
sdk            655.07  17638.54     0.00   0.00    2.60    26.93  670.18   7472.87    15.75   2.30    0.60    11.15    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.95    2.11  80.03  
sdl            656.55  17653.16     0.00   0.00    2.37    26.89  677.64   7473.03    16.05   2.31    0.58    11.03    0.00      0.00     0.00   0.00    0.00     0.00    0.18   62.99    1.96  79.93  
sdm            654.57  17561.50     0.00   0.00    2.56    26.83  670.79   7476.85    15.76   2.30    0.60    11.15    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.98    2.09  80.16  
sdn            654.64  17541.64     0.00   0.00    2.38    26.80  673.14   7472.02    15.90   2.31    0.59    11.10    0.00      0.00     0.00   0.00    0.00     0.00    0.18   61.28    1.97  79.85  
sdo            673.34  17649.85     0.00   0.00    2.27    26.21  677.04   7472.21    16.03   2.31    0.58    11.04    0.00      0.00     0.00   0.00    0.00     0.00    0.18   62.22    1.93  79.98  
sdp            650.65  17476.46     0.00   0.00    2.26    26.86  678.38   7473.77    16.17   2.33    0.58    11.02    0.00      0.00     0.00   0.00    0.00     0.00    0.18   60.67    1.87  79.80  
sdq            650.07  17629.95     0.00   0.00    2.31    27.12  677.22   7475.71    16.05   2.32    0.59    11.04    0.00      0.00     0.00   0.00    0.00     0.00    0.18   62.94    1.91  79.70  
sdr            655.26  17605.45     0.00   0.00    2.52    26.87  672.01   7477.66    15.98   2.32    0.60    11.13    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.34    2.06  79.98  
sds            652.57  17599.76     0.00   0.00    6.64    26.97  455.75   8149.03    11.96   2.56    2.17    17.88    0.00      0.00     0.00   0.00    0.00     0.00    0.18  116.58    5.34  86.73  
sdt            651.14  17549.92     0.00   0.00    2.58    26.95  671.98   7475.30    15.98   2.32    0.60    11.12    0.00      0.00     0.00   0.00    0.00     0.00    0.18   64.62    2.09  79.96  
sdu            678.75  17761.68     0.00   0.00    2.20    26.17  679.47   7472.43    15.94   2.29    0.58    11.00    0.00      0.00     0.00   0.00    0.00     0.00    0.18   60.03    1.90  80.07  
sdv            653.08  17555.64     0.00   0.00    2.35    26.88  673.80   7476.29    15.94   2.31    0.59    11.10    0.00      0.00     0.00   0.00    0.00     0.00    0.18   62.13    1.94  79.75  
sdw            655.08  17615.29     0.00   0.00    2.42    26.89  673.54   7470.69    16.05   2.33    0.59    11.09    0.00      0.00     0.00   0.00    0.00     0.00    0.18   63.18    1.99  79.92  
sdx            652.01  17484.18     0.00   0.00    2.41    26.82  676.29   7475.99    15.84   2.29    0.59    11.05    0.00      0.00     0.00   0.00    0.00     0.00    0.18   61.02    1.98  79.80  
  
   

답변1

스크럽은 주로 전송 속도가 낮은 소규모 읽기를 포함하는 메타데이터 검색 단계에 있습니다. 메타데이터 스캔이 완료될 때까지 데이터 리실버가 느려집니다. ZFS 모듈 매개변수 및/또는 기타 설정을 조작하는 대신 인내심을 가지십시오. 리실버 속도는 가까운 미래에 가속화될 것입니다.

관련 정보