MariaDB 在 500% cpu 時間下瘋狂

MariaDB 在 500% cpu 時間下瘋狂

全球狀況

MariaDB [(none)]> SHOW GLOBAL STATUS;
+--------------------------------------------------------------+--------------------------------------------------+
| Variable_name                                                | Value                                            |
+--------------------------------------------------------------+--------------------------------------------------+
| Aborted_clients                                              | 10                                               |
| Aborted_connects                                             | 17                                               |
| Access_denied_errors                                         | 2808                                             |
| Acl_column_grants                                            | 0                                                |
| Acl_database_grants                                          | 255                                              |
| Acl_function_grants                                          | 0                                                |
| Acl_procedure_grants                                         | 0                                                |
| Acl_proxy_users                                              | 1                                                |
| Acl_role_grants                                              | 0                                                |
| Acl_roles                                                    | 0                                                |
| Acl_table_grants                                             | 2                                                |
| Acl_users                                                    | 253                                              |
| Aria_pagecache_blocks_not_flushed                            | 0                                                |
| Aria_pagecache_blocks_unused                                 | 15706                                            |
| Aria_pagecache_blocks_used                                   | 40                                               |
| Aria_pagecache_read_requests                                 | 287044                                           |
| Aria_pagecache_reads                                         | 20253                                            |
| Aria_pagecache_write_requests                                | 36614                                            |
| Aria_pagecache_writes                                        | 36593                                            |
| Aria_transaction_log_syncs                                   | 1                                                |
| Binlog_commits                                               | 0                                                |
| Binlog_group_commits                                         | 0                                                |
| Binlog_group_commit_trigger_count                            | 0                                                |
| Binlog_group_commit_trigger_lock_wait                        | 0                                                |
| Binlog_group_commit_trigger_timeout                          | 0                                                |
| Binlog_snapshot_file                                         |                                                  |
| Binlog_snapshot_position                                     | 0                                                |
| Binlog_bytes_written                                         | 0                                                |
| Binlog_cache_disk_use                                        | 0                                                |
| Binlog_cache_use                                             | 0                                                |
| Binlog_stmt_cache_disk_use                                   | 0                                                |
| Binlog_stmt_cache_use                                        | 0                                                |
| Busy_time                                                    | 0.000000                                         |
| Bytes_received                                               | 243371501                                        |
| Bytes_sent                                                   | 2355355672                                       |
| Com_admin_commands                                           | 2293                                             |
| Com_alter_db                                                 | 0                                                |
| Com_alter_db_upgrade                                         | 0                                                |
| Com_alter_event                                              | 0                                                |
| Com_alter_function                                           | 0                                                |
| Com_alter_procedure                                          | 0                                                |
| Com_alter_server                                             | 0                                                |
| Com_alter_table                                              | 0                                                |
| Com_alter_tablespace                                         | 0                                                |
| Com_alter_user                                               | 0                                                |
| Com_analyze                                                  | 0                                                |
| Com_assign_to_keycache                                       | 0                                                |
| Com_begin                                                    | 0                                                |
| Com_binlog                                                   | 0                                                |
| Com_call_procedure                                           | 0                                                |
| Com_change_db                                                | 92                                               |
| Com_change_master                                            | 0                                                |
| Com_check                                                    | 0                                                |
| Com_checksum                                                 | 0                                                |
| Com_commit                                                   | 0                                                |
| Com_compound_sql                                             | 0                                                |
| Com_create_db                                                | 0                                                |
| Com_create_event                                             | 0                                                |
| Com_create_function                                          | 0                                                |
| Com_create_index                                             | 0                                                |
| Com_create_procedure                                         | 0                                                |
| Com_create_role                                              | 0                                                |
| Com_create_server                                            | 0                                                |
| Com_create_table                                             | 0                                                |
| Com_create_temporary_table                                   | 0                                                |
| Com_create_trigger                                           | 0                                                |
| Com_create_udf                                               | 0                                                |
| Com_create_user                                              | 0                                                |
| Com_create_view                                              | 0                                                |
| Com_dealloc_sql                                              | 0                                                |
| Com_delete                                                   | 8354                                             |
| Com_delete_multi                                             | 0                                                |
| Com_do                                                       | 0                                                |
| Com_drop_db                                                  | 0                                                |
| Com_drop_event                                               | 0                                                |
| Com_drop_function                                            | 0                                                |
| Com_drop_index                                               | 0                                                |
| Com_drop_procedure                                           | 0                                                |
| Com_drop_role                                                | 0                                                |
| Com_drop_server                                              | 0                                                |
| Com_drop_table                                               | 0                                                |
| Com_drop_temporary_table                                     | 0                                                |
| Com_drop_trigger                                             | 0                                                |
| Com_drop_user                                                | 0                                                |
| Com_drop_view                                                | 0                                                |
| Com_empty_query                                              | 0                                                |
| Com_execute_immediate                                        | 0                                                |
| Com_execute_sql                                              | 0                                                |
| Com_flush                                                    | 0                                                |
| Com_get_diagnostics                                          | 0                                                |
| Com_grant                                                    | 0                                                |

488 rows in set (0.00 sec)

MariaDB [(none)]>

我的cnf

[root@host ~]# cat /etc/my.cnf
#
# This group is read both both by the client and the server
# use it for options that affect everything
#
[client-server]

#
# include all files from the config directory
#
!includedir /etc/my.cnf.d

[mysqld]
log-error=/var/lib/mysql/s102.halabtech.net.err
performance-schema=0
innodb_file_per_table=1
default-storage-engine=MyISAM
max_allowed_packet=268435456
open_files_limit=40000
skip-name-resolve

sql_mode=''

local-infile=0
connect_timeout=25
wait_timeout=30
interactive_timeout=30
slow-query-log=1
long_query_time=5
slow_query_log_file="/var/log/mysql-slow.log"

key_buffer_size = 128M
thread_stack = 128K
thread_cache_size = 8
max_heap_table_size = 256M
query_cache_limit = 4M
query_cache_size = 512M
innodb_buffer_pool_size = 2G

伺服器規格:

Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz

Memory: 4107488k/68927488k available (7792k kernel code, 1900528k absent, 1353716k reserved, 5950k data, 1984k init)

Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         32G     0   32G   0% /dev
tmpfs            32G     0   32G   0% /dev/shm
tmpfs            32G   28M   32G   1% /run
tmpfs            32G     0   32G   0% /sys/fs/cgroup
/dev/md2        437G  273G  142G  66% /
/dev/md1        488M  401M   62M  87% /boot
tmpfs           6.3G     0  6.3G   0% /run/user/0
tmpfs           6.3G     0  6.3G   0% /run/user/987
tmpfs           6.3G     0  6.3G   0% /run/user/1034

更新

MYSQL 慢查詢轉儲:(資料庫實名已更改)

Reading mysql slow query log from /var/log/mysql-slow.log
Count: 2  Time=20.58s (41s)  Lock=0.00s (0s)  Rows_sent=4089261.5 (8178523), Rows_examined=4089261.5 (8178523), Rows_affected=0.0 (0), root[root]@localhost
  SELECT /*!40001 SQL_NO_CACHE */ * FROM `hb_udownloads`

Count: 3  Time=15.38s (46s)  Lock=0.00s (0s)  Rows_sent=81.0 (243), Rows_examined=10026558.0 (30079674), Rows_affected=0.0 (0), server***_fusion[server***_fusion]@localhost
  SELECT *,(select count(id) from tbl_swkey
  where group_id=tbl_keygroup.id and used=0) as selled,(select count(id) from tbl_swkey
  where group_id=tbl_keygroup.id and used=1) as used FROM tbl_keygroup
  order by `displayorder` asc

Count: 1  Time=8.68s (8s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4090482.0 (4090482), Rows_affected=0.0 (0), support***_res[support***_res]@localhost
  UPDATE hb_udownloads SET `upackage_id` = 0 WHERE  upackage_id = 403775

Count: 1  Time=5.64s (5s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4088258.0 (4088258), Rows_affected=0.0 (0), support***_res[support***_res]@localhost
  SELECT  udownload_id AS retval FROM hb_udownloads WHERE  user_id = 415064  AND file_id = 78499  LIMIT 1

Count: 1  Time=5.58s (5s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4088256.0 (4088256), Rows_affected=0.0 (0), support***_res[support***_res]@localhost
  SELECT  udownload_id AS retval FROM hb_udownloads WHERE  user_id = 208286  AND file_id = 202629  LIMIT 1

Count: 1  Time=5.35s (5s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4088255.0 (4088255), Rows_affected=0.0 (0), support***_res[support***_res]@localhost
  SELECT  udownload_id AS retval FROM hb_udownloads WHERE  user_id = 235082  AND file_id = 473624  LIMIT 1

Count: 1  Time=5.21s (5s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4088254.0 (4088254), Rows_affected=0.0 (0), support***_res[supporth***_res]@localhost
  SELECT  udownload_id AS retval FROM hb_udownloads WHERE  user_id = 61350  AND file_id = 493488  LIMIT 1

Count: 1  Time=5.17s (5s)  Lock=0.00s (0s)  Rows_sent=0.0 (0), Rows_examined=4088259.0 (4088259), Rows_affected=0.0 (0), support***_res[support***_res]@localhost
  SELECT  udownload_id AS retval FROM hb_udownloads WHERE  user_id = 338554  AND file_id = 439150  LIMIT 1

請告知為什麼我有時會出現 500% 的峰值,以及應該採取什麼措施來解決該問題,因為我收到了很多 50 倍的錯誤。也請原諒我糟糕的英文。先感謝您。

答案1

高 CPU 使用率通常意味著大量的記憶體表掃描。

查看您的慢速查詢之一:

Time=5.64s (5s)  Rows_sent=0.0 (0)  Rows_examined=4088258.0 (4088258)
SELECT  udownload_id AS retval 
FROM hb_udownloads 
WHERE  user_id = 415064  
AND file_id = 78499  
LIMIT 1

該查詢正在讀取整個 4M 行表,但最多返回其中(在本例中,根本沒有)。
你需要索引在此表上支援此查詢。複合型的user_idfile_id將會是一個好的開始。

另一個效能殺手:

Time=20.58s (41s)  Rows_sent=4089261.5 (8178523)  Rows_examined=4089261.5 (8178523)
SELECT /*!40001 SQL_NO_CACHE */ * 
FROM `hb_udownloads`

切勿使用“選擇 *“在生產代碼中。
總是明確指定您想要的列。我猜想這個表最近一直在「成長」(增加列)。
好的,如果沒有 where 子句,無論如何都會進行表掃描,但這引出了一個問題 - 可憐的客戶端 [應用程式] 將做什麼這個查詢「拋出」了 4M 行?

答案2

其中一些查詢看起來可能需要最佳化。

例如:

Rows_examined=10026558.0 (30079674), Rows_affected=0.0 (0), server***_fusion[server***_fusion]@localhost
  SELECT *,(select count(id) from tbl_swkey
  where group_id=tbl_keygroup.id and used=0) as selled,(select count(id) from tbl_swkey
  where group_id=tbl_keygroup.id and used=1) as used FROM tbl_keygroup
  order by `displayorder` asc

此查詢正在掃描 30000000 行。

您可以更改這一點,例如更改為兩個查詢:

SELECT * FROM tbl_keygroup ORDER by `displayorder` asc;
SELECT count(id) FROM tbl_swkey WHERE used IN (0,1) GROUP BY used;

由於查詢本身涉及無用的聯接和兩次全表掃描,可以減半。

您所展示的大部分內容都指向查詢最佳化的問題,我將查看緩慢的查詢日誌,以確定以最少的表格掃描量檢索資料的最佳方法。

出於同樣的目的,您可能還想對大量資料建立索引。

答案3

** 這應該會使第一個更快:

SELECT  k.*, s.sold, s.used
    FROM ( SELECT group_id,
                  SUM(used = 0) AS sold,
                  SUM(used = 1) AS used
               FROM tbl_swkey
         ) AS s
    JOIN tbl_keygroup AS k  ON s.group_id = k.id
    ORDER BY displayorder

並且有

tbl_swkey:    INDEX(group_id, used)
tbl_keygroup: INDEX(displayorder)

如果您需要進一步討論此查詢,請提供SHOW CREATE TABLE兩個表格、表格大小和EXPLAIN SELECT ...

** 這

SELECT  udownload_id AS retval
    FROM  hb_udownloads
    WHERE  user_id = 415064
      AND  file_id = 78499
    LIMIT  1 

需要 3 列複合索引才能更快:

INDEX(user_id, file_id, udownload_id)

您不關心得到哪個匹配行?或者你應該添加一個ORDER BY?如果您添加ORDER BY,我的索引建議可能需要更改。

** 這是從哪裡來的mysqldump

SELECT /*!40001 SQL_NO_CACHE */ * FROM `hb_udownloads`

如果您對侵入性轉儲的程度不滿意,dba.stackexchange.com 上有一些關於該主題的討論。

相關內容