mysql によるサーバーの高負荷

mysql によるサーバーの高負荷

私の 8 コア サーバーは一定の負荷がかかっており、理想的な時間も 0 です。以下は、mysqltuner.pl スクリプトの出力です。負荷を少なくするためにどのパラメータを変更する必要があるか、どなたか教えていただけませんか?

-bash-4.1# ./mysqltuner.pl
 >>  MySQLTuner 1.7.7 - Major Hayden <[email protected]>
 >>  Bug reports, feature requests, and downloads at http://mysqltuner.com/
 >>  Run with '--help' for additional options and output filtering

[--] Skipped version check for MySQLTuner script
Please enter your MySQL administrative login: root
Please enter your MySQL administrative password: [OK] Currently running supported MySQL version 5.1.73
[OK] Operating on 64-bit architecture

-------- Log file Recommendations ------------------------------------------------------------------
[--] Log file: /var/log/mysqld.log(442M)
[OK] Log file /var/log/mysqld.log exists
[OK] Log file /var/log/mysqld.log is readable.
[OK] Log file /var/log/mysqld.log is not empty
[!!] Log file /var/log/mysqld.log is bigger than 32 Mb
[!!] /var/log/mysqld.log contains 1 warning(s).
[!!] /var/log/mysqld.log contains 857631 error(s).
[--] 360 start(s) detected in /var/log/mysqld.log
[--] 1) 180301 15:15:36 [Note] /usr/libexec/mysqld: ready for connections.
[--] 2) 180227 16:44:00 [Note] /usr/libexec/mysqld: ready for connections.
[--] 3) 180222 14:29:14 [Note] /usr/libexec/mysqld: ready for connections.
[--] 4) 180214 12:59:28 [Note] /usr/libexec/mysqld: ready for connections.
[--] 5) 180212 16:17:48 [Note] /usr/libexec/mysqld: ready for connections.
[--] 6) 180212 12:34:39 [Note] /usr/libexec/mysqld: ready for connections.
[--] 7) 180205 11:54:19 [Note] /usr/libexec/mysqld: ready for connections.
[--] 8) 180124 15:32:20 [Note] /usr/libexec/mysqld: ready for connections.
[--] 9) 180124 14:58:11 [Note] /usr/libexec/mysqld: ready for connections.
[--] 10) 171206 18:08:52 [Note] /usr/libexec/mysqld: ready for connections.
[--] 28 shutdown(s) detected in /var/log/mysqld.log
[--] 1) 180301 15:15:34 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 2) 180227 16:43:59 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 3) 180222 14:29:12 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 4) 180214 12:59:26 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 5) 180212 16:17:47 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 6) 180212 12:34:37 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 7) 180205 11:54:18 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 8) 180124 15:32:19 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 9) 180124 14:58:10 [Note] /usr/libexec/mysqld: Shutdown complete
[--] 10) 171206 18:08:51 [Note] /usr/libexec/mysqld: Shutdown complete

-------- Storage Engine Statistics -----------------------------------------------------------------
[--] Status: +CSV +InnoDB +MRG_MYISAM
[--] Data in MyISAM tables: 65G (Tables: 122)
[--] Data in InnoDB tables: 4G (Tables: 621)
[!!] Total fragmented tables: 1

-------- Security Recommendations ------------------------------------------------------------------
[OK] There are no anonymous accounts for any database users
[OK] All database users have passwords assigned
[!!] User 'cdn@localhost' has user name as password.
[!!] There is no basic password file list!

-------- CVE Security Recommendations --------------------------------------------------------------
[--] Skipped due to --cvefile option undefined

-------- Performance Metrics -----------------------------------------------------------------------
[--] Up for: 20d 21h 19m 50s (6M q [3.409 qps], 13K conn, TX: 8G, RX: 11G)
[--] Reads / Writes: 23% / 77%
[--] Binary logging is disabled
[--] Physical Memory     : 3.7G
[--] Max MySQL memory    : 449.2M
[--] Other process memory: 514.8M
[--] Total buffers: 34.0M global + 2.7M per thread (151 max threads)
[--] P_S Max memory usage: 0B
[--] Galera GCache Max memory usage: 0B
[OK] Maximum reached memory usage: 163.2M (4.36% of installed RAM)
[OK] Maximum possible memory usage: 449.2M (12.01% of installed RAM)
[OK] Overall possible memory usage with other process is compatible with memory available
[OK] Slow queries: 0% (1K/6M)
[OK] Highest usage of available connections: 31% (47/151)
[OK] Aborted connections: 0.06%  (8/13524)
[!!] name resolution is active : a reverse name resolution is made for each new connection and can reduce performance
[!!] Query cache may be disabled by default due to mutex contention.
[!!] Query cache efficiency: 0.0% (0 cached / 1M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (7 temp sorts / 8K sorts)
[OK] No joins without indexes
[OK] Temporary tables created on disk: 0% (0 on disk / 119K total)
[!!] Thread cache is disabled
[!!] Table cache hit rate: 3% (64 open / 2K opened)
[OK] Open file limit used: 0% (182/50K)
[OK] Table locks acquired immediately: 99% (7M immediate / 7M locks)

-------- Performance schema ------------------------------------------------------------------------
[--] Performance schema is disabled.
[--] Memory used by P_S: 0B
[--] Sys schema isn't installed.

-------- ThreadPool Metrics ------------------------------------------------------------------------
[--] ThreadPool stat is disabled.

-------- MyISAM Metrics ----------------------------------------------------------------------------
[OK] Key buffer used: 100.0% (8M used / 8M cache)
[!!] Key buffer size / total MyISAM indexes: 8.0M/25.3G
[!!] Read Key buffer hit rate: 87.0% (530M cached / 68M reads)
[!!] Write Key buffer hit rate: 69.8% (83M cached / 58M writes)

-------- InnoDB Metrics ----------------------------------------------------------------------------
[--] InnoDB is enabled.
[--] InnoDB Thread Concurrency: 8
[!!] InnoDB File per table is not activated
[!!] InnoDB buffer pool / data size: 8.0M/4.0G
[!!] Ratio InnoDB log file size / InnoDB Buffer pool size (125 %): 5.0M * 2/8.0M should be equal 25%
[--] InnoDB Buffer Pool Chunk Size not used or defined in your version
[OK] InnoDB Read buffer efficiency: 99.97% (3114982800 hits/ 3115816604 total)
[!!] InnoDB Write Log efficiency: 83.44% (9668047 hits/ 11586898 total)
[!!] InnoDB log waits: 0.01% (107 waits / 1918851 writes)

-------- AriaDB Metrics ----------------------------------------------------------------------------
[--] AriaDB is disabled.

-------- TokuDB Metrics ----------------------------------------------------------------------------
[--] TokuDB is disabled.

-------- XtraDB Metrics ----------------------------------------------------------------------------
[--] XtraDB is disabled.

-------- RocksDB Metrics ---------------------------------------------------------------------------
[--] RocksDB is disabled.

-------- Spider Metrics ----------------------------------------------------------------------------
[--] Spider is disabled.

-------- Connect Metrics ---------------------------------------------------------------------------
[--] Connect is disabled.

-------- Galera Metrics ----------------------------------------------------------------------------
[--] Galera is disabled.

-------- Replication Metrics -----------------------------------------------------------------------
[--] Galera Synchronous replication: NO
[--] No replication slave(s) for this server.
[--] Binlog format: STATEMENT
[--] XA support enabled: ON
Use of uninitialized value in concatenation (.) or string at ./mysqltuner.pl
        line 233 (#1)
    (W uninitialized) An undefined value was used as if it were already
    defined.  It was interpreted as a "" or a 0, but maybe it was a mistake.
    To suppress this warning assign a defined value to your variables.

    To help you figure out what was undefined, perl will try to tell you the
    name of the variable (if any) that was undefined. In some cases it cannot
    do this, so it also tells you what operation you used the undefined value
    in.  Note, however, that perl optimizes your program and the operation
    displayed in the warning may not necessarily appear literally in your
    program.  For example, "that $foo" is usually optimized into "that "
    . $foo, and the warning will refer to the concatenation (.) operator,
    even though there is no . in your program.

[--]
[--]
[--] This is a standalone server

-------- Recommendations ---------------------------------------------------------------------------
General recommendations:
    /var/log/mysqld.log is > 32Mb, you should analyze why or implement a rotation log strategy such as logrotate!
    Control warning line(s) into /var/log/mysqld.log file
    Control error line(s) into /var/log/mysqld.log file
    Run OPTIMIZE TABLE to defragment tables for better performance
      OPTIMIZE TABLE `tendersinfo_com`.`tenders_archive_new`; -- can free 4628.83363723755 MB
    Total freed space after theses OPTIMIZE TABLE : 4628.83363723755 Mb
    Set up a Secure Password for user@host ( SET PASSWORD FOR 'user'@'SpecificDNSorIp' = PASSWORD('secure_password'); )
    Enable the slow query log to troubleshoot bad queries
    Configure your accounts with ip or subnets only, then update your configuration with skip-name-resolve=1
    Set thread_cache_size to 4 as a starting value
    Increase table_open_cache gradually to avoid file descriptor limits
    Read this before increasing table_open_cache over 64: 
    This is MyISAM only table_cache scalability problem, InnoDB not affected.
    See more details here: https://bugs.mysql.com/bug.php?id=49177
    This bug already fixed in MySQL 5.7.9 and newer MySQL versions.
    Beware that open_files_limit (50000) variable
    should be greater than table_open_cache (64)
    Read this before changing innodb_log_file_size and/or innodb_log_files_in_group: 
Variables to adjust:
    query_cache_size (=0)
    query_cache_type (=0)
    query_cache_limit (> 1M, or use smaller result sets)
    thread_cache_size (start at 4)
    table_open_cache (> 64)
    key_buffer_size (> 25.3G)
    innodb_file_per_table=ON
    innodb_buffer_pool_size (>= 4G) if possible.
    innodb_log_file_size should be (=1M) if possible, so InnoDB total log files size equals to 25% of buffer pool size.
    innodb_log_buffer_size (>= 1M)

答え1

明らかに、mysql は問題ではありません。そうでなければ、ひどいメトリックが表示されます。CPU を消費しているプロセス (top、vmstat、sar ...) を確認し、それがデータベースのエンド ユーザーに直接影響しているかどうかを確認します (ログには表示されていないようです: 遅いクエリはありません)。

答え2

Mysqltuner によるテーブルを最適化するための提案を使用しますtendersinfo_comtenders_archive_new;

4GB の断片化されたテーブル領域を解放できます。

my.cnf-ini [mysqld] セクションの提案 (mysqltuner レポートから取得)

key_buffer_size=24M  # from 8M MyISAM needs more RAM to work well
innodb_buffer_pool_size=2500M  # from 8M  for 62% of RAM
max_connections=100  # from 151 - 47 max_used_connections
thread_cache_size=100  # from 0 for V8 cap of 100 to reduce threads_created
table_open_cache=500  # from 64 - to support ~750 tables
open_file_limit=5000  # from 50000 - for more reasonable limit
expire_logs_days=30  # from 0 for limited historical logs & v8 default days
query_cache_limit=1K  # from 1M conserve RAM and QC not actively used
query_cache_min_res_unit=512  # from 4096 to store more RESULTS if QC ever used

提案を適用した後に追加情報があれば、インスタンスを改善する機会が増えます。 RAM の可用性の増加は真剣に検討する必要があります。 4G で 69G のデータをサポートするのは素晴らしい成果です。ありがとう、MySQL。

答え3

my.cnf-ini [mysqld] セクションに考慮すべき提案

MyISAM に深く関与していることに気付きました。

key_cache_age_threshold-64800  # from 300 seconds to reduce key_reads
key_cache_division_limit=50  # from 100 for HOT/WARM cache
key_cache_block_size=16384  # from 1024 to make more space available when aged out

関連情報