PHP8.1 FPM은 매일 오전 12시 이후에 충돌합니다.

PHP8.1 FPM은 매일 오전 12시 이후에 충돌합니다.

지난 2~3일 동안 저는 PHP8.1-FPM 프로세스가 밤 12시~오전 12시 30분쯤에 충돌하는 것을 발견했습니다.

내 서버의 트래픽은 아침에 가장 많고 밤에 가장 적습니다. 그러나 FPM 프로세스는 하루 종일 잘 실행되고 밤에는 충돌이 발생합니다. 이것은 매우 이상합니다.

로그에서 찾은 내용은 다음과 같습니다.

1일차

[05-Dec-2022 00:19:47] NOTICE: [pool www] child 10868 started
[05-Dec-2022 00:25:02] NOTICE: [pool www] child 9967 exited with code 0 after 7446.963850 seconds from start
[05-Dec-2022 00:25:02] NOTICE: [pool www] child 10871 started
[05-Dec-2022 00:27:37] NOTICE: [pool www] child 9969 exited with code 0 after 7433.300005 seconds from start
[05-Dec-2022 00:27:37] NOTICE: [pool www] child 10872 started
[05-Dec-2022 00:32:41] WARNING: [pool www] child 10194 exited on signal 9 (SIGKILL) after 5713.957769 seconds from start
[05-Dec-2022 00:32:41] NOTICE: [pool www] child 10876 started
[05-Dec-2022 00:32:41] NOTICE: Terminating ...
[05-Dec-2022 00:32:41] NOTICE: exiting, bye-bye!

2일차

[05-Dec-2022 22:42:26] NOTICE: [pool www] child 23687 started
[05-Dec-2022 22:42:26] NOTICE: [pool www] child 21662 exited with code 0 after 11118.073861 seconds from start
[05-Dec-2022 22:42:26] NOTICE: [pool www] child 23688 started
[05-Dec-2022 22:42:31] NOTICE: [pool www] child 21663 exited with code 0 after 11071.941161 seconds from start
[05-Dec-2022 22:42:31] NOTICE: [pool www] child 23689 started
[05-Dec-2022 22:42:41] NOTICE: [pool www] child 21664 exited with code 0 after 11046.438863 seconds from start
[05-Dec-2022 22:42:41] NOTICE: [pool www] child 23690 started
[05-Dec-2022 23:31:34] NOTICE: [pool www] child 22058 exited with code 0 after 11163.770384 seconds from start
[05-Dec-2022 23:31:34] NOTICE: [pool www] child 24124 started
[06-Dec-2022 00:09:32] WARNING: [pool www] child 23076 exited on signal 9 (SIGKILL) after 8637.063701 seconds from start
[06-Dec-2022 00:09:32] NOTICE: [pool www] child 24580 started
[06-Dec-2022 00:09:32] WARNING: [pool www] child 23080 exited on signal 9 (SIGKILL) after 8607.986722 seconds from start
[06-Dec-2022 00:09:33] NOTICE: [pool www] child 24581 started
[06-Dec-2022 00:09:33] WARNING: [pool www] child 23089 exited on signal 9 (SIGKILL) after 8507.937877 seconds from start
[06-Dec-2022 00:09:33] NOTICE: [pool www] child 24582 started
[06-Dec-2022 00:09:33] WARNING: [pool www] child 23380 exited on signal 9 (SIGKILL) after 6752.142496 seconds from start
[06-Dec-2022 00:09:33] NOTICE: [pool www] child 24583 started
[06-Dec-2022 00:09:33] WARNING: [pool www] child 23403 exited on signal 9 (SIGKILL) after 6643.858307 seconds from start
[06-Dec-2022 00:09:33] NOTICE: [pool www] child 24584 started
[06-Dec-2022 00:09:33] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 0 idle, and 147 total children
[06-Dec-2022 00:09:33] NOTICE: Terminating ...
[06-Dec-2022 00:09:33] NOTICE: exiting, bye-bye!
[06-Dec-2022 00:23:49] NOTICE: fpm is running, pid 24611
[06-Dec-2022 00:23:49] NOTICE: ready to handle connections
[06-Dec-2022 00:23:49] NOTICE: systemd monitor interval set to 10000ms
[06-Dec-2022 00:23:53] WARNING: [pool www] seems busy (you may need to increase pm.start_servers, or pm.min/max_spare_servers), spawning 8 children, there are 59 idle, and 63 total children

서버 구성 Digitalocean 4 코어 AMD CPU 8GB RAM $54 USD 기본 드롭릿. PHP8.1 FPM 설정

pm = dynamic 
pm.max_children = 241 
pm.start_servers = 60
pm.min_spare_servers = 60 
pm.max_spare_servers = 180 
pm.max_requests = 500

-- 업데이트 이 애플리케이션은 이미지 서버입니다. 객체 스토리지의 동적으로 크기가 조정된 이미지 또는 다른 서버의 이미지를 제공합니다. 이미지 크기를 조정하고 제공하는 매우 작은 PHP 스크립트가 있습니다. Nginx는 처리된 이미지를 캐싱하는 데 사용됩니다.

-- dmesg 출력 업데이트

[147949.035475] 0 pages in swap cache
[147949.035475] Swap cache stats: add 0, delete 0, find 0/0
[147949.035476] Free swap  = 0kB
[147949.035477] Total swap = 0kB
[147949.035478] 2097014 pages RAM
[147949.035478] 0 pages HighMem/MovableOnly
[147949.035479] 61947 pages reserved
[147949.035480] 0 pages hwpoisoned
[147949.035480] Tasks state (memory values in pages):
[147949.035481] [  pid  ]   uid  tgid total_vm      rss pgtables_bytes swapents oom_score_adj name
[147949.035485] [    380]     0   380     9935      864   106496        0          -250 systemd-journal
[147949.035489] [    417]     0   417    72328     6775   114688        0         -1000 multipathd
[147949.035496] [    539]   103   539    22338      415    73728        0             0 systemd-timesyn
[147949.035498] [    606]   100   606     4029      626    73728        0             0 systemd-network
[147949.035500] [    625]   101   625     6378     1681    94208        0             0 systemd-resolve
[147949.035502] [    650]     0   650     2880     1060    69632        0         -1000 systemd-udevd
[147949.035505] [    747]     0   747     1821      618    57344        0             0 cron
[147949.035506] [    749]   102   749     2191      891    57344        0          -900 dbus-daemon
[147949.035508] [    752]   998   752   345261     1828   208896        0          -900 do-agent
[147949.035511] [    757]     0   757    20699      669    61440        0             0 irqbalance
[147949.035513] [    758]     0   758     8249     2870   102400        0             0 networkd-dispat
[147949.035515] [    767]     0   767    58621      410    81920        0             0 polkitd
[147949.035517] [    768]   104   768    55600     1093    77824        0             0 rsyslogd
[147949.035522] [    771]     0   771   292597     4332   307200        0          -900 snapd
[147949.035524] [    776]     0   776     3873      574    69632        0             0 systemd-logind
[147949.035526] [    778]     0   778    98136     1305   122880        0             0 udisksd
[147949.035529] [    786]     0   786    79236      954   118784        0             0 ModemManager
[147949.035531] [    787]     0   787     1554      192    49152        0             0 agetty
[147949.035534] [    798]     0   798     1543      214    53248        0             0 agetty
[147949.035535] [    826]     0   826     3855     1169    73728        0         -1000 sshd
[147949.035538] [    863]     0   863    27523     2822   114688        0             0 unattended-upgr
[147949.035540] [    904]     0   904    40076     1619   110592        0             0 nginx
[147949.035541] [   1373]     0  1373    73985     1009   163840        0             0 packagekitd
[147949.035544] [   1712]     0  1712   343392      174   180224        0          -900 droplet-agent
[147949.035547] [  12216]     0 12216    53067     2645   151552        0             0 php-fpm8.1
[147949.035549] [  12743]    33 12743    50015    15472   225280        0             0 nginx
[147949.035552] [  12744]    33 12744    51002    16135   233472        0             0 nginx
[147949.035554] [  12745]    33 12745    50614    16019   233472        0             0 nginx
[147949.035556] [  12746]    33 12746    50765    16062   233472        0             0 nginx
[147949.035558] [  12747]    33 12747    40214     5403   147456        0             0 nginx
[147949.035560] [  23025]    33 23025    81250    26391   356352        0             0 php-fpm8.1
[147949.035562] [  23027]    33 23027    63171    13406   245760        0             0 php-fpm8.1
[147949.035564] [  23028]    33 23028    66013    15875   266240        0             0 php-fpm8.1
[147949.035566] [  23029]    33 23029    81755    23040   335872        0             0 php-fpm8.1
[147949.035568] [  23030]    33 23030    81289    25643   352256        0             0 php-fpm8.1
[147949.035570] [  23031]    33 23031    65772    15814   266240        0             0 php-fpm8.1
[147949.035572] [  23032]    33 23032    60680    10956   225280        0             0 php-fpm8.1
[147949.035574] [  23033]    33 23033    81290    22731   327680        0             0 php-fpm8.1
[147949.035576] [  23034]    33 23034    79989    22259   327680        0             0 php-fpm8.1
[147949.035578] [  23035]    33 23035    62802    12404   241664        0             0 php-fpm8.1
[147949.035579] [  23036]    33 23036    81250    22808   327680        0             0 php-fpm8.1
[147949.035581] [  23037]    33 23037    81760    23549   331776        0             0 php-fpm8.1
[147949.035584] [  23038]    33 23038    63141    12398   237568        0             0 php-fpm8.1
[147949.035585] [  23040]    33 23040    65876    16118   266240        0             0 php-fpm8.1
[147949.035588] [  23041]    33 23041    54628     5099   176128        0             0 php-fpm8.1
[147949.035593] [  23042]    33 23042    80737    22410   323584        0             0 php-fpm8.1
[147949.035595] [  23043]    33 23043    63388    12852   245760        0             0 php-fpm8.1
[147949.035596] [  23044]    33 23044    79976    26488   364544        0             0 php-fpm8.1
[147949.035599] [  23045]    33 23045    81353    25677   352256        0             0 php-fpm8.1
[147949.035601] [  23046]    33 23046    64263    14482   253952        0             0 php-fpm8.1
[147949.035603] [  23048]    33 23048    81247    28102   368640        0             0 php-fpm8.1
[147949.035604] [  23049]    33 23049    81763    17517   286720        0             0 php-fpm8.1
[147949.035606] [  23050]    33 23050    54633     5260   176128        0             0 php-fpm8.1
[147949.035608] [  23051]    33 23051    81745    27413   368640        0             0 php-fpm8.1
[147949.035610] [  23052]    33 23052    81250    20292   307200        0             0 php-fpm8.1
[147949.035612] [  23053]    33 23053    62492    12051   237568        0             0 php-fpm8.1
[147949.035614] [  23054]    33 23054    79479    19786   311296        0             0 php-fpm8.1
[147949.035616] [  23055]    33 23055    82274    24380   339968        0             0 php-fpm8.1
[147949.035618] [  23056]    33 23056    81762    23765   335872        0             0 php-fpm8.1
[147949.035619] [  23057]    33 23057    60234    10616   221184        0             0 php-fpm8.1
[147949.035624] [  23058]    33 23058    62266    12114   237568        0             0 php-fpm8.1
[147949.035626] [  23059]    33 23059    64558    15277   253952        0             0 php-fpm8.1
[147949.035628] [  23060]    33 23060    81670    17767   286720        0             0 php-fpm8.1
[147949.035630] [  23061]    33 23061    81740    23569   335872        0             0 php-fpm8.1
[147949.035632] [  23062]    33 23062    81765    27157   364544        0             0 php-fpm8.1
[147949.035634] [  23063]    33 23063    55149     5538   180224        0             0 php-fpm8.1
[147949.035636] [  23064]    33 23064    53607     4475   167936        0             0 php-fpm8.1
[147949.035638] [  23065]    33 23065    81251    26365   356352        0             0 php-fpm8.1
[147949.035640] [  23066]    33 23066    79991    19315   303104        0             0 php-fpm8.1
[147949.035642] [  23067]    33 23067    79803    20677   315392        0             0 php-fpm8.1
[147949.035643] [  23068]    33 23068    65398    16096   262144        0             0 php-fpm8.1
[147949.035645] [  23069]    33 23069    62633    12246   233472        0             0 php-fpm8.1
[147949.035647] [  23070]    33 23070    79262    19484   299008        0             0 php-fpm8.1
[147949.035649] [  23071]    33 23071    79788    30080   376832        0             0 php-fpm8.1
[147949.035651] [  23072]    33 23072    62276    12389   237568        0             0 php-fpm8.1
[147949.035653] [  23073]    33 23073    79974    21148   323584        0             0 php-fpm8.1
[147949.035655] [  23074]    33 23074    81250    20299   307200        0             0 php-fpm8.1
[147949.035656] [  23075]    33 23075    63702    13680   249856        0             0 php-fpm8.1
[147949.035658] [  23077]    33 23077    80732    17341   282624        0             0 php-fpm8.1
[147949.035660] [  23078]    33 23078    79772    20834   303104        0             0 php-fpm8.1
[147949.035662] [  23079]    33 23079    65346    15686   262144        0             0 php-fpm8.1
[147949.035664] [  23081]    33 23081    63206    13763   245760        0             0 php-fpm8.1
[147949.035666] [  23082]    33 23082    81761    24195   339968        0             0 php-fpm8.1
[147949.035667] [  23083]    33 23083    64690    15490   258048        0             0 php-fpm8.1
[147949.035669] [  23084]    33 23084    54811     5039   176128        0             0 php-fpm8.1
[147949.035674] [  23085]    33 23085    63144    12510   237568        0             0 php-fpm8.1
[147949.035676] [  23086]    33 23086    62481    12562   237568        0             0 php-fpm8.1
[147949.035680] [  23087]    33 23087    65662    15683   262144        0             0 php-fpm8.1
[147949.035682] [  23088]    33 23088    81757    27232   364544        0             0 php-fpm8.1
[147949.035684] [  23090]    33 23090    81762    18209   290816        0             0 php-fpm8.1
[147949.035686] [  23091]    33 23091    65152    15280   258048        0             0 php-fpm8.1
[147949.035688] [  23092]    33 23092    79988    21290   327680        0             0 php-fpm8.1
[147949.035690] [  23171]    33 23171    81760    18679   294912        0             0 php-fpm8.1
[147949.035692] [  23173]    33 23173    79410    17029   286720        0             0 php-fpm8.1
[147949.035694] [  23175]    33 23175    65824    16263   266240        0             0 php-fpm8.1
[147949.035696] [  23185]    33 23185    65365    16180   262144        0             0 php-fpm8.1
[147949.035698] [  23186]    33 23186    65231    15066   262144        0             0 php-fpm8.1
[147949.035699] [  23187]    33 23187    79988    20002   315392        0             0 php-fpm8.1
[147949.035701] [  23375]    33 23375    68209    18499   282624        0             0 php-fpm8.1
[147949.035703] [  23378]    33 23378    60998    11722   225280        0             0 php-fpm8.1
[147949.035705] [  23379]    33 23379    79435    18467   286720        0             0 php-fpm8.1
[147949.035707] [  23388]    33 23388    79258    21188   319488        0             0 php-fpm8.1
[147949.035709] [  23389]    33 23389    79453    20039   311296        0             0 php-fpm8.1
[147949.035711] [  23390]    33 23390    61517    11708   229376        0             0 php-fpm8.1
[147949.035713] [  23391]    33 23391    55516     5670   184320        0             0 php-fpm8.1
[147949.035715] [  23392]    33 23392    63335    12141   237568        0             0 php-fpm8.1
[147949.035717] [  23393]    33 23393    81245    19769   303104        0             0 php-fpm8.1
[147949.035721] [  23394]    33 23394    80731    18069   290816        0             0 php-fpm8.1
[147949.035723] [  23395]    33 23395    81229    17634   290816        0             0 php-fpm8.1
[147949.035725] [  23396]    33 23396    62102    11782   233472        0             0 php-fpm8.1
[147949.035727] [  23397]    33 23397    61251    11551   229376        0             0 php-fpm8.1
[147949.035729] [  23398]    33 23398    65025    15476   258048        0             0 php-fpm8.1
[147949.035731] [  23399]    33 23399    56609     6542   192512        0             0 php-fpm8.1
[147949.035733] [  23400]    33 23400    61918    11956   233472        0             0 php-fpm8.1
[147949.035735] [  23401]    33 23401    78978    25135   352256        0             0 php-fpm8.1
[147949.035736] [  23402]    33 23402    61182    11420   229376        0             0 php-fpm8.1
[147949.035738] [  23404]    33 23404    65174    15748   262144        0             0 php-fpm8.1
[147949.035740] [  23405]    33 23405    81247    22004   319488        0             0 php-fpm8.1
[147949.035742] [  23406]    33 23406    53598     4420   167936        0             0 php-fpm8.1
[147949.035744] [  23407]    33 23407    61984    11988   233472        0             0 php-fpm8.1
[147949.035746] [  23408]    33 23408    63977    14485   249856        0             0 php-fpm8.1
[147949.035748] [  23409]    33 23409    81247    25246   348160        0             0 php-fpm8.1
[147949.035749] [  23410]    33 23410    64153    14631   253952        0             0 php-fpm8.1
[147949.035751] [  23411]    33 23411    81279    23444   331776        0             0 php-fpm8.1
[147949.035753] [  23412]    33 23412    65221    15610   262144        0             0 php-fpm8.1
[147949.035755] [  23413]    33 23413    81753    27004   360448        0             0 php-fpm8.1
[147949.035757] [  23414]    33 23414    63471    14123   245760        0             0 php-fpm8.1
[147949.035759] [  23415]    33 23415    63396    14222   245760        0             0 php-fpm8.1
[147949.035761] [  23467]    33 23467    80729    15421   266240        0             0 php-fpm8.1
[147949.035763] [  23469]    33 23469    65609    15655   262144        0             0 php-fpm8.1
[147949.035767] [  23470]    33 23470    63897    14733   249856        0             0 php-fpm8.1
[147949.035769] [  23472]    33 23472    81239    22790   327680        0             0 php-fpm8.1
[147949.035771] [  23473]    33 23473    62872    13170   241664        0             0 php-fpm8.1
[147949.035773] [  23474]    33 23474    64835    15560   258048        0             0 php-fpm8.1
[147949.035775] [  23475]    33 23475    65215    15739   262144        0             0 php-fpm8.1
[147949.035777] [  23476]    33 23476    63137    12595   237568        0             0 php-fpm8.1
[147949.035778] [  23477]    33 23477    78749    17139   282624        0             0 php-fpm8.1
[147949.035780] [  23478]    33 23478    81236    22468   323584        0             0 php-fpm8.1
[147949.035783] [  23479]    33 23479    81238    22144   319488        0             0 php-fpm8.1
[147949.035784] [  23480]    33 23480    61533    12097   229376        0             0 php-fpm8.1
[147949.035786] [  23481]    33 23481    61742    12461   233472        0             0 php-fpm8.1
[147949.035788] [  23668]    33 23668    81120    27387   364544        0             0 php-fpm8.1
[147949.035790] [  23669]    33 23669    81281    25762   352256        0             0 php-fpm8.1
[147949.035792] [  23670]    33 23670    80711    22075   323584        0             0 php-fpm8.1
[147949.035794] [  23671]    33 23671    81629    21215   311296        0             0 php-fpm8.1
[147949.035796] [  23672]    33 23672    54106     4783   172032        0             0 php-fpm8.1
[147949.035798] [  23673]    33 23673    80732    20166   303104        0             0 php-fpm8.1
[147949.035800] [  23674]    33 23674    62566    13115   237568        0             0 php-fpm8.1
[147949.035802] [  23675]    33 23675    62266    11933   237568        0             0 php-fpm8.1
[147949.035804] [  23676]    33 23676    62778    12260   237568        0             0 php-fpm8.1
[147949.035805] [  23677]    33 23677    93296    27104   364544        0             0 php-fpm8.1
[147949.035807] [  23678]    33 23678    79960    26685   364544        0             0 php-fpm8.1
[147949.035809] [  23679]    33 23679    79235    17993   286720        0             0 php-fpm8.1
[147949.035814] [  23680]    33 23680    63334    13892   245760        0             0 php-fpm8.1
[147949.035816] [  23681]    33 23681    82264    22326   323584        0             0 php-fpm8.1
[147949.035817] [  23682]    33 23682    62045    12131   233472        0             0 php-fpm8.1
[147949.035819] [  23683]    33 23683    81649    24578   344064        0             0 php-fpm8.1
[147949.035821] [  23684]    33 23684    81755    25202   348160        0             0 php-fpm8.1
[147949.035823] [  23685]    33 23685    56617     6875   192512        0             0 php-fpm8.1
[147949.035825] [  23686]    33 23686    55403     5763   180224        0             0 php-fpm8.1
[147949.035827] [  23687]    33 23687    78956    21007   319488        0             0 php-fpm8.1
[147949.035829] [  23688]    33 23688    58896     9008   208896        0             0 php-fpm8.1
[147949.035830] [  23689]    33 23689    61751    11976   233472        0             0 php-fpm8.1
[147949.035832] [  23690]    33 23690    79253    29982   372736        0             0 php-fpm8.1
[147949.035834] [  23803]     0 23803     4285      846    73728        0             0 sshd
[147949.035836] [  23806]     0 23806     4260      971    73728        0             0 systemd
[147949.035838] [  23807]     0 23807    42651     1255    90112        0             0 (sd-pam)
[147949.035840] [  23863]     0 23863     2287      943    53248        0             0 bash
[147949.035842] [  23878]     0 23878     2152      868    53248        0             0 htop
[147949.035843] [  24124]    33 24124    63362    13980   245760        0             0 php-fpm8.1
[147949.035846] [  24573]    33 24573    54409     4318   176128        0             0 php-fpm8.1
[147949.035848] [  24574]     0 24574    53067     1478   126976        0             0 php-fpm8.1
[147949.035850] [  24575]     0 24575    53150     1663   126976        0             0 php-fpm8.1
[147949.035854] [  24576]     0 24576    53067     1235   122880        0             0 php-fpm8.1
[147949.035856] [  24577]     0 24577    53067     1589   122880        0             0 php-fpm8.1
[147949.035857] [  24578]     0 24578    53067     1591   122880        0             0 php-fpm8.1
[147949.035859] [  24579]     0 24579    53067     1572   122880        0             0 php-fpm8.1
[147949.035861] [  24580]     0 24580    53067     1615   122880        0             0 php-fpm8.1
[147949.035863] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=systemd-resolved.service,mems_allowed=0,global_oom,task_memcg=/system.slice/php8.1-fpm.service,task=php-fpm8.1,pid=23071,uid=33
[147949.035882] Out of memory: Killed process 23071 (php-fpm8.1) total-vm:319152kB, anon-rss:112388kB, file-rss:1824kB, shmem-rss:6108kB, UID:33 pgtables:368kB oom_score_adj:0

답변1

이 시점에서 서버의 메모리가 부족해졌습니다.

오래 지속되고 메모리 집약적인 여러 작업을 시작하는 예약된 작업이 있을 수 있습니다. 그러면 Linux 커널 메모리 부족 킬러가 실행되어 대부분의 메모리를 소비하는 프로세스가 종료됩니다.

메모리나 스왑 공간을 늘려야 합니다. 또는 애플리케이션의 메모리 사용량을 조사하여 줄일 수 있는지 확인하세요.

관련 정보