我的 apache 錯誤日誌檔案中有此類錯誤訊息:
[Wed Sep 27 18:45:34.289807 2017] [:error] [pid 17543] [client 66.249.79.21:47149] PHP Notice: Undefined variable: catNameUrl in /var/www/dev/page/page_display.inc.php on line 73
[Wed Sep 27 18:59:11.964934 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: getimagesize(../include/images/sizes/3GHI9W6AyEI5/--1000.jpg): failed to open stream: No such file or directory in /var/www/dev/page/index_framed.php on line 66
每小時1000次:
[Wed Sep 27 18:59:11.973506 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.973729 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.973878 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.974017 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
ETC...
事實上,所請求的頁面沒有引用者。這是一個正常的錯誤日誌:
[Wed Sep 27 09:09:39.877580 2017] [:error] [pid 29730] [client XX.XX.XX.XX:18888] PHP Notice: Undefined index: cat in /var/www/dev/page/page_display.inc.php on line 441, referer: https://dev.mywebsite.com/page/redaction/p/10
https://www.abuseipdb.com/check/66.249.79.17 -> 谷歌搜尋引擎蜘蛛
我認為,它打開了錯誤的網址,然後無法打開用於動態創建圖像的檔案。
當這個IP到達我的開發網站時,為什麼日誌中沒有Referer?
答案1
當這個IP到達我的開發網站時,為什麼日誌中沒有Referer?
機器人(包括 Googlebot)通常不會發送引用標頭。
另一個考慮因素是,如果直接訪問頁面,則無需發送引用標頭(在這些情況下,引用標頭通常為空,除非編程為不為空)。
更一般地說,您可能對此感興趣站長提問,其中列出了未發送引用標頭的其他原因(儘管上面列出的兩個是最有可能適用的原因)。
答案2
這很可能是 Google 抓取工具試圖將您網站的內容編入索引。所以我不會擔心。也就是說,你聲明了這一點:
每小時1000次…
並提供這些行作為範例:
[Wed Sep 27 18:59:11.973506 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.973729 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.973878 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
[Wed Sep 27 18:59:11.974017 2017] [:error] [pid 18906] [client 66.249.79.19:58443] PHP Warning: Division by zero in /var/www/dev/include/php/global_images.inc.php on line 335
關鍵部分是“PHP 警告:除以零...”,因為這是網站程式碼和腳本中的錯誤。您的兩個選擇是在生產伺服器上停用 PHP 警告,這將消除這些錯誤訊息,或者(我的首選解決方案)是打開global_images.inc.php
,轉到第 335 行,找出導致除零錯誤的原因並修補該錯誤。
無論您說什麼,您永遠無法完全控制機器人抓取您網站的方式。但如果您想限制錯誤日誌中的噪音,這完全在您的控制之中。清理程式碼或限制網站上的「警告」等級錯誤,不要為此失去更多睡眠。