Web server diagnosis and health. Which logs to watch?

Web server diagnosis and health. Which logs to watch?

Index

• WordPress log
• PHP log
• The web server log
• Nginx helper log
• MySQL log
• Cronjob log
• Fail2ban log
• Let’s Encrypt log
• UFW log

In computer science, a log file is a file that records events that occur in an operating system or other running software. To monitor the health of a web server it is necessary to analyze the various logs available in the system to identify any problems, incompatibilities, critical errors or slowdowns and bottlenecks.

Which logs do we need to analyze to get the most complete picture of the situation? The answer depends on what type of server you are using. In a shared web server you generally have access to only the web server log file, in some cases even PHP and nothing more.

This WordPress site is hosted on a DigitalOcean cloud server . Being an unmanaged service, I have full access via SSH to the machine and can manage all the software installed, such as the web server, PHP, MySQL, various and any applications. Basically I can do what I want with it.

When working in such a not-managed environment , it is important to keep an eye on the stability of the system and this translates into checking the various logs of the applications in use. Ubuntu or any operating system updates, WordPress updates, plugins change version, it may happen that new incompatibilities and new system errors arise that can slow down the machine making it less effective.

In this guide I show you the process I follow to analyze the health of a web server , it can certainly be improved but to start it is more than enough. Let’s see what are the logs I monitor on average every three / four months and after software changes.

WordPress log

Debugging PHP code is part of any project, WordPress comes with a specific debugging system designed to streamline the process and standardize code across cores, plugins, and themes. To enable this log you need to edit the wp-config.php file (found in the main WordPress folder) by adding the following code:

The line define (‘WP_DEBUG’, true) enables WP_DEBUG , a function that allows you to show all PHP errors, notifications and warnings in the WP backend. WP_DEBUG is a PHP constant (a permanent global variable) that can be used to activate the “debug” mode in WordPress. It is assumed to be false by default and is usually set to true in the wp-config.php file on development copies of WordPress.

Error messages can occur for things that work correctly but do not follow the proper data validation conventions in PHP. These warnings are easy to fix when the problematic code has been identified; the resulting code is almost always more resistant to bugs and easier to maintain.

The line define (‘WP_DEBUG_LOG’, true) enables the error log which is saved in /wp-content/debug.log.

The last two lines define (‘WP_DEBUG_DISPLAY’, true) and @ini_set (‘display_errors’, 1) enable the printing of errors in the WP backend. You can disable this feature by simply setting the two values ​​to false and 0 (zero).

WP_DEBUG_DISPLAY is another companion to WP_DEBUG which controls whether or not debug messages are displayed within the HTML of the pages. The default is true , which shows errors and warnings as they are generated. Setting to false will hide all errors. This should be used in conjunction with WP_DEBUG_LOG so that errors can be reviewed later.

PHP log

WordPress runs on PHP it is therefore essential to check the appearance of new errors every now and then to resolve them.

To view any problems you need to refer to the log dedicated to PHP where you can find errors related to outdated commands, memory errors, processes that fail and details on the files that caused the problem.

The log file has a name that varies according to the PHP version you are using, however you can find it in the / var / log / directory.

Operating system	Path	Guy
Debian-Based (Debian, Ubuntu, Linux Mint, etc.)	/etc/php/7.3/apache2/php.ini	Apache Module
Redhat / Fedora / CentOS	/etc/php.ini	All Versions
OpenSUSE	/etc/php/7.3/apache2/php.ini	Apache Module

The web server log

In the guide for the analysis of the web server log we have focused our attention on the access.log file, but the errors generated by the web server are not found in the access file but in the error.log file.

The Apache / Nginx error logs are used to log general error messages. If an error occurs in the web application, it is always a good idea to check the web server’s error log file to see if there is more information on the reason for the error.

In the event that the web server encounters particular situations, it saves the relative information in the error log (error.log). These “situations” fall into different levels of severity defined by a convention:

• Debug– Information to help you determine where the problem is.
• Info– Informative messages that are not necessary to read but which may be useful to know.
• Notice– something normal has happened that is worth noting.
• Warn– something unexpected happened, however it does not cause system instability.
• Error– Something was not successful.
• Crit– Problems that need to be resolved.
• Alert– Quick action is required.
• Emerg– The system is in an unusable state and requires immediate attention.

In this example you can see the export of the log file in HTML version through Goaccess .

Nginx helper log

For webmasters who use Nginx web servers and have installed the Nginx Helper plugin in WordPress it may be useful to have a look at the respective log every now and then. Nginx helper is a plugin that enables useful functions to manage the site on Nginx, such as:

• Removes index.php from permalinks when using WordPress with nginx.
• Adds support for deleting the Redis cache when used as a full page cache created using nginx-srcache-module.
• Adds support for nginx fastcgi_cache_purgeand proxy_cache_purge directive . Provides settings so that you can customize the purge rules.
• Adds support for nginx map {..}on a WordPress multi-site network installation.

MySQL log

MySQL also keeps track of errors in its log if it is activated. Check it cyclically especially after important updates.

MySQL generates three useful logs to analyze while developing any project.

• The error log. Contains information about errors that occur while the server is running (including starting and stopping the server).
• The general query log. General log of what mysqld is doing (connect, disconnect, query).
• The slow query log, my favorite. This log consists of SQL statements that take longer than long_query_timeseconds to execute and require at least min_examined_row_limit rows to be examined. The slow query log can be used to find queries that take a long time to execute and are therefore good candidates for optimization . However, examining a log of long and slow queries can be a time-consuming task. To make this easier, you can use the mysqldumpslow command to process a slow query log file and summarize its contents.

By default, no log files are enabled in MYSQL. All errors will be shown in syslog / var / log / syslog. To enable them just follow the steps below:

STEP 1: Go to this /etc/mysql/conf.d/mysqld_safe_syslog.cnf file and remove or comment out these lines.

STEP 2: Go to the mysql /etc/mysql/my.cnf conf file and add the following lines

To enable the error log, add the following:

To enable the general query log, add the following:

To enable the slow query log add the following:

STEP 3: Save the file and restart mysql using the following commands

To enable runtime logs, log into the mysql client with the command:

then type the following commands:

 

Cronjob log

Cron is one of the most useful tools you can find in any Unix-like operating system. It is used to schedule commands at a specific time. These scheduled commands or tasks are known as “Cron Jobs”. Cron is typically used to perform scheduled backups, monitor disk space, periodically delete files (such as log files) that are no longer needed, perform system maintenance tasks, and much more. If your machine has active cron jobs it is useful to monitor the status to identify any problems.

Cron logs everything in syslog, you can only filter Cron messages with GREP command:

Monitor Cron with syslog

Or if you really want to NERD, you can monitor the log in real time via shell with the TAIL -f command:

Fail2ban log

Have you installed Fail2ban to protect your server ? Well then you will also need to keep an eye on his log to see if anyone gets banned.

Also in this case the tail command is particularly NERD:

Let’s Encrypt log

Do you use Certbot or Let’s Encrypt to manage and update the certificates of your sites? Every now and then take a look at the log file to understand if the certificate updates were successful or if there are technical problems blocking the update.

UFW log

Have you enabled the UFW firewall on your web server? All right, every now and then check the log to figure out what’s going on.

Do you want to add more? I’d like to know what other logs you analyze, leave a comment!