Ispconfig not processing job queue – spamfilter_policy error

Suddenly some of my slave servers stopped processing the job queue. Ispconfig version 3.2.9. Investigating the log files.

/var/log/ispconfig/cron.log shows:

PHP Fatal error:  Uncaught mysqli_sql_exception: Data truncated for column 'warnvirusrecip' at row 1 in /usr/local/ispconfig/server/lib/classes/db_mysql.inc.php:302

See this discussion: https://forum.howtoforge.com/threads/uncaught-mysqli_sql_exception-data-truncated-for-column-warnvirusrecip-at-row-1.90116/

This is how I solved it. On the master server:

mysqldump -uroot -p --add-drop-table dbispconfig spamfilter_policy > spamfilter_policy.sql

Then copy the spamfilter_policy.sql to each slave server having problems processing the queue and do:

mysql -uroot -p dbispconfig < spamfilter_policy.sql

Ispconfig3 monitor tab not opening when using non-english language [BUG – WORKAROUND]

This is caused by a bug when saving the translation file concerning the langauge key for “Let’s Encrypt log”.

When the translation file for the Monitor tab is saved, the necessary backslash before the ‘ in “Let’s” is not saved into the file. The problem is described in this bug report: https://git.ispconfig.org/ispconfig/ispconfig3/-/issues/5881 but for the moment not resolved.

To workaround this problem, manually edit the file /usr/local/ispconfig/interface/web/monitor/lib/lang/XX.lng (where XX is your language). Find the line beginning with:

$wb['Let's Encrypt log'] =

Insert a backslash like this:

$wb['Let\'s Encrypt log'] =

Save the file and your Monitor tab will work again. This must be repeated each time you edit the translations for monitor.

ISPConfig3 localized ‘Welcome to your new email account’ mail not working [SOLUTION]

I made a copy of the file welcome_email_en.txt from /usr/local/ispconfig/server/conf/mail to /usr/local/ispconfig/server/conf-custom/mail and named it welcome_email_se.txt and translated the content of the file, as I wanted my clients to be greeted to their new email account in Swedish. However, the client was still receiving the welcome email in English.

It turned out that the configuration file for ISPConfig has a parameter for language which controls what email templates to use. I thought it would use the country settings from the Client but it doesn’t.

In /usr/local/ispconfig/server/lib check if the file config.inc.local.php exists. If it doesn’t, create one (here is a template for it).

Add the line:

$conf['language'] = 'se';

By using config.inc.local.php and not directly editing config.inc.php you are making sure it doesn’t get overwritten in the next ISPConfig3 update.

In a multiserver setup (master/slaves) this must be done on all the slaves as the ‘Welcome to your new email account’-email is being generated on the server where the new mailbox is located.

Ispconfig3: Let’s Encrypt not working

There can be several reasons for Let’s Encrypt certificates not being issued for sites. These are some things to check:

  • When creating a new website, save it first, then open it again and check Let’s encrypt and then save it (i.e. don’t check the Let’s Encrypt unless the website has been saved at least once in Ispconfig3)
  • Delete any existing certificates under the SSL tab for the website, including self signed
  • Check that all host names for the website and webaliases are pointing to the correct server
  • Use A records, not CNAME
  • Make sure there are no redirects on the website
  • Check for error messages in the log file; /var/log/letsencrypt/letsencrypt.log
  • If you get messages in the letsencrypt.log file like;
    “Please choose an account
    Choises: “
    Then check /etc/letsencrypt/accounts/acme-v01.api.letsencrypt.org/directory and if there are more than one account (subdirectory) move all but one to another location.
  • Check the web logs for the site, both access and error logs, for accesses to .well-known/acme-challenge/* and verify that they are served properly (status code 200)

APS package installation_task stuck in ISPConfig3

When trying to install an APS package on a ISPConfig3 system, the package was never installed and the status “installation_task” remained for the site I tried to install a package.

Investigating /var/log/ispconfig/cron.log revealed:

PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /usr/local/ispconfig/server/lib/classes/aps_installer.inc.php:627

This problem can occur when curl is not installed on the target server, but that was not the problem in this case.

The server had been upgraded to Ubuntu 16.05 which runs PHP 7.0 instead of PHP 5. Curl was installed but the problem was that php7.0-curl was missing. The problem was resolved by:

sudo apt-get install php7.0-curl

The next issue that occured according to the logs was:

PHP Fatal error: Uncaught Error: Class 'ZipArchive' not found in /usr/local/ispconfig/server/lib/classes/aps_installer.inc.php:106

This problem was resolved by:

sudo apt-get install php7.0-zip

After this nothing more had to be done. The ISPConfig cron now successfully installed the APS package automatically.

Ispconfig 3.0 to 3.1 update issue – PHP-FPM and SSL problems

After updating a multi server installation of Ispconfig3 problems occured when editing existing or adding new websites on one of the servers.

  • If the site had SSL enabled the apache2 vhosts file for the site (located in /etc/apache2/sites-availible) would get empty values for SSLCertificateFile, SSLCertificateKeyFile and SSLCertificateChainFile fields, causing it to revert to old configuration or not load the site at all. At some point apache2 wouldn’t reload at all so to get it upp the site had to be disabled by removing /etc/apache2/sites-enabled/100-sitename.vhost file.
  • If the site was running under PHP-FPM the site would report Internal server error 501 caused by the socket file /var/lib/php5-fpm/webXXX.sock was owned by root and not by the web user. This in turned was caused by the fields listen.owner and listen.group in webXXX.conf (for PHP-FPM) was empty.

Since this was a multi server installation and this problem only occured on one server they could be compared. It turned out that the problem was caused in the directory /usr/local/ispconfig/server/plugins-enabled. The files in this directory should be symbolic links to the corresponding files in /usr/local/ispconfig/server/plugins-available, but after a move of the Ispconfig3 installation from an older server, these links had become real files instead of symbolic links.

This meant that the files in /usr/local/ispconfig/server/plugins-enabled was still Ispconfig 3.0 files while the ones in /usr/local/ispconfig/server/plugins-available was Ispconfig 3.1. To resolve the problem, to be on the safe side, make a backup of /usr/local/ispconfig/server/plugins-enabled and then run:

cd /usr/local/ispconfig/server/plugins-enabled
ln -s -f /usr/local/ispconfig/server/plugins-available/apache2_plugin.inc.php apache2_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/apps_vhost_plugin.inc.php apps_vhost_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/aps_plugin.inc.php aps_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/bind_plugin.inc.php bind_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/cron_jailkit_plugin.inc.php cron_jailkit_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/cron_plugin.inc.php cron_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/firewall_plugin.inc.php firewall_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/ftpuser_base_plugin.inc.php ftpuser_base_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/getmail_plugin.inc.php getmail_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/maildrop_plugin.inc.php maildrop_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/mailman_plugin.inc.php mailman_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/mail_plugin_dkim.inc.php mail_plugin_dkim.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/mail_plugin.inc.php mail_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/mysql_clientdb_plugin.inc.php mysql_clientdb_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/network_settings_plugin.inc.php network_settings_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/postfix_filter_plugin.inc.php postfix_filter_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/postfix_server_plugin.inc.php postfix_server_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/shelluser_base_plugin.inc.php shelluser_base_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/shelluser_jailkit_plugin.inc.php shelluser_jailkit_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/software_update_plugin.inc.php software_update_plugin.inc.php
ln -s -f /usr/local/ispconfig/server/plugins-available/webserver_plugin.inc.php webserver_plugin.inc.php

Verify with ls -l /usr/local/ispconfig/server/plugins-enabled that all your files are symbolic links.

 

Ispconfig3 unable to store client on DNS and email domain records

After upgrading from Ispconfig 3.0 to 3.1 odd things started to happen. For example, when saving a DNS secondary zone it would not store the client selected. Same thing when saving an email domian, client info and spam filter setting just would save.

The solution turned out to be simple. Gentlemen – clear your cookies and login again! Problem solved.

Ispconfig3 php5-fpm Error 500 Internal server error

When trying to change a client website on a Ispconfig3 host from Fast-CGI to PHP-FPM the page just gave Error 500 Internal server error. Investigating the site error log showed lines like:

[Sat Oct 22 10:39:56 2016] [error] [client xx.xx.xx.xx] (2)No such file or directory: FastCGI: failed to connect to server "/var/www/clients/client6/web447/cgi-bin/php5-fcgi-*-80-domain.xx": connect() failed
[Sat Oct 22 10:39:56 2016] [error] [client xx.xx.xx.xx] FastCGI: incomplete headers (0 bytes) received from server "/var/www/clients/client6/web447/cgi-bin/php5-fcgi-*-80-domain.xx"

Checking the status of PHP-FPM by the command:

service php5-fpm status

gave the result “not running”.

Investigating the PHP-log file, /var/log/php5-fpm.log, displayed lines like:

[22-Oct-2016 10:34:53] ERROR: [pool web405] cannot get uid for user 'web405'
[22-Oct-2016 10:34:53] ERROR: FPM initialization failed

However, the site widh id 405 and user web405 did no longer exist on the server and has been left there by Ispconfig3 for unknown reasons.

The solution was to manually remove the file /etc/php5/fpm/pool.d/web405.conf file and then:

service php5-fpm restart
service apache2 restart

Ispconfig3 on Ubuntu 12.04 upgrade to 14.04

Upgrading a system running Ispconfig3 on Ubuntu 12.04 (LTS) to 14.04 is quite straight forward. However there are some issues to consider before doing so. It might affect some of the sites that are being hosted.

  • Ubuntu 14.04 will move you from Apache 2.2 to 2.4
  • php will be upgraded from 5.3 to 5.5. Most modern CMS:es like Joomla and WordPress will run on php 5.5 but clients can be using other software or third party extensions that are not ready for php 5.5

Upgrade procedure:

  • Backup, backup and backup. And do some backup again.
  • Prepare your users for some downtime. The upgrade can take up to several hours depending on your server.
  • Upgrade all installed packages so you have the latest versions;
    apt-get update
    apt-get upgrade
  • Then run do-release-upgrade
  • During the upgrade process you probably will be prompted several times about configuration files that have been locally modified. I usually examine the differences using D option and in most cases I select to install the new configuration file using Y option.
  • After Ubuntu has been upgraded and the system has been rebooted you must reconfigure Ispconfig3. I did it by using the update procuedure even though I was running the latest Ispconfig before I upgraded Ubuntu:
    cd /tmp
    wget http://sourceforge.net/projects/ispconfig/files/ISPConfig%203/ISPConfig-3.0.5.4p5/ISPConfig-3.0.5.4p5.tar.gz
    tar xvzf ISPConfig-3.0.5.4p5.tar.gz
    cd ispconfig3_install/install
    php -q update.php
  • Allow Ispconfig3 to reconfigure your services
  • In my case Apache2 wouldn’t start after the upgrade. It was caused by the ruby module and since I don’t use it my simple solution for the moment was just to disable it:
    a2dismod ruby
    service apache2 restart
  • php imap extension had been disabled, so to fix it:
    php5enmod imap
    service apache2 restart
  • If you are hosting PrestaShop sites, you may need to disable php opcache described here or disable encryption by issuing the SQL command: UPDATE `ps_configuration` SET `value` = ‘0’ WHERE `name` = ‘PS_CIPHER_ALGORITHM’;
  • Apache2 configuration files has been moved from /etc/apache2/conf.d to /etc/apache2/conf-availible directory. Then to enable them you need to symlink the configuration file from /etc/apache2/conf-availible to /etc/apache2/conf-enabled and issue the command: service apache2 restart
  • You might experience problems with Postfix after the upgrade with log entries like fatal: no SASL authentication mechanisms and mail not being sent from the mailqueue. In that case:
    apt-get install sasl2-bin
    edit /etc/default/saslauthd and set START=yes
    /etc/init.d/saslauthd start
    service amavis restart
    service postfix restart

    I also had to comment out two lines in /etc/postfix/main.cf:
    #smtpd_sasl_path = private/auth
    #smtpd_sasl_type = dovecot

    And then do:
    service postfix restart

 

Ispconfig3 site cron not executing

When setting up a website in Ispconfig3 I wanted to run a cron job (shell script) for the site (setup under Sites -> Cron jobs) but the job did not execute. When examining the /var/log/auth.log I found lines like these:

Mar 22 10:31:01 servername jk_chrootsh[28726]: abort, homedir ‘/var/www/clients/client6/web284’ for user web284 (5015) does not contain the jail separator <jail>/./<home>

Mar 22 10:31:01 servername jk_chrootsh[28725]: abort, homedir ‘/var/www/clients/client1/web283’ for user web283 (5014) does not contain the jail separator <jail>/./<home>

The solution was to just add a dummy ssh user (using Jailkit as Chroot shell) in Ispconfig3 for the website.
I haven’t verified but I suspect the issue could be caused by the fact that the system originally was set up under Ubuntu 12.04 (LTS) and recently I did a do-release-upgrade to Ubuntu 14.04.