FreeNAS, VMware and iSCSI with MTU 9000

In order to improve perfomance there are recommendations to set the MTU for the iSCSI interface to MTU 9000 instead of the default 1500.

To do this, it important that you set MTU to 9000 on all devices involved; FreeNAS network interface, VMware NICs and vSwitch and all switch ports on the switches connecting those.

Note: When changing these settings, it will cause loss of connectivity so it might be a good idea not to do it on systems in production.

At one point I had forgotten to set the MTU to 9000 for the VMware NIC (only for the vSwitch) causing connectivity problems and error messages in the FreeNAS logs looking like this:

WARNING: 172.16.1.200 (iqn.1998-01.com.vmware:host-585e8872): no ping reply (NOP-Out) after 5 seconds; dropping connection

FreeNAS settings

  • In Network interface setup, add “mtu 9000” to options.

VMware settings

  • Networking -> VMkernel NICs -> edit -> set MTU to 9000 -> Save
  • Networking -> Virtual switches -> edit -> set MTU to 9000 -> Save

Network switch ports

  • Make sure all network switch ports all the way between the FreeNAS and VMware host have MTU set to 9000

Testing

On the VMware host, enable SSH and login using SSH. Use the command:

vmkping -s 8972 172.16.1.100 -d

Replace 172.17.1.100 above with the IP-address of your FreeNAS. -s 8972 sets packet size to 8972 bytes allowing 28 bytes for headers and -d means fragmentation should not be allowed.

If everyting works you will get echo replies. If you get the error message “Message too long” it means somewhere on the way between your VMware host and the FreeNAS there is a limit not allowing MTU 9000.

Akeeba restore error – 1118 – Row size too large (>8126) [solution]

When restoring an Akeeba backup of a WordPress site the restore process was interrupted with the error message saying 1118 – Row size too large (>8126).

Solution:

  • Use SSH to log in to your database server
  • Edit /etc/mysql/my.cnf or if you use a config file under /etc/mysql/conf.d, edit that one
  • Under the [mysqld] section add:
    internal_tmp_disk_storage_engine=MyISAM
    innodb_strict_mode = 0
  • Restart mysql:
    service mysql restart

Edit: After you successfully restored the site, remove the lines and restart MySQL. I didn’t do this and later tried to move another site from this server to another using Akeeba. When the site was installed on the new server, I just got an error message saying “Error Establishing a Database Connection”. To resolve it, I had to go back to the above server, remove the lines from mysql config, restart MySQL, make a new backup of the site using Akeeba and successfully restore it on the new server.

Credit: The solution to this problem was found here.

What Supermicro motherboard model does my server have?

When getting the latest IPMI firmware for your Supermicro server, you need to know what model of the motherboard your server has. Often the server lives in a datacenter somewhere and you don’t want to go there, take down the server, pull it out of the rack and investigate it. If you are running Linux on it, you can simply check the dmesg log for “Supermicro”.

grep Supermicro /var/log/dmesg

Supermicro motherboard model

Supermicro motherboard model

APS package installation_task stuck in ISPConfig3

When trying to install an APS package on a ISPConfig3 system, the package was never installed and the status “installation_task” remained for the site I tried to install a package.

Investigating /var/log/ispconfig/cron.log revealed:

PHP Fatal error: Uncaught Error: Call to undefined function curl_init() in /usr/local/ispconfig/server/lib/classes/aps_installer.inc.php:627

This problem can occur when curl is not installed on the target server, but that was not the problem in this case.

The server had been upgraded to Ubuntu 16.05 which runs PHP 7.0 instead of PHP 5. Curl was installed but the problem was that php7.0-curl was missing. The problem was resolved by:

sudo apt-get install php7.0-curl

The next issue that occured according to the logs was:

PHP Fatal error: Uncaught Error: Class 'ZipArchive' not found in /usr/local/ispconfig/server/lib/classes/aps_installer.inc.php:106

This problem was resolved by:

sudo apt-get install php7.0-zip

After this nothing more had to be done. The ISPConfig cron now successfully installed the APS package automatically.

Ubuntu 14.04, 16.04 and 17.04 Unity, Dash and Launcher missing

If you end up missing the menu etc in Ubuntu 14.04, 16.04 and 17.04, i.e. you only see the desktop but nothing more. You can’t even restart the machine.

It can be resolved like this:

  • Right click on the desktop and select “Start a new terminal”
  • In the terminal window run the following commands;
    sudo apt-get update
    sudo apt-get install --reinstall ubuntu-desktop
    sudo apt-get install --reinstall unity
    sudo shutdown -r now

 

How to convert a certificate PFX file to CRT/KEY using openssl

Your PFX certificate file is protected with a password. It can be converted to CRT and KEY files using SSL:

openssl pkcs12 -in certfile.pfx -nocerts -out keyfile-encrypted.key

When you enter this command you will be asked to type in the pfx file password in order to extract the key. You will be asked to enter a passphrase for the encrypted key. The key will be stored in keyfile-encrypted.key.

The exported keyfile is encrypted but you might need it in unencrypted format. To unencrypt the key, do:

openssl rsa -in keyfile-encrypted.key -out keyfile.key

You will be asked for the passphrase that you entered in the previous step. The unencrypted key will be stored in keyfile.key.

Then it is time to extract the certificate:

openssl pkcs12 -in certfile.pfx -clcerts -nokeys -out certfile.crt

Again, you will need to enter the pfx file password in order to extract the certificate. The certificate will be stored in certfile.crt.

MySQL not sorting swedish characters ÅÄÖ correct [solved]

Problem: When using ORDER BY or GROUP BY, the swedish characters Å,Ä and Ö is sorted as A and O.

Solution: add COLLATE utf8_swedish_ci to your ORDER BY or GROUP BY, for example:

SELECT field1,field2 FROM table1 ORDER BY field1 ASC

should be changed to:

SELECT field1,field2 FROM table1 ORDER BY field1 COLLATE utf8_swedish_ci ASC

Meltdown and Spectre vulnerabilities – what to do?

Meltdown and Spectre are two vulnerabilities present in hardware making it potentially possible for programs to steal information, like passwords etc.

Meltdown affects only Intel processors while Spectre, which is more complex, also partly affects AMD and ARM based processors.

It is not yet known if these vulnerabilities has been exploited by anone. It can affect personal computers, servers, tablets and mobile phones, i.e. more or less any device containing a processor.

More information on: https://spectreattack.com/

What can you do?

  • Check your operating system for updates the upcoming weeks (this is normal good security practice, but make sure you do it frequently)
  • Install and update your virus protection. Even if the antivirus program can’t protect you from the attack it might be able to inform you that your device has got malicious code onboard

You can find security bulletins, security advisorys, faq:s etc for your operating system here: https://meltdownattack.com/#faq-advisory

High load on CPU and disk I/O every hour (Apache, MySQL and mod_pagespeed on Ubuntu)

On one of my Ubuntu servers I noticed a significant peak in CPU load (load average, LA) and disk I/O about every hour. At first, I suspected that MySQL was the cause of this, doing some houeskeeping or garbage cleaning.

However, it turned out it was caused by the Apache module mod_pagespeed. The high load occuered when pagespeed was cleaning out it’s cache.

The solution was to locate the cache on tmpfs instead. This was done by editing the file /etc/apache2/mods-available/pagespeed.conf and change the location of the cache by the line (/run is located on tmpfs which is RAM memory):

ModPagespeedFileCachePath "/run/cache/mod_pagespeed/"

Then restart Apache by:

service apache2 restart

Ispconfig3 php5-fpm Error 500 Internal server error

When trying to change a client website on a Ispconfig3 host from Fast-CGI to PHP-FPM the page just gave Error 500 Internal server error. Investigating the site error log showed lines like:

[Sat Oct 22 10:39:56 2016] [error] [client xx.xx.xx.xx] (2)No such file or directory: FastCGI: failed to connect to server "/var/www/clients/client6/web447/cgi-bin/php5-fcgi-*-80-domain.xx": connect() failed
[Sat Oct 22 10:39:56 2016] [error] [client xx.xx.xx.xx] FastCGI: incomplete headers (0 bytes) received from server "/var/www/clients/client6/web447/cgi-bin/php5-fcgi-*-80-domain.xx"

Checking the status of PHP-FPM by the command:

service php5-fpm status

gave the result “not running”.

Investigating the PHP-log file, /var/log/php5-fpm.log, displayed lines like:

[22-Oct-2016 10:34:53] ERROR: [pool web405] cannot get uid for user 'web405'
[22-Oct-2016 10:34:53] ERROR: FPM initialization failed

However, the site widh id 405 and user web405 did no longer exist on the server and has been left there by Ispconfig3 for unknown reasons.

The solution was to manually remove the file /etc/php5/fpm/pool.d/web405.conf file and then:

service php5-fpm restart
service apache2 restart