Posts

Stefan Helander

FreeNAS, VMware and iSCSI with MTU 9000

/0 Comments/in , /by

In order to improve perfomance there are recommendations to set the MTU for the iSCSI interface to MTU 9000 instead of the default 1500.

To do this, it important that you set MTU to 9000 on all devices involved; FreeNAS network interface, VMware NICs and vSwitch and all switch ports on the switches connecting those.

Note: When changing these settings, it will cause loss of connectivity so it might be a good idea not to do it on systems in production.

At one point I had forgotten to set the MTU to 9000 for the VMware NIC (only for the vSwitch) causing connectivity problems and error messages in the FreeNAS logs looking like this:

WARNING: 172.16.1.200 (iqn.1998-01.com.vmware:host-585e8872): no ping reply (NOP-Out) after 5 seconds; dropping connection

FreeNAS settings

  • In Network interface setup, add “mtu 9000” to options.

VMware settings

  • Networking -> VMkernel NICs -> edit -> set MTU to 9000 -> Save
  • Networking -> Virtual switches -> edit -> set MTU to 9000 -> Save

Network switch ports

  • Make sure all network switch ports all the way between the FreeNAS and VMware host have MTU set to 9000

Testing

On the VMware host, enable SSH and login using SSH. Use the command:

vmkping -s 8972 172.16.1.100 -d

Replace 172.17.1.100 above with the IP-address of your FreeNAS. -s 8972 sets packet size to 8972 bytes allowing 28 bytes for headers and -d means fragmentation should not be allowed.

If everyting works you will get echo replies. If you get the error message “Message too long” it means somewhere on the way between your VMware host and the FreeNAS there is a limit not allowing MTU 9000.

Stefan Helander

VMware ESXi 5.5 guest freezes randomly [solved]

/0 Comments/in /by

On a VMware 5.5 ESXi host I suddenly experienced guest machined that hanged or froze. I had to reset the guest machine to get it up and running again. In the Events tab I could see entries like below at the time of freezing.

Lost access to volume 51d15c1a-17706da4-0ae6-001f29e5c998 (datastore1) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly.
info
2018-10-11 04:03:50
datastore1

Device mpx.vmhba1:C0:T0:L0 performance has deteriorated. I/O latency increased from average value of 69864 microseconds to 4383213 microseconds.
warning
2018-10-11 04:03:50
localhost

I started to check my disks and raid as I suspected a failing disk, but the disks and raid was in working order.

It turned out that the problem started around the time I did a guest snapshot. The datastore had about 14 GB free space out of it’s 924 GB capacity and this seems to be to0 little. After deleting a couple of snapshots the problem disappeared.

When the snapshots had been deleted, I also had to consolidate some of the virtual machines disks.

Edit: A couple of weeks later one of the disks in the raid actually failed without any warning that it was about to fail. After replacing the disk the problems were less frequent but sometimes still occured, and I discovered it was during backups, i.e. with a lot of traffic on the disks.

Moving away one of the most busy clients to another VMware host, the problems disappeared. The disks were SATA disks because this was not really a production system but more of a test and development environment where speed was not so critical. The cause was probably just overloading the disks with more requests than they could handle in combination with a disk about to fail.

Stefan Helander

VMware vSphere client 4.0 crashes when trying to upload a file to datastore

/0 Comments/in , /by

 Suddenly my VMware vSphere client version 4.0.0 started to crash when I try to upload a file to the datastore. This happens on both my Windows Vista and XP boxes running the VMware vSphere client.

The problem is similar to the one described in this KB article concering Windows 7 clients. However, the solution is the same. The VMware vSphere client does not crash if it is started by right clicking and runing it as administrator.