FreeNAS, VMware and iSCSI with MTU 9000

In order to improve perfomance there are recommendations to set the MTU for the iSCSI interface to MTU 9000 instead of the default 1500.

To do this, it important that you set MTU to 9000 on all devices involved; FreeNAS network interface, VMware NICs and vSwitch and all switch ports on the switches connecting those.

Note: When changing these settings, it will cause loss of connectivity so it might be a good idea not to do it on systems in production.

At one point I had forgotten to set the MTU to 9000 for the VMware NIC (only for the vSwitch) causing connectivity problems and error messages in the FreeNAS logs looking like this:

WARNING: 172.16.1.200 (iqn.1998-01.com.vmware:host-585e8872): no ping reply (NOP-Out) after 5 seconds; dropping connection

FreeNAS settings

• In Network interface setup, add “mtu 9000” to options.

VMware settings

• Networking -> VMkernel NICs -> edit -> set MTU to 9000 -> Save
• Networking -> Virtual switches -> edit -> set MTU to 9000 -> Save

Network switch ports

• Make sure all network switch ports all the way between the FreeNAS and VMware host have MTU set to 9000

Testing

On the VMware host, enable SSH and login using SSH. Use the command:

vmkping -s 8972 172.16.1.100 -d

Replace 172.17.1.100 above with the IP-address of your FreeNAS. -s 8972 sets packet size to 8972 bytes allowing 28 bytes for headers and -d means fragmentation should not be allowed.

If everyting works you will get echo replies. If you get the error message “Message too long” it means somewhere on the way between your VMware host and the FreeNAS there is a limit not allowing MTU 9000.