DSS iSCSI + ESXi 5 - Jumbo Frames Problem

[MartinB]

I'm trying to setup ESXi 5.0 with iSCSI so we can utilise MPIO, however i'm coming up against a problem. It's when i enable jumbo frames on the network adapters on the DSS.

The DSS has been factory reset and im starting from scratch.
I have setup 2 adapters for the storage on 2 different ip ranges.
I have enabled jumbo frames in the 2 switches that the adapters are connected to.
I have also enabled jumbo frames on the VMkernel adapters on ESXi.

Now this is where it goes wrong.

If i leave the MTU at 1500 for both adapters on the DSS i can scan the HBA and mount / access the iSCSI datastore no problem.
I can reboot the ESXi host and it boots in a timely fashion (with a v small delay on loading the iscsi portion of the kernel).

Now if i set the MTU of the adapters to 9000 on the DSS it takes forever to scan the HBA (about 7mins). It does finally show the target.
When i come to try and add it and format it, it just hangs and hangs detecting current disk layout just saying Loading...
Then errors with the following:

Call "HostDatastoreSystem.QueryVmfsDatastoreCreateOptions" for object "datastoreSystem-2594" on vCenter Server "VMWARE-VCS" failed.

Now if i drop the MTU on the DSS back down to 1500 everything works perfectly!!

This is really annoying me. Any ideas??

[Gr-R]

Check with the manufacturer on the actual size to set for jumbo frames..... some are not exactly 9000, and verify it is even supported on your NICs.
Also, double check the values are consistent across all hardware in the path.

Finally, with MPIO, you may not need to use jumbo frames, as the performance increase can be negligible. But instead tune the targets for performance.

[MartinB]

You set me on the right course. Thank you.

When i was serching for the MTU of my NICs i came across the readme on the Intel site saying that the Jumbo Frames setting on the switch must be set to at least 22 bytes larger than that of the MTU at the NICs.

When i set the MTU on the switch from 9000 to 9216 (which is the maximum permittable setting) it is now working perfectly.

User configuration error by my part. But just thought id share my solution.