-
Tuning iSCSI settings
Hi All,
Reading the forums and the like this seems to be the "best fit" settings for iSCSI tuning.
headerDigest=None
DataDigest=None
maxoutstandingr2t=8
FirstBurstLength=65536
MaxBurstLength=16776192
maxRecvDataSegmentLen=262144
InitialR2T=No
ImmediateData=Yes
Maxxmitdatasegment=262144
Wthreads=8
When comparing these settings to my ESXi 4.1 iSCSI advanced options I've listed the ones below that can be changed/already set (which I have updated yet)
headerDigest = Prohibited
DataDigest = Prohibited
MaxOutstandingR2T = 1
FirstBurstLength = 262144
MaxBurstLength = 262144
maxRecvDataSegLen = 131072
InitialR2T = unchecked
ImmediateData = checked
My first question is on ESXi 4.1 where do I check/set these 2 settings
Maxxmitdatasegment=262144
Wthreads=8
The second part is what happens if they don't match? will iSCSI just not work? or can it negotiate the settings?
I would like to make the tuning changes to my secondary DSS node, then make the changes to ESXi hosts, (hopefully with out knocking out the iSCSI side of things), manually failover to the secondary node, make the tuning changes to the primary, sync and fail back.
or do I have to bring everything down and change both with a completely offline system?
Cheers
Adam
-
You can find the settings, if you choose a host, and check its hardware configuration(network)
Your best option is to do this all at once on both nodes, while the hosts are down.
Having different frame sizes between nodes, can cause issue with some hosts.
If the iniators do not match, they will adjust, but to the lower settings by default.
Hope this helps.
//G
-
I have looked over the ESXi configuration the only settings NOT listed are these 2
Maxxmitdatasegment=262144
Wthreads=8
Does anyone know where/how to set them in ESXi 4.1?
Cheers
Adam
-
For Wthreads, the iSCSI target employs several threads to perform the actual block I/O to the device.
Depending on your hardware and your (expected) workload, the number of these threads may be carefully adjusted.
The default value of 8 should be sufficient for most purposes.
And this can be only on iscsi target
MaxXmitDataSegmentLength - Sets the maximum data segment length that can be sent.
This value actually used is the minimum of MaxXmitDataSegmentLength and the MaxRecvDataSegmentLength announced by the initiator.
It should be set to multiples of PAGE_SIZE.
Currently the maximum supported value is 64 * PAGE_SIZE, e.g. 262144 if PAGE_SIZE is 4kB.
Configuring too large values may lead to problems allocating sufficient memory, which in turn may lead to SCSI commands timing out at the initiator host.
As you can notice, that changing Wthreads is done on ESX initiator not target, also the same for MaxXmitDataSegmentLength . So the best is to leave them as default or to ask ESX support if they suggest any other value that is better for you.
-
Thanks Al,
That is awesome background and great response :) Just what I was after.
Just an update, I in fact did failover to my secondary node, update the iSCSI settings on my primary node, re synced and failed back to the primary.
Then put each of my VM host machines into maintenance mode (in a DR config meant they migrated all running VM's to my other hosts). Adjusted the iSCSI settings in ESXi and brought the host back online, repeated this for all my hosts.
It has made a VERY noticeable difference to my whole VM setup. "by the seat of your pants feel" things are remarkable more snappy in there response inside the VM guests.
Now the last step for me is the tweak on drbd, as I have BBU controllers on each node (disable barrier checks etc). But I think this one will have to be a complete outage to do.
Cheers and thanks all for you advice.
Adam.
-
