Question about Best Practice Active-Active

Hello,

at the moment I setup my first V7 Active/Active iSCSI cluster.
Here is how I did it... What do you think? Is this best practice? Any errors?


NODE1
2 iSCSI Targets, each with 2 Logical Volumes
Target 1: vmfs01 (2x 600 GB Volume)
Target 2: vmfs02 (2x 2000 GB Volume)

NODE2
2 iSCSI Targets, each with 1 Logical Volume
Target 3: server01 (1x 2500 GB Volume)
Target 4: server02 (1x 1500 GB Volume)

Each with 4 Virtual IP addresses and 3x balance-rr bond for the Storage Replication.


I connect iSCSI Target 1 and 2 to my ESXi Servers and I connect Target 3 and Target 4 to physical Windows 2008 Servers.


Why this way ????

VMware: We only setup 600 GB VMFS Datastores but in this location we have one big fileserver. We don't need much performance, that's the reason why a VM is ok.
iSCSI Target 1 with the 2x 600 GB is the regular VMFS datastore for all Virtual Machines.
iSCSI Target 2 with the 2x 2000 GB is VMFS datastore, too - contain only the data disk of the virtual fileserver.
Benefit: If I have to add more diskspace to the fileserver I can remove iSCSI Target 2 from the cluster without the need to shutdown all Virtual Machines.

Windows Servers: Each server have his own iSCSI Target with only one LUN. So Server 1 never can access the disk of Server 2 by mistake and vice versa.
Second Benefit: If I have to add more diskspace to Server 1 I can keep Server 2 running.



Everythink ok?
Any comments?
Best Practice?



Thanks. :-)

Bye Manuel

I would look to use MPIO for the VIP NICs and not use Bonding. MPIO will give you multiple streams for the I/O's where bonding will give you only 1 stream and you still get path failover with MPIO.


You can directly connect the single designated NIC for the Volume Replication to save on the switch port.

For the VMware I would set the iSCSI Target values that I have listed below and also set in VMware's initiator settings as well. Do this before connecting your VM's. For Windows the defaults
are ok.

1. From the console, press CTRL+ALT+W,
2. Select Tuning options -> iSCSI daemon options -> Target options,
3. Select the target in question,


maxRecvDataSegmentLen=262144
MaxBurstLength= 1048576
Maxxmitdatasegment=262144
FirstBurstLength=65536
DataDigest=None
maxoutstandingr2t=8
InitialR2T=No
ImmediateData=Yes
headerDigest=None
(deprecated)Wthreads=8

nova17.de,

true storage fail over is best achieved with VMWare Storage DRS, the Open-E servers are just big boxes.

the ideal method, is to create identical iscsi luns on each server, so both "nodes" have identical volumes & replicated tasks for active/active.

Hello Todd,
I use MPIO for the VIP NICs - I only use a balance-rr Bond for the Storage Replication NICs:

4 NICs with VIPs presenting iSCSI -- VMware/Win2008 connected with MPIO
3 NICs as balance-rr bond for Storage Replication -- all cables connected directly 1:1 without a Switch inbetween.


About your iSCSI Target Options: At the moment we're running ESX4 but we're planning the upgrade to V5.x. Are these settings optimized for VMware in general?



Hello Alex,
yes I use the Open-E only as a big box, same iSCSI configuration on each side.



Thank you.
Bye, Manuel

I would still set these settings for the ESX 4.x as well.

Hi Todd,

I changed the values. Can you explain in in 2-3 short words what these settings do?
Or how much this will improve the performance?

Thank you :)
Manuel

These are the recommended VMware iSCSI target settings, the performance increase is very difficult to give an exact number but if you test with the defaults to these settings you should see an increase.

...not to open new thread:

We are planning to combine two separate DSS7 iSCSI targets:
- one for VMWare ESXi initiators with parameters (recomended above)
maxRecvDataSegmentLen=262144
MaxBurstLength= 1048576
Maxxmitdatasegment=262144
FirstBurstLength=65536
DataDigest=None
maxoutstandingr2t=8
InitialR2T=No
ImmediateData=Yes
headerDigest=None
(deprecated)Wthreads=8

- and another one for native Win initiators (cluster of 2x Server2008R2)

Is it better to use default parameters on Win and adjust particular target on DSS7?
DelayBetweenReconnect=5
EnableNOPOut=0
ErrorRecoveryLevel=2
FirstBurstLength=#00010000=65536
ImmediateData=1
InitialR2T=0
IPSecConfigTimeout=#0000003c=60
LinkDownTime=#0000000f=15
MaxBurstLength=#00040000=262144
MaxConnectionRetries=#ffffffff
MaxPendingRequests=#000000ff
MaxRecvDataSegmentLength=#00010000=65536
MaxRequestHoldTime=#0000003c=60
MaxTransferLength=#00040000=262144
NetworkReadyRetryCount=#0000000a=10
PortalRetryCount=#00000005=5
SrbTimeoutDelta=#0000000f=15
TCPConnectTime=#0000000f=15
TCPDisconnectTime=#0000000f=15
WMIRequestTimeout=#0000001e=30

or adjust "VMWare recomend" parameters to Windows iSCSI initiator?

Thank you for the answer!
Toni

If you change target values, you will also need to adjust windows initiator.

Why not create multiple iscsi targets, then the targets for windows initiator set them according the windows settings and the targets for vmware set them to match vmware. The settings in open-e are per target and not global I believe

This is the point of my question:

Is it recomended to adjust target settings to initiator defaults?

Default iSCSI settings are different between VMWare, Server2008 and DSS7
(can't find recomended maxoutstandingr2t=? for Win).
Can you recomend propriate settings on DSS console for WinServer2008R2, for example?

Is performance better, if all targets&initiators use equally settings?

Thank you for the answer!
Toni

I would say adjust the targets to match the initiator settings, open-e's target setting s are standard generic that will work by default, but the other vendors have set their initiators for best performance for there OS. Also remember that on VMware if using round robin to set the IOPS setting to 1 for the volumes.

DelayBetweenReconnect=5
EnableNOPOut=0
ErrorRecoveryLevel=2
FirstBurstLength=#00010000=65536
ImmediateData=1
InitialR2T=0
IPSecConfigTimeout=#0000003c=60
LinkDownTime=#0000000f=15
MaxBurstLength=#00040000=262144
MaxConnectionRetries=#ffffffff
MaxPendingRequests=#000000ff=255 - maxoutstandingr2t?
MaxRecvDataSegmentLength=#00010000=65536
MaxRequestHoldTime=#0000003c=60
MaxTransferLength=#00040000=262144 - Maxxmitdatasegment?
NetworkReadyRetryCount=#0000000a=10
PortalRetryCount=#00000005=5
SrbTimeoutDelta=#0000000f=15
TCPConnectTime=#0000000f=15
TCPDisconnectTime=#0000000f=15
WMIRequestTimeout=#0000001e=30

Can I also tune MPIO round robin IOPS setting in Server2008?

I am not sure about windows server settings