Infiniband...

[r3vo]

Yeah, I'd like to see SRP support as well, but until it gets into SCST, we won't see it in Open-E.
OpenE supports IPoIB so yes, you can push replication traffic over it. It *should* be treated just like another eth[*] int.

just my two cents, cause you said should be treated...
indeed, I tested 2 Dssv6 with Infiniband and the interfaces are treated like another eth[x].
I tried also to replicate a volume between the two boxes which was no problem at all.
I couldn't make any detailed performancetests due to time reasons, and I only got 4hdds (sas) in a raid5, but I know that the speed of replication was very fast.
Unfortunately I only got 2 infiniband cards (without switch), so I wasn't able to perform a failover scenario.

[chris74]

Thanks for the replies. As I suspected, there doesnt appear to be anyone running IB with Open-E - mainly because its just too slow using IPoIB and iSCSI. I'm guessing they dont support SRP because it isnt a recognised standard. Its a shame because the benefit of using RDMA over IB coupled with the low cost of the hardware makes for very exciting possibilities when building a SAN on a budget.

IB is faster and much cheaper than 10GbE. All the drivers and software required to run IB/SRP initiators and targets on Linux are fully available. Its just a shame that no-one has yet bundled it all into a software package such as Open-E. When someone finally does this, I'm sure they will make a lot of money.

We are committed to using IB in this cloud and building all the hardware ourselves - so its just a case of spending the time looking at the OFED and SCST target drivers and going through an extended period of configuration and testing to get things just right.

[enealDC]

Open-E has moved to SCST and the status of the SRP target for infiniband is not clear for that project.
If you google, "ofed SRP target" you get a link to a howto document, but I'm not sure what kind of mileage you'll get.
Someone on this thread has already commented on IPoIB for replication and says it seems fast. As I mentioned earlier in the thread, they should be treated like regular eth* interfaces.

[enealDC]

Actually, I mispoke, It's not the SRP target that's uncertain it's the iSER target. Sorry for the confusion.

[chris74]

As far as I can tell, SRP works perfectly fine with the SCST drivers.

http://scst.sourceforge.net/target_srp.html

You said Open-E has moved to SCST - can you clarify this?

Are you suggesting that Open-E may indeed support SRP targets in the short term?

[enealDC]

Sorry for the confusion. I was thinking about the iSER target...

I can't speak directly for open-e so it's not really possible for me to answer if they will or won't do anything. HOWEVER... since the iSCSI target since v6 has been SCST based, the product can support an SRP target. I'm pushing for SAS myself..