4TB limitation on volume replication

[vkeven]

I think its the time for a more "enterprise" version of Open-E , DSS Standard for very small business and a DSS Premium ( with DRBD+ and more control of things inside Open-E ) for bigger one with some sort of priority support contract.

[To-M]

The engineers will have this capability to replicate beyond 4TB (no set time frame for which release - but they are working on it) they will need additional time as we recognize the need for this.

[vkeven]

at the same time if they could use the fonctionnality of DRBD+ 3rd site asynchronious replication this would be a must!

Do you know wich protocol of DRBD ( A,B,C) Open-E is using for replication processus?

Thanks
Keven

[To-M]

We use protocol C - engineers found it to be faster than others.

Our kernel uses 10-20 MB of the Vmalloc memory. The replication Vmalloc usage is about Volume Size/32768.

[vkeven]

Wrong DRBD need 32MB of vmalloc by 1TB of storage, the formula is storage_in_GB/32 and not by 32768 so 4096GB/32 = 128 so I don't know if some people have a rock solid replication volume over 1TB but from the maker of DRBD 10 to 20MB is clearly not correct.

Read this post about memory allocation and 32MB by 1TB , the writer is the PROGRAMMER of DRBD Dipl-Ing Philipp Reisner so I don't know if your enginnier are reading this forum or even DRBD doc and mailing list but I think they should do , 32K by 1GB of storage so 1TB=1000GBx32Kb=32000Kb=32MBx4(for 4096GB or 4TB)=128MB,now I know why people complaining about replication stop working.

Just for Information Protocol C is probably the slowest one but the most secure for your data , so this is good , a control of protocol could be an interesting option , like protocol A for asynchronious remote site.

http://www.drbd.org/users-guide/s-re...protocols.html

[masim]

Hi...

Kevin - thanks for the nice info... but can you explain this:
I am replicating 2.8TB and getting an average steady speed of 62MB/s.

if what you are saying is right, then what I am getting is impossible...

Any one else can confirm this???

[To-M]

Here let me help Keven out as he wanted to place his trophy on our response (yes should have been better but without the blood).

Here is a better description and using Protocol C is better in the end. If you don’t like it build it yourself .

For each device, drbd will (try to) allocate X MB of bitmap, plus some constant amount (<1MB). X = storage_size_in_GB/32, so 1 TB storage -> 32 MB bitmap.

By default Linux allocates 128MB to Vmalloc. For systems using more than 4TB, this may cause an issue - but give us time as we resolve this!!! It will happen, we are not going to just update without hard testing with our main customers unlike what some want it now.