Printable View
Just to check if people are aware that there is a limitation of 4TB with replication function on Open-E because they used DRBD( replication software for linux) free version , Open-E tell me that if enough people need more than a 4TB volume they will look to get the paid version of DRBD( DRBD+) so we will have a 16TB capacity for replicated volume on 32bit system. Some people need it?
I would also be interested in such a feature. We are using a Open-E with 20Tb of space.
Approx. 10Tb will have to be replicated.
If you read there help info in the volume manager it clearly states "The Maximal NAS (NAS), iSCSI (I) and Fibre Channel (FC) volume size (with replication) is limited to 4193120MB (megabytes).
If your volume (including volume replication) fails to be created, increase the Vmalloc size. This can be done via the Hardware Console Tools (ALT+CTRL+W) -> Tuning options -> Vmalloc size."
Just RTFM.....
Sorry but dont know why people do not read it.... it was created for a reason....
This is good to know that others need it as well the more we know that need this will help all. But just to let you know we never stated that we would not do this it is in our plans just not a confirmed date. Keven has the right idea to find out if there are others needing volumes replicating over 4TB. The more we know the better it helps us push the development date.
How much memory Open-e has setup on vmalloc in the kernel? I read on the DRBD mailing list that alot of people have hard time to get volume over 2TB working with default 128M or 192M of vmalloc. Someone have try a full synchro of 4TB?
Linbit ( DRBD programmer ) say that you need 32MB of vmalloc memory for 1TB of storage so the minimum is 128MB , how much open-e kernel is using?
We have a customer that is replicating 4x2.5TB volumes. We would have had just one 10 TB volume (or two 5 terabyte volumes), but the 4TB limitation didn't let us. It wasn't a big deal at all, and we might've needed to split it into 4 volumes anyway, but we would've probably tried it.
Actually, I've had a little problem with the replication stopping randomly every month with that system. I will try increasing the vmalloc!
Hi,
a drbd+ option would be fine, not for replication >=4 TB filesystems, but for the 3rd mirror desaster recovery (DR) option, so we would be able to have an additional DSS server on our DR site.
regards
heimic
Same here, support for a 3rd mirror would be _very good_. Ideally, we would want a failover-failover-failover scenario which had 2 DSS in our server room and a 3rd at a DR site.Quote:
Originally Posted by heimic
I think its the time for a more "enterprise" version of Open-E , DSS Standard for very small business and a DSS Premium ( with DRBD+ and more control of things inside Open-E ) for bigger one with some sort of priority support contract.:cool:
The engineers will have this capability to replicate beyond 4TB (no set time frame for which release - but they are working on it) they will need additional time as we recognize the need for this.
at the same time if they could use the fonctionnality of DRBD+ 3rd site asynchronious replication this would be a must!
Do you know wich protocol of DRBD ( A,B,C) Open-E is using for replication processus?
Thanks
Keven:)
We use protocol C - engineers found it to be faster than others.
Our kernel uses 10-20 MB of the Vmalloc memory. The replication Vmalloc usage is about Volume Size/32768.
Wrong DRBD need 32MB of vmalloc by 1TB of storage, the formula is storage_in_GB/32 and not by 32768 so 4096GB/32 = 128 so I don't know if some people have a rock solid replication volume over 1TB but from the maker of DRBD 10 to 20MB is clearly not correct.
Read this post about memory allocation and 32MB by 1TB , the writer is the PROGRAMMER of DRBD Dipl-Ing Philipp Reisner so I don't know if your enginnier are reading this forum or even DRBD doc and mailing list but I think they should do , 32K by 1GB of storage so 1TB=1000GBx32Kb=32000Kb=32MBx4(for 4096GB or 4TB)=128MB,now I know why people complaining about replication stop working.
Just for Information Protocol C is probably the slowest one but the most secure for your data , so this is good , a control of protocol could be an interesting option , like protocol A for asynchronious remote site.
http://www.drbd.org/users-guide/s-re...protocols.html
Hi...
Kevin - thanks for the nice info... but can you explain this:
I am replicating 2.8TB and getting an average steady speed of 62MB/s.
if what you are saying is right, then what I am getting is impossible...
Any one else can confirm this???
Here let me help Keven out as he wanted to place his trophy on our response (yes should have been better but without the blood).
Here is a better description and using Protocol C is better in the end. If you don’t like it build it yourself :D .
For each device, drbd will (try to) allocate X MB of bitmap, plus some constant amount (<1MB). X = storage_size_in_GB/32, so 1 TB storage -> 32 MB bitmap.
By default Linux allocates 128MB to Vmalloc. For systems using more than 4TB, this may cause an issue - but give us time as we resolve this!!! It will happen, we are not going to just update without hard testing with our main customers unlike what some want it now.
And yes asynchronous is in the works - very soon :)
We are replicating with 2 x 4TB volumes and haven’t had any issues for over 68 days, though we are using DSS ver. 5.0_42 in block io mode.
Not sure if this helps others but on a side note we would have more interest in what Heimic and SeanLeyne stated then the 16TB replication option (DRBD+). Vkeven must be needing something other then what DSS provides currently or wanting blood like Todd said :D
Todd - any chance that we will see asynchronous soon (not asynchronious :D )?
Long time no hear from U!
We want to say mid part of Q2 - though if there are issues or pending projects (especially from you guys for those huge orders :) ) then all this delays it.
Man you are good...
Now i did some research and found that protocol A asynchronous is a completion of request happens as soon as it is completed locally and handed over to the local network stack. In case of a Primary node crash and fail-over to the other node, all WRITEs that have not yet reached the other node are lost.
hopefully this is helpful
Again thanks Todd for these quick responses...
also some must go to vkeven for starting this thread.
How much people has understand the way Open-E replication work before I post this thread?
In just one post I get interest of real power users of Open-E ( the one that want more than 4TB ) , but Open-E is not happy when I'm posting something that could be true , even if I try to contact support I reach a dead end , forum are suppose te be use for discussion,idea,limitation but as long as Open-E don't get hurt. Maybe we should start a user forums of Open-E?
Again we need time to have greater then 4TB replicated volumes, we will have it and if support replied with the same response it is because we need additional time. Right now our focus is on Auto Failover and other projects. Also there are many working with replication beyond 1TB. Having the option of using other DRBD Protocols is a good idea to select from but this will be determined by the engineers and management.
If you are having issues with your replication of volumes please send the logs to support@open-e.com
Now this thread is become active ... yaaaa :)
Keven thanks for this interesting thread and Thanks Todd for the information you provide...
Keven do not blame open-e for not offering a replication for >4TB, frankly how much you are paying for open-e software and how much you will pay for other products. open-e gives me many many features that I need for a very decent price, and every time they improve their product, right now it can compete with very big guys.
Its not just price, its a very nice and stable product that offer a very good solution with a very good support. I do not ask for more..
Todd-- do you give me a discount for this nice talk.. :D :cool:
How about we give you the update when the version of DSS to replicate greater then 4TB comes out :) - man you guys are blood thirsty!
I never say that Open-E is a bad product , its really a good product to be honest, but the fact that its price is low could be one of the problem , this is why I suggest to have a more business oriented version ( like a DSS premium ) with an higher price tag and some form of support contract. In reality Open-E is linux with LVM,XFS,DRBD,ISCSI,Samba,Netatalk,bacula... with a GREAT! Web-UI but we don't have access to command line so if something goes wrong we are stuck to wait for support to answer your question (and you can't buy priority support). I have no problem to paid 5 times the price for Open-E + a support contract but I can't.
Again we need more time to build this up and charge for support for what you are asking(though many will not like!!). We try to please all but it is just too hard to do and no company that I know of can.
Again please give us the time we need to build this...
Kevin,
I may agree with you in that its good to have super open-e edition that cost 5 times more. But come in, in reality can that be real in few months or weeks period. This need time, and that what they are asking.
Give them time and lets see.... if they fulfill there promise that will be great, as we all agree thats its nice product. If not, then too bad, if you do not like it then try something else.
Lets close this, because its taking loooooooooing time to go over all these posts. we need to get knowledge from this and by making this so long there will be no benefit.
Where I put pressure about delay? I just want to know about interest of other people here so if there is enough interest maybe Open-E will put this request upper on the priority list. I also talked about vmalloc and protocol of replication so if they need to touch this part of Open-E maybe they will take care of this request also.
People that don't want to buy priority support can just use Open-E like today with support@open-e.com but people that what to pay for priority support should have an option to do it , the best and fastest way could be trough current partner but some access to command line will be a necessity , a command line access with some advanced administrator user guide could be a solution.Quote:
Originally Posted by To-M
This has been placed on our list and so we will do this, only that we need more time. Again we need more time as there are pre existing orders pending that pay for developers and many other items that is common in the industry. Try asking the bigger companies to lower their prices or add feature in the next release it will not happen.
One more time, we need additional time, we recognize this and I think we understand the need for what you require. I will say this as well, if you send to support@open-e.com your request is also recognized. We save the requests and review many times a year and this has been sent by many already.
Please no more post on this subject as I think we have exhausted to its best.
So after 1 year , where we are with this?
The engineers have this slated for Aug release for the new DSS product.