Synch Vol Replication - is buffered? with Slow Dest?

Can someone please explain to me if/or how synchronous volume replication is buffered in reposnse to (a) a replication link failure or congestion, and (b) destination system performance? I have read some posts about this, but I am still not 100% clear.
(buffering could take place in the source or destination)

On one hand, it would seem that a synchronous replication "step" has been actioned when the destination has either received the packet or comitted it to disk (depending on the daemon's configuration). If that cannot be achieved (failed link or slow destination server), then the source with either (a) accept no more IO write traffic, or (b) start buffering changes. When the link has been restored, then any changes buffered would be transmitted.

I am sure other SANs we have read about using fully synchronous replication say all writes to the source are stopped until the destination is online again. But this Open-E document (http://www.kb.open-e.com/entry/15/) suggests that writes to the source are not stopped, and all changes are correct handled and not lost (via ECC data). This means it is almost acting asynchronously then?

Can we please consider 2 senarios involving a fast DSS source system, plus...

1. a WAN link that can handle the replication traffic on average ok, but not the short bursts the source system experiences (like many real-world system do)
2. a fast LAN link but to a slower destination server (eg. with a slower disk sub-system, or heavily loaded), and as above senario, can keep up on average, but not the peak writes.

So my question is, how does DSS's synchronous volume replication handle these 2 cases.

After understanding this, my next question will be how these could (or not!) work with iSCSI auto-failover. Cheers.

That knowledge-base article is referring to the fact that if the replication link is actually physically severed, then you will still be able to transfer data to the source system, but all the data written to the source will be resynced once the connection is re-established. If the link is severed, the cluster is in a sort of degraded mode. This won't happen really with merely a slow replication link.

By default, Open-E uses synchronous replication, so your write speed is limited by the speed of your destination server and the link to your destination server. This is a good idea for databases where every single transaction counts.

There is an option to enable an asynchronous (i.e. buffering) protocol that will allow replication over slower (i.e. WAN) links while keeping both the source and destination in a "consistent" (in this case, meaning non-corrupt) state but not necessarily in an up-to-date state. In this case, if your primary system blows up, it's possible to lose the last minute or so of data, but besides that you're good.

So, for the case of a slow WAN link or a slow destination system, Open-E has the option to use buffering.

In the case where you have a slow destination disk subsystem (say, 15k sas in source and 7.2k sata in destination), if I were you, I would just put a RAID card with lots (like 4GB) of buffer space in the destination side. That would let you handle a spike in the write load while still being able to use the synchronous replication mode.

Iscsi autofailover I don't think works with asynchronous replication.

You are correct it does not.

Thanks for your comments, sorry for more questions!

Quote:

---

... then you will still be able to transfer data to the source system, but all the data written to the source will be resynced once the connection is re-established. If the link is severed, the cluster is in a sort of degraded mode.

---

Was this doc refering to the asynchronous or the synchronous case?

But if we are talking about the synchronous case, I dont see where the limitation is for the senario I am considering, and it implies we could use iSCSI auto failover in a situation where the data may not be fully up to date (or perhaps a failover would have to wait until the sync status was "ok").

Quote:

---

This won't happen really with merely a slow replication link.

---

Sorry but I dont quite understand why this is the case. As WAN links gets faster, WANs and LANs start to merge, and I can easily see the situation where people wish to use synchronous replication over links that may not be able to cope with intense writes (eg. 100's of MB/s), but can happily cope with average replication traffic (eg. only a few MB/s). In this situation, and if the model supports it, the source & destination must get a little out of step, which could be a good thing (flexible).

The case for asynchronous replication (say) over a slow WAN link, on a daily basis is simple and clear, eg. used like an off-site backup, where day old data is fine.
But when we consider an almost continuous replication over medium speed link, it's not so clear cut, as now we are concerned with the bandwidth of the link - maybe fast enough to keep up most of the time, but not all.

Quote:

---

There is an option to enable an asynchronous (i.e. buffering) protocol that will allow replication over slower (i.e. WAN) links while keeping both the source and destination in a "consistent" (in this case, meaning non-corrupt) state but not necessarily in an up-to-date state.

---

I think need to do a little more technical reading on how this systems works, as the term "consistent" means different things when going between (say) a file on a NTFS formatted Lun, and the various blocks that make it up. If a single block is changed on the source but not on the destination, then the destination may well be potentally corrupted, as far as user data is concerned, but if the replication process has kept track of all changes, then it is still overall "consistent".
Maybe the asynchronous version coordinates the process much better, maing it more certain at any point in time what is consistent.

Quote:

---

I would just put a RAID card with lots (like 4GB) of buffer space in the destination side.

---

When you say "buffer space", you mean RAID cache RAM? The cards we use have 512MB of cache fixed, so we can't change that. If you mean more RAM in the destination DSS system, that is certainly possible, but AFAIK, very little Ram gets used for block-IO volumes (just buffers). Maybe it's possible to replicate a block-IO source volume to a file-IO destination volume, and thus take advantantage of cheap DDR2 Ram? (eg. use 4GB in source & 16GB in destination).

Quote:

---

Iscsi autofailover I don't think works with asynchronous replication.

---

I do understand that, it's well documented, but again, if synchronous volume replication can accommodate some degree of asynchronisity (not sure if that's a real word!), then there's an in-between option.

The synchronous volume replication goes in this way:

1. At the first connection or reconnection the replication compares ECC tables from source and destination. Those parts that have different checksums are transfered first speed limited by speed set in replication task or by speed of the connection. Until all the changes will be send to destination it is stated as INCONSISTENT. This part works like an asynchronous replication.

2. All write access in the synchronous replication action are:
- buffered by source system,
- send to destination system,
- written to disk by destination system,
- confirmed by destination that they are on disk,
- written by source system to disk.
After all those steps next write access is taken.

When connection is dropped the replication task is stopped and no replication is running. The data are written directly to the source disk and ECC checksums are changing. When replication get the connection again the first step starts again.

This shows that whatever - source or destination server is slow, network connection is slow - the speed will be limited by the slowest part of the replication servers. You need to have fast network and fast raid on both sides to have successful working volume replication in failover.

Please note that the async volume replication don't do the 2 point and the replication works all the time more like the 1 point only. In result that gives no guaranty that when the source server will fail the destination server will have consistent data. This is the reason why failover requires synchronous replication.

Thanks you for that explanation, it makes good sense.