How to initially fill share using rsync

[jmo]

Hi!
[...]
Set up "Data replication agent settings" for one of the shares.
Now I want to fill that share with some 100 GB of initial data.
I found that I may use rsync for that purpose.

But I fail:

- when using ssh, the connection is refused at port 22 -- no sshd on Open-e !?!

- when using rsh, after 10-15 seconds, I get this error:
[...]
Any hint how to correct that?

Robert

Robert, have you tried using rsync protocol instead of ssh or rsh? DSS provides an rsync daemon, which the client can connect to directly.

With regards,
Jens

[the_nipper]

Hi Jens,

yes, that's what I asked in my first posting: I used rsync, but I failed. Please look at my first posting where I described the commands that I used and the error messages that I got.

You or anyone else succeeded in using rsync? Then, please, tell me the client command line and the settings for Open-e...

Best,

Robert

[jmo]

Hi Jens,

yes, that's what I asked in my first posting: I used rsync, but I failed. Please look at my first posting where I described the commands that I used and the error messages that I got.

You or anyone else succeeded in using rsync? Then, please, tell me the client command line and the settings for Open-e...

Best,

Robert

Robert,

I'm sorry I wasn't specific enough: rsync knows two general modes of operation:
- using a remote shell to invoke rsync on the remote host and communicate via stdin+out
- using rsyncd on the remote host

DSS is providing the latter (rsyncd), your invocations were for the former (tying both ssh and rsh as the remote shell command).

Yes, I've been using rsync to both populate and to retrieve files via rsync to/from DSS.

With regards,
Jens

[the_nipper]

Again, Jens:

Robert, have you tried using rsync protocol instead of ssh or rsh? DSS provides an rsync daemon, which the client can connect to directly.

Yes, I used rsync. But rsync is a protocol only. To use it, you need a connection. This connection may be done using rsh or ssh, e.g. depends on what the remote side allows.

I tried both ssh and rsh, without success.

Best,

Robert

[jmo]

Again, Jens:

Yes, I used rsync. But rsync is a protocol only. To use it, you need a connection. This connection may be done using rsh or ssh, e.g. depends on what the remote side allows.

I tried both ssh and rsh, without success.

Best,

Robert

Robert,

I'm sorry to disagree with you limiting the connections to rsh and ssh - the following is an excerpt from the rsync command's man page:
--- cut here ---
GENERAL
[...]
There are two different ways for rsync to contact a remote system: using a remote-shell program as the transport (such as ssh or rsh) or contacting an rsync daemon directly via TCP. The remote-shell transport is used when‐ever the source or destination path contains a single colon ( separator after a host specification. Contacting an rsync daemon directly happens when the source or destination path contains a double colon (: separator after a host specification, OR when an rsync:// URL is specified (see also the "USING RSYNC-DAEMON FEATURES VIA A REMOTE-SHELL CONNECTION" section for an exception to this latter rule).
--- cut here ---

DSS only provides the latter - directly connecting to rsyncd via TCP. Neither rsh nor ssh are available on DSS.

With regards,
Jens

[To-L]

Hi Nipper

Try this as it has worked for other users:

You can use our NAS-R3 as a destination server for the rsync process. Enable the Data Replication Agent in WebGUI -> configuration -> NAS settings and also for share(s) that you want to have as destination in NAS Resource. Then set username and password if desired.

Rsync command used to synchronize from your operating system to Open-E server can look similar to this:

cd $SOURCE_DIR
export RSYNC_PASSWORD=${PASSWORD}
rsync -raRx --acls --xattrs --whole-file --delete --force "./"
"$LOGIN@$RSYNC_DEST_IP::$DEST_SHARE"

While using rsync without password please unset RSYNC_PASSWORD variable.

Recomend using a separate NIC for the rsync transfer.

You will need to export your UID and GID from your system then import them to the NAS-R3

For example in UID case - go to NAS resources -> Users -> Function: UID mappings (upload/download).

To upload UID:

* Find configuration file uid_mappings.csv (format:user_name;uid) by clicking "Browse" buuton. This file should be in UTF-8 encoding,
* Press "upload" button to import uid mappings,
* If there will be some errors while importing uids please read uid_mappings_import.log file logs package.

Also you have entered in your question to support please let them know tha

[the_nipper]

Hi To-L,

You will need to export your UID and GID from your system then import them to the NAS-R3

For example in UID case - go to NAS resources -> Users -> Function: UID mappings (upload/download).

I have to do this upload from my system as user root, e.g. UID/GID 0. I guess there's no mapping needed. I have to do this because the data are user homes which I only can access as the superuser.

All user UID/GID are well known to Open-e since it uses the same NIS service as my server.


Question: under these circumstances, is there really need of UID/GID mapping?


Robert/Nipper

[Sh-J]

Dear Robert,

Its good to map them, so your replicated volume can use/access the same groups and users.

How its going now?

Regards,
SJ

[the_nipper]

Hi!

Can't agree. Open-e is configured to use NIS. We have several hundreds of users, and we have daily changes on this database --really!!! (research environment close to a university)

Thus, Open-e MUST be able to work with NIS. It isn't a solution for us to keep out (actually) two Open-e systems manually consistent to NIS.

Best,

Robert