newbie dsync problems

newer
Dovecot 2.4.0 compilation issues...

older
TLS Connections Between Dovecot &...

Kent Borg

23 Jan 2025 23 Jan '25

8:50 p.m.

I have a very basic postfix/dovecot server, with just a few users, and it works. But in the past I have had hardware failures…so I would like use dsync to a nearly matching server for some extra safety, and I have not yet been successful.

I am trying to do this as root over ssh, with a private key for authentication.

I do not managed to get the whole replicator thing working, so as part of simplifying things I am trying to run dsync manually, just on one user (myself). I hope this isn't a silly approach.

What would the manual command look like? The closest example on the man page is:

...

$ doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com

I try to change that to match my case and it has never heard of me:

...

root@la:/etc/dovecot# doveadm sync -1 -u kentborg@mail.borg.org ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kentborg@mail.borg.org doveadm(kentborg@mail.borg.org): Error: User doesn't exist root@la:/etc/dovecot#

I drop the machine part of the name and it knows who I am on this end but can't get to the other end:

...

root@la:/etc/dovecot# doveadm sync -1 -u kentborg ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kent
borg@mail.borg.org kentborg@mail.borg.org's password:

I think I have root's ssh keys set up correctly, I can run this:

...

root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.

If I try to put that in this (incomplete) dsync command:

...

root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg
.org dsync-server root@la:/etc/dovecot# And the remote end outputs the username as "VERSION dsync 3 5", okay, maybe progress.

But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:

...

root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:

Suggestions?

Thanks,

-kb

Show replies by date

Kent Borg

23 Jan 23 Jan

9:11 p.m.

I had a typo (I said I'm a newbie).

On 1/23/25 10:50 AM, Kent Borg via dovecot wrote:

...

This better version also doesn't work:

...

Sorry for the error,

-kb//

cdm567＠yahoo.com

10:26 p.m.

Kent,

You are being prompted for a password, so it isn't using private key authentication. I recommend you get ssh working first, prove you are indeed connecting to your secondary server, and only then introduce doveadm.

Is mail.borg.org the name of your "matching server" or is that the name of your primary server? If so, it looks to me like you are using ssh to connect back to yourself. You should have something like mail.borg.org as primary, mail2.borg.org as backup, and if necessary, add mail2.borg.org to your hosts file if there is no DNS for it. Or even connect via IP address like this:

ssh root@172.16.20.11

I'll leave the discussion on whether to use root in this fashion even makes sense to others. Suffice to say, once you get something working perhaps consider removing the private key and use a non-root user.

Doug

...

Kent Borg

10:33 p.m.

On 1/23/25 12:26 PM, cdm567@yahoo.com wrote:

...

Kent,

You are being prompted for a password, so it isn't using private key authentication. I recommend you get ssh working first, prove you are indeed connecting to your secondary server, and only then introduce doveadm.

Yes, I checked that:

...

I think I have root's ssh keys set up correctly, I can run this:

...
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.

...

Is mail.borg.org the name of your "matching server" or is that the name of your primary server?

mail.borg.org is the name of the (priority 10) backup, I am running this on my (priority 1) primary server, mail2.borg.org, I am pretty certain I am not ssh-ing to myself.

cdm567＠yahoo.com

11:41 p.m.

In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.

...

Kent Borg

24 Jan 24 Jan

12:21 a.m.

On 1/23/25 1:41 PM, cdm567@yahoo.com wrote:

...

In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.

If I take off the "remoteprefix" it logs in, but it doesn't send the user to the other end, the wrapper script on mail.borg.org gets "VERSION dsync 3 5" as the parameter.

...

root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org dsync-server root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password:

According to the man page, that should be the destination:

...

ARGUMENTS destination This argument specifies the synchronized destination. It can be one of:

              location Same as mail_location setting, e.g. maildir:~/Maildir

              remote:login@host Uses dsync_remote_cmd setting to connect to the remote host (usually via ssh)

              remoteprefix:login@host This is the same as remote, except "user@domain\n" is sent before dsync protocol starts.   This allows imple‐ menting a trusted wrapper script that runs doveadm dsync-server by reading the username from the first line.

              tcp:host[:port] Connects to remote doveadm server via TCP.   The default port is specified by doveadm_port setting.

              tcps:host[:port] This is the same as tcp, but with SSL.

              command [arg1 [, arg2, ...]] Runs a local command that connects its standard input & output to a dsync server.

One of the examples on the man page is:

...

doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com

Which I don't understand. What is "mailuser@example.com"? What are the two parameters and the option after that?

Their simpler example:

...

doveadm sync -u username@example.com remote:server-replica.example.com Makes much more sense, but I can't find anything based on that example works.

Does "doveadm sync" maybe not work in version 2.3.19.1?

...

root@la:/etc/dovecot# dovecot --version 2.3.19.1 (9b53102964)

Thanks,

-kb, the Kent who would love to see some working "doveadm sync" examples.

cdm567＠yahoo.com

12:44 a.m.

I don't have a working example because I do my dsync backups on the local machine with output to shared NFS storage that is accessible to both my primary and backup systems. No ssh required or remote connection required. That is provided by NFS.

This excerpt of my backup script runs dsync in a loop where 'USERS' is populated with email account names to be backed up. The backup target location is on separate storage. If you can't figure out the doveadm sync to another server you could NFS share a file system from your secondary system to your primary and do something similar.

for user in ${USERS}; do dsync -u ${user} backup maildir:/home/${user}/backup/mailboxes done

...

-----Original Message----- From: Kent Borg <kentborg-dovecot@borg.org> Sent: Thursday, January 23, 2025 5:22 PM To: cdm567@yahoo.com; dovecot@dovecot.org Subject: Re: newbie dsync problems

On 1/23/25 1:41 PM, cdm567@yahoo.com wrote:

...
In your working example you are connecting as root but in your dsync example your user is remoteprefix:root. Try removing the "remoteprefix:" which is being treated as part of the user name.

If I take off the "remoteprefix" it logs in, but it doesn't send the user to the other end, the wrapper script on mail.borg.org gets "VERSION dsync 3 5" as the parameter.

...
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1052944><mEMeJ8i9kmcQERAAA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync root@mail.borg.org dsync-server root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix:root@mail.borg.org remoteprefix:root@mail.borg.org's password:

According to the man page, that should be the destination:

...
ARGUMENTS destination This argument specifies the synchronized destination. It can be one of:
          location
                 Same as mail_location setting, e.g. maildir:~/Maildir

          remote:login@host
                 Uses dsync_remote_cmd setting to connect to  the
remote host (usually via ssh)
          remoteprefix:login@host
                 This  is  the  same  as remote, except
"user@domain\n" is sent before dsync protocol starts. This allows imple‐ menting a trusted wrapper script that runs doveadm dsync-server by reading the username from the first line.
          tcp:host[:port]
                 Connects to remote doveadm server via TCP.   The
default port is specified by doveadm_port setting.
          tcps:host[:port]
                 This is the same as tcp, but with SSL.

          command [arg1 [, arg2, ...]]
                 Runs  a  local command that connects its standard
input & output to a dsync server.
One of the examples on the man page is:

...
doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com

Which I don't understand. What is "mailuser@example.com"? What are the two parameters and the option after that?

Their simpler example:

...
doveadm sync -u username@example.com remote:server- replica.example.com Makes much more sense, but I can't find anything based on that example works.

Does "doveadm sync" maybe not work in version 2.3.19.1?

...
root@la:/etc/dovecot# dovecot --version 2.3.19.1 (9b53102964)

Thanks,

-kb, the Kent who would love to see some working "doveadm sync" examples.

Kent Borg

1:28 a.m.

On 1/23/25 2:44 PM, Doug via dovecot wrote:

...

I don't have a working example because I do my dsync backups on the local machine with output to shared NFS storage that is accessible to both my primary and backup systems. No ssh required or remote connection required. That is provided by NFS.

Ah, NFS. But I don't think that works so well in my case, my two servers' homes are a very long way apart.

-kb

Kent Borg

25 Jan 25 Jan

9:32 p.m.

So trying to debug replication via getting manual "doveadmin sync" hasn't gotten me there,

Any suggests on how to debug the replication approach? Would it be helpful to post the "/etc/dovecot/conf.d/backup.conf.ext" I created?

Thanks,

-kb

On 1/23/25 10:50 AM, Kent Borg wrote:

...

I have a very basic postfix/dovecot server, with just a few users, and it works. But in the past I have had hardware failures…so I would like use dsync to a nearly matching server for some extra safety, and I have not yet been successful.

I am trying to do this as root over ssh, with a private key for authentication.

I do not managed to get the whole replicator thing working, so as part of simplifying things I am trying to run dsync manually, just on one user (myself). I hope this isn't a silly approach.

What would the manual command look like? The closest example on the man page is:

...
$ doveadm sync -u username@example.com ssh -i id_dsa.dovecot
mailuser@example.com doveadm dsync-server -u username@example.com

I try to change that to match my case and it has never heard of me:

...
root@la:/etc/dovecot# doveadm sync -1 -u kentborg@mail.borg.org ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kentborg@mail.borg.org doveadm(kentborg@mail.borg.org): Error: User doesn't exist root@la:/etc/dovecot#

I drop the machine part of the name and it knows who I am on this end but can't get to the other end:

...
root@la:/etc/dovecot# doveadm sync -1 -u kentborg ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync kentborg@mail.borg.org doveadm dsync-server -u kent
borg@mail.borg.org kentborg@mail.borg.org's password:

I think I have root's ssh keys set up correctly, I can run this:

...
root@la:/etc/dovecot# ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org PTY allocation request failed on channel 0 C-c C-croot@la:/etc/dovecot# …and on the remote end I see some debugging output I put in the remote script, outputting an empty username. Makes sense.

If I try to put that in this (incomplete) dsync command:

...
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg.org Error: Extraneous arguments found: 3 5 doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: read(remote) failed: EOF (version not received) doveadm(kentborg)<1029674><QvOnDPmKkmcqtg8AA8+vgQ>: Error: Remote command returned error 64: ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync mail.borg
.org dsync-server root@la:/etc/dovecot# And the remote end outputs the username as "VERSION dsync 3 5", okay, maybe progress.

But when I try to make the command more complete and send a username to the remote end, and now I am no longer talking to the remote end:

...
root@la:/etc/dovecot# doveadm sync -u kentborg -1 ssh -i /root/.ssh/id_rsa_rc.borg.org.dsync remoteprefix@mail.borg.org remoteprefix@mail.borg.org's password:

Suggestions?

Thanks,

-kb

Age (days ago)

Last active (days ago)

List overview

8 comments

2 participants

participants (2)

cdm567＠yahoo.com
Kent Borg

newbie dsync problems

tags

participants (2)