February 2019 - dovecot

Unexpected behavior with sieve_vacation_to_header_ignore_envelope = yes
by Matthias Petermann 03 Mar '19

03 Mar '19

Hello, we face an issue with the vacation extension of pigeonhole. Our upstream mail server (from which we pull our E-Mails with POP3) uses SPF and performs a SRS re-write of the Return Path. Once the mail is processed via our local postfix, it is delivered via dovecot deliver (using the lda). When the recipient has a vacation rule enabled in sieve, the out-of-office note cannot be sent, as pigeonhole tries to use the Return-Path (which is not accepted as sender by the mail server). We are using the following dovecot / pigeonhole versions: dovecot-2.3.2.1 dovecot-pigeonhole-0.5.2 As of my understanding, the problem was addressed in pigeonhole version v0.4.21 by introducing the sieve_vacation_to_header_ignore_envelope setting. Anyway - even when this is set to "yes" pigeonhole still uses the Return-Path instead of the "From:" header which I had expected: Feb 27 13:22:26 mail dovecot: lda(customer)<48383><3o+yOQGBdlz/vAAAZU03Dg>: Error: sieve: msgid=<e081d2ba-21d5-c32b-3750-69fe0c8a76d2(a)petermann-it.de>: failed to send vacation response to SRS0=gwGXg6ad=RC=petermann-it.de=matthias(a)example.com: <Sendmail program returned error> (temporary error) Are my expectations wrong or did I miss some precondition? Best regards, Matthias 1) This is the email which I use to test (I replaced the actual domain with example.com) ------>8------------------------------------------------------------------------------ Return-Path: <SRS0=gwGXg6ad=RC=petermann-it.de=matthias(a)example.com> Delivered-To: customer[[(a)example.com]] To: Customer <customer(a)example.com> From: Matthias Petermann <matthias(a)petermann-it.de> Subject: Test Message-ID: <e081d2ba-21d5-c32b-3750-69fe0c8a76d2(a)petermann-it.de> Date: Wed, 27 Feb 2019 05:21:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 Test message ------8<------------------------------------------------------------------------------ 2) The related excerpt of the configuration: ------>8------------------------------------------------------------------------------ mail /usr/local/etc/dovecot/conf.d >doveconf | grep -A 10 -B 10 to_header_ignore_envelope result_failure = continue result_internalfail = continue result_success = return-ok skip = never username_filter = } plugin { acl = vfile acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes sieve = file:/srv/vmail/%u/sieve;active=/srv/vmail/%u/active.sieve sieve_vacation_to_header_ignore_envelope = yes } pop3_client_workarounds = pop3_delete_type = default pop3_deleted_flag = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no ------8<------------------------------------------------------------------------------ -- Matthias Petermann <matthias(a)petermann-it.de>

2 2

dovecot[1511]: auth-worker: Fatal: master: service(auth-worker): child 17009 killed with signal 11
by Patrik 28 Feb '19

28 Feb '19

This bug report is similar to this issue: https://www.dovecot.org/list/dovecot/2017-May/108116.html <https://www.dovecot.org/list/dovecot/2017-May/108116.html> I know it is not enough info, but every kill follows with this info: Feb 28 16:57:50 server dovecot[1511]: auth-worker: Fatal: master: service(auth-worker): child 17009 killed with signal 11 (core dumps disabled - https://dovecot.org/bugreport.html#coredumps) Feb 28 16:57:50 server kernel: traps: auth[17009] general protection ip:7fd5d3ae9a64 sp:7fff0a781dc0 error:0 in libmariadb.so.3[7fd5d3add000+26000] It looks like the error is related to the libmariadb.so, of course I am using integration with auth sql. The configuration is like this: # 2.3.4.1 (f79e8e7e4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 () # OS: Linux 4.19.0-2-amd64 x86_64 Debian buster/sid ext4 # Hostname: p3x-dc.example.com auth_mechanisms = plain login mail_location = maildir:/var/mail/vhosts/%d/%n mail_plugins = quota mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe autoexpunge = 30 days special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe autoexpunge = 30 days special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_grace = 10%% quota_max_mail_size = 100M quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M sieve = /etc/dovecot/sieve.conf } protocols = imap lmtp pop3 service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0666 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 12340 } } ssl = required ssl_ca = </etc/ssl/example.com/ca.cer ssl_cert = </etc/ssl/example.com/fullchain.cer ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } protocol lmtp { auth_socket_path = /var/run/dovecot/auth-master mail_plugins = quota sieve } protocol imap { mail_plugins = quota imap_quota } *Patrik* WWW <https://patrikx3.com> | GitHub <https://github.com/patrikx3/> | NPM <https://www.npmjs.com/~patrikx3> | Corifeus <https://corifeus.com> | +36 20 342 8046

2 2

Dovecot Logging and Shell Script Hang
by Asai 28 Feb '19

28 Feb '19

Greetings, We have a PHP based shell script that runs a doveadm backup routine. This script was running just fine, and then we turned on more logging for Dovecot (enabled the mail_log plugin). For some reason, once this was done, the shell script would run for a few minutes and then hang, as it added more log messages to the dsync STDOUT. Once the mail_log plugin was disabled, it ran without hanging. Is there any way to run doveadm and temporarily suspend the mail_log plugin? Or is there a better way to do this? Thanks for the insights. Asai

3 3

Dovecot 2.3.3 Mailbox does not exist
by Kunal A. 27 Feb '19

27 Feb '19

HI, I would deeply aprechiate if someone here could help me address a problem with ACL. I would personally refrain from e-mailing ever so often as I understand other users have more important issues to discuss about. Anyways I hope someone here could help. When I run ;- doveadm acl get -A Public/Archive I keep getting an error that says Mailbox does not exist . See erorr message below : - Error: Can't open mailbox Public/Archive: Mailbox doesn't exist: Public/Archive In the fastmail folder there is a folder called Archive, why isn't it being read? The folder is drwxr-xr-x. with vmail as owner. Deeply appreciate if someone could help with this. Many thanks dovecot -n output :- # 2.3.3 (dcead646b): /etc/dovecot/dovecot.conf # OS: Linux 4.20.10-200.fc29.x86_64 x86_64 Fedora release 29 (Twenty Nine) # Hostname: machine auth_mechanisms = plain login mail_location = maildir:~/Maildir mail_plugins = acl mail_privileged_group = mail mbox_write_locks = fcntl namespace { list = children location = maildir:/run/media/computer/Storage/Email/fastmail/ prefix = Public/ separator = / subscriptions = yes type = public } namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX/ separator = / } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile } postmaster_address = postmaster at example.com protocols = imap pop3 service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_cipher_list = PROFILE=SYSTEM ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = uid=vmail gid=vmail home=/var/mail/vhosts/%d/%n driver = static } protocol imap { mail_plugins = acl imap_acl }

2 2

Re: performance issue with UID SEARCH
by Aki Tuomi 27 Feb '19

27 Feb '19

Without FTS, dovecot needs to open *each* and *every* email when doing text searches, which is understandably rather slow process. Aki On 27.2.2019 10.43, Marc Roos wrote: > > > I am not sure if this any help. From what I understand of maildir it has > lots of separate files, thus uid/gid lookups. Try running something like > nscd, that will cache these lookups? > > > > > -----Original Message----- > From: Aki Tuomi via dovecot [mailto:dovecot@dovecot.org] > Sent: 27 February 2019 06:24 > To: Ben Burke; Dovecot Mailing List > Subject: Re: performance issue with UID SEARCH > > > On 27 February 2019 03:27 Ben Burke via dovecot < > dovecot(a)dovecot.org> wrote: > > > Hi, > > I'm running dovecot 2.2.x and I'm having an issue where I see many > dovecot processes use all the available IO on a server. According > to > iotop the worst offenders seem to be in this state (NOTE: I swapped > in > phony username & IP info): > > dovecot/imap [someusername 123.456.789.012 UID SEARCH] > > The server in question is running with Maildirs on top of an XFS > filesystem. Is there anything I can do to optimize "UID SEARCH" or > find > out why it's being a problem? I've read > https://wiki2.dovecot.org/PerformanceTuning and the linked pages. > > By "being a problem" I mean iostat -xmt 1 /dev/diskdevice shows > 100% > utilization for long periods and in some cases io service times are > > taking many seconds... which causes thunderbird to timeout when > doing > things like appending messages to user "Sent" mailboxes. > > Any ideas? > > Thanks, > Ben Burke > > > Are you using FTS? If not, you should. See > https://wiki.dovecot.org/Plugins/FTS > --- > Aki Tuomi > >

1 0

CASifying dovecot and webmail
by Juan Carlos Sánchez 27 Feb '19

27 Feb '19

Hello: I'm trying to authenticate webmail (roundcube) and dovecot with Apereo CAS. I don't find any good guide and everything I have found is some years old. Has any of you done a recent integration with CAS and can share the details ? Best regards -- ------------------------------------------------------ Juan Carlos Sanchez Hernandez Servicio de Infraestructura e Innovación Universidad Politecnica de Madrid - www.upm.es Rectorado Avda. Ramiro de Maeztu 7 28040 Madrid ------------------------------------------------------

3 2

Re: Linking mailboxes
by Aki Tuomi 27 Feb '19

27 Feb '19

drwx-----x kunal kunal Storage I'm guessing this point is being more restrictive. Can you try changing this into 0755? Also suddenly you are no longer using vmail but random user IDs. This only works with vmail. Aki > On 25 February 2019 19:56 Kunal A. <kunal.oa(a)gmail.com> wrote: > > > Hi, > Yes. Output ;- > > dr-xr-xr-x root root / > drwxr-xr-x root root run > drwxr-xr-x root root media > drwxr-x--x root root computer > drwx-----x kunal kunal Storage > drwx------ vmail vmail Email > drwx------ vmail vmail tmp > > > > On Mon, Feb 25, 2019 at 12:53 PM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > Can you run > > > > namei -vl /run/media/computer/Storage/Email/tmp > > > > the + at the end does not mean that, it means there is an extended POSIX ACL. > > > > Aki > > > > > On 25 February 2019 19:50 Kunal A. <kunal.oa(a)gmail.com> wrote: > > > > > > > > > Hey Aki, > > > Sorry for the trouble but I am still unable to get it working. The file permissions for upper files are now drwxr-xr-x. but I think they need to be drwxr-x--x+ ? > > > Any idea how to change it to that? > > > > > > > > > Error log is below:- > > > > > > doveadm(lightdm): Error: Can't open mailbox Storage/Email: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=976(lightdm) egid=976(lightdm) missing +x perm: /run/media/computer/Storage/Email, dir owned by 5000:5000 mode=0700) > > > doveadm(setroubleshoot): Error: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=975(setroubleshoot) egid=975(setroubleshoot) missing +x perm: /run/media/computer/Storage/Email, dir owned by 5000:5000 mode=0700) > > > doveadm(setroubleshoot): Error: Can't open mailbox Storage/Email: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=975(setroubleshoot) egid=975(setroubleshoot) missing +x perm: /run/media/computer/Storage/Email, dir owned by 5000:5000 mode=0700) > > > doveadm(dovenull): Error: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=974(dovenull) egid=973(dovenull) missing +x perm: /run/media/computer/Storage/Email, dir owned by 5000:5000 mode=0700) > > > ~ > > > > > > > > > > > > > > > On Mon, Feb 25, 2019 at 11:58 AM Kunal A. <kunal.oa(a)gmail.com> wrote: > > > > Dear Aki, > > > > Many thanks again. Thanks for explaining this. > > > > Very grateful for your assistance. > > > > Thanks a bunch ! > > > > Kunal > > > > > > > > > > > > > > > > On Mon, Feb 25, 2019 at 11:56 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > > > > At minimum directories > > > > > > > > > > /run/media/computer/Storage/ > > > > > > > > > > need to have o+x, so you can do > > > > > > > > > > chmod o+x /run /run/media /run/media/computer /run/media/computer/Storage > > > > > > > > > > and > > > > > > > > > > chown -R vmail:vmail /run/media/computer/Storage/Email > > > > > > > > > > Aki > > > > > > > > > > > On 25 February 2019 18:47 Kunal A. via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > > > > Dear Aki, > > > > > > I have given permission to the folders Storage/Email to vmail. > > > > > > But the upper hierarchy is not under vmail . > > > > > > > > > > > > Full directory = /run/media/computer/Storage/Email > > > > > > > > > > > > /run/media/computer = not vmail > > > > > > Storage/Email = vmail > > > > > > > > > > > > How do I get this corrected ? It seems like vmail needs access through all through run/media/computer . > > > > > > > > > > > > Please excuse my ignorance as I am not very familiar with filesystem permissions. :-/ > > > > > > > > > > > > Sincere apologies for the trouble. > > > > > > > > > > > > Thanks > > > > > > Kunal A. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 25, 2019 at 11:36 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > > > > > > I would only run it for the mail folders to be safe. > > > > > > > > > > > > > > Aki > > > > > > > > > > > > > > > On 25 February 2019 18:34 Kunal A. via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > Dear Aki, > > > > > > > > Many thanks for your feedback. Its what I figured I should do but I was not sure how safe it was. > > > > > > > > > > > > > > > > Would it make sense to execute the following to /run/media/computer so that the user/group is vmail? > > > > > > > > sudo chown -R vmail:vmail to the /run folder ? > > > > > > > > > > > > > > > > Also if this practice is safe to perform? > > > > > > > > > > > > > > > > Thanks > > > > > > > > Kunal A > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Feb 25, 2019 at 11:20 AM Aki Tuomi <aki.tuomi(a)open-xchange.com> wrote: > > > > > > > > > > > > > > > > > > > On 25 February 2019 18:19 Aki Tuomi via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 25 February 2019 17:56 Kunal A. via dovecot <dovecot(a)dovecot.org> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hey Guys! > > > > > > > > > > > Many thanks for your feedback. I have kind of gotten it to work but I just struggling to get the permissions of the folder correct. Highly appreciate if someone could help me set the correct permission settings for it . Terribly sorry for my ignorance. > > > > > > > > > > > > > > > > > > > > > > My error msg is listed below : - > > > > > > > > > > > > > > > > > > > > > > doveadm(vmail): Error: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +x perm: /run/media/computer, dir owned by 0:0 mode=0750) > > > > > > > > > > > doveadm(vmail): Error: Can't open mailbox Storage/Email: stat(/run/media/computer/Storage/Email/tmp) failed: Permission denied (euid=5000(vmail) egid=5000(vmail) missing +x perm: /run/media/computer, dir owned by 0:0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > You need to give write permissions to vmail on /run/media/computer/Storage/Email > > > > > > > > > > > > > > > > > > > > You can > > > > > > > > > > - change owner to vmail > > > > > > > > > > - change group to vmail and add rwx for group > > > > > > > > > > - use posix ACLs > > > > > > > > > > - change mode to 0777 (unsafe) > > > > > > > > > > > > > > > > > > > > Aki > > > > > > > > > > > > > > > > > > And the same applies for all files & directories under /run/media/computer/Storage/Email > > > > > > > > > > > > > > > > > > Aki > > > > > > > > > > > > > > > > > > > > > > >

3 2

performance issue with UID SEARCH
by Ben Burke 27 Feb '19

27 Feb '19

Hi, I'm running dovecot 2.2.x and I'm having an issue where I see many dovecot processes use all the available IO on a server. According to iotop the worst offenders seem to be in this state (NOTE: I swapped in phony username & IP info): dovecot/imap [someusername 123.456.789.012 UID SEARCH] The server in question is running with Maildirs on top of an XFS filesystem. Is there anything I can do to optimize "UID SEARCH" or find out why it's being a problem? I've read https://wiki2.dovecot.org/PerformanceTuning and the linked pages. By "being a problem" I mean iostat -xmt 1 /dev/diskdevice shows 100% utilization for long periods and in some cases io service times are taking many seconds... which causes thunderbird to timeout when doing things like appending messages to user "Sent" mailboxes. Any ideas? Thanks, Ben Burke

2 1

new Centos server install yum dependancy error
by Voytek Eymont 26 Feb '19

26 Feb '19

I'm installing a new Centos 7 server, following repo.dovecot.org/ page, but, get this: (I've opted to install all BUT pgsql (use mysql), not sure I really need all other packages..?) should I just go for "dovecot dovecot-mysql dovecot-pigeonhole" ?? tried # yum install dovecot dovecot-debuginfo dovecot-devel dovecot-imaptest dovecot-imaptest-debuginfo dovecot-lua dovecot-mysql dovecot-pigeonhole dovecot-pigeonhole-debuginfo dovecot-pigeonhole-devel ... --> Finished Dependency Resolution Error: Package: 2:dovecot-lua-2.3.4.1-1.x86_64 (dovecot-2.3-latest) Requires: dovecot = 2:2.3.4.1-1 Available: 1:dovecot-2.2.36-3.el7.i686 (base) dovecot = 1:2.2.36-3.el7 Error: Package: 2:dovecot-pigeonhole-devel-2.3.4.1-1.x86_64 (dovecot-2.3-latest) Requires: dovecot-pigeonhole = 2:2.3.4.1-1 Installing: 1:dovecot-pigeonhole-2.2.36-3.el7.x86_64 (base) dovecot-pigeonhole = 1:2.2.36-3.el7 Error: Package: 2:dovecot-imaptest-2.3.4.1-1.x86_64 (dovecot-2.3-latest) Requires: dovecot = 2:2.3.4.1-1 Available: 1:dovecot-2.2.36-3.el7.i686 (base) dovecot = 1:2.2.36-3.el7 You could try using --skip-broken to work around the problem You could try running: rpm -Va --nofiles --nodigest -- Voytek

4 12

Removing a mailbox from a dovecot cluster
by Francis 26 Feb '19

26 Feb '19

Hello, How should I remove a mailbox from a dovecot cluster? I have two dovecot in a cluster. I tried to remove one user's mailbox with theses commands I found with a little google search: doveadm flags add -u user(a)domain.tld "\\Deleted" ALL doveadm expunge -u user(a)domain.tld mailbox "*" DELETED doveadm purge -u user(a)domain.tld rm -fr /var/vmail/domains/domain.tld/user If I do theses commands in a script (so one command after one, very quick), the data just keep coming back from the other server in the cluster (I suppose this is what's happening). If I do theses commands manually, so more slowly, the mailbox is created again but empty this time. If I connect from an IMAP client, I see an empty mailbox. Same thing if I connect an IMAP client to the second server from the cluster. But I still see the data in the storage (mdbox) of the second server (the "m.#" files are still there an they waste space on disks). I tried to do a "doveadm purge" on the second server, but it did nothing more, the data is still keep on the disks. I'm running dovecot 2.2.33.2 on ubuntu 18.04 on both servers. Thanks! -- Francis

1 1