October 2018 - dovecot

Fatal: setgid, imap connections dropped.
by J. de Meijer 19 Oct '18

19 Oct '18

Hi, I'm getting errors with my IMAP setup. Basically, everything seems to work. Mail is delivered nicely from Postfix to Dovecot via LMTP. Dovecot does the authentication to LDAP (also for Postfix). Users are able to send mail via authenticated submission (Postfix) and login into IMAP and POP. However, IMAP connections are dropped frequently with an "ERROR: Connection dropped by IMAP server.". After pressing reload on the webmail, or refreshing in the client might help for a short period. So it fails intermittently. The errors in the maillog are below. It seems to be mixing up users kind of randomly. I think when multiple connections are made at the same time. Did a lot of searching, put couldn't find an answer to this problem. All I can find is related to LDA, which I'm not using. Any help would be appreciated. Errors from the log: Sep 28 00:03:24 mailserver dovecot: imap(userD)<14864><WT8DguF2MspUUoaT>: Fatal: setgid(1012(userD) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1012(userD) instead of 1011(userA)) Sep 28 00:03:24 mailserver dovecot: imap(userD)<17009><recJguF2NMpUUoaT>: Fatal: setgid(1012(userD) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1012(userD) instead of 1011(userA)) Sep 28 00:03:26 mailserver dovecot: imap(userD)<12807><8T0iguF2NspUUoaT>: Fatal: setgid(1012(userD) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1012(userD) instead of 1011(userA)) Sep 28 00:06:59 mailserver dovecot: imap(userD)<15661><UcfOjuF2OcpUUoaT>: Fatal: setgid(1012(userD) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1012(userD) instead of 1011(userA)) Sep 28 00:07:54 mailserver dovecot: imap(userA)<45614><NVkakuF2xO5UUoaT>: Fatal: setgid(1011(userA) from userdb lookup) failed with euid=1012(userD), gid=1012(userD), egid=1012(userD): Operation not permitted (This binary should probably be called with process group set to 1011(userA) instead of 1012(userD)) Sep 28 00:08:08 mailserver dovecot: imap(userF)<45055><AWjtkuF2J/ptSCYM>: Fatal: setgid(1033(userF) from userdb lookup) failed with euid=1012(userD), gid=1012(userD), egid=1012(userD): Operation not permitted (This binary should probably be called with process group set to 1033(userF) instead of 1012(userD)) Sep 28 00:08:08 mailserver dovecot: imap(userF)<46412><87ntkuF2JvptSCYM>: Fatal: setgid(1033(userF) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1033(userF) instead of 1011(userA)) Sep 28 00:08:08 mailserver dovecot: imap(userF)<44858><0nXzkuF2KfptSCYM>: Fatal: setgid(1033(userF) from userdb lookup) failed with euid=1012(userD), gid=1012(userD), egid=1012(userD): Operation not permitted (This binary should probably be called with process group set to 1033(userF) instead of 1012(userD)) Sep 28 00:08:14 mailserver dovecot: imap(userF)<36517><v/NHk+F2K/ptSCYM>: Fatal: setgid(1033(userF) from userdb lookup) failed with euid=1017(userC), gid=1017(userC), egid=1017(userC): Operation not permitted (This binary should probably be called with process group set to 1033(userF) instead of 1017(userC)) Sep 28 00:08:36 mailserver dovecot: imap(userF)<10531><wpKdlOF2MfptSCYM>: Fatal: setgid(1033(userF) from userdb lookup) failed with euid=1011(userA), gid=1011(userA), egid=1011(userA): Operation not permitted (This binary should probably be called with process group set to 1033(userF) instead of 1011(userA)) # dovecot --version 2.3.2.1 (0719df592) # 2.3.2.1 (0719df592): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.2 (7704de5e) # OS: FreeBSD 11.2-RELEASE amd64 # Hostname: callisto auth_cache_size = 10 M auth_debug = yes imap_idle_notify_interval = 29 mins mail_debug = yes mail_fsync = never mail_location = maildir:~/Maildir mail_plugins = " fts fts_solr" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = create special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_autoindex = yes fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ recipient_delimiter = + sieve = file:~/sieve;active=~/.dovecot.sieve sieve_after = /usr/local/etc/dovecot/sieve-after.d sieve_before = /usr/local/etc/dovecot/sieve-before.d sieve_quota_max_storage = 50M } protocols = imap pop3 lmtp sieve service auth { client_limit = 1600 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { inet_listener imap { address = 127.0.0.1, ::1 } process_min_avail = 3 service_count = 1 } service imap { process_min_avail = 3 service_count = 256 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service pop3-login { inet_listener pop3 { address = 127.0.0.1, ::1 } service_count = 1 } ssl = required ssl_cert = </usr/local/etc/ssl/mail.demeijer.com.dovecot.crt ssl_cipher_list = ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256 ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /usr/local/etc/dovecot/dovecot-ldap-user.conf.ext driver = ldap } protocol lda { mail_fsync = optimized mail_plugins = " fts fts_solr sieve" } protocol imap { mail_max_userip_connections = 50 mail_plugins = " fts fts_solr imap_sieve" } protocol lmtp { lmtp_save_to_detail_mailbox = yes mail_fsync = optimized mail_plugins = " fts fts_solr quota sieve" postmaster_address = webmaster(a)example.com } In /usr/local/etc/dovecot/dovecot-ldap.conf.ext: hosts = localhost dn = cn=reader,ou=Roles,dc=example,dc=com dnpass = secretpassword auth_bind = yes base = ou=People,dc=example,dc=com deref = never scope = subtree user_attrs = uid=uid,homeDirectory=home,uidNumber=uid,gidNumber=gid user_filter = (&(objectClass=posixAccount)(uid=%n)) pass_attrs = uid=user,userPassword=password pass_filter = (&(objectClass=posixAccount)(uid=%u)) default_pass_scheme = SSHA dovecot-ldap-user.conf.ext is the same as dovecot-ldap.conf.ext # freebsd-version 11.2-RELEASE-p3 Everything is installed from ports.

2 6

trouble with ssl disable in centos 7 with version 1.0 1.2 and 3.10
by Joseph Harth 19 Oct '18

19 Oct '18

I am trying to use dovecot without ssl in centos 7. It works with imap but I am not able to make it work with pop3. Pop3 only works with ssl=yes. I can use dovecot in centos 4 ok. I get the same error with thunderbird and outlook. But for some reason it works when I telnet to port 110 and do retr 1. I was able to work pam and shadow passwords. this is the output that does not work with tls disabled Oct 18 13:30:17 auth: Debug: Loading modules from directory: /usr/lib64/dovecot/authOct 18 13:30:17 auth: Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.soOct 18 13:30:17 auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.soOct 18 13:30:17 auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.datOct 18 13:30:17 auth: Debug: auth client connected (pid=22513)Oct 18 13:30:17 auth: Debug: client in: AUTH 1 PLAIN service=pop3 session=WXv1I4R4FOXAqAAK lip=192.168.0.37 rip=192.168.0.10 lport=110 rport=58644Oct 18 13:30:17 auth: Debug: client passdb out: CONT 1Oct 18 13:30:17 auth: Debug: client in: CONT 1 == (previous base64 data may contain sensitive data)Oct 18 13:30:17 auth-worker(22515): Debug: Loading modules from directory: /usr/lib64/dovecot/authOct 18 13:30:17 auth-worker(22515): Debug: Module loaded: /usr/lib64/dovecot/auth/lib20_auth_var_expand_crypt.soOct 18 13:30:17 auth-worker(22515): Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.soOct 18 13:30:17 auth-worker(22515): Debug: shadow(joe2,192.168.0.10,<WXv1I4R4FOXAqAAK>): lookupOct 18 13:30:17 auth: Debug: client passdb out: OK 1 user=joe2Oct 18 13:30:17 auth: Debug: master in: REQUEST 2718302209 22513 1 05eac7a2917ce3bc1f0f359b21b138e6 session_pid=22516Oct 18 13:30:17 auth-worker(22515): Debug: passwd(joe2,192.168.0.10,<WXv1I4R4FOXAqAAK>): lookupOct 18 13:30:17 auth: Debug: master userdb out: USER 2718302209 joe2 system_groups_user=joe2 uid=660 gid=661 home=/home/joe2Oct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: Effective uid=660, gid=661, home=/home/joe2Oct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/mail/joe2Oct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: fs: root=/home/joe2/mail, index=, indexpvt=, control=, inbox=/var/mail/joe2, alt=Oct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: Mailbox INBOX: Mailbox opened because: POP3 INBOXOct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: Mailbox INBOX: UID 30: Opened mail because: accessOct 18 13:30:17 pop3(joe2)<22516><WXv1I4R4FOXAqAAK>: Debug: Mailbox INBOX: UID 30: Opened mail because: RETR This is the output with TLS and it works Oct 18 13:30:32 auth: Debug: auth client connected (pid=22542)Oct 18 13:30:33 auth: Debug: client in: AUTH 1 PLAIN service=pop3 secured=tls session=44feJIR4FeXAqAAK lip=192.168.0.37 rip=192.168.0.10 lport=110 rport=58645 ssl_cipher=ECDHE-RSA-AES128-GCM-SHA256 ssl_cipher_bits=128 ssl_pfs=ECDH ssl_protocol=TLSv1.2Oct 18 13:30:33 auth: Debug: client passdb out: CONT 1Oct 18 13:30:33 auth: Debug: client in: CONT 1 == (previous base64 data may contain sensitive data)Oct 18 13:30:33 auth-worker(22515): Debug: shadow(joe2,192.168.0.10,<44feJIR4FeXAqAAK>): lookupOct 18 13:30:33 auth: Debug: client passdb out: OK 1 user=joe2Oct 18 13:30:33 auth: Debug: master in: REQUEST 4165992449 22542 1 cb03b70f92c0c5c030e25a39ea4b8518 session_pid=22543Oct 18 13:30:33 auth-worker(22515): Debug: passwd(joe2,192.168.0.10,<44feJIR4FeXAqAAK>): lookupOct 18 13:30:33 auth: Debug: master userdb out: USER 4165992449 joe2 system_groups_user=joe2 uid=660 gid=661 home=/home/joe2Oct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Effective uid=660, gid=661, home=/home/joe2Oct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/mail/joe2Oct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: fs: root=/home/joe2/mail, index=, indexpvt=, control=, inbox=/var/mail/joe2, alt=Oct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Mailbox INBOX: Mailbox opened because: POP3 INBOXOct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Mailbox INBOX: UID 30: Opened mail because: accessOct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Mailbox INBOX: UID 30: Opened mail because: RETROct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Mailbox INBOX: UID 31: Opened mail because: accessOct 18 13:30:33 pop3(joe2)<22543><44feJIR4FeXAqAAK>: Debug: Mailbox INBOX: UID 31: Opened mail because: RETR

2 1

doveadm expunge with static users
by azurit＠pobox.sk 18 Oct '18

18 Oct '18

Hi, is it possible to use 'doveadm expunge' with static user database? Or can i, somehow, tell dovecot the directory location where i need to do an expunge? Thank you. azur

2 3

Ass(et) protection for mobile users
by Rupert Gallagher 18 Oct '18

18 Oct '18

Hello! Is it possible to hide the public folder when the user is on its mobile phone?

2 6

Problems recovering IMAP Data
by Albert Whale 18 Oct '18

18 Oct '18

HELP, I have experience an issue with the long running IMAP service using Dovecot 2.1.15. I have replaced the interface cards (suspected failure), and I have only recently been able to retrieve my Inbox (1.45GB) via POP3 protocol (I normally run IMAP) I am suspecting a corrupted dovecot.index, but which one? I have counted 260 files in my IMAP tree with the word dovecot. I cannot determine which folder is corrupted with 2.1.5 even with the Mail Debug option. How can I recover the use of my IMAP configuration? Thank you.

2 1

dovecot passdb driver=checkpassword as external script for block ip
by bes 18 Oct '18

18 Oct '18

Refer to https://dovecot.org/pipermail/dovecot/2015-March/099971.html https://wiki.dovecot.org/PasswordDatabase I tried to repeat the same thing. Set these passdb: passdb { args = /myscript.sh ip=%r driver = checkpassword result_failure = return-fail result_success = continue } passdb { args = /etc/dovecot/dovecot-sql.conf.ext ( my working auth method ) driver = sql } Created executable /myscript.sh: #!/bin/sh exit 1 But I saw in the logs: dovecot[15111]: auth: Error: checkpassword(techdept(a)nestormedia.com,192.168.99.128,<Mos6jW94XNzAqGOA>): Child 15115 exited with status 1 dovecot[18016]: imap-login: Login: user=....[successful login] Why result_failure=return-fail from first passdb does not work?

2 2

"Broken MIME parts for mail"
by Poliman - Serwis 18 Oct '18

18 Oct '18

I have many lines like below in error.log file: Oct 16 10:03:22 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: unlink(/var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache) failed: No such file or directory (in mail-cache.c:28) Oct 16 10:03:22 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: Corrupted index cache file /var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache: Broken MIME parts for mail UID 1019 in mailbox INBOX: Cached MIME parts don't match message during parsing: Cached header size mismatch (parts=) Oct 16 10:03:23 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: unlink(/var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache) failed: No such file or directory (in mail-cache.c:28) Oct 16 10:03:23 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: Corrupted index cache file /var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache: Broken MIME parts for mail UID 1052 in mailbox INBOX: Cached MIME parts don't match message during parsing: Cached header size mismatch (parts=) Oct 16 10:03:23 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: unlink(/var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache) failed: No such file or directory (in mail-cache.c:28) Oct 16 10:03:23 s1 dovecot: imap(artur(a)firmastanaszek.pl): Error: Corrupted index cache file /var/vmail/firmastanaszek.pl/artur/Maildir/dovecot.index.cache: Broken MIME parts for mail UID 1098 in mailbox INBOX: Cached MIME parts don't match message during parsing: Cached header size mismatch (parts=) I didn't do anything with Dovecot neither update in last time nor move files or something. Does anybody could tell me what above means and how to fix it?

1 0

Replication 'noreplicate' field usage
by Phil 17 Oct '18

17 Oct '18

I've set up replication as per the wiki under 2.3.3 and in my SQL user query I am returning a 'noreplicate' field, this is a boolean and returns true for all users at the moment. However when I do "doveadm replicator status '*'" it lists all users in my database suggesting it is trying to sync all users regardless of the noreplicate field. ``` user_query = SELECT home, uid, gid, noreplicate FROM users WHERE address = '%u' ``` It seems that the noreplicate field is being ignored, am I using it correctly?

1 0

: Re: Corrupted dbox file/index
by Raymond Sellars 16 Oct '18

16 Oct '18

>On Mon, Oct 15, 2018 at 08:50:21AM +0000, Raymond Sellars wrote: > > Hi > > Looking for some insight into mdbox index file management and recovery from corruptions. > What version of dovecot and what OS are you running? Is NFS linux/bsd/netapp/etc? [Raymond] Linux 2.6.32-279.19.1.el6.x86_64 Red Hat Enterprise Linux Server release 6.7 (Santiago) - its selectively patched a little higher than this. nfs-utils-1.2.3-64.el6.x86_64 nfs-utils-lib-1.1.5-11.el6.x86_64 NFS is provided by SAN array. Either Isilon SAN or VNX SAN. I believe the older Isilon with NFS. [Raymond Sellars] fstab: <ip removed>:/ mailshare-isi /mailshare nfs _netdev,noatime,vers=3,rsize=32768,wsize=32768,bg,intr,tcp 0 0 Thanks Raymond

1 0

Corrupted dbox file/index
by Raymond Sellars 15 Oct '18

15 Oct '18

Hi Looking for some insight into mdbox index file management and recovery from corruptions. I have a two node cluster on NFS with proxy director in front for user stickness. One node (a nominated master) bidirectionally replicates to a 3rd node on a DR site. We periodically get index file corruptions resulting in rebuilds. However the user experience is poor as messages read/deleted from months/years ago all reappear as unread again. We've seen corruption because of NFS NTP time sync problems, proxy not being stick, but also the DR node being off line for a while and then tripping corruption within production when it comes back on. Error message example (1 of): Error: Corrupted dbox file /mailshare/.. (removed) ../home/mail/storage/m.4 (around offset=993548): EOF reading msg header (got 0/30 bytes) https://wiki2.dovecot.org/MailboxFormat/dbox- i've read up on all the documentation I can find and understand " you must not lose the dbox index files, they can't be regenerated without data loss." Questions: #1 Any additional tips for avoiding mdbox index corruptions with dsync? Or should I revert to maildir format? I like the performance premise of the mdbox but these index corruptions are a reliability issue. #2 I'm guessing read status is one of the meta data items lost. But its seems it can't recover it from dovecot.index.backup files either. Any technique to preserve that item as its key to the user experience? #3 If index/transaction logs are so critical is there some kind of check point backups I can take? Native dovecot feature or do I need to script something. #4 I've noticed that rebuilding the index does not work if the dovecot.index.log file is lost (deleted as a hard test). The dovecot.index.cache can be but once the log file i gone messages are not automatically (or manually that i can find) recovered from the storage directory. I've not seen any dovecot.index.log file corruptions but that file seems very high risk. If rebuilding the index only from the log file or a combination process from storage directory? Is there perhaps an option to just use the transaction log and not the index? Although that doesn't sound wise for performance. #5 In additional to status UNREAD we also notice files moved to the trash reappear. Is that expected behavior? Thanks Raymond

2 1