October 2014 - dovecot

[Dovecot] dsync replication errors
by Oli Schacher 09 Sep '15

09 Sep '15

Hi I'm trying to build a cluster of two servers with dsync replication (based on http://wiki2.dovecot.org/Replication) My test setup works fine for very simple tests, I can log in to both servers, copy a message to one of the servers and it successfully apperars in the other account. But, if I try to copy a large amount of messages at once to one of the accounts, my maillogs get flodded with errors(see below) and the mailboxes seem to get out of sync and messages are duplicated over and over again (I originally copied 100 messages and ended up with thousands in both mailboxes until I killed dovecot) I'd appreciate if someone could have a look at my config and tell me what I did wrong. dovecot.conf of both servers, they are identical except for the target ip in mail_replica: dovecot -n # 2.2.beta1 (070ca24e5846+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_plugins = " notify replication" namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = remote:vmail@192.168.23.62 } protocols = pop3 imap service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service replicator { process_min_avail = 1 } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } Log on server1 after I copied 100 messages to an account on that server: Jan 31 10:41:04 doco1 dovecot: imap-login: Login: user=<user1>, method=PLAIN, rip=192.168.23.130, lip=192.168.23.61, mpid=1432, session=<OdjlbJLUmwDAqBeC> Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=72, file=1359625327.M621257P1432.doco1,S=2472,W=2547:2,) Jan 31 10:42:12 doco1 dovecot: dsync-local(user1): Error: Recent flags state corrupted for mailbox INBOX Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=73, file=1359625327.M740847P1432.doco1,S=2417,W=2492:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=74, file=1359625328.M206735P1432.doco1,S=2400,W=2474:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=75, file=1359625328.M668118P1432.doco1,S=2421,W=2496:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=76, file=1359625329.M167578P1432.doco1,S=2480,W=2559:2,) Jan 31 10:42:13 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=77, file=1359625329.M520528P1432.doco1,S=2525,W=2604:2,) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132: 1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133: 1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134: 1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135: 1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136: 1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137: 1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139: 1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140: 1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141: 1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142: 1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143: 1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144: 1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145) Jan 31 10:42:14 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=85: 1359625327.M740847P1432.doco1,S=2417,W=2492 != 2e1b1ee97994566870c02910ea929091 Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=175) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=150) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=176) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=179) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=123) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=147) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=174) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=149) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=173) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=148) Jan 31 10:42:16 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Importing mailbox INBOX failed Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: read(vmail(a)192.168.23.62) failed: EOF Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: Remote command returned error 75 Jan 31 10:42:20 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=230) [...] Log on server2: Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132: 1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133: 1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134: 1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135: 1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136: 1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137: 1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139: 1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140: 1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141: 1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142: 1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143: 1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144: 1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=84, file=1359625330.M608391P1432.doco1,S=2480,W=2559:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=85, file=1359625330.M813949P1432.doco1,S=2427,W=2502:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=86, file=1359625331.M217320P1432.doco1,S=2547,W=2625:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=87, file=1359625331.M688431P1432.doco1,S=2529,W=2609:2,) [...] Thanks Oli -- message transmitted on 100% recycled electrons

7 45

[Dovecot] Dovecot FTS using ElasticSearch
by Darvin Denmian 21 Jul '15

21 Jul '15

Hi, I would like to know if is possible to use ElasticSearch instead of Solr for FTS. Regards, Erick Vitor de Barros.

6 6

[Dovecot] imapc - different settings for different namespaces
by Timo Sirainen 20 Mar '15

20 Mar '15

Originally I was planning on allowing all kinds of mail settings inside namespace {}, including imapc_* settings. But that's a bit difficult to implement (although I think it will happen some day). So for now I was thinking: imapc foo { host = imap.foo.com master_user = foomaster password = foopass } imapc bar { host = imap.bar.com ... } namespace { prefix = foo/ location = imapc:foo } namespace { prefix = bar/ location = imapc:bar } So basically if the path after imapc isn't absolute (/path or ~/path), then treat the path as the imapc section name and use its settings. Anyone have any improvement suggestions? I was planning on adding this to v2.2.3. I'm not sure if the section names should be "imapc" or something else (e.g. imap_remote {}).

2 2

[Dovecot] Sieve vacation not working
by Phill Edwards 24 Feb '15

24 Feb '15

I've been trying all night to get vacation working in my .dovecot.sieve file. The way our email works is we have one account on dovecot which serves my mail and my wife's and we use procmail and sieve to put the emails in different folders depending on who it's for. I'm trying to set up a vacation reply on my wife's email address (aaaaaaaa@optus* below) so people can update to her new address. Here's my file with the email addresses masked: require ["fileinto", "vacation"]; if address :is "to" "xxxxxxxxxxxxxxx(a)ozemail.com.au" { fileinto "Phill_OzEmail_Folder"; stop; } elsif address :is "to" "yyyyyyyyyyyyyyyyy(a)optusnet.com.au" { fileinto "Phill_Optus_Folder"; stop; } elsif address :matches "to" "aaaaaaaa@optus*" { vacation "Please update my email address to bbbbbbbbb(a)exemail.com.au"; fileinto "Sarah_Optus_Folder"; stop; } else { keep; stop; } The problem I'm having is that no auto-replies are sent. I removed .dovecot.lda-dupes and that didn't help; so then I touched it and now it's a 0 byte file and that didn't help. There is no dovecot.sieve.err file and I can't see anything in /var/log/dovecot.log which gives any message about why this wouldn't work. The emails to aaaaaaa@optus* get filed into the right folder so I know the matching is working. I've also tried a reject recipe and that worked fine, so I know that it can send an email if it chooses to. So it's a complete black box mystery as to why it just won't send a reply. Regards, Phill

5 9

Permissions for shared mail don't work
by Damien Desfontaines 17 Feb '15

17 Feb '15

Hello, (Sorry if you already received this mail. I sent it without having subscribed to the list, but I never got any confirmation; so I assumed it has been filtered by the anti-spam or something.) I'm using dovecot v.2.1.7 on Debian Sid, and the output of dovecot -n is available here: http://paste.fulltxt.net/C-O I'm trying to make the mailbox of (system) user "spam" available to user "ted". I followed these instructions: http://wiki2.dovecot.org/SharedMailboxes/Permissions and it doesn't seem to work. To be more precise: - I manually (and recursively) changed the permissions of all files in ~spam/Mail to 660 (and 770 for the directories); - I manually (and recursively) changed the group owner of everything in ~spam/Mail to "mailperso", where "mailperso" is a group whose users are spam and ted; - I added « mail_access_groups=mailperso » into my dovecot config; - and I restarted dovecot. But still, when I send a new email to spam, it appears as being owned by group "spam" and having permissions set to 600, so I can't access it with user "ted". I also tried to set the setgid bit for ~spam/Mail, it didn't change anything, and when I recursively set the setgid for all subdirectories of ~spam/Mail, it gives new mails the correct group (mailperso) but not the correct permissions. Have I missed something, or is this a bug? Thanks in advance and best regards, Damien

3 8

Corrupted index cache after migrating from maildir to mdbox
by Tim 09 Jan '15

09 Jan '15

Hi guys, I'm running dovecot/postfix on my Debian Jessie maschine: # dovecot --version 2.2.13 doveconf -n output: http://p.bastelfreak.de/HsVwS/ today I migrated from maildir(located at /var/vmail) to mdbox with these commands: rsync -av -f"+ */" -f"- *" /var/vmail/ /home/vmail/ rm /home/vmail/*/*/Maildir -rf for i in $(doveadm user \*); do u=$(echo $i | cut -d@ -f1); d=$(echo $i |cut -d@ -f2); dsync -v -u $i mirror mdbox:/home/vmail/$d/$u/mdbox; done Then I changed the mail_location to mdbox:/home/vmail/%d/%n/mdbox and did the following commands: service dovecot restart for i in $(doveadm user \*); do u=$(echo $i | cut -d@ -f1); d=$(echo $i |cut -d@ -f2); dsync -v -u $i mirror maildir:/var/vmail/$d/$u/Mail; done doveadm index -A all INBOX for i in $(doveadm user \*); do doveadm -Dv fts rescan -u $i;done unfortunately I'm getting the following two kinds of error messages for all mail boxes in my mail.log: Error: Corrupted index cache file /home/vmail/%domain/%user/mdbox/mailboxes/%maildir/dbox-Mails/dovecot.index.cache: Broken virtual size for mail UID 2564 and: Error: read(/home/vmail/%domain/%user/mdbox/storage/m.108 ): FETCH BODY[] for mailbox %maildir UID 2564 got too little data: 2417 vs 6319 please let me know if you have any suggestions for fixing this issue or if you need more information. kind regards, Tim

6 19

Dovecot-ee
by Davide 25 Dec '14

25 Dec '14

Hi to all, i found that dovecot-ee repository access is free 0,00 $ cost; i'm running dovecot community 2.2.13 can i migrate my system to dovecot-ee? What are difference between Dovecot-ee and Dovecot-community? -- *Davide Marchi* *T*eorema *F*errara *Srl* Via Spronello, 7 - Ferrara - 44121 Tel. *0532783161* Fax. *0532783368* E-m@il: *davide.marchi(a)mail.cgilfe.it* Skype: *davide.marchi73* Web: *http://www.cgilfe.it* *CONFIDENZIALITA'* *Ai sensi del D.Lgs. 196/2003 si precisa che le informazioni contenute in questo messaggio sono riservate ed a uso esclusivo del destinatario/dei destinatari. Qualora il messaggio in parola Le fosse pervenuto per errore, La invitiamo ad eliminarlo senza copiarlo e a non inoltrarlo a terzi, dandocene gentilmente comunicazione.* *Per favore, pensa all'ambiente. Stampa questa email solo se necessario.*

3 3

2.2.15 Panic in mbox_sync_read_next_mail()
by Hans Morten Kind 27 Nov '14

27 Nov '14

It might not be a fault in dovecot, as the user is accessing the folder locally with alpine while also running imap-sessions. However it would have been nice with a more graceful action than panic? The panic is preceeded by Error: Next message unexpectedly corrupted in mbox file PATH Panic: file mbox-sync.c: line 152 (mbox_sync_read_next_mail): assertion failed: (sync_ctx->input->v_offset != mail_ctx->mail.from_offset || sync_ctx->input->eof) At #7 in the enclosed backtrace the actual values are sync_ctx->input->v_offset = 564 mail_ctx->mail.from_offset = 564 sync_ctx->input->eof = 0 Some will recommend convertion to maildir, but with 25 years history, thousands of active users and dozens terrabytes of mboxes, we are not even considering it. hmk GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i386-pc-solaris2.10"... Reading symbols from /datapool/local/dovecot-2.2.15/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /local/dovecot-2.2.15/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /datapool/local/dovecot-2.2.15/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /local/dovecot-2.2.15/lib/dovecot/libdovecot.so.0 Reading symbols from /datapool/local/program/lib/libssl.so.1.0.0...done. Loaded symbols for /local/program/lib/libssl.so.1.0.0 Reading symbols from /datapool/local/program/lib/libcrypto.so.1.0.0...done. Loaded symbols for /local/program/lib/libcrypto.so.1.0.0 Reading symbols from /datapool/local/program/lib/libz.so...done. Loaded symbols for /local/program/lib/libz.so Reading symbols from /lib/libm.so.2...done. Loaded symbols for /lib/libm.so.2 Reading symbols from /datapool/local/program/lib/libiconv.so.2...done. Loaded symbols for /local/program/lib/libiconv.so.2 Reading symbols from /lib/librt.so.1...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libnsl.so.1...done. Loaded symbols for /lib/libnsl.so.1 Reading symbols from /lib/libsocket.so.1...done. Loaded symbols for /lib/libsocket.so.1 Reading symbols from /lib/libsendfile.so.1...done. Loaded symbols for /lib/libsendfile.so.1 Reading symbols from /lib/libc.so.1...done. Loaded symbols for /lib/libc.so.1 Reading symbols from /lib/libdl.so.1...done. Loaded symbols for /lib/libdl.so.1 Reading symbols from /usr/lib/libz.so...done. Loaded symbols for /usr/lib/libz.so Reading symbols from /lib/libaio.so.1...done. Loaded symbols for /lib/libaio.so.1 Reading symbols from /lib/libmd.so.1...done. Loaded symbols for /lib/libmd.so.1 Reading symbols from /lib/ld.so.1...done. Loaded symbols for /lib/ld.so.1 Core was generated by `dovecot/imap imap-postlogin'. Program terminated with signal 6, Aborted. [New process 86892 ] #0 0xfe80c8e5 in _lwp_kill () from /lib/libc.so.1 (gdb) #0 0xfe80c8e5 in _lwp_kill () from /lib/libc.so.1 No symbol table info available. #1 0xfe807765 in thr_kill () from /lib/libc.so.1 No symbol table info available. #2 0xfe7b376f in raise () from /lib/libc.so.1 No symbol table info available. #3 0xfe7929e1 in abort () from /lib/libc.so.1 No symbol table info available. #4 0xfeda1c82 in default_fatal_finish (type=LOG_TYPE_PANIC, status=0) at failures.c:152 backtrace = 0x8093260 "0xfeda2b9f -> 0xfeda1e9b -> 0xfee9f57a -> 0xfeea207c -> 0xfeea37e9 -> 0xfeea4363 -> 0xfeea45cd -> 0xfeea477d -> 0xfeecb356 -> 0x8074b45 -> 0x805eeda -> 0x805f463 -> 0x806aad1 -> 0x8069a41 -> 0x8069d26"... #5 0xfeda2b9f in i_internal_fatal_handler (ctx=0x80474e0, format=0xfef4c588 "file %s: line %d (%s): assertion failed: (%s)", args=0x8047504 "°Æôþ\230") at failures.c:152 status = 0 #6 0xfeda1e9b in i_panic (format=0xfef4c588 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:152 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = 0x8047504 "°Æôþ\230" #7 0xfee9f57a in mbox_sync_read_next_mail (sync_ctx=0x8047794, mail_ctx=0x804760c) at ../../../../src/lib/array.h:158 __FUNCTION__ = "mbox_sync_read_next_mail" #8 0xfeea207c in mbox_sync_loop (sync_ctx=0x8047794, mail_ctx=0x804760c, partial=false) at ../../../../src/lib/array.h:158 rec = (const struct mail_index_record *) 0x0 uid = 0 messages_count = 66 offset = 564 ret = 1 expunged = false skipped_mails = true uids_broken = false #9 0xfeea37e9 in mbox_sync_do (sync_ctx=0x8047794, flags=0) at ../../../../src/lib/array.h:158 mbox_hdr = (struct mbox_index_header *) 0x80b4a20 mail_ctx = {sync_ctx = 0x8047794, mail = {uid = 0, idx_seq = 0, keywords = {arr = { buffer = 0x0, element_size = 0}, v = 0x0, v_modifiable = 0x0}, flags = 32 ' ', uid_broken = 0, expunged = 0, pseudo = 0, status_broken = 0, xstatus_broken = 0, from_offset = 564, body_size = 0, offset = 564, space = 0}, seq = 2, hdr_offset = 564, body_offset = 564, header_first_change = 4294967295, header_last_change = 0, header = 0x809c758, hdr_md5_sum = "Ô\035\214Ù\217\000²\004é\200\t\230ìøB~", content_length = 18446744073709551615, hdr_pos = {4294967295, 4294967295, 4294967295, 4294967295, 4294967295}, parsed_uid = 0, last_uid_updated_value = 0, last_uid_value_start_pos = 0, have_eoh = 0, need_rewrite = 0, seen_imapbase = 0, updated = 0, recent = 0, dirty = 0, imapbase_rewrite = 0, imapbase_updated = 0} st = (const struct stat *) 0x80c0060 i = 0 ret = 52 partial = 1 #10 0xfeea4363 in mbox_sync_int (mbox=0x80b4868, flags=0, lock_id=0x8047988) at ../../../../src/lib/array.h:158 index_sync_ctx = (struct mail_index_sync_ctx *) 0x80b62e0 sync_view = (struct mail_index_view *) 0x80b6320 trans = (struct mail_index_transaction *) 0x80c5890 sync_ctx = {mbox = 0x80b4868, flags = 0, input = 0x80c0180, file_input = 0x80c0040, write_fd = 12, orig_mtime = 1414582020, orig_atime = 1414582021, orig_size = 3843471, last_stat = { st_dev = 47513605, st_pad1 = {0, 0, 0}, st_ino = 17869, st_mode = 33152, st_nlink = 1, st_uid = 22671, st_gid = 4601, st_rdev = 4294967295, st_pad2 = {0, 0}, st_size = 3843471, st_atim = {tv_sec = 1414582021, tv_nsec = 353242309}, st_mtim = {tv_sec = 1414582020, tv_nsec = 0}, st_ctim = {tv_sec = 1414582021, tv_nsec = 352990278}, st_blksize = 131072, st_blocks = 7693, st_fstype = "zfs", '\0' <repeats 12 times>, st_pad4 = {0, 0, 0, 0, 0, 0, 0, 0}}, index_sync_ctx = 0x80b62e0, sync_view = 0x80b6320, t = 0x80c5890, reset_hdr = { major_version = 0 '\0', minor_version = 0 '\0', base_header_size = 0, header_size = 0, record_size = 0, compat_flags = 0 '\0', unused = "\000\000", indexid = 0, flags = 0, uid_validity = 0, next_uid = 0, messages_count = 0, unused_old_recent_messages_count = 0, seen_messages_count = 0, deleted_messages_count = 0, first_recent_uid = 0, first_unseen_uid_lowwater = 0, first_deleted_uid_lowwater = 0, log_file_seq = 0, log_file_tail_offset = 0, log_file_head_offset = 0, unused_old_sync_size = 0, unused_old_sync_stamp = 0, day_stamp = 0, day_first_uid = {0, 0, 0, 0, 0, 0, 0, 0}}, hdr = 0x80c5b68, header = 0x809c758, from_line = 0x809c6b8, base_uid_validity = 1353318516, base_uid_last = 144140, base_uid_last_offset = 250, mails = {arr = {buffer = 0x809c640, element_size = 52}, v = 0x809c640, v_modifiable = 0x809c640}, sync_changes = 0x80ab998, mail_keyword_pool = 0x80bf220, saved_keywords_pool = 0x80bcbe0, prev_msg_uid = 0, next_uid = 144141, idx_next_uid = 144141, seq = 2, idx_seq = 1, need_space_seq = 0, last_nonrecent_uid = 0, expunged_space = 0, space_diff = 0, dest_first_mail = 0, first_mail_crlf_expunged = 0, keep_recent = 0, readonly = 0, delay_writes = 1, renumber_uids = 0, moved_offsets = 0, ext_modified = 0, index_reset = 0, errors = 0} sync_flags = MAIL_INDEX_SYNC_FLAG_DROP_RECENT ret = 1 changed = 1 delay_writes = true readonly = false __FUNCTION__ = "mbox_sync_int" #11 0xfeea45cd in mbox_sync (mbox=0x80b4868, flags=0) at ../../../../src/lib/array.h:158 lock_id = 47 ret = 134957840 __FUNCTION__ = "mbox_sync" #12 0xfeea477d in mbox_storage_sync_init (box=0x80b4868, flags=0) at ../../../../src/lib/array.h:158 mbox = (struct mbox_mailbox *) 0x80b4868 mbox_sync_flags = 0 ret = 0 #13 0xfeecb356 in mailbox_sync_init (box=0x80b4868, flags=0) at ../../src/lib/array.h:197 _data_stack_cur_id = 4 ctx = (struct mailbox_sync_context *) 0x0 #14 0x08074b45 in imap_sync_init (client=0x80b1850, box=0x80b4868, imap_flags=0, flags=0) at ../../src/lib/array.h:158 ctx = (struct imap_sync_context *) 0x80b61b0 __FUNCTION__ = "imap_sync_init" #15 0x0805eeda in idle_sync_now (box=0x80b4868, ctx=0x80b20e0) at cmd-idle.c:74 __FUNCTION__ = "idle_sync_now" #16 0x0805f463 in cmd_idle (cmd=0x80b2058) at cmd-idle.c:74 client = (struct client *) 0x80b1850 ctx = (struct cmd_idle_context *) 0x80b20e0 #17 0x0806aad1 in command_exec (cmd=0x80b2058) at imap-commands.c:101 hook = (const struct command_hook *) 0x809c078 ret = 8 #18 0x08069a41 in client_command_input (cmd=0x80b2058) at imap-client.c:130 client = (struct client *) 0x80b1850 command = (struct command *) 0x3 __FUNCTION__ = "client_command_input" #19 0x08069d26 in client_command_input (cmd=0x80b2058) at imap-client.c:130 client = (struct client *) 0x80b1850 command = (struct command *) 0x809d3cc __FUNCTION__ = "client_command_input" #20 0x08069e29 in client_handle_next_command (client=0x80b1850, remove_io_r=0x8047b9d) at imap-client.c:130 No locals. #21 0x08069ec6 in client_handle_input (client=0x80b1850) at imap-client.c:130 _data_stack_cur_id = 3 ret = 10 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #22 0x0806a04f in client_input (client=0x80b1850) at imap-client.c:130 cmd = (struct client_command_context *) 0x80aaa44 output = (struct ostream *) 0x80abea8 bytes = 10 __FUNCTION__ = "client_input" #23 0xfedbb235 in io_loop_call_io (io=0x80abf10) at ioloop.c:28 ioloop = (struct ioloop *) 0x809bfc0 t_id = 2 __FUNCTION__ = "io_loop_call_io" #24 0xfedbc994 in io_loop_handler_run_internal (ioloop=0x809bfc0) at ioloop-poll.c:96 ctx = (struct ioloop_handler_context *) 0x809cab0 pollfd = (struct pollfd *) 0x809e148 tv = {tv_sec = 1799, tv_usec = 999001} io = (struct io_file *) 0x80abf10 msecs = 1800000 ret = 0 call = true #25 0xfedbb3ce in io_loop_handler_run (ioloop=0x809bfc0) at ioloop.c:28 No locals. #26 0xfedbb325 in io_loop_run (ioloop=0x809bfc0) at ioloop.c:28 __FUNCTION__ = "io_loop_run" #27 0xfed41dc1 in master_service_run (service=0x809bf08, callback=0x807724a <client_connected>) at master-service.c:75 No locals. #28 0x080775e7 in main (argc=2, argv=0x809bd18) at main.c:72 set_roots = {0x8093080, 0x0} login_set = {auth_socket_path = 0x8093188 " 1\t\b", postlogin_socket_path = 0x80931c0 "", postlogin_timeout_secs = 60, callback = 0x8076fbe <login_client_connected>, failure_callback = 0x80771cd <login_client_failed>, request_auth_token = 1} service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = -1

5 8

v2.2.14 released
by Timo Sirainen 26 Nov '14

26 Nov '14

http://dovecot.org/releases/2.2/dovecot-2.2.14.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.14.tar.gz.sig Looks like it took a bit longer to make this release after RC1 than I thought. I'm now in San Jose area for a few months and still jumping between apartments and trying to adjust to the new timezone. Some of the more important fixes since RC1: - Fixed several race conditions with dovecot.index.cache handling that may have caused unnecessary "cache is corrupted" errors. - auth: If auth client listed userdb and disconnected before finishing, the auth worker process got stuck (and eventually all workers could get used up and requests would start failing). Some of the larger changes since v2.2.13: * lmtp: Delivered-To: header no longer contains <> around the email address. Other MDAs don't have it either. * "Out of disk space" errors are now treated as temporary errors (not the same as "Out of disk quota"). * replication plugin: Use replication only for users who have a non-empty mail_replica setting. + lmtp proxy: Log a line about each mail delivery. + Added login_source_ips setting. This can be used to set the source IP address round-robin from a pool of IPs (in case you run out of TCP ports). + Rawlog settings can use tcp:<host>:<port> as the path. + virtual plugin: Don't keep more than virtual_max_open_mailboxes (default 64) number of backend mailboxes open. + SSL/TLS compression can be disabled with ssl_options=no_compression + acl: Global ACL file now supports "quotes" around patterns. + Added last-login plugin to set user's last-login timestamp on login. + LDAP auth: Allow passdb credentials lookup also with auth_bind=yes - IMAP: MODSEQ was sent in FETCH reply even if CONDSTORE/QRESYNC wasn't enabled. This broke at least old Outlooks. - passdb static treated missing password field the same as an empty password field. - mdbox: Fixed potential infinite looping when scanning a broken mdbox file. - imap-login, pop3-login: Fixed potential crashes when client disconnected unexpectedly. - imap proxy: The connection was hanging in some usage patterns. This mainly affected older Outlooks. - lmtp proxy: The proxy sometimes delivered empty mails in error situations or potentially delivered truncated mails. - fts-lucene: If whitespace_chars was set, we may have ended up indexing some garbage words, growing the index size unnecessarily. - -c and -i parameters for dovecot/doveadm commands were ignored if the config socket was readable. - quota: Quota recalculation didn't include INBOX in some setups. - Mail headers were sometimes added to dovecot.index.cache in wrong order. The main problem this caused was with dsync+imapc incremental syncing when the second sync thought the local mailbox had changed. - doveadm backup didn't notice if emails were missing from the middle of the destination mailbox. Now it deletes and resyncs the mailbox.

7 8

High I/O wait with Dovecot POP3
by Frido Otten 20 Nov '14

20 Nov '14

Hello, In the past we've had 4 loadbalanced servers configured with courier pop3 and dovecot handling respectively POP3 and IMAP connections. These 4 servers have a shared NFS storage on a Netapp FAS3220 controller with SAS disks. This setup is handling 50 POP3 sessions per second without any problems. Now we're installing 6 new loadbalanced servers to do the same job on the same storage, but using Dovecot to handle the POP3 sessions instead of Courier. The IMAP sessions are already handled by those 6 new servers, but we're encountering high iowait issues when using POP3 on those servers. I've applied the hints from http://wiki2.dovecot.org/NFS but this had no improvement. The 6 new servers are quad core Intel(R) Xeon(R) CPU X3323 @ 2.50GHz, with 4 GB RAM connected with gigabit ethernet connection to the storage. The 4 old servers are single core (w/o HT) P4 2,8 GHz with 1 GB RAM. # dovecot --version 2.1.7 # /usr/sbin/dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686-bigmem i686 Debian 6.0.10 ext4 auth_cache_negative_ttl = 0 auth_cache_size = 8 k auth_mechanisms = plain digest-md5 cram-md5 login base_dir = /var/run/imap/ dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no login_greeting = Dovecot MX. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_fsync = always mail_location = maildir:/var/spool/mail/%1u/%1.1u/%u mail_nfs_index = yes mail_nfs_storage = yes maildir_stat_dirs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes namespace { inbox = yes location = prefix = separator = . type = private } namespace { list = children location = maildir:/var/spool/mail/%%1u/%%1.1u/%%u/:INDEX=/var/spool/mail/%1u/%1.1u/%u/shared/%%u prefix = shared.%%u. separator = . subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql-auth.conf driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl quota = maildir quota_rule2 = Trash:storage=+25M } protocols = imap service anvil { client_limit = 4614 } service auth { client_limit = 2560 executable = /usr/lib/dovecot/auth unix_listener auth-master { mode = 0600 user = exim } user = dovecot } service dict { unix_listener dict { group = exim mode = 0660 user = exim } } service imap-login { chroot = login client_limit = 2048 executable = /usr/lib/dovecot/imap-login inet_listener imap { address = *, :: port = 143 } inet_listener imaps { address = *, :: port = 993 } process_limit = 2048 process_min_avail = 3 service_count = 0 user = dovecot vsz_limit = 128 M } service imap { executable = /usr/lib/dovecot/imap process_limit = 2048 } service managesieve-login { chroot = login client_limit = 2048 process_limit = 2048 process_min_avail = 3 service_count = 0 user = dovecot vsz_limit = 128 M } service managesieve { process_limit = 2048 } service pop3-login { chroot = login client_limit = 512 inet_listener pop3 { address = 1.2.3.4 port = 110 } inet_listener pop3s { address = 1.2.3.4 port = 0 } process_limit = 512 process_min_avail = 3 service_count = 0 user = dovecot vsz_limit = 128 M } service pop3 { process_limit = 2560 } ssl_cert = </etc/ssl/certs/wildcardcertificate.crt ssl_key = </etc/ssl/private/wildcardcertificate.key syslog_facility = local7 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql-auth.conf driver = sql } protocol imap { mail_plugins = quota imap_quota acl imap_acl } protocol pop3 { pop3_uidl_format = %v.%u } protocol lda { auth_socket_path = /var/run/imap/auth-master info_log_path = /var/log/dovecot-deliver.log log_path = /var/log/dovecot-deliver.log mail_plugins = sieve quota postmaster_address = Mailer-Daemon(a)mx.domain.nl rejection_reason = Your message with subject "%s" was automatically rejected:%n%r sendmail_path = /usr/sbin/exim } Best regards, Frido

3 3