December 2013 - dovecot

[Dovecot] dsync replication errors
by Oli Schacher 09 Sep '15

09 Sep '15

Hi I'm trying to build a cluster of two servers with dsync replication (based on http://wiki2.dovecot.org/Replication) My test setup works fine for very simple tests, I can log in to both servers, copy a message to one of the servers and it successfully apperars in the other account. But, if I try to copy a large amount of messages at once to one of the accounts, my maillogs get flodded with errors(see below) and the mailboxes seem to get out of sync and messages are duplicated over and over again (I originally copied 100 messages and ended up with thousands in both mailboxes until I killed dovecot) I'd appreciate if someone could have a look at my config and tell me what I did wrong. dovecot.conf of both servers, they are identical except for the target ip in mail_replica: dovecot -n # 2.2.beta1 (070ca24e5846+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.19.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) disable_plaintext_auth = no mail_plugins = " notify replication" namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = remote:vmail@192.168.23.62 } protocols = pop3 imap service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener auth-master { group = vmail mode = 0660 user = vmail } user = root } service replicator { process_min_avail = 1 } ssl = no userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } Log on server1 after I copied 100 messages to an account on that server: Jan 31 10:41:04 doco1 dovecot: imap-login: Login: user=<user1>, method=PLAIN, rip=192.168.23.130, lip=192.168.23.61, mpid=1432, session=<OdjlbJLUmwDAqBeC> Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=72, file=1359625327.M621257P1432.doco1,S=2472,W=2547:2,) Jan 31 10:42:12 doco1 dovecot: dsync-local(user1): Error: Recent flags state corrupted for mailbox INBOX Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=73, file=1359625327.M740847P1432.doco1,S=2417,W=2492:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=74, file=1359625328.M206735P1432.doco1,S=2400,W=2474:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=75, file=1359625328.M668118P1432.doco1,S=2421,W=2496:2,) Jan 31 10:42:12 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=76, file=1359625329.M167578P1432.doco1,S=2480,W=2559:2,) Jan 31 10:42:13 doco1 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=77, file=1359625329.M520528P1432.doco1,S=2525,W=2604:2,) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132: 1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133: 1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134: 1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135: 1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136: 1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137: 1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139: 1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140: 1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141: 1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142: 1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143: 1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144) Jan 31 10:42:14 doco1 dovecot: doveadm(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144: 1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145) Jan 31 10:42:14 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Unexpected GUID mismatch for UID=85: 1359625327.M740847P1432.doco1,S=2417,W=2492 != 2e1b1ee97994566870c02910ea929091 Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=175) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=150) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=176) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M520528P1432.doco1,S=2525,W=2604 (UID=179) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=123) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625327.M740847P1432.doco1,S=2417,W=2492 (UID=147) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=174) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M668118P1432.doco1,S=2421,W=2496 (UID=149) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=173) Jan 31 10:42:15 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625328.M206735P1432.doco1,S=2400,W=2474 (UID=148) Jan 31 10:42:16 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Importing mailbox INBOX failed Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: read(vmail(a)192.168.23.62) failed: EOF Jan 31 10:42:16 doco1 dovecot: dsync-local(user1): Error: Remote command returned error 75 Jan 31 10:42:20 doco1 dovecot: doveadm: Error: dsync-remote(user1): Error: Mailbox INBOX: Remote didn't send mail GUID=1359625329.M167578P1432.doco1,S=2480,W=2559 (UID=230) [...] Log on server2: Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 132: 1359625329.M520528P1432.doco1,S=2525,W=2604 (uid 77 -> 133) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 133: 1359625327.M621257P1432.doco1,S=2472,W=2547 (uid 72 -> 134) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 134: 1359625327.M740847P1432.doco1,S=2417,W=2492 (uid 73 -> 135) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 135: 1359625328.M206735P1432.doco1,S=2400,W=2474 (uid 74 -> 136) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 136: 1359625328.M668118P1432.doco1,S=2421,W=2496 (uid 75 -> 137) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 137: 1359625329.M167578P1432.doco1,S=2480,W=2559 (uid 76 -> 138) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 139: 1359625329.M782065P1432.doco1,S=2461,W=2539 (uid 78 -> 140) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 140: 1359625329.M973834P1432.doco1,S=2523,W=2602 (uid 79 -> 141) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 141: 1359625330.M114922P1432.doco1,S=2546,W=2626 (uid 80 -> 142) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 142: 1359625330.M255922P1432.doco1,S=2467,W=2546 (uid 81 -> 143) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 143: 1359625330.M403257P1432.doco1,S=2534,W=2611 (uid 82 -> 144) Jan 31 10:42:14 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: /mailstore/user1/maildir/dovecot-uidlist: Duplicate file entry at line 144: 1359625330.M486295P1432.doco1,S=2451,W=2529 (uid 83 -> 145) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=84, file=1359625330.M608391P1432.doco1,S=2480,W=2559:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=85, file=1359625330.M813949P1432.doco1,S=2427,W=2502:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=86, file=1359625331.M217320P1432.doco1,S=2547,W=2625:2,) Jan 31 10:42:15 doco2 dovecot: doveadm: Error: dsync-remote(user1): Warning: Maildir /mailstore/user1/maildir: Expunged message reappeared, giving a new UID (old uid=87, file=1359625331.M688431P1432.doco1,S=2529,W=2609:2,) [...] Thanks Oli -- message transmitted on 100% recycled electrons

7 45

[Dovecot] Dovecot FTS using ElasticSearch
by Darvin Denmian 21 Jul '15

21 Jul '15

Hi, I would like to know if is possible to use ElasticSearch instead of Solr for FTS. Regards, Erick Vitor de Barros.

6 6

[Dovecot] imapc - different settings for different namespaces
by Timo Sirainen 20 Mar '15

20 Mar '15

Originally I was planning on allowing all kinds of mail settings inside namespace {}, including imapc_* settings. But that's a bit difficult to implement (although I think it will happen some day). So for now I was thinking: imapc foo { host = imap.foo.com master_user = foomaster password = foopass } imapc bar { host = imap.bar.com ... } namespace { prefix = foo/ location = imapc:foo } namespace { prefix = bar/ location = imapc:bar } So basically if the path after imapc isn't absolute (/path or ~/path), then treat the path as the imapc section name and use its settings. Anyone have any improvement suggestions? I was planning on adding this to v2.2.3. I'm not sure if the section names should be "imapc" or something else (e.g. imap_remote {}).

2 2

[Dovecot] Sieve vacation not working
by Phill Edwards 24 Feb '15

24 Feb '15

I've been trying all night to get vacation working in my .dovecot.sieve file. The way our email works is we have one account on dovecot which serves my mail and my wife's and we use procmail and sieve to put the emails in different folders depending on who it's for. I'm trying to set up a vacation reply on my wife's email address (aaaaaaaa@optus* below) so people can update to her new address. Here's my file with the email addresses masked: require ["fileinto", "vacation"]; if address :is "to" "xxxxxxxxxxxxxxx(a)ozemail.com.au" { fileinto "Phill_OzEmail_Folder"; stop; } elsif address :is "to" "yyyyyyyyyyyyyyyyy(a)optusnet.com.au" { fileinto "Phill_Optus_Folder"; stop; } elsif address :matches "to" "aaaaaaaa@optus*" { vacation "Please update my email address to bbbbbbbbb(a)exemail.com.au"; fileinto "Sarah_Optus_Folder"; stop; } else { keep; stop; } The problem I'm having is that no auto-replies are sent. I removed .dovecot.lda-dupes and that didn't help; so then I touched it and now it's a 0 byte file and that didn't help. There is no dovecot.sieve.err file and I can't see anything in /var/log/dovecot.log which gives any message about why this wouldn't work. The emails to aaaaaaa@optus* get filed into the right folder so I know the matching is working. I've also tried a reject recipe and that worked fine, so I know that it can send an email if it chooses to. So it's a complete black box mystery as to why it just won't send a reply. Regards, Phill

5 9

[Dovecot] Disk Encryption
by Simon Brereton 17 Oct '14

17 Oct '14

Hi As I understand it email headers need to be unencrypted (otherwise DKIM doesn't work). From the MUA to either Postfix, or Dovecot the connection is (or can/should be) secured with TLS/SSL. What I would like to know is if it is possible to encrypt the mailstore? Postfix is using Dovecot for delivery so it's only Dovecot that would need to encrypt/decrypt the mailstore. Is this possible? Is there a terrible reason to do it even if it is possible? I realise that from MTA to MTA there's no guarantee of encryption (and in fact it's very unlikely unless keys have been exchanged), but my primary goal is supplement the physical security of the mail store of mails we already have or have sent. Mostly just idle curiosity as to what has been done, or what could be done. What is worth doing is a separate thread entirely. Thanks. Simon

11 28

[Dovecot] Issues with VANISHED CHANGEDSINCE
by Michael J Rubinsky 09 Jun '14

09 Jun '14

I've been seeing the following wonky behavior with Dovecot. Currently this is with Dovecot 2.0.19, but I was also seeing it in earlier versions as well, including versions from the 1.x series. Some background, this is from Horde's ActiveSync library, when it is trying to determine what UIDs have recently been expunged. I can guarantee that when this command was sent, there was only one or two emails that were removed since the provided MODSEQ value. C: 8 UID FETCH 1:* UID (VANISHED CHANGEDSINCE 587742) S: * VANISHED (EARLIER) 1:119944,119947:123482,123484:128043,128045:130388,130390:130456,130458:132127,132129:135832,135834:136201,136203:136301,136303:136316,136318:138502,138504:140099,140101:141908,141910:142607,142609:142744,142746:142865,142867:143160,143164:143173,143176:143179,143181:143187,143189:143190,143192:143195,143197:143270,143272:143273,143275:143280,143282:143302,143305:143307,143309:143318,143320:143333,143335:143337,143339:143341,143343:143375,143377:143400,143402:143551,143553:143637,143640:143706,143708:144069,144071:144178,144180:144307,144309:144316,144318:144329,144331:144931,144933:144983,144985:145000,145002:145069,145071:145072,145074:145132,145136:145178,145180:145191,145193 Also, to verify it wasn't something screwy with my server, Michael Slusarz provided me with this from his server: <quote> There's definitely something wonky going on in the code. There's a certain tipping point of modseqs where the values become incorrect. For a mailbox with HIGHESTMODSEQ of 54004, I see this: a uid fetch 1:* UID (VANISHED CHANGEDSINCE 53881) * VANISHED (EARLIER) 1:37308,37310:40788,40791:41032,41034:41083 a OK Fetch completed. b uid fetch 1:* UID (VANISHED CHANGEDSINCE 53882) * VANISHED (EARLIER) 41029:41030,41047:41083 b OK Fetch completed. </quote> Thanks for any guidance. -- mike The Horde Project (www.horde.org) mrubinsk(a)horde.org

4 21

[Dovecot] panic with 2.2.10 and GETMETADATA
by Michael Cramer 16 Mar '14

16 Mar '14

i have in my configs the following lines: 10-mail.conf:mail_attribute_dict = file:/srv/vmail/dovecot/metadata/%d/%n/attributes 20-imap.conf:imap_metadata = yes when i test GETMETADATA with telnet i got a pannic in my error.log file and no attributes file in the given location is created root@i920:/etc/dovecot/conf.d# telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Dovecot ready. 01 LOGIN tester(a)domain.com ********* 01 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIA L CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY NOTIFY METADATA COMPRESS=DEFLATE QUOTA ACL RIGHTS=texk] Logged in 02 GETMETADATA "*" (/shared/test) Connection closed by foreign host. in error.log i have these lines: Dec 23 17:20:00 imap(tester(a)domain.com): Panic: file imap-client.c: line 634 (client_command_free): assertion failed: (client->output_cmd_lock == NULL) Dec 23 17:20:00 imap(tester(a)domain.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e900) [0x7fb053bea900] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e9de) [0x7fb053bea9de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fb053ba5bfe] -> dovecot/imap() [0x415c17] -> dovecot/imap() [0x415d8a] -> dovecot/imap() [0x415e15] -> dovecot/imap(client_handle_input+0x115) [0x416115] -> dovecot/imap(client_input+0x72) [0x4164a2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x27) [0x7fb053bfa8f7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xd7) [0x7fb053bfb687] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fb053bfa498] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fb053baadf3] -> dovecot/imap(main+0x2af) [0x40b06f] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fb0537e5de5] -> dovecot/imap() [0x40b1cc] gdb with bt full gives: #0 0x00007f174b6bdf77 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007f174b6c15e8 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007f174baad8f6 in default_fatal_finish (type=<optimized out>, status=status@entry=0) at failures.c:193 backtrace = 0x1801040 "/usr/lib/dovecot/libdovecot.so.0(+0x5e900) [0x7f174baad900] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e9de) [0x7f174baad9de] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f174ba68bfe] -> dovecot/im"... #3 0x00007f174baad9de in i_internal_fatal_handler (ctx=0x7fffe2002eb0, format=<optimized out>, args=<optimized out>) at failures.c:657 status = 0 #4 0x00007f174ba68bfe in i_panic (format=format@entry=0x423040 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:267 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffe2002fa0, reg_save_area = 0x7fffe2002ee0}} #5 0x0000000000415c17 in client_command_free (_cmd=_cmd@entry=0x7fffe2002fc8) at imap-client.c:634 cmd = 0x183a2d0 client = 0x18396f0 state = CLIENT_COMMAND_STATE_WAIT_INPUT __FUNCTION__ = "client_command_free" #6 0x0000000000415d8a in client_command_input (cmd=0x0) at imap-client.c:783 client = 0x18396f0 command = <optimized out> __FUNCTION__ = "client_command_input" #7 0x0000000000415e15 in client_command_input (cmd=0x183a2d0) at imap-client.c:841 client = 0x18396f0 command = <optimized out> __FUNCTION__ = "client_command_input" #8 0x0000000000416115 in client_handle_next_command (remove_io_r=<synthetischer Zeiger>, client=0x18396f0) at imap-client.c:879 No locals. #9 client_handle_input (client=client@entry=0x18396f0) at imap-client.c:891 _data_stack_cur_id = 3 remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #10 0x00000000004164a2 in client_input (client=0x18396f0) at imap-client.c:933 cmd = 0x181db70 output = 0x183a120 bytes = 35 __FUNCTION__ = "client_input" #11 0x00007f174babd8f7 in io_loop_call_io (io=0x183a1f0) at ioloop.c:388 ioloop = 0x1808730 t_id = 2 #12 0x00007f174babe687 in io_loop_handler_run (ioloop=ioloop@entry=0x1808730) at ioloop-epoll.c:220 ctx = 0x18093c0 list = 0x180adf0 io = <optimized out> tv = {tv_sec = 1799, tv_usec = 999100} events_count = <optimized out> msecs = <optimized out> ret = 1 i = 0 call = <optimized out> __FUNCTION__ = "io_loop_handler_run" #13 0x00007f174babd498 in io_loop_run (ioloop=0x1808730) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #14 0x00007f174ba6ddf3 in master_service_run (service=0x18085c0, callback=callback@entry=0x41eea0 <client_connected>) at master-service.c:566 No locals. #15 0x000000000040b06f in main (argc=1, argv=0x1808390) at main.c:400 set_roots = {0x426d00 <imap_setting_parser_info>, 0x0} login_set = {auth_socket_path = 0x1800048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41f250 <login_client_connected>, failure_callback = 0x41ef80 <login_client_failed>, request_auth_token = 1} service_flags = <optimized out> storage_service_flags = <optimized out> username = 0x0 c = <optimized out>

2 2

[Dovecot] Mbox corruption - Inbox beginning with 'FFrom' or 'FrFrom'
by Dimos Alevizos 08 Mar '14

08 Mar '14

Hello, we're having some problems with our dovecot setup. I've seen similar problems in the mailing list some years ago but alas wasn't able to find a solution. Our setup is as follows : An MX farm (postfix) sends mails via LMTP to a director farm (dovecot 2.1.12) which proxies pop3/imap/lmtp traffic to a dovecot farm (dovecot 2.1.16). All mailboxes and indexes are on NFS and all servers are Centos. The problem is that at times we see mailboxes (all of them are in mbox format) beginning with FFrom or FrFrom and of course dovecot says it's not a valid mbox file. If we manually remove the offending extra characters from the beginning of the file everything is working again, but often the same user will show the problem again tomorrow. However it's very rare and so far we haven't been able to reproduce the problem. After activating the mail_log plugin hoping it'll help us locate the problem we run across this case : The user (let's call him user(a)domain.gr) logs in via pop3 and deletes some mails but one of them fails: Jun 19 11:09:18 pop01 dovecot: pop3-login: Login: user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.40, mpid=1389, secured, session=<Ao+rWH3fHgBT660a> Jun 19 11:09:18 director5 dovecot: pop3-login: proxy(user(a)domain.gr): started proxying to 83.235.66.40:110: user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.65, session=<Ao+rWH3fHgBT660a> Jun 19 11:09:24 pop01 dovecot: pop3(user(a)domain.gr): expunge: box=INBOX, uid=20670, msgid=<9f5b1a20a9428ed31a9e7e42ce4411f08d0.20130612070424(a)mail84.us2.rsgsv.net>, size=36526 Jun 19 11:09:24 pop01 dovecot: pop3(user(a)domain.gr): expunge: box=INBOX, uid=20671, msgid=<b27d72dc5a47a3235a232494c6fe8aa0(a)newsletter.stockout.gr>, size=4637 Jun 19 11:09:24 pop01 dovecot: pop3(user(a)domain.gr): expunge: box=INBOX, uid=20672, msgid=<1113670779560.1111910791405.1470.2.32032003(a)scheduler.constantcontact.com>, size=38655 Jun 19 11:09:24 pop01 dovecot: pop3(user(a)domain.gr): expunge: box=INBOX, uid=20673, msgid=<83600a274c1407c5aa4c263e2592cb54(a)debop.gr>, size=1181688 Jun 19 11:09:24 pop01 dovecot: pop3(user(a)domain.gr): expunge: box=INBOX, uid=20674, msgid=<F2A59BE7-007A-4567-9BA5-803446C3D7F8(a)bernier-eliades.gr>, size=61116 Jun 19 11:09:26 pop01 dovecot: pop3(user(a)domain.gr): Error: Next message unexpectedly corrupted in mbox file /var/mail/K/Y/V/domain_gr_user_007 at 1 Jun 19 11:09:26 pop01 dovecot: pop3(user(a)domain.gr): Error: Cached message offset 1 is invalid for mbox file /var/mail/K/Y/V/domain_gr_user_007 Jun 19 11:09:27 pop01 dovecot: pop3(user(a)domain.gr): Disconnected: Logged out top=0/0, retr=2/1502843, del=122/359, size=137448432 Jun 19 11:09:27 pop01 dovecot: pop3(user(a)domain.gr): Warning: Our dotlock file /var/mail/K/Y/V/domain_gr_user_007.lock was deleted (locked 9 secs ago, touched 9 secs ago) Jun 19 11:09:27 pop01 dovecot: pop3(user(a)domain.gr): Error: file_dotlock_delete() failed with mbox file /var/mail/K/Y/V/domain_gr_user_007: No such file or directory Jun 19 11:09:27 director5 dovecot: pop3-login: proxy(user(a)domain.gr): disconnecting 83.235.173.26 (Disconnected by server): user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.65, session=<Ao+rWH3fHgBT660a> However he's still able to log in again : Jun 19 11:11:36 pop01 dovecot: pop3-login: Login: user=<user(a)domain.gr>, method=PLAIN, rip=85.74.231.10, lip=83.235.66.40, mpid=10094, secured, session=<QDLrYH3f7ABVSucK> Jun 19 11:11:36 director3 dovecot: pop3-login: proxy(user(a)domain.gr): started proxying to 83.235.66.40:110: user=<user(a)domain.gr>, method=PLAIN, rip=85.74.231.10, lip=83.235.66.63, session=<QDLrYH3f7ABVSucK> Jun 19 11:11:51 pop01 dovecot: pop3(user(a)domain.gr): Disconnected: Logged out top=0/0, retr=0/0, del=0/237, size=75036938 Jun 19 11:11:51 director3 dovecot: pop3-login: proxy(user(a)domain.gr): disconnecting 85.74.231.10 (Disconnected by server): user=<user(a)domain.gr>, method=PLAIN, rip=85.74.231.10, lip=83.235.66.63, session=<QDLrYH3f7ABVSucK> A few minutes later he receives another mail which is delivered with no problem : Jun 19 11:30:39 corvus amavis[546]: (00546-22) Passed CLEAN, [91.190.168.40] [91.190.168.40] <tk-14439(a)free.splio.com> -> <user(a)domain.gr>, Message-ID: <58175-bWFyaW5hQGJlcm5pZXItZWxpYWR lcy5ncg==(a)free.splio.com>, mail_id: J1P-0WpmQksJ, Hits: 1.412, size: 9025, queued_as: 02BA4AE00414, Tests: [DKIM_VALID=-0.1,DKIM_VERIFIED=-0.1,FREEMAIL_FROM=0.001,HTML_IMAGE_RATIO_04=0.61,INVALID_MSGID=1,URIBL_BLOCKED=0.001], autolearn=disabled, 1533 ms Jun 19 11:30:39 corvus smtp/smtpd[1283]: proxy-accept: END-OF-MESSAGE: 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as 02BA4AE00414; from=<tk-14439(a)free.splio.com> to=<marina(a)bernier-eliades.gr> proto=ESMTP helo=<poc-pbcs2.splio.com> Jun 19 11:30:39 pop01 dovecot: lmtp(4099, user(a)domain.gr) 77O4IHlGwVEDEAAAYg/qxw: msgid=58175-bWFyaW5hQGJlcm5pZXItZWxpYWRlcy5ncg==(a)free.splio.com: from=residence.suddenly(a)gmx.com size=9610 saved mail to INBOX Jun 19 11:30:39 corvus postfix/lmtp[1350]: 02BA4AE00414: to=<user(a)domain.gr>, relay=lmtp.otenet.gr[62.103.147.209]:24, delay=0.12, delays=0.01/0/0/0.11, dsn=2.0.0, status=sent (250 2.0.0 <user(a)domain.gr> 77O4IHlGwVEDEAAAYg/qxw Saved) And then RIGHT after the successful delivery, the mbox somehow ends up corrupted with an FFrom in the beginning : Jun 19 11:31:27 pop01 dovecot: pop3-login: Login: user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.40, mpid=14630, secured, session=<eBLjp33fUwBT660a> Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Error: Syncing INBOX failed: Mailbox isn't a valid mbox file Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Error: Couldn't init INBOX: Mailbox isn't a valid mbox file Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 Jun 19 11:31:27 director5 dovecot: pop3-login: proxy(user(a)domain.gr): Login failed to 83.235.66.40:110: Mailbox isn't a valid mbox file Jun 19 11:31:27 pop01 dovecot: pop3-login: Login: user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.40, mpid=14642, secured, session=<eBLjp33fUwBT660a> Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Error: Syncing INBOX failed: Mailbox isn't a valid mbox file Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Error: Couldn't init INBOX: Mailbox isn't a valid mbox file Jun 19 11:31:27 pop01 dovecot: pop3(user(a)domain.gr): Mailbox init failed top=0/0, retr=0/0, del=0/0, size=0 Jun 19 11:31:27 director5 dovecot: pop3-login: proxy(user(a)domain.gr): Login failed to 83.235.66.40:110: Mailbox isn't a valid mbox file Jun 19 11:31:27 director5 dovecot: pop3-login: Aborted login (proxy dest auth failed): user=<user(a)domain.gr>, method=PLAIN, rip=83.235.173.26, lip=83.235.66.65, session=<eBLjp33fUwBT660a Our config : # 2.1.16: /opt/dovecot-2.1.16/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.22.el5 x86_64 CentOS release 5.9 (Final) auth_cache_negative_ttl = 10 mins auth_cache_size = 5 M auth_cache_ttl = 10 mins auth_verbose = yes default_client_limit = 5000 default_process_limit = 500 deliver_log_format = msgid=%m: from=%f size=%p %$ dict { sqlquota = mysql:/opt/dovecot/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_uid = 20 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = ready login_trusted_networks = 83.235.66.0/24 mail_access_groups = mail disk root mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mbox_lock_timeout = 3 secs mbox_read_locks = dotlock fcntl mmap_disable = yes passdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size quota = dict:User quota::noenforcing:proxy::sqlquota sieve = ~/.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } postmaster_address = postmaster(a)otenet.gr quota_full_tempfail = yes service auth-worker { user = dovenull } service dict { unix_listener dict { group = mail mode = 0666 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { client_limit = 1 inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /opt/dovecot/etc/dovecot/quota-warning.sh user = dovecot } ssl = no userdb { args = /opt/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { mail_plugins = quota sieve quota } protocol lda { mail_plugins = quota sieve quota } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 100 mail_plugins = quota imap_quota quota notify mail_log } protocol pop3 { mail_max_userip_connections = 100 mail_plugins = quota quota mail_log notify pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_fast_size_lookups = yes pop3_lock_session = yes pop3_reuse_xuidl = yes pop3_uidl_format = %08Xu%08Xv } Thank you for your time. Dimos Alevizos

4 10

[Dovecot] tests with metadata module
by Mihai Badici 25 Feb '14

25 Feb '14

--Repost after subscription as list member --- I made some tests with metadata support in dovecot 2.2.10. Previously I had dovecot 2.2.4 with external metadata plugin who work well. I use dovecot with roundcube and roundcube plugins from kolab project . First, the feature activation was tricky and rather undocumented. I spent about half hour googling, after that I set: imap_metadata = yes and mail_attribute_dict = file:%h/shared_metadata I think you cold put those lines in the 20-imap.conf example file commented; that way will be easy for us to activate. Because I changed the location of shared_metadata, all folders was unannotated . I went in folder manager in roundcube and set folder "calendar" as calendar type and so on. That worked and i have: cat shared_metadata: shared/c4268a04da44bb527c5b0000193e0904/vendor/kolab/folder-type event shared/cd5792367fc9cd5196170000193e0904/vendor/kolab/folder-type event shared/33f55d2e8912195209290000193e0904/vendor/kolab/folder-type event shared/32f55d2e8912195209290000193e0904/vendor/kolab/folder-type contact However, it looks like the metadata is not properly read after. Calendar and Contacts folders don't appear in roundcube. I have this crash in dovecot's log: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x67d81) [0xb7625d81] -> /usr/lib/dovecot/libdovecot.so.0(+0x67e11) [0xb7625e11] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb75d6af9] -> dovecot/imap() [0x805d3b2] -> dovecot/imap() [0x805d5bc] -> dovecot/imap() [0x805d6c9] -> dovecot/imap(client_handle_input+0x56) [0x805d836] -> dovecot/imap(client_input+0x72) [0x805dcc2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x38) [0xb7638d98] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x10e) [0xb7639d6e] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0xb7638818] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x28) [0xb75dcb68] -> dovecot/imap(main+0x284) [0x8050f34] -> /lib/libc.so.6(__libc_start_main+0xf5) [0xb743f5a5] -> dovecot/imap() [0x80510a5] Dec 26 09:28:49 imap(mihai.badici): Fatal: master: service(imap): child 4919 killed with signal 6 (core dumps disabled) I logged imap sessions in roundcube and I have: A0003 GETMETADATA "*" (/private/vendor/kolab/folder-type /shared/vendor/kolab/folder-type) On a similar server with the "old" metadata plugin, I have here listed all folders with coresponding annotations. How I can debug further? This is a test server so I can stress it. -- Mihai Bădici http://mihai.badici.ro -----------------------------------------

3 6

[Dovecot] failed: Message has been copied too many times
by Angel L. Mateo 19 Feb '14

19 Feb '14

Hello, I'm running dovecot 2.1.16 in a ubuntu 12.04 server, with lazy_expunge, SiS and mdbox format. The problem I'm having is that the index for one the mailboxes of one of my users is growing too much. This is not the first time of this problem. In previous cases, is because a message is duplicated thousand of times (I haven't found any reason for this). In these cases, moving messages to other folder and moving again to this same folder fixes the problem. So I have tried this, and I have moved all messages from this folder to another one. But this time, after this, indexes directory is still using a lot of space. And the folder seems to have more serious problems: amateo_adm@myotis51:~$ sudo doveadm search -u vlo mailbox BORRADOS.INBOX.MNCS doveadm(vlo): Error: Syncing mailbox BORRADOS.INBOX.MNCS failed: Message has been copied too many times (59306 + -1) I have tried to delete the folder, with the same result: amateo_adm@myotis51:~$ sudo doveadm mailbox delete -u vlo BORRADOS.INBOX.MNCS doveadm(vlo): Error: Can't delete mailbox BORRADOS.INBOX.MNCS: Message has been copied too many times (59306 + -1) And to purge user's mailbox, but it didn't fix the problem. PS: If this could help, BORRADOS. is the namespace used by lazy_expunge plugin. -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337

4 9