From brian.brady1982 at gmail.com Sun Nov 1 07:27:13 2015 From: brian.brady1982 at gmail.com (brian) Date: Sun, 1 Nov 2015 18:27:13 +1100 Subject: Initialization Failed: In-Reply-To: References: <56345451.9040304@gmail.com> Message-ID: <5635BED1.1030408@gmail.com> Yup, that did it. For some unknown reason I thought it had it twice in the tutorial, but when I checked, it did not. :S I've taken it out and all seems to be working now. Thanks muchly. On 31/10/15 18:08, Christian Kivalo wrote: > > Hi, > >> Error: user brian at myactualdomain.com: Initialization failed: Namespace >> '': Unknown setting: /var/mail/vmail/myactualdomain.com/brian/mail > .... >> mail_location = maildir:~/Maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs > You have double "maildir:" keyword. > > Regards > Christian From ms at mike2k.de Sun Nov 1 17:03:36 2015 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 01 Nov 2015 18:03:36 +0100 Subject: fts_solr does not quote lucene's special characters Message-ID: <73b23752e767509df021ed9d5d012deb@www.mike2k.de> Hello, I am using Dovecot 2.2.19 with the solr backend for full text search and experience the following issue. When I issue a search that includes characters that are part of lucene's query syntax (e.g. the double quote character), dovecot does not escape the special characters and issues a request with invalid syntax to the solr server. For example, I search for foo"bar, then my IMAP client (roundcube) sends the following to dovecot: A0003 UID THREAD REFS US-ASCII ALL UNDELETED BODY "foo\"bar" Dovecot in turn uses the following request URI to query SOLR: "GET /solr/select?fl=uid,score&rows=113739&sort=uid+asc&q=body:%22foo%22bar%22&fq=%2Bbox:4d86808c56d44282b35cea8700dda537+%2Buser:%22myuser%22 HTTP/1.1" 400 994 Solr responds with error 400 and the following body: type Status report message null description The request sent by the client was syntactically incorrect. This is because the " character in foo"bar is not quoted in the solr request. When I adapt the request to quote the " character solr succeeds with the expected result: "GET /solr/select?fl=uid,score&rows=113739&sort=uid+asc&q=body:%22foo%5C%22bar%22&fq=%2Bbox:4d86808c56d44282b35cea8700dda537+%2Buser:%22myuser%22 HTTP/1.1" 200 535 A fix for this would be highly appreciated. Cheers, Michael From rdossant at redhat.com Mon Nov 2 13:28:43 2015 From: rdossant at redhat.com (Rafael dos Santos) Date: Mon, 2 Nov 2015 14:28:43 +0100 Subject: auth_request_get_var_expand test fails on big endian arches. Message-ID: Hi, when running the tests in a big endian machine, I get the following error: 0 / 1 tests failed test-auth-request-var-expand.c:99: Assert failed: strcmp(str_c(str), test_output) == 0 test-auth-request-var-expand.c:104: Assert failed: strcmp(str_c(str), test_output) == 0 auth request var expand short and long ............................... : FAILED test: random seed #1 was 1446803094 auth request var expand flags ........................................ : ok test-auth-request-var-expand.c:158: Assert failed: strcmp(str_c(str), test_output) == 0 auth request var expand long-only .................................... : FAILED auth request var expand usernames .................................... : ok auth request var expand funcs ........................................ : ok 2 / 5 tests failed Makefile:1257: recipe for target 'check-test' failed Looking into it I found that auth_request_get_var_expand_table ends up calling the net_ip2addr function. This function, in turn, uses ntohl to convert from network byte order to host byte order. In big endian machines, that means that the string output will be ------------- big endian --------------- +user at +domain1@+domain2 +user +domain1 at +domain2 +service 7.91.205.21 73.150.2.210 54321 +password +mech secured 21 210 valid instead of ---------- expected output --------- +user at +domain1@+domain2 +user +domain1 at +domain2 +service 21.205.91.7 210.2.150.73 54321 +password +mech secured 21 210 valid Let me know in case you need help testing on big endian arches. Att. -- Rafael Fonseca From lists at tigertech.com Mon Nov 2 20:12:38 2015 From: lists at tigertech.com (Robert L Mathews) Date: Mon, 2 Nov 2015 12:12:38 -0800 Subject: IMAP COPY creates invalid index data with zlib and mail_log plugins enabled but zlib_save off In-Reply-To: <5633EA00.2080603@tigertech.com> References: <5633EA00.2080603@tigertech.com> Message-ID: <5637C3B6.6020001@tigertech.com> On 10/30/15 3:06 PM, Robert L Mathews wrote: > I've noticed that maildir IMAP COPY commands can generate invalid > dovecot.index entries when all seven of the following are true: As a followup to my own post, I believe I've tracked this problem down to this code at lines 119-124 of /src/plugins/zlib/zlib-plugin.c: /* don't uncompress input when we are reading a mail that we're just in the middle of saving, and we didn't do the compression ourself. in such situation we're probably checking if the user-given input looks compressed */ if (_mail->saving && zuser->save_handler == NULL) return zmail->super.istream_opened(_mail, stream); When these lines are removed, the problem no longer happens. I'm guessing that an IMAP COPY with the seven circumstances I mentioned matches "we are reading a mail that we're just in the middle of saving, and we didn't do the compression ourself". But the mail apparently does need decompressing in this situation. It seems odd for the code to not simply decompress the mail in all cases. But I'm not sure what other code relies on this check, so I have no idea whether removing these lines might cause problems. -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From marcin at mejor.pl Tue Nov 3 10:49:51 2015 From: marcin at mejor.pl (=?UTF-8?Q?Marcin_Miros=c5=82aw?=) Date: Tue, 3 Nov 2015 11:49:51 +0100 Subject: [dovceot-2.2.19] doveadm fetch with fts_squat returns random results Message-ID: <5638914F.7070600@mejor.pl> Hi all! I've got problem with results returned by doveadm fetch. The are random and incorrect. I'll try show you: - trying to fetch emails with both: nonexistent header name and non existent value of header name: -- doveadm fetch -u user at domain uid HEADER X-Sasdasdsendasasdasdasdasaer XXtuxie |grep -c uid 0 - trying to fetch email with: non existent header name and existent value of header (value exists usually in header X-Sender) -- # doveadm fetch -u user at domain uid HEADER X-Sasdasdsendasasdasdasdasaer tuxie |grep -c uid 13743 second search: -- # doveadm fetch -u user at domain uid HEADER X-Sasdasdsendasasdasdasdasaer tuxie |grep -c uid 16639 another search with the same criteria: -- doveadm fetch -u user at domain uid HEADER X-Sasdasdsendasasdasdasdasaer tuxie |grep -c uid 20296 so the number of fetched emails is random and none of them contains such header: -- # doveadm fetch -u user at domain hdr HEADER X-Sasdasdsendasasdasdasdasaer tuxie |grep -c X-Sasdasdsendasasdasdasdasaer 0 - now I'm searching for email with existent header contained existent value: -- # doveadm fetch -u user at domain uid HEADER X-Sender tuxie |grep -c uid 50699 second search: -- # doveadm fetch -u user at domain uid HEADER X-Sender tuxie |grep -c uid 50375 It looks for me that searching is done only for value of header not for both header name with given value. Next I turn off fts_squat in config and now results are correct. If someone uses doevadm expunge with fts_squat should be carefull. I just tried fts_lucene - no problem. Regards, Marcin From marcin at mejor.pl Tue Nov 3 11:48:10 2015 From: marcin at mejor.pl (=?UTF-8?Q?Marcin_Miros=c5=82aw?=) Date: Tue, 3 Nov 2015 12:48:10 +0100 Subject: [dovceot-2.2.19] doveadm fts expand constantly gives Segfault Message-ID: <56389EFA.9080302@mejor.pl> Hi! When I issue: doveadm fts expand -A body xxx I'm always getting Segfault. I can't find descrption of this command in manual, is it possible to add some information to manual? # gdb -q doveadm Reading symbols from doveadm...Reading symbols from /usr/lib64/debug//usr/bin/doveadm.debug...done. done. (gdb) set args fts expand -A body XXX (gdb) run Starting program: /usr/bin/doveadm fts expand -A body XXX Warning: couldn't activate thread debugging using libthread_db: Cannot find new threads: generic error warning: File "/lib64/libthread_db-1.0.so" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load". To enable execution of this file add add-auto-load-safe-path /lib64/libthread_db-1.0.so line to your configuration file "/root/.gdbinit". To completely disable this security protection add set auto-load safe-path / line to your configuration file "/root/.gdbinit". For more information about this security protection see the "Auto-loading safe path" section in the GDB manual. E.g., run from the shell: info "(gdb)Auto-loading safe path" warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available. Program received signal SIGSEGV, Segmentation fault. 0x00007ffff65a61af in fts_user_get_search_tokenizer (user=) at fts-user.c:248 248 fts-user.c: No such file or directory. (gdb) bt #0 0x00007ffff65a61af in fts_user_get_search_tokenizer (user=) at fts-user.c:248 #1 0x00007ffff65a2f8a in fts_search_arg_expand (backend=0x555555802f10, argp=0x7fffffffdf10, pool=0x5555557e56e0) at fts-search-args.c:108 #2 fts_search_args_expand_tree (backend=backend at entry=0x555555802f10, pool=0x5555557e56e0, argp=argp at entry=0x7fffffffdf10) at fts-search-args.c:171 #3 0x00007ffff65a31a5 in fts_search_args_expand (backend=backend at entry=0x555555802f10, args=0x5555557e5700) at fts-search-args.c:199 #4 0x00007ffff2118b2e in cmd_fts_expand_run (ctx=0x5555557e2f10, user=) at doveadm-fts.c:127 #5 0x000055555557a945 in doveadm_mail_next_user (ctx=0x5555557e2f10, input=, error_r=0x7fffffffe028) at doveadm-mail.c:384 #6 0x000055555557bf53 in doveadm_mail_all_users (wildcard_user=0x0, argv=, ctx=0x5555557e2f10) at doveadm-mail.c:452 #7 doveadm_mail_cmd (argv=, argc=, cmd=) at doveadm-mail.c:621 #8 doveadm_mail_try_run (cmd_name=, argc=4, argv=0x5555557ce3a0) at doveadm-mail.c:710 #9 0x000055555556ba6a in main (argc=5, argv=0x5555557ce398) at doveadm.c:345 (gdb) mail_plugins = acl fts fts_lucene listescape quota notify mail_log stats zlib protocol lmtp { mail_plugins = acl fts fts_lucene listescape quota notify mail_log stats zlib sieve } protocol lda { mail_plugins = quota sieve } protocol imap { mail_max_userip_connections = 25 mail_plugins = acl fts fts_lucene listescape quota notify mail_log stats zlib antispam imap_acl imap_quota imap_stats imap_zlib last_login } protocol pop3 { mail_plugins = acl fts fts_lucene listescape quota notify mail_log stats zlib last_login } Thanks, Marcin From johnbclements at gmail.com Tue Nov 3 19:49:21 2015 From: johnbclements at gmail.com (John Clements) Date: Tue, 3 Nov 2015 11:49:21 -0800 Subject: dovecot-lda can't create /var/mail dotlocks on debian Message-ID: I've been using dovecot+postfix happily for many years, and I'm now configuring it for a new machine. However, I'm running into an old problem again, and thinking that there must be a better solution. The problem is that dovecot-lda is unable to create dotlock files in the /var/mail directory. Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream version 2.2.13) OS: Debian Jessie Currently, my mail directory has these permissions: clements at desmond:~$ ls -ld /var/mail drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail clements at desmond:~$ ls -l /var/mail total 8 -rw------- 1 clements mail 1382 Nov 2 21:59 clements -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon I've added mail_privileged_group = mail to allow creation of the dotlock files. When I configure postfix to deliver using dovecot-lda, I get logs that look like this: Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: setegid(privileged) failed: Operation not permitted Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to INBOX: BUG: Unknown internal error In order to isolate the error, I took postfix out of the equation, and called dovecot-lda directly: clements at desmond:/tmp$ cat bogusmail From: clements at XXXDOMAIN.org To: granitemon at localhost Date: November 3 2015 Subject: graaaah this is the body clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < bogusmail BUG: Unknown internal error clements at desmond:/tmp$ In response to this, mail.log now contains this similar error: Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save failed to INBOX: BUG: Unknown internal error Nov 3 11:34:57 desmond dovecot: lda(clements): Error: setegid(privileged) failed: Operation not permitted I've tried a number of "random internet search" solutions, including - changing perms on mail files from 660 to 600 - enabling 'mail_access_groups=mail' in 10-mail.conf - adding individual users to the mail group. I guess I'm pretty confident that if dovecot is writing "BUG: Unknown internal error" in the logs, that this is is actually a bug in dovecot. OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to see if a relevant-looking bug had been fixed, but nothing jumped out at me. OBresearch: searching the dovecot mailing list, I found one *extremely* relevant thread called "Re: [Dovecot] started with dovecot sieve ", but there didn't seem to be a solution contained in the thread. Final note: this doesn't appear to be confined to debian jessie: I took a look at my existing installation, and I see that in fact I just went ahead and made /var/mail world-writeable, which seems... sub-optimal. I'm sure I could do that here, too, but I'd certainly rather not. Thanks in advance, and let me know if I've left out relevant crucial information. Best, John Clements From larryrtx at gmail.com Tue Nov 3 19:52:03 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 3 Nov 2015 13:52:03 -0600 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: what is the full permissions of /var/mail? ls -lda /var/mail On Tue, Nov 3, 2015 at 1:49 PM, John Clements wrote: > I've been using dovecot+postfix happily for many years, and I'm now > configuring it for a new machine. However, I'm running into an old problem > again, and thinking that there must be a better solution. > > The problem is that dovecot-lda is unable to create dotlock files in the > /var/mail directory. > > Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream version > 2.2.13) > OS: Debian Jessie > > Currently, my mail directory has these permissions: > > clements at desmond:~$ ls -ld /var/mail > drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail > clements at desmond:~$ ls -l /var/mail > total 8 > -rw------- 1 clements mail 1382 Nov 2 21:59 clements > -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon > > I've added > mail_privileged_group = mail > to allow creation of the dotlock files. > > When I configure postfix to deliver using dovecot-lda, I get logs that look > like this: > > Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: > setegid(privileged) failed: Operation not permitted > Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< > 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to INBOX: > BUG: Unknown internal error > > In order to isolate the error, I took postfix out of the equation, and > called dovecot-lda directly: > > clements at desmond:/tmp$ cat bogusmail > From: clements at XXXDOMAIN.org > To: granitemon at localhost > Date: November 3 2015 > Subject: graaaah > > this is the body > clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < > bogusmail > BUG: Unknown internal error > clements at desmond:/tmp$ > > In response to this, mail.log now contains this similar error: > > Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save > failed to INBOX: BUG: Unknown internal error > Nov 3 11:34:57 desmond dovecot: lda(clements): Error: setegid(privileged) > failed: Operation not permitted > > > I've tried a number of "random internet search" solutions, including > - changing perms on mail files from 660 to 600 > - enabling 'mail_access_groups=mail' in 10-mail.conf > - adding individual users to the mail group. > > I guess I'm pretty confident that if dovecot is writing "BUG: Unknown > internal error" in the logs, that this is is actually a bug in dovecot. > > OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to see if > a relevant-looking bug had been fixed, but nothing jumped out at me. > OBresearch: searching the dovecot mailing list, I found one *extremely* > relevant thread called "Re: [Dovecot] started with dovecot sieve > ", but > there didn't seem to be a solution contained in the thread. > > Final note: this doesn't appear to be confined to debian jessie: I took a > look at my existing installation, and I see that in fact I just went ahead > and made /var/mail world-writeable, which seems... sub-optimal. I'm sure I > could do that here, too, but I'd certainly rather not. > > Thanks in advance, and let me know if I've left out relevant crucial > information. > > Best, > > John Clements > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 From johnbclements at gmail.com Tue Nov 3 20:10:47 2015 From: johnbclements at gmail.com (John Clements) Date: Tue, 3 Nov 2015 12:10:47 -0800 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: clements at desmond:/var/log$ ls -lda /var/mail drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail Best, John Clements On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman wrote: > what is the full permissions of /var/mail? > > > ls -lda /var/mail > > On Tue, Nov 3, 2015 at 1:49 PM, John Clements > wrote: > >> I've been using dovecot+postfix happily for many years, and I'm now >> configuring it for a new machine. However, I'm running into an old problem >> again, and thinking that there must be a better solution. >> >> The problem is that dovecot-lda is unable to create dotlock files in the >> /var/mail directory. >> >> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream version >> 2.2.13) >> OS: Debian Jessie >> >> Currently, my mail directory has these permissions: >> >> clements at desmond:~$ ls -ld /var/mail >> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >> clements at desmond:~$ ls -l /var/mail >> total 8 >> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >> >> I've added >> mail_privileged_group = mail >> to allow creation of the dotlock files. >> >> When I configure postfix to deliver using dovecot-lda, I get logs that >> look >> like this: >> >> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >> setegid(privileged) failed: Operation not permitted >> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to INBOX: >> BUG: Unknown internal error >> >> In order to isolate the error, I took postfix out of the equation, and >> called dovecot-lda directly: >> >> clements at desmond:/tmp$ cat bogusmail >> From: clements at XXXDOMAIN.org >> To: granitemon at localhost >> Date: November 3 2015 >> Subject: graaaah >> >> this is the body >> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >> bogusmail >> BUG: Unknown internal error >> clements at desmond:/tmp$ >> >> In response to this, mail.log now contains this similar error: >> >> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save >> failed to INBOX: BUG: Unknown internal error >> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: setegid(privileged) >> failed: Operation not permitted >> >> >> I've tried a number of "random internet search" solutions, including >> - changing perms on mail files from 660 to 600 >> - enabling 'mail_access_groups=mail' in 10-mail.conf >> - adding individual users to the mail group. >> >> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >> internal error" in the logs, that this is is actually a bug in dovecot. >> >> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to see if >> a relevant-looking bug had been fixed, but nothing jumped out at me. >> OBresearch: searching the dovecot mailing list, I found one *extremely* >> relevant thread called "Re: [Dovecot] started with dovecot sieve >> ", but >> there didn't seem to be a solution contained in the thread. >> >> Final note: this doesn't appear to be confined to debian jessie: I took a >> look at my existing installation, and I see that in fact I just went ahead >> and made /var/mail world-writeable, which seems... sub-optimal. I'm sure I >> could do that here, too, but I'd certainly rather not. >> >> Thanks in advance, and let me know if I've left out relevant crucial >> information. >> >> Best, >> >> John Clements >> > > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com > US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 > From larryrtx at gmail.com Tue Nov 3 20:12:30 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 3 Nov 2015 14:12:30 -0600 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: Hrm. if you turn up the debug on lda, do you get any more of a clue? Those permissions look fine to me. On Tue, Nov 3, 2015 at 2:10 PM, John Clements wrote: > clements at desmond:/var/log$ ls -lda /var/mail > drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail > > > Best, > > John Clements > > On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman > wrote: > >> what is the full permissions of /var/mail? >> >> >> ls -lda /var/mail >> >> On Tue, Nov 3, 2015 at 1:49 PM, John Clements >> wrote: >> >>> I've been using dovecot+postfix happily for many years, and I'm now >>> configuring it for a new machine. However, I'm running into an old >>> problem >>> again, and thinking that there must be a better solution. >>> >>> The problem is that dovecot-lda is unable to create dotlock files in the >>> /var/mail directory. >>> >>> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream >>> version >>> 2.2.13) >>> OS: Debian Jessie >>> >>> Currently, my mail directory has these permissions: >>> >>> clements at desmond:~$ ls -ld /var/mail >>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>> clements at desmond:~$ ls -l /var/mail >>> total 8 >>> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >>> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >>> >>> I've added >>> mail_privileged_group = mail >>> to allow creation of the dotlock files. >>> >>> When I configure postfix to deliver using dovecot-lda, I get logs that >>> look >>> like this: >>> >>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >>> setegid(privileged) failed: Operation not permitted >>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >>> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to INBOX: >>> BUG: Unknown internal error >>> >>> In order to isolate the error, I took postfix out of the equation, and >>> called dovecot-lda directly: >>> >>> clements at desmond:/tmp$ cat bogusmail >>> From: clements at XXXDOMAIN.org >>> To: granitemon at localhost >>> Date: November 3 2015 >>> Subject: graaaah >>> >>> this is the body >>> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >>> bogusmail >>> BUG: Unknown internal error >>> clements at desmond:/tmp$ >>> >>> In response to this, mail.log now contains this similar error: >>> >>> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save >>> failed to INBOX: BUG: Unknown internal error >>> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: >>> setegid(privileged) >>> failed: Operation not permitted >>> >>> >>> I've tried a number of "random internet search" solutions, including >>> - changing perms on mail files from 660 to 600 >>> - enabling 'mail_access_groups=mail' in 10-mail.conf >>> - adding individual users to the mail group. >>> >>> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >>> internal error" in the logs, that this is is actually a bug in dovecot. >>> >>> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to see >>> if >>> a relevant-looking bug had been fixed, but nothing jumped out at me. >>> OBresearch: searching the dovecot mailing list, I found one *extremely* >>> relevant thread called "Re: [Dovecot] started with dovecot sieve >>> ", but >>> there didn't seem to be a solution contained in the thread. >>> >>> Final note: this doesn't appear to be confined to debian jessie: I took a >>> look at my existing installation, and I see that in fact I just went >>> ahead >>> and made /var/mail world-writeable, which seems... sub-optimal. I'm sure >>> I >>> could do that here, too, but I'd certainly rather not. >>> >>> Thanks in advance, and let me know if I've left out relevant crucial >>> information. >>> >>> Best, >>> >>> John Clements >>> >> >> >> >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >> > > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 From larryrtx at gmail.com Tue Nov 3 20:13:50 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 3 Nov 2015 14:13:50 -0600 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: and, are you SURE that dovecot-lda has mail in it's group list when it is executing? On Tue, Nov 3, 2015 at 2:12 PM, Larry Rosenman wrote: > Hrm. if you turn up the debug on lda, do you get any more of a clue? > > Those permissions look fine to me. > > > On Tue, Nov 3, 2015 at 2:10 PM, John Clements > wrote: > >> clements at desmond:/var/log$ ls -lda /var/mail >> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >> >> >> Best, >> >> John Clements >> >> On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman >> wrote: >> >>> what is the full permissions of /var/mail? >>> >>> >>> ls -lda /var/mail >>> >>> On Tue, Nov 3, 2015 at 1:49 PM, John Clements >>> wrote: >>> >>>> I've been using dovecot+postfix happily for many years, and I'm now >>>> configuring it for a new machine. However, I'm running into an old >>>> problem >>>> again, and thinking that there must be a better solution. >>>> >>>> The problem is that dovecot-lda is unable to create dotlock files in the >>>> /var/mail directory. >>>> >>>> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream >>>> version >>>> 2.2.13) >>>> OS: Debian Jessie >>>> >>>> Currently, my mail directory has these permissions: >>>> >>>> clements at desmond:~$ ls -ld /var/mail >>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>> clements at desmond:~$ ls -l /var/mail >>>> total 8 >>>> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >>>> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >>>> >>>> I've added >>>> mail_privileged_group = mail >>>> to allow creation of the dotlock files. >>>> >>>> When I configure postfix to deliver using dovecot-lda, I get logs that >>>> look >>>> like this: >>>> >>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >>>> setegid(privileged) failed: Operation not permitted >>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >>>> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to INBOX: >>>> BUG: Unknown internal error >>>> >>>> In order to isolate the error, I took postfix out of the equation, and >>>> called dovecot-lda directly: >>>> >>>> clements at desmond:/tmp$ cat bogusmail >>>> From: clements at XXXDOMAIN.org >>>> To: granitemon at localhost >>>> Date: November 3 2015 >>>> Subject: graaaah >>>> >>>> this is the body >>>> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >>>> bogusmail >>>> BUG: Unknown internal error >>>> clements at desmond:/tmp$ >>>> >>>> In response to this, mail.log now contains this similar error: >>>> >>>> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save >>>> failed to INBOX: BUG: Unknown internal error >>>> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: >>>> setegid(privileged) >>>> failed: Operation not permitted >>>> >>>> >>>> I've tried a number of "random internet search" solutions, including >>>> - changing perms on mail files from 660 to 600 >>>> - enabling 'mail_access_groups=mail' in 10-mail.conf >>>> - adding individual users to the mail group. >>>> >>>> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >>>> internal error" in the logs, that this is is actually a bug in dovecot. >>>> >>>> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to see >>>> if >>>> a relevant-looking bug had been fixed, but nothing jumped out at me. >>>> OBresearch: searching the dovecot mailing list, I found one *extremely* >>>> relevant thread called "Re: [Dovecot] started with dovecot sieve >>>> ", but >>>> there didn't seem to be a solution contained in the thread. >>>> >>>> Final note: this doesn't appear to be confined to debian jessie: I took >>>> a >>>> look at my existing installation, and I see that in fact I just went >>>> ahead >>>> and made /var/mail world-writeable, which seems... sub-optimal. I'm >>>> sure I >>>> could do that here, too, but I'd certainly rather not. >>>> >>>> Thanks in advance, and let me know if I've left out relevant crucial >>>> information. >>>> >>>> Best, >>>> >>>> John Clements >>>> >>> >>> >>> >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>> >> >> > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com > US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 From johnbclements at gmail.com Tue Nov 3 20:40:42 2015 From: johnbclements at gmail.com (John Clements) Date: Tue, 3 Nov 2015 12:40:42 -0800 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: Well, first, here are the logs I generated: Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Effective uid=1003, gid=1003, home=/home/granitemon Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/mail/granitemon Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: fs: root=/home/granitemon/mail, index=, indexpvt=, control=, inbox=/var/mail/granitemon, alt= Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: userdb lookup skipped, username taken from USER environment Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Destination address: granitemon at desmond.brinckerhoff.org (source: user at hostname) Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: setegid(privileged) failed: Operation not permitted Nov 3 12:23:05 desmond dovecot: lda(granitemon): msgid=< 20151103202305.88BE05FF39 at desmond.brinckerhoff.org>: save failed to INBOX: BUG: Unknown internal error Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: setegid(privileged) failed: Operation not permitted Nov 3 12:23:05 desmond postfix/local[26490]: 88BE05FF39: to=, relay=local, delay=0.04, delays=0.01/0.01/0/0.02, dsn=4.3.0, status=deferred (temporary failure) At this point... well, I don't understand why dovecot signals an "Unknown internal error," but I think I understand that even if I *do* get this working, I'm pretty much throwing in the towel, because since postfix invokes the lda as the user receiving the mail, then this only works if all users receiving mail are in the mail group, which means any of them can mess up any other's mbox. So, it looks like even if this bug is fixed, I'm left with two obvious choices: - make /var/mail writeable by all users that receive mail, or - use LMTP instead. Many thanks for your help, John Clements On Tue, Nov 3, 2015 at 12:13 PM, Larry Rosenman wrote: > and, are you SURE that dovecot-lda has mail in it's group list when it is > executing? > > On Tue, Nov 3, 2015 at 2:12 PM, Larry Rosenman wrote: > >> Hrm. if you turn up the debug on lda, do you get any more of a clue? >> >> Those permissions look fine to me. >> >> >> On Tue, Nov 3, 2015 at 2:10 PM, John Clements >> wrote: >> >>> clements at desmond:/var/log$ ls -lda /var/mail >>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>> >>> >>> Best, >>> >>> John Clements >>> >>> On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman >>> wrote: >>> >>>> what is the full permissions of /var/mail? >>>> >>>> >>>> ls -lda /var/mail >>>> >>>> On Tue, Nov 3, 2015 at 1:49 PM, John Clements >>>> wrote: >>>> >>>>> I've been using dovecot+postfix happily for many years, and I'm now >>>>> configuring it for a new machine. However, I'm running into an old >>>>> problem >>>>> again, and thinking that there must be a better solution. >>>>> >>>>> The problem is that dovecot-lda is unable to create dotlock files in >>>>> the >>>>> /var/mail directory. >>>>> >>>>> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream >>>>> version >>>>> 2.2.13) >>>>> OS: Debian Jessie >>>>> >>>>> Currently, my mail directory has these permissions: >>>>> >>>>> clements at desmond:~$ ls -ld /var/mail >>>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>>> clements at desmond:~$ ls -l /var/mail >>>>> total 8 >>>>> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >>>>> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >>>>> >>>>> I've added >>>>> mail_privileged_group = mail >>>>> to allow creation of the dotlock files. >>>>> >>>>> When I configure postfix to deliver using dovecot-lda, I get logs that >>>>> look >>>>> like this: >>>>> >>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >>>>> setegid(privileged) failed: Operation not permitted >>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >>>>> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to >>>>> INBOX: >>>>> BUG: Unknown internal error >>>>> >>>>> In order to isolate the error, I took postfix out of the equation, and >>>>> called dovecot-lda directly: >>>>> >>>>> clements at desmond:/tmp$ cat bogusmail >>>>> From: clements at XXXDOMAIN.org >>>>> To: granitemon at localhost >>>>> Date: November 3 2015 >>>>> Subject: graaaah >>>>> >>>>> this is the body >>>>> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >>>>> bogusmail >>>>> BUG: Unknown internal error >>>>> clements at desmond:/tmp$ >>>>> >>>>> In response to this, mail.log now contains this similar error: >>>>> >>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: save >>>>> failed to INBOX: BUG: Unknown internal error >>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: >>>>> setegid(privileged) >>>>> failed: Operation not permitted >>>>> >>>>> >>>>> I've tried a number of "random internet search" solutions, including >>>>> - changing perms on mail files from 660 to 600 >>>>> - enabling 'mail_access_groups=mail' in 10-mail.conf >>>>> - adding individual users to the mail group. >>>>> >>>>> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >>>>> internal error" in the logs, that this is is actually a bug in dovecot. >>>>> >>>>> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to >>>>> see if >>>>> a relevant-looking bug had been fixed, but nothing jumped out at me. >>>>> OBresearch: searching the dovecot mailing list, I found one *extremely* >>>>> relevant thread called "Re: [Dovecot] started with dovecot sieve >>>>> ", but >>>>> there didn't seem to be a solution contained in the thread. >>>>> >>>>> Final note: this doesn't appear to be confined to debian jessie: I >>>>> took a >>>>> look at my existing installation, and I see that in fact I just went >>>>> ahead >>>>> and made /var/mail world-writeable, which seems... sub-optimal. I'm >>>>> sure I >>>>> could do that here, too, but I'd certainly rather not. >>>>> >>>>> Thanks in advance, and let me know if I've left out relevant crucial >>>>> information. >>>>> >>>>> Best, >>>>> >>>>> John Clements >>>>> >>>> >>>> >>>> >>>> -- >>>> Larry Rosenman http://www.lerctr.org/~ler >>>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>>> >>> >>> >> >> >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >> > > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com > US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 > From larryrtx at gmail.com Tue Nov 3 20:44:06 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Tue, 3 Nov 2015 14:44:06 -0600 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Effective uid=1003, gid=1003, home=/home/granitemon Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: setegid(privileged) failed: Operation not permitted so it's running as the normal user, and NOT with the mail group. I'm using exim with LMTP. LMTP is NOT a bad thing, and might make your life easier. It does allow you to add sieve scripting if you want to via pigeonhole. Sorry, I'm at a loss, as I do NOT run postfix. I'm not sure what it needs to invoke dovecot-lda with gid mail in the group list. On Tue, Nov 3, 2015 at 2:40 PM, John Clements wrote: > Well, first, here are the logs I generated: > > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Effective > uid=1003, gid=1003, home=/home/granitemon > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Namespace inbox: > type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, > subscriptions=yes location=mbox:~/mail:INBOX=/var/mail/granitemon > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: fs: > root=/home/granitemon/mail, index=, indexpvt=, control=, > inbox=/var/mail/granitemon, alt= > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: userdb lookup > skipped, username taken from USER environment > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: none: root=, > index=, indexpvt=, control=, inbox=, alt= > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Destination > address: granitemon at desmond.brinckerhoff.org (source: user at hostname) > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: > setegid(privileged) failed: Operation not permitted > Nov 3 12:23:05 desmond dovecot: lda(granitemon): msgid=< > 20151103202305.88BE05FF39 at desmond.brinckerhoff.org>: save failed to > INBOX: BUG: Unknown internal error > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: > setegid(privileged) failed: Operation not permitted > Nov 3 12:23:05 desmond postfix/local[26490]: 88BE05FF39: > to=, relay=local, delay=0.04, > delays=0.01/0.01/0/0.02, dsn=4.3.0, status=deferred (temporary failure) > > At this point... well, I don't understand why dovecot signals an "Unknown > internal error," but I think I understand that even if I *do* get this > working, I'm pretty much throwing in the towel, because since postfix > invokes the lda as the user receiving the mail, then this only works if all > users receiving mail are in the mail group, which means any of them can > mess up any other's mbox. > > So, it looks like even if this bug is fixed, I'm left with two obvious > choices: > - make /var/mail writeable by all users that receive mail, or > - use LMTP instead. > > Many thanks for your help, > > John Clements > > > On Tue, Nov 3, 2015 at 12:13 PM, Larry Rosenman > wrote: > >> and, are you SURE that dovecot-lda has mail in it's group list when it is >> executing? >> >> On Tue, Nov 3, 2015 at 2:12 PM, Larry Rosenman >> wrote: >> >>> Hrm. if you turn up the debug on lda, do you get any more of a clue? >>> >>> Those permissions look fine to me. >>> >>> >>> On Tue, Nov 3, 2015 at 2:10 PM, John Clements >>> wrote: >>> >>>> clements at desmond:/var/log$ ls -lda /var/mail >>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>> >>>> >>>> Best, >>>> >>>> John Clements >>>> >>>> On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman >>>> wrote: >>>> >>>>> what is the full permissions of /var/mail? >>>>> >>>>> >>>>> ls -lda /var/mail >>>>> >>>>> On Tue, Nov 3, 2015 at 1:49 PM, John Clements >>>> > wrote: >>>>> >>>>>> I've been using dovecot+postfix happily for many years, and I'm now >>>>>> configuring it for a new machine. However, I'm running into an old >>>>>> problem >>>>>> again, and thinking that there must be a better solution. >>>>>> >>>>>> The problem is that dovecot-lda is unable to create dotlock files in >>>>>> the >>>>>> /var/mail directory. >>>>>> >>>>>> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream >>>>>> version >>>>>> 2.2.13) >>>>>> OS: Debian Jessie >>>>>> >>>>>> Currently, my mail directory has these permissions: >>>>>> >>>>>> clements at desmond:~$ ls -ld /var/mail >>>>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>>>> clements at desmond:~$ ls -l /var/mail >>>>>> total 8 >>>>>> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >>>>>> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >>>>>> >>>>>> I've added >>>>>> mail_privileged_group = mail >>>>>> to allow creation of the dotlock files. >>>>>> >>>>>> When I configure postfix to deliver using dovecot-lda, I get logs >>>>>> that look >>>>>> like this: >>>>>> >>>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >>>>>> setegid(privileged) failed: Operation not permitted >>>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >>>>>> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to >>>>>> INBOX: >>>>>> BUG: Unknown internal error >>>>>> >>>>>> In order to isolate the error, I took postfix out of the equation, and >>>>>> called dovecot-lda directly: >>>>>> >>>>>> clements at desmond:/tmp$ cat bogusmail >>>>>> From: clements at XXXDOMAIN.org >>>>>> To: granitemon at localhost >>>>>> Date: November 3 2015 >>>>>> Subject: graaaah >>>>>> >>>>>> this is the body >>>>>> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >>>>>> bogusmail >>>>>> BUG: Unknown internal error >>>>>> clements at desmond:/tmp$ >>>>>> >>>>>> In response to this, mail.log now contains this similar error: >>>>>> >>>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: >>>>>> save >>>>>> failed to INBOX: BUG: Unknown internal error >>>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: >>>>>> setegid(privileged) >>>>>> failed: Operation not permitted >>>>>> >>>>>> >>>>>> I've tried a number of "random internet search" solutions, including >>>>>> - changing perms on mail files from 660 to 600 >>>>>> - enabling 'mail_access_groups=mail' in 10-mail.conf >>>>>> - adding individual users to the mail group. >>>>>> >>>>>> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >>>>>> internal error" in the logs, that this is is actually a bug in >>>>>> dovecot. >>>>>> >>>>>> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to >>>>>> see if >>>>>> a relevant-looking bug had been fixed, but nothing jumped out at me. >>>>>> OBresearch: searching the dovecot mailing list, I found one >>>>>> *extremely* >>>>>> relevant thread called "Re: [Dovecot] started with dovecot sieve >>>>>> ", >>>>>> but >>>>>> there didn't seem to be a solution contained in the thread. >>>>>> >>>>>> Final note: this doesn't appear to be confined to debian jessie: I >>>>>> took a >>>>>> look at my existing installation, and I see that in fact I just went >>>>>> ahead >>>>>> and made /var/mail world-writeable, which seems... sub-optimal. I'm >>>>>> sure I >>>>>> could do that here, too, but I'd certainly rather not. >>>>>> >>>>>> Thanks in advance, and let me know if I've left out relevant crucial >>>>>> information. >>>>>> >>>>>> Best, >>>>>> >>>>>> John Clements >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Larry Rosenman http://www.lerctr.org/~ler >>>>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>>>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>>>> >>>> >>>> >>> >>> >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>> >> >> >> >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >> > > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 From johnbclements at gmail.com Tue Nov 3 20:46:35 2015 From: johnbclements at gmail.com (John Clements) Date: Tue, 3 Nov 2015 12:46:35 -0800 Subject: dovecot-lda can't create /var/mail dotlocks on debian In-Reply-To: References: Message-ID: Yep, yep, yep, consider this solved. I believe I understand the issues involved, now. Many thanks for your help! John Clements On Tue, Nov 3, 2015 at 12:44 PM, Larry Rosenman wrote: > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Effective > uid=1003, gid=1003, home=/home/granitemon > > > Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: > setegid(privileged) failed: Operation not permitted > > > so it's running as the normal user, and NOT with the mail group. > > I'm using exim with LMTP. LMTP is NOT a bad thing, and might make your > life easier. It does allow you to add sieve scripting if you want to via > pigeonhole. > > Sorry, I'm at a loss, as I do NOT run postfix. I'm not sure what it needs > to invoke dovecot-lda with gid mail in the group list. > > > > On Tue, Nov 3, 2015 at 2:40 PM, John Clements > wrote: > >> Well, first, here are the logs I generated: >> >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Effective >> uid=1003, gid=1003, home=/home/granitemon >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Namespace inbox: >> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, >> subscriptions=yes location=mbox:~/mail:INBOX=/var/mail/granitemon >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: fs: >> root=/home/granitemon/mail, index=, indexpvt=, control=, >> inbox=/var/mail/granitemon, alt= >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: userdb lookup >> skipped, username taken from USER environment >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: none: root=, >> index=, indexpvt=, control=, inbox=, alt= >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Debug: Destination >> address: granitemon at desmond.brinckerhoff.org (source: user at hostname) >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: >> setegid(privileged) failed: Operation not permitted >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): msgid=< >> 20151103202305.88BE05FF39 at desmond.brinckerhoff.org>: save failed to >> INBOX: BUG: Unknown internal error >> Nov 3 12:23:05 desmond dovecot: lda(granitemon): Error: >> setegid(privileged) failed: Operation not permitted >> Nov 3 12:23:05 desmond postfix/local[26490]: 88BE05FF39: >> to=, relay=local, delay=0.04, >> delays=0.01/0.01/0/0.02, dsn=4.3.0, status=deferred (temporary failure) >> >> At this point... well, I don't understand why dovecot signals an "Unknown >> internal error," but I think I understand that even if I *do* get this >> working, I'm pretty much throwing in the towel, because since postfix >> invokes the lda as the user receiving the mail, then this only works if all >> users receiving mail are in the mail group, which means any of them can >> mess up any other's mbox. >> >> So, it looks like even if this bug is fixed, I'm left with two obvious >> choices: >> - make /var/mail writeable by all users that receive mail, or >> - use LMTP instead. >> >> Many thanks for your help, >> >> John Clements >> >> >> On Tue, Nov 3, 2015 at 12:13 PM, Larry Rosenman >> wrote: >> >>> and, are you SURE that dovecot-lda has mail in it's group list when it >>> is executing? >>> >>> On Tue, Nov 3, 2015 at 2:12 PM, Larry Rosenman >>> wrote: >>> >>>> Hrm. if you turn up the debug on lda, do you get any more of a clue? >>>> >>>> Those permissions look fine to me. >>>> >>>> >>>> On Tue, Nov 3, 2015 at 2:10 PM, John Clements >>>> wrote: >>>> >>>>> clements at desmond:/var/log$ ls -lda /var/mail >>>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>>> >>>>> >>>>> Best, >>>>> >>>>> John Clements >>>>> >>>>> On Tue, Nov 3, 2015 at 11:52 AM, Larry Rosenman >>>>> wrote: >>>>> >>>>>> what is the full permissions of /var/mail? >>>>>> >>>>>> >>>>>> ls -lda /var/mail >>>>>> >>>>>> On Tue, Nov 3, 2015 at 1:49 PM, John Clements < >>>>>> johnbclements at gmail.com> wrote: >>>>>> >>>>>>> I've been using dovecot+postfix happily for many years, and I'm now >>>>>>> configuring it for a new machine. However, I'm running into an old >>>>>>> problem >>>>>>> again, and thinking that there must be a better solution. >>>>>>> >>>>>>> The problem is that dovecot-lda is unable to create dotlock files in >>>>>>> the >>>>>>> /var/mail directory. >>>>>>> >>>>>>> Dovecot version: 1:2.2.13-12~deb8u1 (I'm guessing this is upstream >>>>>>> version >>>>>>> 2.2.13) >>>>>>> OS: Debian Jessie >>>>>>> >>>>>>> Currently, my mail directory has these permissions: >>>>>>> >>>>>>> clements at desmond:~$ ls -ld /var/mail >>>>>>> drwxrwsr-x 2 root mail 4096 Nov 2 22:07 /var/mail >>>>>>> clements at desmond:~$ ls -l /var/mail >>>>>>> total 8 >>>>>>> -rw------- 1 clements mail 1382 Nov 2 21:59 clements >>>>>>> -rw------- 1 granitemon mail 530 Nov 2 22:07 granitemon >>>>>>> >>>>>>> I've added >>>>>>> mail_privileged_group = mail >>>>>>> to allow creation of the dotlock files. >>>>>>> >>>>>>> When I configure postfix to deliver using dovecot-lda, I get logs >>>>>>> that look >>>>>>> like this: >>>>>>> >>>>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): Error: >>>>>>> setegid(privileged) failed: Operation not permitted >>>>>>> Nov 3 11:12:20 desmond dovecot: lda(granitemon): msgid=< >>>>>>> 20151103181306.A4B5B5FF32 at desmond.XXXDOMAIN.org>: save failed to >>>>>>> INBOX: >>>>>>> BUG: Unknown internal error >>>>>>> >>>>>>> In order to isolate the error, I took postfix out of the equation, >>>>>>> and >>>>>>> called dovecot-lda directly: >>>>>>> >>>>>>> clements at desmond:/tmp$ cat bogusmail >>>>>>> From: clements at XXXDOMAIN.org >>>>>>> To: granitemon at localhost >>>>>>> Date: November 3 2015 >>>>>>> Subject: graaaah >>>>>>> >>>>>>> this is the body >>>>>>> clements at desmond:/tmp$ /usr/lib/dovecot/dovecot-lda -e -d clements < >>>>>>> bogusmail >>>>>>> BUG: Unknown internal error >>>>>>> clements at desmond:/tmp$ >>>>>>> >>>>>>> In response to this, mail.log now contains this similar error: >>>>>>> >>>>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): msgid=unspecified: >>>>>>> save >>>>>>> failed to INBOX: BUG: Unknown internal error >>>>>>> Nov 3 11:34:57 desmond dovecot: lda(clements): Error: >>>>>>> setegid(privileged) >>>>>>> failed: Operation not permitted >>>>>>> >>>>>>> >>>>>>> I've tried a number of "random internet search" solutions, including >>>>>>> - changing perms on mail files from 660 to 600 >>>>>>> - enabling 'mail_access_groups=mail' in 10-mail.conf >>>>>>> - adding individual users to the mail group. >>>>>>> >>>>>>> I guess I'm pretty confident that if dovecot is writing "BUG: Unknown >>>>>>> internal error" in the logs, that this is is actually a bug in >>>>>>> dovecot. >>>>>>> >>>>>>> OBresearch: I read through the release notes of 2.2.14 -- 2.2.19 to >>>>>>> see if >>>>>>> a relevant-looking bug had been fixed, but nothing jumped out at me. >>>>>>> OBresearch: searching the dovecot mailing list, I found one >>>>>>> *extremely* >>>>>>> relevant thread called "Re: [Dovecot] started with dovecot sieve >>>>>>> ", >>>>>>> but >>>>>>> there didn't seem to be a solution contained in the thread. >>>>>>> >>>>>>> Final note: this doesn't appear to be confined to debian jessie: I >>>>>>> took a >>>>>>> look at my existing installation, and I see that in fact I just went >>>>>>> ahead >>>>>>> and made /var/mail world-writeable, which seems... sub-optimal. I'm >>>>>>> sure I >>>>>>> could do that here, too, but I'd certainly rather not. >>>>>>> >>>>>>> Thanks in advance, and let me know if I've left out relevant crucial >>>>>>> information. >>>>>>> >>>>>>> Best, >>>>>>> >>>>>>> John Clements >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Larry Rosenman http://www.lerctr.org/~ler >>>>>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>>>>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Larry Rosenman http://www.lerctr.org/~ler >>>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>>> >>> >>> >>> >>> -- >>> Larry Rosenman http://www.lerctr.org/~ler >>> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com >>> US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 >>> >> >> > > > -- > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com > US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961 > From jtam.home at gmail.com Wed Nov 4 01:24:37 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 3 Nov 2015 17:24:37 -0800 (PST) Subject: ssl-params: slow startup (patch for consideration) Message-ID: Based on the recent found weaknesses in DH key exchange, http://weakdh.org/ I increased ssl_dh_parameters_length to 2048 bits, and found waited for 5+ minutes for dovecot to come back online after a restart. Unless you got a fast machine, the initialization of DH parameters can exceed your patience. Regeneration may not be a problem (if ssl_parameters_regenerate=0 or if Dovecot uses old parameters until regeneration finishes), but for cold starts, the server can be tied up for a few minutes creating DH parameters while clients queue up. I ran "openssl dhparam 2048" and got wildly varying run times of 1m45s, 11m56s, 0.4s, 2m19s, 3h23s. Most of the time was spent testing primality of candidate p *and* (p-1)/2 -- so called "safe prime". If you're unlucky, this can take a long time. However, it appears "safe" primes are not what they're cracked up to be -- they offer some guarantees, but are not safer than non-safe primes. Creating DH parameters without requiring primality of (p-1)/2 (i.e. what "openssl dhparam -dsaparam" does) results in much lower run-time bounds. I cribbed some OpenSSL code to create this (untested) patch. -------------------------------------------------------------------------------- --- iostream-openssl-params.c~ Tue Nov 3 16:08:38 2015 +++ iostream-openssl-params.c Tue Nov 3 15:43:39 2015 @@ -6,5 +6,2 @@ -/* 2 or 5. Haven't seen their difference explained anywhere, but 2 is the - default.. */ -#define DH_GENERATOR 2 @@ -14,2 +11,3 @@ DH *dh; + DSA *dsa; unsigned char *p; @@ -17,3 +15,13 @@ - dh = DH_generate_parameters(bitsize, DH_GENERATOR, NULL, NULL); + dsa = DSA_generate_parameters(bitsize, NULL, 0, NULL, NULL, NULL, NULL); + if (dsa == NULL) { + *error_r = t_strdup_printf( + "DSA_generate_parameters(bits=%d) failed: %s", + bitsize, openssl_iostream_error()); + return -1; + } + + dh = DSA_dup_DH(dsa); + DSA_free(dsa); + if (dh == NULL) { @@ -20,4 +28,4 @@ *error_r = t_strdup_printf( - "DH_generate_parameters(bits=%d, gen=%d) failed: %s", - bitsize, DH_GENERATOR, openssl_iostream_error()); + "DSA_dup_DH() failed: %s", + openssl_iostream_error()); return -1; -------------------------------------------------------------------------------- The other way to prevent long startup times is to pre-compute the DH parameter using "openssl dhparam -out dh2048.pem 2048". I can contribute a patch to do this (read file, convert it into ssl-parameters.dat, then set/behave like ssl_parameters_regenerate=0), but I couldn't figure out the best place to do this. ssl_params_if_unchanged()? Joseph Tam From sca at andreasschulze.de Wed Nov 4 13:22:07 2015 From: sca at andreasschulze.de (A. Schulze) Date: Wed, 04 Nov 2015 14:22:07 +0100 Subject: ssl-params: slow startup (patch for consideration) In-Reply-To: Message-ID: <20151104142207.Horde.dGMfUJONcvkZIA_VxDQ9kkF@andreasschulze.de> Joseph Tam: > Based on the recent found weaknesses in DH key exchange, > > http://weakdh.org/ > > I increased ssl_dh_parameters_length to 2048 bits, and found waited > for 5+ minutes for dovecot to come back online after a restart. > Unless you got a fast machine, the initialization of DH parameters can > exceed your patience. > > Regeneration may not be a problem (if ssl_parameters_regenerate=0 or if > Dovecot uses old parameters until regeneration finishes), but for cold > starts, the server can be tied up for a few minutes creating DH parameters > while clients queue up. > > I ran "openssl dhparam 2048" and got wildly varying run times of 1m45s, > 11m56s, 0.4s, 2m19s, 3h23s. Most of the time was spent testing primality > of candidate p *and* (p-1)/2 -- so called "safe prime". If you're > unlucky, this can take a long time. > > However, it appears "safe" primes are not what they're cracked up to be > -- they offer some guarantees, but are not safer than non-safe primes. > Creating DH parameters without requiring primality of (p-1)/2 (i.e. what > "openssl dhparam -dsaparam" does) results in much lower run-time bounds. > > I cribbed some OpenSSL code to create this (untested) patch. precomputing ssl-params is also possible without patching but it's a little bit tricky 1. generate a temporary minimal config cat < /tmp/ssl-params.conf ssl_dh_parameters_length = 4096 state_dir = /tmp/ EOF 2. calculation rm -f /tmp/ssl-parameters.dat* nice -n 19 /path/to/ssl-params -c /tmp/ssl-params.conf 3. move the result to your running dovecot mv /tmp/ssl-parameters.dat /path/to/ssl-parameters.dat doveadm reload rm /tmp/ssl-params.conf Long version in german: https://andreasschulze.de/dovecot/ssl-params Andreas From those.lists at gmail.com Wed Nov 4 19:47:46 2015 From: those.lists at gmail.com (Forrest) Date: Wed, 4 Nov 2015 14:47:46 -0500 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) Message-ID: <563A60E2.4050404@gmail.com> I have been attempting to use the cyrus2dovecot script, to no avail. I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. inboxes=the "myaccount" that was copied over /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ --cyrus-sub /home/varimap/user/%h/%u.sub \ --dovecot-inbox /home/myaccount/dovecot/Maildir \ myaccount the log output complains of: cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. All directory paths are correct. Thanks. From jjhoffart at icloud.com Wed Nov 4 20:18:56 2015 From: jjhoffart at icloud.com (jjhoffart) Date: Wed, 04 Nov 2015 20:18:56 +0000 (GMT) Subject: ldap and proxy/proxy_maybe Message-ID: Hello, ???I am trying to setup dovecot director and am receiving a lot of "Aborted login (proxy dest auth failed)" and "error proxying loops to itself" messages in the dovecot log. I don't fully understand the dovecot-ldap.conf file config? could someone please let me know if the following is correct. If so what could be causing the errors mentioned above, if not what needs to be corrected? hosts ? ? ? ? ? = 127.0.0.1:389 ldap_version? ? = 3 auth_bind ? ? ? = yes dn? ? ? ? ? ? ? = cn=vmail,dc=something,dc=somewhere dnpass? ? ? ? ? = hi89Dfnptc55jRWG9oLQJUfWOH0CgV base? ? ? ? ? ? = o=domains,dc=campus,dc=local scope ? ? ? ? ? = subtree deref ? ? ? ? ? = never # Below two are required by command 'doveadm mailbox ...' iterate_attrs ? = mail=user iterate_filter? = (&(objectClass=mailUser)(accountStatus=active)(enabledService=mail)) user_filter ? ? = (&(objectClass=mailUser)(accountStatus=active)(enabledService=mail)(enabledService=%Ls%Lc)(|(mail=%u)(&(enabledService=shadowaddress)(shadowAddress=%u)))) user_attrs? ? ? = mail=user,homeDirectory=home,=mail=maildir:~/Maildir/,mailQuota=quota_rule=*:bytes=%$ # Used for dn lookup pass_filter ? ? = (&(objectClass=mailUser)(accountStatus=active)(enabledService=mail)(enabledService=%Ls%Lc)(|(mail=%u)(&(enabledService=shadowaddress)(shadowAddress=%u)))) pass_attrs? ? ? = mail=user,userPassword=password,someAttribute=proxy,hostName=host default_pass_scheme = CRYPT From raubvogel at gmail.com Wed Nov 4 22:19:22 2015 From: raubvogel at gmail.com (Mauricio Tavares) Date: Wed, 4 Nov 2015 17:19:22 -0500 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <563A60E2.4050404@gmail.com> References: <563A60E2.4050404@gmail.com> Message-ID: On Nov 4, 2015 2:48 PM, "Forrest" wrote: > > I have been attempting to use the cyrus2dovecot script, to no avail. > > I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. > > In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. > > > inboxes=the "myaccount" that was copied over > > /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ > --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ > --cyrus-sub /home/varimap/user/%h/%u.sub \ > --dovecot-inbox /home/myaccount/dovecot/Maildir \ > myaccount > > > > the log output complains of: > > cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. > > and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. > > All directory paths are correct. > > > Thanks. I've never used cyrrus, so I wonder if what the script does is create new index and rename the files containing email. Which leads to filename convention. What I do know is that the times I had issues with indexes, I just deleted the index file and let Dovecot create a new one. From jtam.home at gmail.com Wed Nov 4 22:45:55 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 4 Nov 2015 14:45:55 -0800 (PST) Subject: ssl-params: slow startup (patch for consideration) In-Reply-To: References: Message-ID: A. Schulze writes: > precomputing ssl-params is also possible without patching but it's a > little bit tricky > ... > Long version in german: https://andreasschulze.de/dovecot/ssl-params Nice. (You should probably point out to ensure ssl_parameters_regenerate is zero, otherwise all this work will get wiped out!) Joseph Tam From terry at dop.com Thu Nov 5 07:18:56 2015 From: terry at dop.com (terry at dop.com) Date: Wed, 4 Nov 2015 23:18:56 -0800 Subject: Fw: new message Message-ID: <0000fdf80cd2$6e019fe9$c77450e4$@dop.com> Hello! New message, please read terry at dop.com From terry at dop.com Thu Nov 5 07:22:04 2015 From: terry at dop.com (Doug Wierenga) Date: Wed, 4 Nov 2015 23:22:04 -0800 Subject: Fw: new message Message-ID: <00007386532c$38db4871$32e5fc57$@dop.com> Hello! New message, please read Doug Wierenga From dovecot at randy.pensive.org Thu Nov 5 08:21:49 2015 From: dovecot at randy.pensive.org (Randall Gellens) Date: Thu, 5 Nov 2015 00:21:49 -0800 Subject: dsync imapc authentication mechanisms Message-ID: I'm trying to use dsync to mirror mail onto a remote IMAP server. The remote server is generating an error that authentication via plain text is not permitted. Does dsync imapc support any non-clear-text authentication mechanisms? The remote server supports: AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=NTLM -- Randall Gellens Opinions are personal; facts are suspect; I speak for myself only -------------- Randomly selected tag: --------------- At a dinner party one should eat wisely but not too well, and talk well but not too wisely. --W. Somerset Maugham From sca at andreasschulze.de Thu Nov 5 09:34:18 2015 From: sca at andreasschulze.de (A. Schulze) Date: Thu, 05 Nov 2015 10:34:18 +0100 Subject: ssl-params: slow startup (patch for consideration) In-Reply-To: References: Message-ID: <20151105103418.Horde.msQBiqkwGxYYt18cWuqSqrl@andreasschulze.de> Joseph Tam: > A. Schulze writes: > >> precomputing ssl-params is also possible without patching but it's a >> little bit tricky >> ... >> Long version in german: https://andreasschulze.de/dovecot/ssl-params > > Nice. > > (You should probably point out to ensure ssl_parameters_regenerate is > zero, otherwise all this work will get wiped out!) as I read the current source ssl_parameters_regenerate is not used anymore At least in 2.2.1[5...9] ssl_parameters_regenerate *has no effect anymore* For that reason I developed this workaround. Andreas From djk at tobit.co.uk Thu Nov 5 21:39:57 2015 From: djk at tobit.co.uk (djk) Date: Thu, 5 Nov 2015 21:39:57 +0000 Subject: replication delays Message-ID: <563BCCAD.7060201@tobit.co.uk> I have two dovecot instances that replicate to each other. Mail arrives into "post" and it is generally accessed via IMAP on "france". Changes (moves/expunge etc) on "france" seem to replicate back to "post" more or less immediately. But this seems to happen only for this user. (Some) other users' (e.g. me) mail generally replicates more or less immediately in both directions. Nov 5 17:40:55 post dovecot: lmtp(willem at xxx): xF9mK6eUO1YsFQAAdYyp2A: sieve: msgid=<20151105173923.14970.95605 at ssa-app-xxx>: stored mail into mailbox 'SSA' Nov 5 17:41:00 post dovecot: lmtp(willem at xxx): yV9mK6eUO1YsFQAAdYyp2A: sieve: msgid=<20151105173924.14970.91073 at ssa-app-xxx>: stored mail into mailbox 'SSA' Nov 5 20:39:09 france dovecot: dsync-server(willem): save: box=SSA, uid=3130, msgid=<20151105173923.14970.95605 at ssa-app-xxx>, size=9724 Nov 5 20:39:09 france dovecot: dsync-server(willem): save: box=SSA, uid=3131, msgid=<20151105173924.14970.91073 at ssa-app-xxx>, size=9197 Any suggestions as to where to start looking? As you can see I am using 2.2.18 with mdbox and a simple all vmail based system with not many users. I am not seeing any error messages in my mail logs that suggest there any delays happening. Although I have seen a few things like this happening from time to time a busier account: Nov 5 12:12:07 france dovecot: dsync-server(djk): Error: Couldn't lock /home/imap/djk/.dovecot-sync.lock: Timed out after 30 seconds Nov 5 12:12:13 france dovecot: dsync-server(djk at tobit.co.uk): Error: Couldn't lock /home/imap/djk/.dovecot-sync.lock: Timed out after 30 seconds Nov 5 12:16:49 france dovecot: dsync-server(djk): Error: Couldn't lock /home/imap/djk/.dovecot-sync.lock: Timed out after 30 seconds root at france:/var/log# dovecot -n # 2.2.18: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: Linux 3.13.0-66-generic x86_64 Ubuntu 14.04.3 LTS auth_verbose = yes doveadm_password = first_valid_uid = 2000 imapc_features = rfc822.size fetch-headers imapc_host = imapc_port = 993 imapc_ssl = imaps imapc_ssl_verify = no imapc_user = %n lda_mailbox_autocreate = yes login_greeting = IMAP ready. mail_attachment_dir = %h/attachments mail_attachment_min_size = 64 k mail_location = mdbox:%h:INBOX=%h/mail mail_plugins = " fts fts_lucene mail_log notify notify replication" mail_prefetch_count = 20 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mdbox_rotate_size = 10 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { fts = lucene fts_lucene = whitespace_chars=@. mail_log_events = delete undelete expunge copy save mailbox_delete mailbox_rename mailbox_create mail_log_fields = uid box msgid size mail_replica = tcps:post: sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster at tobit.co.uk protocols = " imap lmtp sieve sieve" service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener exim-auth { group = Debian-exim mode = 0660 user = Debian-exim } } service config { unix_listener config { mode = 0666 user = vmail } } service doveadm { inet_listener { port = ssl = yes } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = } unix_listener lmtp { mode = 0666 } user = vmail } service managesieve-login { inet_listener sieve { port = } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve { process_limit = 100 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = vmail } } ssl_cert = I?m almost embarrassed to submit the following patch. ==== cut here ==== --- dovecot-2.2.19/src/lib/bits.h.orig Thu Mar 19 08:42:32 2015 +++ dovecot-2.2.19/src/lib/bits.h Thu Nov 5 15:10:35 2015 @@ -10,7 +10,9 @@ #include #include -#include +#ifdef HAVE_STDINT_H +# include +#endif #define UINT64_SUM_OVERFLOWS(a, b) \ (a > (uint64_t)-1 - b) From teemu.huovila at dovecot.fi Fri Nov 6 08:59:44 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Fri, 6 Nov 2015 10:59:44 +0200 Subject: Patch for Dovecot 2.2.19 compilation failure In-Reply-To: References: Message-ID: <563C6C00.9040503@dovecot.fi> Contributions of all sizes are appreciated. Thank you. br, Teemu Huovila On 06.11.2015 00:21, Hammer, Adam wrote: > I?m almost embarrassed to submit the following patch. > > ==== cut here ==== > > --- dovecot-2.2.19/src/lib/bits.h.orig Thu Mar 19 08:42:32 2015 > +++ dovecot-2.2.19/src/lib/bits.h Thu Nov 5 15:10:35 2015 > @@ -10,7 +10,9 @@ > > #include > #include > -#include > +#ifdef HAVE_STDINT_H > +# include > +#endif > > #define UINT64_SUM_OVERFLOWS(a, b) \ > (a > (uint64_t)-1 - b) > > From a.meyer at nimmini.de Fri Nov 6 16:09:00 2015 From: a.meyer at nimmini.de (Andreas Meyer) Date: Fri, 6 Nov 2015 17:09:00 +0100 Subject: Problem with permissions of the logfile Message-ID: <20151106170900.29d1bc62@itxbox.bitcorner.intern> Hello! I want to have dovecot log into a separate file and set log_path = /var/log/dovecot instead of log_path = syslog Dovecot does not start if I do this. Tried to change permissions and overschip of the file, no luck, ha. Deleted it in the hope dovecot creates it but no. If I set log_path = syslog dovecot starts and I see it is running as # ps aux |grep dove root 2429 0.2 0.0 3416 2568 ? Ss 16:42 0:00 /usr/sbin/dovecot -F dovecot 2431 0.0 0.0 3100 2080 ? S 16:42 0:00 dovecot/anvil root 2432 0.0 0.0 3228 2316 ? S 16:42 0:00 dovecot/log root 2434 0.0 0.0 4220 3300 ? S 16:42 0:00 dovecot/config What must be done to have doveot log into the specified file? Greetings Andreas From a.meyer at nimmini.de Fri Nov 6 16:09:20 2015 From: a.meyer at nimmini.de (Andreas Meyer) Date: Fri, 6 Nov 2015 17:09:20 +0100 Subject: Problem with permissions of the logfile Message-ID: <20151106170920.13c2cc51@itxbox.bitcorner.intern> Hello! I want to have dovecot log into a separate file and set log_path = /var/log/dovecot instead of log_path = syslog Dovecot does not start if I do this. Tried to change permissions and overschip of the file, no luck, ha. Deleted it in the hope dovecot creates it but no. If I set log_path = syslog dovecot starts and I see it is running as # ps aux |grep dove root 2429 0.2 0.0 3416 2568 ? Ss 16:42 0:00 /usr/sbin/dovecot -F dovecot 2431 0.0 0.0 3100 2080 ? S 16:42 0:00 dovecot/anvil root 2432 0.0 0.0 3228 2316 ? S 16:42 0:00 dovecot/log root 2434 0.0 0.0 4220 3300 ? S 16:42 0:00 dovecot/config What must be done to have dovecot log into the specified file? Greetings Andreas From a.meyer at nimmini.de Fri Nov 6 16:43:29 2015 From: a.meyer at nimmini.de (Andreas Meyer) Date: Fri, 6 Nov 2015 17:43:29 +0100 Subject: Problem with permissions of the logfile In-Reply-To: References: <20151106170920.13c2cc51@itxbox.bitcorner.intern> Message-ID: <20151106174329.18ae990f@itxbox.bitcorner.intern> Hello! To stay on the list. "S. Muthu Kumar" schrieb am 06.11.15 um 21:49:32 Uhr: > log_path here means the logging channel path. It refers to the logger > daemon to use to generate logs. > > Most of the linux/unix based applications prefer to use common system > logger to take care of writing the logs preferably under /var/log/ as > there are other structures(like rotating those logs periodically). > > do refer to man pages of syslog.conf or rsyslog.conf and see how you > can generate individual log files for devecot via (r)syslog daemon > while you have delegated dovecot logging responsibility to (r)syslog. I must admit I don't understand what is going on. I am running rsyslog on two servers and on both servers version 2.2.13 of dovecot. On the first I set log_path = /var/log/dovecot1 with -rw------- 1 root root 1093533 6. Nov 17:24 dovecot1 in /var/log and everthing is fine. If I do this on the newly set up server, dovecot does not start. Ofcourse I could set it up to log to syslog and prepare rsyslog.conf but why is it so? Regards Andreas From ml+dovecot at valo.at Fri Nov 6 16:46:20 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Fri, 06 Nov 2015 17:46:20 +0100 Subject: Problem with permissions of the logfile In-Reply-To: <20151106170920.13c2cc51@itxbox.bitcorner.intern> References: <20151106170920.13c2cc51@itxbox.bitcorner.intern> Message-ID: <28DF4292-5A11-4B44-B625-58B1A0B1CDEC@valo.at> Hi Am 06. November 2015 17:09:20 MEZ, schrieb Andreas Meyer : >Hello! > >I want to have dovecot log into a separate file and set >log_path = /var/log/dovecot instead of log_path = syslog > >Dovecot does not start if I do this. Tried to change >permissions and overschip of the file, no luck, ha. >Deleted it in the hope dovecot creates it but no. > >If I set log_path = syslog dovecot starts and I see >it is running as > ># ps aux |grep dove >root 2429 0.2 0.0 3416 2568 ? Ss 16:42 0:00 >/usr/sbin/dovecot -F >dovecot 2431 0.0 0.0 3100 2080 ? S 16:42 0:00 >dovecot/anvil >root 2432 0.0 0.0 3228 2316 ? S 16:42 0:00 >dovecot/log >root 2434 0.0 0.0 4220 3300 ? S 16:42 0:00 >dovecot/config > >What must be done to have dovecot log into the specified file? > >Greetings > > Andreas Is the user dovecot is running as allowed to write to that file? You could configure your syslog daemon to write all dovecot logging to a different file and don't write it to standard syslog file. Regards Christian From nathan at bravenet.com Fri Nov 6 17:37:53 2015 From: nathan at bravenet.com (Nathan Coulson) Date: Fri, 6 Nov 2015 09:37:53 -0800 Subject: Questions on supporting Shared Mailboxes using imapc:// In-Reply-To: <5617FA83.9060404@bravenet.com> References: <5617FA83.9060404@bravenet.com> Message-ID: <563CE571.6070903@bravenet.com> On 2015-10-09 10:33 AM, Nathan Coulson wrote: > > I was not able to get a shared mailbox setup working using imapc. > Tested using mailboxes on the same server, as well as other servers. > > We are using dovecot 2.2.10 from Centos 7, following the guide at > http://wiki2.dovecot.org/SharedMailboxes/ClusterSetup > > > imapc_host=192.168.5.5 #(Local dovecot server) > imapc_master_user= %u > imapc_password=TempPass > #imapc_user=test2 at example.com # Added for testing. > namespace { > type = shared > separator = / > prefix = shared/%%u/ > list = children > location = imapc:~/shared/%%u/ > } > namespace { # used as a control to verify that shared mailboxes work > type = shared > separator = / > prefix = shared2/%%u/ > list = children > location = sdbox:%%h:INDEXPVT=~/shared2/%%u > } > > > dict { > acl = mysql:/etc/dovecot/dovecot-sql_aggelos_shares.conf.ext > } > > plugin { > acl = vfile > acl_shared_dict = proxy::acl > } > > mail_plugins = acl > > protocol imap { > mail_plugins = acl imap_acl > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf.ext > } > > > passdb { # for master user logins > driver = sql > args = /etc/dovecot/dovecot-sql_aggelos_aclmaster.conf.ext > master = yes > pass = yes > } > > userdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf.ext > } > > > > > > I can login using test2 at example.com*test at example.com (and see all of > test2's email), and it looks like it authenticates ok via imap, but > shared never shows up. No issues using shared2 which uses direct > access to the mailbox > > > doveadm acl debug -u test at example.com shared/test2 at example.com (Fails, > uses imapc) > > doveadm(test at example.com): Info: Mailbox 'INBOX' is in namespace > 'shared/test2 at example.com/' > doveadm(test at example.com): Info: Mailbox path: > /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.INBOX > doveadm(test at example.com): Info: All message flags are shared across > users in mailbox > doveadm(test at example.com): Info: User test at example.com has no rights > for mailbox > doveadm(test at example.com): Error: User test at example.com is missing > 'lookup' right > doveadm(test at example.com): Info: Mailbox shared/test2 at example.com is > NOT visible in LIST > > > > doveadm acl debug -u test at example.com shared/test2 at example.com/Junk > (Fails, contains a folder that exists) > doveadm(test at example.com): Info: Mailbox 'Junk' is in namespace > 'shared/test2 at example.com/' > doveadm(test at example.com): Info: Mailbox path: > /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.Junk > doveadm(test at example.com): Info: All message flags are shared across > users in mailbox > doveadm(test at example.com): Info: User test at example.com has no rights > for mailbox > doveadm(test at example.com): Error: User test at example.com is missing > 'lookup' right > doveadm(test at example.com): Info: Mailbox shared/test2 at example.com/Junk > is NOT visible in LIST > > > doveadm acl debug -u test at example.com shared/test2 at example.com/z > (Fails, contains a folder that does not exist. Expected result) > > doveadm(test at example.com): Error: Mailbox 'z' in namespace > 'shared/test2 at example.com/' doesn't exist in > /misc/1/2/mail/test at example.com-4/shared/test2 at example.com/.z > > > > doveadm acl debug -u test at example.com shared2/test2 at example.com > (Works, using direct storage) > > doveadm(test at example.com): Info: Mailbox 'INBOX' is in namespace > 'shared2/test2 at example.com/' > doveadm(test at example.com): Info: Mailbox path: > /misc/1/2/mail/test2 at example.com-87/mailboxes/INBOX/dbox-Mails > doveadm(test at example.com): Info: Per-user private flags in mailbox: \Seen > doveadm(test at example.com): Info: User test at example.com has rights: > lookup read write-seen > doveadm(test at example.com): Info: Mailbox found from dovecot-acl-list > doveadm(test at example.com): Info: User test2 at example.com found from ACL > shared dict > doveadm(test at example.com): Info: Mailbox shared2/test2 at example.com is > visible in LIST > > > > It feels like the acl information in the vfiles is not available when > using imapc. > > > > Our setup is as follows: (Currently does not use Dovecot Director, but > has the same issues where shared mailboxes may not be local on the > server) > * Front servers, that are running dovecot (proxy to backend servers) > * Backend Servers, each handle their own local users. > > > Thank you > Adding to this, Is it possible to set the location for the VFile's when using imapc? If so, I believe that would allow this to work. (Available over NFS [later GFS2] on remote servers, Technically we could also use the storage directly but I was concerned about 2 dovecot processes on different servers using the same sdbox storage location). -- Nathan Coulson System Administrator for Bravenet www.bravenet.com nathan at bravenet.com From ksmutthu at gmail.com Sat Nov 7 02:21:39 2015 From: ksmutthu at gmail.com (S. Muthu Kumar) Date: Sat, 7 Nov 2015 07:51:39 +0530 Subject: Problem with permissions of the logfile In-Reply-To: <28DF4292-5A11-4B44-B625-58B1A0B1CDEC@valo.at> References: <20151106170920.13c2cc51@itxbox.bitcorner.intern> <28DF4292-5A11-4B44-B625-58B1A0B1CDEC@valo.at> Message-ID: because linux/application developers think it is the best way to do, some of which I can think of (r)syslog provides a centralized log handling mechanisms like in case you want to collect all your servers logs to get them digested/analyzed in one place for better monitoring of your servers, other reason is to have the failure file permissions and other situations to be handled by (r)syslog rather the application suffering to start because the log file is not there or needed permissions are not there. While rotating logs copytruncate is not the default rotating method configured in logrotate. In that case you end up maintain log rotate configuration too if you want your application to handle writing its logs and to avoid service restart. Am sure there are many other use cases which is worth using (r)syslog to handle log writing. On 11/6/15, Christian Kivalo wrote: > Hi > > Am 06. November 2015 17:09:20 MEZ, schrieb Andreas Meyer > : >>Hello! >> >>I want to have dovecot log into a separate file and set >>log_path = /var/log/dovecot instead of log_path = syslog >> >>Dovecot does not start if I do this. Tried to change >>permissions and overschip of the file, no luck, ha. >>Deleted it in the hope dovecot creates it but no. >> >>If I set log_path = syslog dovecot starts and I see >>it is running as >> >># ps aux |grep dove >>root 2429 0.2 0.0 3416 2568 ? Ss 16:42 0:00 >>/usr/sbin/dovecot -F >>dovecot 2431 0.0 0.0 3100 2080 ? S 16:42 0:00 >>dovecot/anvil >>root 2432 0.0 0.0 3228 2316 ? S 16:42 0:00 >>dovecot/log >>root 2434 0.0 0.0 4220 3300 ? S 16:42 0:00 >>dovecot/config >> >>What must be done to have dovecot log into the specified file? >> >>Greetings >> >> Andreas > Is the user dovecot is running as allowed to write to that file? > > You could configure your syslog daemon to write all dovecot logging to a > different file and don't write it to standard syslog file. > > Regards > Christian > From bytesplit at gmail.com Sat Nov 7 20:31:21 2015 From: bytesplit at gmail.com (Philon) Date: Sat, 7 Nov 2015 21:31:21 +0100 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <563A60E2.4050404@gmail.com> References: <563A60E2.4050404@gmail.com> Message-ID: Hi there, I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. Philon > Am 04.11.2015 um 20:47 schrieb Forrest : > > I have been attempting to use the cyrus2dovecot script, to no avail. > > I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. > > In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. > > > inboxes=the "myaccount" that was copied over > > /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ > --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ > --cyrus-sub /home/varimap/user/%h/%u.sub \ > --dovecot-inbox /home/myaccount/dovecot/Maildir \ > myaccount > > > > the log output complains of: > > cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. > > and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. > > All directory paths are correct. > > > Thanks. From those.lists at gmail.com Sat Nov 7 21:35:21 2015 From: those.lists at gmail.com (Forrest) Date: Sat, 7 Nov 2015 16:35:21 -0500 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: References: <563A60E2.4050404@gmail.com> Message-ID: <563E6E99.2090405@gmail.com> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) On 11/7/15 3:31 PM, Philon wrote: > Hi there, > > I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. > > > Philon > > >> Am 04.11.2015 um 20:47 schrieb Forrest : >> >> I have been attempting to use the cyrus2dovecot script, to no avail. >> >> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >> >> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >> >> >> inboxes=the "myaccount" that was copied over >> >> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >> --cyrus-sub /home/varimap/user/%h/%u.sub \ >> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >> myaccount >> >> >> >> the log output complains of: >> >> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >> >> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >> >> All directory paths are correct. >> >> >> Thanks. From stephane.cottin at vixns.com Sun Nov 8 10:50:13 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Sun, 8 Nov 2015 11:50:13 +0100 Subject: antispam plugin pipe backend error when moving multiple emails Message-ID: <1B80B1F2-2FA3-4F42-A8F7-DD8AEDFA0CB9@vixns.com> Hi, I've got some trouble with the dovecot antispam plugin and the pipe backend. I'm using dovecot 2.2.18 with maildirs and zlib compression enabled. When moving 2 or more emails at once from the Junk folder to another one, I always have the following error : "Failed to copy to temporary file" In the server logs : imap(vvv at vvv.vvv): Error: read(zlib(/data/Maildir/.test/tmp/1446974366.M123890P936.vvv)) failed: Cached message size smaller than expected (13553 < 13562, box=test, UID=0) The same operation with one email at a time, on the same emails, works as expected. St?phane From lists at searchy.net Sun Nov 8 21:23:00 2015 From: lists at searchy.net (Frank de Bot (lists)) Date: Sun, 08 Nov 2015 22:23:00 +0100 Subject: Dsync problems Message-ID: <563FBD34.2070700@searchy.net> Hi, I'm setting up a migration between 2 dovecot servers 2.1.12 to 2.2.19 . I've ran into some issues with dsync. I use the following command for syncing: doveadm -o mail_fsync=never -o imapc_host=%remote% -o imapc_master_user=master -o imapc_password=%masterpw% -o imapc_features="rfc822.size fetch-headers" -o mail_prefetch_count=20 -o imapc_user=%user% backup -1 -R -u %user% imapc: I execute a full sync first and when the mailserver are being migrated it only would have to do incremental. The first is run is without problems. But I find the following problem: - Flags on current mails aren't updated. If I set message to unread, the sync won't pick them up. - On the origin dovecot server I had removed some messages and purged the inbox. Those message are now in the Trash and are created again and again when I sync Is this normal behaviour? It renders it pretty much useless since data consistency isn't guaranteed. Regards, Frank de Bot From fila at pruda.com Mon Nov 9 23:28:49 2015 From: fila at pruda.com (fila at pruda.com) Date: Mon, 9 Nov 2015 15:28:49 -0800 Subject: Fw: new message Message-ID: <0000692843f2$e39d2ad7$c1858174$@pruda.com> Hey! New message, please read fila at pruda.com From fila at pruda.com Mon Nov 9 23:35:42 2015 From: fila at pruda.com (dovecot) Date: Mon, 9 Nov 2015 15:35:42 -0800 Subject: Fw: new message Message-ID: <0000dd422c0b$87fca113$9e55d411$@pruda.com> Hey! New message, please read dovecot From fila at pruda.com Mon Nov 9 23:35:49 2015 From: fila at pruda.com (dovecot) Date: Mon, 9 Nov 2015 15:35:49 -0800 Subject: Fw: new message Message-ID: <00009dfe054f$b4fd2c20$07ea98a1$@pruda.com> Hey! New message, please read dovecot From mancyborg at gmail.com Tue Nov 10 00:44:41 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Tue, 10 Nov 2015 01:44:41 +0100 Subject: hunting the fatty Message-ID: <20151110014441.065cfb4b@linux> Hello dear list, I've recently discovered 'doveadm stats' and I'm trying to use "doveadm stats dump user" and "doveadm stats dump session" to understand the pop/imap users that put more stress on the hard disks. My problem is that some users refuse to delete their emails from the server, so they keep 20GB of maildir files on the server, the webmail (roundcube) takes forever to open the inbox, the imap searches takes forever and meanwhile all the users wait. (already tried roundcube + memcache(d) but didn't help) So my problem is not the storage usage itself: I don't care if the user gets tons of emails with big attachments; my problem is when the user opens / searches an imap folder with more than 10K mails and iostat util goes 100% for minutes. So I've enabled dovecot's stats and enjoying "doveadm stats top", "stats-top.pl" and "doveadm stats dump user/session", but talking about "doveadm stats dump user" and its output fields: user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits I'm not sure which of those fields can help me and I can't find any relevant documentation. So here are my questions: 1. is there a documentation for those 21 fields and for 'doveadm stats' in general ? 2. what's the difference between disk_output, read_bytes, read_count and mail_read_bytes ? 3. which field of those is, in your opinion, more representative for expressing the workload that gives me problems ? 4. which settings do I need to store 1 week worth of stats ? I'm currenty using the 'standard' values: stats_refresh = 30 secs stats_track_cmds = yes stats_memory_limit = 16 M stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_session_min_time = 15 mins stats_user_min_time = 1 hours Can you please tell me the correct parameters to store 1 week of stats ? Thank you, Mike From vlb at cfcl.com Tue Nov 10 05:35:12 2015 From: vlb at cfcl.com (Vicki Brown) Date: Mon, 9 Nov 2015 21:35:12 -0800 Subject: procmail as a content_filter with dovecot Message-ID: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> I am using Mac OS X Server. OS X Server uses postfix with dovecot set as mail_transport. I want to add procmail to the mix. I cannot use the (typical) mailbox_command = procmail -a "$EXTENSION" because mailbox_transport = dovecot overrides any mailbox_command parameter and it is NOT Possible to change, remove, or otherwise vary the mailbox_transport on OS X server. I'm thinking I should be able to set up a content_filter for procmail, but I've found nothing online to tell me if this is possible or how to declare it. Can anyone help? -- Vicki cfcl.com/vlb twitter.com/vlb From me at junc.eu Tue Nov 10 06:49:27 2015 From: me at junc.eu (Benny Pedersen) Date: Tue, 10 Nov 2015 07:49:27 +0100 Subject: procmail as a content_filter with dovecot In-Reply-To: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> Message-ID: <20151110064951.69A6523619@talvi.dovecot.org> On November 10, 2015 6:36:00 AM Vicki Brown wrote: > Can anyone help? dovecot supports sieve, so why the need for procmail ? but if mta is configured to use procmail then next step is just configure procmailrc to use mda dovecot-lda From ml+dovecot at valo.at Tue Nov 10 07:50:50 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Tue, 10 Nov 2015 08:50:50 +0100 Subject: hunting the fatty In-Reply-To: <20151110014441.065cfb4b@linux> References: <20151110014441.065cfb4b@linux> Message-ID: <2e2f2d5602081281bd81b918722cd0c4@valo.at> Hi, On 2015-11-10 01:44, mancyborg at gmail.com wrote: > Hello dear list, > I've recently discovered 'doveadm stats' and I'm trying to use > "doveadm stats dump user" and "doveadm stats dump session" > to understand the pop/imap users that put more stress on the hard > disks. > > My problem is that some users refuse to delete their emails from the > server, > so they keep 20GB of maildir files on the server, the webmail > (roundcube) takes forever to open the inbox, > the imap searches takes forever > and meanwhile all the users wait. > (already tried roundcube + memcache(d) but didn't help) What is forever in your context? I'm using roundcube and a folder with about 78k mails opens in < 1 sec unsorted. A folder with about 37k messages from a mailinglist and thread sort takes < 3 sec. My roundcube shows 200 messages per page by default. On a side note, are you using an imap proxy for roundcube? It doesn't help you with your dovecot problem but it speeds up roundcube. To speed up imap searches i can recommend to implement fts-solr with dovecot (or maybe fts-elasticsearch, am wanting to try that but solr works...). That will speed up your searches after mailboxes are indexed. > So my problem is not the storage usage itself: > I don't care if the user gets tons of emails with big attachments; > my problem is when the user opens / searches an imap folder with more > than 10K mails > and iostat util goes 100% for minutes. Dovecot should be very quick to open even folders with a huge amount of files due to its indexes. I'm unable to reproduce any significant numbers in iostat when accessing large mailfolders with roundcube. Whats your configuration, filesystem, ... > So I've enabled dovecot's stats and enjoying "doveadm stats top", > "stats-top.pl" and "doveadm stats dump user/session", > but talking about "doveadm stats dump user" and its output fields: > > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > > I'm not sure which of those fields can help me > and I can't find any relevant documentation. > > So here are my questions: > > 1. is there a documentation for those 21 fields and for 'doveadm > stats' in general ? > 2. what's the difference between disk_output, read_bytes, read_count > and mail_read_bytes ? > 3. which field of those is, in your opinion, more representative for > expressing the workload that gives me problems ? > 4. which settings do I need to store 1 week worth of stats ? > > I'm currenty using the 'standard' values: > > stats_refresh = 30 secs > stats_track_cmds = yes > stats_memory_limit = 16 M > stats_command_min_time = 1 mins > stats_domain_min_time = 12 hours > stats_ip_min_time = 12 hours > stats_session_min_time = 15 mins > stats_user_min_time = 1 hours > > Can you please tell me the correct parameters to store 1 week of stats > ? For stats somebody else has to jump in, i have only enabled the plugin to see what to get out of it but not made any use of it. Please share your doveconf -n output > Thank you, > Mike regards christian From bytesplit at gmail.com Tue Nov 10 08:55:37 2015 From: bytesplit at gmail.com (Philon) Date: Tue, 10 Nov 2015 09:55:37 +0100 Subject: procmail as a content_filter with dovecot In-Reply-To: <20151110064951.69A6523619@talvi.dovecot.org> References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: <78A86E91-38AB-4AA1-A732-9BF0C8C1AD6D@googlemail.com> Like Benny said: dovecot has sieve, even on OSX: LMGIFY :) : https://support.apple.com/kb/PH8750 is a bit dated as for Snow Leopard? but here I find something more recent: http://www.markreinmuth.com/2014/07/server-side-mail-rules-in-os-x-server/ I can only recommend to install Roundcube and the managesieve plugin. That will give you a nice webui for editing your filters. Philon > Am 10.11.2015 um 07:49 schrieb Benny Pedersen : > > On November 10, 2015 6:36:00 AM Vicki Brown wrote: > >> Can anyone help? > > dovecot supports sieve, so why the need for procmail ? > > but if mta is configured to use procmail then next step is just configure procmailrc to use mda dovecot-lda From f0andrey at gmail.com Tue Nov 10 09:54:46 2015 From: f0andrey at gmail.com (Andrey Fesenko) Date: Tue, 10 Nov 2015 12:54:46 +0300 Subject: LDAP schema for dovecot proxy? Message-ID: Hello, I want to deploy dovecot proxy/director with the backend and authorization in LDAP. Dovecot wiki specifies only what is necessary to apply additional arguments that the scheme would have earned a proxy, but no solid LDAP schema. Is there such a scheme, such as the existing scheme http://www.zytrax.com/books/ldap/ape/courier.html http://pro-ldap.ru/sources/schema/qmail.schema http://open.rhx.it/phamm/schema/phamm.schema Unfortunately for the tasks they are not good because they just no dovecot-specific variables From sergey.schwartz at bgoperator.com Tue Nov 10 12:46:25 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Tue, 10 Nov 2015 15:46:25 +0300 Subject: lmtp panic Message-ID: <5641E721.9060008@bgoperator.com> Gents, I've just upgraded to the latest build of dovecot , now lmtp delivery process panics for just one user Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f50e380ddbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f50e380f555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f50e37fe6ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f50e37ff277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f50e3a62b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f50e597ebbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f50e597ffb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50e597ede8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f887d2b0dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f887d2b2555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f887d2a16ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f887d2a2277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f887d505b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f887f421bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f887f422fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f887f421de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7fd26aef5dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7fd26aef7555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7fd26aed32e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7fd26aee66ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7fd26aee7277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7fd26b14ab81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fd26d066bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7fd26d067fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fd26d066c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd26d066de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7494 killed with signal 6 (core dumped) Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7497 killed with signal 6 (core dumped) Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7499 killed with signal 6 (core dumped) Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 6719 killed with signal 6 (core dumped) Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7502 killed with signal 6 (core dumped) Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: Connection closed (in DATA finished) Oleg has lots and lots of small emails :) I'm running ubuntu server 14.04 x86_64 dovecot 2.2.19 (0b1c73b01a5a) build from http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main -- Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz at bgoperator.com From vlb at cfcl.com Tue Nov 10 20:01:52 2015 From: vlb at cfcl.com (Vicki Brown) Date: Tue, 10 Nov 2015 12:01:52 -0800 Subject: * Re: procmail as a content_filter with dovecot In-Reply-To: <20151110064951.69A6523619@talvi.dovecot.org> References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: > dovecot supports sieve, so why the need for procmail ? Because I already HAVE procmail recipes and know proemial. The point is to make what I have, work. > On Nov 9, 2015, at 22:49, Benny Pedersen wrote: > > On November 10, 2015 6:36:00 AM Vicki Brown wrote: > >> Can anyone help? > > dovecot supports sieve, so why the need for procmail ? > > but if mta is configured to use procmail then next step is just configure procmailrc to use mda dovecot-lda > -- Vicki cfcl.com/vlb twitter.com/vlb From me at junc.eu Tue Nov 10 21:01:53 2015 From: me at junc.eu (Benny Pedersen) Date: Tue, 10 Nov 2015 22:01:53 +0100 Subject: * Re: procmail as a =?UTF-8?Q?content=5Ffilter=20with=20dovec?= =?UTF-8?Q?ot?= In-Reply-To: References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: Vicki Brown skrev den 2015-11-10 21:01: >> dovecot supports sieve, so why the need for procmail ? > Because I already HAVE procmail recipes and know proemial. +1 > The point is to make what I have, work. https://www.google.dk/search?q=procmailrc+to+sieve and you can imho learn more with roundcube, that will help you make the power out of sieve, but i know it if one have learned procmailrc its will be staying that way, until one see how simple sieve rules can be horde webmail can change rules from procmail to sieve, so if server changes from procmail to another filter engine it would just need a new save of rules, since horde saves meta data to rebuild procmail rules, and this metadata can make sieve rules aswell have fun From bytesplit at gmail.com Tue Nov 10 21:10:30 2015 From: bytesplit at gmail.com (Philon) Date: Tue, 10 Nov 2015 22:10:30 +0100 Subject: * Re: procmail as a content_filter with dovecot In-Reply-To: References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: Vicki Your point here is that you are using Mac OS X. This as you have already learned is overriding your changes with what the guys in Cupertino think is good for you. So I?d rather move on to sieve then fighting my way against this not-so-servery OS. You will get killed on the next OS upgrade I tell you! Sieve is not bad! And OSX is not a server OS as long as you not switch everything over to macports and leave the Server GUI app behind... Make your decision :) > Am 10.11.2015 um 21:01 schrieb Vicki Brown : > >> dovecot supports sieve, so why the need for procmail ? > > Because I already HAVE procmail recipes and know proemial. > > The point is to make what I have, work. > > >> On Nov 9, 2015, at 22:49, Benny Pedersen wrote: >> >> On November 10, 2015 6:36:00 AM Vicki Brown wrote: >> >>> Can anyone help? >> >> dovecot supports sieve, so why the need for procmail ? >> >> but if mta is configured to use procmail then next step is just configure procmailrc to use mda dovecot-lda >> > > -- Vicki > cfcl.com/vlb > twitter.com/vlb From vlb at cfcl.com Tue Nov 10 22:38:51 2015 From: vlb at cfcl.com (Vicki Brown) Date: Tue, 10 Nov 2015 14:38:51 -0800 Subject: * Re: procmail as a content_filter with dovecot In-Reply-To: References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: Thank you. This may be helpful. > On Nov 10, 2015, at 13:01, Benny Pedersen wrote: > > > https://www.google.dk/search?q=procmailrc+to+sieve > > > have fun -- Vicki cfcl.com/vlb twitter.com/vlb From mancyborg at gmail.com Wed Nov 11 02:25:44 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Wed, 11 Nov 2015 03:25:44 +0100 Subject: hunting the fatty In-Reply-To: <2e2f2d5602081281bd81b918722cd0c4@valo.at> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> Message-ID: <20151111032544.0c8bf2b1@linux> On Tue, 10 Nov 2015 08:50:50 +0100 Christian Kivalo wrote: > Hi, > > On 2015-11-10 01:44, mancyborg at gmail.com wrote: > > Hello dear list, > > I've recently discovered 'doveadm stats' and I'm trying to use > > "doveadm stats dump user" and "doveadm stats dump session" > > to understand the pop/imap users that put more stress on the hard > > disks. > > > > My problem is that some users refuse to delete their emails from the > > server, > > so they keep 20GB of maildir files on the server, the webmail > > (roundcube) takes forever to open the inbox, > > the imap searches takes forever > > and meanwhile all the users wait. > > (already tried roundcube + memcache(d) but didn't help) > > What is forever in your context? > I'm using roundcube and a folder with about 78k mails opens in < 1 sec > unsorted. A folder with about 37k messages from a mailinglist and thread > sort takes < 3 sec. My roundcube shows 200 messages per page by default. > On a side note, are you using an imap proxy for roundcube? It doesn't > help you with your dovecot problem but it speeds up roundcube. > > To speed up imap searches i can recommend to implement fts-solr with > dovecot (or maybe fts-elasticsearch, am wanting to try that but solr > works...). That will speed up your searches after mailboxes are indexed. > > > So my problem is not the storage usage itself: > > I don't care if the user gets tons of emails with big attachments; > > my problem is when the user opens / searches an imap folder with more > > than 10K mails > > and iostat util goes 100% for minutes. > > Dovecot should be very quick to open even folders with a huge amount of > files due to its indexes. > > I'm unable to reproduce any significant numbers in iostat when accessing > large mailfolders with roundcube. > > Whats your configuration, filesystem, ... > > > So I've enabled dovecot's stats and enjoying "doveadm stats top", > > "stats-top.pl" and "doveadm stats dump user/session", > > but talking about "doveadm stats dump user" and its output fields: > > > > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > > > > I'm not sure which of those fields can help me > > and I can't find any relevant documentation. > > > > So here are my questions: > > > > 1. is there a documentation for those 21 fields and for 'doveadm > > stats' in general ? > > 2. what's the difference between disk_output, read_bytes, read_count > > and mail_read_bytes ? > > 3. which field of those is, in your opinion, more representative for > > expressing the workload that gives me problems ? > > 4. which settings do I need to store 1 week worth of stats ? > > > > I'm currenty using the 'standard' values: > > > > stats_refresh = 30 secs > > stats_track_cmds = yes > > stats_memory_limit = 16 M > > stats_command_min_time = 1 mins > > stats_domain_min_time = 12 hours > > stats_ip_min_time = 12 hours > > stats_session_min_time = 15 mins > > stats_user_min_time = 1 hours > > > > Can you please tell me the correct parameters to store 1 week of stats > > ? > > For stats somebody else has to jump in, i have only enabled the plugin > to see what to get out of it but not made any use of it. > > Please share your doveconf -n output > > > Thank you, > > Mike > > regards > christian By 'forever' I mean more than 1 minute. So there is no documentation / manual for 'doveadm stats' ? Do I have to read the source to know which field does what ? I mean the output fields of "doveadm stats dump user": user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits what's the difference between disk_output, read_bytes, read_count and mail_read_bytes ? (sorry to restate the same question, just making sure about it) Thank you, Mike From mancyborg at gmail.com Wed Nov 11 02:44:22 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Wed, 11 Nov 2015 03:44:22 +0100 Subject: hunting the fatty In-Reply-To: <2e2f2d5602081281bd81b918722cd0c4@valo.at> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> Message-ID: <20151111034422.71ede74e@linux> On Tue, 10 Nov 2015 08:50:50 +0100 Christian Kivalo wrote: > Hi, > > On 2015-11-10 01:44, mancyborg at gmail.com wrote: > > Hello dear list, > > I've recently discovered 'doveadm stats' and I'm trying to use > > "doveadm stats dump user" and "doveadm stats dump session" > > to understand the pop/imap users that put more stress on the hard > > disks. > > > > My problem is that some users refuse to delete their emails from the > > server, > > so they keep 20GB of maildir files on the server, the webmail > > (roundcube) takes forever to open the inbox, > > the imap searches takes forever > > and meanwhile all the users wait. > > (already tried roundcube + memcache(d) but didn't help) > > What is forever in your context? > I'm using roundcube and a folder with about 78k mails opens in < 1 sec > unsorted. A folder with about 37k messages from a mailinglist and thread > sort takes < 3 sec. My roundcube shows 200 messages per page by default. > On a side note, are you using an imap proxy for roundcube? It doesn't > help you with your dovecot problem but it speeds up roundcube. > > To speed up imap searches i can recommend to implement fts-solr with > dovecot (or maybe fts-elasticsearch, am wanting to try that but solr > works...). That will speed up your searches after mailboxes are indexed. > > > So my problem is not the storage usage itself: > > I don't care if the user gets tons of emails with big attachments; > > my problem is when the user opens / searches an imap folder with more > > than 10K mails > > and iostat util goes 100% for minutes. > > Dovecot should be very quick to open even folders with a huge amount of > files due to its indexes. > > I'm unable to reproduce any significant numbers in iostat when accessing > large mailfolders with roundcube. > > Whats your configuration, filesystem, ... > > > So I've enabled dovecot's stats and enjoying "doveadm stats top", > > "stats-top.pl" and "doveadm stats dump user/session", > > but talking about "doveadm stats dump user" and its output fields: > > > > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > > > > I'm not sure which of those fields can help me > > and I can't find any relevant documentation. > > > > So here are my questions: > > > > 1. is there a documentation for those 21 fields and for 'doveadm > > stats' in general ? > > 2. what's the difference between disk_output, read_bytes, read_count > > and mail_read_bytes ? > > 3. which field of those is, in your opinion, more representative for > > expressing the workload that gives me problems ? > > 4. which settings do I need to store 1 week worth of stats ? > > > > I'm currenty using the 'standard' values: > > > > stats_refresh = 30 secs > > stats_track_cmds = yes > > stats_memory_limit = 16 M > > stats_command_min_time = 1 mins > > stats_domain_min_time = 12 hours > > stats_ip_min_time = 12 hours > > stats_session_min_time = 15 mins > > stats_user_min_time = 1 hours > > > > Can you please tell me the correct parameters to store 1 week of stats > > ? > > For stats somebody else has to jump in, i have only enabled the plugin > to see what to get out of it but not made any use of it. > > Please share your doveconf -n output > > > Thank you, > > Mike > > regards > christian By 'forever' I mean more than 1 minute. So there is no documentation / manual for 'doveadm stats' ? Do I have to read the source to know which field does what ? I mean the output fields of "doveadm stats dump user": user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits what's the difference between disk_output, read_bytes, read_count and mail_read_bytes ? (sorry to restate the same question, just making sure about it) Thank you, Mike From lists at wiesinger.com Wed Nov 11 08:08:29 2015 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Wed, 11 Nov 2015 09:08:29 +0100 Subject: * Re: procmail as a content_filter with dovecot In-Reply-To: References: <0742A343-7014-4BEE-B989-DFF22058FC69@cfcl.com> <20151110064951.69A6523619@talvi.dovecot.org> Message-ID: <5642F77D.3090401@wiesinger.com> On 10.11.2015 23:38, Vicki Brown wrote: > Thank you. This may be helpful. > >> On Nov 10, 2015, at 13:01, Benny Pedersen wrote: >> >> >> https://www.google.dk/search?q=procmailrc+to+sieve >> >> Find my enhanced version + wrapper script at with custom configuration at: http://www.wiesinger.com/opensource/procmail2sieve/ Ciao, Gerhard From klemen at arnes.si Wed Nov 11 11:08:27 2015 From: klemen at arnes.si (klemen at arnes.si) Date: Wed, 11 Nov 2015 12:08:27 +0100 Subject: locked ldap users Message-ID: <564321AB.9020809@arnes.si> Hello. I'm having a problem regarding locked users in ldap. We are using 389DS ldap server. We lock our users with nsAccountLock=true. If user successfully logs into dovecot, his credentials gets cached. When this user is locked its credentials still stay in cache. The problem I'm having is that our ldap server returns error code 53 ("Unwilling to perform - Account inactivated"). Dovecot takes this error code and decides that ldap doesnt work so it takes users credentials from cache (like stated in docs). User can still login untill his credentials are cleared from cache. Error message in logs: ldap(USERNAME,193.X.Y.Z,): ldap_bind() failed: Server is unwilling to perform Oct 20 14:39:31 SERVER dovecot: auth: ldap(USERNAME,193.X.Y.Z,): Falling back to expired data from cache Does dovecot reacts to error code 53 as it should? Maybe our ldap server should return different error code - like 49/533 (Account_disabled)? How would dovecot react to error code 49? Currenty we solve this problem by taking list of locked users and clear cache every minute. thank you and best regards, Klemen Andreuzzi Arnes From adm at tenhi.ru Wed Nov 11 11:32:24 2015 From: adm at tenhi.ru (=?UTF-8?Q?=D0=A1=D1=82=D0=B0=D0=BD=D0=B8=D1=81=D0=BB=D0=B0=D0=B2_?= =?UTF-8?Q?=D0=A7=D0=B5=D1=80=D0=BA=D0=B0=D1=81=D0=BE=D0=B2?=) Date: Wed, 11 Nov 2015 14:32:24 +0300 Subject: Dovecot+AD Message-ID: <14b5e631a1077fc0e1c1a56bf6e42891@tenhi.ru> Hello. I have a iRedmail server integrated with AD. I make it following theese instructions http://www.iredmail.org/docs/active.directory.html In my AD i have a user attributes: sAMAccountName = user1 userPrincipalName = csi at int.mdom.com proxyAddresses = mail1 at out.lor.com i need login with user user1 without domain and then have an address mail1 at out.lor.com i user it in /etc/dovecot/dovecot-ldap.conf ---- hosts = dc1.int.mdom.com:389 ldap_version = 3 auth_bind = yes dn = userksa dnpass = sdjfuihJKHSAJKDHQUWjdskfh base = OU=Users,OU=QOL,DC=int,DC=mdom,DC=com scope = subtree deref = never user_filter = (&(objectClass=user)(sAMAccountName=%n) (memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) pass_filter = (&(objectClass=user)(sAMAccountName=%n) (memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) default_pass_scheme = CRYPT user_attrs = =home=/var/vmail/vmail1/%Ld/%Ln/Maildir/,=mail=maildir:/var/vmail/vmail1/%Ld/%Ln/Maildir/ ---- i have tried to do: #user_filter = (&(proxyAddresses=%u) (objectClass=person)(memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) #pass_filter = (&(proxyAddresses=%u) (objectClass=person)(memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) but nothing works. From adm at tenhi.ru Wed Nov 11 11:36:44 2015 From: adm at tenhi.ru (=?UTF-8?Q?=D0=A1=D1=82=D0=B0=D0=BD=D0=B8=D1=81=D0=BB=D0=B0=D0=B2_?= =?UTF-8?Q?=D0=A7=D0=B5=D1=80=D0=BA=D0=B0=D1=81=D0=BE=D0=B2?=) Date: Wed, 11 Nov 2015 14:36:44 +0300 Subject: Active directory Message-ID: Hello. I have a iRedmail server integrated with AD. I make it following theese instructions http://www.iredmail.org/docs/active.directory.html In my AD i have a user attributes: sAMAccountName = user1 userPrincipalName = csi at int.mdom.com proxyAddresses = mail1 at out.lor.com i need login with user user1 without domain and then have an address mail1 at out.lor.com i user it in /?etc/?dovecot/?dovecot-?ldap.conf -?-?-?-? hosts = dc1.int.mdom.com:389 ldap_version = 3 auth_bind = yes dn = userksa dnpass = sdjfuihJKHSAJKDHQUWjdskfh base = OU=Users,OU=QOL,DC=int,DC=mdom,DC=com scope = subtree deref = never user_filter = (&(objectClass=user)(sAMAccountName=%n) (memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) pass_filter = (&(objectClass=user)(sAMAccountName=%n) (memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) default_pass_scheme = CRYPT user_attrs = =home=/var/vmail/vmail1/%Ld/%Ln/Maildir/,=mail=maildir:/var/vmail/vmail1/%Ld/%Ln/Maildir/ -?-?-?-? i have tried to do: #user_filter = (&(proxyAddresses=%u) (objectClass=person)(memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) #pass_filter = (&(proxyAddresses=%u) (objectClass=person)(memberof=CN=Mail_Users,OU=Groups,OU=SFP,DC=int,DC=mdom,DC=com)) but nothing works. From sergey.schwartz at bgoperator.com Wed Nov 11 14:06:38 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Wed, 11 Nov 2015 17:06:38 +0300 Subject: lmtp panic In-Reply-To: <5641E721.9060008@bgoperator.com> References: <5641E721.9060008@bgoperator.com> Message-ID: <56434B6E.7080902@bgoperator.com> After 1 or 2 hours of recieveing this kind of errors mail server runs out of file descriptors Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: Too many open files Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz at bgoperator.com 10.11.2015 15:46, Sergey Schwartz ?????: > Gents, > > > > I've just upgraded to the latest build of dovecot , now lmtp delivery > process panics for just one user > > > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7f50e380ddbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7f50e380f555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7f50e37fe6ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7f50e37ff277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7f50e3a62b81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> > dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7f50e597ebbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7f50e597ffb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f50e597ede8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7f887d2b0dbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7f887d2b2555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7f887d2a16ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7f887d2a2277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7f887d505b81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> > dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7f887f421bbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7f887f422fb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7f887f421de8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] > Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local > Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Panic: file str.c: line 22 (str_new_const): assertion failed: > (str[len] == '\0') > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) > [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) > [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) > [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) > [0x7fd26aef5dbd] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) > [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) > [0x7fd26aef7555] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7fd26aed32e7] > -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) > [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) > [0x7fd26aee66ea] -> > /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) > [0x7fd26aee7277] -> > /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) > [0x7fd26b14ab81] -> > /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) > [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> > dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) > [0x7fd26d066bbc] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) > [0x7fd26d067fb1] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) > [0x7fd26d066c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) > [0x7fd26d066de8] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) > [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7494 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7497 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7499 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 6719 killed with signal 6 (core > dumped) > Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): > Fatal: master: service(lmtp): child 7502 killed with signal 6 (core > dumped) > Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: > Connection closed (in DATA finished) > > > Oleg has lots and lots of small emails :) > > I'm running ubuntu server 14.04 x86_64 > dovecot 2.2.19 (0b1c73b01a5a) build from > http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main > From edevena at gsy.it Wed Nov 11 14:23:09 2015 From: edevena at gsy.it (Eugenio De Vena) Date: Wed, 11 Nov 2015 15:23:09 +0100 Subject: log client certificate Message-ID: <56434F4D.6050609@gsy.it> Hello, I would like to log the client certificate ( don't want to authenticate using client certificate but just "log" the client certificate if present ( both Subject and Public key ) ). Is it possibile ? Eugenio -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3697 bytes Desc: Firma crittografica S/MIME URL: From ml+dovecot at valo.at Wed Nov 11 15:34:33 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Wed, 11 Nov 2015 16:34:33 +0100 Subject: hunting the fatty In-Reply-To: <20151111034422.71ede74e@linux> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> Message-ID: <673c398bcdf5c28c7a8ec65724672cce@valo.at> On 2015-11-11 03:44, mancyborg at gmail.com wrote: > On Tue, 10 Nov 2015 08:50:50 +0100 > Christian Kivalo wrote: > >> Hi, >> >> On 2015-11-10 01:44, mancyborg at gmail.com wrote: >> > Hello dear list, >> > I've recently discovered 'doveadm stats' and I'm trying to use >> > "doveadm stats dump user" and "doveadm stats dump session" >> > to understand the pop/imap users that put more stress on the hard >> > disks. >> > >> > My problem is that some users refuse to delete their emails from the >> > server, >> > so they keep 20GB of maildir files on the server, the webmail >> > (roundcube) takes forever to open the inbox, >> > the imap searches takes forever >> > and meanwhile all the users wait. >> > (already tried roundcube + memcache(d) but didn't help) >> >> What is forever in your context? >> I'm using roundcube and a folder with about 78k mails opens in < 1 sec >> unsorted. A folder with about 37k messages from a mailinglist and >> thread >> sort takes < 3 sec. My roundcube shows 200 messages per page by >> default. >> On a side note, are you using an imap proxy for roundcube? It doesn't >> help you with your dovecot problem but it speeds up roundcube. >> >> To speed up imap searches i can recommend to implement fts-solr with >> dovecot (or maybe fts-elasticsearch, am wanting to try that but solr >> works...). That will speed up your searches after mailboxes are >> indexed. >> >> > So my problem is not the storage usage itself: >> > I don't care if the user gets tons of emails with big attachments; >> > my problem is when the user opens / searches an imap folder with more >> > than 10K mails >> > and iostat util goes 100% for minutes. >> >> Dovecot should be very quick to open even folders with a huge amount >> of >> files due to its indexes. >> >> I'm unable to reproduce any significant numbers in iostat when >> accessing >> large mailfolders with roundcube. >> >> Whats your configuration, filesystem, ... >> >> > So I've enabled dovecot's stats and enjoying "doveadm stats top", >> > "stats-top.pl" and "doveadm stats dump user/session", >> > but talking about "doveadm stats dump user" and its output fields: >> > >> > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits >> > >> > I'm not sure which of those fields can help me >> > and I can't find any relevant documentation. >> > >> > So here are my questions: >> > >> > 1. is there a documentation for those 21 fields and for 'doveadm >> > stats' in general ? >> > 2. what's the difference between disk_output, read_bytes, read_count >> > and mail_read_bytes ? >> > 3. which field of those is, in your opinion, more representative for >> > expressing the workload that gives me problems ? >> > 4. which settings do I need to store 1 week worth of stats ? >> > >> > I'm currenty using the 'standard' values: >> > >> > stats_refresh = 30 secs >> > stats_track_cmds = yes >> > stats_memory_limit = 16 M >> > stats_command_min_time = 1 mins >> > stats_domain_min_time = 12 hours >> > stats_ip_min_time = 12 hours >> > stats_session_min_time = 15 mins >> > stats_user_min_time = 1 hours >> > >> > Can you please tell me the correct parameters to store 1 week of stats >> > ? >> >> For stats somebody else has to jump in, i have only enabled the plugin >> to see what to get out of it but not made any use of it. >> >> Please share your doveconf -n output >> >> > Thank you, >> > Mike >> >> regards >> christian > > By 'forever' I mean more than 1 minute. That is really long. This should not take that long. > So there is no documentation / manual for 'doveadm stats' ? > Do I have to read the source to know which field does what ? I don't know of more than whats on the dovecot wiki stats plugin page at http://wiki2.dovecot.org/Statistics > I mean the output fields of "doveadm stats dump user": > > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > > what's the difference between disk_output, read_bytes, read_count and > mail_read_bytes ? From the wiki page: disk_output: Number of bytes written to disk -> i'd go for disk_input if your interested in reads read_bytes: Number of bytes read using read() syscalls read_count: Number of read() syscalls mail_read_bytes: Number of message bytes read() Not really much information but a base to start tests from. Make yourself a testaccount and test. Thats the best way to figure stuff out by yourself. > (sorry to restate the same question, just making sure about it) > > Thank you, > Mike Regards Christian From mancyborg at gmail.com Wed Nov 11 15:51:05 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Wed, 11 Nov 2015 16:51:05 +0100 Subject: hunting the fatty In-Reply-To: <673c398bcdf5c28c7a8ec65724672cce@valo.at> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> <673c398bcdf5c28c7a8ec65724672cce@valo.at> Message-ID: <20151111165105.4bc9fb35@linux> On Wed, 11 Nov 2015 16:34:33 +0100 Christian Kivalo wrote: > On 2015-11-11 03:44, mancyborg at gmail.com wrote: > > On Tue, 10 Nov 2015 08:50:50 +0100 > > Christian Kivalo wrote: > > > >> Hi, > >> > >> On 2015-11-10 01:44, mancyborg at gmail.com wrote: > >> > Hello dear list, > >> > I've recently discovered 'doveadm stats' and I'm trying to use > >> > "doveadm stats dump user" and "doveadm stats dump session" > >> > to understand the pop/imap users that put more stress on the hard > >> > disks. > >> > > >> > My problem is that some users refuse to delete their emails from the > >> > server, > >> > so they keep 20GB of maildir files on the server, the webmail > >> > (roundcube) takes forever to open the inbox, > >> > the imap searches takes forever > >> > and meanwhile all the users wait. > >> > (already tried roundcube + memcache(d) but didn't help) > >> > >> What is forever in your context? > >> I'm using roundcube and a folder with about 78k mails opens in < 1 sec > >> unsorted. A folder with about 37k messages from a mailinglist and > >> thread > >> sort takes < 3 sec. My roundcube shows 200 messages per page by > >> default. > >> On a side note, are you using an imap proxy for roundcube? It doesn't > >> help you with your dovecot problem but it speeds up roundcube. > >> > >> To speed up imap searches i can recommend to implement fts-solr with > >> dovecot (or maybe fts-elasticsearch, am wanting to try that but solr > >> works...). That will speed up your searches after mailboxes are > >> indexed. > >> > >> > So my problem is not the storage usage itself: > >> > I don't care if the user gets tons of emails with big attachments; > >> > my problem is when the user opens / searches an imap folder with more > >> > than 10K mails > >> > and iostat util goes 100% for minutes. > >> > >> Dovecot should be very quick to open even folders with a huge amount > >> of > >> files due to its indexes. > >> > >> I'm unable to reproduce any significant numbers in iostat when > >> accessing > >> large mailfolders with roundcube. > >> > >> Whats your configuration, filesystem, ... > >> > >> > So I've enabled dovecot's stats and enjoying "doveadm stats top", > >> > "stats-top.pl" and "doveadm stats dump user/session", > >> > but talking about "doveadm stats dump user" and its output fields: > >> > > >> > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > >> > > >> > I'm not sure which of those fields can help me > >> > and I can't find any relevant documentation. > >> > > >> > So here are my questions: > >> > > >> > 1. is there a documentation for those 21 fields and for 'doveadm > >> > stats' in general ? > >> > 2. what's the difference between disk_output, read_bytes, read_count > >> > and mail_read_bytes ? > >> > 3. which field of those is, in your opinion, more representative for > >> > expressing the workload that gives me problems ? > >> > 4. which settings do I need to store 1 week worth of stats ? > >> > > >> > I'm currenty using the 'standard' values: > >> > > >> > stats_refresh = 30 secs > >> > stats_track_cmds = yes > >> > stats_memory_limit = 16 M > >> > stats_command_min_time = 1 mins > >> > stats_domain_min_time = 12 hours > >> > stats_ip_min_time = 12 hours > >> > stats_session_min_time = 15 mins > >> > stats_user_min_time = 1 hours > >> > > >> > Can you please tell me the correct parameters to store 1 week of stats > >> > ? > >> > >> For stats somebody else has to jump in, i have only enabled the plugin > >> to see what to get out of it but not made any use of it. > >> > >> Please share your doveconf -n output > >> > >> > Thank you, > >> > Mike > >> > >> regards > >> christian > > > > By 'forever' I mean more than 1 minute. > > That is really long. This should not take that long. > > > So there is no documentation / manual for 'doveadm stats' ? > > Do I have to read the source to know which field does what ? > > I don't know of more than whats on the dovecot wiki stats plugin page at > http://wiki2.dovecot.org/Statistics > > > I mean the output fields of "doveadm stats dump user": > > > > user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits > > > > what's the difference between disk_output, read_bytes, read_count and > > mail_read_bytes ? > > From the wiki page: > disk_output: Number of bytes written to disk -> i'd go for disk_input if > your interested in reads > read_bytes: Number of bytes read using read() syscalls > read_count: Number of read() syscalls > mail_read_bytes: Number of message bytes read() > > Not really much information but a base to start tests from. > > Make yourself a testaccount and test. Thats the best way to figure stuff > out by yourself. > > > (sorry to restate the same question, just making sure about it) > > > > Thank you, > > Mike > > Regards > Christian Ok thank you :) From rick at havokmon.com Wed Nov 11 15:50:29 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 11 Nov 2015 09:50:29 -0600 Subject: OT - Re: hunting the fatty In-Reply-To: <673c398bcdf5c28c7a8ec65724672cce@valo.at> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> <673c398bcdf5c28c7a8ec65724672cce@valo.at> Message-ID: <20151111095029.Horde.EPATZmIsmsmea8UojGPhKg1@www.vfemail.net> LOL. This is a horrible subject line.? I've been trying to resolve a DDOS issue, and ignoring a lot of email.? Here I had been thinking this was a sex-spam, and I just got around to wondering why the spam system isn't working quite right and they kept coming in.? :P Not sure if this was suggested, but use SSD's.? With a ton of mail combined with Maildir random reads/writes, SSD's are like night and day. Rick From mancyborg at gmail.com Wed Nov 11 15:59:35 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Wed, 11 Nov 2015 16:59:35 +0100 Subject: OT - Re: hunting the fatty In-Reply-To: <20151111095029.Horde.EPATZmIsmsmea8UojGPhKg1@www.vfemail.net> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> <673c398bcdf5c28c7a8ec65724672cce@valo.at> <20151111095029.Horde.EPATZmIsmsmea8UojGPhKg1@www.vfemail.net> Message-ID: <20151111165935.131db304@linux> On Wed, 11 Nov 2015 09:50:29 -0600 Rick Romero wrote: > LOL. > > This is a horrible subject line.? I've been trying to resolve a DDOS > issue, and ignoring a lot of email.? > > Here I had been thinking this was a sex-spam, and I just got around to > wondering why the spam system isn't working quite right and they kept > coming in.? :P > > Not sure if this was suggested, but use SSD's.? With a ton of mail > combined with Maildir random reads/writes, SSD's are like night and day. > > Rick Very sorry about nasty subject ;) Btw is it worth the offload on i/o by having some Dovecot index or cache on an SSD ? From rick at havokmon.com Wed Nov 11 16:20:07 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 11 Nov 2015 10:20:07 -0600 Subject: OT - Re: hunting the fatty In-Reply-To: <20151111165935.131db304@linux> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> <673c398bcdf5c28c7a8ec65724672cce@valo.at> <20151111095029.Horde.EPATZmIsmsmea8UojGPhKg1@www.vfemail.net> <20151111165935.131db304@linux> Message-ID: <20151111102007.Horde.D1BXuRGs-EZSgQKsOWJssg1@www.vfemail.net> Quoting mancyborg at gmail.com: > On Wed, 11 Nov 2015 09:50:29 -0600 > Rick Romero wrote: > >> LOL. >> >> This is a horrible subject line.? I've been trying to resolve a DDOS >> issue, and ignoring a lot of email.? >> >> Here I had been thinking this was a sex-spam, and I just got around to >> wondering why the spam system isn't working quite right and they kept >> coming in.? :P >> >> Not sure if this was suggested, but use SSD's.? With a ton of mail >> combined with Maildir random reads/writes, SSD's are like night and day. >> >> Rick > > Very sorry about nasty subject ;) > Btw is it worth the offload on i/o by having some Dovecot index or cache > on an SSD ? Lately I've separated the indexes onto another SSD-based volume, just to save read/writes, but I don't think it actually improved anything. Rick From mancyborg at gmail.com Wed Nov 11 16:56:08 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Wed, 11 Nov 2015 17:56:08 +0100 Subject: OT - Re: hunting the fatty In-Reply-To: <20151111102007.Horde.D1BXuRGs-EZSgQKsOWJssg1@www.vfemail.net> References: <20151110014441.065cfb4b@linux> <2e2f2d5602081281bd81b918722cd0c4@valo.at> <20151111034422.71ede74e@linux> <673c398bcdf5c28c7a8ec65724672cce@valo.at> <20151111095029.Horde.EPATZmIsmsmea8UojGPhKg1@www.vfemail.net> <20151111165935.131db304@linux> <20151111102007.Horde.D1BXuRGs-EZSgQKsOWJssg1@www.vfemail.net> Message-ID: <20151111175608.270fcb5f@linux> On Wed, 11 Nov 2015 10:20:07 -0600 Rick Romero wrote: > Quoting mancyborg at gmail.com: > > > On Wed, 11 Nov 2015 09:50:29 -0600 > > Rick Romero wrote: > > > >> LOL. > >> > >> This is a horrible subject line.? I've been trying to resolve a DDOS > >> issue, and ignoring a lot of email.? > >> > >> Here I had been thinking this was a sex-spam, and I just got around to > >> wondering why the spam system isn't working quite right and they kept > >> coming in.? :P > >> > >> Not sure if this was suggested, but use SSD's.? With a ton of mail > >> combined with Maildir random reads/writes, SSD's are like night and day. > >> > >> Rick > > > > Very sorry about nasty subject ;) > > Btw is it worth the offload on i/o by having some Dovecot index or cache > > on an SSD ? > > Lately I've separated the indexes onto another SSD-based volume, just to > save read/writes, but I don't think it actually improved anything. > > Rick I'm going to try dm-cache and flashcache asap .. From frust at iti.cs.tu-bs.de Thu Nov 12 07:03:17 2015 From: frust at iti.cs.tu-bs.de (Frank Rust) Date: Thu, 12 Nov 2015 08:03:17 +0100 Subject: Problems after upgrade from 2.0 to 2.2 Message-ID: Hi all, after upgrade from some 2.0 version to 2.2.19 (debian) i face map login problems: # doveconf -n # 2.2.19 (ca91d540fd87): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 # OS: Linux 2.6.32-5-amd64 x86_64 Debian 8.2 ext4 auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot-debug.log hostname = test.my.domain.de info_log_path = /var/log/dovecot.log mail_access_groups = smmta mail_debug = yes mail_location = mbox:/extra/mail/mbox.dir/%u/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/%u/index:CONTROL=/extra/mail/mbox.dir/%u/control:INBOX=/extra/mail/%u mail_plugins = quota mail_privileged_group = smmta managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append quota = fs:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = file:~/sieve;active=~/.dovecot.sieve } postmaster_address = postmaster at my.domain.de protocols = " imap sieve pop3" quota_full_tempfail = yes ssl = required ssl_cert = , method=PLAIN, rip=134.169.39.105, lip=134.169.39.22, mpid=9475, TLS, session=<...> dovecot-debug.log: Nov 12 07:47:27 imap-login: Debug: SSL: elliptic curve secp384r1 will be used for ECDH and ECDHE key exchanges Nov 12 07:47:27 auth: Debug: auth client connected (pid=11158) Nov 12 07:47:27 imap-login: Debug: SSL: where=0x10, ret=1: before/accept initialization [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: before/accept initialization [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2002, ret=-1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2002, ret=-1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2002, ret=-1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2001, ret=1: unknown state [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x20, ret=1: SSL negotiation finished successfully [134.169.39.105] Nov 12 07:47:27 imap-login: Debug: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [134.169.39.105] Nov 12 07:47:27 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=QkLwUFIkXsCGqSdp lip=134.169.39.22 rip=134.169.39.105 lport=993 rport=49246 resp=xxxx (previous base64 data removed) Nov 12 07:47:27 auth-worker(11140): Debug: pam(frank,134.169.39.105): lookup service=dovecot Nov 12 07:47:27 auth-worker(11140): Debug: pam(frank,134.169.39.105): #1/1 style=1 msg=Password: Nov 12 07:47:27 auth: Debug: client passdb out: OK 1 user=frank Nov 12 07:47:27 auth: Debug: master in: REQUEST 3038248961 11158 1 4e2362c2872f9288b7c8c3ebd8469620 session_pid=11159 request_auth_token Nov 12 07:47:27 auth-worker(11140): Debug: pam(frank,134.169.39.105): lookup Nov 12 07:47:27 auth: Debug: master userdb out: USER 3038248961 frank system_groups_user=frank uid=1000 gid=100 home=/home/frank auth_token=1e04d2c61fb327ec14c1e54ff678d73a325d20bf Nov 12 07:47:27 imap(frank): Debug: Loading modules from directory: /usr/lib/dovecot/modules Nov 12 07:47:27 imap(frank): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Nov 12 07:47:27 imap(frank): Debug: Effective uid=1000, gid=100, home=/home/frank Nov 12 07:47:27 imap(frank): Debug: Quota root: name=User quota backend=fs args= Nov 12 07:47:27 imap(frank): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 Nov 12 07:47:27 imap(frank): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Nov 12 07:47:27 imap(frank): Debug: Quota warning: bytes=1020054732 (95%) messages=0 reverse=no command=quota-warning 95 frank Nov 12 07:47:27 imap(frank): Debug: Quota warning: bytes=858993459 (80%) messages=0 reverse=no command=quota-warning 80 frank Nov 12 07:47:27 imap(frank): Debug: Quota grace: root=User quota bytes=107374182 (10%) Nov 12 07:47:27 imap(frank): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location=mbox:/extra/mail/mbox.dir/frank/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/frank/index:CONTROL=/extra/mail/mbox.dir/frank/control:INBOX=/extra/mail/frank Nov 12 07:47:27 imap(frank): Debug: fs: root=/extra/mail/mbox.dir/frank/mailboxes, index=/extra/mail/mbox.dir/frank/index, indexpvt=, control=/extra/mail/mbox.dir/frank/control, inbox=/extra/mail/frank, alt= Nov 12 07:47:27 imap(frank): Debug: fs quota add mailbox dir = /extra/mail/mbox.dir/frank/mailboxes Nov 12 07:47:27 imap(frank): Debug: fs quota block device = /dev/mapper/ownCloudVG-ownCloudData Nov 12 07:47:27 imap(frank): Debug: fs quota mount point = /extra Nov 12 07:47:27 imap(frank): Debug: fs quota mount type = ext4 Nov 12 07:47:27 imap-login: Debug: SSL alert: close notify [134.169.39.105] What could cause the problem? The problem is not quota related (I enhanced the allowed quota and got the same results) best regards, Frank From ml+dovecot at valo.at Thu Nov 12 07:58:45 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Thu, 12 Nov 2015 08:58:45 +0100 Subject: Problems after upgrade from 2.0 to 2.2 In-Reply-To: References: Message-ID: <456329dfe8017e2f58905ceb9bcbf7a6@valo.at> Hi, On 2015-11-12 08:03, Frank Rust wrote: > Hi all, > after upgrade from some 2.0 version to 2.2.19 (debian) i face map > login problems: > > # doveconf -n > # 2.2.19 (ca91d540fd87): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.9 > # OS: Linux 2.6.32-5-amd64 x86_64 Debian 8.2 ext4 > auth_debug = yes > auth_debug_passwords = yes > auth_verbose = yes > auth_verbose_passwords = plain > debug_log_path = /var/log/dovecot-debug.log > hostname = test.my.domain.de > info_log_path = /var/log/dovecot.log > mail_access_groups = smmta > mail_debug = yes > mail_location = > mbox:/extra/mail/mbox.dir/%u/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/%u/index:CONTROL=/extra/mail/mbox.dir/%u/control:INBOX=/extra/mail/%u > mail_plugins = quota > mail_privileged_group = smmta > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate > namespace inbox { > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } Missing inbox = yes in your default namespace. See wiki http://wiki2.dovecot.org/Namespaces > passdb { > driver = pam > } > plugin { > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename flag_change append > quota = fs:User quota > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+100M > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = file:~/sieve;active=~/.dovecot.sieve > } > postmaster_address = postmaster at my.domain.de > protocols = " imap sieve pop3" > quota_full_tempfail = yes > ssl = required > ssl_cert = ssl_cipher_list = > EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA > EECDH RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 > userdb { > driver = passwd > } > verbose_ssl = yes > protocol lda { > mail_plugins = > } > > [ snip ] > Nov 12 07:47:27 imap(frank): Debug: Namespace inbox: type=private, > prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes ^^^^ inbox not set. > location=mbox:/extra/mail/mbox.dir/frank/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/frank/index:CONTROL=/extra/mail/mbox.dir/frank/control:INBOX=/extra/mail/frank > Nov 12 07:47:27 imap(frank): Debug: fs: > root=/extra/mail/mbox.dir/frank/mailboxes, > index=/extra/mail/mbox.dir/frank/index, indexpvt=, > control=/extra/mail/mbox.dir/frank/control, inbox=/extra/mail/frank, > alt= > Nov 12 07:47:27 imap(frank): Debug: fs quota add mailbox dir = > /extra/mail/mbox.dir/frank/mailboxes > Nov 12 07:47:27 imap(frank): Debug: fs quota block device = > /dev/mapper/ownCloudVG-ownCloudData > Nov 12 07:47:27 imap(frank): Debug: fs quota mount point = /extra > Nov 12 07:47:27 imap(frank): Debug: fs quota mount type = ext4 > Nov 12 07:47:27 imap-login: Debug: SSL alert: close notify > [134.169.39.105] > > What could cause the problem? > > The problem is not quota related (I enhanced the allowed quota and got > the same results) > > > best regards, > Frank regards christian From frust at iti.cs.tu-bs.de Thu Nov 12 08:33:50 2015 From: frust at iti.cs.tu-bs.de (Frank Rust) Date: Thu, 12 Nov 2015 09:33:50 +0100 Subject: Problems after upgrade from 2.0 to 2.2 In-Reply-To: <456329dfe8017e2f58905ceb9bcbf7a6@valo.at> References: <456329dfe8017e2f58905ceb9bcbf7a6@valo.at> Message-ID: <9E2DAAB8-BD83-4014-BE32-5177BDC3D8BD@iti.cs.tu-bs.de> Thanks a lot! This was added by a new debian package file! They added the 15-mailboxes.conf file with this input-namespace. It?s a bad thing. For changed configuration the installer asks which version, old or new to take, but new files are installed without confirmation. But that?s a debian problem, not dovecot. Best regards, Frank > Am 12.11.2015 um 08:58 schrieb Christian Kivalo : > > Hi, > > On 2015-11-12 08:03, Frank Rust wrote: >> Hi all, >> after upgrade from some 2.0 version to 2.2.19 (debian) i face map >> login problems: >> # doveconf -n >> # 2.2.19 (ca91d540fd87): /etc/dovecot/dovecot.conf >> # Pigeonhole version 0.4.9 >> # OS: Linux 2.6.32-5-amd64 x86_64 Debian 8.2 ext4 >> auth_debug = yes >> auth_debug_passwords = yes >> auth_verbose = yes >> auth_verbose_passwords = plain >> debug_log_path = /var/log/dovecot-debug.log >> hostname = test.my.domain.de >> info_log_path = /var/log/dovecot.log >> mail_access_groups = smmta >> mail_debug = yes >> mail_location = >> mbox:/extra/mail/mbox.dir/%u/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/%u/index:CONTROL=/extra/mail/mbox.dir/%u/control:INBOX=/extra/mail/%u >> mail_plugins = quota >> mail_privileged_group = smmta >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date index ihave duplicate >> namespace inbox { >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> } > > Missing inbox = yes in your default namespace. > > See wiki http://wiki2.dovecot.org/Namespaces > >> passdb { >> driver = pam >> } >> plugin { >> mail_log_events = delete undelete expunge copy mailbox_delete >> mailbox_rename flag_change append >> quota = fs:User quota >> quota_rule = *:storage=1G >> quota_rule2 = Trash:storage=+100M >> quota_warning = storage=95%% quota-warning 95 %u >> quota_warning2 = storage=80%% quota-warning 80 %u >> sieve = file:~/sieve;active=~/.dovecot.sieve >> } >> postmaster_address = postmaster at my.domain.de >> protocols = " imap sieve pop3" >> quota_full_tempfail = yes >> ssl = required >> ssl_cert = > ssl_cipher_list = >> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA >> EECDH RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS >> ssl_key = > ssl_prefer_server_ciphers = yes >> ssl_protocols = !SSLv2 !SSLv3 >> userdb { >> driver = passwd >> } >> verbose_ssl = yes >> protocol lda { >> mail_plugins = >> } > > [ snip ] > >> Nov 12 07:47:27 imap(frank): Debug: Namespace inbox: type=private, >> prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes > ^^^^ > inbox not set. > >> location=mbox:/extra/mail/mbox.dir/frank/mailboxes:DIRNAME=mBoX-MeSsAgEs:INDEX=/extra/mail/mbox.dir/frank/index:CONTROL=/extra/mail/mbox.dir/frank/control:INBOX=/extra/mail/frank >> Nov 12 07:47:27 imap(frank): Debug: fs: >> root=/extra/mail/mbox.dir/frank/mailboxes, >> index=/extra/mail/mbox.dir/frank/index, indexpvt=, >> control=/extra/mail/mbox.dir/frank/control, inbox=/extra/mail/frank, >> alt= >> Nov 12 07:47:27 imap(frank): Debug: fs quota add mailbox dir = >> /extra/mail/mbox.dir/frank/mailboxes >> Nov 12 07:47:27 imap(frank): Debug: fs quota block device = >> /dev/mapper/ownCloudVG-ownCloudData >> Nov 12 07:47:27 imap(frank): Debug: fs quota mount point = /extra >> Nov 12 07:47:27 imap(frank): Debug: fs quota mount type = ext4 >> Nov 12 07:47:27 imap-login: Debug: SSL alert: close notify [134.169.39.105] >> What could cause the problem? >> The problem is not quota related (I enhanced the allowed quota and got >> the same results) >> best regards, >> Frank > > regards > christian From ml+dovecot at valo.at Thu Nov 12 08:39:05 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Thu, 12 Nov 2015 09:39:05 +0100 Subject: Problems after upgrade from 2.0 to 2.2 In-Reply-To: <9E2DAAB8-BD83-4014-BE32-5177BDC3D8BD@iti.cs.tu-bs.de> References: <456329dfe8017e2f58905ceb9bcbf7a6@valo.at> <9E2DAAB8-BD83-4014-BE32-5177BDC3D8BD@iti.cs.tu-bs.de> Message-ID: <1597195c8bc9ab49c5123f870cf58485@valo.at> On 2015-11-12 09:33, Frank Rust wrote: > Thanks a lot! > > This was added by a new debian package file! They added the > 15-mailboxes.conf file with this input-namespace. > It?s a bad thing. For changed configuration the installer asks which > version, old or new to take, but new files are installed without > confirmation. But that?s a debian problem, not dovecot. The 15-mailboxes.conf is also present when you compile dovecot yourself. > Best regards, > Frank regards christian From hs at schlittermann.de Thu Nov 12 10:56:44 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Thu, 12 Nov 2015 11:56:44 +0100 Subject: fail: doveadm backup -s "" with a huuge number of mailboxes Message-ID: <20151112105644.GB5773@jumper.schlittermann.de> Hello, I'm using 2.2.13. For syncing I'm using repetitions of about the following (simplified for readability): state_file=.s doveadm \ -o imapc_master_user= \ -o imapc_password= \ -o imapc_host= \ -o mail_fsync=never \ backup -s "$(<.s)" -u "" -R imapc: >.s_new mv .s_new .s The user has about 4501 IMAP folders. This makes a huge state output of about 128 kB. This in turn leads to an overly long command line in the next iteration. (excuse: command line too long) Is there any chance to solve this? I think, the only possibility would be to have doveadm-backup -s read the state from a *file*, wouldn't it? (Apart from the fact that I do not see a manpage for this, except entries in the Wiki (but this may be due to poor packaging) of the Debian dovecot package.) Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From lists at searchy.net Thu Nov 12 20:35:07 2015 From: lists at searchy.net (Frank de Bot (lists)) Date: Thu, 12 Nov 2015 21:35:07 +0100 Subject: FTS, multiple virtualmailbox entries for single mailbox Message-ID: <5644F7FB.6060301@searchy.net> Hi, i have a dovecot setup with virtual users. Each user has 2 entries for his mailbox. One is with the username formatted as %username%@%default_realm% , the other %username%@%domain% I've done some tests with it, but I don't complete understand how it works - I've emptied all document in solr so it empty. Doveadm is instructed to do the same with 'doveadm -D -v fts rescan -A' - I reindex %username%@%default_realm% with 'doveadm -D index -u %username%@%default_realm% '*' - With fluke I can see 4 indexes documents. - I reindex %username%@%domain% - No documents are added to Solr, dovecot finds it's up-to-date - After 'doveadm -D -v fts rescan -A' the documents are added The part I don't understand is that a certain key used for indexing, but dovecot checks based on something else. If find this inconsistent. How can I manage to get virtualmailboxes where multiple logins are allowed on 1 mailbox and FTS is working? I think it won't work well this way if a user logs in with only a certain username with the default realm. Message received by dovecot will be indexed in Solr with %username%@%domain% , but there will be no indexed documents for %username%@%default_realm% A earlier threads suggested to be able to override the username for FTS, but it was tought to be not a good idea. Regards, Frank de Bot From tss at iki.fi Thu Nov 12 22:12:21 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 13 Nov 2015 00:12:21 +0200 Subject: fail: doveadm backup -s "" with a huuge number of mailboxes In-Reply-To: <20151112105644.GB5773@jumper.schlittermann.de> References: <20151112105644.GB5773@jumper.schlittermann.de> Message-ID: On 12 Nov 2015, at 12:56, Heiko Schlittermann wrote: > > Hello, > > I'm using 2.2.13. > > For syncing I'm using repetitions of about the > following (simplified for readability): > > state_file=.s > doveadm \ > -o imapc_master_user= \ > -o imapc_password= \ > -o imapc_host= \ > -o mail_fsync=never \ > backup -s "$(<.s)" -u "" -R imapc: >.s_new > mv .s_new .s > > The user has about 4501 IMAP folders. This makes a huge > state output of about 128 kB. This in turn leads to an overly long > command line in the next iteration. (excuse: command line too long) > > Is there any chance to solve this? I think, the only possibility would > be to have doveadm-backup -s read the state from a *file*, wouldn't it? Right now no way to read it from a file. But is this Linux? I thought Linux had unlimited (or at least very large) command line length nowadays. From mfoley at ohprs.org Thu Nov 12 22:13:50 2015 From: mfoley at ohprs.org (Mark Foley) Date: Thu, 12 Nov 2015 17:13:50 -0500 Subject: How to Restore emails Message-ID: <201511122213.tACMDoDj025630@mail.hprs.local> I have a user who accidentially deleted a large number of emails prior to a certain date. He wants them back. I do have a tarfile backup of these messages. Is there a good way to restore them? Can I simply restore them to the appropriate Maildir directory and dovecot will just "figure it out"? Thanks, --Mark From hs at schlittermann.de Thu Nov 12 22:30:29 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Thu, 12 Nov 2015 23:30:29 +0100 Subject: fail: doveadm backup -s "" with a huuge number of mailboxes In-Reply-To: References: <20151112105644.GB5773@jumper.schlittermann.de> Message-ID: <20151112223029.GB4735@jumper.schlittermann.de> Timo Sirainen (Do 12 Nov 2015 23:12:21 CET): ? > > The user has about 4501 IMAP folders. This makes a huge > > state output of about 128 kB. This in turn leads to an overly long > > command line in the next iteration. (excuse: command line too long) > > > > Is there any chance to solve this? I think, the only possibility would > > be to have doveadm-backup -s read the state from a *file*, wouldn't it? > > Right now no way to read it from a file. But is this Linux? I thought Linux had unlimited (or at least very large) command line length nowadays. Yes, the product of size and number of the arguments is limited: *** 189961 args ( 10 chars each) with total length 1899610 *** 99503 args ( 20 chars each) with total length 1990060 *** 2087 args ( 1000 chars each) with total length 2087000 *** 16 args (128000 chars each) with total length 2048000 The upper limit for the sum of parameters seems to be ~2 MB But, x127=$(perl -e 'print "x" x (127 * 2**10)') # 127 KB x128=$(perl -e 'print "x" x (128 * 2**10)') # 128 KB /bin/echo $x127 # works /bin/echo $x127 $x127 # works /bin/echo $x128 # breaks /bin/echo $x128 $x128 # breaks It seems that a *single* command line parameter is limited to somewhat near 128 kB. The complete amount of command line data is near 2 MB. Conclusion: doveadm sync should be able to read a state *file*. IMHO Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From mfoley at ohprs.org Thu Nov 12 22:31:39 2015 From: mfoley at ohprs.org (Mark Foley) Date: Thu, 12 Nov 2015 17:31:39 -0500 Subject: How to Restore emails In-Reply-To: <201511122213.tACMDoDj025630@mail.hprs.local> References: <201511122213.tACMDoDj025630@mail.hprs.local> Message-ID: <201511122231.tACMVd1R028712@mail.hprs.local> According to a message to this list from Oli Schacher, http://www.dovecot.org/list/dovecot/2011-June/059493.html, all I need to do is copy the deleted emails to their original folder and dovecot will take care of it: On Tue Jun 7 08:43:54 EEST 2011 Oli Schacher wrote: >> OK, but if I want to restore a particular mail from backups, what're >> the steps involved? What I mean is do I just copy and paste that mail >> file from backups to the user account maildir folder and what program >> should I run to synchronize it. > > > exactly, just copy the mail from your backup back into the users > maildir (usually into 'cur'). Make sure the permissions of the restored > file are correct. No need to synchronize anything, dovecot automatically > detects the added message. Unless someone out there advises me against doing this, I'm going to try it. I've also seen `dovadm import` as a possible suggestion, though if a simple copy works I don't see why anyone would use `dovadm import`. --Mark -----Original Message----- From: Mark Foley Date: Thu, 12 Nov 2015 17:13:50 -0500 To: dovecot at dovecot.org Subject: How to Restore emails I have a user who accidentially deleted a large number of emails prior to a certain date. He wants them back. I do have a tarfile backup of these messages. Is there a good way to restore them? Can I simply restore them to the appropriate Maildir directory and dovecot will just "figure it out"? Thanks, --Mark From hs at schlittermann.de Thu Nov 12 22:36:52 2015 From: hs at schlittermann.de (Heiko Schlittermann) Date: Thu, 12 Nov 2015 23:36:52 +0100 Subject: How to Restore emails In-Reply-To: <201511122231.tACMVd1R028712@mail.hprs.local> References: <201511122213.tACMDoDj025630@mail.hprs.local> <201511122231.tACMVd1R028712@mail.hprs.local> Message-ID: <20151112223652.GC4735@jumper.schlittermann.de> Hi, Mark Foley (Do 12 Nov 2015 23:31:39 CET): > According to a message to this list from Oli Schacher, > http://www.dovecot.org/list/dovecot/2011-June/059493.html, all I need to do is copy the deleted > emails to their original folder and dovecot will take care of it: > ? > > exactly, just copy the mail from your backup back into the users > > maildir (usually into 'cur'). Make sure the permissions of the restored > > file are correct. No need to synchronize anything, dovecot automatically > > detects the added message. About the 'cur' I'd not be too sure. If you *mv* the files there from a directory on the same filesystem, you should be fine, but if copy the files, I'd be careful. Probably you want to mimic the maildir behaviour: copy the files to tmp/ mv the files to cur/ (not sure, if new/ would be fine to, because new/ is the natural place after tmp/. I'm not sure, what this does to the message state the client sees.) Best regards from Dresden/Germany Viele Gr??e aus Dresden Heiko Schlittermann -- SCHLITTERMANN.de ---------------------------- internet & unix support - Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - gnupg encrypted messages are welcome --------------- key ID: F69376CE - ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: Digital signature URL: From mfoley at ohprs.org Fri Nov 13 02:07:37 2015 From: mfoley at ohprs.org (Mark Foley) Date: Thu, 12 Nov 2015 21:07:37 -0500 Subject: How to Restore emails In-Reply-To: <20151112223652.GC4735@jumper.schlittermann.de> References: <201511122213.tACMDoDj025630@mail.hprs.local> <201511122231.tACMVd1R028712@mail.hprs.local> <20151112223652.GC4735@jumper.schlittermann.de> Message-ID: <201511130207.tAD27b4u014410@mail.hprs.local> Thanks for the reply Heiko > About the 'cur' I'd not be too sure. Well, I'm just restoring to .Deleted Items, so it shouldn't be a problem. The tar restore should preserve names, permission, etc. I'll backup the current Maildir, the try the restore and see what happens. I'll report back. --Mark -----Original Message----- > Date: Thu, 12 Nov 2015 23:36:52 +0100 > From: Heiko Schlittermann > To: dovecot at dovecot.org > Subject: Re: How to Restore emails > > Hi, > > Mark Foley (Do 12 Nov 2015 23:31:39 CET): > > According to a message to this list from Oli Schacher, > > http://www.dovecot.org/list/dovecot/2011-June/059493.html, all I need to do is copy the deleted > > emails to their original folder and dovecot will take care of it: > > > ??? > > > exactly, just copy the mail from your backup back into the users > > > maildir (usually into 'cur'). Make sure the permissions of the restored > > > file are correct. No need to synchronize anything, dovecot automatically > > > detects the added message. > > About the 'cur' I'd not be too sure. If you *mv* the files there from a > directory on the same filesystem, you should be fine, but if copy the > files, I'd be careful. Probably you want to mimic the maildir behaviour: > > copy the files to tmp/ > mv the files to cur/ (not sure, if new/ would be fine to, > because new/ is the natural place after > tmp/. I'm not sure, what this does to the > message state the client sees.) > > Best regards from Dresden/Germany > Viele Gr??e aus Dresden > Heiko Schlittermann > -- > SCHLITTERMANN.de ---------------------------- internet & unix support - > Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - > gnupg encrypted messages are welcome --------------- key ID: F69376CE - > ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - From rblayzor.bulk at inoc.net Fri Nov 13 03:45:22 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Thu, 12 Nov 2015 22:45:22 -0500 Subject: lmtp seg fault Message-ID: dovecot -n # 2.2.19: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: FreeBSD 10.2-RELEASE-p7 amd64 doveconf: Warning: Dovecot was last started using /etc/dovecot/dovecot.conf, but this config is /usr/local/etc/dovecot/dovecot.conf auth_cache_size = 10 M auth_cache_ttl = 5 mins auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 10 base_dir = /var/run/dovecot/ default_client_limit = 2000 default_process_limit = 200 disable_plaintext_auth = no first_valid_uid = 25 imap_idle_notify_interval = 4 mins log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Ready to serve. mail_fsync = always mail_location = maildir:~/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_temp_dir = /var/tmp mailbox_list_index = yes maildir_very_dirty_syncs = yes managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body environment mailbox date index ihave duplicate mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = maildir:User quota quota_exceeded_message = Quota exceeded (Mailbox is full) quota_rule = *:storage=1G quota_rule2 = Trash:ignore sieve = ~/.dovecot.sieve sieve_default = /etc/dovecot/sieve/global/JunkFolder.sieve sieve_dir = ~/.sieve sieve_extensions = -enotify sieve_global_dir = /etc/dovecot/sieve/global/ zlib_save = xz zlib_save_level = 6 } pop3_fast_size_lookups = yes pop3_save_uidl = yes protocols = imap pop3 lmtp sieve rejection_reason = Your message to <%t> was rejected:%n%n %r%n rejection_subject = [BOUNCE] %s service imap-login { inet_listener imap { port = 143 } process_min_avail = 4 service_count = 100 } service imap { executable = imap last-login } service last-login { executable = script-login /etc/dovecot/post-login.sh user = mailnull } service lmtp { process_min_avail = 4 unix_listener lmtp { mode = 0666 } user = mailnull } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service managesieve { process_limit = 100 } service pop3-login { inet_listener pop3 { port = 110 } process_min_avail = 4 service_count = 100 } service pop3 { executable = pop3 last-login } ssl = no ssl_cipher_list = ALL:!SSLv2:!LOW:!ADH:!NULL:!DES:@STRENGTH ssl_protocols = !SSLv2 !SSLv3 submission_host = localhost:25 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol lmtp { mail_plugins = zlib quota sieve } protocol lda { mail_plugins = zlib quota } protocol imap { mail_max_userip_connections = 8 mail_plugins = zlib quota imap_quota } protocol pop3 { mail_max_userip_connections = 2 mail_plugins = zlib quota } protocol sieve { mail_max_userip_connections = 3 managesieve_implementation_string = Dovecot managesieve_max_line_length = 65536 } GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Core was generated by `lmtp'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/dovecot/libdovecot-lda.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-lda.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/local/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/local/lib/dovecot/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.6 Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/liblzma.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/liblzma.so.5 Reading symbols from /usr/local/lib/dovecot/lib90_sieve_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot/lib90_sieve_plugin.so Reading symbols from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000010dd08c0 in buffer_free () from /usr/local/lib/dovecot/libdovecot.so.0 (gdb) bt #0 0x0000000010dd08c0 in buffer_free () from /usr/local/lib/dovecot/libdovecot.so.0 #1 0x0000000010b12908 in mail_transaction_log_file_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #2 0x0000000010b165e0 in mail_transaction_log_find_file () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #3 0x0000000010b149d7 in mail_transaction_log_view_set () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #4 0x0000000010b03f4a in mail_index_sync_map () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #5 0x0000000010afb288 in mail_index_map () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #6 0x0000000010b11245 in mail_index_reset_error () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #7 0x0000000010b109ad in mail_index_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #8 0x0000000010b10853 in mail_index_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #9 0x0000000010aebd0b in index_storage_mailbox_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #10 0x0000000010aa3aec in maildir_is_backend_readonly () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #11 0x0000000010aa30f1 in maildir_is_backend_readonly () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #12 0x0000000011e15052 in zlib_plugin_deinit () from /usr/local/lib/dovecot/lib20_zlib_plugin.so #13 0x0000000010a7cfe2 in mailbox_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #14 0x0000000010a7cee4 in mailbox_open () from /usr/local/lib/dovecot/libdovecot-storage.so.0 #15 0x0000000010832330 in mail_deliver_save_open () from /usr/local/lib/dovecot/libdovecot-lda.so.0 #16 0x00000000128b4e1f in sieve_action_opr_optional_read () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 #17 0x00000000128ac9b3 in sieve_result_execute () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 #18 0x00000000128bc5bc in sieve_execute () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 #19 0x000000001266b06e in sieve_plugin_init () from /usr/local/lib/dovecot/lib90_sieve_plugin.so #20 0x00000000108329aa in mail_deliver () from /usr/local/lib/dovecot/libdovecot-lda.so.0 #21 0x00000000004087f1 in cmd_xclient () #22 0x0000000000407fbf in cmd_data () #23 0x0000000010de7bd3 in io_loop_call_io () from /usr/local/lib/dovecot/libdovecot.so.0 #24 0x0000000010de96ef in io_loop_handler_run_internal () from /usr/local/lib/dovecot/libdovecot.so.0 #25 0x0000000010de80d4 in io_loop_handler_run () from /usr/local/lib/dovecot/libdovecot.so.0 #26 0x0000000010de7eb8 in io_loop_run () from /usr/local/lib/dovecot/libdovecot.so.0 #27 0x0000000010d841d8 in master_service_run () from /usr/local/lib/dovecot/libdovecot.so.0 #28 0x0000000000405531 in main () From skdovecot at smail.inf.fh-brs.de Fri Nov 13 08:08:55 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 13 Nov 2015 09:08:55 +0100 (CET) Subject: How to Restore emails In-Reply-To: <201511130207.tAD27b4u014410@mail.hprs.local> References: <201511122213.tACMDoDj025630@mail.hprs.local> <201511122231.tACMVd1R028712@mail.hprs.local> <20151112223652.GC4735@jumper.schlittermann.de> <201511130207.tAD27b4u014410@mail.hprs.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 12 Nov 2015, Mark Foley wrote: >> About the 'cur' I'd not be too sure. > > Well, I'm just restoring to .Deleted Items, so it shouldn't be a problem. The tar restore > should preserve names, permission, etc. I'll backup the current Maildir, the try the restore > and see what happens. I'll report back. a problem will arise, if Dovecot indexes the directory when the file is not fully restorred, hence, changes later. Therefore, either prevent any access in Dovecot (deliver and retrieval) or use Heiko's approach, because mv is atomic on the same filesystem (rename instead of copy). The next problem comes with duplicates, because Maildir saves flags, keywords and status in the filename. I, therefore, copy a backup to another subdir, say "tmp2", then run fdupes (or similiar program) over cur, new and tmp2 to find duplicates, delete them in tmp2, and finally "mv -i " (you never know ;-) ) the remaining files from tmp2 to cur or new. > -----Original Message----- >> Date: Thu, 12 Nov 2015 23:36:52 +0100 >> From: Heiko Schlittermann >> To: dovecot at dovecot.org >> Subject: Re: How to Restore emails >> >> Hi, >> >> Mark Foley (Do 12 Nov 2015 23:31:39 CET): >>> According to a message to this list from Oli Schacher, >>> http://www.dovecot.org/list/dovecot/2011-June/059493.html, all I need to do is copy the deleted >>> emails to their original folder and dovecot will take care of it: >>> >> ??? >>>> exactly, just copy the mail from your backup back into the users >>>> maildir (usually into 'cur'). Make sure the permissions of the restored >>>> file are correct. No need to synchronize anything, dovecot automatically >>>> detects the added message. >> >> About the 'cur' I'd not be too sure. If you *mv* the files there from a >> directory on the same filesystem, you should be fine, but if copy the >> files, I'd be careful. Probably you want to mimic the maildir behaviour: >> >> copy the files to tmp/ >> mv the files to cur/ (not sure, if new/ would be fine to, >> because new/ is the natural place after >> tmp/. I'm not sure, what this does to the >> message state the client sees.) >> >> Best regards from Dresden/Germany >> Viele Gr??e aus Dresden >> Heiko Schlittermann >> -- >> SCHLITTERMANN.de ---------------------------- internet & unix support - >> Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - >> gnupg encrypted messages are welcome --------------- key ID: F69376CE - >> ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVkWal3z1H7kL/d9rAQLcOwgAkkQTsr3/G3Hx8ic7Dbt+rq6nMEqbtaLJ xv0S3YstOn4cfnadSCb3uWoAa3CGtdpS99dgr7U09Xyx6Jbq2WNElHgumVZVoJUz qffFhMHl//QVPDDEHvsHJTnjuQEpLo6QAnd/2Sq6pJ5zQ4hL6Nf6WSJkEZ15GB8V Zl9s9lnMDR3zEnKUzpFSAE4QVaM8PwByU91h2tc61Y+kVGFLrSJYfgfsLsNT+T4C dC4YhSa85Y8nHj9grpk/aY1f8QndThRpSxDixMuLeYphY/Nc8XljAn986+7wA72Z TszSvNrzNVz0qKxy7q+raTJ66bpxrlEwNCH9Fb33rC5Zk7WDwtfHAw== =cBzu -----END PGP SIGNATURE----- From sergey.schwartz at bgoperator.com Fri Nov 13 09:40:44 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Fri, 13 Nov 2015 12:40:44 +0300 Subject: lmtp panic In-Reply-To: <56434B6E.7080902@bgoperator.com> References: <5641E721.9060008@bgoperator.com> <56434B6E.7080902@bgoperator.com> Message-ID: <5645B01C.3020209@bgoperator.com> root at mx10:~# doveadm force-resync -u oleg.vasilyev at bgoperator.com noreply-05-08-2015 doveadm(oleg.vasilyev at bgoperator.com): Warning: fscking index file /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage/dovecot.map.index doveadm(oleg.vasilyev at bgoperator.com): Warning: mdbox /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage: rebuilding indexes doveadm(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) doveadm(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f0f6f04e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x82158) [0x7f0f6f04e158] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0f6eff58de] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f0f6f3a6c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_compress+0xaad) [0x7f0f6f38c83d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c7) [0x7f0f6f39f4b7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x1109) [0x7f0f6f329189] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x825) [0x7f0f6f32aaf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) [0x7f0f6f32ac7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x42) [0x7f0f6f32ad32] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) [0x7f0f6f3106eb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7f0f6f310837] -> doveadm(+0x258e4) [0x7f0f6fc9d8e4] -> doveadm(+0x2547e) [0x7f0f6fc9d47e] -> doveadm(doveadm_mail_try_run+0x2ac) [0x7f0f6fc9e4dc] -> doveadm(main+0x3a5) [0x7f0f6fc8f245] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0f6ec28ec5] -> doveadm(+0x1734e) [0x7f0f6fc8f34e] root at mx10:~# doveconf -n # 2.2.19 (0b1c73b01a5a): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 # OS: Linux 3.13.0-34-generic x86_64 Ubuntu 14.04.3 LTS xfs auth_cache_negative_ttl = 0 auth_cache_size = 512 M auth_default_realm = bgoperator.com auth_failure_delay = 5 secs auth_worker_max_count = 4096 default_client_limit = 40960 default_process_limit = 8192 default_vsz_limit = 2 G disable_plaintext_auth = no first_valid_gid = 500 mail_location = mdbox:/srv/vmail/current/%d/%n mail_plugins = acl zlib notify replication mail_shared_explicit_inbox = yes mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mdbox_rotate_size = 32 M namespace { list = yes location = mdbox:/srv/vmail/current/%%d/%%n prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/virtual/base.conf.ext driver = sql } passdb { args = /etc/dovecot/virtual/master.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = file:/srv/vmail/current/shared.db/000-shared-mailboxes.db sieve = file:/srv/vmail/current/%d/%n/sieve;active=/srv/vmail/current/%d/%n/active.sieve sieve_max_redirects = 0 zlib_save = xz zlib_save_level = 6 } protocols = " imap lmtp sieve" service auth-worker { process_limit = 4096 } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { process_limit = 4096 process_min_avail = 24 } service imap { process_limit = 4096 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } ssl_cert = After 1 or 2 hours of recieveing this kind of errors mail server runs > out of file descriptors > > Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: Too > many open files > > > Best regards, > Sergey Schwartz > > Senior System Administrator > Biblio Globus Tour Operator > www.bgoperator.ru > > T: +7 495 5042500 ext 1532 > E: sergey.schwartz at bgoperator.com > > 10.11.2015 15:46, Sergey Schwartz ?????: >> Gents, >> >> >> >> I've just upgraded to the latest build of dovecot , now lmtp delivery >> process panics for just one user >> >> >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Panic: file str.c: line 22 (str_new_const): assertion failed: >> (str[len] == '\0') >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Panic: file str.c: line 22 (str_new_const): assertion failed: >> (str[len] == '\0') >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >> [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >> [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >> [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >> [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >> [0x7f50e380ddbd] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >> [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >> [0x7f50e380f555] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >> [0x7f50e37eb2e7] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >> [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >> [0x7f50e37fe6ea] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >> [0x7f50e37ff277] -> >> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >> [0x7f50e3a62b81] -> >> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >> [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> >> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >> [0x7f50e597ebbc] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >> [0x7f50e597ffb1] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >> [0x7f50e597ec45] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50e597ede8] >> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >> [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >> [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >> [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >> [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >> [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >> [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >> [0x7f887d2b0dbd] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >> [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >> [0x7f887d2b2555] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >> [0x7f887d28e2e7] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >> [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >> [0x7f887d2a16ea] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >> [0x7f887d2a2277] -> >> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >> [0x7f887d505b81] -> >> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >> [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> >> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >> [0x7f887f421bbc] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >> [0x7f887f422fb1] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >> [0x7f887f421c45] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f887f421de8] >> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >> [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >> [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] >> Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local >> Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Panic: file str.c: line 22 (str_new_const): assertion failed: >> (str[len] == '\0') >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >> [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >> [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >> [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >> [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >> [0x7fd26aef5dbd] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >> [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >> [0x7fd26aef7555] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >> [0x7fd26aed32e7] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >> [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >> [0x7fd26aee66ea] -> >> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >> [0x7fd26aee7277] -> >> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >> [0x7fd26b14ab81] -> >> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >> [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> >> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >> [0x7fd26d066bbc] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >> [0x7fd26d067fb1] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >> [0x7fd26d066c45] -> >> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd26d066de8] >> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >> [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> >> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >> [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Fatal: master: service(lmtp): child 7494 killed with signal 6 (core >> dumped) >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Fatal: master: service(lmtp): child 7497 killed with signal 6 (core >> dumped) >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Fatal: master: service(lmtp): child 7499 killed with signal 6 (core >> dumped) >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Fatal: master: service(lmtp): child 6719 killed with signal 6 (core >> dumped) >> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >> Fatal: master: service(lmtp): child 7502 killed with signal 6 (core >> dumped) >> Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: >> Connection closed (in DATA finished) >> >> >> Oleg has lots and lots of small emails :) >> >> I'm running ubuntu server 14.04 x86_64 >> dovecot 2.2.19 (0b1c73b01a5a) build from >> http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main >> From mfoley at ohprs.org Fri Nov 13 18:39:03 2015 From: mfoley at ohprs.org (Mark Foley) Date: Fri, 13 Nov 2015 13:39:03 -0500 Subject: How to Restore emails In-Reply-To: References: <201511122213.tACMDoDj025630@mail.hprs.local> <201511122231.tACMVd1R028712@mail.hprs.local> <20151112223652.GC4735@jumper.schlittermann.de> <201511130207.tAD27b4u014410@mail.hprs.local> Message-ID: <201511131839.tADId3PK026226@mail.hprs.local> On Fri, 13 Nov 2015 09:08:55 CET Steffen Kaiser wrote: > a problem will arise, if Dovecot indexes the directory when the file is > not fully restorred, hence, changes later. > > Therefore, either prevent any access in Dovecot (deliver and retrieval) or > use Heiko's approach, because mv is atomic on the same filesystem (rename > instead of copy). I did shut down Dovecot before restoring the files. > The next problem comes with duplicates, because Maildir saves flags, > keywords and status in the filename. Good point. I was only restoring the ".Deleted Items" mail folder and sub-folder, so I wasn't too concerned with the flags since they shouldn't really be changing much for deleted messages. Still, you are right, it is possible that the user could have altered the flags on some of his deleted messages (since he's keeping these around for months essentially as another email archive). I did a tar restore of that mail folder with the end-of-October full backup, and again with the incremental backup from the previous day. The user reported all message were restored OK. If he had changed flags he'd certainly end up with duplicates. Next time I will restore to a temp directory and check the flags and not restore files with the same name but different flags. --Mark -----Original Message----- > Date: Fri, 13 Nov 2015 09:08:55 +0100 (CET) > From: Steffen Kaiser > To: Mark Foley > Subject: Re: How to Restore emails > Cc: dovecot at dovecot.org > > On Thu, 12 Nov 2015, Mark Foley wrote: > > >> About the 'cur' I'd not be too sure. > > > > Well, I'm just restoring to .Deleted Items, so it shouldn't be a problem. The tar restore > > should preserve names, permission, etc. I'll backup the current Maildir, the try the restore > > and see what happens. I'll report back. > > a problem will arise, if Dovecot indexes the directory when the file is > not fully restorred, hence, changes later. > > Therefore, either prevent any access in Dovecot (deliver and retrieval) or > use Heiko's approach, because mv is atomic on the same filesystem (rename > instead of copy). > > The next problem comes with duplicates, because Maildir saves flags, > keywords and status in the filename. > > I, therefore, copy a backup to another subdir, say "tmp2", then run fdupes > (or similiar program) over cur, new and tmp2 to find duplicates, delete > them in tmp2, and finally "mv -i " (you never know ;-) ) the remaining > files from tmp2 to cur or new. > > > -----Original Message----- > >> Date: Thu, 12 Nov 2015 23:36:52 +0100 > >> From: Heiko Schlittermann > >> To: dovecot at dovecot.org > >> Subject: Re: How to Restore emails > >> > >> Hi, > >> > >> Mark Foley (Do 12 Nov 2015 23:31:39 CET): > >>> According to a message to this list from Oli Schacher, > >>> http://www.dovecot.org/list/dovecot/2011-June/059493.html, all I need to do is copy the deleted > >>> emails to their original folder and dovecot will take care of it: > >>> > >> ??? > >>>> exactly, just copy the mail from your backup back into the users > >>>> maildir (usually into 'cur'). Make sure the permissions of the restored > >>>> file are correct. No need to synchronize anything, dovecot automatically > >>>> detects the added message. > >> > >> About the 'cur' I'd not be too sure. If you *mv* the files there from a > >> directory on the same filesystem, you should be fine, but if copy the > >> files, I'd be careful. Probably you want to mimic the maildir behaviour: > >> > >> copy the files to tmp/ > >> mv the files to cur/ (not sure, if new/ would be fine to, > >> because new/ is the natural place after > >> tmp/. I'm not sure, what this does to the > >> message state the client sees.) > >> > >> Best regards from Dresden/Germany > >> Viele Gr??e aus Dresden > >> Heiko Schlittermann > >> -- > >> SCHLITTERMANN.de ---------------------------- internet & unix support - > >> Heiko Schlittermann, Dipl.-Ing. (TU) - {fon,fax}: +49.351.802998{1,3} - > >> gnupg encrypted messages are welcome --------------- key ID: F69376CE - > >> ! key id 7CBF764A and 972EAC9F are revoked since 2015-01 ------------ - > > > > - -- > Steffen Kaiser From fsantiago at garbage-juice.com Sun Nov 15 00:46:29 2015 From: fsantiago at garbage-juice.com (fsantiago at garbage-juice.com) Date: Sat, 14 Nov 2015 19:46:29 -0500 Subject: usage of recipient delimiters Message-ID: <8756157b8d733fe85f78541a155cfe51@garbage-juice.com> Hello, I have a forum system which can potentially have users send email to it for posting purposes that arrives addressed to a particular user account with a recipient delimiter: user+1a2b3c4d5e6f at domain.com The forum server is supposed to check the "user" account mailbox's inbox for the messages and post them to the site automatically. This doesn't work right now because when the messages arrive, they get placed into a newly and automatically created folder named after the alpha numeric string after the + in the address (eg... 1a2b3c4d5e6f). I feel dovecot is likely doing this. any ideas why and how i can shut this off and just allow them to arrive and stay in the inbox? My version is: 2.2.10 My config is: # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 4.1.5-x86_64-linode61 x86_64 CentOS Linux release 7.1.1503 (Core) auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } first_valid_uid = 2000 last_valid_uid = 2000 listen = * [::] log_path = /var/log/dovecot.log mail_gid = 2000 mail_location = maildir:%Lh/Maildir/:INDEX=%Lh/Maildir/ mail_plugins = quota mailbox_alias acl mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:%%Lh/Maildir/:INDEX=%%Lh/Maildir/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl auth_socket_path = /var/run/dovecot/auth-master mailbox_alias_new = Sent Messages mailbox_alias_new2 = Sent Items mailbox_alias_old = Sent mailbox_alias_old2 = Sent quota = dict:user::proxy::quotadict quota_rule = *:storage=1G quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = %Lh/sieve/dovecot.sieve sieve_before = /var/vmail/sieve/dovecot.sieve sieve_dir = %Lh/sieve sieve_global_dir = /var/vmail/sieve } protocols = pop3 imap sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { address = 127.0.0.1 port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { address = 127.0.0.1 port = 4190 } } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl = required ssl_cert = References: <8756157b8d733fe85f78541a155cfe51@garbage-juice.com> Message-ID: <46565331-04B0-49BC-9437-E0062A4B4970@valo.at> Hi, Am 15. November 2015 01:46:29 MEZ, schrieb fsantiago at garbage-juice.com: >Hello, > >I have a forum system which can potentially have users send email to it > >for posting purposes that arrives addressed to a particular user >account >with a recipient delimiter: > >user+1a2b3c4d5e6f at domain.com > >The forum server is supposed to check the "user" account mailbox's >inbox >for the messages and post them to the site automatically. This doesn't >work right now because when the messages arrive, they get placed into a > >newly and automatically created folder named after the alpha numeric >string after the + in the address (eg... 1a2b3c4d5e6f). > >I feel dovecot is likely doing this. any ideas why and how i can shut >this off and just allow them to arrive and stay in the inbox? > >My version is: > >2.2.10 > >My config is: > ># 2.2.10: /etc/dovecot/dovecot.conf ># OS: Linux 4.1.5-x86_64-linode61 x86_64 CentOS Linux release 7.1.1503 ... >userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql >} >protocol lda { > auth_socket_path = /var/run/dovecot/auth-master > lda_mailbox_autocreate = yes With this setting in 15-lda.conf nonexistant folders are automatically created > lda_mailbox_autosubscribe = yes > log_path = /var/log/dovecot-sieve.log > mail_plugins = quota mailbox_alias acl sieve > postmaster_address = root >} >protocol lmtp { > info_log_path = /var/log/dovecot-lmtp.log > lmtp_save_to_detail_mailbox = yes Tries to save to the detail mailbox when a recipient address contains the detail. look at in 20-lmtp.conf > mail_plugins = quota sieve > postmaster_address = postmaster > recipient_delimiter = + >} >protocol imap { > imap_client_workarounds = tb-extra-mailbox-sep > mail_max_userip_connections = 20 > mail_plugins = quota mailbox_alias acl imap_quota imap_acl >} >protocol pop3 { > mail_max_userip_connections = 20 > mail_plugins = quota mailbox_alias acl > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv >} > > >Thank you. > >- Fabian S. Regards Christian From fsantiago at garbage-juice.com Sun Nov 15 12:20:06 2015 From: fsantiago at garbage-juice.com (Fabian Santiago) Date: Sun, 15 Nov 2015 07:20:06 -0500 Subject: usage of recipient delimiters In-Reply-To: <46565331-04B0-49BC-9437-E0062A4B4970@valo.at> References: <8756157b8d733fe85f78541a155cfe51@garbage-juice.com> <46565331-04B0-49BC-9437-E0062A4B4970@valo.at> Message-ID: Thank you Christian. That led me right and it's working as I wish now. Sincerely, Fabian Santiago Sent from my iPhone > On Nov 15, 2015, at 4:25 AM, Christian Kivalo wrote: > > Hi, > > Am 15. November 2015 01:46:29 MEZ, schrieb fsantiago at garbage-juice.com: >> Hello, >> >> I have a forum system which can potentially have users send email to it >> >> for posting purposes that arrives addressed to a particular user >> account >> with a recipient delimiter: >> >> user+1a2b3c4d5e6f at domain.com >> >> The forum server is supposed to check the "user" account mailbox's >> inbox >> for the messages and post them to the site automatically. This doesn't >> work right now because when the messages arrive, they get placed into a >> >> newly and automatically created folder named after the alpha numeric >> string after the + in the address (eg... 1a2b3c4d5e6f). >> >> I feel dovecot is likely doing this. any ideas why and how i can shut >> this off and just allow them to arrive and stay in the inbox? >> >> My version is: >> >> 2.2.10 >> >> My config is: >> >> # 2.2.10: /etc/dovecot/dovecot.conf >> # OS: Linux 4.1.5-x86_64-linode61 x86_64 CentOS Linux release 7.1.1503 > > ... > >> userdb { >> args = /etc/dovecot/dovecot-mysql.conf >> driver = sql >> } >> protocol lda { >> auth_socket_path = /var/run/dovecot/auth-master >> lda_mailbox_autocreate = yes > > With this setting in 15-lda.conf nonexistant folders are automatically created > >> lda_mailbox_autosubscribe = yes >> log_path = /var/log/dovecot-sieve.log >> mail_plugins = quota mailbox_alias acl sieve >> postmaster_address = root >> } >> protocol lmtp { >> info_log_path = /var/log/dovecot-lmtp.log >> lmtp_save_to_detail_mailbox = yes > > Tries to save to the detail mailbox when a recipient address contains the detail. look at in 20-lmtp.conf > >> mail_plugins = quota sieve >> postmaster_address = postmaster >> recipient_delimiter = + >> } >> protocol imap { >> imap_client_workarounds = tb-extra-mailbox-sep >> mail_max_userip_connections = 20 >> mail_plugins = quota mailbox_alias acl imap_quota imap_acl >> } >> protocol pop3 { >> mail_max_userip_connections = 20 >> mail_plugins = quota mailbox_alias acl >> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >> pop3_uidl_format = %08Xu%08Xv >> } >> >> >> Thank you. >> >> - Fabian S. > > Regards > Christian > From tm at del.bg Tue Nov 17 08:35:11 2015 From: tm at del.bg (Teodor Milkov) Date: Tue, 17 Nov 2015 10:35:11 +0200 Subject: doveadm import/sync with read-only source location Message-ID: <564AE6BF.1070702@del.bg> Hello, Currently doveadm tool can't be used with read-only source locations because it's trying to create lock files. I'd guess lock files shouldn't be necessary on a read-only media, because no one else is going to change anything while our doveadm instance is running anyway. Example: doveadm import -u test at example.com maildir:/backup.0/domains/example.com/postmaster/Maildir restored ALL doveadm(test at example.com): Error: open(/backup.0/domains/example.com/postmaster/Maildir/.Trash/dovecot-uidlist.lock) failed: Read-only file system Perhaps this should be something like doveadm backup -R flag, but in the opposite direction ? i.e. we have a [full] backup command, but no [partial] restore command right now. Or am I missing something? Best regards, Teodor From ac at stacksoft.ru Tue Nov 17 11:03:59 2015 From: ac at stacksoft.ru (Anton Cheyvhchalov) Date: Tue, 17 Nov 2015 14:03:59 +0300 Subject: doveadm import/sync with read-only source location In-Reply-To: <564AE6BF.1070702@del.bg> References: <564AE6BF.1070702@del.bg> Message-ID: <20151117140359.3be2aba0@tahmasib.office.stacksoft.ru> +1 to that idea. Ask the same couple month before during migration from 1.x to 2.x but got no answer from someone who able to change this. On Tue, 17 Nov 2015 10:35:11 +0200 Teodor Milkov wrote: > Hello, > > Currently doveadm tool can't be used with read-only source locations > because it's trying to create lock files. > > I'd guess lock files shouldn't be necessary on a read-only media, > because no one else is going to change anything while our doveadm > instance is running anyway. Example: > > doveadm import -u test at example.com > maildir:/backup.0/domains/example.com/postmaster/Maildir restored ALL > doveadm(test at example.com): Error: > open(/backup.0/domains/example.com/postmaster/Maildir/.Trash/dovecot-uidlist.lock) > failed: Read-only file system > > Perhaps this should be something like doveadm backup -R flag, but in the > opposite direction ? i.e. we have a [full] backup command, but no > [partial] restore command right now. Or am I missing something? > > Best regards, > Teodor From sergey.schwartz at bgoperator.com Tue Nov 17 11:04:24 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Tue, 17 Nov 2015 14:04:24 +0300 Subject: lmtp panic In-Reply-To: <5645B01C.3020209@bgoperator.com> References: <5641E721.9060008@bgoperator.com> <56434B6E.7080902@bgoperator.com> <5645B01C.3020209@bgoperator.com> Message-ID: <564B09B8.9090208@bgoperator.com> Gentlemen, imap hits the same problem Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8215e) [0x7f8c8ce0915e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8224c) [0x7f8c8ce0924c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8c8cdb090e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f8c8d161c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_add+0x6eb) [0x7f8c8d14babb] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x280) [0x7f8c8d131f90] -> /usr/lib/dovecot/libdovecot.so.0(+0x6bace) [0x7f8c8cdf2ace] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x82) [0x7f8c8cdf7cb2] -> /usr/lib/dovecot/libdovecot.so.0(+0x6b429) [0x7f8c8cdf2429] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f8c8cdfb564] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x317) [0x7f8c8d16f827] -> dovecot/imap(+0x1effe) [0x7f8c8d5ffffe] -> dovecot/imap(+0x1d3fc) [0x7f8c8d5fe3fc] -> dovecot/imap(imap_fetch_more+0x3a) [0x7f8c8d5ff4fa] -> dovecot/imap(cmd_fetch+0x354) [0x7f8c8d5f1934] -> dovecot/imap(command_exec+0x85) [0x7f8c8d5fcbe5] -> dovecot/imap(+0x1a152) [0x7f8c8d5fb152] -> dovecot/imap(+0x1a1e0) [0x7f8c8d5fb1e0] -> dovecot/imap(client_handle_input+0x13d) [0x7f8c8d5fb54d] -> dovecot/imap(client_input+0x85) [0x7f8c8d5fb915] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f8c8ce1cc4c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f8c8ce1e0b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f8c8ce1ccd5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8c8ce1ce78] Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Fatal: master: service(imap): child 7445 killed with signal 6 (core dumped) any ideas how to fix this ? Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz at bgoperator.com 13.11.2015 12:40, Sergey Schwartz ?????: > root at mx10:~# doveadm force-resync -u oleg.vasilyev at bgoperator.com > noreply-05-08-2015 > doveadm(oleg.vasilyev at bgoperator.com): Warning: fscking index file > /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage/dovecot.map.index > doveadm(oleg.vasilyev at bgoperator.com): Warning: mdbox > /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage: rebuilding > indexes > doveadm(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: > line 37 (mail_index_uint32_to_offset): assertion failed: (offset < > 0x40000000) > doveadm(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: > /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f0f6f04e0de] -> > /usr/lib/dovecot/libdovecot.so.0(+0x82158) [0x7f0f6f04e158] -> > /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0f6eff58de] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) > [0x7f0f6f3a6c58] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_compress+0xaad) > [0x7f0f6f38c83d] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c7) > [0x7f0f6f39f4b7] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x1109) > [0x7f0f6f329189] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x825) > [0x7f0f6f32aaf5] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) > [0x7f0f6f32ac7c] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x42) > [0x7f0f6f32ad32] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) > [0x7f0f6f3106eb] -> > /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) > [0x7f0f6f310837] -> doveadm(+0x258e4) [0x7f0f6fc9d8e4] -> > doveadm(+0x2547e) [0x7f0f6fc9d47e] -> > doveadm(doveadm_mail_try_run+0x2ac) [0x7f0f6fc9e4dc] -> > doveadm(main+0x3a5) [0x7f0f6fc8f245] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f0f6ec28ec5] -> doveadm(+0x1734e) [0x7f0f6fc8f34e] > > > root at mx10:~# doveconf -n > # 2.2.19 (0b1c73b01a5a): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.9 > # OS: Linux 3.13.0-34-generic x86_64 Ubuntu 14.04.3 LTS xfs > auth_cache_negative_ttl = 0 > auth_cache_size = 512 M > auth_default_realm = bgoperator.com > auth_failure_delay = 5 secs > auth_worker_max_count = 4096 > default_client_limit = 40960 > default_process_limit = 8192 > default_vsz_limit = 2 G > disable_plaintext_auth = no > first_valid_gid = 500 > mail_location = mdbox:/srv/vmail/current/%d/%n > mail_plugins = acl zlib notify replication > mail_shared_explicit_inbox = yes > mailbox_list_index = yes > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate > mdbox_rotate_size = 32 M > namespace { > list = yes > location = mdbox:/srv/vmail/current/%%d/%%n > prefix = shared/%%u/ > separator = / > subscriptions = no > type = shared > } > namespace inbox { > inbox = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > separator = / > } > passdb { > args = /etc/dovecot/virtual/base.conf.ext > driver = sql > } > passdb { > args = /etc/dovecot/virtual/master.conf.ext > driver = sql > } > plugin { > acl = vfile > acl_shared_dict = > file:/srv/vmail/current/shared.db/000-shared-mailboxes.db > sieve = > file:/srv/vmail/current/%d/%n/sieve;active=/srv/vmail/current/%d/%n/active.sieve > sieve_max_redirects = 0 > zlib_save = xz > zlib_save_level = 6 > } > protocols = " imap lmtp sieve" > service auth-worker { > process_limit = 4096 > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0666 > user = postfix > } > } > service imap-login { > process_limit = 4096 > process_min_avail = 24 > } > service imap { > process_limit = 4096 > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > } > ssl_cert = ssl_key = syslog_facility = local0 > userdb { > args = /etc/dovecot/virtual/base.conf.ext > driver = sql > } > protocol lmtp { > mail_plugins = acl zlib notify replication sieve > } > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = acl zlib notify replication imap_acl > } > > > > Best regards, > Sergey Schwartz > > Senior System Administrator > Biblio Globus Tour Operator > www.bgoperator.ru > > T: +7 495 5042500 ext 1532 > E: sergey.schwartz at bgoperator.com > > 11.11.2015 17:06, Sergey Schwartz ?????: >> After 1 or 2 hours of recieveing this kind of errors mail server runs >> out of file descriptors >> >> Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: Too >> many open files >> >> >> Best regards, >> Sergey Schwartz >> >> Senior System Administrator >> Biblio Globus Tour Operator >> www.bgoperator.ru >> >> T: +7 495 5042500 ext 1532 >> E: sergey.schwartz at bgoperator.com >> >> 10.11.2015 15:46, Sergey Schwartz ?????: >>> Gents, >>> >>> >>> >>> I've just upgraded to the latest build of dovecot , now lmtp >>> delivery process panics for just one user >>> >>> >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>> (str[len] == '\0') >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>> (str[len] == '\0') >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>> [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>> [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>> [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>> [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >>> [0x7f50e380ddbd] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>> [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >>> [0x7f50e380f555] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>> [0x7f50e37eb2e7] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >>> [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>> [0x7f50e37fe6ea] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>> [0x7f50e37ff277] -> >>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>> [0x7f50e3a62b81] -> >>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>> [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> >>> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>> [0x7f50e597ebbc] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>> [0x7f50e597ffb1] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>> [0x7f50e597ec45] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50e597ede8] >>> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>> [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>> [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>> [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>> [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>> [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >>> [0x7f887d2b0dbd] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>> [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >>> [0x7f887d2b2555] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>> [0x7f887d28e2e7] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >>> [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>> [0x7f887d2a16ea] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>> [0x7f887d2a2277] -> >>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>> [0x7f887d505b81] -> >>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>> [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> >>> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>> [0x7f887f421bbc] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>> [0x7f887f422fb1] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>> [0x7f887f421c45] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f887f421de8] >>> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>> [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] >>> Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local >>> Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>> (str[len] == '\0') >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>> [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>> [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>> [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>> [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) >>> [0x7fd26aef5dbd] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>> [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) >>> [0x7fd26aef7555] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>> [0x7fd26aed32e7] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) >>> [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>> [0x7fd26aee66ea] -> >>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>> [0x7fd26aee7277] -> >>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>> [0x7fd26b14ab81] -> >>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>> [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> >>> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>> [0x7fd26d066bbc] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>> [0x7fd26d067fb1] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>> [0x7fd26d066c45] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd26d066de8] >>> -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>> [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> >>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>> [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(lmtp): child 7494 killed with signal 6 (core >>> dumped) >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(lmtp): child 7497 killed with signal 6 (core >>> dumped) >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(lmtp): child 7499 killed with signal 6 (core >>> dumped) >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(lmtp): child 6719 killed with signal 6 (core >>> dumped) >>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(lmtp): child 7502 killed with signal 6 (core >>> dumped) >>> Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: >>> Connection closed (in DATA finished) >>> >>> >>> Oleg has lots and lots of small emails :) >>> >>> I'm running ubuntu server 14.04 x86_64 >>> dovecot 2.2.19 (0b1c73b01a5a) build from >>> http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main >>> From madduck at madduck.net Tue Nov 17 20:51:18 2015 From: madduck at madduck.net (martin f krafft) Date: Wed, 18 Nov 2015 09:51:18 +1300 Subject: Disabling auth fallback to PAM Message-ID: <20151117205118.GA29237@fishbowl.rw.madduck.net> Hi folks, According to the wiki,? it's considered a feature of Dovecot and its ability to support multiple authentication sources that "if the password doesn't match in the first database, it checks the next one". ?) http://wiki.dovecot.org/Authentication/MultipleDatabases I think it's great that Dovecot allows auth sources to be stacked like this, but I am not sold on the idea that the next database ought to be tried when a *password* does not match. Let me elaborate: If the first database has knowledge of a user, then it can (should) be considered authoritative, and if the provided password does not match, it's an authentication error right away. Only if the first source does not posess any knowledge about a given user, then should Dovecot proceed to query/check with the next database. Can this be configured somehow? If not, would it make sense to make this behaviour configurable? Thanks, -- @martinkrafft | http://madduck.net/ | http://two.sentenc.es/ "the ships hung in the sky in much the same way that bricks don't." -- hitchhiker's guide to the galaxy spamtraps: madduck.bogus at madduck.net -------------- next part -------------- A non-text attachment was scrubbed... Name: digital_signature_gpg.asc Type: application/pgp-signature Size: 1107 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current) URL: From infoomatic at gmx.at Wed Nov 18 16:50:39 2015 From: infoomatic at gmx.at (Infoo Matic) Date: Wed, 18 Nov 2015 17:50:39 +0100 Subject: =?utf-8?q?bug_in_shared_folders=2Flazy=5Fexpunge?= Message-ID: <403-564cac80-3-280d9640@238841513> Hi list, I have a problem I was not able to resolve with my current setup (quite basic, the relevant part appended [1]). When userA shares a folder with userB then userB cannot access his mailbox. It's the same problem others have mentioned in *) http://www.dovecot.org/list/dovecot/2014-August/097449.html *) http://dovecot.org/pipermail/dovecot/2014-May/096355.html Could we please have a fix? TIA infoomatic [1] namespace { list = children location = mdbox:/var/lib/vmail/mail/%d/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace expunged { hidden = yes list = no location = mdbox:/var/lib/vmail/expunged/mail/%d/%u:INDEX=/var/lib/vmail/expunged/index/%d/%u:MAILBOXDIR=expunged prefix = .EXPUNGED/ subscriptions = no } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } From hadmut at danisch.de Wed Nov 18 21:00:33 2015 From: hadmut at danisch.de (Hadmut Danisch) Date: Wed, 18 Nov 2015 22:00:33 +0100 Subject: Shortcomings of Maildir++ layout In-Reply-To: References: <55D9918A.7080003@danisch.de> Message-ID: <564CE6F1.6000409@danisch.de> > Sounds more like you just introduced a new self-created problem that would require yet another workaround.. ;-) Definitely wrong. Appending .dir to directories and .folder to folder files keeps files completely distinct and name collissions are not possible. > But what was not clear from your mail, is what exactly you identified as "the problems" with standard Maildir++ setup? Having regular files filenames beginning with a . is a no-go and flawed by design. Extremely error-prone No serious designer would ever do that. Furthermore, it make it difficult to backup files from particular folder, since not in distinct folder (and thus not mbox-design). Even worse, both folders foo/bar and foo.bar both get the filename .foo.bar Maildir++-setup is broken by design in several ways. That's the problem with Maildir++. Appending smileys doesn't help. Hadmut On 24.08.2015 15:28, Maarten Bezemer wrote: > > On Sun, 23 Aug 2015, Hadmut Danisch wrote: > >> astonished to see, what problems come with Maildir++ and how complicated >> dovecot tries to deal with them. > .. > [fast fwd to "solution"] > >> INBOX -> ~/mail/inbox >> abc -> ~/mail/abc >> abc/def -> ~/mail/abc.dir/def or ~/mail/___abc___/def > > This would not work. > For example, how would you handle a folder named "abc.dir"? In your > scheme, that would end up as ~/mail/abc.dir which would still result > in a name clash. > How would you handle abc/def/ghi? Something like > ~/mail/abc.dir/def.dir/ghi or? That would not make it very workable > either. > > > > From troeder at univention.de Thu Nov 19 07:38:56 2015 From: troeder at univention.de (=?windows-1252?Q?Daniel_Tr=F6der?=) Date: Thu, 19 Nov 2015 08:38:56 +0100 Subject: Shortcomings of Maildir++ layout In-Reply-To: <564CE6F1.6000409@danisch.de> References: <55D9918A.7080003@danisch.de> <564CE6F1.6000409@danisch.de> Message-ID: <564D7C90.4070501@univention.de> On 11/18/2015 22:00, Hadmut Danisch wrote: > Having regular files filenames beginning with a . is a no-go and flawed > by design. Extremely error-prone No serious designer would ever do that. Sounds like a troll - should be ignored? ? yes - his blog is full of bad propaganda? definitively a troll. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Sat Nov 21 01:14:11 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Nov 2015 03:14:11 +0200 Subject: Disabling auth fallback to PAM In-Reply-To: <20151117205118.GA29237@fishbowl.rw.madduck.net> References: <20151117205118.GA29237@fishbowl.rw.madduck.net> Message-ID: > On 17 Nov 2015, at 22:51, martin f krafft wrote: > > Hi folks, > > According to the wiki,? it's considered a feature of Dovecot and its > ability to support multiple authentication sources that "if the > password doesn't match in the first database, it checks the next > one". > > ?) http://wiki.dovecot.org/Authentication/MultipleDatabases > > I think it's great that Dovecot allows auth sources to be stacked > like this, but I am not sold on the idea that the next database > ought to be tried when a *password* does not match. Let me > elaborate: > > If the first database has knowledge of a user, then it can (should) > be considered authoritative, and if the provided password does not > match, it's an authentication error right away. Only if the first > source does not posess any knowledge about a given user, then should > Dovecot proceed to query/check with the next database. > > Can this be configured somehow? > If not, would it make sense to make this behaviour configurable? Well, your topic is PAM.. And PAM doesn't necessarily tell you if the problem is that the user doesn't exist or that the password doesn't match. Another similar problem is checkpassword script. And LDAP with auth_bind=yes. And some ways of configuring SQL.. But.. Right now passdb has result_success, result_failure and result_internalfail. I suppose it should be possible to add result_user_unknown there that defaults to result_failure if it's not explicitly set. It wouldn't work with all passdb setups, but it would work for some. I've added it to my TODO list, but that's quite long already and this is near the bottom of it. So if you want it to be added to Dovecot anytime soon please send a patch. Shouldn't be difficult to implement. From tss at iki.fi Sat Nov 21 01:23:21 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Nov 2015 03:23:21 +0200 Subject: Shortcomings of Maildir++ layout In-Reply-To: <564CE6F1.6000409@danisch.de> References: <55D9918A.7080003@danisch.de> <564CE6F1.6000409@danisch.de> Message-ID: <3A06A2A7-A9FF-476F-8F35-5F0466EB8335@iki.fi> > On 18 Nov 2015, at 23:00, Hadmut Danisch wrote: > >> Sounds more like you just introduced a new self-created problem that > would require yet another workaround.. ;-) > > Definitely wrong. > > Appending .dir to directories and .folder to folder files keeps files > completely distinct and name collissions are not possible. > > >> But what was not clear from your mail, is what exactly you identified > as "the problems" with standard Maildir++ setup? > > > Having regular files filenames beginning with a . is a no-go and flawed > by design. Extremely error-prone No serious designer would ever do that. Dovecot doesn't force you to use LAYOUT=Maildir++. Just use LAYOUT=fs and you can also specify a DIRNAME= to avoid collisions with cur/new/tmp names. Actually, Dovecot doesn't even force you to use Maildir at all. Its own sdbox/mdbox mailbox formats are much more efficient if you don't care about using a standard mailbox format. > Even worse, both folders foo/bar and foo.bar both get the filename > .foo.bar Not with listescape plugin enabled. And if namespace separator is '/' as you seem to imply, newer Dovecot versions will give an error instead of silently translating foo/bar to .foo.bar (newer = probably last 5+ years or something). From tss at iki.fi Sat Nov 21 01:25:06 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Nov 2015 03:25:06 +0200 Subject: lmtp panic In-Reply-To: <564B09B8.9090208@bgoperator.com> References: <5641E721.9060008@bgoperator.com> <56434B6E.7080902@bgoperator.com> <5645B01C.3020209@bgoperator.com> <564B09B8.9090208@bgoperator.com> Message-ID: <19C39DF2-A856-41FF-BBBB-3D12FEFFFAF1@iki.fi> Do you have a huge >4GB dovecot.index.cache file? > On 17 Nov 2015, at 13:04, Sergey Schwartz wrote: > > Gentlemen, > > imap hits the same problem > > Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) > Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8215e) [0x7f8c8ce0915e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8224c) [0x7f8c8ce0924c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8c8cdb090e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f8c8d161c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_add+0x6eb) [0x7f8c8d14babb] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x280) [0x7f8c8d131f90] -> /usr/lib/dovecot/libdovecot.so.0(+0x6bace) [0x7f8c8cdf2ace] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x82) [0x7f8c8cdf7cb2] -> /usr/lib/dovecot/libdovecot.so.0(+0x6b429) [0x7f8c8cdf2429] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f8c8cdfb564] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x317) [0x7f8c8d16f827] -> dovecot/imap(+0x1effe) [0x7f8c8d5ffffe] -> dovecot/imap(+0x1d3fc) [0x7f8c8d5fe3fc] -> dovecot/imap(imap_fetch_more+0x3a) [0x7f8c8d5ff4fa] -> dovecot/imap(cmd_fetch+0x354) [0x7f8c8d5f1934] -> dovecot/imap(command_exec+0x85) [0x7f8c8d5fcbe5] -> dovecot/imap(+0x1a152) [0x7f8c8d5fb152] -> dovecot/imap(+0x1a1e0) [0x7f8c8d5fb1e0] -> dovecot/imap(client_handle_input+0x13d) [0x7f8c8d5fb54d] -> dovecot/imap(client_input+0x85) [0x7f8c8d5fb915] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f8c8ce1cc4c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f8c8ce1e0b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f8c8ce1ccd5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8c8ce1ce78] > Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Fatal: master: service(imap): child 7445 killed with signal 6 (core dumped) > > any ideas how to fix this ? > > Best regards, > Sergey Schwartz > > Senior System Administrator > Biblio Globus Tour Operator > www.bgoperator.ru > > T: +7 495 5042500 ext 1532 > E: sergey.schwartz at bgoperator.com > > 13.11.2015 12:40, Sergey Schwartz ?????: >> root at mx10:~# doveadm force-resync -u oleg.vasilyev at bgoperator.com noreply-05-08-2015 >> doveadm(oleg.vasilyev at bgoperator.com): Warning: fscking index file /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage/dovecot.map.index >> doveadm(oleg.vasilyev at bgoperator.com): Warning: mdbox /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage: rebuilding indexes >> doveadm(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) >> doveadm(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f0f6f04e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x82158) [0x7f0f6f04e158] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0f6eff58de] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f0f6f3a6c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_compress+0xaad) [0x7f0f6f38c83d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c7) [0x7f0f6f39f4b7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x1109) [0x7f0f6f329189] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x825) [0x7f0f6f32aaf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) [0x7f0f6f32ac7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x42) [0x7f0f6f32ad32] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) [0x7f0f6f3106eb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7f0f6f310837] -> doveadm(+0x258e4) [0x7f0f6fc9d8e4] -> doveadm(+0x2547e) [0x7f0f6fc9d47e] -> doveadm(doveadm_mail_try_run+0x2ac) [0x7f0f6fc9e4dc] -> doveadm(main+0x3a5) [0x7f0f6fc8f245] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0f6ec28ec5] -> doveadm(+0x1734e) [0x7f0f6fc8f34e] >> >> >> root at mx10:~# doveconf -n >> # 2.2.19 (0b1c73b01a5a): /etc/dovecot/dovecot.conf >> # Pigeonhole version 0.4.9 >> # OS: Linux 3.13.0-34-generic x86_64 Ubuntu 14.04.3 LTS xfs >> auth_cache_negative_ttl = 0 >> auth_cache_size = 512 M >> auth_default_realm = bgoperator.com >> auth_failure_delay = 5 secs >> auth_worker_max_count = 4096 >> default_client_limit = 40960 >> default_process_limit = 8192 >> default_vsz_limit = 2 G >> disable_plaintext_auth = no >> first_valid_gid = 500 >> mail_location = mdbox:/srv/vmail/current/%d/%n >> mail_plugins = acl zlib notify replication >> mail_shared_explicit_inbox = yes >> mailbox_list_index = yes >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate >> mdbox_rotate_size = 32 M >> namespace { >> list = yes >> location = mdbox:/srv/vmail/current/%%d/%%n >> prefix = shared/%%u/ >> separator = / >> subscriptions = no >> type = shared >> } >> namespace inbox { >> inbox = yes >> location = >> mailbox Drafts { >> auto = subscribe >> special_use = \Drafts >> } >> mailbox Junk { >> auto = subscribe >> special_use = \Junk >> } >> mailbox Sent { >> auto = subscribe >> special_use = \Sent >> } >> mailbox Trash { >> auto = subscribe >> special_use = \Trash >> } >> prefix = >> separator = / >> } >> passdb { >> args = /etc/dovecot/virtual/base.conf.ext >> driver = sql >> } >> passdb { >> args = /etc/dovecot/virtual/master.conf.ext >> driver = sql >> } >> plugin { >> acl = vfile >> acl_shared_dict = file:/srv/vmail/current/shared.db/000-shared-mailboxes.db >> sieve = file:/srv/vmail/current/%d/%n/sieve;active=/srv/vmail/current/%d/%n/active.sieve >> sieve_max_redirects = 0 >> zlib_save = xz >> zlib_save_level = 6 >> } >> protocols = " imap lmtp sieve" >> service auth-worker { >> process_limit = 4096 >> } >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0666 >> user = postfix >> } >> } >> service imap-login { >> process_limit = 4096 >> process_min_avail = 24 >> } >> service imap { >> process_limit = 4096 >> } >> service lmtp { >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0600 >> user = postfix >> } >> } >> ssl_cert = > ssl_key = > syslog_facility = local0 >> userdb { >> args = /etc/dovecot/virtual/base.conf.ext >> driver = sql >> } >> protocol lmtp { >> mail_plugins = acl zlib notify replication sieve >> } >> protocol imap { >> mail_max_userip_connections = 50 >> mail_plugins = acl zlib notify replication imap_acl >> } >> >> >> >> Best regards, >> Sergey Schwartz >> >> Senior System Administrator >> Biblio Globus Tour Operator >> www.bgoperator.ru >> >> T: +7 495 5042500 ext 1532 >> E: sergey.schwartz at bgoperator.com >> >> 11.11.2015 17:06, Sergey Schwartz ?????: >>> After 1 or 2 hours of recieveing this kind of errors mail server runs out of file descriptors >>> >>> Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>> >>> >>> Best regards, >>> Sergey Schwartz >>> >>> Senior System Administrator >>> Biblio Globus Tour Operator >>> www.bgoperator.ru >>> >>> T: +7 495 5042500 ext 1532 >>> E: sergey.schwartz at bgoperator.com >>> >>> 10.11.2015 15:46, Sergey Schwartz ?????: >>>> Gents, >>>> >>>> >>>> >>>> I've just upgraded to the latest build of dovecot , now lmtp delivery process panics for just one user >>>> >>>> >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f50e380ddbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f50e380f555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f50e37fe6ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f50e37ff277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f50e3a62b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f50e597ebbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f50e597ffb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50e597ede8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f887d2b0dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f887d2b2555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f887d2a16ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f887d2a2277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f887d505b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f887f421bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f887f422fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f887f421de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7fd26aef5dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7fd26aef7555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7fd26aed32e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7fd26aee66ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7fd26aee7277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7fd26b14ab81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fd26d066bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7fd26d067fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fd26d066c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd26d066de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7494 killed with signal 6 (core dumped) >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7497 killed with signal 6 (core dumped) >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7499 killed with signal 6 (core dumped) >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 6719 killed with signal 6 (core dumped) >>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7502 killed with signal 6 (core dumped) >>>> Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: Connection closed (in DATA finished) >>>> >>>> >>>> Oleg has lots and lots of small emails :) >>>> >>>> I'm running ubuntu server 14.04 x86_64 >>>> dovecot 2.2.19 (0b1c73b01a5a) build from http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main >>>> From tss at iki.fi Sat Nov 21 01:35:30 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 21 Nov 2015 03:35:30 +0200 Subject: doveadm import/sync with read-only source location In-Reply-To: <20151117140359.3be2aba0@tahmasib.office.stacksoft.ru> References: <564AE6BF.1070702@del.bg> <20151117140359.3be2aba0@tahmasib.office.stacksoft.ru> Message-ID: On 17 Nov 2015, at 13:03, Anton Cheyvhchalov wrote: > > > +1 to that idea. > > Ask the same couple month before during migration from 1.x to 2.x but got no answer from someone who able to change this. Troublesome.. Probably nicest way to implement this would be to add a new "maildir_lock_method" setting, which defaults to dotlock as it is now, but could be changed to fcntl/flock similar to as with mbox. I think fcntl/flock would work with read-only filesystems as well. This feature would also improve performance for most setups. It's something I've had in my mind a long time. But it's not really important enough, especially because people who care about performance would use sdbox/mdbox anyway. I'm happy to get a patch implementing it though. > On Tue, 17 Nov 2015 10:35:11 +0200 > Teodor Milkov wrote: > >> Hello, >> >> Currently doveadm tool can't be used with read-only source locations >> because it's trying to create lock files. >> >> I'd guess lock files shouldn't be necessary on a read-only media, >> because no one else is going to change anything while our doveadm >> instance is running anyway. Example: >> >> doveadm import -u test at example.com >> maildir:/backup.0/domains/example.com/postmaster/Maildir restored ALL >> doveadm(test at example.com): Error: >> open(/backup.0/domains/example.com/postmaster/Maildir/.Trash/dovecot-uidlist.lock) >> failed: Read-only file system >> >> Perhaps this should be something like doveadm backup -R flag, but in the >> opposite direction ? i.e. we have a [full] backup command, but no >> [partial] restore command right now. Or am I missing something? >> >> Best regards, >> Teodor From madduck at madduck.net Sat Nov 21 06:12:13 2015 From: madduck at madduck.net (martin f krafft) Date: Sat, 21 Nov 2015 19:12:13 +1300 Subject: Disabling auth fallback to PAM In-Reply-To: References: <20151117205118.GA29237@fishbowl.rw.madduck.net> Message-ID: <20151121061213.GA4016@fishbowl.rw.madduck.net> also sprach Timo Sirainen [2015-11-21 14:14 +1300]: > Well, your topic is PAM. Is it? My point is that PAM should not even be asked if an authentication source beforehand knows about a user but the password cannot be verified. > But.. Right now passdb has result_success, result_failure and > result_internalfail. I suppose it should be possible to add > result_user_unknown there that defaults to result_failure if it's > not explicitly set. result_user_known should be resturned when the authentication source does not know about a user. If the authentication source knows a user but fails to authenticate him/her due to a password mismatch, the result should rather be result_auth_failure. Those two should really replace result_failure and the dovecot authentication stack should only continue on result_user_known or result_internalfail. If we get result_success or result_auth_failure, then authentication is done and no further sources should be considered. -- @martinkrafft | http://madduck.net/ | http://two.sentenc.es/ only by counting could humans demonstrate their independence of computers. -- douglas adams, "the hitchhiker's guide to the galaxy" spamtraps: madduck.bogus at madduck.net -------------- next part -------------- A non-text attachment was scrubbed... Name: digital_signature_gpg.asc Type: application/pgp-signature Size: 1107 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current) URL: From sergey.schwartz at bgoperator.com Mon Nov 23 09:46:18 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Mon, 23 Nov 2015 12:46:18 +0300 Subject: lmtp panic In-Reply-To: <19C39DF2-A856-41FF-BBBB-3D12FEFFFAF1@iki.fi> References: <5641E721.9060008@bgoperator.com> <56434B6E.7080902@bgoperator.com> <5645B01C.3020209@bgoperator.com> <564B09B8.9090208@bgoperator.com> <19C39DF2-A856-41FF-BBBB-3D12FEFFFAF1@iki.fi> Message-ID: <5652E06A.2070607@bgoperator.com> Timo, -rw------- 1 vmail vmail *1,7G* ????. 16 22:54 ./mailboxes/noreply-05-08-2015/dbox-Mails/dovecot.index.cache the largest one Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz at bgoperator.com 21.11.2015 04:25, Timo Sirainen ?????: > Do you have a huge >4GB dovecot.index.cache file? > >> On 17 Nov 2015, at 13:04, Sergey Schwartz wrote: >> >> Gentlemen, >> >> imap hits the same problem >> >> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) >> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8215e) [0x7f8c8ce0915e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8224c) [0x7f8c8ce0924c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f8c8cdb090e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f8c8d161c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_add+0x6eb) [0x7f8c8d14babb] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x280) [0x7f8c8d131f90] -> /usr/lib/dovecot/libdovecot.so.0(+0x6bace) [0x7f8c8cdf2ace] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x82) [0x7f8c8cdf7cb2] -> /usr/lib/dovecot/libdovecot.so.0(+0x6b429) [0x7f8c8cdf2429] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) [0x7f8c8ce13483] -> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) [0x7f8c8ce13e7d] -> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) [0x7f8c8cdfb564] -> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x317) [0x7f8c8d16f827] -> dovecot/imap(+0x1effe) [0x7f8c8d5ffffe] -> dovecot/imap(+0x1d3fc) [0x7f8c8d5fe3fc] -> dovecot/imap(imap_fetch_more+0x3a) [0x7f8c8d5ff4fa] -> dovecot/imap(cmd_fetch+0x354) [0x7f8c8d5f1934] -> dovecot/imap(command_exec+0x85) [0x7f8c8d5fcbe5] -> dovecot/imap(+0x1a152) [0x7f8c8d5fb152] -> dovecot/imap(+0x1a1e0) [0x7f8c8d5fb1e0] -> dovecot/imap(client_handle_input+0x13d) [0x7f8c8d5fb54d] -> dovecot/imap(client_input+0x85) [0x7f8c8d5fb915] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f8c8ce1cc4c] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f8c8ce1e0b1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f8c8ce1ccd5] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8c8ce1ce78] >> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): Fatal: master: service(imap): child 7445 killed with signal 6 (core dumped) >> >> any ideas how to fix this ? >> >> Best regards, >> Sergey Schwartz >> >> Senior System Administrator >> Biblio Globus Tour Operator >> www.bgoperator.ru >> >> T: +7 495 5042500 ext 1532 >> E: sergey.schwartz at bgoperator.com >> >> 13.11.2015 12:40, Sergey Schwartz ?????: >>> root at mx10:~# doveadm force-resync -u oleg.vasilyev at bgoperator.com noreply-05-08-2015 >>> doveadm(oleg.vasilyev at bgoperator.com): Warning: fscking index file /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage/dovecot.map.index >>> doveadm(oleg.vasilyev at bgoperator.com): Warning: mdbox /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage: rebuilding indexes >>> doveadm(oleg.vasilyev at bgoperator.com): Panic: file mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) >>> doveadm(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f0f6f04e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x82158) [0x7f0f6f04e158] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0f6eff58de] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) [0x7f0f6f3a6c58] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_compress+0xaad) [0x7f0f6f38c83d] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c7) [0x7f0f6f39f4b7] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x1109) [0x7f0f6f329189] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x825) [0x7f0f6f32aaf5] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) [0x7f0f6f32ac7c] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x42) [0x7f0f6f32ad32] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) [0x7f0f6f3106eb] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) [0x7f0f6f310837] -> doveadm(+0x258e4) [0x7f0f6fc9d8e4] -> doveadm(+0x2547e) [0x7f0f6fc9d47e] -> doveadm(doveadm_mail_try_run+0x2ac) [0x7f0f6fc9e4dc] -> doveadm(main+0x3a5) [0x7f0f6fc8f245] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f0f6ec28ec5] -> doveadm(+0x1734e) [0x7f0f6fc8f34e] >>> >>> >>> root at mx10:~# doveconf -n >>> # 2.2.19 (0b1c73b01a5a): /etc/dovecot/dovecot.conf >>> # Pigeonhole version 0.4.9 >>> # OS: Linux 3.13.0-34-generic x86_64 Ubuntu 14.04.3 LTS xfs >>> auth_cache_negative_ttl = 0 >>> auth_cache_size = 512 M >>> auth_default_realm = bgoperator.com >>> auth_failure_delay = 5 secs >>> auth_worker_max_count = 4096 >>> default_client_limit = 40960 >>> default_process_limit = 8192 >>> default_vsz_limit = 2 G >>> disable_plaintext_auth = no >>> first_valid_gid = 500 >>> mail_location = mdbox:/srv/vmail/current/%d/%n >>> mail_plugins = acl zlib notify replication >>> mail_shared_explicit_inbox = yes >>> mailbox_list_index = yes >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate >>> mdbox_rotate_size = 32 M >>> namespace { >>> list = yes >>> location = mdbox:/srv/vmail/current/%%d/%%n >>> prefix = shared/%%u/ >>> separator = / >>> subscriptions = no >>> type = shared >>> } >>> namespace inbox { >>> inbox = yes >>> location = >>> mailbox Drafts { >>> auto = subscribe >>> special_use = \Drafts >>> } >>> mailbox Junk { >>> auto = subscribe >>> special_use = \Junk >>> } >>> mailbox Sent { >>> auto = subscribe >>> special_use = \Sent >>> } >>> mailbox Trash { >>> auto = subscribe >>> special_use = \Trash >>> } >>> prefix = >>> separator = / >>> } >>> passdb { >>> args = /etc/dovecot/virtual/base.conf.ext >>> driver = sql >>> } >>> passdb { >>> args = /etc/dovecot/virtual/master.conf.ext >>> driver = sql >>> } >>> plugin { >>> acl = vfile >>> acl_shared_dict = file:/srv/vmail/current/shared.db/000-shared-mailboxes.db >>> sieve = file:/srv/vmail/current/%d/%n/sieve;active=/srv/vmail/current/%d/%n/active.sieve >>> sieve_max_redirects = 0 >>> zlib_save = xz >>> zlib_save_level = 6 >>> } >>> protocols = " imap lmtp sieve" >>> service auth-worker { >>> process_limit = 4096 >>> } >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> group = postfix >>> mode = 0666 >>> user = postfix >>> } >>> } >>> service imap-login { >>> process_limit = 4096 >>> process_min_avail = 24 >>> } >>> service imap { >>> process_limit = 4096 >>> } >>> service lmtp { >>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>> group = postfix >>> mode = 0600 >>> user = postfix >>> } >>> } >>> ssl_cert = >> ssl_key = >> syslog_facility = local0 >>> userdb { >>> args = /etc/dovecot/virtual/base.conf.ext >>> driver = sql >>> } >>> protocol lmtp { >>> mail_plugins = acl zlib notify replication sieve >>> } >>> protocol imap { >>> mail_max_userip_connections = 50 >>> mail_plugins = acl zlib notify replication imap_acl >>> } >>> >>> >>> >>> Best regards, >>> Sergey Schwartz >>> >>> Senior System Administrator >>> Biblio Globus Tour Operator >>> www.bgoperator.ru >>> >>> T: +7 495 5042500 ext 1532 >>> E: sergey.schwartz at bgoperator.com >>> >>> 11.11.2015 17:06, Sergey Schwartz ?????: >>>> After 1 or 2 hours of recieveing this kind of errors mail server runs out of file descriptors >>>> >>>> Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: Too many open files >>>> >>>> >>>> Best regards, >>>> Sergey Schwartz >>>> >>>> Senior System Administrator >>>> Biblio Globus Tour Operator >>>> www.bgoperator.ru >>>> >>>> T: +7 495 5042500 ext 1532 >>>> E: sergey.schwartz at bgoperator.com >>>> >>>> 10.11.2015 15:46, Sergey Schwartz ?????: >>>>> Gents, >>>>> >>>>> >>>>> >>>>> I've just upgraded to the latest build of dovecot , now lmtp delivery process panics for just one user >>>>> >>>>> >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f50e5991bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f50e380ddbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f50e37f43b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f50e380f555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f50e37eb2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f50e37eb46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f50e37fe6ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f50e37ff277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f50e3a62b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f50e597ebbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f50e597ffb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f50e597ec45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f50e597ede8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7f887f434bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f887d2b0dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7f887d2973b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f887d2b2555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7f887d28e2e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f887d28e46b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7f887d2a16ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7f887d2a2277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7f887d505b81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7f887f421bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7f887f422fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7f887f421c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f887f421de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Panic: file str.c: line 22 (str_new_const): assertion failed: (str[len] == '\0') >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) [0x7fd26d079bf8] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7fd26aef5dbd] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) [0x7fd26aedc3b1] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7fd26aef7555] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) [0x7fd26aed32e7] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd26aed346b] -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) [0x7fd26aee66ea] -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) [0x7fd26aee7277] -> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) [0x7fd26b14ab81] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) [0x7fd26d066bbc] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) [0x7fd26d067fb1] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) [0x7fd26d066c45] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7fd26d066de8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7494 killed with signal 6 (core dumped) >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7497 killed with signal 6 (core dumped) >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7499 killed with signal 6 (core dumped) >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 6719 killed with signal 6 (core dumped) >>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): Fatal: master: service(lmtp): child 7502 killed with signal 6 (core dumped) >>>>> Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: Connection closed (in DATA finished) >>>>> >>>>> >>>>> Oleg has lots and lots of small emails :) >>>>> >>>>> I'm running ubuntu server 14.04 x86_64 >>>>> dovecot 2.2.19 (0b1c73b01a5a) build from http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main >>>>> From sergey.schwartz at bgoperator.com Mon Nov 23 16:25:29 2015 From: sergey.schwartz at bgoperator.com (Sergey Schwartz) Date: Mon, 23 Nov 2015 19:25:29 +0300 Subject: lmtp panic In-Reply-To: <5652E06A.2070607@bgoperator.com> References: <5641E721.9060008@bgoperator.com> <56434B6E.7080902@bgoperator.com> <5645B01C.3020209@bgoperator.com> <564B09B8.9090208@bgoperator.com> <19C39DF2-A856-41FF-BBBB-3D12FEFFFAF1@iki.fi> <5652E06A.2070607@bgoperator.com> Message-ID: <56533DF9.4090008@bgoperator.com> Timo, I've changed postfix lmtp delivery config to deal with storm conditions (oleg's emails), so that other users didn't wait for their email for hours. lmtp_destination_recipient_limit = 1 Number of error messages had dramatically reduced. It looks like concurrency is the root cause of the problem. I'm rebuilding oleg's mailbox just in case something was broken. dovecot.index.cache files looks tiny now -rw------- 1 vmail vmail 7,9M ????. 23 18:39 ./mailboxes/INBOX/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 452 ????. 23 13:47 ./mailboxes/Drafts/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 2,1M ????. 23 17:27 ./mailboxes/Sent/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 28K ????. 23 16:58 ./mailboxes/Junk/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 32K ????. 23 18:26 ./mailboxes/Trash/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 29M ????. 23 19:07 ./mailboxes/noreplybg/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 532 ????. 20 17:04 ./mailboxes/noreplybg/2015q2/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 49M ????. 23 19:05 ./mailboxes/noreplybg/2015q3/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 26M ????. 23 18:00 ./mailboxes/noreplybg/2015q4/dbox-Mails/dovecot.index.cache -rw------- 1 vmail vmail 349K ????. 23 16:31 ./mailboxes/provisa/dbox-Mails/dovecot.index.cache sort emails by folders, not sure if this will help in case of mdbox root at mx10:/etc/postfix# doveadm -f table mailbox status -u oleg.vasilyev at bgoperator.com -t all noreplybg/2015q3 messages recent unseen vsize 1307684 1307684 851074 11566574000 root at mx10:/etc/postfix# doveadm -f table mailbox status -u oleg.vasilyev at bgoperator.com -t all noreplybg/2015q4 messages recent unseen vsize 754266 754266 753931 9271563127 root at mx10:/etc/postfix# Best regards, Sergey Schwartz Senior System Administrator Biblio Globus Tour Operator www.bgoperator.ru T: +7 495 5042500 ext 1532 E: sergey.schwartz at bgoperator.com 23.11.2015 12:46, Sergey Schwartz ?????: > Timo, > > -rw------- 1 vmail vmail *1,7G* ????. 16 22:54 > ./mailboxes/noreply-05-08-2015/dbox-Mails/dovecot.index.cache > > the largest one > > Best regards, > Sergey Schwartz > > Senior System Administrator > Biblio Globus Tour Operator > www.bgoperator.ru > > T: +7 495 5042500 ext 1532 > E: sergey.schwartz at bgoperator.com > > 21.11.2015 04:25, Timo Sirainen ?????: >> Do you have a huge >4GB dovecot.index.cache file? >> >>> On 17 Nov 2015, at 13:04, Sergey Schwartz >>> wrote: >>> >>> Gentlemen, >>> >>> imap hits the same problem >>> >>> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): >>> Panic: file mail-index-util.c: line 37 >>> (mail_index_uint32_to_offset): assertion failed: (offset < 0x40000000) >>> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): >>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x8215e) >>> [0x7f8c8ce0915e] -> /usr/lib/dovecot/libdovecot.so.0(+0x8224c) >>> [0x7f8c8ce0924c] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>> [0x7f8c8cdb090e] -> >>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) >>> [0x7f8c8d161c58] -> >>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_add+0x6eb) >>> [0x7f8c8d14babb] -> >>> /usr/lib/dovecot/libdovecot-storage.so.0(index_mail_parse_header+0x280) >>> [0x7f8c8d131f90] -> /usr/lib/dovecot/libdovecot.so.0(+0x6bace) >>> [0x7f8c8cdf2ace] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) >>> [0x7f8c8ce13483] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) >>> [0x7f8c8ce13e7d] -> >>> /usr/lib/dovecot/libdovecot.so.0(message_parse_header_next+0x82) >>> [0x7f8c8cdf7cb2] -> /usr/lib/dovecot/libdovecot.so.0(+0x6b429) >>> [0x7f8c8cdf2429] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read+0x53) >>> [0x7f8c8ce13483] -> >>> /usr/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x3d) >>> [0x7f8c8ce13e7d] -> >>> /usr/lib/dovecot/libdovecot.so.0(message_get_header_size+0x74) >>> [0x7f8c8cdfb564] -> >>> /usr/lib/dovecot/libdovecot-storage.so.0(imap_msgpart_open+0x317) >>> [0x7f8c8d16f827] -> dovecot/imap(+0x1effe) [0x7f8c8d5ffffe] -> >>> dovecot/imap(+0x1d3fc) [0x7f8c8d5fe3fc] -> >>> dovecot/imap(imap_fetch_more+0x3a) [0x7f8c8d5ff4fa] -> >>> dovecot/imap(cmd_fetch+0x354) [0x7f8c8d5f1934] -> >>> dovecot/imap(command_exec+0x85) [0x7f8c8d5fcbe5] -> >>> dovecot/imap(+0x1a152) [0x7f8c8d5fb152] -> dovecot/imap(+0x1a1e0) >>> [0x7f8c8d5fb1e0] -> dovecot/imap(client_handle_input+0x13d) >>> [0x7f8c8d5fb54d] -> dovecot/imap(client_input+0x85) [0x7f8c8d5fb915] >>> -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>> [0x7f8c8ce1cc4c] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>> [0x7f8c8ce1e0b1] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>> [0x7f8c8ce1ccd5] -> >>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f8c8ce1ce78] >>> Nov 17 14:02:06 mx10 dovecot: imap(oleg.vasilyev at bgoperator.com): >>> Fatal: master: service(imap): child 7445 killed with signal 6 (core >>> dumped) >>> >>> any ideas how to fix this ? >>> >>> Best regards, >>> Sergey Schwartz >>> >>> Senior System Administrator >>> Biblio Globus Tour Operator >>> www.bgoperator.ru >>> >>> T: +7 495 5042500 ext 1532 >>> E: sergey.schwartz at bgoperator.com >>> >>> 13.11.2015 12:40, Sergey Schwartz ?????: >>>> root at mx10:~# doveadm force-resync -u oleg.vasilyev at bgoperator.com >>>> noreply-05-08-2015 >>>> doveadm(oleg.vasilyev at bgoperator.com): Warning: fscking index file >>>> /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage/dovecot.map.index >>>> doveadm(oleg.vasilyev at bgoperator.com): Warning: mdbox >>>> /srv/vmail/current/bgoperator.com/oleg.vasilyev/storage: rebuilding >>>> indexes >>>> doveadm(oleg.vasilyev at bgoperator.com): Panic: file >>>> mail-index-util.c: line 37 (mail_index_uint32_to_offset): assertion >>>> failed: (offset < 0x40000000) >>>> doveadm(oleg.vasilyev at bgoperator.com): Error: Raw backtrace: >>>> /usr/lib/dovecot/libdovecot.so.0(+0x820de) [0x7f0f6f04e0de] -> >>>> /usr/lib/dovecot/libdovecot.so.0(+0x82158) [0x7f0f6f04e158] -> >>>> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0f6eff58de] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_uint32_to_offset+0x78) >>>> [0x7f0f6f3a6c58] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_cache_compress+0xaad) >>>> [0x7f0f6f38c83d] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x2c7) >>>> [0x7f0f6f39f4b7] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_rebuild_in_context+0x1109) >>>> [0x7f0f6f329189] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync_begin+0x825) >>>> [0x7f0f6f32aaf5] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x4c) >>>> [0x7f0f6f32ac7c] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x42) >>>> [0x7f0f6f32ad32] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x3b) >>>> [0x7f0f6f3106eb] -> >>>> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x37) >>>> [0x7f0f6f310837] -> doveadm(+0x258e4) [0x7f0f6fc9d8e4] -> >>>> doveadm(+0x2547e) [0x7f0f6fc9d47e] -> >>>> doveadm(doveadm_mail_try_run+0x2ac) [0x7f0f6fc9e4dc] -> >>>> doveadm(main+0x3a5) [0x7f0f6fc8f245] -> >>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>>> [0x7f0f6ec28ec5] -> doveadm(+0x1734e) [0x7f0f6fc8f34e] >>>> >>>> >>>> root at mx10:~# doveconf -n >>>> # 2.2.19 (0b1c73b01a5a): /etc/dovecot/dovecot.conf >>>> # Pigeonhole version 0.4.9 >>>> # OS: Linux 3.13.0-34-generic x86_64 Ubuntu 14.04.3 LTS xfs >>>> auth_cache_negative_ttl = 0 >>>> auth_cache_size = 512 M >>>> auth_default_realm = bgoperator.com >>>> auth_failure_delay = 5 secs >>>> auth_worker_max_count = 4096 >>>> default_client_limit = 40960 >>>> default_process_limit = 8192 >>>> default_vsz_limit = 2 G >>>> disable_plaintext_auth = no >>>> first_valid_gid = 500 >>>> mail_location = mdbox:/srv/vmail/current/%d/%n >>>> mail_plugins = acl zlib notify replication >>>> mail_shared_explicit_inbox = yes >>>> mailbox_list_index = yes >>>> managesieve_notify_capability = mailto >>>> managesieve_sieve_capability = fileinto reject envelope >>>> encoded-character vacation subaddress comparator-i;ascii-numeric >>>> relational regex imap4flags copy include variables body enotify >>>> environment mailbox date index ihave duplicate >>>> mdbox_rotate_size = 32 M >>>> namespace { >>>> list = yes >>>> location = mdbox:/srv/vmail/current/%%d/%%n >>>> prefix = shared/%%u/ >>>> separator = / >>>> subscriptions = no >>>> type = shared >>>> } >>>> namespace inbox { >>>> inbox = yes >>>> location = >>>> mailbox Drafts { >>>> auto = subscribe >>>> special_use = \Drafts >>>> } >>>> mailbox Junk { >>>> auto = subscribe >>>> special_use = \Junk >>>> } >>>> mailbox Sent { >>>> auto = subscribe >>>> special_use = \Sent >>>> } >>>> mailbox Trash { >>>> auto = subscribe >>>> special_use = \Trash >>>> } >>>> prefix = >>>> separator = / >>>> } >>>> passdb { >>>> args = /etc/dovecot/virtual/base.conf.ext >>>> driver = sql >>>> } >>>> passdb { >>>> args = /etc/dovecot/virtual/master.conf.ext >>>> driver = sql >>>> } >>>> plugin { >>>> acl = vfile >>>> acl_shared_dict = >>>> file:/srv/vmail/current/shared.db/000-shared-mailboxes.db >>>> sieve = >>>> file:/srv/vmail/current/%d/%n/sieve;active=/srv/vmail/current/%d/%n/active.sieve >>>> sieve_max_redirects = 0 >>>> zlib_save = xz >>>> zlib_save_level = 6 >>>> } >>>> protocols = " imap lmtp sieve" >>>> service auth-worker { >>>> process_limit = 4096 >>>> } >>>> service auth { >>>> unix_listener /var/spool/postfix/private/auth { >>>> group = postfix >>>> mode = 0666 >>>> user = postfix >>>> } >>>> } >>>> service imap-login { >>>> process_limit = 4096 >>>> process_min_avail = 24 >>>> } >>>> service imap { >>>> process_limit = 4096 >>>> } >>>> service lmtp { >>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>> group = postfix >>>> mode = 0600 >>>> user = postfix >>>> } >>>> } >>>> ssl_cert = >>> ssl_key = >>> syslog_facility = local0 >>>> userdb { >>>> args = /etc/dovecot/virtual/base.conf.ext >>>> driver = sql >>>> } >>>> protocol lmtp { >>>> mail_plugins = acl zlib notify replication sieve >>>> } >>>> protocol imap { >>>> mail_max_userip_connections = 50 >>>> mail_plugins = acl zlib notify replication imap_acl >>>> } >>>> >>>> >>>> >>>> Best regards, >>>> Sergey Schwartz >>>> >>>> Senior System Administrator >>>> Biblio Globus Tour Operator >>>> www.bgoperator.ru >>>> >>>> T: +7 495 5042500 ext 1532 >>>> E: sergey.schwartz at bgoperator.com >>>> >>>> 11.11.2015 17:06, Sergey Schwartz ?????: >>>>> After 1 or 2 hours of recieveing this kind of errors mail server >>>>> runs out of file descriptors >>>>> >>>>> Nov 11 16:22:20 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:24 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:25 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:27 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:28 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:30 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:31 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:32 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:22:55 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> Nov 11 16:23:00 mx10 dovecot: config: Error: net_accept() failed: >>>>> Too many open files >>>>> >>>>> >>>>> Best regards, >>>>> Sergey Schwartz >>>>> >>>>> Senior System Administrator >>>>> Biblio Globus Tour Operator >>>>> www.bgoperator.ru >>>>> >>>>> T: +7 495 5042500 ext 1532 >>>>> E: sergey.schwartz at bgoperator.com >>>>> >>>>> 10.11.2015 15:46, Sergey Schwartz ?????: >>>>>> Gents, >>>>>> >>>>>> >>>>>> >>>>>> I've just upgraded to the latest build of dovecot , now lmtp >>>>>> delivery process panics for just one user >>>>>> >>>>>> >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>>>>> (str[len] == '\0') >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>>>>> (str[len] == '\0') >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>>>>> [0x7f50e596b0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>>>>> [0x7f50e596b1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>>>>> [0x7f50e59128de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>>>>> [0x7f50e5991bf8] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f50e380ddbd] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>>>>> [0x7f50e37f43b1] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f50e380f555] >>>>>> -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>>>>> [0x7f50e37eb2e7] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f50e37eb46b] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>>>>> [0x7f50e37fe6ea] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>>>>> [0x7f50e37ff277] -> >>>>>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>>>>> [0x7f50e3a62b81] -> >>>>>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>>>>> [0x7f50e5f26899] -> dovecot/lmtp(+0x6a04) [0x7f50e6357a04] -> >>>>>> dovecot/lmtp(+0x72d7) [0x7f50e63582d7] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>>>>> [0x7f50e597ebbc] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>>>>> [0x7f50e597ffb1] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>>>>> [0x7f50e597ec45] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>>>>> [0x7f50e597ede8] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>>>>> [0x7f50e59182e3] -> dovecot/lmtp(main+0x165) [0x7f50e6356135] -> >>>>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>>>>> [0x7f50e5545ec5] -> dovecot/lmtp(+0x5223) [0x7f50e6356223] >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>>>>> [0x7f887f40e0de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>>>>> [0x7f887f40e1cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>>>>> [0x7f887f3b58de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>>>>> [0x7f887f434bf8] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7f887d2b0dbd] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>>>>> [0x7f887d2973b1] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7f887d2b2555] >>>>>> -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>>>>> [0x7f887d28e2e7] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7f887d28e46b] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>>>>> [0x7f887d2a16ea] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>>>>> [0x7f887d2a2277] -> >>>>>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>>>>> [0x7f887d505b81] -> >>>>>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>>>>> [0x7f887f9c9899] -> dovecot/lmtp(+0x6a04) [0x7f887fdfaa04] -> >>>>>> dovecot/lmtp(+0x72d7) [0x7f887fdfb2d7] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>>>>> [0x7f887f421bbc] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>>>>> [0x7f887f422fb1] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>>>>> [0x7f887f421c45] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>>>>> [0x7f887f421de8] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>>>>> [0x7f887f3bb2e3] -> dovecot/lmtp(main+0x165) [0x7f887fdf9135] -> >>>>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>>>>> [0x7f887efe8ec5] -> dovecot/lmtp(+0x5223) [0x7f887fdf9223] >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7502): Connect from local >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(7509): Connect from local >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Panic: file str.c: line 22 (str_new_const): assertion failed: >>>>>> (str[len] == '\0') >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x820de) >>>>>> [0x7fd26d0530de] -> /usr/lib/dovecot/libdovecot.so.0(+0x821cc) >>>>>> [0x7fd26d0531cc] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) >>>>>> [0x7fd26cffa8de] -> /usr/lib/dovecot/libdovecot.so.0(+0xa8bf8) >>>>>> [0x7fd26d079bf8] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5ddbd) [0x7fd26aef5dbd] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_match+0xf1) >>>>>> [0x7fd26aedc3b1] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x5f555) [0x7fd26aef7555] >>>>>> -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_continue+0xe7) >>>>>> [0x7fd26aed32e7] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_interpreter_run+0x2b) [0x7fd26aed346b] >>>>>> -> /usr/lib/dovecot/libdovecot-sieve.so.0(+0x4e6ea) >>>>>> [0x7fd26aee66ea] -> >>>>>> /usr/lib/dovecot/libdovecot-sieve.so.0(sieve_execute+0x47) >>>>>> [0x7fd26aee7277] -> >>>>>> /usr/lib/dovecot/modules/lib90_sieve_plugin.so(+0x3b81) >>>>>> [0x7fd26b14ab81] -> >>>>>> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x49) >>>>>> [0x7fd26d60e899] -> dovecot/lmtp(+0x6a04) [0x7fd26da3fa04] -> >>>>>> dovecot/lmtp(+0x72d7) [0x7fd26da402d7] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x4c) >>>>>> [0x7fd26d066bbc] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x101) >>>>>> [0x7fd26d067fb1] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x25) >>>>>> [0x7fd26d066c45] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) >>>>>> [0x7fd26d066de8] -> >>>>>> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) >>>>>> [0x7fd26d0002e3] -> dovecot/lmtp(main+0x165) [0x7fd26da3e135] -> >>>>>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) >>>>>> [0x7fd26cc2dec5] -> dovecot/lmtp(+0x5223) [0x7fd26da3e223] >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Fatal: master: service(lmtp): child 7494 killed with signal 6 >>>>>> (core dumped) >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Fatal: master: service(lmtp): child 7497 killed with signal 6 >>>>>> (core dumped) >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Fatal: master: service(lmtp): child 7499 killed with signal 6 >>>>>> (core dumped) >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Fatal: master: service(lmtp): child 6719 killed with signal 6 >>>>>> (core dumped) >>>>>> Nov 10 15:36:49 mx10 dovecot: lmtp(oleg.vasilyev at bgoperator.com): >>>>>> Fatal: master: service(lmtp): child 7502 killed with signal 6 >>>>>> (core dumped) >>>>>> Nov 10 15:36:51 mx10 dovecot: lmtp(7509): Disconnect from local: >>>>>> Connection closed (in DATA finished) >>>>>> >>>>>> >>>>>> Oleg has lots and lots of small emails :) >>>>>> >>>>>> I'm running ubuntu server 14.04 x86_64 >>>>>> dovecot 2.2.19 (0b1c73b01a5a) build from >>>>>> http://xi.rename-it.nl/debian/ jessie-auto/dovecot-2.2 main >>>>>> From f0andrey at gmail.com Wed Nov 25 13:42:36 2015 From: f0andrey at gmail.com (Andrey Fesenko) Date: Wed, 25 Nov 2015 16:42:36 +0300 Subject: CentOS rpm dovecot 2.2.10 auth/db-ldap.c TLS bug/patch Message-ID: Hello, CentOS rpm dovecot 2.2.10 ?ontains bug auth/db-ldap.c TLS (not connect LDAP+TLS server ldaps://), exist bug/patch https://bugs.centos.org/view.php?id=8267 As far as the correct patch in upstream dovecot quite a lot of changes at this point if there is a correct patch? From goetz.reinicke at filmakademie.de Wed Nov 25 14:05:53 2015 From: goetz.reinicke at filmakademie.de (=?UTF-8?Q?G=c3=b6tz_Reinicke_-_IT_Koordinator?=) Date: Wed, 25 Nov 2015 15:05:53 +0100 Subject: sieve is working/forwarding mail - but not for all users Message-ID: <5655C041.80904@filmakademie.de> Hi, we have dovecot-ee-2.2.18.2 and pigeonhole/managesieve 0.4.8 running for some time. Today some users informed us that they did not get mails from one project account forwarded to there personal accounts any more. This worked till one week ago and I cant think of any changes we made... The project account keeps a copy of received mails. I tried two different accounts to configure forwarding to internal and external mail addresses which is working. Question: Any hint or idea? How may I debug sieve forwarding? Thanks and regards . G?tz -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5501 bytes Desc: S/MIME Cryptographic Signature URL: From rs at sys4.de Wed Nov 25 14:17:08 2015 From: rs at sys4.de (Robert Schetterer) Date: Wed, 25 Nov 2015 15:17:08 +0100 Subject: sieve is working/forwarding mail - but not for all users In-Reply-To: <5655C041.80904@filmakademie.de> References: <5655C041.80904@filmakademie.de> Message-ID: <5655C2E4.8080706@sys4.de> Am 25.11.2015 um 15:05 schrieb G?tz Reinicke - IT Koordinator: > Hi, > > we have dovecot-ee-2.2.18.2 and pigeonhole/managesieve 0.4.8 running for > some time. > > Today some users informed us that they did not get mails from one > project account forwarded to there personal accounts any more. > > This worked till one week ago and I cant think of any changes we made... > > The project account keeps a copy of received mails. > > I tried two different accounts to configure forwarding to internal and > external mail addresses which is working. > > Question: Any hint or idea? How may I debug sieve forwarding? > > Thanks and regards . G?tz > look at the related rules and logs ( search for sieve ) i.e http://wiki.dovecot.org/LDA/Sieve/Dovecot ... Compile and Runtime Logging Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From bind at enas.net Wed Nov 25 14:32:07 2015 From: bind at enas.net (Urban Loesch) Date: Wed, 25 Nov 2015 15:32:07 +0100 Subject: sieve is working/forwarding mail - but not for all users In-Reply-To: <5655C041.80904@filmakademie.de> References: <5655C041.80904@filmakademie.de> Message-ID: <5655C667.5080500@enas.net> Hi, to how many addresses do you redirect the incoming mails? Could it be that you hit the "sieve_max_redirects = n" configuration parameter? Regards Urban Am 25.11.2015 um 15:05 schrieb G?tz Reinicke - IT Koordinator: > Hi, > > we have dovecot-ee-2.2.18.2 and pigeonhole/managesieve 0.4.8 running for > some time. > > Today some users informed us that they did not get mails from one > project account forwarded to there personal accounts any more. > > This worked till one week ago and I cant think of any changes we made... > > The project account keeps a copy of received mails. > > I tried two different accounts to configure forwarding to internal and > external mail addresses which is working. > > Question: Any hint or idea? How may I debug sieve forwarding? > > Thanks and regards . G?tz > From goetz.reinicke at filmakademie.de Wed Nov 25 14:57:39 2015 From: goetz.reinicke at filmakademie.de (=?UTF-8?Q?G=c3=b6tz_Reinicke_-_IT_Koordinator?=) Date: Wed, 25 Nov 2015 15:57:39 +0100 Subject: sieve is working/forwarding mail - but not for all users - SOLVED In-Reply-To: <5655C667.5080500@enas.net> References: <5655C041.80904@filmakademie.de> <5655C667.5080500@enas.net> Message-ID: <5655CC63.8060302@filmakademie.de> Thanks all, found a config file with the adjusted sieve_max_redirects which was saved during an update. Regards . G?tz Am 25.11.15 um 15:32 schrieb Urban Loesch: > Hi, > > to how many addresses do you redirect the incoming mails? > Could it be that you hit the "sieve_max_redirects = n" configuration parameter? > > Regards > Urban > > Am 25.11.2015 um 15:05 schrieb G?tz Reinicke - IT Koordinator: >> Hi, >> >> we have dovecot-ee-2.2.18.2 and pigeonhole/managesieve 0.4.8 running for >> some time. >> >> Today some users informed us that they did not get mails from one >> project account forwarded to there personal accounts any more. >> >> This worked till one week ago and I cant think of any changes we made... >> >> The project account keeps a copy of received mails. >> >> I tried two different accounts to configure forwarding to internal and >> external mail addresses which is working. >> >> Question: Any hint or idea? How may I debug sieve forwarding? >> >> Thanks and regards . G?tz >> -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5501 bytes Desc: S/MIME Cryptographic Signature URL: From renaud at allard.it Wed Nov 25 15:16:45 2015 From: renaud at allard.it (Renaud Allard) Date: Wed, 25 Nov 2015 16:16:45 +0100 Subject: LZ4 mailbox corruption Message-ID: <5655D0DD.4080004@allard.it> Hello, When I look at email files on my server (mdbox format), they contain random LZ4 stuff. I have now changed the compression to gzip and new mails look cleaner. But older ones have problems. A mail file looks like this: 2 M1e C5655b6a8 ^A^BN 00000000000006F4 Dovecot-LZ4^M*<9B>^@^A^@^@^@^@^F
<83>Return-path: Envelope-to: nikademus at llorien.org Delivery-date: Mon, 22 Nov 2004 09:50:58 +0100 Received: from }^@^G [66.135.39.84] by mail.}^@< with esmtp (TLS-1.0:DHE_RSA_3DES_EDE_CBC_SHA:24) id=1CW9uQ-0002gj-El for^@^C^_;^@8(localhost. ^@^Ddomain [127.0.0.1])^@^O^S^A^H<82>(8.12.8/^G^@^R)^@^HESMTP id iAM9gl4l021209^@^_<^@^B^_>^@^A3:42:47 -06^Ac(from -^B^A^@Ohost^@^\<86>/Submit)^@!cd^@N7; 4^B <8C>^@^_DZ^B^D &^@^KMessage-Id: <200411220942.^Y^A^Dq^@^O^B ^BTo: Renaud AllardB^A^E Then, I randomly see errors like this in maillog: truncated lz4 chunk at 555395 Given I am still able to read the mails using imap, I think there might be a way to recover them to a proper format. However, using dsync to make a mirror throws a lot of errors like below. # dsync -v -u recover -o "mail_plugins=zlib" mirror dbox:~/ndbox dsync(recover): Error: lz4.read(//home/vmail/recover/mdbox/storage/m.2): corrupted lz4 chunk at 43551 dsync(recover): Error: write(/home/vmail/recover/ndbox/mailboxes/INBOX/dbox-Mails/.temp.1448464321.P24341Q6M965521.isildur.arnor.org) failed: Invalid argument dsync(recover): Error: copy: i_stream_read(zlib(//home/vmail/recover/mdbox/storage/m.2)) failed: read(//home/vmail/recover/mdbox/storage/m.2) failed: lz4.read(//home/vmail/recover/mdbox/storage/m.2): corrupted lz4 chunk at 43551 dsync(recover): Error: read(zlib(//home/vmail/recover/mdbox/storage/m.2)) failed: read(//home/vmail/recover/mdbox/storage/m.2) failed: lz4.read(//home/vmail/recover/mdbox/storage/m.2): corrupted lz4 chunk at 43551 (uid=42028, box=INBOX) dsync(recover): Error: Mailbox INBOX: read(msg input) failed: read(//home/vmail/recover/mdbox/storage/m.2) failed: lz4.read(//home/vmail/recover/mdbox/storage/m.2): corrupted lz4 chunk at 43551 dsync(recover): Error: read(zlib(//home/vmail/recover/mdbox/storage/m.2)) failed: read(//home/vmail/recover/mdbox/storage/m.2) failed: lz4.read(//home/vmail/recover/mdbox/storage/m.2): corrupted lz4 chunk at 43551 (uid=42028, box=INBOX) dsync(recover): Error: unlink(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????) failed: No such file or directory dsync(recover): Error: unlink(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????//home/vmail/recover/mdbox/mailboxes/Drafts/dbox-Mails/dovecot.index.log.2) failed: No such file or directory dsync(recover): Error: unlink(Internal error occurred. Refer to server log for more information. [2015-11-25 16:12:01]) failed: No such file or directory dsync(recover): Error: unlink(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????) failed: No such file or directory dsync(recover): Error: unlink(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????) failed: File name too long dsync(recover): Error: unlink(Internal error occurred. Refer to server log for more information. [2015-11-25 16:12:01]) failed: No such file or directory Any help welcome -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4213 bytes Desc: S/MIME Cryptographic Signature URL: From madduck at madduck.net Thu Nov 26 02:12:02 2015 From: madduck at madduck.net (martin f krafft) Date: Thu, 26 Nov 2015 15:12:02 +1300 Subject: Let lmtp create target directories Message-ID: <20151126021202.GA30712@fishbowl.rw.madduck.net> Hello, we're using vmm? to manage our postfix+dovecot virtual mail setup, which allows us to give every virtual user a separate EUID and every domain a separate EGID for additional security (vs. handling all virtual mail with a single "vmail" user). As a consequence, however, vmm must itself create the user directories with the appropriate owners, and to do so, it requires root rights. I am trying to investigate getting rid of this need?. Since Dovecot quite happily creates ~/Maildir when necessary, couldn't it also create parents? The home directory should be trivial (same EUID/EGID), but grandparents etc. might need a different policy (e.g. 0/EGID for the grandparent, 0/0 for great-grandparents, etc.). Is this something that could fall within the realm of Dovecot's lmtp? Or is the lmtp invoked as the user and doesn't actually drop root? If so, might there be another way? ?) http://vmm.localdomain.org/? ?) Hallo Pascal ?) http://bugs.debian.org/804382 Thanks, -- @martinkrafft | http://madduck.net/ | http://two.sentenc.es/ "perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away." -- antoine de saint-exup?ry spamtraps: madduck.bogus at madduck.net -------------- next part -------------- A non-text attachment was scrubbed... Name: digital_signature_gpg.asc Type: application/pgp-signature Size: 1107 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/sig-policy/999bbcc4/current) URL: From mancyborg at gmail.com Thu Nov 26 06:51:27 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Thu, 26 Nov 2015 07:51:27 +0100 Subject: imap logging ? Message-ID: <20151126075127.484f05d8@linux> Hi I'm trying to log my users imap actions, like when creating a folder, moving an email or deleting an email. So I've enabled 'mail_debug' and I'm checking /var/log/dovecot/debug.log this is what happens when I delete an email: Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): query: SELECT password FROM mailbox WHERE username = 'XXX' and active = 1 and restrictedAccess = 0 Nov 26 07:46:38 auth: Debug: client out: OK 1 user=XXX Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): SELECT '/var/vmail/XXX/XXX' as home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota) AS quota_rule FROM mailbox WHERE username = 'XXX' Nov 26 07:46:38 auth: Debug: master out: USER 374472705 XXX home=/var/vmail/XXX/XXX uid=5000 gid=5000 quota_rule=*:storage=524288 Nov 26 07:46:38 imap(XXX): Debug: Effective uid=5000, gid=5000, home=/var/vmail/XXX/XXX Nov 26 07:46:38 imap(XXX): Debug: Quota root: name=User quota backend=maildir args= Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=* bytes=536870912 messages=0 Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 Nov 26 07:46:38 imap(XXX): Debug: maildir++: root=/var/vmail/XXX/XXX/Maildir, index=/var/vmail/XXX/XXX/Maildir/indexes, control=, inbox=/var/vmail/XXX/XXX/Maildir, alt= and when creating a folder, access an email or moving an email, the output is basically the same: I'm unable to find the actual IMAP command. So, question: is there a way to log IMAP commands to a file ? Thank you, Mike From bind at enas.net Thu Nov 26 07:52:55 2015 From: bind at enas.net (Urban Loesch) Date: Thu, 26 Nov 2015 08:52:55 +0100 Subject: [SPAM: high] imap logging ? In-Reply-To: <20151126075127.484f05d8@linux> References: <20151126075127.484f05d8@linux> Message-ID: <5656BA57.2040002@enas.net> Hi, perhaps this is what you need. http://wiki2.dovecot.org/Plugins/MailLog No "mail_debug enabled" neccessary. Regards Urban Am 26.11.2015 um 07:51 schrieb mancyborg at gmail.com: > Hi I'm trying to log my users imap actions, like when creating a folder, moving an email or deleting an email. > So I've enabled 'mail_debug' and I'm checking /var/log/dovecot/debug.log > this is what happens when I delete an email: > > Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): query: SELECT password FROM mailbox WHERE username = 'XXX' and active = 1 and restrictedAccess = 0 > Nov 26 07:46:38 auth: Debug: client out: OK 1 user=XXX > Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): SELECT '/var/vmail/XXX/XXX' as home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota) AS quota_rule FROM mailbox WHERE username = 'XXX' > Nov 26 07:46:38 auth: Debug: master out: USER 374472705 XXX home=/var/vmail/XXX/XXX uid=5000 gid=5000 quota_rule=*:storage=524288 > Nov 26 07:46:38 imap(XXX): Debug: Effective uid=5000, gid=5000, home=/var/vmail/XXX/XXX > Nov 26 07:46:38 imap(XXX): Debug: Quota root: name=User quota backend=maildir args= > Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=* bytes=536870912 messages=0 > Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 > Nov 26 07:46:38 imap(XXX): Debug: maildir++: root=/var/vmail/XXX/XXX/Maildir, index=/var/vmail/XXX/XXX/Maildir/indexes, control=, inbox=/var/vmail/XXX/XXX/Maildir, alt= > > and when creating a folder, access an email or moving an email, the output is basically the same: > I'm unable to find the actual IMAP command. > > So, question: is there a way to log IMAP commands to a file ? > > Thank you, > Mike > From sami.ketola at dovecot.fi Thu Nov 26 11:30:29 2015 From: sami.ketola at dovecot.fi (Sami Ketola) Date: Thu, 26 Nov 2015 13:30:29 +0200 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <563E6E99.2090405@gmail.com> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> Message-ID: <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> Hi, With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. http://wiki2.dovecot.org/Migration Sami > On 07 Nov 2015, at 23:35, Forrest wrote: > > Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) > > > > On 11/7/15 3:31 PM, Philon wrote: >> Hi there, >> >> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >> >> >> Philon >> >> >>> Am 04.11.2015 um 20:47 schrieb Forrest : >>> >>> I have been attempting to use the cyrus2dovecot script, to no avail. >>> >>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>> >>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>> >>> >>> inboxes=the "myaccount" that was copied over >>> >>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>> myaccount >>> >>> >>> >>> the log output complains of: >>> >>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>> >>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>> >>> All directory paths are correct. >>> >>> >>> Thanks. From emmanuel.fuste at thalesgroup.com Thu Nov 26 13:36:45 2015 From: emmanuel.fuste at thalesgroup.com (FUSTE Emmanuel) Date: Thu, 26 Nov 2015 14:36:45 +0100 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> Message-ID: <56570AED.1020506@thalesgroup.com> Hello, Because it did not work ? In a similar situation, we where forced to use isync/mbsync in imap to imap mode because dsync did not work. It was reported here more than a year ago (May 2014). Time to time, I see the same report from other trying to use dsync to do a migration to dovecot. Dsync is a very appealing and elegant solution to this usage, but it does not always work in real world. Regards, Emmanuel Le 26/11/2015 12:30, Sami Ketola a ?crit : > Hi, > > With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. > > http://wiki2.dovecot.org/Migration > > Sami > > >> On 07 Nov 2015, at 23:35, Forrest wrote: >> >> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) >> >> >> >> On 11/7/15 3:31 PM, Philon wrote: >>> Hi there, >>> >>> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >>> >>> >>> Philon >>> >>> >>>> Am 04.11.2015 um 20:47 schrieb Forrest : >>>> >>>> I have been attempting to use the cyrus2dovecot script, to no avail. >>>> >>>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>>> >>>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>>> >>>> >>>> inboxes=the "myaccount" that was copied over >>>> >>>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>>> myaccount >>>> >>>> >>>> >>>> the log output complains of: >>>> >>>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>>> >>>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>>> >>>> All directory paths are correct. >>>> >>>> >>>> Thanks. From g.danti at assyoma.it Thu Nov 26 14:02:49 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Thu, 26 Nov 2015 15:02:49 +0100 Subject: Questions about hardlinks, alternate storage and compression In-Reply-To: <55A50413.7070602@assyoma.it> References: <55A3806D.7060307@us.es> <55A38958.6020700@assyoma.it> <55A4E1CE.30403@assyoma.it> <55A50413.7070602@assyoma.it> Message-ID: Il 14-07-2015 14:44 Gionatan Danti ha scritto: > On 14/07/15 12:26, Steffen Kaiser wrote: > >> >> You asked about "newer dovecot versions", v2.2 does so. >> > > Fair enough :) > > So, with v2.2+ the hardlink approach is irremediably gone, at least > with LMTP (and without relying to SiS)? Dear list, sorry if I resume this (relatively) old post, but I would like to know if someone has some good suggestions/ideas. A quick recap... System spec: - CentOS 6.6 x64 - dovecot-2.0.9-8.el6_6.4.x86_64 RPM package/version - sdbox mail store I have two problems: 1) hardlinks are not created when the same email is sent to multiple rcpt, nor when sending to the very same user (a non-linked copy is kept both in Inbox and Sent) 2) I would like to have an alternate storage and to enable compression on it only, leaving the main location without compression. It is possible? Relative to the hard-link problem, I found that not using before_sieve and after_sieve solves my problem, so that hardlinks are created correctly. However, this is far from ideal because I lose global sieve rules (global/defaul rules are only applied when no customized rules exist). Any idea to how to solve these two problem? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From sami.ketola at dovecot.fi Thu Nov 26 14:24:08 2015 From: sami.ketola at dovecot.fi (Sami Ketola) Date: Thu, 26 Nov 2015 16:24:08 +0200 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <56570AED.1020506@thalesgroup.com> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> <56570AED.1020506@thalesgroup.com> Message-ID: <257B4D4E-923B-488D-9285-F85AE87D3EF1@dovecot.fi> Hi, I think you tried to read cyrus mails folders directly. I was talking about fething mails from cyrus over imapc connection. Sami > On 26 Nov 2015, at 15:36, FUSTE Emmanuel wrote: > > Hello, > > Because it did not work ? > In a similar situation, we where forced to use isync/mbsync in imap to > imap mode because dsync did not work. > It was reported here more than a year ago (May 2014). > Time to time, I see the same report from other trying to use dsync to do > a migration to dovecot. > Dsync is a very appealing and elegant solution to this usage, but it > does not always work in real world. > > Regards, > Emmanuel > > Le 26/11/2015 12:30, Sami Ketola a ?crit : >> Hi, >> >> With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. >> >> http://wiki2.dovecot.org/Migration >> >> Sami >> >> >>> On 07 Nov 2015, at 23:35, Forrest wrote: >>> >>> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) >>> >>> >>> >>> On 11/7/15 3:31 PM, Philon wrote: >>>> Hi there, >>>> >>>> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >>>> >>>> >>>> Philon >>>> >>>> >>>>> Am 04.11.2015 um 20:47 schrieb Forrest : >>>>> >>>>> I have been attempting to use the cyrus2dovecot script, to no avail. >>>>> >>>>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>>>> >>>>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>>>> >>>>> >>>>> inboxes=the "myaccount" that was copied over >>>>> >>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>>>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>>>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>>>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>>>> myaccount >>>>> >>>>> >>>>> >>>>> the log output complains of: >>>>> >>>>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>>>> >>>>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>>>> >>>>> All directory paths are correct. >>>>> >>>>> >>>>> Thanks. From g.danti at assyoma.it Thu Nov 26 14:27:40 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Thu, 26 Nov 2015 15:27:40 +0100 Subject: [g.danti@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression] In-Reply-To: <20151126141529.GO18514@frodo.gerdesas.com> References: <20151126141529.GO18514@frodo.gerdesas.com> Message-ID: Il 26-11-2015 15:15 John R. Dennison ha scritto: > You are strongly encouraged to update that CentOS system. Current is > 6.7 (released some 3 months ago) and dovecot-2.0.9-19. Ouch! I copied outdated information from my old post. My current system _is_ CentOS 6.7 with dovecot dovecot-2.0.9-19.el6_7.2.x86_64 Sorry for the confusion. Still, the problems remain > If you find you need a more modern dovecot version you will find that > dovecot-2.2.19-1 is in the GhettoForge 3rd-party repo, packaged and > maintained by a respected CentOS community member. Information > on this repo can be found by following the links off of: > > http://wiki.centos.org/AdditionalResources/Repositories > The point is that I _don't know_ if more recent Dovecot version have the same problem or not. Moreover, as this is a production machine, I would like to leave a key package as Dovecot as near as possible to the shipping O.S. Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From me at junc.eu Thu Nov 26 15:01:07 2015 From: me at junc.eu (Benny Pedersen) Date: Thu, 26 Nov 2015 16:01:07 +0100 Subject: [g.danti@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression] In-Reply-To: References: <20151126141529.GO18514@frodo.gerdesas.com> Message-ID: <20151126150115.2F0BA23488@talvi.dovecot.org> On November 26, 2015 3:28:22 PM Gionatan Danti wrote: > Il 26-11-2015 15:15 John R. Dennison ha scritto: >> You are strongly encouraged to update that CentOS system. Current is >> 6.7 (released some 3 months ago) and dovecot-2.0.9-19. > > Ouch! I copied outdated information from my old post. > My current system _is_ CentOS 6.7 with dovecot centos 7.x is latest, so if you need help upgrade make a bug report to centos, not waste your time here since you choiced a precompiled problem, reporting here 7s helpfull if you use dovecot compiled with the latest version here, this migth be a unknown bug you then have found, but not if you use precompiled problems, then report to the maintainers of such software From emmanuel.fuste at thalesgroup.com Thu Nov 26 15:49:20 2015 From: emmanuel.fuste at thalesgroup.com (FUSTE Emmanuel) Date: Thu, 26 Nov 2015 16:49:20 +0100 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <257B4D4E-923B-488D-9285-F85AE87D3EF1@dovecot.fi> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> <56570AED.1020506@thalesgroup.com> <257B4D4E-923B-488D-9285-F85AE87D3EF1@dovecot.fi> Message-ID: <56572A00.1050308@thalesgroup.com> Hi, No, I tried fetching over imapc too exactly as you suggested. In my case it was not from cyrus, but from CriticalPath. isync was finally able to do the job, preserving flags and doing UIDs mapping. The most boring part was generating proper config file for thousands of accounts. A working imapc/dsync would have been better. Emmanuel. Le 26/11/2015 15:24, Sami Ketola a ?crit : > Hi, > > I think you tried to read cyrus mails folders directly. I was talking about fething mails from cyrus over imapc connection. > > Sami > >> On 26 Nov 2015, at 15:36, FUSTE Emmanuel wrote: >> >> Hello, >> >> Because it did not work ? >> In a similar situation, we where forced to use isync/mbsync in imap to >> imap mode because dsync did not work. >> It was reported here more than a year ago (May 2014). >> Time to time, I see the same report from other trying to use dsync to do >> a migration to dovecot. >> Dsync is a very appealing and elegant solution to this usage, but it >> does not always work in real world. >> >> Regards, >> Emmanuel >> >> Le 26/11/2015 12:30, Sami Ketola a ?crit : >>> Hi, >>> >>> With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. >>> >>> http://wiki2.dovecot.org/Migration >>> >>> Sami >>> >>> >>>> On 07 Nov 2015, at 23:35, Forrest wrote: >>>> >>>> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) >>>> >>>> >>>> >>>> On 11/7/15 3:31 PM, Philon wrote: >>>>> Hi there, >>>>> >>>>> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >>>>> >>>>> >>>>> Philon >>>>> >>>>> >>>>>> Am 04.11.2015 um 20:47 schrieb Forrest : >>>>>> >>>>>> I have been attempting to use the cyrus2dovecot script, to no avail. >>>>>> >>>>>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>>>>> >>>>>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>>>>> >>>>>> >>>>>> inboxes=the "myaccount" that was copied over >>>>>> >>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>>>>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>>>>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>>>>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>>>>> myaccount >>>>>> >>>>>> >>>>>> >>>>>> the log output complains of: >>>>>> >>>>>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>>>>> >>>>>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>>>>> >>>>>> All directory paths are correct. >>>>>> >>>>>> >>>>>> Thanks. From tss at iki.fi Thu Nov 26 20:37:10 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Nov 2015 22:37:10 +0200 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <56572A00.1050308@thalesgroup.com> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> <56570AED.1020506@thalesgroup.com> <257B4D4E-923B-488D-9285-F85AE87D3EF1@dovecot.fi> <56572A00.1050308@thalesgroup.com> Message-ID: <6B456ED0-CE0B-467B-8EB2-38B995B1FBD2@iki.fi> v2.2.19 has many fixes related to dsync+imapc, which were found while migrating several million users from GMail. I'm not aware of any problems with it now. Also even before v2.2.19 dsync+imapc has been used to successfully do many large migrations. > On 26 Nov 2015, at 17:49, FUSTE Emmanuel wrote: > > Hi, > > No, I tried fetching over imapc too exactly as you suggested. > In my case it was not from cyrus, but from CriticalPath. > isync was finally able to do the job, preserving flags and doing UIDs > mapping. The most boring part was generating proper config file for > thousands of accounts. > A working imapc/dsync would have been better. > > Emmanuel. > > Le 26/11/2015 15:24, Sami Ketola a ?crit : >> Hi, >> >> I think you tried to read cyrus mails folders directly. I was talking about fething mails from cyrus over imapc connection. >> >> Sami >> >>> On 26 Nov 2015, at 15:36, FUSTE Emmanuel wrote: >>> >>> Hello, >>> >>> Because it did not work ? >>> In a similar situation, we where forced to use isync/mbsync in imap to >>> imap mode because dsync did not work. >>> It was reported here more than a year ago (May 2014). >>> Time to time, I see the same report from other trying to use dsync to do >>> a migration to dovecot. >>> Dsync is a very appealing and elegant solution to this usage, but it >>> does not always work in real world. >>> >>> Regards, >>> Emmanuel >>> >>> Le 26/11/2015 12:30, Sami Ketola a ?crit : >>>> Hi, >>>> >>>> With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. >>>> >>>> http://wiki2.dovecot.org/Migration >>>> >>>> Sami >>>> >>>> >>>>> On 07 Nov 2015, at 23:35, Forrest wrote: >>>>> >>>>> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) >>>>> >>>>> >>>>> >>>>> On 11/7/15 3:31 PM, Philon wrote: >>>>>> Hi there, >>>>>> >>>>>> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >>>>>> >>>>>> >>>>>> Philon >>>>>> >>>>>> >>>>>>> Am 04.11.2015 um 20:47 schrieb Forrest : >>>>>>> >>>>>>> I have been attempting to use the cyrus2dovecot script, to no avail. >>>>>>> >>>>>>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>>>>>> >>>>>>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>>>>>> >>>>>>> >>>>>>> inboxes=the "myaccount" that was copied over >>>>>>> >>>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>>>>>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>>>>>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>>>>>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>>>>>> myaccount >>>>>>> >>>>>>> >>>>>>> >>>>>>> the log output complains of: >>>>>>> >>>>>>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>>>>>> >>>>>>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>>>>>> >>>>>>> All directory paths are correct. >>>>>>> >>>>>>> >>>>>>> Thanks. From tss at iki.fi Thu Nov 26 20:45:20 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Nov 2015 22:45:20 +0200 Subject: FTS, multiple virtualmailbox entries for single mailbox In-Reply-To: <5644F7FB.6060301@searchy.net> References: <5644F7FB.6060301@searchy.net> Message-ID: <82B48685-8A3C-4722-BAE5-2DECA27E4A28@iki.fi> > On 12 Nov 2015, at 22:35, Frank de Bot (lists) wrote: > > Hi, > > i have a dovecot setup with virtual users. Each user has 2 entries for > his mailbox. One is with the username formatted as > %username%@%default_realm% , the other %username%@%domain% passdb and userdb should change the username by returning "user" field, so it'll be the same regardless of how the user logins. From tss at iki.fi Thu Nov 26 20:50:38 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Nov 2015 22:50:38 +0200 Subject: Dsync problems In-Reply-To: <563FBD34.2070700@searchy.net> References: <563FBD34.2070700@searchy.net> Message-ID: <8304CECC-3864-41EE-B885-66F51D6D8E64@iki.fi> > On 08 Nov 2015, at 23:23, Frank de Bot (lists) wrote: > > Hi, > > I'm setting up a migration between 2 dovecot servers 2.1.12 to 2.2.19 . > I've ran into some issues with dsync. > > I use the following command for syncing: > > doveadm -o mail_fsync=never -o imapc_host=%remote% -o > imapc_master_user=master -o imapc_password=%masterpw% -o > imapc_features="rfc822.size fetch-headers" -o mail_prefetch_count=20 -o > imapc_user=%user% backup -1 -R -u %user% imapc: -1 parameter is ignored with backup. > I execute a full sync first and when the mailserver are being migrated > it only would have to do incremental. The first is run is without problems. > But I find the following problem: > > - Flags on current mails aren't updated. If I set message to unread, the > sync won't pick them up. See if adding -f parameter helps. > - On the origin dovecot server I had removed some messages and purged > the inbox. Those message are now in the Trash and are created again and > again when I sync I don't understand what this means. On origin the mails were moved to Trash? And what do you mean created again and again? From tss at iki.fi Thu Nov 26 20:53:11 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 26 Nov 2015 22:53:11 +0200 Subject: dsync imapc authentication mechanisms In-Reply-To: References: Message-ID: > On 05 Nov 2015, at 10:21, Randall Gellens wrote: > > I'm trying to use dsync to mirror mail onto a remote IMAP server. > The remote server is generating an error that authentication via > plain text is not permitted. Does dsync imapc support any > non-clear-text authentication mechanisms? The remote server supports: > > AUTH=CRAM-MD5 AUTH=DIGEST-MD5 AUTH=NTLM imapc supports SSL/TLS to avoid plaintext auth, but right now it can only support LOGIN and PLAIN mechanisms. It could support others, but somebody would just need to implement it to src/lib-sasl/. Afterwards you can use imapc_sasl_mechanisms to set the wanted SASL mechanism. From mancyborg at gmail.com Fri Nov 27 05:43:21 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Fri, 27 Nov 2015 06:43:21 +0100 Subject: [SPAM: high] imap logging ? In-Reply-To: <5656BA57.2040002@enas.net> References: <20151126075127.484f05d8@linux> <5656BA57.2040002@enas.net> Message-ID: <20151127064321.24826610@linux> Works wonders thank you! Finally I can use my accusation finger with my users! http://cdn.scooppick.com/wp-content/uploads/2015/03/Keeps-Talking.jpg On Thu, 26 Nov 2015 08:52:55 +0100 Urban Loesch wrote: > Hi, > > perhaps this is what you need. > > http://wiki2.dovecot.org/Plugins/MailLog > > No "mail_debug enabled" neccessary. > > Regards > Urban > > Am 26.11.2015 um 07:51 schrieb mancyborg at gmail.com: > > Hi I'm trying to log my users imap actions, like when creating a folder, moving an email or deleting an email. > > So I've enabled 'mail_debug' and I'm checking /var/log/dovecot/debug.log > > this is what happens when I delete an email: > > > > Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): query: SELECT password FROM mailbox WHERE username = 'XXX' and active = 1 and restrictedAccess = 0 > > Nov 26 07:46:38 auth: Debug: client out: OK 1 user=XXX > > Nov 26 07:46:38 auth-worker(1555): Debug: sql(XXX,127.0.0.1): SELECT '/var/vmail/XXX/XXX' as home, 5000 AS uid, 5000 AS gid, concat('*:storage=', quota) AS quota_rule FROM mailbox WHERE username = 'XXX' > > Nov 26 07:46:38 auth: Debug: master out: USER 374472705 XXX home=/var/vmail/XXX/XXX uid=5000 gid=5000 quota_rule=*:storage=524288 > > Nov 26 07:46:38 imap(XXX): Debug: Effective uid=5000, gid=5000, home=/var/vmail/XXX/XXX > > Nov 26 07:46:38 imap(XXX): Debug: Quota root: name=User quota backend=maildir args= > > Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=* bytes=536870912 messages=0 > > Nov 26 07:46:38 imap(XXX): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 > > Nov 26 07:46:38 imap(XXX): Debug: maildir++: root=/var/vmail/XXX/XXX/Maildir, index=/var/vmail/XXX/XXX/Maildir/indexes, control=, inbox=/var/vmail/XXX/XXX/Maildir, alt= > > > > and when creating a folder, access an email or moving an email, the output is basically the same: > > I'm unable to find the actual IMAP command. > > > > So, question: is there a way to log IMAP commands to a file ? > > > > Thank you, > > Mike > > From mancyborg at gmail.com Fri Nov 27 07:08:00 2015 From: mancyborg at gmail.com (mancyborg at gmail.com) Date: Fri, 27 Nov 2015 08:08:00 +0100 Subject: logging imap searches ? Message-ID: <20151127080800.4c6217f2@linux> Hi thanks to this list support I'm using Dovecot's MailLog plugin http://wiki2.dovecot.org/Plugins/MailLog to log the following imap events: delete undelete expunge copy mailbox_delete mailbox_rename flag_change save mailbox_create and I'm very happy. Question: is it possible to also log imap searches https://tools.ietf.org/html/rfc3501#section-6.4.4 ? Thank you, Mike From g.danti at assyoma.it Fri Nov 27 08:44:51 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Fri, 27 Nov 2015 09:44:51 +0100 Subject: [g.danti@assyoma.it: [Dovecot] Re: Questions about hardlinks, alternate storage and compression] In-Reply-To: <20151126150115.2F0BA23488@talvi.dovecot.org> References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> Message-ID: > centos 7.x is latest, so if you need help upgrade make a bug report to > centos, not waste your time here since you choiced a precompiled > problem, reporting here 7s helpfull if you use dovecot compiled with > the latest version here, this migth be a unknown bug you then have > found, but not if you use precompiled problems, then report to the > maintainers of such software Hi, does it means that when you use before_sieve and after_sieve in your dovecot configuration the hard-linking part works? I don't need help to upgrade to CentOS 7.x, but before going this route I would understand if this is how dovecot/pigenhole are expected to work (don't use hardlinks) or if I misconfigured something. RedHat / CentOS are a very common base OS, so I wonder if I am the only one hitting this problem or no. Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From emmanuel.fuste at thalesgroup.com Fri Nov 27 09:31:36 2015 From: emmanuel.fuste at thalesgroup.com (FUSTE Emmanuel) Date: Fri, 27 Nov 2015 10:31:36 +0100 Subject: Problems Converting from Cyrus to Dovecot (cyrus2dovecot) In-Reply-To: <6B456ED0-CE0B-467B-8EB2-38B995B1FBD2@iki.fi> References: <563A60E2.4050404@gmail.com> <563E6E99.2090405@gmail.com> <4AC25F7D-0DE6-4E43-A510-67AEBBDCE0E5@dovecot.fi> <56570AED.1020506@thalesgroup.com> <257B4D4E-923B-488D-9285-F85AE87D3EF1@dovecot.fi> <56572A00.1050308@thalesgroup.com> <6B456ED0-CE0B-467B-8EB2-38B995B1FBD2@iki.fi> Message-ID: <565822F8.9080002@thalesgroup.com> Hello Timo, Yes, I follow closely the commit messages of the dovecot-cvs list and lots of things have moved in this area. I will try and expect to be able to use dsync+imapc for our futures migrations. Best regards, Emmanuel. Le 26/11/2015 21:37, Timo Sirainen a ?crit : > v2.2.19 has many fixes related to dsync+imapc, which were found while migrating several million users from GMail. I'm not aware of any problems with it now. Also even before v2.2.19 dsync+imapc has been used to successfully do many large migrations. > >> On 26 Nov 2015, at 17:49, FUSTE Emmanuel wrote: >> >> Hi, >> >> No, I tried fetching over imapc too exactly as you suggested. >> In my case it was not from cyrus, but from CriticalPath. >> isync was finally able to do the job, preserving flags and doing UIDs >> mapping. The most boring part was generating proper config file for >> thousands of accounts. >> A working imapc/dsync would have been better. >> >> Emmanuel. >> >> Le 26/11/2015 15:24, Sami Ketola a ?crit : >>> Hi, >>> >>> I think you tried to read cyrus mails folders directly. I was talking about fething mails from cyrus over imapc connection. >>> >>> Sami >>> >>>> On 26 Nov 2015, at 15:36, FUSTE Emmanuel wrote: >>>> >>>> Hello, >>>> >>>> Because it did not work ? >>>> In a similar situation, we where forced to use isync/mbsync in imap to >>>> imap mode because dsync did not work. >>>> It was reported here more than a year ago (May 2014). >>>> Time to time, I see the same report from other trying to use dsync to do >>>> a migration to dovecot. >>>> Dsync is a very appealing and elegant solution to this usage, but it >>>> does not always work in real world. >>>> >>>> Regards, >>>> Emmanuel >>>> >>>> Le 26/11/2015 12:30, Sami Ketola a ?crit : >>>>> Hi, >>>>> >>>>> With imapsync you will lose message UIDs which means that IMAP clients need to clear their local caches and redownload all messages. Why not use dovecot dsync over imapc instead? It tries to preserve UIDs and Flags. >>>>> >>>>> http://wiki2.dovecot.org/Migration >>>>> >>>>> Sami >>>>> >>>>> >>>>>> On 07 Nov 2015, at 23:35, Forrest wrote: >>>>>> >>>>>> Thank you for the reply. I did find imapsync whilst perusing Google. I will give it a shot, it sounds more realistic/reliable. I have a hoard of emails going back to 1999, so I want as few errors as possible :) >>>>>> >>>>>> >>>>>> >>>>>> On 11/7/15 3:31 PM, Philon wrote: >>>>>>> Hi there, >>>>>>> >>>>>>> I was in the same position, but for mutliple accounts. Still you might want to look at imapsync (https://github.com/imapsync/imapsync), isync and offlineimap. There are more alternatives listed at the imapsync homepage. >>>>>>> >>>>>>> >>>>>>> Philon >>>>>>> >>>>>>> >>>>>>>> Am 04.11.2015 um 20:47 schrieb Forrest : >>>>>>>> >>>>>>>> I have been attempting to use the cyrus2dovecot script, to no avail. >>>>>>>> >>>>>>>> I have many years of content that I want to convert from Cyrus to Dovecot; with the above not working, what are other options out there? Another idea I had is simply set up another IMAP server (using Dovecot) and drag-and-drop and just wait, which I may end up doing. >>>>>>>> >>>>>>>> In the above, I copied over my entire /var/imap and /var/spool/imap to another system; there is only one account (mine), so calling the script was fairly easy; it just doesn't work. >>>>>>>> >>>>>>>> >>>>>>>> inboxes=the "myaccount" that was copied over >>>>>>>> >>>>>>>> /home/myaccount/cyrus2dovecot --cyrus-inbox /home/myaccount/inboxes/%u \ >>>>>>>> --cyrus-seen /home/myaccount/varimap/user/%h/%u.seen \ >>>>>>>> --cyrus-sub /home/varimap/user/%h/%u.sub \ >>>>>>>> --dovecot-inbox /home/myaccount/dovecot/Maildir \ >>>>>>>> myaccount >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> the log output complains of: >>>>>>>> >>>>>>>> cyrus2dovecot [myaccount]: (warning) Index record missing for: INBOX/62020. >>>>>>>> >>>>>>>> and correctly complains about squat indices, as that's not a file it would handle. There is no output into the Maildir, however. >>>>>>>> >>>>>>>> All directory paths are correct. >>>>>>>> >>>>>>>> >>>>>>>> Thanks. From it at vantibolli.com Fri Nov 27 10:47:16 2015 From: it at vantibolli.com (Shakeel Shahzad) Date: Fri, 27 Nov 2015 15:47:16 +0500 Subject: Sieve Max Redirect Message-ID: <565834B4.5040809@vantibolli.com> Hello, I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my dovecot.conf file, I have set sieve_max_redirects as 500. In dovecot (2.2.9) configuration file, I see sieve_max_redirect as 100 but when I add email addresses more than 32, I see that redirect fails. Can you please help me to see this issue and fix it through? If you need anything from my side, please let me know. Regards... Shakeel From tss at iki.fi Fri Nov 27 12:42:56 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Nov 2015 14:42:56 +0200 Subject: auth_request_get_var_expand test fails on big endian arches. In-Reply-To: References: Message-ID: On 02 Nov 2015, at 15:28, Rafael dos Santos wrote: > > Hi, > > when running the tests in a big endian machine, I get the following error: > > 0 / 1 tests failed > test-auth-request-var-expand.c:99: Assert failed: strcmp(str_c(str), > test_output) == 0 > test-auth-request-var-expand.c:104: Assert failed: strcmp(str_c(str), > test_output) == 0 > auth request var expand short and long ............................... : > FAILED Should be fixed by http://hg.dovecot.org/dovecot-2.2/rev/cf956e34dc11 From rdossant at redhat.com Fri Nov 27 12:49:33 2015 From: rdossant at redhat.com (Rafael dos Santos) Date: Fri, 27 Nov 2015 13:49:33 +0100 Subject: auth_request_get_var_expand test fails on big endian arches. In-Reply-To: References: Message-ID: Hi, On 27 November 2015 at 13:42, Timo Sirainen wrote: > On 02 Nov 2015, at 15:28, Rafael dos Santos wrote: > > > > Hi, > > > > when running the tests in a big endian machine, I get the following > error: > > > > 0 / 1 tests failed > > test-auth-request-var-expand.c:99: Assert failed: strcmp(str_c(str), > > test_output) == 0 > > test-auth-request-var-expand.c:104: Assert failed: strcmp(str_c(str), > > test_output) == 0 > > auth request var expand short and long ............................... : > > FAILED > > Should be fixed by http://hg.dovecot.org/dovecot-2.2/rev/cf956e34dc11 > I will test on a big-endian machine and report back. Att. -- Rafael Fonseca From tss at iki.fi Fri Nov 27 12:59:20 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Nov 2015 14:59:20 +0200 Subject: IMAP COPY creates invalid index data with zlib and mail_log plugins enabled but zlib_save off In-Reply-To: <5637C3B6.6020001@tigertech.com> References: <5633EA00.2080603@tigertech.com> <5637C3B6.6020001@tigertech.com> Message-ID: > On 02 Nov 2015, at 22:12, Robert L Mathews wrote: > > On 10/30/15 3:06 PM, Robert L Mathews wrote: > >> I've noticed that maildir IMAP COPY commands can generate invalid >> dovecot.index entries when all seven of the following are true: > > As a followup to my own post, I believe I've tracked this problem down > to this code at lines 119-124 of /src/plugins/zlib/zlib-plugin.c: > > /* don't uncompress input when we are reading a mail that we're just > in the middle of saving, and we didn't do the compression ourself. > in such situation we're probably checking if the user-given input > looks compressed */ > if (_mail->saving && zuser->save_handler == NULL) > return zmail->super.istream_opened(_mail, stream); > > When these lines are removed, the problem no longer happens. > > I'm guessing that an IMAP COPY with the seven circumstances I mentioned > matches "we are reading a mail that we're just in the middle of saving, > and we didn't do the compression ourself". But the mail apparently does > need decompressing in this situation. > > It seems odd for the code to not simply decompress the mail in all > cases. But I'm not sure what other code relies on this check, so I have > no idea whether removing these lines might cause problems. The check is there to make sure that people can't try to exploit bugs in compression libraries by uploading something that Dovecot attempts to decompress later on. So if the input looks compressed it's simply rejected. (If zlib_save is enabled this isn't a problem, because the compressed-looking input is compressed again.) Fixed: http://hg.dovecot.org/dovecot-2.2/rev/574c1e5b3d90 From skdovecot at smail.inf.fh-brs.de Fri Nov 27 13:09:02 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 27 Nov 2015 14:09:02 +0100 (CET) Subject: Sieve Max Redirect In-Reply-To: <565834B4.5040809@vantibolli.com> References: <565834B4.5040809@vantibolli.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 27 Nov 2015, Shakeel Shahzad wrote: > I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my dovecot.conf > file, I have set sieve_max_redirects as 500. In dovecot (2.2.9) configuration > file, I see sieve_max_redirect as 100 but when I add email addresses more > than 32, I see that redirect fails. Can you please help me to see this issue > and fix it through? If you need anything from my side, please let me know. The error message(s) you get :-) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVlhV73z1H7kL/d9rAQJjJQgAx5OJ+mPXRn0XUefam64wbJrCTFglJldk 3nR446FzCXZVIy4KqCGP2XIP8f6XB/iubRodZxwv165Tb1jc+nsEA6a8GDHD4LHu /FTxYEYMx/xbvoZyKRGHyrSK9Gj3Ee3+EJAY34GUGBfCGmBp9/Dzj/db4j/GkbpW P5vR3MvyH9K6uWEShLI1MghBGas2QG1+0nSYeG8VKhaot6G+sfvHfdCkroivHeNM Al3tpstGfweAKqucVDWvPAQnI83lo/j3rIZxHkeILjXTEvMrARDT4/qI4sblpcrY wqJFlTbGeimZrOyV3I4OQFkVxJG15Qn+00wxLRAziezOBpbFeCyl6w== =RERJ -----END PGP SIGNATURE----- From rdossant at redhat.com Fri Nov 27 13:14:16 2015 From: rdossant at redhat.com (Rafael dos Santos) Date: Fri, 27 Nov 2015 14:14:16 +0100 Subject: auth_request_get_var_expand test fails on big endian arches. In-Reply-To: References: Message-ID: Timo, On 27 November 2015 at 13:49, Rafael dos Santos wrote: > > >> Should be fixed by http://hg.dovecot.org/dovecot-2.2/rev/cf956e34dc11 >> > > > I will test on a big-endian machine and report back. > I tested the patch and now all tests pass. Thank you for the fix! Att. -- Rafael Fonseca From tss at iki.fi Fri Nov 27 13:25:14 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 27 Nov 2015 15:25:14 +0200 Subject: fts_solr does not quote lucene's special characters In-Reply-To: <73b23752e767509df021ed9d5d012deb@www.mike2k.de> References: <73b23752e767509df021ed9d5d012deb@www.mike2k.de> Message-ID: On 01 Nov 2015, at 19:03, Michael Stilkerich wrote: > > Hello, > > I am using Dovecot 2.2.19 with the solr backend for full text search and experience the following issue. > > When I issue a search that includes characters that are part of lucene's query syntax (e.g. the double quote character), dovecot does not escape the special characters and issues a request with invalid syntax to the solr server. http://hg.dovecot.org/dovecot-2.2/rev/9d5c59d98857 should help. From dovecot at d9c.eu Fri Nov 27 13:59:21 2015 From: dovecot at d9c.eu (Andreas Fischer) Date: Fri, 27 Nov 2015 14:59:21 +0100 Subject: doveadm-backup verbosity? Message-ID: <565861B9.80208@d9c.eu> Hi, according to the man page doveadm-backup, the verbosity switch "-v" should do the following: "Enables verbosity, including progress counter" However, i see no difference running the following command with or without the "-v" switch: >>doveadm -v backup -R -A remote:myhost The command itself works without problem, two mailboxes are correctly back-upped, but the output is only an empty line. Not sure if this is expected behavior. Regards # 2.2.19: /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.9 (357ac0a0e68b+) # OS: Linux 4.2.5-1-ARCH x86_64 dsync_remote_cmd = ssh -p 19022 -l root %{host} doveadm dsync-server -u%u first_valid_uid = 99 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate editheader passdb { args = nopassword=y driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve sieve_extensions = +editheader } protocols = service anvil { process_min_avail = 0 unix_listener anvil-auth-penalty { mode = 064 } } ssl = no userdb { args = /etc/dovecot/users default_fields = uid=nobody gid=nobody home=/backup/tmp/mail/%d/%n mail=sdbox:~/box driver = passwd-file } From it at vantibolli.com Fri Nov 27 14:31:58 2015 From: it at vantibolli.com (Shakeel Shahzad) Date: Fri, 27 Nov 2015 19:31:58 +0500 Subject: Sieve Max Redirect In-Reply-To: References: <565834B4.5040809@vantibolli.com> Message-ID: <5658695E.1020509@vantibolli.com> @Steffan: This is the error I see in log file: sieve: info: started log at Nov 27 14:30:15. main script: line 42: error: total number of actions exceeds policy limit (33 > 32). Regards... Shakeel On 11/27/2015 06:09 PM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 27 Nov 2015, Shakeel Shahzad wrote: > >> I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my >> dovecot.conf file, I have set sieve_max_redirects as 500. In dovecot >> (2.2.9) configuration file, I see sieve_max_redirect as 100 but when >> I add email addresses more than 32, I see that redirect fails. Can >> you please help me to see this issue and fix it through? If you need >> anything from my side, please let me know. > > The error message(s) you get :-) > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVlhV73z1H7kL/d9rAQJjJQgAx5OJ+mPXRn0XUefam64wbJrCTFglJldk > 3nR446FzCXZVIy4KqCGP2XIP8f6XB/iubRodZxwv165Tb1jc+nsEA6a8GDHD4LHu > /FTxYEYMx/xbvoZyKRGHyrSK9Gj3Ee3+EJAY34GUGBfCGmBp9/Dzj/db4j/GkbpW > P5vR3MvyH9K6uWEShLI1MghBGas2QG1+0nSYeG8VKhaot6G+sfvHfdCkroivHeNM > Al3tpstGfweAKqucVDWvPAQnI83lo/j3rIZxHkeILjXTEvMrARDT4/qI4sblpcrY > wqJFlTbGeimZrOyV3I4OQFkVxJG15Qn+00wxLRAziezOBpbFeCyl6w== > =RERJ > -----END PGP SIGNATURE----- From stephan at rename-it.nl Fri Nov 27 15:03:44 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 27 Nov 2015 16:03:44 +0100 Subject: Sieve Max Redirect In-Reply-To: <5658695E.1020509@vantibolli.com> References: <565834B4.5040809@vantibolli.com> <5658695E.1020509@vantibolli.com> Message-ID: <565870D0.6060607@rename-it.nl> Op 27-11-2015 om 15:31 schreef Shakeel Shahzad: > @Steffan: This is the error I see in log file: > > > sieve: info: started log at Nov 27 14:30:15. > main script: line 42: error: total number of actions exceeds policy > limit (33 > 32). http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Configurable_Limits Regards, Stephan. > On 11/27/2015 06:09 PM, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Fri, 27 Nov 2015, Shakeel Shahzad wrote: >> >>> I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my >>> dovecot.conf file, I have set sieve_max_redirects as 500. In dovecot >>> (2.2.9) configuration file, I see sieve_max_redirect as 100 but when >>> I add email addresses more than 32, I see that redirect fails. Can >>> you please help me to see this issue and fix it through? If you need >>> anything from my side, please let me know. >> >> The error message(s) you get :-) >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVlhV73z1H7kL/d9rAQJjJQgAx5OJ+mPXRn0XUefam64wbJrCTFglJldk >> 3nR446FzCXZVIy4KqCGP2XIP8f6XB/iubRodZxwv165Tb1jc+nsEA6a8GDHD4LHu >> /FTxYEYMx/xbvoZyKRGHyrSK9Gj3Ee3+EJAY34GUGBfCGmBp9/Dzj/db4j/GkbpW >> P5vR3MvyH9K6uWEShLI1MghBGas2QG1+0nSYeG8VKhaot6G+sfvHfdCkroivHeNM >> Al3tpstGfweAKqucVDWvPAQnI83lo/j3rIZxHkeILjXTEvMrARDT4/qI4sblpcrY >> wqJFlTbGeimZrOyV3I4OQFkVxJG15Qn+00wxLRAziezOBpbFeCyl6w== >> =RERJ >> -----END PGP SIGNATURE----- From it at vantibolli.com Fri Nov 27 15:34:33 2015 From: it at vantibolli.com (Shakeel Shahzad) Date: Fri, 27 Nov 2015 20:34:33 +0500 Subject: Sieve Max Redirect In-Reply-To: <565870D0.6060607@rename-it.nl> References: <565834B4.5040809@vantibolli.com> <5658695E.1020509@vantibolli.com> <565870D0.6060607@rename-it.nl> Message-ID: <56587809.2020204@vantibolli.com> Thank you very much @Steffan. I just needed to add sieve_max_actions. Regards... Shakeel On 11/27/2015 08:03 PM, Stephan Bosch wrote: > > > Op 27-11-2015 om 15:31 schreef Shakeel Shahzad: >> @Steffan: This is the error I see in log file: >> >> >> sieve: info: started log at Nov 27 14:30:15. >> main script: line 42: error: total number of actions exceeds policy >> limit (33 > 32). > > http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#Configurable_Limits > > > Regards, > > Stephan. > > > >> On 11/27/2015 06:09 PM, Steffen Kaiser wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Fri, 27 Nov 2015, Shakeel Shahzad wrote: >>> >>>> I have configured iRedMail ( 0.9.1) on Ubuntu (14.04 LTS). In my >>>> dovecot.conf file, I have set sieve_max_redirects as 500. In >>>> dovecot (2.2.9) configuration file, I see sieve_max_redirect as 100 >>>> but when I add email addresses more than 32, I see that redirect >>>> fails. Can you please help me to see this issue and fix it through? >>>> If you need anything from my side, please let me know. >>> >>> The error message(s) you get :-) >>> >>> - -- Steffen Kaiser >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1 >>> >>> iQEVAwUBVlhV73z1H7kL/d9rAQJjJQgAx5OJ+mPXRn0XUefam64wbJrCTFglJldk >>> 3nR446FzCXZVIy4KqCGP2XIP8f6XB/iubRodZxwv165Tb1jc+nsEA6a8GDHD4LHu >>> /FTxYEYMx/xbvoZyKRGHyrSK9Gj3Ee3+EJAY34GUGBfCGmBp9/Dzj/db4j/GkbpW >>> P5vR3MvyH9K6uWEShLI1MghBGas2QG1+0nSYeG8VKhaot6G+sfvHfdCkroivHeNM >>> Al3tpstGfweAKqucVDWvPAQnI83lo/j3rIZxHkeILjXTEvMrARDT4/qI4sblpcrY >>> wqJFlTbGeimZrOyV3I4OQFkVxJG15Qn+00wxLRAziezOBpbFeCyl6w== >>> =RERJ >>> -----END PGP SIGNATURE----- From a.helwig at heinlein-support.de Fri Nov 27 15:51:12 2015 From: a.helwig at heinlein-support.de (Andre Helwig) Date: Fri, 27 Nov 2015 16:51:12 +0100 Subject: ldap paging Message-ID: <56587BF0.6010608@heinlein-support.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi List, in ldapsearch you are able to use paging in case of limited results on ldap server side with "-E pr=10000/noprompt" Is there any way to get the same effect for the iterate query? Thanks Andre -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJWWHvwAAoJEAoTNwRDnEhRVmMH/0+vGRx7urQ49bHbzc4asD7r XlbwS7whkDKrJpflK39PLPw5BJoptKhwLNOueotFv9Pq8EYQBkN/gX69BzxeoFk8 DZpiP/ogYQIG1lUijes4UkbttVpnviy9/mLBKjIfIkvGDu0q3zwuq5876q6t2Qxx kJkpLSizxAfnGLeD03txVPAc8wGzxhxkyEdosQ+ARriYI1Ce+dfK28z6hSAEXMA+ 4JZEvIsy6P+xepFCFYILFWeRO2YBzqHtgtjKu8KCiyBdUPG0Nii4z2YmN2AoCj7m rQ8l4CeTDw+lk0pMy26kCsmh+PK2/OXCUGjxgYJOR3mxn2HNSIfUXFVWs4ri464= =qWS+ -----END PGP SIGNATURE----- From p.heinlein at heinlein-support.de Sat Nov 28 07:26:10 2015 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sat, 28 Nov 2015 08:26:10 +0100 Subject: How to delete "proxy" attribute in passdb lookup? Message-ID: <56595712.5060205@heinlein-support.de> Hi, I have a complicated SQL backend with stored procedures to generate passdb/userdb queries. This result have the proxy-attribute set and it's very complicated to change that procedures. On one system I tried to override/delete the proxy attribute: passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf.ext override_fields = proxy=n } Looks like that doesn't work. "proxy" is always set. doveadm auth lookup username passdb: username user : username host : mailserver proxy hostip : xx.xx.xx.xx Is there any way to override/delete an existing true/false attribute like "proxy"? Peer -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin From sytse at pobox.com Sat Nov 28 17:14:08 2015 From: sytse at pobox.com (Sytse Kuijk) Date: Sat, 28 Nov 2015 18:14:08 +0100 Subject: Connection between postfix and dovecot failing Message-ID: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> Hi all, Hope someone has been here and done that before. I am running a Mac mini with OSX server version 10.5, that is on El Capitan, but the problem I have started on Yosemite (the previous version) and did not change at all in the upgrade (I had hoped it would have solved itself). As you might know, OSX server uses postfix and dovecot as mailing engines, good choice. Since we suffered a short power outage a little over a week ago, the mail has failed in delivering messages to the local mailboxes. We can send mail out, check the mailboxes (they never change right now),etc. but incoming mail was first just not delivered and after a week warning are sent back to the senders, telling there is a temporary problem (hope that this message is right and I can solve the issue). What seems to go wrong is that postfix cannot connect to the local MTA. The error message that appears in several places is: recipient=ann.cools at surikata.be offset=769 dsn_orig_rcpt=rfc822;ann.cools at surikata.be status=4.4.1 action=delayed reason=delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused I find all sorts of explanations and cures, but none work up to now. I tried to find lock files that might be blocking, deleted some files ending in lck with zero size, with no result. I tried to switch off the spam/virus/junk features, no difference (some mail said that messages in the queue will not get through, but new ones would. Not so, not a single message up to now). I also get alerts in the Server application, warning that: An unassigned mail directory has been detected and is scheduled to be deleted. The unassigned mail directory located in: /Library/Server/Mail/Data/mail/702C16A8-6B2D-42D8-9362-659DD2949AFC is not associated with any user account and is scheduled to be deleted in 1 days. In the mean time no mail, and unfortunately, we need it badly. Any suggestions? Million thanks for your kind support. sytse -- ------------------------------------------ - Sytse Kuijk - ___ - R.S.R. n.v. - o|o - Gijzelaarsstraat 16 - \_____(.)_____/ - 2000 Antwerpen - - GSM: +32 468 20 00 00 - ___ - Phone: +32 3 216 4328 - o|o - E-mail: sytse at pobox.com - \_____(.)_____/ ------------------------------------------ iThink therefore iMac From sytse at pobox.com Sat Nov 28 17:34:32 2015 From: sytse at pobox.com (Sytse Kuijk) Date: Sat, 28 Nov 2015 18:34:32 +0100 Subject: Connection between postfix and dovecot failing In-Reply-To: <26F1C14B-D972-46DB-8D11-6A0BEE2BC8C8@pettijohn-web.com> References: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> <26F1C14B-D972-46DB-8D11-6A0BEE2BC8C8@pettijohn-web.com> Message-ID: <193B8D42-A9B9-4C58-A822-8BACFD3770B5@pobox.com> Well, I tried that. I can connect a telnet session on that port and that does not refuse the connection, it sits there waiting, but I do not know the protocol so I cannot check if it does what it is supposed to. I also checked who is listening on that port, here is the dialog: sh-3.2# sudo lsof -i -n -P | grep 10024 perl5.18 75844 _amavisd 5u IPv4 0x866f6119e39f833d 0t0 TCP 127.0.0.1:10024 (LISTEN) perl5.18 75844 _amavisd 6u IPv6 0x866f6119e03f253d 0t0 TCP [::1]:10024 (LISTEN) looks sort of correct to me? sytse -- ------------------------------------------ - Sytse Kuijk - ___ - R.S.R. n.v. - o|o - Gijzelaarsstraat 16 - \_____(.)_____/ - 2000 Antwerpen - - GSM: +32 468 20 00 00 - ___ - Phone: +32 3 216 4328 - o|o - E-mail: sytse at pobox.com - \_____(.)_____/ ------------------------------------------ iThink therefore iMac > On 28 nov. 2015, at 18:31, Edgar Pettifogging wrote: > > I would bet that whatever is listening at 127.0.0.1 port 10024 didn't get restarted when the power came back on. Amavis perhaps or another proxy server of some sort. > > Sent from my iPhone > >> On Nov 28, 2015, at 11:14 AM, Sytse Kuijk wrote: >> >> Hi all, >> >> Hope someone has been here and done that before. >> I am running a Mac mini with OSX server version 10.5, that is on El Capitan, but the problem I have started on Yosemite (the previous version) and did not change at all in the upgrade (I had hoped it would have solved itself). >> >> As you might know, OSX server uses postfix and dovecot as mailing engines, good choice. >> Since we suffered a short power outage a little over a week ago, the mail has failed in delivering messages to the local mailboxes. >> We can send mail out, check the mailboxes (they never change right now),etc. but incoming mail was first just not delivered and after a week warning are sent back to the senders, telling there is a temporary problem (hope that this message is right and I can solve the issue). >> >> What seems to go wrong is that postfix cannot connect to the local MTA. The error message that appears in several places is: >> >> recipient=ann.cools at surikata.be >> offset=769 >> dsn_orig_rcpt=rfc822;ann.cools at surikata.be >> status=4.4.1 >> action=delayed >> reason=delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused >> >> I find all sorts of explanations and cures, but none work up to now. >> I tried to find lock files that might be blocking, deleted some files ending in lck with zero size, with no result. >> I tried to switch off the spam/virus/junk features, no difference (some mail said that messages in the queue will not get through, but new ones would. Not so, not a single message up to now). >> I also get alerts in the Server application, warning that: >> >> An unassigned mail directory has been detected and is scheduled to be deleted. >> >> The unassigned mail directory located in: >> /Library/Server/Mail/Data/mail/702C16A8-6B2D-42D8-9362-659DD2949AFC >> is not associated with any user account and is scheduled to be deleted in 1 days. >> >> In the mean time no mail, and unfortunately, we need it badly. >> >> Any suggestions? >> >> Million thanks for your kind support. >> >> sytse >> -- >> ------------------------------------------ >> - Sytse Kuijk - ___ >> - R.S.R. n.v. - o|o >> - Gijzelaarsstraat 16 - \_____(.)_____/ >> - 2000 Antwerpen - >> - GSM: +32 468 20 00 00 - ___ >> - Phone: +32 3 216 4328 - o|o >> - E-mail: sytse at pobox.com - \_____(.)_____/ >> ------------------------------------------ >> >> iThink therefore iMac From sytse at pobox.com Sat Nov 28 21:42:15 2015 From: sytse at pobox.com (Sytse Kuijk) Date: Sat, 28 Nov 2015 22:42:15 +0100 Subject: Connection between postfix and dovecot failing In-Reply-To: <613AFCD4-F2EF-43D4-A420-7427EBB27102@pettijohn-web.com> References: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> <26F1C14B-D972-46DB-8D11-6A0BEE2BC8C8@pettijohn-web.com> <193B8D42-A9B9-4C58-A822-8BACFD3770B5@pobox.com> <613AFCD4-F2EF-43D4-A420-7427EBB27102@pettijohn-web.com> Message-ID: <4D25D461-C5F4-4601-901A-6771C2A5247D@pobox.com> Well, post fix tries, but cannot connect. Sounds to me like problem at the receiving side. In any case, one question would be: is amavis part of dovecot or of postfix? sytse -- ------------------------------------------ - Sytse Kuijk - ___ - R.S.R. n.v. - o|o - Gijzelaarsstraat 16 - \_____(.)_____/ - 2000 Antwerpen - - GSM: +32 468 20 00 00 - ___ - Phone: +32 3 216 4328 - o|o - E-mail: sytse at pobox.com - \_____(.)_____/ ------------------------------------------ iThink therefore iMac > On 28 nov. 2015, at 22:24, Edgar Pettifogging wrote: > > This should probably move to postfix list. Doesn't look like dovecot gets a chance to deliver. > > Sent from my iPhone > >> On Nov 28, 2015, at 11:34 AM, Sytse Kuijk wrote: >> >> Well, I tried that. I can connect a telnet session on that port and that does not refuse the connection, it sits there waiting, but I do not know the protocol so I cannot check if it does what it is supposed to. >> >> I also checked who is listening on that port, here is the dialog: >> >> sh-3.2# sudo lsof -i -n -P | grep 10024 >> perl5.18 75844 _amavisd 5u IPv4 0x866f6119e39f833d 0t0 TCP 127.0.0.1:10024 (LISTEN) >> perl5.18 75844 _amavisd 6u IPv6 0x866f6119e03f253d 0t0 TCP [::1]:10024 (LISTEN) >> >> looks sort of correct to me? >> >> sytse >> >> -- >> >> ------------------------------------------ >> - Sytse Kuijk - ___ >> - R.S.R. n.v. - o|o >> - Gijzelaarsstraat 16 - \_____(.)_____/ >> - 2000 Antwerpen - >> - GSM: +32 468 20 00 00 - ___ >> - Phone: +32 3 216 4328 - o|o >> - E-mail: sytse at pobox.com - \_____(.)_____/ >> ------------------------------------------ >> >> iThink therefore iMac >> >>> On 28 nov. 2015, at 18:31, Edgar Pettifogging wrote: >>> >>> I would bet that whatever is listening at 127.0.0.1 port 10024 didn't get restarted when the power came back on. Amavis perhaps or another proxy server of some sort. >>> >>> Sent from my iPhone >>> >>>> On Nov 28, 2015, at 11:14 AM, Sytse Kuijk wrote: >>>> >>>> Hi all, >>>> >>>> Hope someone has been here and done that before. >>>> I am running a Mac mini with OSX server version 10.5, that is on El Capitan, but the problem I have started on Yosemite (the previous version) and did not change at all in the upgrade (I had hoped it would have solved itself). >>>> >>>> As you might know, OSX server uses postfix and dovecot as mailing engines, good choice. >>>> Since we suffered a short power outage a little over a week ago, the mail has failed in delivering messages to the local mailboxes. >>>> We can send mail out, check the mailboxes (they never change right now),etc. but incoming mail was first just not delivered and after a week warning are sent back to the senders, telling there is a temporary problem (hope that this message is right and I can solve the issue). >>>> >>>> What seems to go wrong is that postfix cannot connect to the local MTA. The error message that appears in several places is: >>>> >>>> recipient=ann.cools at surikata.be >>>> offset=769 >>>> dsn_orig_rcpt=rfc822;ann.cools at surikata.be >>>> status=4.4.1 >>>> action=delayed >>>> reason=delivery temporarily suspended: connect to 127.0.0.1[127.0.0.1]:10024: Connection refused >>>> >>>> I find all sorts of explanations and cures, but none work up to now. >>>> I tried to find lock files that might be blocking, deleted some files ending in lck with zero size, with no result. >>>> I tried to switch off the spam/virus/junk features, no difference (some mail said that messages in the queue will not get through, but new ones would. Not so, not a single message up to now). >>>> I also get alerts in the Server application, warning that: >>>> >>>> An unassigned mail directory has been detected and is scheduled to be deleted. >>>> >>>> The unassigned mail directory located in: >>>> /Library/Server/Mail/Data/mail/702C16A8-6B2D-42D8-9362-659DD2949AFC >>>> is not associated with any user account and is scheduled to be deleted in 1 days. >>>> >>>> In the mean time no mail, and unfortunately, we need it badly. >>>> >>>> Any suggestions? >>>> >>>> Million thanks for your kind support. >>>> >>>> sytse >>>> -- >>>> ------------------------------------------ >>>> - Sytse Kuijk - ___ >>>> - R.S.R. n.v. - o|o >>>> - Gijzelaarsstraat 16 - \_____(.)_____/ >>>> - 2000 Antwerpen - >>>> - GSM: +32 468 20 00 00 - ___ >>>> - Phone: +32 3 216 4328 - o|o >>>> - E-mail: sytse at pobox.com - \_____(.)_____/ >>>> ------------------------------------------ >>>> >>>> iThink therefore iMac From noeldude at gmail.com Sat Nov 28 22:29:37 2015 From: noeldude at gmail.com (Noel) Date: Sat, 28 Nov 2015 16:29:37 -0600 Subject: Connection between postfix and dovecot failing In-Reply-To: <4D25D461-C5F4-4601-901A-6771C2A5247D@pobox.com> References: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> <26F1C14B-D972-46DB-8D11-6A0BEE2BC8C8@pettijohn-web.com> <193B8D42-A9B9-4C58-A822-8BACFD3770B5@pobox.com> <613AFCD4-F2EF-43D4-A420-7427EBB27102@pettijohn-web.com> <4D25D461-C5F4-4601-901A-6771C2A5247D@pobox.com> Message-ID: <565A2AD1.1080107@gmail.com> On 11/28/2015 3:42 PM, Sytse Kuijk wrote: > Well, post fix tries, but cannot connect. Sounds to me like problem at the receiving side. > In any case, one question would be: is amavis part of dovecot or of postfix? > > sytse > Neither. Amavis (hopefully you're using the version called amavisd-new) is a third party content filter you've enabled separately. There is an amavis-users mail list where you can get help specific to amavisd-new. Apparently amavis isn't working correctly. If restarting amavis doesn't help, look in the logs for errors from amavis. You can (temporarily) disable the content filter and get mail flowing by commenting out any "content_filter" statements in postfix main.cf or master.cf, then requeue all deferred mail with "postsuper -r ALL" Further discussion of this should move to the amavis-users list. https://www.ijs.si/software/amavisd/ -- Noel Jones From tss at iki.fi Sun Nov 29 00:01:35 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 29 Nov 2015 02:01:35 +0200 Subject: How to delete "proxy" attribute in passdb lookup? In-Reply-To: <56595712.5060205@heinlein-support.de> References: <56595712.5060205@heinlein-support.de> Message-ID: On 28 Nov 2015, at 09:26, Peer Heinlein wrote: > Is there any way to override/delete an existing true/false attribute > like "proxy"? Not right now, because any value at all is treated as "true" (and I'm pretty sure even empty value). I suppose the best solution would be to finally change all the passdb/userdb/plugin boolean values to actually support "no" value as not being enabled. It's something that we're thinking about for Dovecot v2.3. It would probably also be useful to add support for removing a field entirely, such as returning "-proxy" (where the =value would be ignored). I'm not entirely sure how useful that would be though. From angusf at multizone.co.uk Sun Nov 29 10:55:10 2015 From: angusf at multizone.co.uk (Angus Fox) Date: Sun, 29 Nov 2015 10:55:10 +0000 Subject: Connection between postfix and dovecot failing In-Reply-To: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> References: <46BA591E-AF2D-4E47-A34F-C6CEF077732F@pobox.com> Message-ID: <7885CF41-E652-45DB-AED1-974678B0FF0A@multizone.co.uk> On 28 Nov 2015, at 17:14, Sytse Kuijk wrote: >I am running a Mac mini with OSX server version 10.5, that is on El Capitan, but the problem I have started on Yosemite (the previous version) and did not change at all in the upgrade (I had hoped it would have solved itself). ? Since we suffered a short power outage a little over a week ago, the mail has failed in delivering messages to the local mailboxes. So it sounds like you ran an upgrade from Mac OS X 10.10.x and OS X Server 5.x or 4.x to Mac OS X 10.11.1 and OS X Server 5.0.15 in order to fix a problem caused by a power outage. I would suggest reverting to a backup from immediately before the power outage and getting it working first before trying to run the upgrade (and the migration it runs) - I have had OS X Server become unrecoverable before due to a power failure and had to rebuild from scratch. Upgrading could not fix the underlying problems with the state that the filesystem was in because there had been a data loss. This was a very painful experience. Sorry this isn?t specifically Dovecot related, although I have been running Dovecot for years now on a live Mac OS X computer and upgraded to OS X Server 5.0.15 recently with very few issues other than the usual Apple simplification complexities and the fact that the OS X Server installer took a *very* long time to do the migration. Angus From p.heinlein at heinlein-support.de Sun Nov 29 17:00:24 2015 From: p.heinlein at heinlein-support.de (Peer Heinlein) Date: Sun, 29 Nov 2015 18:00:24 +0100 Subject: How to delete "proxy" attribute in passdb lookup? In-Reply-To: References: <56595712.5060205@heinlein-support.de> Message-ID: <565B2F28.6090009@heinlein-support.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Am 29.11.2015 um 01:01 schrieb Timo Sirainen: Hi Timo, > Not right now, because any value at all is treated as "true" (and > I'm pretty sure even empty value). I suppose the best solution > would be to finally change all the passdb/userdb/plugin boolean > values to actually support "no" value as not being enabled. It's > something that we're thinking about for Dovecot v2.3. Okay, I expected that, thanks for clarification. > It would probably also be useful to add support for removing a > field entirely, such as returning "-proxy" (where the =value would > be ignored). I'm not entirely sure how useful that would be > though. Why don't just enable override_fields = proxy= so, just use "attribue=" to delete it? Peer - -- Heinlein Support GmbH Schwedter Str. 8/9b, 10119 Berlin http://www.heinlein-support.de Tel: 030 / 405051-42 Fax: 030 / 405051-19 Zwangsangaben lt. ?35a GmbHG: HRB 93818 B / Amtsgericht Berlin-Charlottenburg, Gesch?ftsf?hrer: Peer Heinlein -- Sitz: Berlin -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJWWy8oAAoJEAOLLpq5E82HEEkH/2y4kcOzGTSlIHvJ+SIwT+hu Lnw2wugXSe/o6VQF+f9O/VB2V21QUy8JaDVHiWMhkqZXjReMzse5zGC5f4rkJe+p 2cPHHxhfaU4KJinNPMEJNMlewwXMPvCWzO8bVUqSC5s1umSe9Ihq8uM6c0wXbd0X bQ2pMyZEOz8z3DxdvxhPpkj2SupLCJ8Q3QBhrt/lFupa8bGNGSS/KeR9NRUsLHVJ 5KuJZwHl745TMSaCS7Z4NhaUdBsvRfuirg8AhcWQ1sExW+nDYgOvTlECNp427aIT PvkfDkROT/yX0dsQzX4jej0d/YlaQaJVUhxY4syuy807dclVF+sBTXHqgiqKqRI= =q87r -----END PGP SIGNATURE----- From tss at iki.fi Sun Nov 29 18:25:43 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 29 Nov 2015 20:25:43 +0200 Subject: How to delete "proxy" attribute in passdb lookup? In-Reply-To: <565B2F28.6090009@heinlein-support.de> References: <56595712.5060205@heinlein-support.de> <565B2F28.6090009@heinlein-support.de> Message-ID: On 29 Nov 2015, at 19:00, Peer Heinlein wrote: > >> It would probably also be useful to add support for removing a >> field entirely, such as returning "-proxy" (where the =value would >> be ignored). I'm not entirely sure how useful that would be >> though. > > Why don't just enable > > override_fields = proxy= > > so, just use "attribue=" to delete it? There could be some settings which are wanted to be explicitly cleared. For example in dovecot.conf have "mail_plugins=foo" while in passdb it could be disabled with "mail_plugins=". There are probably some better examples. From stephan at rename-it.nl Sun Nov 29 19:20:08 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 29 Nov 2015 20:20:08 +0100 Subject: Sieve "foreverypart" and "mime" extensions (RFC 5703) now implemented Message-ID: <565B4FE8.2070306@rename-it.nl> Hi, I have finally managed to implement the Sieve "foreverypart" and "mime" extensions (RFC 5703). These are now included in the main Mercurial repository and will be included in the next release. I will be fixing a few minor details in the coming week; e.g., interaction with "include" and "editheader" extensions needs more work. Still, in the mean time, those of you who are interested can already test this feature a bit. It is not a small feature, so knowing bugs early is helpful. You can install it from the Mercurial repository or get the latest Debian packages from Xi. When I finish the last details (probably next week), I will release a first RC version of Pigeonhole v0.4.10 right away. Regards, Stephan. From rs at sys4.de Sun Nov 29 19:48:18 2015 From: rs at sys4.de (Robert Schetterer) Date: Sun, 29 Nov 2015 20:48:18 +0100 Subject: Sieve "foreverypart" and "mime" extensions (RFC 5703) now implemented In-Reply-To: <565B4FE8.2070306@rename-it.nl> References: <565B4FE8.2070306@rename-it.nl> Message-ID: <565B5682.9040402@sys4.de> Am 29.11.2015 um 20:20 schrieb Stephan Bosch: > Hi, > > I have finally managed to implement the Sieve "foreverypart" and "mime" > extensions (RFC 5703). These are now included in the main Mercurial > repository and will be included in the next release. > > I will be fixing a few minor details in the coming week; e.g., > interaction with "include" and "editheader" extensions needs more work. > Still, in the mean time, those of you who are interested can already > test this feature a bit. It is not a small feature, so knowing bugs > early is helpful. You can install it from the Mercurial repository or > get the latest Debian packages from Xi. > > When I finish the last details (probably next week), I will release a > first RC version of Pigeonhole v0.4.10 right away. > > Regards, > > Stephan. > Hi Stephan, thats good news, thx for coding so i can go on with tests about forward mail without SPF DMARC conflicts via sieve i ve written a german blog about that a year ago https://sys4.de/de/blog/2014/10/29/email-forward-mit-sieve-ohne-spf-dmarc-und-dkim-konflikte/ Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From g.danti at assyoma.it Mon Nov 30 08:21:51 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Mon, 30 Nov 2015 09:21:51 +0100 Subject: Questions about hardlinks, alternate storage and compression] In-Reply-To: References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> Message-ID: <565C071F.1000302@assyoma.it> Hi all, I have some updates on the hard-link discussion. First, let me explain that I installed a test machine with CentOS 7.1 and dovecot/pigeonhole version 2.2.10-4 and the results where identical on what I had on CentOS 6.7 and dovecot 2.0.9-19 The bottom line is that hardlinking works only when no, or at most only one, RCPT have sieve filtering. For example: - if no RCPT has sieve filters, hardlink is OK - if only one RCPT has sieve filters, hardlink is OK - if two or more RCPTs have sieve filters, hardlink is NOT working. Some time ago I reported that, when disabling the sieve_before and sieve_after options, hardlinks worked; however, this really was a side effect of what written above. So, let me do a straigth question: is someone using dovecot/LMTP with hardlinking? To me, this seems a _very_ important feature, and I wonder if I am doing something wrong or if the feature (hardlink+sieve) simply does not exists. Thanks. On 27/11/2015 09:44, Gionatan Danti wrote: >> centos 7.x is latest, so if you need help upgrade make a bug report to >> centos, not waste your time here since you choiced a precompiled >> problem, reporting here 7s helpfull if you use dovecot compiled with >> the latest version here, this migth be a unknown bug you then have >> found, but not if you use precompiled problems, then report to the >> maintainers of such software > > Hi, > does it means that when you use before_sieve and after_sieve in your > dovecot configuration the hard-linking part works? > > I don't need help to upgrade to CentOS 7.x, but before going this route > I would understand if this is how dovecot/pigenhole are expected to work > (don't use hardlinks) or if I misconfigured something. > > RedHat / CentOS are a very common base OS, so I wonder if I am the only > one hitting this problem or no. > Thanks. > -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From tss at iki.fi Mon Nov 30 13:23:10 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 30 Nov 2015 15:23:10 +0200 Subject: Questions about hardlinks, alternate storage and compression] In-Reply-To: <565C071F.1000302@assyoma.it> References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> <565C071F.1000302@assyoma.it> Message-ID: On 30 Nov 2015, at 10:21, Gionatan Danti wrote: > > So, let me do a straigth question: is someone using dovecot/LMTP with hardlinking? To me, this seems a _very_ important feature, and I wonder if I am doing something wrong or if the feature (hardlink+sieve) simply does not exists. Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO entry about it: - remove mail_deliver_session after all, do all the stuff transparently by hooking into mailbox_copy(). - use this hook also to do the mail deduplication: 1) sort all destination users, 2) create mail_user only once for each user, 3) remember in src_mail the previously copied mail, 4) use that for mailbox_copy()ing to following recipients - make sure this removes duplicate dbox mails when sieve saves mail to multiple mailboxes From frido at 0tten.nl Mon Nov 30 14:31:41 2015 From: frido at 0tten.nl (Frido Otten) Date: Mon, 30 Nov 2015 15:31:41 +0100 Subject: Sieve, shared folders, different daemons Message-ID: <565C5DCD.1090706@0tten.nl> Hi, In our dovecot setup we use 3 different config files. One for maildropping(sieve), one for dovecot/imap on standard ports with ssl config and one for dovecot/imap on standard ports+1 with a different ssl certificate. Currently the client using the standard port+1 config wants to make use of shared folders, but the thing is that the current namespace separator in all configs is '.'. This is conflicting with shared folders. Can we simply run one of the dovecot/imap daemons configured with a different separator and shared folders without having impact on sieve and the current connected users to the standard ports? Regards, Frido -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: OpenPGP digital signature URL: From g.danti at assyoma.it Mon Nov 30 15:48:50 2015 From: g.danti at assyoma.it (Gionatan Danti) Date: Mon, 30 Nov 2015 16:48:50 +0100 Subject: Questions about hardlinks, alternate storage and compression] In-Reply-To: References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> <565C071F.1000302@assyoma.it> Message-ID: <565C6FE2.3080105@assyoma.it> Hi Timo, glad to know it is in your TODO list ;) Any rough ETA on that? Thanks. On 30/11/2015 14:23, Timo Sirainen wrote: > On 30 Nov 2015, at 10:21, Gionatan Danti wrote: >> >> So, let me do a straigth question: is someone using dovecot/LMTP with hardlinking? To me, this seems a _very_ important feature, and I wonder if I am doing something wrong or if the feature (hardlink+sieve) simply does not exists. > > Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO entry about it: > > - remove mail_deliver_session after all, do all the stuff transparently > by hooking into mailbox_copy(). > - use this hook also to do the mail deduplication: 1) sort all destination > users, 2) create mail_user only once for each user, 3) remember in > src_mail the previously copied mail, 4) use that for mailbox_copy()ing > to following recipients > - make sure this removes duplicate dbox mails when sieve saves mail to > multiple mailboxes > -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8 From tss at iki.fi Mon Nov 30 22:23:07 2015 From: tss at iki.fi (Timo Sirainen) Date: Tue, 1 Dec 2015 00:23:07 +0200 Subject: Questions about hardlinks, alternate storage and compression] In-Reply-To: <565C6FE2.3080105@assyoma.it> References: <20151126141529.GO18514@frodo.gerdesas.com> <20151126150115.2F0BA23488@talvi.dovecot.org> <565C071F.1000302@assyoma.it> <565C6FE2.3080105@assyoma.it> Message-ID: On 30 Nov 2015, at 17:48, Gionatan Danti wrote: > > Hi Timo, > glad to know it is in your TODO list ;) It's been for many years. > Any rough ETA on that? Right now it doesn't seem likely to be developed anytime soon. > Thanks. > > On 30/11/2015 14:23, Timo Sirainen wrote: >> On 30 Nov 2015, at 10:21, Gionatan Danti wrote: >>> >>> So, let me do a straigth question: is someone using dovecot/LMTP with hardlinking? To me, this seems a _very_ important feature, and I wonder if I am doing something wrong or if the feature (hardlink+sieve) simply does not exists. >> >> Hardlink+Sieve has never worked. The fix is a bit complicated. Here's my TODO entry about it: >> >> - remove mail_deliver_session after all, do all the stuff transparently >> by hooking into mailbox_copy(). >> - use this hook also to do the mail deduplication: 1) sort all destination >> users, 2) create mail_user only once for each user, 3) remember in >> src_mail the previously copied mail, 4) use that for mailbox_copy()ing >> to following recipients >> - make sure this removes duplicate dbox mails when sieve saves mail to >> multiple mailboxes >> > > -- > Danti Gionatan > Supporto Tecnico > Assyoma S.r.l. - www.assyoma.it > email: g.danti at assyoma.it - info at assyoma.it > GPG public key ID: FF5F32A8