From dovecot at randy.pensive.org Thu Jan 1 00:28:03 2015 From: dovecot at randy.pensive.org (Randall Gellens) Date: Wed, 31 Dec 2014 16:28:03 -0800 Subject: dovecot: imap Error: fts_solr: Query didn't return uid Message-ID: Anyone know what this means and what the implications are? dovecot: imap(user) Error: fts_solr: Query didn't return uid -- Randall Gellens Opinions are personal; facts are suspect; I speak for myself only -------------- Randomly selected tag: --------------- Men fear thought as they fear nothing else on earth, more than ruin, more even than death --Bertrand Russell From patrickdk at patrickdk.com Thu Jan 1 04:27:38 2015 From: patrickdk at patrickdk.com (Patrick Domack) Date: Wed, 31 Dec 2014 23:27:38 -0500 Subject: Postfix and Dovecot SASL - wiki update In-Reply-To: <6DE2147F-0A36-4E69-9D32-8E8D214C5767@langille.org> Message-ID: <20141231232738.Horde.6tSNqzAKyFSVLWG2AWvIZg3@mail.patrickdk.com> Disabling chroot doesn't fix the issue, it just worked around it. Aparently your chroot is not configured correctly, likely using debian based system, and your resolv.conf in the chroot is not updated correctly. Quoting Dan Langille : > I wanted to mention I updated the Postfix and Dovecot SASL wiki > entry yesterday. > > http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > > The entry: > > submission inet n - - - - smtpd > > was incorrect. It should be: > > submission inet n - n - - smtpd > > By specifying n for chroot, it ensures that DNS lookups can occur. > Otherwise, if you are using reject_unknown_recipient_domain > the following error will occur: Recipient address rejected: Domain not found > > Hope that helps. > ? > Dan Langille > http://langille.org/ From dan at langille.org Thu Jan 1 04:51:44 2015 From: dan at langille.org (Dan Langille) Date: Wed, 31 Dec 2014 23:51:44 -0500 Subject: Postfix and Dovecot SASL - wiki update In-Reply-To: <20141231232738.Horde.6tSNqzAKyFSVLWG2AWvIZg3@mail.patrickdk.com> References: <20141231232738.Horde.6tSNqzAKyFSVLWG2AWvIZg3@mail.patrickdk.com> Message-ID: On Dec 31, 2014, at 11:27 PM, Patrick Domack wrote: > > Disabling chroot doesn't fix the issue, it just worked around it. Oh?. > Aparently your chroot is not configured correctly, likely using debian based system, and your resolv.conf in the chroot is not updated correctly. Using FreeBSD here, not Debian. I have no chroot configured for Postfix at all. The wiki page does not clearly mention this requirement. > Quoting Dan Langille : > >> I wanted to mention I updated the Postfix and Dovecot SASL wiki entry yesterday. >> >> http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL >> >> The entry: >> >> submission inet n - - - - smtpd >> >> was incorrect. It should be: >> >> submission inet n - n - - smtpd >> >> By specifying n for chroot, it ensures that DNS lookups can occur. Otherwise, if you are using reject_unknown_recipient_domain >> the following error will occur: Recipient address rejected: Domain not found >> >> Hope that helps. >> ? >> Dan Langille >> http://langille.org/ ? Dan Langille http://langille.org/ From wojtek at puchar.net Thu Jan 1 12:29:28 2015 From: wojtek at puchar.net (Wojciech Puchar) Date: Thu, 1 Jan 2015 13:29:28 +0100 (CET) Subject: dovecot 2 low TCP speed (fetching big mail) Message-ID: i use dovecot 2 under FreeBSD (dovecot2-2.2.15 compiled from ports). tried both with kqueue enabled or not. everything works very fast, EXCEPT fetching big mail. tried multiple clients (thunderbird on windows, alpine on the same server, alpine on other unix server connected by 1Gb/s LAN) and fetching peaks at 1MB/s. NO SSL! could you please point me where to search for a problem. hints: writing over IMAP (eg. writing to Sent folder) works at full speed. Doesn't seem like a problem with unix server, every other service goes at speed 70-100MB/s both directions (like samba, ftp). tried to turn off "kqueue" compile time option, no difference. Please give me a hint at least what to check. Does dovecot use other way to sent data over TCP socket than other programs? thank you. From stephan at rename-it.nl Thu Jan 1 13:10:12 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 01 Jan 2015 14:10:12 +0100 Subject: Sieve permissions issue following update In-Reply-To: References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> Message-ID: <54A54734.4010805@rename-it.nl> On 12/31/2014 5:05 PM, Robert Blayzor wrote: > On Dec 10, 2014, at 1:52 AM, Steffen Kaiser wrote: > > I've been following this thread and have been seeing a similar problem. Dovecot 2.2.5 and pigeonhole-0.4.6 > > Yet, dovecot still tries to compile it under the user in that path. > > > Dec 31 15:55:11 dovecot: lda(fred): Error: sieve: binary save: failed to create temporary file: open(/etc/dovecot/sieve/default.svbin.localhost.87581.) failed: Permission denied (euid=1002(fred) egid=1002(fred) missing +w perm: /etc/dovecot/sieve, dir owned by 26:0 mode=0755) > Dec 31 15:55:11 dovecot: lda(fred): Error: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/etc/dovecot/sieve/default.sieve' need to be pre-compiled using the sievec tool > Dec 31 15:55:11 dovecot: lda(fred): sieve: msgid=<63706CEA-E77F-45BE-B848-1E664773EBDE at inoc.net>: stored mail into mailbox 'INBOX' Could you enable mail_debug? That should show why it is trying to recompile the Sieve script. Regards, Stephan. From rblayzor.bulk at inoc.net Thu Jan 1 13:36:40 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Thu, 1 Jan 2015 08:36:40 -0500 Subject: Sieve permissions issue following update In-Reply-To: <54A54734.4010805@rename-it.nl> References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> Message-ID: <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> On Jan 1, 2015, at 8:10 AM, Stephan Bosch wrote: > > Could you enable mail_debug? That should show why it is trying to > recompile the Sieve script. Well, that it does! And it's saying the script is "not up to date" and tries to recompile it. However, I'm not sure why it would say it's NOT up to date, it most certainly was manually compiled by me and not touched afterwards. Would commented likes, starting with "#" in the script have anything to do with it? Jan 01 13:32:30 lda(rt): Debug: sieve: file storage: Using script storage path: /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 lda(rt): Debug: sieve: file script: Opened script `default' from `/etc/dovecot/sieve/default.sieve' Jan 01 13:32:30 lda(rt): Debug: sieve: Using the following location for user's Sieve script: /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 lda(rt): Debug: sieve: Loading script /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 lda(rt): Debug: sieve: Script binary /etc/dovecot/sieve/default.svbin is not up-to-date Jan 01 13:32:30 lda(rt): Debug: sieve: Script `default' from /etc/dovecot/sieve/default.sieve successfully compiled Jan 01 13:32:30 lda(rt): Error: sieve: binary save: failed to create temporary file: open(/etc/dovecot/sieve/default.svbin.dogpile.devnull.us.679.) failed: Permission denied (euid=1002(rt) egid=1002(rt) missing +w perm: /etc/dovecot/sieve, dir owned by 26:0 mode=0755) From gheskett at wdtv.com Thu Jan 1 14:12:07 2015 From: gheskett at wdtv.com (Gene Heskett) Date: Thu, 1 Jan 2015 09:12:07 -0500 Subject: Sieve permissions issue following update In-Reply-To: <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> References: <548732CB.4070606@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> Message-ID: <201501010912.07573.gheskett@wdtv.com> On Thursday 01 January 2015 08:36:40 Robert Blayzor did opine And Gene did reply: > On Jan 1, 2015, at 8:10 AM, Stephan Bosch wrote: > > Could you enable mail_debug? That should show why it is trying to > > recompile the Sieve script. > > Well, that it does! And it's saying the script is "not up to date" and > tries to recompile it. However, I'm not sure why it would say it's > NOT up to date, it most certainly was manually compiled by me and not > touched afterwards. Would commented likes, starting with "#" in the > script have anything to do with it? > > > Jan 01 13:32:30 lda(rt): Debug: sieve: file storage: Using script > storage path: /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 > lda(rt): Debug: sieve: file script: Opened script `default' from > `/etc/dovecot/sieve/default.sieve' Jan 01 13:32:30 lda(rt): Debug: > sieve: Using the following location for user's Sieve script: > /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 lda(rt): Debug: > sieve: Loading script /etc/dovecot/sieve/default.sieve Jan 01 13:32:30 > lda(rt): Debug: sieve: Script binary /etc/dovecot/sieve/default.svbin > is not up-to-date Jan 01 13:32:30 lda(rt): Debug: sieve: Script > `default' from /etc/dovecot/sieve/default.sieve successfully compiled > Jan 01 13:32:30 lda(rt): Error: sieve: binary save: failed to create > temporary file: > open(/etc/dovecot/sieve/default.svbin.dogpile.devnull.us.679.) failed: > Permission denied (euid=1002(rt) egid=1002(rt) missing +w perm: > /etc/dovecot/sieve, dir owned by 26:0 mode=0755) Obviously, the last 3 lines are showing a perms problem. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS From rblayzor.bulk at inoc.net Thu Jan 1 14:31:56 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Thu, 1 Jan 2015 09:31:56 -0500 Subject: Sieve permissions issue following update In-Reply-To: <201501010912.07573.gheskett@wdtv.com> References: <548732CB.4070606@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> <201501010912.07573.gheskett@wdtv.com> Message-ID: <48EE92D8-42FA-47A4-B00A-4A4727338A55@inoc.net> On Jan 1, 2015, at 9:12 AM, Gene Heskett wrote: > > Obviously, the last 3 lines are showing a perms problem. Yes, I know it's a permissions problem. But there should be NO permissions problem as it should not be trying to recompile the script. The script was already pre-compiled and has not changed. (though it thinks it's "out of date" ?). The only "fix" would be to chmod 777 the directory where the default script is so that EVERYONE could compile it at the location. (even though it shouldn't need to be because it was already precompiled) But that would be rather silly now, wouldn't it? These are default sieve scripts that are not in the users homedir, so they have no permission to compile and write them in a directory they don't own. -Robert From stephan at rename-it.nl Thu Jan 1 14:50:07 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 01 Jan 2015 15:50:07 +0100 Subject: Sieve permissions issue following update In-Reply-To: <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> Message-ID: <54A55E9F.8030905@rename-it.nl> On 1/1/2015 2:36 PM, Robert Blayzor wrote: > On Jan 1, 2015, at 8:10 AM, Stephan Bosch wrote: >> Could you enable mail_debug? That should show why it is trying to >> recompile the Sieve script. > > Well, that it does! And it's saying the script is "not up to date" and tries to recompile it. However, I'm not sure why it would say it's NOT up to date, it most certainly was manually compiled by me and not touched afterwards. Would commented likes, starting with "#" in the script have anything to do with it? > > > Jan 01 13:32:30 lda(rt): Debug: sieve: file storage: Using script storage path: /etc/dovecot/sieve/default.sieve > Jan 01 13:32:30 lda(rt): Debug: sieve: file script: Opened script `default' from `/etc/dovecot/sieve/default.sieve' > Jan 01 13:32:30 lda(rt): Debug: sieve: Using the following location for user's Sieve script: /etc/dovecot/sieve/default.sieve > Jan 01 13:32:30 lda(rt): Debug: sieve: Loading script /etc/dovecot/sieve/default.sieve > Jan 01 13:32:30 lda(rt): Debug: sieve: Script binary /etc/dovecot/sieve/default.svbin is not up-to-date > Jan 01 13:32:30 lda(rt): Debug: sieve: Script `default' from /etc/dovecot/sieve/default.sieve successfully compiled > Jan 01 13:32:30 lda(rt): Error: sieve: binary save: failed to create temporary file: open(/etc/dovecot/sieve/default.svbin.dogpile.devnull.us.679.) failed: Permission denied (euid=1002(rt) egid=1002(rt) missing +w perm: /etc/dovecot/sieve, dir owned by 26:0 mode=0755) Hmm. This smells like a bug. I notice that your modification times of the .sieve and .svbin file are exactly the same (that is somewhat unusual). I'm looking at a potential bug that would explain your problem. To confirm, could you try running sievec again, so that the .svbin is actually newer than the .sieve? Regards, Stephan. From rblayzor.bulk at inoc.net Thu Jan 1 14:58:16 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Thu, 1 Jan 2015 09:58:16 -0500 Subject: Sieve permissions issue following update In-Reply-To: <54A55E9F.8030905@rename-it.nl> References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> <54A55E9F.8030905@rename-it.nl> Message-ID: On Jan 1, 2015, at 9:50 AM, Stephan Bosch wrote: > > Hmm. This smells like a bug. I notice that your modification times of > the .sieve and .svbin file are exactly the same (that is somewhat > unusual). I'm looking at a potential bug that would explain your problem. > > To confirm, could you try running sievec again, so that the .svbin is > actually newer than the .sieve? Sorry about that. ls -l was only showing minutes the actual file mtime *is* newer: ls -l -rw-r--r-- 1 root wheel 168 Jan 1 13:37 default.sieve -rw-r--r-- 1 root wheel 300 Jan 1 13:37 default.svbin stat -f %Sm default.sieve Jan 1 13:37:42 2015 stat -f %Sm default.svbin Jan 1 13:37:51 2015 I did just run it again... same problem: -rw-r--r-- 1 root wheel 168 Jan 1 13:37 default.sieve -rw-r--r-- 1 root wheel 300 Jan 1 14:55 default.svbin Jan 1 14:56:52 dovecot: lda(fred): Error: sieve: binary save: failed to create temporary file: open(/etc/dovecot/sieve/default.svbin.localhost.1435.) failed: Permission denied (euid=1002(fred) egid=1002(fred) missing +w perm: /etc/dovecot/sieve, dir owned by 26:0 mode=0755) Jan 1 14:56:52 dovecot: lda(fred): Error: sieve: The LDA Sieve plugin does not have permission to save global Sieve script binaries; global Sieve scripts like `/etc/dovecot/sieve/default.sieve' need to be pre-compiled using the sievec tool TIA From edgar at pettijohn-web.com Thu Jan 1 15:00:11 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Thu, 01 Jan 2015 09:00:11 -0600 Subject: dovecot 2 low TCP speed (fetching big mail) In-Reply-To: References: Message-ID: <54A560FB.9020506@pettijohn-web.com> I was just messing around with my pf.conf and then I read this, so just a thought do you have any kind of firewall that may be slowing the process?? On 01/01/15 06:29, Wojciech Puchar wrote: > i use dovecot 2 under FreeBSD (dovecot2-2.2.15 compiled from ports). > > tried both with kqueue enabled or not. > > everything works very fast, EXCEPT fetching big mail. > > tried multiple clients (thunderbird on windows, alpine on the same > server, alpine on other unix server connected by 1Gb/s LAN) and > fetching peaks at 1MB/s. NO SSL! > > could you please point me where to search for a problem. > > hints: > > writing over IMAP (eg. writing to Sent folder) works at full speed. > > Doesn't seem like a problem with unix server, every other service goes > at speed 70-100MB/s both directions (like samba, ftp). > > tried to turn off "kqueue" compile time option, no difference. > > Please give me a hint at least what to check. Does dovecot use other > way to sent data over TCP socket than other programs? > > thank you. From rblayzor.bulk at inoc.net Thu Jan 1 15:17:37 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Thu, 1 Jan 2015 10:17:37 -0500 Subject: Sieve permissions issue following update In-Reply-To: References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> <54A55E9F.8030905@rename-it.nl> Message-ID: On Jan 1, 2015, at 9:58 AM, Robert Blayzor wrote: > >> Hmm. This smells like a bug. I notice that your modification times of >> the .sieve and .svbin file are exactly the same (that is somewhat >> unusual). I'm looking at a potential bug that would explain your problem. >> >> To confirm, could you try running sievec again, so that the .svbin is >> actually newer than the .sieve? If it makes any difference at all... I only see this using "dovecot-lda". If I change my Exim transport to use Dovecot's LMTP, I do not see this problem. For the record also, the script DOES still execute (the compiled version that exists), even after the error... -- Robert From stephan at rename-it.nl Thu Jan 1 16:22:02 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 01 Jan 2015 17:22:02 +0100 Subject: Sieve permissions issue following update In-Reply-To: References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> <54A55E9F.8030905@rename-it.nl> Message-ID: <54A5742A.8040207@rename-it.nl> On 1/1/2015 4:17 PM, Robert Blayzor wrote: > On Jan 1, 2015, at 9:58 AM, Robert Blayzor wrote: >>> Hmm. This smells like a bug. I notice that your modification times of >>> the .sieve and .svbin file are exactly the same (that is somewhat >>> unusual). I'm looking at a potential bug that would explain your problem. >>> >>> To confirm, could you try running sievec again, so that the .svbin is >>> actually newer than the .sieve? > > If it makes any difference at all... I only see this using "dovecot-lda". If I change my Exim transport to use Dovecot's LMTP, I do not see this problem. That is odd. You can try the latest version. I've added some more debugging regarding the up-to-date check. > > For the record also, the script DOES still execute (the compiled version that exists), even after the error... It compiles, so it can be executed. It just cannot store the binary for future use. So, it will work as normal, but it is not efficient as it compiles the Sieve script for every incoming message. Regards, Stephan. From wojtek at puchar.net Thu Jan 1 22:36:21 2015 From: wojtek at puchar.net (Wojciech Puchar) Date: Thu, 1 Jan 2015 23:36:21 +0100 (CET) Subject: dovecot 2 low TCP speed (fetching big mail) In-Reply-To: <54A560FB.9020506@pettijohn-web.com> References: <54A560FB.9020506@pettijohn-web.com> Message-ID: tried with ipfw firewall turned off. no difference. anyway - thanks for advice. On Thu, 1 Jan 2015, Edgar Pettijohn wrote: > I was just messing around with my pf.conf and then I read this, so just a > thought do you have any kind of firewall that may be slowing the process?? > > On 01/01/15 06:29, Wojciech Puchar wrote: >> i use dovecot 2 under FreeBSD (dovecot2-2.2.15 compiled from ports). >> >> tried both with kqueue enabled or not. >> >> everything works very fast, EXCEPT fetching big mail. >> >> tried multiple clients (thunderbird on windows, alpine on the same server, >> alpine on other unix server connected by 1Gb/s LAN) and fetching peaks at >> 1MB/s. NO SSL! >> >> could you please point me where to search for a problem. >> >> hints: >> >> writing over IMAP (eg. writing to Sent folder) works at full speed. >> >> Doesn't seem like a problem with unix server, every other service goes at >> speed 70-100MB/s both directions (like samba, ftp). >> >> tried to turn off "kqueue" compile time option, no difference. >> >> Please give me a hint at least what to check. Does dovecot use other way to >> sent data over TCP socket than other programs? >> >> thank you. > > From mail at marc-stuermer.de Fri Jan 2 08:09:36 2015 From: mail at marc-stuermer.de (Marc =?utf-8?b?U3TDvHJtZXI=?=) Date: Fri, 02 Jan 2015 09:09:36 +0100 Subject: Disabling compressed storage for one public namespace - is it possible? Message-ID: <20150102090936.Horde.yjsqoSK6w3GssKzrdvPLsQ1@webmail.no-carrier.info> Greetings, I've got Dovecot 2.2.X running with gzip compressed storage enabled by default. My question though is: is it possible to disable this compressed storage for one specific, public namespace only? I haven't found quite much information about it on the wiki. Thanks in advance. From user+dovecot at localhost.localdomain.org Fri Jan 2 14:35:39 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Fri, 02 Jan 2015 14:35:39 +0000 Subject: Disabling compressed storage for one public namespace - is it possible? In-Reply-To: <20150102090936.Horde.yjsqoSK6w3GssKzrdvPLsQ1@webmail.no-carrier.info> References: <20150102090936.Horde.yjsqoSK6w3GssKzrdvPLsQ1@webmail.no-carrier.info> Message-ID: <54A6ACBB.3080203@localhost.localdomain.org> On 01/02/2015 08:09 AM, Marc St?rmer wrote: > Greetings, > > I've got Dovecot 2.2.X running with gzip compressed storage enabled by > default. > > My question though is: is it possible to disable this compressed > storage for one specific, public namespace only? I haven't found quite > much information about it on the wiki. Hi Marc, in one setup we use 'plugin {zlib_save = bz2}' for the mdbox storage. The public namespace uses the Maildir++ format. All messages in the public namespace are stored w/o compression (no zlib related configuration). Regards, Pascal -- The trapper recommends today: deadbeef.1500204 at localdomain.org From listas at adminlinux.com.br Fri Jan 2 17:14:47 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Fri, 02 Jan 2015 15:14:47 -0200 Subject: Designing an Object Storage Plugin for a specific platform In-Reply-To: <54A16E8E.6060207@adminlinux.com.br> References: <54A16E8E.6060207@adminlinux.com.br> Message-ID: <54A6D207.1040202@adminlinux.com.br> Did someone has already developed something similar and have any tips for me? Thanks! -- Thiago Henrique Em 29-12-2014 13:09, Listas at Adminlinux escreveu: > Hi! > > I'm planning to develop a OSP (Object Storage Plugin) to support a > specific platform of my company. > > Is there any API documentation or sample code of a simple plugin that > can be used as a basis for an implementation? > > Thanks! > -- > Thiago Henrique From superinterstellar at gmail.com Sat Jan 3 05:25:57 2015 From: superinterstellar at gmail.com (Kev Lau) Date: Sat, 3 Jan 2015 12:25:57 +0700 Subject: Help with installing Apache Solr with Dovecot 2.2 Message-ID: Hi, I am relatively new to Apache Solr and I am trying to get it to integrate with Dovecot 2.2 running on Ubuntu 14.04 and needed a little help with configuration. The following steps I have already configured. Appreciate if someone could help, so please help me! 1. Apache Solr installed, up and running 6406 [main] INFO org.eclipse.jetty.server.AbstractConnector ? Started SocketConnector at 0.0.0.0:8983 I can then ssh into it and see the running solr admin. ssh -t -L 8983:localhost:8983 user at mydomain.com 2. Schema file added startTime: 3 minutes ago instanceDir: /opt/solr/solr/collection1/ dataDir: /opt/solr/solr/collection1/data/ In my collection1 core selector I have the following files:- 1.solrconfig.xml 2.schema.xml The output of schema.xml is shown below:- id body 3. Dovecot package dovecot-solr installed 4. Dovecot configured with following files to run on solr-fts. In 10-mail.conf:- # Space separated list of plugins to load for all services. Plugins specific to # IMAP, LDA, etc. are added to this list in their own .conf files. #mail_plugins = mail_plugins = fts fts_solr In 90-plugin.conf:- plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } I would really appreciate if someone could please help me here. Request for someone to guide me here. Thanks a lot Kevin From mail at marc-stuermer.de Sat Jan 3 09:45:20 2015 From: mail at marc-stuermer.de (Marc =?utf-8?b?U3TDvHJtZXI=?=) Date: Sat, 03 Jan 2015 10:45:20 +0100 Subject: Help with installing Apache Solr with Dovecot 2.2 In-Reply-To: Message-ID: <20150103104520.Horde.45hja5f3mCAoViq4mJnxmQ2@webmail.no-carrier.info> Zitat von Kev Lau : > Hi, > I am relatively new to Apache Solr and I am trying to get it to integrate > with Dovecot 2.2 running on Ubuntu 14.04 and needed a little help with > configuration. How do you store your user data? Aside that, you should later setup the necessary cronjobs for Solr to run smoothly. If unsure what's the problem turn on mail_debug=yes in your dovecot.conf . From superinterstellar at gmail.com Sat Jan 3 10:27:22 2015 From: superinterstellar at gmail.com (Kev Lau) Date: Sat, 3 Jan 2015 17:27:22 +0700 Subject: Help with installing Apache Solr with Dovecot 2.2 In-Reply-To: <20150103104520.Horde.45hja5f3mCAoViq4mJnxmQ2@webmail.no-carrier.info> References: <20150103104520.Horde.45hja5f3mCAoViq4mJnxmQ2@webmail.no-carrier.info> Message-ID: Dear Marc, Thanks for your feedback! 1. I store it in MySQL. 2. I will add a cronjob to run Solr if I can figure out why its not working on test runs. Maybe I might try to setup a cronjob for solr to avoid having to manually start it everytime. 3. Will turn on the mail_debug in dovecot.conf . Where will it log the problems? Thanks Kevin On Sat, Jan 3, 2015 at 4:45 PM, Marc St?rmer wrote: > > Zitat von Kev Lau : > > Hi, >> I am relatively new to Apache Solr and I am trying to get it to integrate >> with Dovecot 2.2 running on Ubuntu 14.04 and needed a little help with >> configuration. >> > > How do you store your user data? > > Aside that, you should later setup the necessary cronjobs for Solr to run > smoothly. > > If unsure what's the problem turn on mail_debug=yes in your dovecot.conf . > From superinterstellar at gmail.com Sat Jan 3 12:19:12 2015 From: superinterstellar at gmail.com (Kev Lau) Date: Sat, 3 Jan 2015 19:19:12 +0700 Subject: Help with installing Apache Solr with Dovecot 2.2 In-Reply-To: References: <20150103104520.Horde.45hja5f3mCAoViq4mJnxmQ2@webmail.no-carrier.info> Message-ID: Dear Marc, Found the log for mail-debug. Apologies for the bad question. Thanks Kevin On Sat, Jan 3, 2015 at 5:27 PM, Kev Lau wrote: > Dear Marc, > Thanks for your feedback! > > 1. I store it in MySQL. > > 2. I will add a cronjob to run Solr if I can figure out why its not > working on test runs. Maybe I might try to setup a cronjob for solr to > avoid having to manually start it everytime. > > 3. Will turn on the mail_debug in dovecot.conf . Where will it log the > problems? > > Thanks > Kevin > > On Sat, Jan 3, 2015 at 4:45 PM, Marc St?rmer > wrote: > >> >> Zitat von Kev Lau : >> >> Hi, >>> I am relatively new to Apache Solr and I am trying to get it to integrate >>> with Dovecot 2.2 running on Ubuntu 14.04 and needed a little help with >>> configuration. >>> >> >> How do you store your user data? >> >> Aside that, you should later setup the necessary cronjobs for Solr to run >> smoothly. >> >> If unsure what's the problem turn on mail_debug=yes in your dovecot.conf . >> > > From superinterstellar at gmail.com Sat Jan 3 16:52:29 2015 From: superinterstellar at gmail.com (Kev Lau) Date: Sat, 3 Jan 2015 23:52:29 +0700 Subject: Help with installing Apache Solr with Dovecot 2.2 In-Reply-To: References: <20150103104520.Horde.45hja5f3mCAoViq4mJnxmQ2@webmail.no-carrier.info> Message-ID: Hi Marc, I have debug turned on but i keep getting this in my logs:- Jan 03 17:36:41 imap(user at mydomain.com): Info: Disconnected: Logged out in=478 out=4938 Jan 03 17:37:41 imap-login: Info: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=20247, secured, session=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=20255, secured, session=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=20259, secured, session= wrote: > Dear Marc, > Found the log for mail-debug. Apologies for the bad question. > Thanks > Kevin > > On Sat, Jan 3, 2015 at 5:27 PM, Kev Lau > wrote: > >> Dear Marc, >> Thanks for your feedback! >> >> 1. I store it in MySQL. >> >> 2. I will add a cronjob to run Solr if I can figure out why its not >> working on test runs. Maybe I might try to setup a cronjob for solr to >> avoid having to manually start it everytime. >> >> 3. Will turn on the mail_debug in dovecot.conf . Where will it log the >> problems? >> >> Thanks >> Kevin >> >> On Sat, Jan 3, 2015 at 4:45 PM, Marc St?rmer >> wrote: >> >>> >>> Zitat von Kev Lau : >>> >>> Hi, >>>> I am relatively new to Apache Solr and I am trying to get it to >>>> integrate >>>> with Dovecot 2.2 running on Ubuntu 14.04 and needed a little help with >>>> configuration. >>>> >>> >>> How do you store your user data? >>> >>> Aside that, you should later setup the necessary cronjobs for Solr to >>> run smoothly. >>> >>> If unsure what's the problem turn on mail_debug=yes in your dovecot.conf >>> . >>> >> >> > From dovecot.org at stelb.de Sat Jan 3 22:09:02 2015 From: dovecot.org at stelb.de (Stefan Le Breton) Date: Sat, 03 Jan 2015 23:09:02 +0100 Subject: dovecot-lda segfaults with replication Message-ID: <19fa77aec98de4eb13757c66655be25c@stelb.de> Hi, I have set up replication, but dovecot-lda now fails with a segmentation fault. Problem seems to be in replication/replication-plugin.c line 253: struct replication_user *ruser = REPLICATION_USER_CONTEXT(ctx->ns->user); resulting in ruser = 0x0 # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.0-RELEASE amd64 gdb Backtrace + doveconf -n below. Regards Stefan gdb backtrace: #0 0x0000000802001bc4 in replication_mail_transaction_commit (txn=0x801c811b0, changes=0x7fffffffd3d0) at replication-plugin.c:258 ctx = (struct replication_mail_txn_context *) 0x801c811b0 ruser = (struct replication_user *) 0x0 priority = REPLICATION_PRIORITY_NONE #1 0x0000000801490749 in notify_contexts_mail_transaction_commit (t=0x801c24600, changes=0x7fffffffd3d0) at notify-plugin.c:132 ctx = (struct notify_context *) 0x801c8d020 mail_txn = (struct notify_mail_txn *) 0x801c282b0 #2 0x0000000801491a53 in notify_transaction_commit (t=0x801c24600, changes_r=0x7fffffffd3d0) at notify-storage.c:182 lt = (struct notify_transaction_context *) 0x0 lbox = (union mailbox_module_context *) 0x801cade50 #3 0x0000000800ae1ff9 in mailbox_transaction_commit_get_changes (_t=0x801ce01b0, changes_r=0x7fffffffd3d0) at mail-storage.c:1916 _data_stack_cur_id = 3 t = (struct mailbox_transaction_context *) 0x801c24600 box = (struct mailbox *) 0x801cad840 save_count = 1 ret = -128 #4 0x0000000800ae1f49 in mailbox_transaction_commit (t=0x801ce01b0) at mail-storage.c:1897 changes = {pool = 0x801cd5420, uid_validity = 1234722882, saved_uids = { arr = {buffer = 0x801cd5440, element_size = 8}, v = 0x801cd5440, v_modifiable = 0x801cd5440}, ignored_modseq_changes = 0, changed = true, no_read_perm = false} ret = 8 #5 0x000000080244bfb2 in sieve_action_opr_optional_read () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 No symbol table info available. #6 0x0000000802443924 in sieve_result_implicit_keep () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 No symbol table info available. #7 0x0000000802443dcd in sieve_result_execute () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 No symbol table info available. #8 0x0000000802452f1c in sieve_execute () from /usr/local/lib/dovecot-2.2-pigeonhole/libdovecot-sieve.so.0 No symbol table info available. #9 0x0000000802205912 in sieve_plugin_init () from /usr/local/lib/dovecot/lib90_sieve_plugin.so No symbol table info available. #10 0x0000000800829104 in mail_deliver (ctx=0x7fffffffda08, storage_r=0x7fffffffd960) at mail-deliver.c:400 ret = 8 #11 0x0000000000402f2e in main (argc=7, argv=0x7fffffffdb08) at main.c:437 set_roots = {0x604970, 0x0} ctx = {pool = 0x801c29420, set = 0x801c67988, session = 0x801c29440, dup_ctx = 0x801c237c0, session_id = 0x0, src_mail = 0x801c48840, src_envelope_sender = 0x0, dest_user = 0x801c87040, dest_addr = 0x7fffffffddaf "mail at maserver.de", final_dest_addr = 0x7fffffffddaf "mail at maserver.de", dest_mailbox_name = 0x403d68 "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tempfail_error = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} service_flags = 1027 user = 0x7fffffffdd9e "mail at maserver.de" errstr = 0x37f path = 0x7fffffffddc0 "/tmp/mail" lda_set = (struct lda_settings *) 0x801c67988 storage_service = (struct mail_storage_service_ctx *) 0x801c46840 service_user = (struct mail_storage_service_user *) 0x801c67190 service_input = {module = 0x403d55 "lda", service = 0x403d55 "lda", username = 0x7fffffffdd9e "mail at maserver.de", session_id = 0x0, local_ip = { family = 0, u = {ip6 = {__u6_addr = {__u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__u6_addr = { __u6_addr8 = '\0' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = {s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = 0, flags_override_remove = 0, no_userdb_lookup = 0} storage = (struct mail_storage *) 0x0 user_source = 0x404090 "" destaddr_source = 0x403d73 "-a parameter" process_euid = 0 stderr_rejection = false ret = 1 c = -1 error = MAIL_ERROR_NONE # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.0-RELEASE amd64 auth_mechanisms = plain login digest-md5 doveadm_password = welcomemyguest first_valid_gid = 0 first_valid_uid = 125 listen = * login_greeting = McImap mail_location = maildir:~/.maildir mail_log_prefix = "%Us(%u): " mail_plugins = " notify replication" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate passdb { args = dovecot driver = pam } passdb { args = /usr/local/etc/dovecot/dovecot-mysql.conf driver = sql } plugin { antispam_backend = mailtrain antispam_mail_notspam = some at where antispam_mail_sendmail = /usr/sbin/sendmail antispam_mail_spam = some at elsewhere antispam_spam_pattern = INBOX.spam* plugin_debug = yes } protocols = imap sieve service aggregator { fifo_listener replication-notify-fifo { user = vmail } unix_listener replication-notify { user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = root } service doveadm { inet_listener { port = 31337 ssl = yes } } service managesieve-login { inet_listener sieve { port = 4190 } } service replicator { unix_listener replicator-doveadm { group = vmail mode = 0600 user = vmail } } ssl_cert = Hi all, I am upgrading a dovecot 1.2 installation to a 2.2 installation and have found and fixed a number of problems.. I've seen several postings in the archive about ntlm_auth not working, and it is true, there are several regressions in dovecot here. The first and simplest is that the enablement instructions in the wikki are wrong. ntlm_auth must be called as root, which is what 1.2 did. 2.2 defaults to calling it as the auth user which subtly doesn't work: Dec 30 20:30:21 quartz dovecot[8439]: auth: Error: Login for user []\[jgg]@[wakko] failed due to [Reading winbind reply failed!] Dec 30 20:30:21 quartz dovecot[8439]: auth: Error: ../auth/ntlmssp/ntlmssp_server.c:454: Checking NTLMSSP password for \jgg failed: NT_STATUS_UNSUCCESSFUL Dec 30 20:30:21 quartz dovecot[8439]: auth: Error: GENSEC login failed: NT_STATUS_UNSUCCESSFUL Dec 30 20:30:21 quartz dovecot[8439]: auth: Error: winbind: ntlm_auth exited with exit code 0 The fix is simple, run auth as root: service auth { user = root } This ended up as debian bug #774263 which has a few more details. Regards, Jason From jgunthorpe at obsidianresearch.com Sun Jan 4 04:43:57 2015 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Sat, 3 Jan 2015 21:43:57 -0700 Subject: [PATCH] LAYOUT=imapdir is broken in v2.2 Message-ID: <20150104044357.GB20575@obsidianresearch.com> The next thing I noticed in my v1.2 -> 2.2 upgrade is that mail_location = maildir:[..]:LAYOUT=imapdir is broken, the symptom is dovecot returning this to the client when requesting any mailbox beyond INBOX: Character not allowed in mailbox name: ' Which is actually trying to say "Character not allowed in mailbox name: '\0'", but since the %c is not escaped it ends up with the truncated string. This patch fixes it: diff --git a/src/lib-storage/list/mailbox-list-maildir.c b/src/lib-storage/list/mailbox-list-maildir.c index c99a2900a6d6..ae5f35d955ac 100644 --- a/src/lib-storage/list/mailbox-list-maildir.c +++ b/src/lib-storage/list/mailbox-list-maildir.c @@ -46,6 +46,7 @@ static struct mailbox_list *imapdir_list_alloc(void) list = p_new(pool, struct maildir_mailbox_list, 1); list->list = imapdir_mailbox_list; list->list.pool = pool; + list->sep = '.'; list->global_temp_prefix = IMAPDIR_GLOBAL_TEMP_PREFIX; list->temp_prefix = p_strconcat(pool, list->global_temp_prefix, Analysis: I noticed this while upgrading a dovecot install from 1.2.15 (squeeze) to 2.2.13 (jessie). This upstream commit author Timo Sirainen Thu Jan 20 20:59:07 2011 +0200 (2011-01-20) changeset 12586 a2780b694b2d parent 12585 b748c622e896 child 12587 c3a258ee96c4 lib-storage: mailbox_alloc() now takes a virtual mailbox name and other related API changes. All storage_name <-> vname conversions now go through the same two mailbox_list methods. This has many benefits, such as: * listescape plugin is now much simpler and bugfree * allows changing lib-storage API to use UTF-8 mailbox names in future * allows creation of "mailbox aliases" plugin Restructed the _alloc functions to move the hierarchy_sep from the initializer into the _alloc call itself: @@ -29,6 +30,7 @@ static struct mailbox_list *maildir_list_alloc(void) list = p_new(pool, struct maildir_mailbox_list, 1); list->list = maildir_mailbox_list; list->list.pool = pool; + list->sep = '.'; list->global_temp_prefix = MAILDIR_GLOBAL_TEMP_PREFIX; list->temp_prefix = p_strconcat(pool, list->global_temp_prefix, [..] struct mailbox_list maildir_mailbox_list = { .name = MAILBOX_LIST_NAME_MAILDIRPLUSPLUS, - .hierarchy_sep = '.', .props = MAILBOX_LIST_PROP_NO_MAILDIR_NAME | MAILBOX_LIST_PROP_NO_ALT_DIR | MAILBOX_LIST_PROP_NO_NOSELECT, [..] struct mailbox_list imapdir_mailbox_list = { .name = MAILBOX_LIST_NAME_IMAPDIR, - .hierarchy_sep = '.', .props = MAILBOX_LIST_PROP_NO_MAILDIR_NAME | MAILBOX_LIST_PROP_NO_ALT_DIR | MAILBOX_LIST_PROP_NO_NOSELECT, Noting that heierarchy_sep was removed from maildir_mailbox_list and imapdir_mailbox_list but only added to maildir_list_alloc(), and not imapdir_list_alloc(). This ultimately results in mailbox_list_get_hierarchy_sep() returning '\0' and mailbox_verify_name() failing everything (all strings contain '\0' according to strchr). This ended up as debian bug #774533 Regards, Jason From jgunthorpe at obsidianresearch.com Sun Jan 4 05:49:54 2015 From: jgunthorpe at obsidianresearch.com (Jason Gunthorpe) Date: Sat, 3 Jan 2015 22:49:54 -0700 Subject: ntlm_auth random failures with dovecot Message-ID: <20150104054954.GC20575@obsidianresearch.com> I'm still a bit fuzzy on exactly what has blown up here since my 1.2 install (or maybe it was broken then and I never noticed), but it looks like the way dovecot is calling out to ntlm_auth is violating the --helper-protocol=squid-2.5-ntlmssp scheme. The issue is how it handles simultaneous clients connecting - for instance launching thunderbird with NTLM auth creates multiple imapds that all have to be auth'd. Since dovecot doesn't (and apparrently didn't in 1.2?) serialize this it ends up sending a jumble to ntlm_auth. Strace sayth, as example: read(0, "YR xxxxxxx=\n", 4096) = 48 read(0, "YR xxxxxxx=\n", 4096) = 48 read(0, "KK xxxxxxx=\n",4096) = 176 read(0, "KK xxxxxxx=\n",4096) = 176 That is two clients connecting at once, and the sequence has become jumbled. Fiddling around with ntlm_auth manually I can get it to give me this: YR xxx # 1 TT xxx # 1 YR xxx # 2 TT xxx # 2 KK xxx # 2 AF jgg # 2 KK xxx # 1 Called NTLMSSP after state machine was 'done' GENSEC login failed: NT_STATUS_INVALID_PARAMETER NA NT_STATUS_INVALID_PARAMETER Ie, reordering the sequence (# 1 and # 2) causes it to tell you that, no, the sequence cannot be reordered. To me this says the samba folks expect that the YY/TT/KK/AF sequence is *NOT* reordered. The implication is that the mech-winbind in dovecot must seralize everything, and it doesn't! So, this is fairly broken, I can hit these failure causes with a high probability when using thunderbird. Any thoughts on how to repair this? The simplest answer would be to pool and assign a ntlm_auth process to each incoming auth context, or to actually serialize auth. But it can't treat ntlm_auth as a stateless helper. Jason From bra at fsn.hu Sun Jan 4 16:08:21 2015 From: bra at fsn.hu (Nagy, Attila) Date: Sun, 04 Jan 2015 17:08:21 +0100 Subject: mdbox GUID Message-ID: <54A96575.2080703@fsn.hu> Hi, The dbox page says: "Each message has a 128 bit globally unique identifier (GUID)" I guess this is what guid_128_generate() generates. Is this believed to be really globally unique? As far as I can tell, in the current architecture they are -given there are enough time precision-, because all dovecot daemons are single threaded, so they have concurrency but doesn't have parallelism (they are not multi threaded). I would like to list all of the messages, with this GUID as the unique identifier, so a collision would be fatal here... From teemu.huovila at dovecot.fi Mon Jan 5 10:49:33 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 05 Jan 2015 12:49:33 +0200 Subject: Crashes with tracebacks In-Reply-To: <5492C743.6040406@acm.org> References: <5492C743.6040406@acm.org> Message-ID: <54AA6C3D.5060805@dovecot.fi> On 12/18/2014 02:23 PM, Timothe Litt wrote: > Crashes, redux. I hope I have provided all the information required for > a solution. Many thanks in advance for having a look. > > I have 71 core files for a user, that all happened in the space of about > 6 hours. It appears that mail delivered to 'Junk E-mail' is being > accessed. I suspect they're all the same issue. I saw the same syslog > entry a while back; did a resync & enabled process dumps. Naturally, it > went away -- until this cluster of crashes. > > File system is ext3. It is NFS mounted by other machines, but only the > local machine should be touching the mail directories. The user does > not have an interactive login - it's an e-mail only account. > > This user's IMAP client is AppleMail. The delivery agent is procmail; > Junk is detected by spamassassin; clamav is also present. The patches mentioned in http://markmail.org/message/xqu3yr52c6hjxqk2 might fix your issue. You could also consider switching over to LMTP or dovecot-lda as the mail delivery method. br, Teemu Huovila From kinchan at kinchan.com Mon Jan 5 20:53:27 2015 From: kinchan at kinchan.com (Yoshito Takeuchi) Date: Tue, 6 Jan 2015 05:53:27 +0900 Subject: 'ssl_cipher_list' setting Message-ID: Hi, I am Yoshi, Japanese. I used FreeBSD 10.1 Dovecot 2.2.15 I want pop3s, so I made /usr/local/etc/dovecot/local.conf ssl = yes ssl_cert = SSLv3 ) I did trouble. /var/log/maillog Jan 6 05:41:53 example dovecot: pop3-login: Disconnected (no auth attempts in 0 secs): user=<>, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, TLS handshaking, session=<5e9 zuO0LVwB+PO8D> Is this bug ? or I did miss setting ? Thank you for reading. Yoshi From h.reindl at thelounge.net Mon Jan 5 20:59:14 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 05 Jan 2015 21:59:14 +0100 Subject: 'ssl_cipher_list' setting In-Reply-To: References: Message-ID: <54AAFB22.2080206@thelounge.net> Am 05.01.2015 um 21:53 schrieb Yoshito Takeuchi: > I used > > FreeBSD 10.1 > Dovecot 2.2.15 > > I want pop3s, so I made > > /usr/local/etc/dovecot/local.conf > > ssl = yes > ssl_cert = ssl_key = ssl_ca = ssl_protocols = !SSLv2 !SSLv3 > ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLV3:!EXP:!aNULL:!RC4 > > It's work fine. > But, change > > ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLv3:!EXP:!aNULL:!RC4 > > ( SSLV3 -> SSLv3 ) > > I did trouble > > /var/log/maillog > > Jan 6 05:41:53 example dovecot: pop3-login: Disconnected (no auth attempts in 0 > secs): user=<>, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, TLS > handshaking, session=<5e9 > zuO0LVwB+PO8D> > > Is this bug ? or I did miss setting? !SSLV3 was wrong and not recognized !SSLv3 is recognized but bullshit since you want to disable SSLv3 but not all ciphers which are still valid for newer TLS versions you do that already correctly with "ssl_protocols" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From kinchan at kinchan.com Tue Jan 6 19:10:13 2015 From: kinchan at kinchan.com (Yoshito Takeuchi) Date: Wed, 7 Jan 2015 04:10:13 +0900 Subject: 'ssl_cipher_list' setting In-Reply-To: <54AAFB22.2080206@thelounge.net> References: <54AAFB22.2080206@thelounge.net> Message-ID: Hi Reindl Harald Thank you for your reply. I got it. I changed my /usr/local/etc/dovecot/local.conf below. ssl = yes ssl_cert = : > > Am 05.01.2015 um 21:53 schrieb Yoshito Takeuchi: >> >> I used >> >> FreeBSD 10.1 >> Dovecot 2.2.15 >> >> I want pop3s, so I made >> >> /usr/local/etc/dovecot/local.conf >> >> ssl = yes >> ssl_cert = > ssl_key = > ssl_ca = > ssl_protocols = !SSLv2 !SSLv3 >> ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLV3:!EXP:!aNULL:!RC4 >> >> It's work fine. >> But, change >> >> ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLv3:!EXP:!aNULL:!RC4 >> >> ( SSLV3 -> SSLv3 ) >> >> I did trouble >> >> /var/log/maillog >> >> Jan 6 05:41:53 example dovecot: pop3-login: Disconnected (no auth >> attempts in 0 >> secs): user=<>, rip=xxx.xxx.xxx.xxx, lip=xxx.xxx.xxx.xxx, TLS >> handshaking, session=<5e9 >> zuO0LVwB+PO8D> >> >> Is this bug ? or I did miss setting? > > > !SSLV3 was wrong and not recognized > !SSLv3 is recognized but bullshit since you want to disable SSLv3 but not > all ciphers which are still valid for newer TLS versions > > you do that already correctly with "ssl_protocols" > From pug at felsing.net Wed Jan 7 06:44:39 2015 From: pug at felsing.net (Christian Felsing) Date: Wed, 07 Jan 2015 07:44:39 +0100 Subject: Schema Patch to get Dovecot running with Solr 4.10.3 Message-ID: <54ACD5D7.5000101@felsing.net> Hello, AFAIK a known problem with newer Apache Solr versions. If you patch Dovecots solr-schema.xml Dovecot will work with Solr 4.10.3: --- solr-schema.xml 2015-01-05 09:12:51.080196122 +0100 +++ /home/solr/solr/example/solr/collection1/conf/schema.xml 2015-01-05 09:31:48.320206660 +0100 @@ -51,6 +51,7 @@ + This patch was created regarding Dovecot 2.2.15, for Dovecot configuration please follow instructions at http://wiki2.dovecot.org/Plugins/FTS/Solr Christian From alessio at skye.it Wed Jan 7 09:49:44 2015 From: alessio at skye.it (Alessio Cecchi) Date: Wed, 07 Jan 2015 10:49:44 +0100 Subject: Schema Patch to get Dovecot running with Solr 4.10.3 In-Reply-To: <54ACD5D7.5000101@felsing.net> References: <54ACD5D7.5000101@felsing.net> Message-ID: <54AD0138.4070003@skye.it> Il 07/01/2015 07:44, Christian Felsing ha scritto: > Hello, > > AFAIK a known problem with newer Apache Solr versions. If you patch Dovecots > solr-schema.xml Dovecot will work with Solr 4.10.3: > > > --- solr-schema.xml 2015-01-05 09:12:51.080196122 +0100 > +++ /home/solr/solr/example/solr/collection1/conf/schema.xml 2015-01-05 > 09:31:48.320206660 +0100 > @@ -51,6 +51,7 @@ > > > > + > > > > > > > This patch was created regarding Dovecot 2.2.15, for Dovecot configuration > please follow instructions at http://wiki2.dovecot.org/Plugins/FTS/Solr > > Christian Hi, for Dovecot 2.2.15 and Solr 4.10 I'm using this schema: https://raw.githubusercontent.com/extremeshok/solr-dovecot2/master/schema.xml From eslist at ols.es Wed Jan 7 19:03:11 2015 From: eslist at ols.es (David Saez) Date: Wed, 07 Jan 2015 20:03:11 +0100 Subject: Cached message size smaller than expected Message-ID: <54AD82EF.9040307@ols.es> Hi Since i upgraded to Dovecot 2.2.15 i start experiencing "Cached message size smaller than expected" errors that render a whole imap folder unusable as dovecot panics. I used Timo Sirainen maildir-size-fix.pl v1.1 script to try to fix the sizes but it does not seem to work trying to access the maildir produces this log in dovecot: Jan 07 19:34:13 imap(forumed at forumed.biz): Error: read(zlib(/var/spool/exim/mail/forumed.biz/forumed/.Sent/cur/1386527120.M104960P2012.mail,S=7725131:2,S)) failed: Cached message size smaller than expected (7725131 < 7729083, box=Sent, UID=2734) Jan 07 19:34:13 imap(forumed at forumed.biz): Error: Maildir filename has wrong S value, renamed the file from /var/spool/exim/mail/forumed.biz/forumed/.Sent/cur/1386527120.M104960P2012.mail,S=7725131:2,S to /var/spool/exim/mail/forumed.biz/forumed/.Sent/cur/1386527120.M104960P2012.mail,S=7729084:2,S Jan 07 19:34:13 imap(forumed at forumed.biz): Error: Corrupted index cache file /var/spool/exim/mail/forumed.biz/forumed/.Sent/dovecot.index.cache: Broken physical size for mail UID 2734 Jan 07 19:34:13 imap(forumed at forumed.biz): Panic: file istream.c: line 167 (i_stream_read): assertion failed: (old_size <= _stream->pos - _stream->skip) Jan 07 19:34:13 imap(forumed at forumed.biz): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x75e1a) [0x7feb14469e1a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x75e96) [0x7feb14469e96] -> /usr/local/lib/dovecot/libdovecot.so.0(i_error+0) [0x7feb1441b2bf] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read+0x214) [0x7feb144731f4] -> /usr/local/lib/dovecot/libdovecot.so.0(i_stream_read_data+0x52) [0x7feb144733b2] -> /usr/local/lib/dovecot/libdovecot.so.0(message_get_body_size+0xa9) [0x7feb1445da89] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mail_init_stream+0x16f) [0x7feb1473795f] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x2dacc) [0x7feb14701acc] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_get_stream+0x4d) [0x7feb1470f3ed] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x2e3e8) [0x7feb147023e8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_get_virtual_size+0x2b) [0x7feb1470f07b] -> dovecot/imap() [0x41b2f5] -> dovecot/imap() [0x419d6c] -> dovecot/imap(imap_fetch_more+0x31) [0x419fa1] -> dovecot/imap() [0x40ebb0] -> dovecot/imap(command_exec+0x3d) [0x4183ad] -> dovecot/imap() [0x416a7e] -> dovecot/imap(client_output+0x10b) [0x417adb] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x931d9) [0x7feb144871d9] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x5b) [0x7feb1447b0db] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0xaf) [0x7feb1447c27f] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7feb1447b169] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7feb1447b1f8] -> /usr/local/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7feb14420193] -> dovecot/imap(main+0x2b0) [0x4210d0] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7feb140b1c4d] -> dovecot/imap() [0x40bea9] Jan 07 19:34:13 imap(forumed at forumed.biz): Fatal: master: service(imap): child 9337 killed with signal 6 (core not dumped) 1386527120.M104960P2012.mail,S=7725131:2,S is actually a gzip compressed mail, it's compressed size (file system size) is 7725131 bytes after dovecot panic it's automatically renamed to 1386527120.M104960P2012.mail,S=7739837:2,S by dovecot dovecot -n # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 auth_cache_size = 10 M auth_cache_ttl = 1 days auth_mechanisms = plain login auth_username_chars = abcdefghijklmnopqrstuvwxyz01234567890.-_@ disable_plaintext_auth = no hostname = pop.olsns.net log_path = /usr/local/log/dovecot.log mail_fsync = never mail_gid = mail mail_plugins = " zlib" mail_uid = exim mailbox_list_index = yes maildir_very_dirty_syncs = yes namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } plugin { zlib_save = gz zlib_save_level = 6 } pop3_fast_size_lookups = yes protocols = imap pop3 service auth { user = root } service imap-login { executable = /usr/local/libexec/dovecot/imap-login inet_listener imap { address = 82.98.162.142 127.0.0.1 } service_count = 100 user = dovecot } service imap { service_count = 100 } service pop3-login { executable = /usr/local/libexec/dovecot/pop3-login inet_listener pop3 { address = 82.98.162.142 } service_count = 100 user = dovecot } service pop3 { executable = /usr/local/libexec/dovecot/pop3 service_count = 100 } ssl_cert = I am attempting to pull email from gmail IMAP to my local machine and with the configuration I have I only seem to get messages from the INBOX folder. Hoping I could get some assistance getting all the gmail folders to download. Here is the imapc config: imapc_host = 64.233.171.108 imapc_user = %u imapc_master_user = master imapc_password = somepass imapc_features = rfc822.size # If you have Dovecot v2.2.8+ you may get a significant performance improvement with fetch-headers: imapc_features = $imapc_features fetch-headers # Read multiple mails in parallel, improves performance mail_prefetch_count = 20 # If the old IMAP server uses INBOX. namespace prefix, set: imapc_list_prefix = Gmail # for SSL: imapc_port = 993 imapc_ssl = imaps #imapc_ssl_ca_dir = /etc/ssl imapc_ssl_verify = no And the doveadm command I am running: doveadm -D -o imapc_user=$username -o imapc_password=$escaped_password backup -R -x '\All' -x '\Flagged' -x '\Important' -u $username imapc: From hummel at pasteur.fr Thu Jan 8 13:48:47 2015 From: hummel at pasteur.fr (hummel at pasteur.fr) Date: Thu, 8 Jan 2015 14:48:47 +0100 Subject: auth-deny : from file to LDAP Message-ID: <20150108134847.GA6519@parmesan.sis.pasteur.fr> Hello Timo, I want to deny access to some users. For now I'm doing it using 2 passdb's and listing users in a text file. !include auth-deny.conf.ext -> passwd-file driver !include auth-ldap.conf.ext -> ldap driver I want to do the same using only LDAP. I'm not quite sure how to do it : a) should I . change the driver of the first passdb from passwd-file to ldap . for user to be rejected, add an LDAP attribute named "foo" with a value of "yes" and map it with something like this : pass_attrs = ....,foo=deny in dovecot-ldap.conf.ext ? b) or could I use only one ldap passdb by changing the pass_filter from pass_filter = (&(objectClass=posixAccount)(uid=%u)) to something like pass_filter = (&(objectClass=posixAccount)(uid=%u)(!foo=yes)) ? Thanks -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From hummel at pasteur.fr Thu Jan 8 14:47:12 2015 From: hummel at pasteur.fr (Thomas HUMMEL) Date: Thu, 8 Jan 2015 15:47:12 +0100 Subject: auth-deny : from file to LDAP In-Reply-To: <20150108134847.GA6519@parmesan.sis.pasteur.fr> References: <20150108134847.GA6519@parmesan.sis.pasteur.fr> Message-ID: <20150108144712.GA13205@parmesan.sis.pasteur.fr> On Thu, Jan 08, 2015 at 02:48:47PM +0100, hummel at pasteur.fr wrote: > Hello Timo, > a) should I > > . change the driver of the first passdb from passwd-file to ldap > . for user to be rejected, add an LDAP attribute named "foo" with a value of "yes" and map it with something like this : > > pass_attrs = ....,foo=deny in dovecot-ldap.conf.ext ? > This doesn't seem to work but maybe am I misunderstanding the logic : I thought that in the passdb{} section of auth-deny.conf.ext, you could comment "deny = yes" as long as the passdb returned an extra_field mapped on "deny" with the value of "yes" for users you'd want to deny access to: is that the case ? Maybe it's just something like : "if user is found in passdb but "deny = yes" is not stated in the passdb{} section, then access is granted ? > b) or could I use only one ldap passdb by changing the pass_filter > > from > > pass_filter = (&(objectClass=posixAccount)(uid=%u)) > > to something like > > pass_filter = (&(objectClass=posixAccount)(uid=%u)(!foo=yes)) > > ? This is working but I don't know if this is the recommended way of doing it. Thanks. -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From laszlo.toth at linguamatics.com Thu Jan 8 16:42:25 2015 From: laszlo.toth at linguamatics.com (Laszlo Toth) Date: Thu, 08 Jan 2015 16:42:25 +0000 Subject: replication hangs Message-ID: <54AEB371.8090101@linguamatics.com> Hi, I have 2 Dovecot servers (2.2.10) which are replicating each other. Replication works fine except for one user. [root at dovecot2 ~]# doveadm -D sync -u username dovecot1 doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/lib15_notify_plugin.so doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/lib20_replication_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(username): Debug: Effective uid=489, gid=489, home=/data/mail/username doveadm(username): Debug: fs: root=/data/mail/username, index=, indexpvt=, control=, inbox=, alt= doveadm(username): Debug: Namespace : Using permissions from /data/mail/username: mode=0700 gid=default dsync-local(username): Debug: brain M: in state=master_recv_handshake dsync-local(username): Debug: brain M: out state=send_mailbox_tree changed=1 dsync-local(username): Debug: brain M: in state=send_mailbox_tree dsync-local(username): Debug: brain M: out state=send_mailbox_tree_deletes changed=1 dsync-local(username): Debug: brain M: in state=send_mailbox_tree_deletes dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=1 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree dsync-local(username): Debug: brain M: out state=recv_mailbox_tree_deletes changed=1 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(username): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree_deletes dsync-local(username): Debug: brain M: out state=recv_mailbox_tree_deletes changed=0 dsync-local(username): Debug: brain M: in state=recv_mailbox_tree_deletes ...and it hangs. Both servers are CentOS 6.6 x64. Replication status for the problematic user: [root at deoveadm2 ~]# doveadm replicator status username username priority fast sync full sync failed username low 42:10:59 42:10:59 y I tried to reindex the problematic mailbox, tried to restart both mail servers but the issue still here. I found no error messages in /var/log/maillog. What should I try next? Thanks, Laszlo From redflag237 at googlemail.com Thu Jan 8 21:33:53 2015 From: redflag237 at googlemail.com (Jonas Plitt) Date: Thu, 8 Jan 2015 22:33:53 +0100 Subject: Dovecot replication - notify not working Message-ID: Dear Dovecot-Admins, I've set up a pair of Dovecot Servers, please find config of server one attached. They are configured to replicate changes over a tcp connection using port 12345, set up as described in http://wiki2.dovecot.org/Replication article page. Adding the user postboxes to replication using "doveadm replicator add '*' " syncs the mailboxes as expected. Unfortunately both of the servers does not propagate any changes, e.G. new mail in Sent folder. Removing and adding the mailboxes again syncs them again. Can anybody help me to get the notify working, please? many thanks in advance, Jonas -------------- next part -------------- # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab094.8 x86_64 Ubuntu 14.04.1 LTS auth_mechanisms = plain login doveadm_password = MySecret doveadm_port = 12345 dsync_remote_cmd = ssh -p 4711 -l%{login} %{host} doveadm dsync-server -u%u listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = Dovecot (Ubuntu) ready. mail_gid = mail mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication" mail_privileged_group = vmail mail_uid = vmail mailbox_idle_check_interval = 1 mins namespace inbox { inbox = yes location = prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { mail_replica = tcp:luigi.servers.netz1000.de:12345 quota = dict:user::file:/var/vmail/%d/%n/.quotausage replication_full_sync_interval = 1 hour sieve = /var/vmail/%d/%n/.sieve } postmaster_address = postmaster at sara.servers.netz1000.de protocols = imap pop3 service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service config { unix_listener config { user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { client_limit = 1000 inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 500 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0666 } } ssl_ca = Hi all, when hardening dovecot against the POODLE vulnerability, we followed the advise to disable SSL2 and SSL3 but this is giving problems with some email clients (claws-mail). ssl_protocols = !SSLv2 !SSLv3 results in the following error: dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, session=<2C8jBjIMmQBVGNd1> Our smtp server is postfix, can you please suggest a better 'ssl_protocols' and 'ssl_cipher_list' configuration ? We are running Debian 7 Wheezy Thank you, RuggedInbox team From CMarcus at Media-Brokers.com Fri Jan 9 07:38:18 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 09 Jan 2015 02:38:18 -0500 Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> Message-ID: <54AF856A.9040306@Media-Brokers.com> doveconf -n output? On 1/9/2015 2:07 AM, ml at ruggedinbox.com wrote: > Hi all, when hardening dovecot against the POODLE vulnerability, > we followed the advise to disable SSL2 and SSL3 > but this is giving problems with some email clients (claws-mail). > > ssl_protocols = !SSLv2 !SSLv3 > > results in the following error: > > dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, > rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: > error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, > session=<2C8jBjIMmQBVGNd1> > > Our smtp server is postfix, can you please suggest a better > 'ssl_protocols' and 'ssl_cipher_list' configuration ? > We are running Debian 7 Wheezy > > Thank you, > RuggedInbox team > From ml at ruggedinbox.com Fri Jan 9 07:44:55 2015 From: ml at ruggedinbox.com (ml at ruggedinbox.com) Date: Fri, 09 Jan 2015 07:44:55 +0000 Subject: dovecot on wheezy, best ssl configuration =?UTF-8?Q?=3F?= In-Reply-To: <54AF856A.9040306@Media-Brokers.com> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF856A.9040306@Media-Brokers.com> Message-ID: Hi thanks for your help, follows the doveconf -n output: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab094.7 x86_64 Debian 7.7 simfs auth_mechanisms = plain login auth_verbose = yes debug_log_path = /var/log/dovecot/debug.log default_client_limit = 8192 default_process_limit = 2048 director_username_hash = %Lu dsync_remote_cmd = ssh -l%{login} %{host} doveadm dsync-server -u%u -U first_valid_gid = 5000 first_valid_uid = 5000 imap_id_send = name * last_valid_gid = 5000 last_valid_uid = 5000 login_greeting = Welcome to ruggedinbox.com mail_gid = vmail mail_location = maildir:/var/vmail/%d/%n/Maildir:INDEX=/var/vmail/%d/%n/Maildir/indexes mail_max_userip_connections = 25 mail_privileged_group = vmail mail_shared_explicit_inbox = no mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/mysql.conf driver = sql } postmaster_address = postmaster at ruggedinbox.com protocols = imap pop3 sieve replication_full_sync_interval = 1 days service auth { client_limit = 0 drop_priv_before_exec = no executable = auth idle_kill = 0 process_limit = 1 process_min_avail = 0 service_count = 0 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-client { mode = 0600 } unix_listener auth-login { mode = 0600 user = $default_internal_user } unix_listener auth-master { mode = 0600 } unix_listener auth-userdb { mode = 0666 user = $default_internal_user } unix_listener login/login { mode = 0666 } user = $default_internal_user vsz_limit = 128 M } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login idle_kill = 0 inet_listener imap { port = 143 ssl = no } inet_listener imaps { port = 993 ssl = yes } process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 128 M } service imap { client_limit = 1 drop_priv_before_exec = no executable = imap idle_kill = 0 process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 unix_listener login/imap { group = mode = 0666 user = } vsz_limit = 128 M } service lmtp { client_limit = 1 drop_priv_before_exec = no executable = lmtp idle_kill = 0 process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 unix_listener lmtp { mode = 0666 } vsz_limit = 128 M } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login idle_kill = 0 inet_listener pop3 { port = 110 ssl = no } inet_listener pop3s { port = 995 ssl = yes } process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 128 M } service pop3 { client_limit = 1 drop_priv_before_exec = no executable = pop3 idle_kill = 0 process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 unix_listener login/pop3 { mode = 0666 } vsz_limit = 128 M } shutdown_clients = no ssl_cert = doveconf -n output? > > On 1/9/2015 2:07 AM, ml at ruggedinbox.com wrote: >> Hi all, when hardening dovecot against the POODLE vulnerability, >> we followed the advise to disable SSL2 and SSL3 >> but this is giving problems with some email clients (claws-mail). >> >> ssl_protocols = !SSLv2 !SSLv3 >> >> results in the following error: >> >> dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): >> user=<>, >> rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: >> error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, >> session=<2C8jBjIMmQBVGNd1> >> >> Our smtp server is postfix, can you please suggest a better >> 'ssl_protocols' and 'ssl_cipher_list' configuration ? >> We are running Debian 7 Wheezy >> >> Thank you, >> RuggedInbox team >> From philipp at devh.de Fri Jan 9 07:50:12 2015 From: philipp at devh.de (Philipp Resch) Date: Fri, 09 Jan 2015 08:50:12 +0100 Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> Message-ID: <54AF8834.1000708@devh.de> Am 09.01.2015 um 08:07 schrieb ml at ruggedinbox.com: > Hi all, when hardening dovecot against the POODLE vulnerability, > we followed the advise to disable SSL2 and SSL3 > but this is giving problems with some email clients (claws-mail). > > ssl_protocols = !SSLv2 !SSLv3 > > results in the following error: > > dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, > rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: > error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, > session=<2C8jBjIMmQBVGNd1> > > Our smtp server is postfix, can you please suggest a better > 'ssl_protocols' and 'ssl_cipher_list' configuration ? > We are running Debian 7 Wheezy > > Thank you, > RuggedInbox team Hi, this is my config on Wheezy. I don't know if it's 'best', but it works for us: # SSL protocols to use ssl_protocols = !SSLv2 !SSLv3 # Prefer the server's order of ciphers over client's. ssl_prefer_server_ciphers = yes ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA:ECDHE-RSA-DES-CBC3-SHA:DES-CBC3-SHA:!SSLv2 Cheers, Philipp -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4296 bytes Desc: S/MIME Cryptographic Signature URL: From ml at ruggedinbox.com Fri Jan 9 07:58:22 2015 From: ml at ruggedinbox.com (ml at ruggedinbox.com) Date: Fri, 09 Jan 2015 07:58:22 +0000 Subject: dovecot on wheezy, best ssl configuration =?UTF-8?Q?=3F?= In-Reply-To: <54AF8834.1000708@devh.de> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF8834.1000708@devh.de> Message-ID: <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> Hi thanks for your help! Trying to set your same parameters, when restarting dovecot, gives the error: doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 136: Unknown setting: ssl_prefer_server_ciphers doveconf: Error: managesieve-login: dump-capability process returned 89 doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 136: Unknown setting: ssl_prefer_server_ciphers [....] Restarting IMAP/POP3 mail server: dovecotdoveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 136: Unknown setting: ssl_prefer_server_ciphers doveconf: Error: managesieve-login: dump-capability process returned 89 doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 136: Unknown setting: ssl_prefer_server_ciphers and if trying to comment the line with 'ssl_prefer_server_ciphers', dovecot restarts fine but same problem as before, claws-mail can't connect. dovecot version is 2.1.7 any hints ? On 2015-01-09 07:50, Philipp Resch wrote: > Am 09.01.2015 um 08:07 schrieb ml at ruggedinbox.com: >> Hi all, when hardening dovecot against the POODLE vulnerability, >> we followed the advise to disable SSL2 and SSL3 >> but this is giving problems with some email clients (claws-mail). >> >> ssl_protocols = !SSLv2 !SSLv3 >> >> results in the following error: >> >> dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): >> user=<>, >> rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: >> error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, >> session=<2C8jBjIMmQBVGNd1> >> >> Our smtp server is postfix, can you please suggest a better >> 'ssl_protocols' and 'ssl_cipher_list' configuration ? >> We are running Debian 7 Wheezy >> >> Thank you, >> RuggedInbox team > > Hi, > > this is my config on Wheezy. I don't know if it's 'best', but it works > for us: > > # SSL protocols to use > ssl_protocols = !SSLv2 !SSLv3 > # Prefer the server's order of ciphers over client's. > ssl_prefer_server_ciphers = yes > ssl_cipher_list = > ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA:ECDHE-RSA-DES-CBC3-SHA:DES-CBC3-SHA:!SSLv2 > > > Cheers, > Philipp From skdovecot at smail.inf.fh-brs.de Fri Jan 9 08:00:53 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 9 Jan 2015 09:00:53 +0100 (CET) Subject: auth-deny : from file to LDAP In-Reply-To: <20150108144712.GA13205@parmesan.sis.pasteur.fr> References: <20150108134847.GA6519@parmesan.sis.pasteur.fr> <20150108144712.GA13205@parmesan.sis.pasteur.fr> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 8 Jan 2015, Thomas HUMMEL wrote: > On Thu, Jan 08, 2015 at 02:48:47PM +0100, hummel at pasteur.fr wrote: >> Hello Timo, > >> a) should I >> >> . change the driver of the first passdb from passwd-file to ldap >> . for user to be rejected, add an LDAP attribute named "foo" with a value of "yes" and map it with something like this : >> >> pass_attrs = ....,foo=deny in dovecot-ldap.conf.ext ? >> > > This doesn't seem to work but maybe am I misunderstanding the logic : > > I thought that in the passdb{} section of auth-deny.conf.ext, you could comment > "deny = yes" as long as the passdb returned an extra_field mapped on "deny" > with the value of "yes" for users you'd want to deny access to: is that the > case ? > > Maybe it's just something like : "if user is found in passdb but "deny = > yes" is not stated in the passdb{} section, then access is granted ? > The deny=yes is a special syntax: If this passdb matches -> deny, there is no ExtraField "deny". >> b) or could I use only one ldap passdb by changing the pass_filter >> >> from >> >> pass_filter = (&(objectClass=posixAccount)(uid=%u)) >> >> to something like >> >> pass_filter = (&(objectClass=posixAccount)(uid=%u)(!foo=yes)) >> > > This is working but I don't know if this is the recommended way of doing it. Actually I use "(!(deniedService=%Ls))", but keep in mind that you do not "deny" an user knowingly, but that this user is not found. The semantic is different. What you could try - I do not remember anybody posting something like this - - is to combine a ldap passdb with deny=yes. The doc http://wiki2.dovecot.org/PasswordDatabase does not restrict the deny=yes to just passwd-file, hence, if you create yet another LDAP conf file that matches only denied users and write: passdb { driver = ldap args = /etc/dovecot/dovecot-ldap_denied_users.conf.ext deny = yes } - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVK+Ktnz1H7kL/d9rAQJo7wgAmDYddi3TShOOiOhcFYrM3YN9T3OaMex7 EU9BKMIn2l8DcPBixWSemwDpOsiprzKgMa0hqxVK9jXT0a5FaQaJqo/l0u7/x5uB EPEw71baztB1YPwiyyU2HLL5CIBVdWaXlMNtQyKoh14GWiMgdJaTcvM9nZGteaYJ qAxPD3zifcpZRoU2L2TpMJRyMVdnAgm8p90hulCEXOGY3QNxzKa6BEUuZsZTrV/e quqwDWYxe1Mkng36lz4K2bh5xB6NVsbyq0OzdhfJe5RODCVu0dptHn8KJPMvgB5a 2qYPraXoenNr6NBNfUvFGD+x+rjse3SB5AoKiO5KZRS3XelOIECiRA== =ztWz -----END PGP SIGNATURE----- From philipp at devh.de Fri Jan 9 08:06:16 2015 From: philipp at devh.de (Philipp Resch) Date: Fri, 09 Jan 2015 09:06:16 +0100 Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF8834.1000708@devh.de> <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> Message-ID: <54AF8BF8.40308@devh.de> Am 09.01.2015 um 08:58 schrieb ml at ruggedinbox.com: > Hi thanks for your help! > Trying to set your same parameters, when restarting dovecot, gives the > error: > > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf > line 136: Unknown setting: ssl_prefer_server_ciphers > doveconf: Error: managesieve-login: dump-capability process returned 89 > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf > line 136: Unknown setting: ssl_prefer_server_ciphers > [....] Restarting IMAP/POP3 mail server: dovecotdoveconf: Fatal: Error > in configuration file /etc/dovecot/dovecot.conf line 136: Unknown > setting: ssl_prefer_server_ciphers > doveconf: Error: managesieve-login: dump-capability process returned 89 > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf > line 136: Unknown setting: ssl_prefer_server_ciphers > > and if trying to comment the line with 'ssl_prefer_server_ciphers', > dovecot restarts fine but same problem as before, claws-mail can't connect. > > dovecot version is 2.1.7 > > any hints ? > > > On 2015-01-09 07:50, Philipp Resch wrote: >> Am 09.01.2015 um 08:07 schrieb ml at ruggedinbox.com: >>> Hi all, when hardening dovecot against the POODLE vulnerability, >>> we followed the advise to disable SSL2 and SSL3 >>> but this is giving problems with some email clients (claws-mail). >>> >>> ssl_protocols = !SSLv2 !SSLv3 >>> >>> results in the following error: >>> >>> dovecot: pop3-login: Disconnected (no auth attempts in 1 secs): user=<>, >>> rip=XXX, lip=XXX, TLS handshaking: SSL_accept() failed: >>> error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher, >>> session=<2C8jBjIMmQBVGNd1> >>> >>> Our smtp server is postfix, can you please suggest a better >>> 'ssl_protocols' and 'ssl_cipher_list' configuration ? >>> We are running Debian 7 Wheezy >>> >>> Thank you, >>> RuggedInbox team >> >> Hi, >> >> this is my config on Wheezy. I don't know if it's 'best', but it works >> for us: >> >> # SSL protocols to use >> ssl_protocols = !SSLv2 !SSLv3 >> # Prefer the server's order of ciphers over client's. >> ssl_prefer_server_ciphers = yes >> ssl_cipher_list = >> ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA:ECDHE-RSA-DES-CBC3-SHA:DES-CBC3-SHA:!SSLv2 >> >> >> >> Cheers, >> Philipp Hi, yes, the ssl_prefer_server_ciphers setting was introduced in 2.2.x It seems as if claws mail is preferring SSLv3, have you tried connecting with another client (e.g. Thunderbird)? If that works you might want to check with the CM devs... Cheers, Philipp -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4296 bytes Desc: S/MIME Cryptographic Signature URL: From skdovecot at smail.inf.fh-brs.de Fri Jan 9 08:11:25 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 9 Jan 2015 09:11:25 +0100 (CET) Subject: Quota, doveadm, postfixadmin In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 29 Dec 2014, Ricardo wrote: I'm not sure if I understood your question(s), so I summarize quota in a top post: you did not have posted your config, hence, if the quota is _not_ calculated each time, you query the quota, you get a cached value. Try this: doveadm quota recalc -u ... doveadm quota get -u ... Do you have a domain quota? Then this will fail. > I have Postfix+Dovecot+MySQL+Postfixadmin with CentOS 6.6, a virtual > machines for testing, and then pass it to servers in production, my problem > is the next: > > I add 30 MB for domain.local, then cast 10 MB for test1 at domain.local and 20 > MB for test2 at domain.local, so far so good. > Sending an attachment from one account to another, the example of > test1 at domain.local to test2 at domain.local, test2 at domain.local receive the > email with the attachment. > > If I write in the shell doveadm quota get -u test2 at domain.local I get the > following: > > Quota > name > Type Value Limit % > user > STORAGE 3391 20000 16 > user > MESSAGE 3 10000 0 > > Informs me that I this consuming 16% of the 20 MB assigned to this account, > but if I delete the file and again write doveadm quota get -u > test2 at domain.local I release this: > > Quota > name > Type Value Limit % > user > STORAGE 6782 20000 33 > user > MESSAGE 4 10000 0 > > Normal serious you continue using 16%, not 33%, when you go into the trash, > because that happens from one place to another, right? These statistics the > Postfixadmin Dame them so much as writing in the shell doveadm quota get -u > test2 at domain.local. I delete the file in the trash and to again write > doveadm quota get -u test2 at domain.local still see me. > > Quota > name > Type Value Limit % > user > STORAGE 6782 20000 33 > user > MESSAGE 4 10000 0 > > But I realize that so that it resets to 0%, I have to close the mail > client, if not would follow using 33% and is when is 0% as shown here. > > Quota > name > Type Value Limit % > user > STORAGE 1 20000 0 > user > MESSAGE 2 10000 0 > > Is this normal? The client to send emails and exceed quota will having to > always close the mail client, so you reset and stays at 0%? > > Can anyone help me? I have looked online but I can't find information about > this. Has happened to someone? > > Thanks so much. > > Best regards. > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVK+NLXz1H7kL/d9rAQIvXAgAu6wGugsqWm4V1V2FpvnpSFFIqOKu6eO2 Wiu8JFo11K5LYsuUmc2j8YgQVZJHHtYuUSqcwF71JFBNIhYymPoEQN8sAz8D0KLe 2CFpLWbasDr50pC8tTbO9Qrr1C6OCQJz68aiQaBGQPfKXf9Dpim59JhAnh5DbI2T xFAj8lpzzDKfHeAeFqKtotmhhZtcCN9Cgkes3EsSxdEAAVQ8KmHtb5hXrqtVxrTv lzqF5wS7poC+eKYKyW08nZgvAntAntTpZNJIGR3s8cs9sWJzfgIgqoZNQD9wHEzN aruOo/OKwPC2UbGboylKElG1+ZZMX2z9a1CNJf7Ph5OrGA7DT9XrdA== =N4en -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Fri Jan 9 08:16:25 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 9 Jan 2015 09:16:25 +0100 (CET) Subject: Authcache and user changing In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 29 Dec 2014, Lazy wrote: > > I have noticed that during auth cache hits usernames are not updated. > (We use ldap backend > and change username with > user_attrs = uid=user, mailMessageStore=home, > mailQuotaSize=quota_rule=*:bytes=%$ > > cold cache > > lmtp(14414): Debug: auth input: testmon_testmon > home=/vmail/te/testmon_testmon quota_rule=*:bytes=104857600 > lmtp(14414): Debug: changed username to testmon_testmon > lmtp(14414): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 > > > hot cache > > lmtp(14715): Debug: auth input: iqmon at mon.test.pl > home=/vmail/iq/testmon_testmon quota_rule=*:bytes=104857600 > lmtp(14715): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 > > this kills our dictionary based quota (users have multiple quota > instances depending on > timing and alias the email was delivered to. > > Is there a way to force addition of user to the auth cache ? Did you posted your doveconf -n and ldap settings somewhere? - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVK+OWXz1H7kL/d9rAQLCawf/ROHvFecRZorrmGadAUmOk7Nj26m4WBH9 gQVbtOlPN4Vi8An1ldkTtyAqs8M7d67rGp3pW5ncZtd6BbseaaSBSNXaVXomZoBO RwOxrXQ0c7Em+yDegKwz085Iu4h6vTkzlsnDhRl1aiEn4r05U8GSyKTV4hchB1Ej rkfEILSWHK/bzvbeGz4qwxpFRyd4CMabK9IsFz0qiQFhavE8AKnnega4HDk2Vsdr sABB8xI25++DD4yKBKDsAari1PkoPrndzURexdH/chJFGpD/eFbO7t9owtyFnaU9 eNZYH1H81TjkHG8u4Rq2AoTFZt2RnptfHtCbs0lQnhuddrmH6gv8ZQ== =QIur -----END PGP SIGNATURE----- From CMarcus at Media-Brokers.com Fri Jan 9 08:34:19 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 09 Jan 2015 03:34:19 -0500 Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: <54AF8BF8.40308@devh.de> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF8834.1000708@devh.de> <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> <54AF8BF8.40308@devh.de> Message-ID: <54AF928B.1050003@Media-Brokers.com> On 1/9/2015 3:06 AM, Philipp Resch wrote: > It seems as if claws mail is preferring SSLv3 And since dovecot is really not affected by the poodle vulnerability, if you can't upgrade (I believe 2.2 is in the backports repo?), probably easiest to just reenable SSLv3... From hummel at pasteur.fr Fri Jan 9 10:02:48 2015 From: hummel at pasteur.fr (Thomas HUMMEL) Date: Fri, 9 Jan 2015 11:02:48 +0100 Subject: auth-deny : from file to LDAP In-Reply-To: References: <20150108134847.GA6519@parmesan.sis.pasteur.fr> <20150108144712.GA13205@parmesan.sis.pasteur.fr> Message-ID: <20150109100248.GA16884@parmesan.sis.pasteur.fr> On Fri, Jan 09, 2015 at 09:00:53AM +0100, Steffen Kaiser wrote: > The deny=yes is a special syntax: If this passdb matches -> deny, there is > no ExtraField "deny". Thanks for your answer. That's what I thought after my tests. This explains why I was still able to log in... > but keep in mind that you do not "deny" an user knowingly, but that this user > is not found. The semantic is different. I know, I thought about that. But still what could be the unwanted side effects ? > What you could try - I do not remember anybody posting something like this - > - is to combine a ldap passdb with deny=yes. I thought about that too, but that would mean setting up another LDAP directory, which I find a little bit overkill. Thanks. -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From lazy404 at gmail.com Fri Jan 9 10:36:04 2015 From: lazy404 at gmail.com (Lazy) Date: Fri, 9 Jan 2015 11:36:04 +0100 Subject: Authcache and user changing In-Reply-To: References: Message-ID: 2015-01-09 9:16 GMT+01:00 Steffen Kaiser : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > On Mon, 29 Dec 2014, Lazy wrote: > >> >> I have noticed that during auth cache hits usernames are not updated. >> (We use ldap backend >> and change username with >> user_attrs = uid=user, mailMessageStore=home, >> mailQuotaSize=quota_rule=*:bytes=%$ >> >> cold cache >> >> lmtp(14414): Debug: auth input: testmon_testmon >> home=/vmail/te/testmon_testmon quota_rule=*:bytes=104857600 >> lmtp(14414): Debug: changed username to testmon_testmon >> lmtp(14414): Debug: Added userdb setting: >> plugin/quota_rule=*:bytes=104857600 >> >> >> hot cache >> >> lmtp(14715): Debug: auth input: iqmon at mon.test.pl >> home=/vmail/iq/testmon_testmon quota_rule=*:bytes=104857600 >> lmtp(14715): Debug: Added userdb setting: >> plugin/quota_rule=*:bytes=104857600 >> >> this kills our dictionary based quota (users have multiple quota >> instances depending on >> timing and alias the email was delivered to. >> >> Is there a way to force addition of user to the auth cache ? > > > Did you posted your doveconf -n and ldap settings somewhere? I have atached a dirty hack that fixes the issue for me (username changes are cached in userdb auth cache). config follows dovecot -n passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } dovecot-ldap.conf.ext debug_level = 32 auth_bind = no ldap_version = 3 base = ou=accounts,o=test,c=pl user_attrs = uid=user, mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ user_filter = (&(&(!(accountStatus=deleted))(objectClass=qMailUser))(|(mail=%u)(uid=%u)(mailAlternateAddress=%u))) pass_attrs = uid=user, userPassword=password pass_filter = (&(objectClass=qMailUser)(|(mail=%u)(uid=%u)(mailAlternateAddress=%u))) iterate_attrs = uid=user iterate_filter = (&(&(objectClass=qmailUser)(!(accountStatus=deleted)))) full doveconf -n # 2.2.15: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.7 auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 5 mins auth_debug = yes auth_master_user_separator = * auth_mechanisms = plain login auth_username_format = %u auth_verbose = yes base_dir = /var/run/dovecot/ deliver_log_format = msgid=%m f:%f s:%s %$ disable_plaintext_auth = no import_environment = TZ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = Delivered-To listen = 10.0.55.22 login_greeting = Imap ready. login_trusted_networks = 10.0.55.2/32 10.0.55.3/32 mail_debug = yes mail_gid = 300 mail_location = maildir:~/Maildir:INDEX=/var/dovecot_indexes%h mail_plugins = quota expire notify mail_log mail_uid = 300 maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox SPAM { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { expire = SPAM expire_dict = redis:host=127.0.0.1:prefix=expire/ last_login_key = %u mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid from subject size quota = dict:User quota::redis:host=127.0.0.1:prefix=user/ quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve_global_dir = /etc/sieve_global sieve_max_redirects = 20 } postmaster_address = postmaster at test.pl protocols = imap pop3 lmtp sieve recipient_delimiter = service auth { unix_listener auth-userdb { mode = 0777 } } service dict { unix_listener dict { group = vmail user = vmail } } service doveadm { inet_listener { port = 4567 } } service imap-login { process_min_avail = 8 service_count = 0 } service imap { process_limit = 14000 } service lmtp { inet_listener lmtp { address = dovecot2 port = 24 } process_min_avail = 5 user = vmail } service pop3-login { process_min_avail = 8 service_count = 0 } service pop3 { process_limit = 10000 } service quota-warning { executable = script /usr/local/bin/quota_warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no syslog_facility = local2 userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { auth_username_format = %u mail_plugins = quota expire notify mail_log sieve syslog_facility = local3 } protocol lda { mail_plugins = quota expire notify mail_log sieve syslog_facility = local3 } protocol imap { mail_max_userip_connections = 30 mail_plugins = quota expire notify mail_log imap_quota last_login } protocol sieve { plugin { sieve = ~/.dovecot.sieve sieve_max_script_size = 50K sieve_quota_max_scripts = 5 sieve_quota_max_storage = 500K sieve_storage = ~/sieve/ } } protocol pop3 { mail_plugins = quota expire notify mail_log last_login pop3_client_workarounds = outlook-no-nuls pop3_fast_size_lookups = yes pop3_uidl_format = %f } -- Michal Grzedzicki -------------- next part -------------- A non-text attachment was scrubbed... Name: cache_user.patch Type: application/octet-stream Size: 2248 bytes Desc: not available URL: From tim at bastelfreak.de Fri Jan 9 13:13:39 2015 From: tim at bastelfreak.de (Tim) Date: Fri, 09 Jan 2015 14:13:39 +0100 Subject: Corrupted index cache after migrating from maildir to mdbox In-Reply-To: <546F3853.2030001@bastelfreak.de> References: <54275AD9.2080900@bastelfreak.de> <408E7528-3DA5-4469-ADB8-0278679DC3A7@iki.fi> <546F0D10.5060909@bastelfreak.de> <546F1AC8.8020800@solu.fi> <546F3853.2030001@bastelfreak.de> Message-ID: <54AFD403.2040105@bastelfreak.de> On 21.11.2014 14:04, Tim wrote: > On 21.11.2014 11:58, Toni Mattila wrote: >> Hi, >> >> I think your issue is due to your corrupted maildir sizes in filenames >> so you got corrupted index that way to your mdbox. That is not >> something you can easily fix afterwords other than fixing the maildir >> first and then migrate to mdbox. >> > well, I don't have the maildir files anymore or they are way out of > date. I can try to mirror the directory structure with doveadm to create > another mdbox instance. maybe this will it also fix. I successfully mirrored all mailboxes with dsync, but this didn't fix anything. My mailclients are still complaining about the virtual size. or broken index cache files. Any thoughts here? Timo maybe? >> Best Regards, >> Toni >> >> On 21.11.2014 11:59, Tim wrote: >>> Hi Timo, >>> sorry for the late reply, missed this mail. >>> On 15.10.2014 05:05, Timo Sirainen wrote: >>>> On 27 Sep 2014, at 17:48, Tim wrote: >>>> >>>>> unfortunately I'm getting the following two kinds of error messages >>>>> for >>>>> all mail boxes in my mail.log: >>>>> >>>>> Error: Corrupted index cache file >>>>> /home/vmail/%domain/%user/mdbox/mailboxes/%maildir/dbox-Mails/dovecot.index.cache: >>>>> >>>>> Broken virtual size for mail UID 2564 >>>>> >>>>> and: >>>>> >>>>> Error: read(/home/vmail/%domain/%user/mdbox/storage/m.108 >>>>> ): FETCH BODY[] for mailbox %maildir UID 2564 got too little data: >>>>> 2417 >>>>> vs 6319 >>>> If that happens to all the mails, something went wrong. Did you >>>> maybe use zlib plugin while writing the messages (compressed), but >>>> now you're trying to read the mails without zlib plugin? >>> the migration from maildir to mdbox is a few weeks in the past so I >>> don't remember correctly. IMHO I migrated from maildir with disabled >>> zlib to mdbox. with mdbox, zlib was active the hole time and it is still >>> active. Here is my doveconf -n output: http://p.bastelfreak.de/l2wq/ . >>> Do you need anything else for debugging? >>>> The error messages may be somewhat confusing. The problem isn't >>>> dovecot.index.cache file, but that the message sizes in the m.* >>>> files don't match the contents. There's no easy way to fix those, >>>> because they should just about never be wrong. From skdovecot at smail.inf.fh-brs.de Fri Jan 9 13:17:35 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 9 Jan 2015 14:17:35 +0100 (CET) Subject: auth-deny : from file to LDAP In-Reply-To: <20150109100248.GA16884@parmesan.sis.pasteur.fr> References: <20150108134847.GA6519@parmesan.sis.pasteur.fr> <20150108144712.GA13205@parmesan.sis.pasteur.fr> <20150109100248.GA16884@parmesan.sis.pasteur.fr> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 9 Jan 2015, Thomas HUMMEL wrote: > On Fri, Jan 09, 2015 at 09:00:53AM +0100, Steffen Kaiser wrote: > >> The deny=yes is a special syntax: If this passdb matches -> deny, there is >> no ExtraField "deny". > > Thanks for your answer. That's what I thought after my tests. This explains why > I was still able to log in... > >> but keep in mind that you do not "deny" an user knowingly, but that this user >> is not found. The semantic is different. > > I know, I thought about that. But still what could be the unwanted side effects > ? The logs contain different entries. Denied users show up as failed login attempts and if you might have running a log analyzer that acts on that security breach. Also, later you might add another passdb, which could succeed on that user, who is denied in LDAP. >> What you could try - I do not remember anybody posting something like this - >> - is to combine a ldap passdb with deny=yes. > > I thought about that too, but that would mean setting up another LDAP > directory, which I find a little bit overkill. You need not no 2nd LDAP directory, but another configuration file with another LDAP search filter, that hits denied users only. This could be implemented by a new attribute. You will gain the log information about the denied user. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVK/U73z1H7kL/d9rAQL8EQf+OpSOGTlFaiqH+VGvsnK6BHkywXGjkgML 0+360Agc2WwI/2GLHbOS0vRjs2Wzm91FVyXXD5kd1CXTOjF1Xp7rVoaR1B9aQtoi m1DOTdzJ/CbKH8b2NOvEpUdRJVj/qGCD6DWkIOppWA4pjSr45xX5znCJA45Y7xxi C6kfUMx8tRmb5byUR5qyjrZeMxHF1mrmoH4UaIa3mvFHiv1WSFom8NM+pnJ99yoH IOz3ZQ5mxP3l6jAV5c3w1P/6fgDVazHvji5ecvj+JncmEhEvwV4XeGrbfTuk9DnB f9JwygXXBmSA58Ncgz1eJi/E7dDSijrPG3+7QR2obDIoqJLjzEOBEQ== =WZji -----END PGP SIGNATURE----- From fuckspam at wheres5.com Fri Jan 9 17:49:09 2015 From: fuckspam at wheres5.com (Hoggins!) Date: Fri, 09 Jan 2015 18:49:09 +0100 Subject: PROXY protocol Message-ID: <54B01495.6040300@wheres5.com> Hello folks, Any plans on implementing the PROXY protocol to allow Dovecot being behind a TCP proxy, and still logging the real IP address of the users ? See : http://blog.haproxy.com/haproxy/proxy-protocol/ Thanks ! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From eslist at ols.es Fri Jan 9 19:25:55 2015 From: eslist at ols.es (David Saez) Date: Fri, 09 Jan 2015 20:25:55 +0100 Subject: pop3 seen Message-ID: <54B02B43.3070805@ols.es> Hi Is there a way to know if a message has been dowloaded via pop3 and not seen via imap ? From bertrand.caplet at chunkz.net Fri Jan 9 19:34:25 2015 From: bertrand.caplet at chunkz.net (Bertrand Caplet) Date: Fri, 09 Jan 2015 20:34:25 +0100 Subject: pop3 seen In-Reply-To: <54B02B43.3070805@ols.es> References: <54B02B43.3070805@ols.es> Message-ID: <54B02D41.4020906@chunkz.net> > Hi > > Is there a way to know if a message has been dowloaded via pop3 > and not seen via imap ? Hey, Why don't you use only imap ? It would be so simpler. Regards, -- CHUNKZ.NET - dodgy DIYer and computer technician Bertrand Caplet, Flers (FR) Feel free to send encrypted/signed messages Key ID: FF395BD9 GPG FP: DE10 73FD 17EB 5544 A491 B385 1EDA 35DC FF39 5BD9 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From eslist at ols.es Fri Jan 9 20:28:19 2015 From: eslist at ols.es (David Saez) Date: Fri, 09 Jan 2015 21:28:19 +0100 Subject: pop3 seen In-Reply-To: <54B02D41.4020906@chunkz.net> References: <54B02B43.3070805@ols.es> <54B02D41.4020906@chunkz.net> Message-ID: <54B039E3.2080805@ols.es> Hi this are my users, not me, i just want to have a way to delete old messages that have only been seen by pop3 >> Hi >> >> Is there a way to know if a message has been dowloaded via pop3 >> and not seen via imap ? > > Hey, > Why don't you use only imap ? It would be so simpler. > > Regards, > From ml at ruggedinbox.com Fri Jan 9 21:30:51 2015 From: ml at ruggedinbox.com (ml at ruggedinbox.com) Date: Fri, 09 Jan 2015 21:30:51 +0000 Subject: dovecot on wheezy, best ssl configuration =?UTF-8?Q?=3F?= In-Reply-To: <54AF928B.1050003@Media-Brokers.com> References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF8834.1000708@devh.de> <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> <54AF8BF8.40308@devh.de> <54AF928B.1050003@Media-Brokers.com> Message-ID: On 2015-01-09 08:34, Charles Marcus wrote: > On 1/9/2015 3:06 AM, Philipp Resch wrote: >> It seems as if claws mail is preferring SSLv3 > > And since dovecot is really not affected by the poodle vulnerability, > if > you can't upgrade (I believe 2.2 is in the backports repo?), probably > easiest to just reenable SSLv3... Hi thanks Charles and thanks to all for your help. We decided to reenable SSLv3. We'll upgrade Dovecot when Debian will officially dist upgrade to version 8 :) From brian at eliassen.org Thu Jan 8 00:25:11 2015 From: brian at eliassen.org (Brian Eliassen) Date: Wed, 7 Jan 2015 16:25:11 -0800 Subject: Solaris 10 make check core dumps Message-ID: Hello everyone, I'm doing a standard build on Solaris 10 with Dovecot 2.2.15 without anything fancy: ./configure make make check make install Configure output looks like this: *** Install prefix . : /usr/local File offsets ... : 64bit I/O polling .... : poll I/O notifys .... : none SSL ............ : yes (OpenSSL) GSSAPI ......... : no passdbs ........ : static passwd passwd-file shadow pam checkpassword : -bsdauth -sia -ldap -sql -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword : -ldap -sql -vpopmail -nss SQL drivers .... : : -pgsql -mysql -sqlite Full text search : squat : -lucene -solr *** Everything works properly including the "make" until I try a "make check" at which time I receive the following failure. It would appear that something in test-http-auth is most unhappy. http auth challenges valid [0]: parse `Basic realm="WallyWorld"' ..... : ok http auth challenges valid [0]: [0]->scheme = Basic .................. : ok /bin/bash: line 1: 28057 Segmentation Fault (core dumped) ./$bin make[2]: *** [check-test] Error 1 make[2]: Leaving directory `/export/home/brian/software/dovecot-2.2.15/ src/lib-http' make[1]: *** [check-recursive] Error 1 make[1]: Leaving directory `/export/home/brian/software/dovecot-2.2.15/ src' make: *** [check-recursive] Error 1 This creates: -rw------- 1 root 1000 3523871 Jan 7 10:43 core Can this be safely ignored and I can proceed to "make install" or should I look somewhere else for the problem? Here is the mdb backtrace: $ mdb test-http-auth core Loading modules: [ libc.so.1 ld.so.1 ] > ::stack libc.so.1`strlen+0x50(29487, ffbffb00, ffbff43d, 0, 0, 0) libc.so.1`vsnprintf+0x70(4f2c0, 10f, 29478, ffbffaf8, 10, ff1c7078) t_noalloc_strdup_vprintf+0x3c(4f2c0, ffbffaf8, ffbffa2c, 1, 19, 0) p_strdup_vprintf+0xc(4cb90, 29478, ffbffaf8, ff1c759c, 4cc38, 4cb8c) t_strdup_printf+0x38(29478, 0, 0, 61fefeff, 80808080, 1010101) test_http_auth_challenges_valid+0x150(29000, 29400, 0, 4f250, 4cbb8, 0) test_run_funcs+0x24(4ca74, 1c00, ff1c7940, 4, ff312a00, ff13866c) test_run+0xc(4ca74, ffbffc7c, ffbffc84, 4f030, ff310140, 0) _start+0x5c(0, 0, 0, 0, 0, 0) Any help would be greatly appreciated. Brian From rhopfer at ica.jku.at Wed Jan 7 11:08:27 2015 From: rhopfer at ica.jku.at (Roland Hopferwieser) Date: Wed, 07 Jan 2015 12:08:27 +0100 Subject: Kerberos problems with /dev/null as dovecot home Message-ID: <54AD13AB.9040603@ica.jku.at> Hi, this is for documentation if someone else has the same problem. After migration from Dovecot 1 to 2.2. it was no longer able use Kerberos (Heimdal). I found out that dovecot tries to read ~/.krb5/config in the home of the dovecot user. This file doesn't need to exist, but if the home is set to /dev/null, it will not work. Maybe it would be nice to have an error message for this case. Regards, Roland From rs at sys4.de Fri Jan 9 23:15:23 2015 From: rs at sys4.de (Robert Schetterer) Date: Sat, 10 Jan 2015 00:15:23 +0100 Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: References: <0387611010ffcac0b0ebd128c390d8f0@ruggedinbox.com> <54AF8834.1000708@devh.de> <4f72732b4859ca2818a4b52c0b923df5@ruggedinbox.com> <54AF8BF8.40308@devh.de> <54AF928B.1050003@Media-Brokers.com> Message-ID: <54B0610B.4080206@sys4.de> Am 09.01.2015 um 22:30 schrieb ml at ruggedinbox.com: > On 2015-01-09 08:34, Charles Marcus wrote: >> On 1/9/2015 3:06 AM, Philipp Resch wrote: >>> It seems as if claws mail is preferring SSLv3 >> >> And since dovecot is really not affected by the poodle vulnerability, if >> you can't upgrade (I believe 2.2 is in the backports repo?), probably >> easiest to just reenable SSLv3... > > > Hi thanks Charles and thanks to all for your help. > We decided to reenable SSLv3. > We'll upgrade Dovecot when Debian will officially dist upgrade to > version 8 :) update https://bugzilla.redhat.com/show_bug.cgi?id=1153970 http://git.claws-mail.org/?p=claws.git;a=commit;h=c6dc3e229f361f11ab4920d84bb11b5821bc4e86 http://git.claws-mail.org/?p=claws.git;a=patch;h=c6dc3e229f361f11ab4920d84bb11b5821bc4e86 >From c6dc3e229f361f11ab4920d84bb11b5821bc4e86 Mon Sep 17 00:00:00 2001 From: Colin Leroy Date: Thu, 16 Oct 2014 14:35:46 +0200 Subject: [PATCH] Disable SSL3.0 entirely as a Poodle fix. --- src/common/ssl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/common/ssl.c b/src/common/ssl.c index f612299..569c808 100644 --- a/src/common/ssl.c +++ b/src/common/ssl.c @@ -323,7 +323,7 @@ gboolean ssl_init_socket(SockInfo *sockinfo) sockinfo->gnutls_priority, r); } else { - gnutls_priority_set_direct(session, "NORMAL", NULL); + gnutls_priority_set_direct(session, "NORMAL:-VERS-SSL3.0", NULL); } gnutls_record_disable_padding(session); -- 1.7.10.4 Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From jyri.hovila at turvamies.fi Sat Jan 10 00:07:17 2015 From: jyri.hovila at turvamies.fi (Jyri Hovila [Turvamies.fi]) Date: Sat, 10 Jan 2015 02:07:17 +0200 Subject: Moving or "upgrading" from MD5 to SSHA512 Message-ID: <67761249.20150110020717@turvamies.fi> Hello, world! I have a long-running Dovecot & Postfix installation using PostgreSQL back-end. Until now I've been using MD5 hashing but would like to "upgrade" to the salted SSHA512. Is there a way to configure Dovecot so that it would automatically detect the type of the hash stored in the database, so that users who have changed their password (and thus being hashed with SHA512) and users still having an MD5 hash would both be able to authenticate at the same time? Yours, Jyri -- +358-50-5632104 (24/7) +358-46-8822157 (backup) jyri.hovila at turvamies.fi From jtam.home at gmail.com Sat Jan 10 00:11:35 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 9 Jan 2015 16:11:35 -0800 (PST) Subject: dovecot on wheezy, best ssl configuration ? In-Reply-To: References: Message-ID: ml at ruggedinbox.com writes: > Our smtp server is postfix, can you please suggest a better > 'ssl_protocols' and 'ssl_cipher_list' configuration ? > We are running Debian 7 Wheezy A useful command to know is "openssl ciphers" run on the server that will tell you the ciphers available given a protocol and cipher list spec. If it comes out to empty, your client won't be able to negotiate any SSL sessions, and you'll have include more ciphers. For example, TLSv1 protocol minus any low-grade encryption or SSLv2 ciphers: $ openssl ciphers -tlsv1 'ALL:\!LOW:\!SSLv2' ADH-AES256-SHA:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:AES256-SHA:ADH-AES128-SHA:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:AES128-SHA:ADH-DES-CBC3-SHA:EXP-ADH-DES-CBC-SHA:ADH-RC4-MD5:EXP-ADH-RC4-MD5:EDH-RSA-DES-CBC3-SHA:EXP-EDH-RSA-DES-CBC-SHA:EDH-DSS-DES-CBC3-SHA:EXP-EDH-DSS-DES-CBC-SHA:DES-CBC3-SHA:EXP-DES-CBC-SHA:IDEA-CBC-SHA:EXP-RC2-CBC-MD5:RC4-SHA:RC4-MD5:EXP-RC4-MD5 Joseph Tam From duihi77 at gmail.com Sat Jan 10 06:17:22 2015 From: duihi77 at gmail.com (Duane Hill) Date: Sat, 10 Jan 2015 00:17:22 -0600 Subject: pop3 seen In-Reply-To: <54B02B43.3070805@ols.es> References: <54B02B43.3070805@ols.es> Message-ID: <1919601476.20150110001722@gmail.com> On Friday, January 9, 2015, 1:25:55 PM, David wrote: > Hi > Is there a way to know if a message has been dowloaded via pop3 > and not seen via imap ? I usually see such activity through monitoring of the Dovecot logs. -- Duane Hill duihi77 at gmail.com "If at first you don't succeed, so much for sky diving." From eslist at ols.es Sat Jan 10 07:25:06 2015 From: eslist at ols.es (David Saez) Date: Sat, 10 Jan 2015 08:25:06 +0100 Subject: pop3 seen In-Reply-To: <1919601476.20150110001722@gmail.com> References: <54B02B43.3070805@ols.es> <1919601476.20150110001722@gmail.com> Message-ID: <54B0D3D2.8090105@ols.es> Hi I will prefer some way to flag the messages when they are seen by pop and/or imap > On Friday, January 9, 2015, 1:25:55 PM, David wrote: > >> Hi > >> Is there a way to know if a message has been dowloaded via pop3 >> and not seen via imap ? > > I usually see such activity through monitoring of the Dovecot logs. > From list_dovecot at bluerosetech.com Sat Jan 10 07:35:37 2015 From: list_dovecot at bluerosetech.com (Darren Pilgrim) Date: Fri, 09 Jan 2015 23:35:37 -0800 Subject: Moving or "upgrading" from MD5 to SSHA512 In-Reply-To: <67761249.20150110020717@turvamies.fi> References: <67761249.20150110020717@turvamies.fi> Message-ID: <54B0D649.4010104@bluerosetech.com> On 1/9/2015 4:07 PM, Jyri Hovila [Turvamies.fi] wrote: > Hello, world! > > I have a long-running Dovecot & Postfix installation using PostgreSQL > back-end. > > Until now I've been using MD5 hashing but would like to "upgrade" to > the salted SSHA512. > > Is there a way to configure Dovecot so that it would automatically > detect the type of the hash stored in the database, so that users who > have changed their password (and thus being hashed with SHA512) and > users still having an MD5 hash would both be able to authenticate at > the same time? Have a read of this: http://wiki2.dovecot.org/HowTo/ConvertPasswordSchemes From jyri.hovila at turvamies.fi Sat Jan 10 10:45:25 2015 From: jyri.hovila at turvamies.fi (Jyri Hovila [Turvamies.fi]) Date: Sat, 10 Jan 2015 12:45:25 +0200 Subject: Moving or "upgrading" from MD5 to SSHA512 In-Reply-To: <54B0D649.4010104@bluerosetech.com> References: <67761249.20150110020717@turvamies.fi> <54B0D649.4010104@bluerosetech.com> Message-ID: <906443151.20150110124525@turvamies.fi> Hi! > Have a read of this: > http://wiki2.dovecot.org/HowTo/ConvertPasswordSchemes In fact no. Thanks! - Jyri -- +358-50-5632104 (24/7) +358-46-8822157 (backup) jyri.hovila at turvamies.fi From juan at inti.gob.ar Sat Jan 10 16:03:35 2015 From: juan at inti.gob.ar (Juan Bernhard) Date: Sat, 10 Jan 2015 13:03:35 -0300 Subject: pop3 seen In-Reply-To: <54B0D3D2.8090105@ols.es> References: <54B02B43.3070805@ols.es> <1919601476.20150110001722@gmail.com> <54B0D3D2.8090105@ols.es> Message-ID: <54B14D57.3060504@inti.gob.ar> El 10/01/2015 a las 04:25 a.m., David Saez escibi?: > Hi > > I will prefer some way to flag the messages when they are seen > by pop and/or imap > >> On Friday, January 9, 2015, 1:25:55 PM, David wrote: >> >>> Hi >> >>> Is there a way to know if a message has been dowloaded via pop3 >>> and not seen via imap ? >> >> I usually see such activity through monitoring of the Dovecot logs. >> Hi! You should enable the command logs, there is a plug in to enable that, mail_log plugin i think, check the file conf.d/10-logging.conf. The pop3 command to download a mail is RETR , in theory you could map the uid number to the mail file, using the dovecot-uidlist file for each mailbox. I let you the job to find out wich file is every mail ;) I really don't know that (some log, I guess). If you want to see on imap too, you have to enable the flag change state to see when a mail is marked as read. This is not complete, but it will give you a start point to search info. Saludos! From lista at xdrv.co.uk Sat Jan 10 17:16:37 2015 From: lista at xdrv.co.uk (James) Date: Sat, 10 Jan 2015 17:16:37 +0000 Subject: Solaris 10 make check core dumps In-Reply-To: References: Message-ID: <54B15E75.1090804@xdrv.co.uk> On 08/01/2015 00:25, Brian Eliassen wrote: > I'm doing a standard build on Solaris 10 with Dovecot 2.2.15 without > anything fancy: > > ./configure > make > make check > make install > > Configure output looks like this: > *** > Install prefix . : /usr/local Oooh, I suggest you use /opt ./configure \ --prefix=${PREFIX} \ --sysconfdir=/etc/opt/${PREFIX} \ --localstatedir=/var/opt/${PREFIX} \ ... $ man -s 5 filesystem $ mdb test-http-auth core Loading modules: [ libc.so.1 ld.so.1 ] > ::stack libc.so.1`strlen+0x50(29487, ffbffb00, ffbff43d, 0, 0, 0) libc.so.1`vsnprintf+0x70(4f2c0, 10f, 29478, ffbffaf8, 10, ff1c7078) t_noalloc_strdup_vprintf+0x3c(4f2c0, ffbffaf8, ffbffa2c, 1, 19, 0) p_strdup_vprintf+0xc(4cb90, 29478, ffbffaf8, ff1c759c, 4cc38, 4cb8c) t_strdup_printf+0x38(29478, 0, 0, 61fefeff, 80808080, 1010101) Spot the zeros. It's doing the equivalent of 'vsnprintf(s, "%s", 0);'. The check will run by changing the test code and not attempting to print nulls, patch file attached. I've not thought about this deeply and this is no more than a hack to get the test to run, I didn't study the result. On face value the core dump says dovecot has failed the test because its string functions don't take nulls, that is if dovecot ever sends them nulls which I can't say other than dovecot does run without dumping core. If there is the possibility nulls need to be printed the dovecot functions in src/lib/strfuncs.c need fixing. -------------- next part -------------- --- ../original/src/lib-http/test-http-auth.c 2014-09-24 20:34:27.000000000 +0100 +++ src/lib-http/test-http-auth.c 2015-01-10 15:22:59.083365723 +0000 @@ -120,7 +120,7 @@ strcmp(chalo->scheme, chalt->scheme) == 0); if (chalo->data == NULL || chalt->data == NULL) { test_out(t_strdup_printf("[%d]->data = %s", - index, str_sanitize(chalo->data, 80)), + index, str_sanitize(chalo->data == NULL ? "NULL" : chalo->data, 80)), chalo->data == chalt->data); } else { test_out(t_strdup_printf("[%d]->data = %s", @@ -230,7 +230,7 @@ strcmp(out.scheme, test->scheme) == 0); if (out.data == NULL || test->data == NULL) { test_out(t_strdup_printf("->data = %s", - str_sanitize(out.data, 80)), + str_sanitize(out.data == NULL ? "NULL" : out.data, 80)), out.data == test->data); } else { test_out(t_strdup_printf("->data = %s", --- ../original/src/lib-http/test-http-request-parser.c 2014-06-02 12:50:10.000000000 +0100 +++ src/lib-http/test-http-request-parser.c 2015-01-10 15:25:52.198118307 +0000 @@ -269,7 +269,7 @@ if (payload == NULL || test->payload == NULL) { test_out(t_strdup_printf("request->payload = %s", - str_sanitize(payload, 80)), + str_sanitize(payload == NULL ? "NULL" : payload, 80)), payload == test->payload); } else { test_out(t_strdup_printf("request->payload = %s", --- ../original/src/lib-imap/test-imap-url.c 2014-07-03 10:44:49.000000000 +0100 +++ src/lib-imap/test-imap-url.c 2015-01-10 15:57:26.923109229 +0000 @@ -637,14 +637,14 @@ strcmp(urlp->host_name, urlt->host_name) == 0); } if (urlp->userid == NULL || urlt->userid == NULL) { - test_out_quiet(t_strdup_printf("url->userid = %s", urlp->userid), + test_out_quiet(t_strdup_printf("url->userid = %s", urlp->userid == NULL ? "NULL" : urlp->userid), urlp->userid == urlt->userid); } else { test_out_quiet(t_strdup_printf("url->userid = %s", urlp->userid), strcmp(urlp->userid, urlt->userid) == 0); } if (urlp->auth_type == NULL || urlt->auth_type == NULL) { - test_out_quiet(t_strdup_printf("url->auth_type = %s", urlp->auth_type), + test_out_quiet(t_strdup_printf("url->auth_type = %s", urlp->auth_type == NULL ? "NULL" : urlp->auth_type), urlp->auth_type == urlt->auth_type); } else { test_out_quiet(t_strdup_printf("url->auth_type = %s", urlp->auth_type), @@ -665,7 +665,7 @@ urlp->have_host_ip == urlt->have_host_ip); } if (urlp->mailbox == NULL || urlt->mailbox == NULL) { - test_out_quiet(t_strdup_printf("url->mailbox = %s", urlp->mailbox), + test_out_quiet(t_strdup_printf("url->mailbox = %s", urlp->mailbox == NULL ? "NULL" : urlp->mailbox), urlp->mailbox == urlt->mailbox); } else { test_out_quiet(t_strdup_printf("url->mailbox = %s", urlp->mailbox), @@ -676,7 +676,7 @@ test_out_quiet(t_strdup_printf("url->uid = %u", urlp->uid), urlp->uid == urlt->uid); if (urlp->section == NULL || urlt->section == NULL) { - test_out_quiet(t_strdup_printf("url->section = %s", urlp->section), + test_out_quiet(t_strdup_printf("url->section = %s", urlp->section == NULL ? "NULL" : urlp->section), urlp->section == urlt->section); } else { test_out_quiet(t_strdup_printf("url->section = %s", urlp->section), @@ -688,7 +688,7 @@ urlp->partial_size == urlt->partial_size); if (urlp->search_program == NULL || urlt->search_program == NULL) { test_out_quiet(t_strdup_printf( - "url->search_program = %s", urlp->search_program), + "url->search_program = %s", urlp->search_program == NULL ? "NULL" : urlp->search_program), urlp->search_program == urlt->search_program); } else { test_out_quiet(t_strdup_printf( @@ -718,7 +718,7 @@ if (urlp->uauth_access_user == NULL || urlt->uauth_access_user == NULL) { test_out_quiet(t_strdup_printf("url->uauth_access_user = %s", - urlp->uauth_access_user), + urlp->uauth_access_user == NULL ? "NULL" : urlp->uauth_access_user), urlp->uauth_access_user == urlt->uauth_access_user); } else { test_out_quiet(t_strdup_printf("url->uauth_access_user = %s", @@ -728,7 +728,7 @@ } if (urlp->uauth_mechanism == NULL || urlt->uauth_mechanism == NULL) { test_out_quiet(t_strdup_printf( - "url->uauth_mechanism = %s", urlp->uauth_mechanism), + "url->uauth_mechanism = %s", urlp->uauth_mechanism == NULL ? "NULL" : urlp->uauth_mechanism), urlp->uauth_mechanism == urlt->uauth_mechanism); } else { test_out_quiet(t_strdup_printf( @@ -737,7 +737,7 @@ } if (urlp->uauth_token == NULL || urlt->uauth_token == NULL) { test_out_quiet(t_strdup_printf( - "url->uauth_token = %s", urlp->uauth_token), + "url->uauth_token = %s", urlp->uauth_token == NULL ? (unsigned char *) "NULL" : urlp->uauth_token), urlp->uauth_token == urlt->uauth_token); } else { bool equal = urlp->uauth_token_size == urlt->uauth_token_size; From dave.mehler at gmail.com Sat Jan 10 18:34:49 2015 From: dave.mehler at gmail.com (David Mehler) Date: Sat, 10 Jan 2015 13:34:49 -0500 Subject: Dovecot on Fedora 20 or 21 Message-ID: Hello, Is anyone running Dovecot on either a Fedora 20 or 21 system? I'm having an issue, on a system reboot, which I admit does not happen often, Dovecot fails to start in the systemctl list, output is status failed. The issue seems to be Dovecot can not bind to the ipv6 address. Now later if I manually log in to the box and start dovecot it works just fine no problems. I've googled and seen similar issues to this. Does anyone have a solution? Thanks. Dave. From fuckspam at wheres5.com Sat Jan 10 18:37:19 2015 From: fuckspam at wheres5.com (Hoggins!) Date: Sat, 10 Jan 2015 19:37:19 +0100 Subject: Dovecot on Fedora 20 or 21 In-Reply-To: References: Message-ID: <54B1715F.3060809@wheres5.com> Hello Dave, Are you using NetworkManager or /etc/init.d/network to set up your network interfaces ? Le 10/01/2015 19:34, David Mehler a ?crit : > Hello, > > Is anyone running Dovecot on either a Fedora 20 or 21 system? I'm > having an issue, on a system reboot, which I admit does not happen > often, Dovecot fails to start in the systemctl list, output is status > failed. The issue seems to be Dovecot can not bind to the ipv6 > address. Now later if I manually log in to the box and start dovecot > it works just fine no problems. I've googled and seen similar issues > to this. Does anyone have a solution? > > Thanks. > Dave. > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mine at michi.su Sat Jan 10 18:40:51 2015 From: mine at michi.su (Michael) Date: Sat, 10 Jan 2015 19:40:51 +0100 Subject: Dovecot on Fedora 20 or 21 In-Reply-To: References: Message-ID: <54B17233.6070702@michi.su> On 10.01.2015 19:34, David Mehler wrote: > Hello, > > Is anyone running Dovecot on either a Fedora 20 or 21 system? I'm > having an issue, on a system reboot, which I admit does not happen > often, Dovecot fails to start in the systemctl list, output is status > failed. The issue seems to be Dovecot can not bind to the ipv6 > address. Now later if I manually log in to the box and start dovecot > it works just fine no problems. I've googled and seen similar issues > to this. Does anyone have a solution? Sounds like Dovecot just starts too early. Before the v6 interface is ready. You should be able to either define dependencies or delays in systemctl, right? Michael From h.reindl at thelounge.net Sat Jan 10 18:54:00 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 10 Jan 2015 19:54:00 +0100 Subject: Dovecot on Fedora 20 or 21 In-Reply-To: References: Message-ID: <54B17548.1000606@thelounge.net> Am 10.01.2015 um 19:34 schrieb David Mehler: > Is anyone running Dovecot on either a Fedora 20 or 21 system? surely > having an issue, on a system reboot, which I admit does not happen > often, Dovecot fails to start in the systemctl list, output is status > failed. The issue seems to be Dovecot can not bind to the ipv6 > address. Now later if I manually log in to the box and start dovecot > it works just fine no problems. I've googled and seen similar issues > to this. Does anyone have a solution? you could avoid that by have it loisten on the global IP (0.0.0.0 for ipv4 and ::: for ipv6) instead configure a specific IP for the service, otherwise you need to make sure ordering with After= or Before= in the systemd-units [root at srv-rhsoft:~]$ cat /usr/lib/systemd/system/dovecot.service [Unit] Description=Dovecot IMAP/POP3 Proxy After=network.service systemd-networkd.service network-online.target dbmail-imapd.service dbmail-pop3d.service mysqld.service mysqld-dbmail.service Before=postfix.service [Service] Type=simple ExecStart=/usr/sbin/dovecot -F Environment="LANG=en_GB.UTF-8" NonBlocking=yes Restart=always RestartSec=1 PrivateTmp=true CapabilityBoundingSet=CAP_CHOWN CAP_DAC_OVERRIDE CAP_IPC_LOCK CAP_KILL CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID CAP_SYS_CHROOT ReadOnlyDirectories=/etc ReadOnlyDirectories=/usr ReadOnlyDirectories=/proc ReadOnlyDirectories=/sys ReadOnlyDirectories=/var/lib ReadWriteDirectories=/var/lib/dovecot InaccessibleDirectories=/boot InaccessibleDirectories=/home InaccessibleDirectories=/media InaccessibleDirectories=/root InaccessibleDirectories=/run/console InaccessibleDirectories=/run/dbus InaccessibleDirectories=/run/lock InaccessibleDirectories=/run/mount InaccessibleDirectories=/run/systemd/generator InaccessibleDirectories=/run/systemd/system InaccessibleDirectories=/run/systemd/users InaccessibleDirectories=/run/udev InaccessibleDirectories=/run/user InaccessibleDirectories=/usr/local/scripts InaccessibleDirectories=/var/lib/rpm InaccessibleDirectories=/var/lib/yum InaccessibleDirectories=/var/lib/dbus InaccessibleDirectories=/var/lib/systemd [Install] WantedBy=multi-user.target -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From lista at xdrv.co.uk Mon Jan 12 11:10:20 2015 From: lista at xdrv.co.uk (James) Date: Mon, 12 Jan 2015 11:10:20 +0000 Subject: Solaris 10 make check core dumps In-Reply-To: <54B15E75.1090804@xdrv.co.uk> References: <54B15E75.1090804@xdrv.co.uk> Message-ID: <54B3AB9C.2080204@xdrv.co.uk> On 10/01/2015 17:16, James wrote: > On 08/01/2015 00:25, Brian Eliassen wrote: >> Install prefix . : /usr/local > > Oooh, I suggest you use /opt > > ./configure \ > --prefix=${PREFIX} \ > --sysconfdir=/etc/opt/${PREFIX} \ > --localstatedir=/var/opt/${PREFIX} \ > ... > > $ man -s 5 filesystem Ha! I changed my concept of "prefix" as I wrote. My apologies for needing a second attempt. I also divide up /var/ and /etc by project. PREFIX=/opt/abcd ./configure \ --prefix=${PREFIX} \ --sysconfdir=/etc${PREFIX}/dovecot \ --localstatedir=/var${PREFIX}/dovecot \ ... The other way I think of prefix is more generally, if "abcd" is the chosen identifier the file prefix is "/opt/abcd" or "/opt/${prefix}". This identifier is used as a prefix in several places like the SVR4 package name and service name. From petar at smokva.net Mon Jan 12 12:05:38 2015 From: petar at smokva.net (Petar Bogdanovic) Date: Mon, 12 Jan 2015 13:05:38 +0100 Subject: pigeonhole: upgrade 0.4.3 -> 0.4.6 broke script storage Message-ID: <20150112120538.GA17521@pintail.smokva.net> Hi, I ran into some troubles while upgrading pigeonhole: crane dovecot: lda(petar): Error: sieve: Failed to access user storage (temporary failure) crane dovecot: lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed After reading through the changelog, I noticed that there have been some changes in the script storage location syntax but that old prefix-less paths should still work fine. My lda-configuration is very simple: # lda protocol lda { mail_plugins = sieve postmaster_address = } plugin { sieve = /data/dovecot/user/%u.sieve sieve_extensions = +editheader sieve_max_redirects = 50 } And even if I change $sieve to: sieve = file:/data/dovecot/user/%u.sieve the error messages disappear but based on the sudden unfiltered wave of mails in my INBOX that I usually store somewhere else, I guess the path still doesn't work. Any ideas? Thanks, Petar Bogdanovic -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From tlx at leuxner.net Mon Jan 12 12:15:47 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 12 Jan 2015 13:15:47 +0100 Subject: HG e3640ccaa76d crashes upon IMAP SEARCH Message-ID: <20150112121547.GA38901@nihlus.leuxner.net> $ doveconf -n | head # 2.2.15 (e3640ccaa76d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 # OS: Linux 3.16.0-0.bpo.4-amd64 x86_64 Debian 7.8 $ /usr/lib/dovecot/imap -u tlx at leuxner.net * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL RIGHTS=texk] Logged in as tlx at leuxner.net a select "Shops/Amazon" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk NotJunk) * OK [PERMANENTFLAGS (\Answered \Flagged \Seen \Draft $NotJunk NotJunk \*)] Flags permitted. * 491 EXISTS * 0 RECENT * OK [UNSEEN 491] First unseen. * OK [UIDVALIDITY 1213986801] UIDs valid * OK [UIDNEXT 568] Predicted next UID * OK [HIGHESTMODSEQ 1029] Highest a OK [READ-WRITE] Select completed (0.003 secs). b search charset utf-8 body "Sleeve" imap(tlx at leuxner.net): Panic: file message-decoder.c: line 346 (message_decode_body): assertion failed: (ctx->translation_size <= sizeof(ctx->translation_buf)) imap(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x7749f) [0x7ffe68e6d49f] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7ffe68e6d57a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7ffe68e1df65] -> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x688) [0x7ffe68e5c858] -> /usr/lib/dovecot/libdovecot.so.0(message_search_more+0xa1) [0x7ffe68e60881] -> /usr/lib/dovecot/libdovecot.so.0(message_search_msg+0x5b) [0x7ffe68e60a4b] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa9b7b) [0x7ffe69187b7b] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_search_args_foreach+0x45) [0x7ffe6915d675] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xa951a) [0x7ffe6918751a] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xab76a) [0x7ffe6918976a] -> /usr/lib/dovecot/libdovecot-storage.so.0(index_storage_search_next_nonblock+0xe2) [0x7ffe69189e32] -> /usr/lib/dovecot/modules/lib90_stats_plugin.so(+0x2daa) [0x7ffe66dc3daa] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x27) [0x7ffe691627a7] -> /usr/lib/dovecot/imap() [0x41da2f] -> /usr/lib/dovecot/imap() [0x41e0a2] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xaa) [0x7ffe68e7f6ca] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run_internal+0x69) [0x7ffe68e804e9] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9) [0x7ffe68e7f549] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7ffe68e7f5c8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7ffe68e23433] -> /usr/lib/dovecot/imap(main+0x2a1) [0x40c591] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7ffe68a88ead] -> /usr/lib/dovecot/imap() [0x40c6f5] I tried to make it dump a core but failed... Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From stephan at rename-it.nl Mon Jan 12 12:19:18 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 12 Jan 2015 13:19:18 +0100 Subject: pigeonhole: upgrade 0.4.3 -> 0.4.6 broke script storage In-Reply-To: <20150112120538.GA17521@pintail.smokva.net> References: <20150112120538.GA17521@pintail.smokva.net> Message-ID: <54B3BBC6.7060409@rename-it.nl> Petar Bogdanovic schreef op 12-1-2015 om 13:05: > Hi, > > I ran into some troubles while upgrading pigeonhole: > > crane dovecot: lda(petar): Error: sieve: Failed to access user storage (temporary failure) > crane dovecot: lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed > > After reading through the changelog, I noticed that there have been some > changes in the script storage location syntax but that old prefix-less > paths should still work fine. > > My lda-configuration is very simple: > > # lda > protocol lda { > mail_plugins = sieve > postmaster_address = > } > plugin { > sieve = /data/dovecot/user/%u.sieve > sieve_extensions = +editheader > sieve_max_redirects = 50 > } > > And even if I change $sieve to: > > sieve = file:/data/dovecot/user/%u.sieve > > the error messages disappear but based on the sudden unfiltered wave of > mails in my INBOX that I usually store somewhere else, I guess the path > still doesn't work. You can enable mail_debug=yes to see where it is looking for the Sieve scripts. Regards, Stephan. From josh at ascendant.com.au Mon Jan 12 12:29:34 2015 From: josh at ascendant.com.au (Joshua Atkins) Date: Mon, 12 Jan 2015 23:29:34 +1100 Subject: Initial development work on fts-elasticsearch Message-ID: Hi all, I have started work on an ElasticSearch FTS plugin for Dovecot. It's available now on GitHub at: https://github.com/ascendantcom/fts-elasticsearch Please note that it is still in heavy development right now. It supports indexing, searching and expunging with some caveats (no multi-mailbox search or attachments). Pull requests, suggestions and bug reports very welcome! Cheers, Josh From redflag237 at googlemail.com Mon Jan 12 12:29:29 2015 From: redflag237 at googlemail.com (Jonas Plitt) Date: Mon, 12 Jan 2015 13:29:29 +0100 Subject: Dovecot replication over TCP/SSL, certificate error Message-ID: Hi All, I'm running TCP-based dsync replication on two dovecot nodes. Nowdays i tried to enable SSL (TCPS). I changed mail_replica prefix from tcp:* to tcps:* and added ssl=yes to the inet_listener. Then on running *doveadm sync* i'm getting the following message: " *doveadm(example at example.com ): Error: Couldn't initialize SSL context: Can't load CA certs from directory /etc/ssl/certs: error:02001024:system library:fopen:File name too longdoveadm: Error: Failed to iterate through some users*" this is my config (part): *ssl_cert = References: Message-ID: <54B3BF6D.9010706@thelounge.net> Am 12.01.2015 um 13:29 schrieb Jonas Plitt: > *doveadm(example at example.com ): Error: Couldn't > initialize SSL context: Can't load CA certs from directory /etc/ssl/certs: > error:02001024:system library:fopen:File name too longdoveadm: Error: > Failed to iterate through some users*" > > this is my config (part): > > *ssl_cert = /etc/ssl/certsssl_client_ca_file = > > The file startcom-ba-bundle contains the complete chain. The file > startcom-ca contains only the ca certificate. Can anybody help, please? did you read the "File name too long"? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From redflag237 at googlemail.com Mon Jan 12 12:51:12 2015 From: redflag237 at googlemail.com (Jonas Plitt) Date: Mon, 12 Jan 2015 13:51:12 +0100 Subject: Dovecot replication over TCP/SSL, certificate error In-Reply-To: <54B3BF6D.9010706@thelounge.net> References: <54B3BF6D.9010706@thelounge.net> Message-ID: Of cource i did. Even this one does not work: *ssl_cert = : > > > Am 12.01.2015 um 13:29 schrieb Jonas Plitt: > >> *doveadm(example at example.com ): Error: Couldn't >> initialize SSL context: Can't load CA certs from directory /etc/ssl/certs: >> error:02001024:system library:fopen:File name too longdoveadm: Error: >> Failed to iterate through some users*" >> >> this is my config (part): >> >> *ssl_cert = > > > /etc/ssl/certsssl_client_ca_file = >> > >> The file startcom-ba-bundle contains the complete chain. The file >> startcom-ca contains only the ca certificate. Can anybody help, please? >> > > did you read the "File name too long"? > > -- Mit freundlichen Gr??en, with kind regards, Jonas Plitt From petar at smokva.net Mon Jan 12 13:19:26 2015 From: petar at smokva.net (Petar Bogdanovic) Date: Mon, 12 Jan 2015 14:19:26 +0100 Subject: pigeonhole: upgrade 0.4.3 -> 0.4.6 broke script storage In-Reply-To: <54B3BBC6.7060409@rename-it.nl> References: <20150112120538.GA17521@pintail.smokva.net> <54B3BBC6.7060409@rename-it.nl> Message-ID: <20150112131926.GA17729@pintail.smokva.net> On Mon, Jan 12, 2015 at 01:19:18PM +0100, Stephan Bosch wrote: > > Petar Bogdanovic schreef op 12-1-2015 om 13:05: > >Hi, > > > >I ran into some troubles while upgrading pigeonhole: > > > > crane dovecot: lda(petar): Error: sieve: Failed to access user storage (temporary failure) > > crane dovecot: lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed > > > >After reading through the changelog, I noticed that there have been some > >changes in the script storage location syntax but that old prefix-less > >paths should still work fine. > > > >My lda-configuration is very simple: > > > > # lda > > protocol lda { > > mail_plugins = sieve > > postmaster_address = > > } > > plugin { > > sieve = /data/dovecot/user/%u.sieve > > sieve_extensions = +editheader > > sieve_max_redirects = 50 > > } > > > >And even if I change $sieve to: > > > > sieve = file:/data/dovecot/user/%u.sieve > > > >the error messages disappear but based on the sudden unfiltered wave of > >mails in my INBOX that I usually store somewhere else, I guess the path > >still doesn't work. > > You can enable mail_debug=yes to see where it is looking for the Sieve > scripts. Thanks. The following output is with: sieve = /data/dovecot/user/%u.sieve master: Dovecot v2.2.15 starting up for imap lda: Debug: Loading modules from directory: /usr/pkg/lib/dovecot lda: Debug: Module loaded: /usr/pkg/lib/dovecot/lib90_sieve_plugin.so lda(petar): Debug: Effective uid=2000, gid=2000, home=/etc/skel lda(petar): Debug: maildir++: root=/data/dovecot/user/petar, index=, indexpvt=, control=, inbox=/data/dovecot/user/petar, alt= lda(petar): Debug: userdb lookup skipped, username taken from USER environment lda(petar): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= lda(petar): Debug: Destination address: petar@ (source: user at hostname) lda(petar): Debug: sieve: Pigeonhole version 0.4.6 (3e924b1b6c5c+) initializing lda(petar): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. lda(petar): Debug: sieve: file storage: Performing auto-detection lda(petar): Debug: sieve: file storage: access(/etc/skel, rwx) failed: Permission denied lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed lda(petar): Error: sieve: Failed to access user storage (temporary failure) The inbox-path seems found but the sieve path isn't mentioned. ($HOME is /etc/skel and its read-only.) The following output is with: sieve = file:/data/dovecot/user/%u.sieve master: Dovecot v2.2.15 starting up for imap lda: Debug: Loading modules from directory: /usr/pkg/lib/dovecot lda: Debug: Module loaded: /usr/pkg/lib/dovecot/lib90_sieve_plugin.so lda(petar): Debug: Effective uid=2000, gid=2000, home=/etc/skel lda(petar): Debug: maildir++: root=/data/dovecot/user/petar, index=, indexpvt=, control=, inbox=/data/dovecot/user/petar, alt= lda(petar): Debug: userdb lookup skipped, username taken from USER environment lda(petar): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= lda(petar): Debug: Destination address: petar@ (source: user at hostname) lda(petar): Debug: sieve: Pigeonhole version 0.4.6 (3e924b1b6c5c+) initializing lda(petar): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. lda(petar): Debug: sieve: file storage: Active script path is unconfigured; using default (path=~/.dovecot.sieve) lda(petar): Debug: sieve: file storage: Using script storage path: /data/dovecot/user/petar.sieve lda(petar): Debug: sieve: file storage: Using active Sieve script path: /etc/skel/.dovecot.sieve lda(petar): Debug: sieve: file storage: Relative path to sieve storage in active link: /data/dovecot/user/petar.sieve/ lda(petar): Debug: sieve: file storage: Using script storage path: /etc/skel/.dovecot.sieve lda(petar): Debug: sieve: file script: File `/etc/skel/.dovecot.sieve' not found lda(petar): Debug: sieve: No active Sieve script exists in user storage `/data/dovecot/user/petar.sieve' (trying default script location instead) lda(petar): Debug: sieve: No default script configured for user lda(petar): Debug: sieve: User has no personal script lda(petar): Debug: sieve: No scripts to execute: reverting to default delivery. # ls -la /data/dovecot/user/petar.* -rw------- 1 petar petar 32946 /data/dovecot/user/petar.sieve -rw------- 1 petar petar 532 /data/dovecot/user/petar.sieve.log -rw------- 1 petar petar 31571 /data/dovecot/user/petar.svbin Shouldn't both variants at least fail identically if no prefix implies `file:'? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: not available URL: From wgrcunha at gmail.com Mon Jan 12 20:29:42 2015 From: wgrcunha at gmail.com (Francisco Wagner C. Freire) Date: Mon, 12 Jan 2015 18:29:42 -0200 Subject: PROXY protocol In-Reply-To: <54B01495.6040300@wheres5.com> References: <54B01495.6040300@wheres5.com> Message-ID: +1 On Fri, Jan 9, 2015 at 3:49 PM, Hoggins! wrote: > Hello folks, > > Any plans on implementing the PROXY protocol to allow Dovecot being > behind a TCP proxy, and still logging the real IP address of the users ? > See : http://blog.haproxy.com/haproxy/proxy-protocol/ > > Thanks ! > > From stephan at rename-it.nl Mon Jan 12 21:28:35 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 12 Jan 2015 22:28:35 +0100 Subject: pigeonhole: upgrade 0.4.3 -> 0.4.6 broke script storage In-Reply-To: <20150112131926.GA17729@pintail.smokva.net> References: <20150112120538.GA17521@pintail.smokva.net> <54B3BBC6.7060409@rename-it.nl> <20150112131926.GA17729@pintail.smokva.net> Message-ID: <54B43C83.40509@rename-it.nl> On 1/12/2015 2:19 PM, Petar Bogdanovic wrote: > On Mon, Jan 12, 2015 at 01:19:18PM +0100, Stephan Bosch wrote: >> Petar Bogdanovic schreef op 12-1-2015 om 13:05: >>> Hi, >>> >>> I ran into some troubles while upgrading pigeonhole: >>> >>> crane dovecot: lda(petar): Error: sieve: Failed to access user storage (temporary failure) >>> crane dovecot: lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed >>> >>> After reading through the changelog, I noticed that there have been some >>> changes in the script storage location syntax but that old prefix-less >>> paths should still work fine. >>> >>> My lda-configuration is very simple: >>> >>> # lda >>> protocol lda { >>> mail_plugins = sieve >>> postmaster_address = >>> } >>> plugin { >>> sieve = /data/dovecot/user/%u.sieve >>> sieve_extensions = +editheader >>> sieve_max_redirects = 50 >>> } >>> >>> And even if I change $sieve to: >>> >>> sieve = file:/data/dovecot/user/%u.sieve >>> >>> the error messages disappear but based on the sudden unfiltered wave of >>> mails in my INBOX that I usually store somewhere else, I guess the path >>> still doesn't work. >> You can enable mail_debug=yes to see where it is looking for the Sieve >> scripts. > Thanks. The following output is with: > > sieve = /data/dovecot/user/%u.sieve > > master: Dovecot v2.2.15 starting up for imap > lda: Debug: Loading modules from directory: /usr/pkg/lib/dovecot > lda: Debug: Module loaded: /usr/pkg/lib/dovecot/lib90_sieve_plugin.so > lda(petar): Debug: Effective uid=2000, gid=2000, home=/etc/skel > lda(petar): Debug: maildir++: root=/data/dovecot/user/petar, index=, indexpvt=, control=, inbox=/data/dovecot/user/petar, alt= > lda(petar): Debug: userdb lookup skipped, username taken from USER environment > lda(petar): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= > lda(petar): Debug: Destination address: petar@ (source: user at hostname) > lda(petar): Debug: sieve: Pigeonhole version 0.4.6 (3e924b1b6c5c+) initializing > lda(petar): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. > lda(petar): Debug: sieve: file storage: Performing auto-detection > lda(petar): Debug: sieve: file storage: access(/etc/skel, rwx) failed: Permission denied > lda(petar): Error: sieve: file storage: Could not find storage root directory; path was left unconfigured and autodetection failed > lda(petar): Error: sieve: Failed to access user storage (temporary failure) > > The inbox-path seems found but the sieve path isn't mentioned. > ($HOME is /etc/skel and its read-only.) > > The following output is with: > > sieve = file:/data/dovecot/user/%u.sieve > > master: Dovecot v2.2.15 starting up for imap > lda: Debug: Loading modules from directory: /usr/pkg/lib/dovecot > lda: Debug: Module loaded: /usr/pkg/lib/dovecot/lib90_sieve_plugin.so > lda(petar): Debug: Effective uid=2000, gid=2000, home=/etc/skel > lda(petar): Debug: maildir++: root=/data/dovecot/user/petar, index=, indexpvt=, control=, inbox=/data/dovecot/user/petar, alt= > lda(petar): Debug: userdb lookup skipped, username taken from USER environment > lda(petar): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= > lda(petar): Debug: Destination address: petar@ (source: user at hostname) > lda(petar): Debug: sieve: Pigeonhole version 0.4.6 (3e924b1b6c5c+) initializing > lda(petar): Debug: sieve: include: sieve_global is not set; it is currently not possible to include `:global' scripts. > lda(petar): Debug: sieve: file storage: Active script path is unconfigured; using default (path=~/.dovecot.sieve) > lda(petar): Debug: sieve: file storage: Using script storage path: /data/dovecot/user/petar.sieve > lda(petar): Debug: sieve: file storage: Using active Sieve script path: /etc/skel/.dovecot.sieve > lda(petar): Debug: sieve: file storage: Relative path to sieve storage in active link: /data/dovecot/user/petar.sieve/ > lda(petar): Debug: sieve: file storage: Using script storage path: /etc/skel/.dovecot.sieve > lda(petar): Debug: sieve: file script: File `/etc/skel/.dovecot.sieve' not found > lda(petar): Debug: sieve: No active Sieve script exists in user storage `/data/dovecot/user/petar.sieve' (trying default script location instead) > lda(petar): Debug: sieve: No default script configured for user > lda(petar): Debug: sieve: User has no personal script > lda(petar): Debug: sieve: No scripts to execute: reverting to default delivery. > > # ls -la /data/dovecot/user/petar.* > -rw------- 1 petar petar 32946 /data/dovecot/user/petar.sieve > -rw------- 1 petar petar 532 /data/dovecot/user/petar.sieve.log > -rw------- 1 petar petar 31571 /data/dovecot/user/petar.svbin > > > Shouldn't both variants at least fail identically if no prefix implies `file:'? There is some backwards compatibility that makes these cases different. But, first of all, it is not a good idea to have a read-only home directory. A writable home directory is important; e.g., to store vacation response status information. That said, I do think this a bug, especially since it worked before. I've investigated this a bit and this is going to be a somewhat bigger change that I need to test thoroughly, so this may take a couple of days. A workaround for now would be to give it a dummy sieve_dir setting that points to a writable directory. Regards, Stephan. From ms at sys4.de Tue Jan 13 08:32:19 2015 From: ms at sys4.de (Michael Schwartzkopff) Date: Tue, 13 Jan 2015 09:32:19 +0100 Subject: dovecot and glusterfs Message-ID: <3425862.ATKZrtYu7l@nb003> Hi, I did some experiments with dovecot on a glusterfs on the active nodes without a director. So I had concurrent access to the files. With the help of the available documentation about NFS and fcntl locks I managed to find out the following: With the plain mbox format dovecot seems to apply and to honor the fcntl locks. But since this format is not used any more in real setups, it is useless. With mdbox and maildir format I could reliably crash my mail storage just by delivering mails to the both dovecots via LMTP to the same user. In maildir dovecot seems not the set / respect the fnctl locks of the index file. dotlocks do not seems to work either with mdbox. So I think the only solution os to use a director in a real world setup. Or is there any non-obvious trick that I did not check? I think it would be really nice if dovecot could use glusterfs for realy scalable postbox servers. Mit freundlichen Gr??en, Michael Schwartzkopff -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From skdovecot at smail.inf.fh-brs.de Tue Jan 13 09:41:26 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 13 Jan 2015 10:41:26 +0100 (CET) Subject: Maildir permissions on creation? In-Reply-To: <3784011419594020@web20h.yandex.ru> References: <2333451419423675@web25m.yandex.ru> <8B93B457-D114-447B-A547-B59069DB99F7@brain-force.ch> <2519001419582918@web3o.yandex.ru> <2530321419583057@web3o.yandex.ru> <3784011419594020@web20h.yandex.ru> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 26 Dec 2014, Von Random wrote: > Yes, and it is not my use case, sadly. ACLs are meant for imap, not > filesystem access and mail_access_groups is also useless for some > reason. It just does not change anything. Probably because I don't use > mail_location and variables when I provide path for home. (It's stored > as is in a database on mailbox creation.) I think Tobi means this page: http://wiki2.dovecot.org/SharedMailboxes/Permissions There you see from which directory filesystem permissions are copied from on creation. > 26.12.2014, 12:07, "Tobi" : >> Have you checked the dovecot wiki for 'filepermissions in shared mailboxes' ? >> Explains how dovecot set ACL when creating mailboxes >> >> Am 26. Dezember 2014 09:37:37 MEZ, schrieb Von Random : >>> Err, my bad, of course 0640 on files in case of setgid on directories. >>> >>> 26.12.2014, 11:36, "Von Random" : >>>> ?The main reason is that I don't want to provide backup user with >>> unnecessary write permissions within maildirs and mail user within >>> backup logs dir. I was talking about mail_user:mail_group 0750 on dirs >>> and 0640 on files. (Or, possibly, mail_user:backup_group 2750 and >>> 2640.) >>>> ?26.12.2014, 11:19, "Tobi" : >>>>> ??Whats the reason you do not want to use default dovecot user. Your >>> idea would assume that at least the group must have write access. For >>> me a no-go on mailboxes. >>>>> ??Am 24. Dezember 2014 13:21:15 MEZ, schrieb Von Random >>> : >>>>>> ??Hello. >>>>>> >>>>>> ??In my configuration dovecot reads home from mysql and uses no >>> variables >>>>>> ??within it. It uses Maildir++ storage with virtual users. I also >>> happen >>>>>> ??to use LMTP. >>>>>> >>>>>> ??I want to use a backup solution that does not involve running >>> itself as >>>>>> ??root. Neither do I want to run it as dovecot's mail user. >>>>>> >>>>>> ??And there lies the problem: dovecot creates maildirs with 0700 and >>>>>> ??files within them inherit that set of permissions. And there seems >>> to >>>>>> ??be no sane way to control it. I think I've figured out what to >>> patch in >>>>>> ??order to change that default, but if possible, I'd like to avoid >>> doing >>>>>> ??that. >>>>>> >>>>>> ??tl;dr: is it possible to change the default set of permissions for >>> new >>>>>> ??maildirs created by dovecot? >>>>> ??- -- >>>>> ??Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail >>> gesendet. >>>>>>> -----BEGIN PGP SIGNATURE----- >>>>>>> ??Version: APG v1.1.1 >>>>>>> >>>>>>> ??iQI7BAEBCgAlBQJUnRoiHhxUb2JpIDx0b2JzdGVyQGJyYWluLWZvcmNlLmNoPgAK >>>>>>> ??CRA1HOYlqGik5L9iD/9FIE2JzWfB7fWJEbI3vVg+ar4LRGwW1xNIf8ahlytPbsmH >>>>>>> ??D9dFP3kWpJH6TO41AksNjz/FxHcUqSrlsKYTFUsDZCyb9Zyxd8Jv+dH04FwUzw+y >>>>>>> ??kq+ayoylnTN7Q8mQnqkUpWEuYd5ohWbbUUSguwzN8Zd1T4jCFR9/Lvu0cweE6meu >>>>>>> ??mBGwvXBbfBP4El+mAM2lKnoMXQpMycS6aX7FwQ/fBaoWSkzg/2az69UtqG6LLF67 >>>>>>> ??ZhJkbn17cpT3y/l/2nV5urdcFVh8RoSXNuYyMN4r7IDKQ6BPsneD+839LS9X/gP/ >>>>>>> ??QMuo5nO5xdx4q1bT9gtv1zz5eKAJ1f3R2oPZyxdpiV5PruSY3J3kvu0cF0aJvQF6 >>>>>>> ??+s0iDsoDBcueB6JEUnYAarteWvBvxRnqb8ytju1xbIEkKLZtaS8Gf2cpZ8eA+Ha/ >>>>>>> ??D0zkYfq7bt+Ra7BY6Qh7XMfiN5z9eWe5RqOmVLZRLf9N75U6wWa+fmXpvlqEtGNV >>>>>>> ??pKsgif/ivCHiA24+JSJ1PBRjwO36Tu4MsSlT2WnkvyFBQzFzg9T00fl+uTXzgNEz >>>>>>> ??SBta2wmXk9XVJ9mPzfNrAaAA3+T6H90Sj1CbRt7NZx30UdwYoIL5XXEtnG0p1XdS >>>>>>> ??moe91H2UE+NT+jgj7emAOLxJf86vIwiHCqI0Zl/7wVNJEnKBXJP5WIHcdEDEHg== >>>>>>> ??=Tl/z >>>>>>> ??-----END PGP SIGNATURE----- >> >> - -- >> Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. >> -----BEGIN PGP SIGNATURE----- >> Version: APG v1.1.1 >> >> iQI7BAEBCgAlBQJUnSVbHhxUb2JpIDx0b2JzdGVyQGJyYWluLWZvcmNlLmNoPgAK >> CRA1HOYlqGik5GvTEACYhFfqS5hK4rAoHCzDd5vFwCLIE/ESheJhREbpewE4k7Nq >> BGR1hKygYeAnj3w1CB9R4sgRBOTHSpRuIzk1ZMId5UuQOBxy8ukRzA5xiAoP22lw >> AxB6Ek7e7VLz0998GmoN7uw1t5tczR1SkHscs0XfuBCBJrvo79DHYs7CP18oDTic >> m5NVJS0MZzJGWPSDGwN6WPwRbLZMTZHc79h+WTmYz9Fzet2As71npvP0PiLd4hQP >> xMlgUXCF1vXTUPUGf/NtQfy/xjaww4sEMqOxpigjLLMmnz+OFPo9Mar0uWvNBCgq >> 3rTUqAUaMcLS8ANyvqzPXXba67dl+rNYuK33tWom+H17JzNf1rdYKxtc7IVQgm/E >> RfYhIejH9yQR0/8CHw+ySF3mJdiQsZHMKL4PwXhRzp9OuOU8RZts1bve8pJEHGSr >> ZQjqjiNB/DyE0s9uNh94U58mwGT8FKHQPR52EPF2WyNlyet1aYUCNQlXm2Qe+3FI >> k4D4eJpRfWIHS5x7NALuwrki+/OqFvDphzKCTmhIC2Qa8UnKvxaS9VccW2Z5D9R9 >> PHoOb2pgm5bIOUtsWUCykDiTwh5IA0jReoGPRlXmFK/tuhHawdrbfUlQ/YrVAX8w >> p+FhchB5e5LNnBOjIXDB9c+viuobF3qo3uoOqjAwGTkEqIdwsiswCUPfjZJD+A== >> =XfBH >> -----END PGP SIGNATURE----- > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLToRnz1H7kL/d9rAQKEaQf/WyLdX/Kd8/RE3YptqbZ7EL+kA9Fg1m1b Mi+LL6fw/Fya+OW2woRa9zQEi3/IGMGZGwbFN5ZvEp+EdIfHZMcy4Gi9a/I1+KDD Q8ePq4lZ4oKvjMrWNk2Jvn+JUszdtzDs0ONzh9JG9zOXPOa9n3CDZ9jdSw6bHNyT KuvB6YIOxz2a8QJ6IQtzWrO36fu13rt0n0NTfgsuaNpX0nYJdcsJ1QC5WBHAEHKe rty5WQH/xcaDkUrOWOL+wAM4jFT5Uou9y9yxLsi3GUCMd03Efp+t3dFZgScES8Ib WrpIDJmw6wUpfVFRfJnpUdfLt9rCxdwBZphPiH1CKHYrRwwGQE9Ujw== =o00L -----END PGP SIGNATURE----- From bra at fsn.hu Tue Jan 13 10:42:59 2015 From: bra at fsn.hu (Nagy, Attila) Date: Tue, 13 Jan 2015 11:42:59 +0100 Subject: [PATCH] Make Delivered-To optional Message-ID: <54B4F6B3.2070009@fsn.hu> Hi, I've read at least one e-mail on this list about making delivered-to in lmtpd optional, but now I need this too, so I made a patch. The default remains as is now (enabled). Rationale: I would like to dsync users and I have catchall POP mailboxes (meaning: a single mailbox gets the mails for a lot of e-mail addresses). If I deliver the e-mails to Dovecot's lmtpd as the original RCPT TO, the Delivered-To header can be used for the POP client to sort the messages on their side to the correct mailbox, but because Dovecot sees this as a new user, it adds it to the replicator.db. Now imagine a single mailbox with thousands of e-mail addresses, this will make thousands of entries in replicator.db, all of which Dovecot would like to replicate independently... If I deliver to the mailbox uid, Dovecot dsync will work right (only one user will be added, no matter how many different e-mail addresses end up in this mailbox), but the Delivered-To will contain the uid, so the user can't use that to sort the messages into their final destination on their side. So the solution here is that I disable the addition of Delivered-To header in Dovecot (because it doesn't know what is the original address) and add it in an upper layer, so during the lmtp transaction, the message will already contain the right value. Currently Dovecot adds another Delivered-To line, which confuses the clients. Please add this feature to Dovecot. Thanks, -------------- next part -------------- diff -r e3640ccaa76d doc/example-config/conf.d/20-lmtp.conf --- a/doc/example-config/conf.d/20-lmtp.conf Sat Jan 10 04:32:42 2015 +0200 +++ b/doc/example-config/conf.d/20-lmtp.conf Tue Jan 13 11:29:20 2015 +0100 @@ -13,8 +13,11 @@ # Verify quota before replying to RCPT TO. This adds a small overhead. #lmtp_rcpt_check_quota = no +# Insert Delivered-To header to the messages, delivered through LMTP. +#lmtp_add_delivered_to = yes + protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins). #mail_plugins = $mail_plugins } - \ No newline at end of file + diff -r e3640ccaa76d src/lmtp/commands.c --- a/src/lmtp/commands.c Sat Jan 10 04:32:42 2015 +0200 +++ b/src/lmtp/commands.c Tue Jan 13 11:29:20 2015 +0100 @@ -996,7 +996,7 @@ if (array_count(&client->state.rcpt_to) > 0) { str_printfa(str, "Return-Path: <%s>\r\n", client->state.mail_from); - if (rcpt_to != NULL) + if (rcpt_to != NULL && client->lmtp_set->lmtp_add_delivered_to) str_printfa(str, "Delivered-To: %s\r\n", rcpt_to); } diff -r e3640ccaa76d src/lmtp/lmtp-settings.c --- a/src/lmtp/lmtp-settings.c Sat Jan 10 04:32:42 2015 +0200 +++ b/src/lmtp/lmtp-settings.c Tue Jan 13 11:29:20 2015 +0100 @@ -60,6 +60,7 @@ DEF(SET_BOOL, lmtp_proxy), DEF(SET_BOOL, lmtp_save_to_detail_mailbox), DEF(SET_BOOL, lmtp_rcpt_check_quota), + DEF(SET_BOOL, lmtp_add_delivered_to), DEF(SET_STR, lmtp_address_translate), DEF(SET_STR_VARS, login_greeting), DEF(SET_STR, login_trusted_networks), @@ -71,6 +72,7 @@ .lmtp_proxy = FALSE, .lmtp_save_to_detail_mailbox = FALSE, .lmtp_rcpt_check_quota = FALSE, + .lmtp_add_delivered_to = TRUE, .lmtp_address_translate = "", .login_greeting = PACKAGE_NAME" ready.", .login_trusted_networks = "" diff -r e3640ccaa76d src/lmtp/lmtp-settings.h --- a/src/lmtp/lmtp-settings.h Sat Jan 10 04:32:42 2015 +0200 +++ b/src/lmtp/lmtp-settings.h Tue Jan 13 11:29:20 2015 +0100 @@ -8,6 +8,7 @@ bool lmtp_proxy; bool lmtp_save_to_detail_mailbox; bool lmtp_rcpt_check_quota; + bool lmtp_add_delivered_to; const char *lmtp_address_translate; const char *login_greeting; const char *login_trusted_networks; From marcin at mejor.pl Tue Jan 13 11:08:45 2015 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Tue, 13 Jan 2015 12:08:45 +0100 Subject: dovecot-2.2.15 process dovecot/imap segfaults when dict process has troubles Message-ID: <54B4FCBD.4040801@mejor.pl> Hi! I hope it's not too late to wish all the best in 2015 year! I'm writing the email because I've noticed that if dict process has any troubles with configuration then dovecot/imap throws segfault: log: Jan 12 16:51:59 mail dovecot: dict: Error: Failed to initialize dictionary 'lastlogin': dict pgsql: Can't open configuration file /etc/dovecot/dovecot-dict-sql.conf.ext: Permission denied Jan 12 16:51:59 mail dovecot: dict: Error: Failed to initialize dictionary 'lastlogin': dict pgsql: Can't open configuration file /etc/dovecot/dovecot-dict-sql.conf.ext: Permission denied Jan 12 16:52:00 mail dovecot: master: Error: service(imap): child 24243 killed with signal 11 (core dumped) and here is backtrace: (gdb) bt #0 io_remove_full (_io=0x7fb02b0afce0, closed=false) at ioloop.c:119 #1 0x00007fb02aac5f40 in io_remove (io=) at ioloop.c:154 #2 0x00007fb02aa841b2 in client_dict_finish_transaction (dict=dict at entry=0x7fb02b0afc20, id=, ret=ret at entry=-1) at dict-client.c:274 #3 0x00007fb02aa84286 in client_dict_disconnect (dict=dict at entry=0x7fb02b0afc20) at dict-client.c:459 #4 0x00007fb02aa8493d in client_dict_deinit (_dict=0x7fb02b0afc20) at dict-client.c:519 #5 0x00007fb02aa28ece in last_login_dict_deinit (user=) at last-login-plugin.c:34 #6 0x00007fb02aa28f92 in last_login_user_deinit (user=0x7fb02b0ab400) at last-login-plugin.c:46 #7 0x00007fb02ab820bf in mail_user_unref (_user=_user at entry=0x7fb02b0bbdc8) at mail-user.c:163 #8 0x00007fb02ac1ee0d in client_default_destroy (client=0x7fb02b0bbd50, reason=) at imap-client.c:284 #9 0x00007fb02ac29549 in client_kill_idle (client=0x7fb02b0bbd50) at main.c:105 #10 0x00007fb02ac296a5 in client_kill_idle (client=0x7fb02b0bbd50) at main.c:129 #11 imap_die () at main.c:121 #12 0x00007fb02aac661d in io_loop_call_io (io=0x7fb02b093ce0) at ioloop.c:498 #13 0x00007fb02aac7a15 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x7fb02b091740) at ioloop-epoll.c:220 #14 0x00007fb02aac66d5 in io_loop_handler_run (ioloop=ioloop at entry=0x7fb02b091740) at ioloop.c:545 #15 0x00007fb02aac677a in io_loop_run (ioloop=0x7fb02b091740) at ioloop.c:522 #16 0x00007fb02aa5b5aa in master_service_run (service=0x7fb02b0915d0, callback=) at master-service.c:566 #17 0x00007fb02ac116cd in main (argc=, argv=) at main.c:412 Marcin From nick.z.edwards at gmail.com Tue Jan 13 11:24:24 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 13 Jan 2015 21:24:24 +1000 Subject: Moving or "upgrading" from MD5 to SSHA512 In-Reply-To: <67761249.20150110020717@turvamies.fi> References: <67761249.20150110020717@turvamies.fi> Message-ID: In MySQL we use default_pass_scheme = CRYPT Assuming this is not a 10yo box, and your libs are modern, this allow for md5/shaxxx/whatever Dont touch postgresql so never bothered to look, but I guess it accepts the same parameter. On 1/10/15, Jyri Hovila [Turvamies.fi] wrote: > Hello, world! > > I have a long-running Dovecot & Postfix installation using PostgreSQL > back-end. > > Until now I've been using MD5 hashing but would like to "upgrade" to the > salted SSHA512. > > Is there a way to configure Dovecot so that it would automatically detect > the type of the hash stored in the database, so that users who have changed > their password (and thus being hashed with SHA512) and users still having an > MD5 hash would both be able to authenticate at the same time? > > Yours, > > Jyri > -- > +358-50-5632104 (24/7) > +358-46-8822157 (backup) > jyri.hovila at turvamies.fi > From nick.z.edwards at gmail.com Tue Jan 13 11:40:34 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 13 Jan 2015 21:40:34 +1000 Subject: dovecot and glusterfs In-Reply-To: <3425862.ATKZrtYu7l@nb003> References: <3425862.ATKZrtYu7l@nb003> Message-ID: On 1/13/15, Michael Schwartzkopff wrote: > Hi, > > I did some experiments with dovecot on a glusterfs on the active nodes > without > a director. So I had concurrent access to the files. > > With the help of the available documentation about NFS and fcntl locks I > managed to find out the following: > > With the plain mbox format dovecot seems to apply and to honor the fcntl > locks. But since this format is not used any more in real setups, it is > useless. > > With mdbox and maildir format I could reliably crash my mail storage just by > > delivering mails to the both dovecots via LMTP to the same user. In maildir > > dovecot seems not the set / respect the fnctl locks of the index file. > dotlocks > do not seems to work either with mdbox. > > So I think the only solution os to use a director in a real world setup. Or > is > there any non-obvious trick that I did not check? Interesting, we use NFSv3 dovecot LDA with maildir, we have at present two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we added a few more over Christmas, so I think about 32 pop3 servers, but with only 4 imap servers incl webmail (IMAP is not heavily used here due to government spy laws) talking to NAS storage server backend, *we do not use director* at all and has never been an issue. Director IIRC solves the problem of IMAP inconsistency, but we never see advantage when we tested, no doubt it solves some fancy setup problem, but since director can not help with pop3, it was not worth the hassle. never had any problems with webmail either, load balancers seem to look after it well We did see lot of people with issues with LMTP, but we never saw these with lda. > I think it would be really nice if dovecot could use glusterfs for realy > scalable postbox servers. There was some nasty problems with that with cyrus, so much so, most uni's here who used that moved to a NAS backend and no one has whinged since :-> From ms at sys4.de Tue Jan 13 12:02:06 2015 From: ms at sys4.de (Michael Schwartzkopff) Date: Tue, 13 Jan 2015 13:02:06 +0100 Subject: dovecot and glusterfs In-Reply-To: References: <3425862.ATKZrtYu7l@nb003> Message-ID: <8583678.2KJkKX4ukM@nb003> Am Dienstag, 13. Januar 2015, 21:40:34 schrieb Nick Edwards: > On 1/13/15, Michael Schwartzkopff wrote: > > Hi, > > > > I did some experiments with dovecot on a glusterfs on the active nodes > > without > > a director. So I had concurrent access to the files. > > > > With the help of the available documentation about NFS and fcntl locks I > > managed to find out the following: > > > > With the plain mbox format dovecot seems to apply and to honor the fcntl > > locks. But since this format is not used any more in real setups, it is > > useless. > > > > With mdbox and maildir format I could reliably crash my mail storage just > > by > > > > delivering mails to the both dovecots via LMTP to the same user. In > > maildir > > > > dovecot seems not the set / respect the fnctl locks of the index file. > > dotlocks > > do not seems to work either with mdbox. > > > > So I think the only solution os to use a director in a real world setup. > > Or > > is > > there any non-obvious trick that I did not check? > > Interesting, we use NFSv3 dovecot LDA with maildir, we have at present > two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we > added a few more over Christmas, so I think about 32 pop3 servers, > but with only 4 imap servers incl webmail (IMAP is not heavily used > here due to government spy laws) talking to NAS storage server > backend, *we do not use director* at all and has never been an issue. > Director IIRC solves the problem of IMAP inconsistency, but we never > see advantage when we tested, no doubt it solves some fancy setup > problem, but since director can not help with pop3, it was not worth > the hassle. never had any problems with webmail either, load balancers > seem to look after it well Yes. NFS has its own locking. I wanted to use plain glusterfs client without the detour of NFS. Thanks for your hint. Mit freundlichen Gr??en, Michael Schwartzkopff -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From m at sys4.de Tue Jan 13 12:30:15 2015 From: m at sys4.de (Marc Schiffbauer) Date: Tue, 13 Jan 2015 13:30:15 +0100 Subject: dovecot and glusterfs In-Reply-To: <3425862.ATKZrtYu7l@nb003> References: <3425862.ATKZrtYu7l@nb003> Message-ID: <20150113123015.GA7538@schiffbauer.net> * Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr: >Hi, > >I did some experiments with dovecot on a glusterfs on the active nodes without >a director. So I had concurrent access to the files. > >With the help of the available documentation about NFS and fcntl locks I >managed to find out the following: > >With the plain mbox format dovecot seems to apply and to honor the fcntl >locks. But since this format is not used any more in real setups, it is >useless. Did you try sdbox, too? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From ms at sys4.de Tue Jan 13 12:46:30 2015 From: ms at sys4.de (Michael Schwartzkopff) Date: Tue, 13 Jan 2015 13:46:30 +0100 Subject: dovecot and glusterfs In-Reply-To: <20150113123015.GA7538@schiffbauer.net> References: <3425862.ATKZrtYu7l@nb003> <20150113123015.GA7538@schiffbauer.net> Message-ID: <1958707.kuIYjfN8ek@nb003> Am Dienstag, 13. Januar 2015, 13:30:15 schrieb Marc Schiffbauer: > * Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr: > >Hi, > > > >I did some experiments with dovecot on a glusterfs on the active nodes > >without a director. So I had concurrent access to the files. > > > >With the help of the available documentation about NFS and fcntl locks I > >managed to find out the following: > > > >With the plain mbox format dovecot seems to apply and to honor the fcntl > >locks. But since this format is not used any more in real setups, it is > >useless. > > Did you try sdbox, too? not yet. Will try as soon as I find some time. Mit freundlichen Gr??en, Michael Schwartzkopff -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From marcin at mejor.pl Tue Jan 13 16:04:02 2015 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Tue, 13 Jan 2015 17:04:02 +0100 Subject: dovecot-2.2.15 process dovecot/imap segfaults when dict process has troubles In-Reply-To: References: <54B4FCBD.4040801@mejor.pl> Message-ID: <54B541F2.8030702@mejor.pl> W dniu 13.01.2015 o 13:53, Lazy pisze: > I can't reproduce this with redis dict. > > Could You provide us with a doveconf -n ? > Does this also happen with other dict engines ? Working configuration: auth_cache_size = 64 k auth_cache_ttl = 10 mins auth_master_user_separator = * auth_mechanisms = plain login deliver_log_format = msgid=%m: from=%f: phys=%p: virt=%w %$ dict { domainquota = pgsql:/etc/dovecot/dovecot-dict-sql-domain.conf.ext lastlogin = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext quota = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } first_valid_gid = 999 first_valid_uid = 999 last_valid_gid = 999 last_valid_uid = 999 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * mail_gid = 999 mail_location = maildir:~/.maildir:INDEX=/data/poczta/indeksy/%d/%n mail_log_prefix = "%s(%u) <%{session}>: " mail_plugins = acl fts fts_squat listescape quota notify mail_log stats zlib mail_temp_dir = /data/poczta/tmp mail_uid = 999 mailbox_list_index = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include vari ables body enotify environment mailbox date ihave duplicate namespace { location = maildir:%%h/.maildir:INDEX=/data/poczta/indeksy/shared/%d/%n prefix = shared/%%n/ type = shared } namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Archives { special_use = \Archive } mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } mailbox Wiadomo?ci-?mieci { special_use = \Junk } mailbox Wiadomo&AVs-ci-&AVs-mieci { special_use = \Junk } prefix = separator = / } passdb { driver = pam master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/data/poczta/acl/global-acls:cache_secs=60 acl_shared_dict = file:/data/poczta/acl/shared-mailboxes.db fts = squat fts_squat = partial=4 full=10 last_login_dict = proxy::lastlogin last_login_key = lastlogin/%u/%s/%r listescape_char = \ quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = file:~/sieve;active=~/.dovecot.sieve sieve_default = /etc/dovecot/sieve/sieve_default_spam-to-junk.sieve stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 1M stats_refresh = 30 s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } postmaster_address = postmaster at example.com protocols = imap pop3 lmtp sieve quota_full_tempfail = yes service auth-worker { user = root } service auth { unix_listener auth-client { group = mail mode = 0660 } unix_listener auth-userdb { mode = 0777 } } service dict { unix_listener dict { group = poczta mode = 0660 } } service stats { fifo_listener stats-mail { group = poczta mode = 0660 } } ssl_cert = Hello, I want to test dovecot proxying, on a user basis, in a simple set up (proxy, no proxy_maybe and forward the passwd to the remote server). This seem pretty simple except that, in order to test it alongside the production server, I set up 3 servers : . the proxy, listening to pop and imap on non standard ports 9143, 9993, 9110, 9995 . a proxy destination for some users, listening on 10143, 10993, 109110, 10995 . another proxy destination for some other users, listening on 20145, 20993, 209110, 20995 But there seems to be only one 'port' extra_field, so how can I handle the proxying of both IMAP and POP3 (especially for the same user who can use both) ? Thanks. -- Thomas Hummel | Institut Pasteur | Groupe Exploitation et Infrastructure From list at airstreamcomm.net Tue Jan 13 17:33:32 2015 From: list at airstreamcomm.net (List) Date: Tue, 13 Jan 2015 11:33:32 -0600 Subject: dovecot and glusterfs In-Reply-To: <8583678.2KJkKX4ukM@nb003> References: <3425862.ATKZrtYu7l@nb003> <8583678.2KJkKX4ukM@nb003> Message-ID: <54B556EC.1020900@airstreamcomm.net> On 1/13/15, 6:02 AM, Michael Schwartzkopff wrote: > Am Dienstag, 13. Januar 2015, 21:40:34 schrieb Nick Edwards: >> On 1/13/15, Michael Schwartzkopff wrote: >>> Hi, >>> >>> I did some experiments with dovecot on a glusterfs on the active nodes >>> without >>> a director. So I had concurrent access to the files. >>> >>> With the help of the available documentation about NFS and fcntl locks I >>> managed to find out the following: >>> >>> With the plain mbox format dovecot seems to apply and to honor the fcntl >>> locks. But since this format is not used any more in real setups, it is >>> useless. >>> >>> With mdbox and maildir format I could reliably crash my mail storage just >>> by >>> >>> delivering mails to the both dovecots via LMTP to the same user. In >>> maildir >>> >>> dovecot seems not the set / respect the fnctl locks of the index file. >>> dotlocks >>> do not seems to work either with mdbox. >>> >>> So I think the only solution os to use a director in a real world setup. >>> Or >>> is >>> there any non-obvious trick that I did not check? >> Interesting, we use NFSv3 dovecot LDA with maildir, we have at present >> two dozen front end SMTP servers (using dovecot-lda) and some, hrmm we >> added a few more over Christmas, so I think about 32 pop3 servers, >> but with only 4 imap servers incl webmail (IMAP is not heavily used >> here due to government spy laws) talking to NAS storage server >> backend, *we do not use director* at all and has never been an issue. >> Director IIRC solves the problem of IMAP inconsistency, but we never >> see advantage when we tested, no doubt it solves some fancy setup >> problem, but since director can not help with pop3, it was not worth >> the hassle. never had any problems with webmail either, load balancers >> seem to look after it well > Yes. NFS has its own locking. I wanted to use plain glusterfs client without > the detour of NFS. Thanks for your hint. > > Mit freundlichen Gr??en, > > Michael Schwartzkopff > The last time we experimented with Glusterfs (two years ago) the native client was actually not able to maintain consistency as well as the NFS for a reason that I cannot remember anymore. We used maildir, and when using NFS we were able to deliver about a hundred thousand emails per hour and do a couple hundred thousand IMAP and POP3 retrievals per hour against a modest four node Gluster cluster with four Dovecot/Postfix servers (running in vmware). From ms at sys4.de Tue Jan 13 22:01:50 2015 From: ms at sys4.de (Michael Schwartzkopff) Date: Tue, 13 Jan 2015 23:01:50 +0100 Subject: dovecot and glusterfs In-Reply-To: <20150113123015.GA7538@schiffbauer.net> References: <3425862.ATKZrtYu7l@nb003> <20150113123015.GA7538@schiffbauer.net> Message-ID: <19490839.mVvdvmhfIa@nb003> Am Dienstag, 13. Januar 2015, 13:30:15 schrieb Marc Schiffbauer: > * Michael Schwartzkopff schrieb am 13.01.15 um 09:32 Uhr: > >Hi, > > > >I did some experiments with dovecot on a glusterfs on the active nodes > >without a director. So I had concurrent access to the files. > > > >With the help of the available documentation about NFS and fcntl locks I > >managed to find out the following: > > > >With the plain mbox format dovecot seems to apply and to honor the fcntl > >locks. But since this format is not used any more in real setups, it is > >useless. > > Did you try sdbox, too? sdbox also gives an error: lmtp(30999, user1 at example.net): Error: sdbox: /srv/mail/user1 at example.net/mailboxes/INBOX/dbox-Mails/u.69 already exists, rebuilding index Mit freundlichen Gr??en, Michael Schwartzkopff -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64, +49 (162) 165 0044 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From jyri.hovila at turvamies.fi Tue Jan 13 22:08:07 2015 From: jyri.hovila at turvamies.fi (Jyri Hovila [Turvamies.fi]) Date: Wed, 14 Jan 2015 00:08:07 +0200 Subject: Moving or "upgrading" from MD5 to SSHA512 In-Reply-To: <54B595FA.1040600@bluerosetech.com> References: <67761249.20150110020717@turvamies.fi> <54B0D649.4010104@bluerosetech.com> <906443151.20150110124525@turvamies.fi> <54B595FA.1040600@bluerosetech.com> Message-ID: <1573149518.20150114000807@turvamies.fi> Hi! My "no" referred to the fact that I had not read the article before. =D Thanks for linking it - I'm sure it will be very useful. I've been terribly busy during the last couple of days and have therefore not been able to comment much on the list. I'll come back there as soon as things calm down a bit. =) Yours, Jyri -- +358-50-5632104 (24/7) +358-46-8822157 (backup) jyri.hovila at turvamies.fi From rblayzor.bulk at inoc.net Wed Jan 14 00:28:53 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 13 Jan 2015 19:28:53 -0500 Subject: pigeonhole ereject vs reject Message-ID: Currently pigeonhole supports reject which would generate a NDR for each message. (If I understand the current documentation) Using Dovecot LMTP it would be more optimal to kick a 5xx back to the primary MTA to reject the delivery rather than generating more back scatter NDRs. Anyone know if this is possible now (to forgo the NDR)'s or do we need to wait for ereject ? If so, how much work is involved? -- Robert inoc.net!rblayzor http://inoc.net/ From h.reindl at thelounge.net Wed Jan 14 00:34:10 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 14 Jan 2015 01:34:10 +0100 Subject: pigeonhole ereject vs reject In-Reply-To: References: Message-ID: <54B5B982.3070103@thelounge.net> Am 14.01.2015 um 01:28 schrieb Robert Blayzor: > Currently pigeonhole supports reject which would generate a NDR for each message. (If I understand the current documentation) > > Using Dovecot LMTP it would be more optimal to kick a 5xx back to the primary MTA to reject the delivery rather than generating more back scatter NDRs and what would that change? nothing if you think about how mail works! * the MTA receives the message * the MTA confirms with 2xx status code * later the delivery server rejects * the MTA *must* create a bounce just don't reject mails after you confirmed you have received them in the SMTP session and if you don't want a mail after that DISCARD it by consider legal implications - there is nothing between -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From rblayzor.bulk at inoc.net Wed Jan 14 01:22:24 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 13 Jan 2015 20:22:24 -0500 Subject: pigeonhole ereject vs reject In-Reply-To: <54B5B982.3070103@thelounge.net> References: <54B5B982.3070103@thelounge.net> Message-ID: <448D9D04-EE38-4AFE-BF6B-440DF60D9A2B@inoc.net> From rblayzor.bulk at inoc.net Wed Jan 14 01:23:24 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 13 Jan 2015 20:23:24 -0500 Subject: pigeonhole ereject vs reject In-Reply-To: <54B5B982.3070103@thelounge.net> References: <54B5B982.3070103@thelounge.net> Message-ID: <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> On Jan 13, 2015, at 7:34 PM, Reindl Harald wrote: > > and what would that change? > nothing if you think about how mail works! > > * the MTA receives the message > * the MTA confirms with 2xx status code > * later the delivery server rejects > * the MTA *must* create a bounce > > just don't reject mails after you confirmed you have received them in the SMTP session and if you don't want a mail after that DISCARD it by consider legal implications - there is nothing between > The above is not entirely true. You are assuming that your MTA it's sending a 2xx accepting the message immediately before delivery via LMTP completes. With PRDR (in Exim for example, or without) a 5xx during the LMTP transport should issue a 5xx error back to the sending MTA, not a 2xx. Therefore, there would be no NDR generated by the receiving system. The senders MTA would have to generate the NDR, but that's not my problem at that point. Of course WITHOUT PRDR this is a little bit more of an issue since it would be a rejection for all recipients of the message. -- Robert inoc.net!rblayzor http://inoc.net/ From h.reindl at thelounge.net Wed Jan 14 01:30:28 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 14 Jan 2015 02:30:28 +0100 Subject: pigeonhole ereject vs reject In-Reply-To: <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> References: <54B5B982.3070103@thelounge.net> <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> Message-ID: <54B5C6B4.9020706@thelounge.net> Am 14.01.2015 um 02:23 schrieb Robert Blayzor: > On Jan 13, 2015, at 7:34 PM, Reindl Harald wrote: >> >> and what would that change? >> nothing if you think about how mail works! >> >> * the MTA receives the message >> * the MTA confirms with 2xx status code >> * later the delivery server rejects >> * the MTA *must* create a bounce >> >> just don't reject mails after you confirmed you have received them in the SMTP session and if you don't want a mail after that DISCARD it by consider legal implications - there is nothing between >> > > The above is not entirely true. You are assuming that your MTA it's sending a 2xx accepting the message immediately before delivery via LMTP completes. With PRDR (in Exim for example, or without) a 5xx during the LMTP transport should issue a 5xx error back to the sending MTA, not a 2xx. Therefore, there would be no NDR generated by the receiving system. The senders MTA would have to generate the NDR, but that's not my problem at that point. Of course WITHOUT PRDR this is a little bit more of an issue since it would be a rejection for all recipients of the message. i assume a sane MTA like postfix with a queue and so be able to receive and confirm messages independent of the final destination - even if you use typically LMTP there could be an external transport for a RCPT and the same message can have internal and external destinations so what you want in your OP is just DISCARD in a sieve script and there is no point in "Using Dovecot LMTP it would be more optimal to kick a 5xx back" when the desired result is DISCARD why do you want the burden of keep the SMTP session with the client open until the mail is finally stored? that don't scale! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From rblayzor.bulk at inoc.net Wed Jan 14 01:40:16 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 13 Jan 2015 20:40:16 -0500 Subject: pigeonhole ereject vs reject In-Reply-To: <54B5C6B4.9020706@thelounge.net> References: <54B5B982.3070103@thelounge.net> <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> <54B5C6B4.9020706@thelounge.net> Message-ID: <9C3D712F-B7E6-4785-B4E2-B2503ABAB0E8@inoc.net> On Jan 13, 2015, at 8:30 PM, Reindl Harald wrote: > > so what you want in your OP is just DISCARD in a sieve script and there is no point in "Using Dovecot LMTP it would be more optimal to kick a 5xx back" when the desired result is DISCARD > > why do you want the burden of keep the SMTP session with the client open until the mail is finally stored? that don't scale! Sieve is all about policy. A 5xx reject would let the sending server know the message could not be delivered due a failure. (ie: user policy rejection, without receiving MTA generating NDR backscatter). I would rather not just accept it and the message disappear into ether without the sender receiving any notification of why. -- Robert inoc.net!rblayzor http://inoc.net/ From h.reindl at thelounge.net Wed Jan 14 01:44:51 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 14 Jan 2015 02:44:51 +0100 Subject: pigeonhole ereject vs reject In-Reply-To: <9C3D712F-B7E6-4785-B4E2-B2503ABAB0E8@inoc.net> References: <54B5B982.3070103@thelounge.net> <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> <54B5C6B4.9020706@thelounge.net> <9C3D712F-B7E6-4785-B4E2-B2503ABAB0E8@inoc.net> Message-ID: <54B5CA13.7040301@thelounge.net> Am 14.01.2015 um 02:40 schrieb Robert Blayzor: > On Jan 13, 2015, at 8:30 PM, Reindl Harald wrote: >> >> so what you want in your OP is just DISCARD in a sieve script and there is no point in "Using Dovecot LMTP it would be more optimal to kick a 5xx back" when the desired result is DISCARD >> >> why do you want the burden of keep the SMTP session with the client open until the mail is finally stored? that don't scale! > > Sieve is all about policy no - it is about *filter* mails > A 5xx reject would let the sending server know the message could not be delivered due a failure which belongs in the MTA and not the LDA > (ie: user policy rejection, without receiving MTA generating NDR backscatter). > I would rather not just accept it and the message disappear into ether without > the sender receiving any notification of why. hence you reject messages on MTA level before LMTP is called because taht happens in case of sender based filters in the envelope-level and in case of subject filters at least before the mailbody -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From edgar at pettijohn-web.com Wed Jan 14 01:59:44 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Tue, 13 Jan 2015 19:59:44 -0600 Subject: blowfish crypt Message-ID: <54B5CD90.7090300@pettijohn-web.com> I'm setting up a new server and am planning on using blowfish crypt. This page from the wiki http://wiki2.dovecot.org/Authentication/PasswordSchemes says the password will begin with $2a$, however mine are starting $2b$. Is this wrong on the wiki or am I doing something wrong? Just want to catch this before it becomes a problem. Thanks, Edgar From edgar at pettijohn-web.com Wed Jan 14 02:04:00 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Tue, 13 Jan 2015 20:04:00 -0600 Subject: blowfish crypt In-Reply-To: <54B5CD90.7090300@pettijohn-web.com> References: <54B5CD90.7090300@pettijohn-web.com> Message-ID: <54B5CE90.80608@pettijohn-web.com> Disregard i was using the encrypt command instead of doveadm. On 01/13/15 19:59, Edgar Pettijohn wrote: > I'm setting up a new server and am planning on using blowfish crypt. > This page from the wiki > http://wiki2.dovecot.org/Authentication/PasswordSchemes says the > password will begin with $2a$, however mine are starting $2b$. Is > this wrong on the wiki or am I doing something wrong? Just want to > catch this before it becomes a problem. > > Thanks, > > Edgar From rblayzor.bulk at inoc.net Wed Jan 14 02:04:28 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 13 Jan 2015 21:04:28 -0500 Subject: pigeonhole ereject vs reject In-Reply-To: <54B5CA13.7040301@thelounge.net> References: <54B5B982.3070103@thelounge.net> <8B05A91A-041D-4556-B53D-D345F204C8A7@inoc.net> <54B5C6B4.9020706@thelounge.net> <9C3D712F-B7E6-4785-B4E2-B2503ABAB0E8@inoc.net> <54B5CA13.7040301@thelounge.net> Message-ID: On Jan 13, 2015, at 8:44 PM, Reindl Harald wrote: >> A 5xx reject would let the sending server know the message could not be delivered due a failure > > which belongs in the MTA and not the LDA > >> (ie: user policy rejection, without receiving MTA generating NDR backscatter). >> I would rather not just accept it and the message disappear into ether without >> the sender receiving any notification of why. > > hence you reject messages on MTA level before LMTP is called because taht happens in case of sender based filters in the envelope-level and in case of subject filters at least before the mailbody RFC5429 clarifies the need and why "ereject" should be used vs. reject with NDR. https://tools.ietf.org/html/rfc5429 Maybe in your MTA at data time you just accept the message and bounce later (or just discard it). I have the flexibility and the choice not to. -- Robert inoc.net!rblayzor http://inoc.net/ From gdrub13 at gmail.com Wed Jan 14 14:50:24 2015 From: gdrub13 at gmail.com (rub zorghy) Date: Wed, 14 Jan 2015 15:50:24 +0100 Subject: Questions regarding imaptest Message-ID: Hi, The measurements were created under the following conditions : ? operating system : Red Hat Enterprise Linux Server release 6.6 (Santiago) kernel in version 2.6.32-504.el6.x86_64 ? virtual server (VMware) with an Intel(R) Xeon(R) 4vCPU E5649 x 2,526 GHz (2 cores per virtual socket) and 4 GB RAM ? 7,200 RPM SATA 1TB (FC SAN IBM System Storage N3400) ? all file systems had been formated in ext4 ? Dovecot v.2.0.9 ? imaptest v.1.0.0 ? LDAP 389 Directory Server v.1.2.2 ? free pagecache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches and services restarted before the test "Tuning" based upon the book ? *Dovecot: POP3/IMAP servers for enterprises and ISPs* ? Peer Heinlein : ---%<------------------------------------------ imap {} (10-master.conf). service imap { [...] process_limit = 8192 [...] } auth {} service auth { [...] client_limit = 9516 [...] } process_min_avail = 16 ---%<------------------------------------------ mail_max_userip_connections = 100 The test results shows : Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 324 142 128 263 299 405 92 153 320 336 335 800/800 212 92 92 182 166 233 59 87 178 201 202 800/800 - 4051 stalled for 16 secs in command: 8 APPEND "INBOX" () {3251+} 315 160 134 300 275 400 85 136 300 317 317 800/800 - 4051 stalled for 17 secs in command: 8 APPEND "INBOX" () {3251+} 247 130 163 292 251 366 99 115 239 240 240 800/800 222 122 117 234 279 383 112 147 263 222 222 800/800 188 93 94 177 168 232 65 87 162 196 195 800/800 290 126 119 260 262 383 102 126 283 282 283 800/800 240 125 107 243 259 353 102 130 253 256 256 800/800 276 158 144 291 280 400 101 149 266 261 261 800/800 254 95 89 189 235 313 78 118 243 253 253 800/800 82 829 926 861 722 593 455 628 1695 2252 4 ms/cmd avg Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% 30% 5% 148 59 55 126 119 174 43 64 130 147 147 800/800 - 8770 stalled for 16 secs in command: 4 FETCH 1:2 (INTERNALDATE UID INTERNALDATE) 261 128 139 247 252 369 101 125 258 269 269 800/800 302 159 158 307 282 398 109 126 296 298 297 800/800 234 96 118 239 270 369 98 138 357 428 427 800/800 361 213 220 450 383 596 151 197 240 165 166 800/800 192 93 92 206 256 352 104 118 297 289 290 800/800 - 9806 stalled for 16 secs in command: 4 FETCH 1 (RFC822.SIZE) 245 85 72 162 128 190 52 73 157 174 173 800/800 - 9806 stalled for 17 secs in command: 4 FETCH 1 (RFC822.SIZE) 294 152 165 348 360 526 136 179 313 308 309 800/800 272 87 79 174 164 229 54 85 218 230 230 800/800 218 152 152 310 325 467 117 152 265 236 221 798/800 119 830 852 859 707 566 511 702 1672 2161 6 ms/cmd avg I have three questions : 1) how to interpret each row except the last one ? 2) how to explain the stalled commands ? 3) why benchmark shows too few read disk I/O ? Thanks so much, rub From mlnospam at yahoo.com Wed Jan 14 15:05:07 2015 From: mlnospam at yahoo.com (ML mail) Date: Wed, 14 Jan 2015 15:05:07 +0000 (UTC) Subject: [SERVERBUG] failed to send mail with SA and antispam plugin Message-ID: <1230729371.1068223.1421247907704.JavaMail.yahoo@jws106126.mail.bf1.yahoo.com> Hello, I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: [SERVERBUG] failed to send mail Does anyone have an idea what could be the issue here? It is quite annoying as the mail then stays in the Spam folder and can not been re-learnt has ham. Best regards ML From laeeth at laeeth.com Wed Jan 14 17:39:36 2015 From: laeeth at laeeth.com (Laeeth Isharc) Date: Wed, 14 Jan 2015 17:39:36 +0000 Subject: dovecot.org slow & imap search now blazing fast Message-ID: <54B6A9D8.6080300@laeeth.com> Hi. In case you were not aware, I wanted to let you know that dovecot.org has been very slow (often almost unusable) for at least several days now. Also, I upgraded my dovecot from 2.2.10 to 2.2.15. Thank you for fixing IMAP search. I have c. 2mm messages, and it had become unusable before but it is now very fast. (I saw a reference to an optimisation on the mailing list from back in Oct 2013, so maybe that was it). Regards, Laeeth From laeeth at laeeth.com Wed Jan 14 17:42:18 2015 From: laeeth at laeeth.com (Laeeth Isharc) Date: Wed, 14 Jan 2015 17:42:18 +0000 Subject: attn rick van rein (midget) Message-ID: <54B6AA7A.6020801@laeeth.com> From 11 Oct 2013 (Rick van Rein) "Hello, I love Dovecot, but when developing a small IMAP tool, I ran into searching behaviour can easily be optimised. Please forgive a rather detailed suggestion. This was on Dovecot 1.2.15 on Debian Squeeze. My tool? It's called "midget" and retrieves documents from an IMAP box based on their mid: or cid: identifier, as per RFC 2392. I thought this would be useful to retrieve email attachments into a remote shell environment. When using Kerberos, the credentials and a strong hint of one's mail address are already present anyway, DNS SRV does the rest. If you want to see the early code, I'll be happy to share it here." Hi Rick. Thanks for the suggested optimisation - I am not sure if it is you who I have to think for contributing towards making IMAP search fast again. In any case, if you see this I would be interested to see the source for midget if you are willing to share. If you don't want to post to the list then my email is laeeth at laeeth.com Thanks. Laeeth From user+dovecot at localhost.localdomain.org Wed Jan 14 20:06:50 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Wed, 14 Jan 2015 20:06:50 +0000 Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: <1230729371.1068223.1421247907704.JavaMail.yahoo@jws106126.mail.bf1.yahoo.com> References: <1230729371.1068223.1421247907704.JavaMail.yahoo@jws106126.mail.bf1.yahoo.com> Message-ID: <54B6CC5A.5000700@localhost.localdomain.org> On 01/14/2015 03:05 PM, ML mail wrote: > Hello, > > I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: > > [SERVERBUG] failed to send mail > > Does anyone have an idea what could be the issue here? It is quite annoying as the mail then stays in the Spam folder and can not been re-learnt has ham. > Someone has stolen my crystal ball. :-( Please provide `doveconf -n` output. Regards, Pascal -- The trapper recommends today: beeffeed.1501421 at localdomain.org From tss at iki.fi Wed Jan 14 23:24:17 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 14 Jan 2015 15:24:17 -0800 Subject: HG e3640ccaa76d crashes upon IMAP SEARCH In-Reply-To: <20150112121547.GA38901@nihlus.leuxner.net> References: <20150112121547.GA38901@nihlus.leuxner.net> Message-ID: <4CC2F226-F880-41F8-9FEB-FE54050354E6@iki.fi> On 12 Jan 2015, at 04:15, Thomas Leuxner wrote: > $ doveconf -n | head > # 2.2.15 (e3640ccaa76d): /etc/dovecot/dovecot.conf > b search charset utf-8 body "Sleeve" > imap(tlx at leuxner.net): Panic: file message-decoder.c: line 346 (message_decode_body): assertion failed: (ctx->translation_size <= sizeof(ctx->translation_buf)) Hopefully fixed now? Looks like there have been all kinds of small bugs with searching. From tlx at leuxner.net Wed Jan 14 23:57:57 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 15 Jan 2015 00:57:57 +0100 Subject: HG e3640ccaa76d crashes upon IMAP SEARCH In-Reply-To: <4CC2F226-F880-41F8-9FEB-FE54050354E6@iki.fi> References: <20150112121547.GA38901@nihlus.leuxner.net> <4CC2F226-F880-41F8-9FEB-FE54050354E6@iki.fi> Message-ID: <20150114235757.GA43905@nihlus.leuxner.net> * Timo Sirainen 2015.01.15 00:24: > > $ doveconf -n | head > > # 2.2.15 (e3640ccaa76d): /etc/dovecot/dovecot.conf > > > b search charset utf-8 body "Sleeve" > > imap(tlx at leuxner.net): Panic: file message-decoder.c: line 346 (message_decode_body): assertion failed: (ctx->translation_size <= sizeof(ctx->translation_buf)) > > Hopefully fixed now? Looks like there have been all kinds of small bugs with searching. Works now with latest commits. Tested with manual approach above and MUAs. Thanks! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From mlnospam at yahoo.com Thu Jan 15 07:10:08 2015 From: mlnospam at yahoo.com (ML mail) Date: Thu, 15 Jan 2015 07:10:08 +0000 (UTC) Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: <54B6CC5A.5000700@localhost.localdomain.org> References: <54B6CC5A.5000700@localhost.localdomain.org> Message-ID: <67482780.1472250.1421305808440.JavaMail.yahoo@jws10672.mail.bf1.yahoo.com> Hehe sure. Here would be the output of dovecot -n on my maibox server: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 auth_master_user_separator = * auth_mechanisms = plain login cram-md5 disable_plaintext_auth = no first_valid_uid = 150 last_valid_uid = 150 login_trusted_networks = X.Y.Z.X mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { special_use = \Trash } prefix = INBOX. separator = . } passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { antispam_backend = pipe antispam_pipe_program = /usr/bin/spamc antispam_pipe_program_args = -d;my-mx-server.domain.com;-u;amavis antispam_pipe_program_notspam_args = -L;ham antispam_pipe_program_spam_args = -L;spam antispam_pipe_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = error antispam_spam = INBOX.Spam;INBOX.Junk antispam_trash = INBOX.trash;trash;INBOX.Trash;Trash;INBOX.Deleted Items;Deleted Items;INBOX.Deleted Messages;Deleted Messages antispam_verbose_debug = 1 quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = INBOX.Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_after = /etc/dovecot/sieve/after.d sieve_before = /etc/dovecot/sieve/before.d sieve_dir = ~/sieve } postmaster_address = postmaster at domain.com protocols = " imap lmtp sieve pop3" service auth { inet_listener { port = 12345 } } service imap { process_limit = 4096 } service lmtp { inet_listener lmtp { port = 24 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl_cert = wrote: On 01/14/2015 03:05 PM, ML mail wrote: > Hello, > > I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: > > [SERVERBUG] failed to send mail > > Does anyone have an idea what could be the issue here? It is quite annoying as the mail then stays in the Spam folder and can not been re-learnt has ham. > Someone has stolen my crystal ball. :-( Please provide `doveconf -n` output. Regards, Pascal -- The trapper recommends today: beeffeed.1501421 at localdomain.org From skdovecot at smail.inf.fh-brs.de Thu Jan 15 07:30:01 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 15 Jan 2015 08:30:01 +0100 (CET) Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: <67482780.1472250.1421305808440.JavaMail.yahoo@jws10672.mail.bf1.yahoo.com> References: <54B6CC5A.5000700@localhost.localdomain.org> <67482780.1472250.1421305808440.JavaMail.yahoo@jws10672.mail.bf1.yahoo.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 15 Jan 2015, ML mail wrote: It would be better to have the original question & discussion at the top, so one could cut and read nicely, > On Wednesday, January 14, 2015 9:06 PM, Pascal Volk wrote: > On 01/14/2015 03:05 PM, ML mail wrote: > >> Hello, >> >> I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: >> >> [SERVERBUG] failed to send mail > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 [cut] > plugin { > antispam_backend = pipe > antispam_pipe_program = /usr/bin/spamc > antispam_pipe_program_args = -d;my-mx-server.domain.com;-u;amavis > antispam_pipe_program_notspam_args = -L;ham > antispam_pipe_program_spam_args = -L;spam > antispam_pipe_tmpdir = /tmp > antispam_signature = X-Spam-Flag > antispam_signature_missing = error > antispam_spam = INBOX.Spam;INBOX.Junk > antispam_trash = INBOX.trash;trash;INBOX.Trash;Trash;INBOX.Deleted Items;Deleted Items;INBOX.Deleted Messages;Deleted Messages > antispam_verbose_debug = 1 What does the Plugin log? [cut] > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = vmail > } You seem to use a virtual user configuration, so the antispam plugin executes spamc as user vmail, correct? What happens if you run the program manually? sudo -u vmail /usr/bin/spamc -d my-mx-server.domain.com -u amavis -L ham \ < message You can also trace the program with a wrapper script. Replace antispam_pipe_program with /usr/local/bin/spamc-wrapper ==== BEGIN /usr/local/bin/spamc-wrapper #/bin/bash ( # when and who date;id; let i=0 # what echo /usr/bin/spamc "$@" # see embedded spaces in arguments for arg; do let i=i+1 echo "Arg#$i: '$arg'" done # call original program /usr/bin/spamc "$@" # log return code / exit code rc=$? echo rc=$rc # Make sure the rc is returned back to caller exit $rc # log everything into a file ) >>/tmp/spamc-wrapper.log 2>&1 ==== END If you have lots of simultaneous calls, create one log file per call -> add .$$ to filename. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLdsenz1H7kL/d9rAQI5wQf/R/8IIWAgcLX8061FBI5gFxEH8jP8UVMb MoSkkRT88teQJYuDxjr8zA92MX/2HDruRstzwxgJ1WxPefOvETpzGs1wrFqtWABM 2qB/ENfmkyOgmvpfsX5j6armYgOGEK0j5lf/ulV2i110wAJqjWY+9hBekFs7g4Th j29D42kv5Tl0XVwJbbYfzH/gUB+kyLxV7ja3dWWmmRRNV9am4Du36zua5AB2BzYq kuSqXfxClbnRYA/Ajy8H1KYhmx8wRtjkNijxt8B7R5f04E8hLrFVd5lKZIieuO6e oNu45xHeW3mYas0I3jWBf0u5pt1XlP7RLOtdB3D15CsW42PPsDcKxw== =Beji -----END PGP SIGNATURE----- From jtam.home at gmail.com Thu Jan 15 10:28:38 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 15 Jan 2015 02:28:38 -0800 (PST) Subject: dovecot.org slow & imap search now blazing fast In-Reply-To: References: Message-ID: From: Laeeth Isharc > In case you were not aware, I wanted to let you know that dovecot.org > has been very slow (often almost unusable) for at least several days now. Not for me as of the writring of this Email. Seems OK to me. Check your browser, network, etc. Joseph Tam From mlnospam at yahoo.com Thu Jan 15 11:22:38 2015 From: mlnospam at yahoo.com (ML mail) Date: Thu, 15 Jan 2015 11:22:38 +0000 (UTC) Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: References: Message-ID: <520997517.1542791.1421320958387.JavaMail.yahoo@jws10663.mail.bf1.yahoo.com> Thank you for your helpful hints on debugging this issue. I wanted first to get the mail which generates this error but unfortunately the user already deleted it as well from his trash. Now I got another hint and it looks like this mail had a big attachment to it. Is it possible that spamc generated this error due to the size of the mail? Regards ML On Thursday, January 15, 2015 8:29 AM, Steffen Kaiser wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 15 Jan 2015, ML mail wrote: It would be better to have the original question & discussion at the top, so one could cut and read nicely, > On Wednesday, January 14, 2015 9:06 PM, Pascal Volk wrote: > On 01/14/2015 03:05 PM, ML mail wrote: > >> Hello, >> >> I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: >> >> [SERVERBUG] failed to send mail > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 [cut] > plugin { > antispam_backend = pipe > antispam_pipe_program = /usr/bin/spamc > antispam_pipe_program_args = -d;my-mx-server.domain.com;-u;amavis > antispam_pipe_program_notspam_args = -L;ham > antispam_pipe_program_spam_args = -L;spam > antispam_pipe_tmpdir = /tmp > antispam_signature = X-Spam-Flag > antispam_signature_missing = error > antispam_spam = INBOX.Spam;INBOX.Junk > antispam_trash = INBOX.trash;trash;INBOX.Trash;Trash;INBOX.Deleted Items;Deleted Items;INBOX.Deleted Messages;Deleted Messages > antispam_verbose_debug = 1 What does the Plugin log? [cut] > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = vmail > } You seem to use a virtual user configuration, so the antispam plugin executes spamc as user vmail, correct? What happens if you run the program manually? sudo -u vmail /usr/bin/spamc -d my-mx-server.domain.com -u amavis -L ham \ < message You can also trace the program with a wrapper script. Replace antispam_pipe_program with /usr/local/bin/spamc-wrapper ==== BEGIN /usr/local/bin/spamc-wrapper #/bin/bash ( # when and who date;id; let i=0 # what echo /usr/bin/spamc "$@" # see embedded spaces in arguments for arg; do let i=i+1 echo "Arg#$i: '$arg'" done # call original program /usr/bin/spamc "$@" # log return code / exit code rc=$? echo rc=$rc # Make sure the rc is returned back to caller exit $rc # log everything into a file ) >>/tmp/spamc-wrapper.log 2>&1 ==== END If you have lots of simultaneous calls, create one log file per call -> add .$$ to filename. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLdsenz1H7kL/d9rAQI5wQf/R/8IIWAgcLX8061FBI5gFxEH8jP8UVMb MoSkkRT88teQJYuDxjr8zA92MX/2HDruRstzwxgJ1WxPefOvETpzGs1wrFqtWABM 2qB/ENfmkyOgmvpfsX5j6armYgOGEK0j5lf/ulV2i110wAJqjWY+9hBekFs7g4Th j29D42kv5Tl0XVwJbbYfzH/gUB+kyLxV7ja3dWWmmRRNV9am4Du36zua5AB2BzYq kuSqXfxClbnRYA/Ajy8H1KYhmx8wRtjkNijxt8B7R5f04E8hLrFVd5lKZIieuO6e oNu45xHeW3mYas0I3jWBf0u5pt1XlP7RLOtdB3D15CsW42PPsDcKxw== =Beji -----END PGP SIGNATURE----- From dave at icdsoft.com Thu Jan 15 14:06:22 2015 From: dave at icdsoft.com (David Davidov) Date: Thu, 15 Jan 2015 16:06:22 +0200 Subject: How to quote the special wildcard characters '*' and '?' Message-ID: <54B7C95E.2080702@icdsoft.com> Hello, I am searching for a way to execute doveadm command against mailbox containing a special wildcard character in its name. For example: # doveadm expunge -u 'me?@example.com' mailbox 'INBOX.Junk Mail' SAVEDBEFORE 7 The result is: doveadm(root): Error: User listing returned failure doveadm: Error: Failed to iterate through some users with exit code 75 In the mail log: dovecot: auth: Error: Trying to iterate users, but userdbs don't support it To be clear: the question mark is a part of the mailbox's name. Not an attempt to use it as wildcard. Is there a way 1. To quote the special characters or 2. To disable wildcard matcing in the -u parameter regards, -- David Davidov | System Administrator | ICDSoft Ltd. From gdrub13 at gmail.com Thu Jan 15 17:39:15 2015 From: gdrub13 at gmail.com (rub zorghy) Date: Thu, 15 Jan 2015 18:39:15 +0100 Subject: Questions regarding imaptest In-Reply-To: References: Message-ID: Sorry, I forgot to specify my workload : ./imaptest-1.0.0/src/imaptest secs=300 clients=800 seed=123 userfile=/tmp/userlist_imaptest no_tracking mbox=/tmp/dovecot-crlf with dovecot-crlf : http://www.dovecot.org/tmp/dovecot-crlf Kind regards, rub 2015-01-14 15:50 GMT+01:00 rub zorghy : > Hi, > > The measurements were created under the following conditions : > > ? operating system : Red Hat Enterprise Linux Server release 6.6 > (Santiago) kernel in version 2.6.32-504.el6.x86_64 > ? virtual server (VMware) with an Intel(R) Xeon(R) 4vCPU E5649 x 2,526 GHz > (2 cores per virtual socket) and 4 GB RAM > ? 7,200 RPM SATA 1TB (FC SAN IBM System Storage N3400) > ? all file systems had been formated in ext4 > ? Dovecot v.2.0.9 > ? imaptest v.1.0.0 > ? LDAP 389 Directory Server v.1.2.2 > ? free pagecache, dentries and inodes: echo 3 > /proc/sys/vm/drop_caches > and services restarted before the test > > "Tuning" based upon the book ? *Dovecot: POP3/IMAP servers for > enterprises and ISPs* ? Peer Heinlein : > > ---%<------------------------------------------ > > imap {} (10-master.conf). > service imap { > [...] > process_limit = 8192 > [...] > } > > auth {} > service auth { > [...] > client_limit = 9516 > [...] > } > > process_min_avail = 16 > > ---%<------------------------------------------ > > mail_max_userip_connections = 100 > > The test results shows : > > Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo > 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% > 30% 5% > 324 142 128 263 299 405 92 153 320 336 335 800/800 > 212 92 92 182 166 233 59 87 178 201 202 800/800 > - 4051 stalled for 16 secs in command: 8 APPEND "INBOX" () {3251+} > 315 160 134 300 275 400 85 136 300 317 317 800/800 > - 4051 stalled for 17 secs in command: 8 APPEND "INBOX" () {3251+} > 247 130 163 292 251 366 99 115 239 240 240 800/800 > 222 122 117 234 279 383 112 147 263 222 222 800/800 > 188 93 94 177 168 232 65 87 162 196 195 800/800 > 290 126 119 260 262 383 102 126 283 282 283 800/800 > 240 125 107 243 259 353 102 130 253 256 256 800/800 > 276 158 144 291 280 400 101 149 266 261 261 800/800 > 254 95 89 189 235 313 78 118 243 253 253 800/800 > 82 829 926 861 722 593 455 628 1695 2252 4 ms/cmd avg > > Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo > 100% 50% 50% 100% 100% 100% 50% 100% 100% 100% 100% > 30% 5% > 148 59 55 126 119 174 43 64 130 147 147 800/800 > - 8770 stalled for 16 secs in command: 4 FETCH 1:2 (INTERNALDATE UID > INTERNALDATE) > 261 128 139 247 252 369 101 125 258 269 269 800/800 > 302 159 158 307 282 398 109 126 296 298 297 800/800 > 234 96 118 239 270 369 98 138 357 428 427 800/800 > 361 213 220 450 383 596 151 197 240 165 166 800/800 > 192 93 92 206 256 352 104 118 297 289 290 800/800 > - 9806 stalled for 16 secs in command: 4 FETCH 1 (RFC822.SIZE) > 245 85 72 162 128 190 52 73 157 174 173 800/800 > - 9806 stalled for 17 secs in command: 4 FETCH 1 (RFC822.SIZE) > 294 152 165 348 360 526 136 179 313 308 309 800/800 > 272 87 79 174 164 229 54 85 218 230 230 800/800 > 218 152 152 310 325 467 117 152 265 236 221 798/800 > 119 830 852 859 707 566 511 702 1672 2161 6 ms/cmd avg > > I have three questions : > > 1) how to interpret each row except the last one ? > > 2) how to explain the stalled commands ? > > 3) why benchmark shows too few read disk I/O ? > > Thanks so much, > > rub > > From chayes at afo.net Thu Jan 15 22:42:10 2015 From: chayes at afo.net (Cliff Hayes) Date: Thu, 15 Jan 2015 16:42:10 -0600 Subject: pigeonhole - how to whitelist Message-ID: <54B84242.6090907@afo.net> Hello, When new users are added we start them with a spam rule that routes spam to their junk folder. I don't see a way to assign priority ... so how does a user whitelist a spam-flagged email? Are the rules applied in some order? Alphabetically perhaps? If so I can name the spam rule z-spam. Thanks in advance. From skdovecot at smail.inf.fh-brs.de Fri Jan 16 07:33:28 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 16 Jan 2015 08:33:28 +0100 (CET) Subject: pigeonhole - how to whitelist In-Reply-To: <54B84242.6090907@afo.net> References: <54B84242.6090907@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 15 Jan 2015, Cliff Hayes wrote: > When new users are added we start them with a spam rule that routes spam to > their junk folder. I don't see a way to assign priority ... so how does a > user whitelist a spam-flagged email? Are the rules applied in some order? > Alphabetically perhaps? If so I can name the spam rule z-spam. rules do have exactly one order, in which they appear in the Sieve script. But you certainly mean something different. Maybe a particular Sieve front-end, that assembles the Sieve script together? See, http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#multiscript There is one personal script the user may change and you can define one or more scripts to be executed before or after the personal script. So, if this would be pigeonhole problem, you define the spam processing in an "after" global script and let have the user whitelist a message, the personal script must file the message somewhere and stop script processing, see the paragraphe after "sieve_after = ". - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLi+yHz1H7kL/d9rAQL7VwgAnJyDcjCccum3681zpRl7wwm7BWgJq/9D jYGTOg162a/MO1nCcJTV+D0jETe4eaLe7QLLbYrhHyjdOoeHk32w9fMmNtrFsDQS PnddE8o0xIxEquuabBbY5grx9KWKBoriZvaN6XbBh+kC+GxAQWkZ8P+4WA5NHZCc /FbwD/3Nf5C7rZbujgkxLdhaGD+pb9EfE9+fq6WZD8+/avU/Gfm91N1H0a/I5vGf OgeErUwBH35iA0Z++cCv7tT7i4stwHAyF12LVnr9uQQE4XtDXAgQjzzeC/eY008b iyB0+i3edeR6peCh+MJ7NIn3ptNEilf8jHAfv5WrnCtRM9uSZvJPmQ== =8CZj -----END PGP SIGNATURE----- From mlnospam at yahoo.com Fri Jan 16 08:46:19 2015 From: mlnospam at yahoo.com (ML mail) Date: Fri, 16 Jan 2015 08:46:19 +0000 (UTC) Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: <520997517.1542791.1421320958387.JavaMail.yahoo@jws10663.mail.bf1.yahoo.com> References: <520997517.1542791.1421320958387.JavaMail.yahoo@jws10663.mail.bf1.yahoo.com> Message-ID: <161978983.2077751.1421397979208.JavaMail.yahoo@jws10633.mail.bf1.yahoo.com> Thanks to your help Steffen I was able to find out the issue which was simply the size of the Spam mail as you can see here: spamc[16545]: skipped message, greater than max message size (512000 bytes) The spam mail was around 900 kbytes as such I have changed the spamc limit to 1MB. Bastard spammer who abuses this limit by attaching a big image... Regards ML On Thursday, January 15, 2015 12:23 PM, ML mail wrote: Thank you for your helpful hints on debugging this issue. I wanted first to get the mail which generates this error but unfortunately the user already deleted it as well from his trash. Now I got another hint and it looks like this mail had a big attachment to it. Is it possible that spamc generated this error due to the size of the mail? Regards ML On Thursday, January 15, 2015 8:29 AM, Steffen Kaiser wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 15 Jan 2015, ML mail wrote: It would be better to have the original question & discussion at the top, so one could cut and read nicely, > On Wednesday, January 14, 2015 9:06 PM, Pascal Volk wrote: > On 01/14/2015 03:05 PM, ML mail wrote: > >> Hello, >> >> I am using the antispam plugin of Dovecot with SpamAssassin and in some cases when users move back mails from the Spam folder to their INBOX (false positive) they get the following error message: >> >> [SERVERBUG] failed to send mail > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 ext4 [cut] > plugin { > antispam_backend = pipe > antispam_pipe_program = /usr/bin/spamc > antispam_pipe_program_args = -d;my-mx-server.domain.com;-u;amavis > antispam_pipe_program_notspam_args = -L;ham > antispam_pipe_program_spam_args = -L;spam > antispam_pipe_tmpdir = /tmp > antispam_signature = X-Spam-Flag > antispam_signature_missing = error > antispam_spam = INBOX.Spam;INBOX.Junk > antispam_trash = INBOX.trash;trash;INBOX.Trash;Trash;INBOX.Deleted Items;Deleted Items;INBOX.Deleted Messages;Deleted Messages > antispam_verbose_debug = 1 What does the Plugin log? [cut] > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = vmail > } You seem to use a virtual user configuration, so the antispam plugin executes spamc as user vmail, correct? What happens if you run the program manually? sudo -u vmail /usr/bin/spamc -d my-mx-server.domain.com -u amavis -L ham \ < message You can also trace the program with a wrapper script. Replace antispam_pipe_program with /usr/local/bin/spamc-wrapper ==== BEGIN /usr/local/bin/spamc-wrapper #/bin/bash ( # when and who date;id; let i=0 # what echo /usr/bin/spamc "$@" # see embedded spaces in arguments for arg; do let i=i+1 echo "Arg#$i: '$arg'" done # call original program /usr/bin/spamc "$@" # log return code / exit code rc=$? echo rc=$rc # Make sure the rc is returned back to caller exit $rc # log everything into a file ) >>/tmp/spamc-wrapper.log 2>&1 ==== END If you have lots of simultaneous calls, create one log file per call -> add .$$ to filename. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLdsenz1H7kL/d9rAQI5wQf/R/8IIWAgcLX8061FBI5gFxEH8jP8UVMb MoSkkRT88teQJYuDxjr8zA92MX/2HDruRstzwxgJ1WxPefOvETpzGs1wrFqtWABM 2qB/ENfmkyOgmvpfsX5j6armYgOGEK0j5lf/ulV2i110wAJqjWY+9hBekFs7g4Th j29D42kv5Tl0XVwJbbYfzH/gUB+kyLxV7ja3dWWmmRRNV9am4Du36zua5AB2BzYq kuSqXfxClbnRYA/Ajy8H1KYhmx8wRtjkNijxt8B7R5f04E8hLrFVd5lKZIieuO6e oNu45xHeW3mYas0I3jWBf0u5pt1XlP7RLOtdB3D15CsW42PPsDcKxw== =Beji -----END PGP SIGNATURE----- From h.reindl at thelounge.net Fri Jan 16 13:12:16 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 16 Jan 2015 14:12:16 +0100 Subject: [SERVERBUG] failed to send mail with SA and antispam plugin In-Reply-To: <161978983.2077751.1421397979208.JavaMail.yahoo@jws10633.mail.bf1.yahoo.com> References: <520997517.1542791.1421320958387.JavaMail.yahoo@jws10663.mail.bf1.yahoo.com> <161978983.2077751.1421397979208.JavaMail.yahoo@jws10633.mail.bf1.yahoo.com> Message-ID: <54B90E30.9000604@thelounge.net> Am 16.01.2015 um 09:46 schrieb ML mail: > Thanks to your help Steffen I was able to find out the issue which was simply the size of the Spam mail as you can see here: > > spamc[16545]: skipped message, greater than max message size (512000 bytes) > > The spam mail was around 900 kbytes as such I have changed the spamc limit to 1MB. Bastard spammer who abuses this limit by attaching a big image... that's common for years now, even as we used a Barracuda Networks device where you need to add &expert=1 in the extended settings to raise that limit given that only a small amount makes it to SA here i increased that to 5 MB, in case of large images there is no performance impact, only large messages with most plaintext are ressource hungry to scan in fact i have seen such bastards attach 2 MB images to the typical spam mailbody to bypass scanners -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From moiseev at mezonplus.ru Fri Jan 16 14:03:29 2015 From: moiseev at mezonplus.ru (Alexander Moisseev) Date: Fri, 16 Jan 2015 17:03:29 +0300 Subject: dsync-remote: Error: Failed to set attribute Message-ID: <54B91A31.2000306@mezonplus.ru> 1. I am looking for backup solution for mdbox with SIS and experimenting with dsync. I am planning to do it like that: doveadm backup [deSISing] => transmitting to remote host => dsync-server [SISing] Is there more convenient method? 2. I have dovecot 2.2.15 on both hosts. The local host is a mail server. Dovecot on the remote host neither running nor configured (I have created an empty dovecot.conf in the config dir). When I tried to back up mailbox from local to remote host with command doveadm backup -P -u user at domain.tld \ ssh -c blowfish -i /tmp/vmail/.ssh/id_dsa vmail at remotehost \ doveadm \ -o mail_location=mdbox:/tmp/vmail/domain.tld/user/mdbox \ -o mail_attachment_dir=/tmp/vmail/attachments \ dsync-server I got error messages: dsync-remote(vmail): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/server/sieve/files/roundcube: Internal attributes cannot be changed directly dsync-remote(vmail): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/server/sieve/files/main: Internal attributes cannot be changed directly dsync-remote(vmail): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/server/sieve/files/test: Internal attributes cannot be changed directly dsync-remote(vmail): Error: Mailbox INBOX: Failed to set attribute vendor/vendor.dovecot/pvt/server/sieve/default: Internal attributes cannot be changed directly What do that errors mean? Does dsync try to replicate sieve scripts? -- Alexander From chayes at afo.net Fri Jan 16 18:45:35 2015 From: chayes at afo.net (Cliff Hayes) Date: Fri, 16 Jan 2015 12:45:35 -0600 Subject: pigeonhole - how to whitelist In-Reply-To: References: <54B84242.6090907@afo.net> Message-ID: <54B95C4F.9050906@afo.net> Thanks. That's exactly what I needed. However I have a permission problem. I added the parameter to 90-sieve.conf and created the directory but now I get the following permissions errors in maillog even though I have the file and directory wide open with 777 permissions: Error: yY/0JHtauVQfPgAAU+Cu/Q: sieve: failed to open sieve dir: stat(/var/lib/dovecot/sieve/after.d/) failed: Permission denied (euid=526(cliffhayes) egid=12(mail) missing +x perm: /var/lib/dovecot, euid is not dir owner) On 1/16/2015 1:33 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Thu, 15 Jan 2015, Cliff Hayes wrote: > >> When new users are added we start them with a spam rule that routes >> spam to their junk folder. I don't see a way to assign priority ... >> so how does a user whitelist a spam-flagged email? Are the rules >> applied in some order? Alphabetically perhaps? If so I can name the >> spam rule z-spam. > > rules do have exactly one order, in which they appear in the Sieve script. > > But you certainly mean something different. Maybe a particular Sieve > front-end, that assembles the Sieve script together? > > See, http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#multiscript > > There is one personal script the user may change and you can define one > or more scripts to be executed before or after the personal script. So, > if this would be pigeonhole problem, you define the spam processing in > an "after" global script and let have the user whitelist a message, the > personal script must file the message somewhere and stop script > processing, see the paragraphe after "sieve_after = ". > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVLi+yHz1H7kL/d9rAQL7VwgAnJyDcjCccum3681zpRl7wwm7BWgJq/9D > jYGTOg162a/MO1nCcJTV+D0jETe4eaLe7QLLbYrhHyjdOoeHk32w9fMmNtrFsDQS > PnddE8o0xIxEquuabBbY5grx9KWKBoriZvaN6XbBh+kC+GxAQWkZ8P+4WA5NHZCc > /FbwD/3Nf5C7rZbujgkxLdhaGD+pb9EfE9+fq6WZD8+/avU/Gfm91N1H0a/I5vGf > OgeErUwBH35iA0Z++cCv7tT7i4stwHAyF12LVnr9uQQE4XtDXAgQjzzeC/eY008b > iyB0+i3edeR6peCh+MJ7NIn3ptNEilf8jHAfv5WrnCtRM9uSZvJPmQ== > =8CZj > -----END PGP SIGNATURE----- > From mc at aiguphonie.com Fri Jan 16 22:57:48 2015 From: mc at aiguphonie.com (Martin =?utf-8?Q?Carl=C3=A9?=) Date: Sat, 17 Jan 2015 00:57:48 +0200 Subject: dsync SSL fails since 2.2.15 Message-ID: Hello list, dovecot ran rock-solid on OSX Mavericks for about 1 year replicating my mail between 2 servers via dsync with SSL as that is well described here: http://wiki2.dovecot.org/Replication After upgrading to 2.2.15, dsync gets stuck with the Error: "Received invalid SSL certificate" even though neither any of the dovecot configs nor the certs, keys or the CA have changed! When I simply outcomment SSL and switch dsync to use tcp (instead of tcps) everthing replications still works like a charm. Please help me to get SSL back working! I did a lot of testing and come up with a concrete QUESTION below, hopefully leading the way out of this trap. What happend ============= 2 days before I upgraded one of the machines to OSX Yosemite. Along with this, I also upgraded to dovecot 2.2.15 via homebrew (unfortunately on both machines at once). During this process, also openssl was updated to "OpenSSL 1.0.1k 8 Jan 2015". If checking the unchanged certs against the CA, however, the results are still "OK". 1st check: OK ========== sudo /usr/bin/openssl verify -CAfile /etc/ssl/ca/dovecotCA.pem /etc/ssl/certs/dovecot_on27_signed_cert.pem Password: /etc/ssl/certs/dovecot_on27_signed_cert.pem: OK 2nd check: OK (providing the CAfile and connecting to the doveadm_port) =========== openssl s_client -CAfile /etc/ssl/ca/dovecotCA.pem -connect on27.linkpc.net:8082 CONNECTED(00000003) depth=1 CN = dovecotCA2, O = dovecot, OU = dovecot, ST = dovecot, C = AF, L = dovecot, emailAddress = mc at aiguphonie.com verify return:1 depth=0 CN = on27.linkpc.net, O = dovecot, OU = dovecot, ST = dovecot, C = AF, L = dovecot, emailAddress = mc at aiguphonie.com verify return:1 --- Certificate chain 0 s:/CN=on27.linkpc.net/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com i:/CN=dovecotCA2/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com --- Server certificate -----BEGIN CERTIFICATE----- dmVjb3RDQTIxEDAOBgNVBAoMB2RvdmVjb3QxEDAOBgNVBAsMB2RvdmVjb3QxEDAO [...] +g== -----END CERTIFICATE----- subject=/CN=on27.linkpc.net/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com issuer=/CN=dovecotCA2/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com --- No client certificate CA names sent --- SSL handshake has read 1709 bytes and written 487 bytes --- New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384 Server public key is 2048 bit Secure Renegotiation IS supported Compression: zlib compression Expansion: zlib compression SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES256-GCM-SHA384 Session-ID: C4DDBA1FA50039FA5D94EF2359BA037B3903D66B6B637CA0733A9216BFCC3996 Session-ID-ctx: Master-Key: 0495D21CA11AA54856D78B48C3DBE9B70EFFB65F13224B430D2B4B2F80F12BE5A89F31454F9577F22F5DDC26FDBAAFAC Key-Arg : None PSK identity: None PSK identity hint: None SRP username: None TLS session ticket lifetime hint: 300 (seconds) TLS session ticket: [...] 0090 - 2d 97 37 15 bd a9 be 68-c1 79 fa dd d8 75 76 3f -.7....h.y...uv? Compression: 1 (zlib compression) Start Time: 1421443766 Timeout : 300 (sec) Verify return code: 0 (ok) --- - Yet, testing dsync yields: ERROR ========================================================================================== sudo -u _vmail doveadm -v sync -u test tcps:on27.linkpc.net Password: doveadm(test): Info: Received invalid SSL certificate: certificate signature failure: /CN=on27.linkpc.net/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com doveadm(test): Error: doveadm server disconnected before handshake: Received invalid SSL certificate: certificate signature failure: /CN=on27.linkpc.net/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com doveadm(test): Fatal: Disconnected from remote: Received invalid SSL certificate: certificate signature failure: /CN=on27.linkpc.net/O=dovecot/OU=dovecot/ST=dovecot/C=AF/L=dovecot/emailAddress=mc at aiguphonie.com QUESTION ========= So the question clearly is, how does dovecot check the cert against the CA exactly? Is there a call to the openssl cmd or is the library linked into dovecotadm? If liked, what version is used and how can I possibly change it? or: What's wrong with my CA and cert(s) all of a sudden? How can I create new CA for two certs fitting the (new) needs of dovecotadm? THANK YOU! ========================================================================================== Here are my full but rather simple configs of both machines: ========================================================================================== 1st machine: Yosemite # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 14.0.0 x86_64 base_dir = /var/run/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull doveadm_password = secret doveadm_port = 8082 log_path = /usr/local/var/log/dovecot/error mail_home = /var/vmail/%n mail_location = maildir:~/mail mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = " notify replication" namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot_authDBs/authDBs_on27/passwd.dovecot driver = passwd-file } plugin { mail_replica = tcps:nephelism.linkpc.net replication_full_sync_interval = 1 hour } protocols = imap service aggregator { fifo_listener replication-notify-fifo { user = _vmail } unix_listener replication-notify { user = _vmail } } service auth { unix_listener auth-userdb { group = _vmail mode = 0666 user = _vmail } } service doveadm { inet_listener { port = 8082 ssl = yes } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 user = _vmail } } ssl = required ssl_cert = From tlx at leuxner.net Sat Jan 17 09:51:27 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 17 Jan 2015 10:51:27 +0100 Subject: Changeset c02f29ca104d badly breaks LMTP/mdbox Message-ID: <20150117095127.GA55916@nihlus.leuxner.net> $ gdb /usr/lib/dovecot/lmtp core_lmtp.48883.nihlus.leuxner.net GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/lib/dovecot/lmtp...Reading symbols from /usr/lib/debug/usr/lib/dovecot/lmtp...done. done. [New LWP 48883] warning: Can't read pathname for load map: Input/output error. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `dovecot/lmtp'. Program terminated with signal 6, Aborted. #0 0x00007f0ca373b165 in raise () from /lib/x86_64-linux-gnu/libc.so.6 (gdb) bt full #0 0x00007f0ca373b165 in raise () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #1 0x00007f0ca373e3e0 in abort () from /lib/x86_64-linux-gnu/libc.so.6 No symbol table info available. #2 0x00007f0ca3b0ef85 in default_fatal_finish (type=, status=status at entry=0) at failures.c:202 backtrace = 0x1385320 "/usr/lib/dovecot/libdovecot.so.0(+0x79f8f) [0x7f0ca3b0ef8f] -> /usr/lib/dovecot/libdovecot.so.0(+0x79fee) [0x7f0ca3b0efee] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0ca3abebb5] -> /usr/lib/d"... #3 0x00007f0ca3b0efee in i_internal_fatal_handler (ctx=0x7fffdfafecb0, format=, args=) at failures.c:666 status = 0 #4 0x00007f0ca3abebb5 in i_panic (format=format at entry=0x7f0ca3e61438 "file %s: line %d (%s): assertion failed: (%s)") at failures.c:276 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0, timestamp_usecs = 0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fffdfafeda0, reg_save_area = 0x7fffdfafece0}} #5 0x00007f0ca3dc1d7a in mdbox_map_append_finish (ctx=0x142ac90) at mdbox-map.c:1153 appends = 0x0 last = 0x142b1d0 count = cur_offset = __FUNCTION__ = "mdbox_map_append_finish" #6 0x00007f0ca3dc3d08 in mdbox_save_finish_write (_ctx=0x142ab50) at mdbox-save.c:205 _data_stack_cur_id = 6 ctx = 0x142ab50 mail = 0x142c4a0 #7 mdbox_save_finish (ctx=0x142ab50) at mdbox-save.c:228 ret = 21152928 #8 0x00007f0ca3e070e2 in mailbox_save_finish (_ctx=_ctx at entry=0x7fffdfafee88) at mail-storage.c:2145 _data_stack_cur_id = 5 ctx = 0x142ab50 t = 0x1426a80 keywords = 0x0 pvt_flags = 0 copying_via_save = true ret = __FUNCTION__ = "mailbox_save_finish" #9 0x00007f0ca3dfec32 in mail_storage_copy (ctx=0x0, ctx at entry=0x142ab50, mail=mail at entry=0x13eaf20) at mail-copy.c:107 No locals. #10 0x00007f0ca3dc42bf in mdbox_copy (_ctx=0x142ab50, mail=0x13eaf20) at mdbox-save.c:444 ctx = 0x142ab50 save_mail = 0x13eaf20 src_mbox = rec = {map_uid = 21146464, save_date = 0} guid_data = 0x142ab68 wanted_guid = "\360=B\001\000\000\000\000\340\357E\242\f\177\000" #11 0x00007f0ca245f21b in acl_copy (ctx=0x142ab50, mail=0x13eaf20) at acl-mailbox.c:435 t = abox = 0x1423df0 #12 0x00007f0ca3e0728d in mailbox_copy (_ctx=_ctx at entry=0x7fffdfafefa8, mail=mail at entry=0x13eaf20) at mail-storage.c:2215 _data_stack_cur_id = 4 ctx = 0x142ab50 t = 0x1426a80 keywords = 0x0 pvt_flags = 0 backend_mail = 0x13eaf20 ret = __FUNCTION__ = "mailbox_copy" #13 0x00007f0ca17ae91e in act_store_execute (action=, aenv=0x1421d60, tr_context=0x14220f8) at sieve-actions.c:584 trans = 0x14220f8 mail = 0x13eaf20 save_ctx = 0x0 keywords = 0x0 backends_equal = status = 1 #14 0x00007f0ca17a5ca5 in sieve_result_transaction_execute (first=, result=) at sieve-result.c:1172 act = 0x1422088 rsef = sef = rac = 0x1422088 status = #15 sieve_result_execute (result=0x1421d30, keep=keep at entry=0x0) at sieve-result.c:1387 status = result_status = first_action = 0x1422088 last_action = 0x0 implicit_keep = true ret = #16 0x00007f0ca17b5e45 in sieve_execute (sbin=, msgdata=, senv=, ehandler=ehandler at entry=0x140cba0, flags=flags at entry=SIEVE_RUNTIME_FLAG_NOGLOBAL, keep=keep at entry=0x0) at sieve.c:560 result = 0x1421d30 ret = 1 #17 0x00007f0ca1a113f5 in lda_sieve_singlescript_execute (srctx=0x7fffdfaff170) at lda-sieve-plugin.c:491 sbin = 0x14196a0 rtflags = SIEVE_RUNTIME_FLAG_NOGLOBAL debug = false svinst = 0x14092e0 script = 0x140d3c0 user_script = cpflags = SIEVE_COMPILE_FLAG_NOGLOBAL ret = ehandler = 0x140cba0 error = #18 lda_sieve_execute (storage_r=0x7fffdfaff338, srctx=0x7fffdfaff170) at lda-sieve-plugin.c:927 msgdata = {mail = 0x13eaf20, return_path = 0x13b63c8 "bounces+20150117-lists=leuxner.net at packages.qa.debian.org", orig_envelope_to = 0x13b6c40 "tlx at leuxner.net", final_envelope_to = 0x13b6c40 "tlx at leuxner.net", auth_user = 0x13efc38 "tlx at leuxner.net", id = 0x13eb8a0 "<1421485513.5048.5.camel at kaspar.in>"} estatus = {last_storage = 0x13f9cb0, message_saved = 0, message_forwarded = 0, tried_default_save = 0, keep_original = 0, store_failed = 0} debug = ret = mdctx = 0x7fffdfaff360 svinst = scriptenv = {action_log_format = 0x13d1c98 "msgid=%m: %$", user = 0x13efb50, default_mailbox = 0x4094dc "INBOX", postmaster_address = 0x13b5740 "postmaster at spectre.leuxner.net", mailbox_autocreate = false, mailbox_autosubscribe = false, script_context = 0x7fffdfaff360, smtp_start = 0x7f0ca1a10190 , smtp_add_rcpt = 0x7f0ca1a10180 , smtp_send = 0x7f0ca1a10170 , smtp_finish = 0x7f0ca1a10160 , duplicate_check = 0x7f0ca1a10110 , duplicate_mark = 0x7f0ca1a10140 , duplicate_flush = 0x7f0ca1a10100 , reject_mail = 0x7f0ca1a100f0 , exec_status = 0x7fffdfaff0f0, trace_stream = 0x0, trace_config = {level = SIEVE_TRLVL_NONE, flags = 0}} #19 lda_sieve_deliver_mail (mdctx=, storage_r=0x7fffdfaff338) at lda-sieve-plugin.c:986 _data_stack_cur_id = 3 srctx = {svinst = 0x14092e0, mdctx = 0x7fffdfaff360, home_dir = 0x13f1708 "/var/vmail/domains/leuxner.net/tlx", scripts = 0x1384d28, script_count = 1, user_script = 0x140d3c0, main_script = 0x140d3c0, msgdata = 0x7fffdfaff100, scriptenv = 0x7fffdfaff1d0, user_ehandler = 0x140cba0, master_ehandler = 0x140ca90, userlog = 0x1384da8 "/var/vmail/domains/leuxner.net/tlx/.dovecot.sieve.log"} debug = svenv = {hostname = 0x13d1c18 "spectre.leuxner.net", domainname = 0x0, base_dir = 0x13f0bd0 "/var/run/dovecot", username = 0x13efc38 "tlx at leuxner.net", home_dir = 0x13f1708 "/var/vmail/domains/leuxner.net/tlx", temp_dir = 0x13f18d0 "/tmp", flags = SIEVE_FLAG_HOME_RELATIVE, location = SIEVE_ENV_LOCATION_MDA, delivery_phase = SIEVE_DELIVERY_PHASE_DURING} i = ret = 0 #20 0x00007f0ca40ad794 in mail_deliver (ctx=ctx at entry=0x7fffdfaff360, storage_r=storage_r at entry=0x7fffdfaff338) at mail-deliver.c:400 ret = #21 0x000000000040691c in client_deliver (session=0x13ef740, src_mail=0x13eaf20, rcpt=0x13b6440, client=0x13b5340) at commands.c:779 set_parser = line = str = mail_error = MAIL_ERROR_NONE ret = input = ns = sets = storage = 0x0 mail_set = username = dctx = {pool = 0x13ef720, set = 0x13d1ba8, session = 0x13ef740, dup_ctx = 0x1409210, session_id = 0x13b63b0 "pAQhD7wmulTzvgAAgUOSbA", src_mail = 0x13eaf20, src_envelope_sender = 0x13b63c8 "bounces+20150117-lists=leuxner.net at packages.qa.debian.org", dest_user = 0x13efb50, dest_addr = 0x13b6c40 "tlx at leuxner.net", final_dest_addr = 0x13b6c40 "tlx at leuxner.net", dest_mailbox_name = 0x4094dc "INBOX", dest_mail = 0x0, var_expand_table = 0x0, tempfail_error = 0x0, tried_default_save = false, saved_mail = false, save_dest_mail = false, mailbox_full = false, dsn = false} lda_set = 0x13d1ba8 error = #22 client_deliver_next (session=0x13ef740, src_mail=0x13eaf20, client=0x13b5340) at commands.c:822 rcpts = 0x13b6440 count = 1 #23 client_input_data_write_local (input=, client=0x13b5340) at commands.c:918 src_mail = 0x13eaf20 first_uid = 4294967295 session = 0x13ef740 old_uid = 0 #24 client_input_data_write (client=0x13b5340) at commands.c:1035 input = 0x13d0020 ret = true #25 client_input_data_handle (client=0x13b5340) at commands.c:1130 data = size = 543 ret = #26 0x00007f0ca3b2107b in io_loop_call_io (io=0x13b5b10) at ioloop.c:501 ioloop = 0x138a730 t_id = 2 __FUNCTION__ = "io_loop_call_io" #27 0x00007f0ca3b220fb in io_loop_handler_run_internal (ioloop=ioloop at entry=0x138a730) at ioloop-epoll.c:220 ctx = 0x138f5e0 events = 0xbef3 event = 0x1390460 list = 0x13b5b70 io = tv = {tv_sec = 299, tv_usec = 992673} events_count = msecs = ret = 1 i = call = __FUNCTION__ = "io_loop_handler_run_internal" #28 0x00007f0ca3b21109 in io_loop_handler_run (ioloop=ioloop at entry=0x138a730) at ioloop.c:548 No locals. #29 0x00007f0ca3b21188 in io_loop_run (ioloop=0x138a730) at ioloop.c:525 __FUNCTION__ = "io_loop_run" #30 0x00007f0ca3ac4083 in master_service_run (service=0x138a5c0, callback=callback at entry=0x405230 ) at master-service.c:566 No locals. #31 0x000000000040508c in main (argc=1, argv=0x138a390) at main.c:123 set_roots = {0x60b700, 0x409fa0, 0x0} service_flags = storage_service_flags = 675 c = -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From tim.odriscoll at lambrookschool.co.uk Sat Jan 17 17:29:29 2015 From: tim.odriscoll at lambrookschool.co.uk (Tim ODriscoll) Date: Sat, 17 Jan 2015 17:29:29 +0000 Subject: Shared mailboxes not shown or accessible Message-ID: <20150117172929.Horde.BDo2dEUERzW0MfTvnC-HFQ1@mail1.lambrookschool.co.uk> Dear All, I'm trying (and failing) to implement a shared mailbox folder. My goal is to share the inbox of a user 'invoices' to a test user 'rubbish'. My ACLs appear to be correct: # doveadm acl get -u invoices inbox ID Global Rights user=rubbish create delete expunge insert lookup post read write write-deleted write-seen The shared-dict is being generated: # cat /var/lib/dovecot/db/shared-mailboxes shared/shared-boxes/user/rubbish/invoices 1 Yet when I try to list the mailboxes, the shared namespace has no children: . login rubbish password . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA ACL RIGHTS=texk] Logged in . list "" "*" * LIST (\HasNoChildren \Trash) "/" Trash * LIST (\HasNoChildren \Drafts) "/" Drafts * LIST (\HasNoChildren \Sent) "/" Sent * LIST (\Noselect \HasNoChildren) "/" shared * LIST (\HasNoChildren) "/" INBOX . OK List completed. I've gone through all the documentation I can find but still can't sort this out. Any help would be greatly appreciated. Tim My dovecot config: # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-32-generic x86_64 Ubuntu 14.04 LTS xfs auth_mechanisms = plain login auth_username_format = %Ln default_vsz_limit = 384 M disable_plaintext_auth = no listen = * mail_access_groups = mail mail_location = maildir:/home/email/LHNET/%Lu/Maildir:INBOX=/home/email/LHNET/%Lu/Maildir mail_plugins = acl namespace { list = yes location = maildir:/home/email/LHNET/%%Lu/Maildir:INDEXPVT=/home/email/LHNET/%Lu/Maildir/shared/%%Lu:CONTROL=/home/email/LHNET/%Lu/Maildir/shared/%%Lu prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes quota = maildir:User quota quota_grace = 10%% quota_rule = *:storage=3G quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0666 user = postfix } unix_listener auth-userdb { group = mail mode = 0640 user = vmail } } service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 ssl = yes } } service imap { process_limit = 1024 vsz_limit = 384 M } ssl_cert = Hi All, I upgrade my mail server from doveot 1.1.7 to 2.2.13 and encounter problems with file locking issues. The server has around 400 clients using IMAP and I get tons of "Warning: fscking index file .. dovecot.index" and "Error: Corrupted transaction log". Sometimes the transaction log problem is followed by "Panic: file mail-transaction-log.c: line 350 (mail_transaction_log_set_mailbox_sync_pos): assertion failed: (file_offset >= log->head->saved_tail_offset)" I tried all offered locking methods (fcntl, flock, dotlock) without success. The server runs inside a virtual guest (linux-vserver which is similar to LXC) using an overlay filesystem. I suspect the overlayfs but have no idea how to test this, besides - as I understand - dotlock should not rely on the fs locking at all :( I even wonder why the old dovecot worked for nearly 10 years without any problems, even under higher loads. The config is pretty straightforward: # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.4.57-vs2.3.3.9-g67e6589-dirty x86_64 Gentoo Base System release 2.2 auth_cache_negative_ttl = 0 auth_cache_size = 50 M auth_mechanisms = plain cram-md5 debug_log_path = /var/log/dovecot-debug.log default_client_limit = 900 default_process_limit = 250 disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 info_log_path = /var/log/dovecot.log last_valid_gid = 89 last_valid_uid = 89 listen = * lock_method = dotlock login_greeting = Serverpilot.net Mail Service mail_gid = 89 mail_location = maildir:~/.maildir mail_uid = 89 namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 service imap-login { process_min_avail = 5 service_count = 0 } ssl_cert = From mail at oliwel.de Fri Jan 16 11:24:06 2015 From: mail at oliwel.de (Oliver Welter) Date: Fri, 16 Jan 2015 12:24:06 +0100 Subject: Outlook and TLSv.1 Message-ID: <54B8F4D6.9090500@oliwel.de> Hi Folks, after adding TLSv1.2 to by TLS options a lot of Outlook users complaint about connection errors, openssl s_client and Thunderbird works fine. I found some posts about this but none of them had a real solution on this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 ssl_cert = From sendmailroot at afo.net Fri Jan 16 19:43:16 2015 From: sendmailroot at afo.net (Sendmail Root) Date: Fri, 16 Jan 2015 13:43:16 -0600 Subject: pigeonhole - how to whitelist In-Reply-To: <54B95C4F.9050906@afo.net> References: <54B84242.6090907@afo.net> <54B95C4F.9050906@afo.net> Message-ID: <54B969D4.8020505@afo.net> I fixed it by moving the directory from the recommended /var/lib/dovecot/sieve/after.d/ to /tmp If that is not the recommended resolution, please advise. On 1/16/2015 12:45 PM, Cliff Hayes wrote: > Thanks. > That's exactly what I needed. > However I have a permission problem. > I added the parameter to 90-sieve.conf and created the directory but now > I get the following permissions errors in maillog even though I have the > file and directory wide open with 777 permissions: > > Error: yY/0JHtauVQfPgAAU+Cu/Q: sieve: failed to open sieve dir: > stat(/var/lib/dovecot/sieve/after.d/) failed: Permission denied > (euid=526(cliffhayes) egid=12(mail) missing +x perm: /var/lib/dovecot, > euid is not dir owner) > > > On 1/16/2015 1:33 AM, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Thu, 15 Jan 2015, Cliff Hayes wrote: >> >>> When new users are added we start them with a spam rule that routes >>> spam to their junk folder. I don't see a way to assign priority ... >>> so how does a user whitelist a spam-flagged email? Are the rules >>> applied in some order? Alphabetically perhaps? If so I can name the >>> spam rule z-spam. >> >> rules do have exactly one order, in which they appear in the Sieve >> script. >> >> But you certainly mean something different. Maybe a particular Sieve >> front-end, that assembles the Sieve script together? >> >> See, http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#multiscript >> >> There is one personal script the user may change and you can define one >> or more scripts to be executed before or after the personal script. So, >> if this would be pigeonhole problem, you define the spam processing in >> an "after" global script and let have the user whitelist a message, the >> personal script must file the message somewhere and stop script >> processing, see the paragraphe after "sieve_after = ". >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVLi+yHz1H7kL/d9rAQL7VwgAnJyDcjCccum3681zpRl7wwm7BWgJq/9D >> jYGTOg162a/MO1nCcJTV+D0jETe4eaLe7QLLbYrhHyjdOoeHk32w9fMmNtrFsDQS >> PnddE8o0xIxEquuabBbY5grx9KWKBoriZvaN6XbBh+kC+GxAQWkZ8P+4WA5NHZCc >> /FbwD/3Nf5C7rZbujgkxLdhaGD+pb9EfE9+fq6WZD8+/avU/Gfm91N1H0a/I5vGf >> OgeErUwBH35iA0Z++cCv7tT7i4stwHAyF12LVnr9uQQE4XtDXAgQjzzeC/eY008b >> iyB0+i3edeR6peCh+MJ7NIn3ptNEilf8jHAfv5WrnCtRM9uSZvJPmQ== >> =8CZj >> -----END PGP SIGNATURE----- >> > From michael at bigmichi1.de Sun Jan 18 08:15:49 2015 From: michael at bigmichi1.de (Michael Cramer) Date: Sun, 18 Jan 2015 08:15:49 +0000 Subject: Initial development work on fts-elasticsearch In-Reply-To: Message-ID: <20150118081549.Horde.tMFjhuWelFcYt7WIL1rglw1@horde.bigmichi1.de> Zitat von Joshua Atkins : > Hi all, > > I have started work on an ElasticSearch FTS plugin for Dovecot. > > It's available now on GitHub at: > > https://github.com/ascendantcom/fts-elasticsearch > > Please note that it is still in heavy development right now. It > supports indexing, searching and expunging with some caveats (no > multi-mailbox search or attachments). > > Pull requests, suggestions and bug reports very welcome! > > Cheers, > Josh nice work, but where can i found the fts-header files? dovecot-dev package doesn't include it. From rs at sys4.de Sun Jan 18 08:45:16 2015 From: rs at sys4.de (Robert Schetterer) Date: Sun, 18 Jan 2015 09:45:16 +0100 Subject: Outlook and TLSv.1 In-Reply-To: <54B8F4D6.9090500@oliwel.de> References: <54B8F4D6.9090500@oliwel.de> Message-ID: <54BB729C.1000106@sys4.de> Am 16.01.2015 um 12:24 schrieb Oliver Welter: > Hi Folks, > > after adding TLSv1.2 to by TLS options a lot of Outlook users complaint > about connection errors, openssl s_client and Thunderbird works fine. > > I found some posts about this but none of them had a real solution on > this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. > > I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 > > ssl_cert = ssl_cipher_list = ALL:!EXPORT:!LOW:!MEDIUM:!aNULL:+RC4:@STRENGTH > ssl_dh_parameters_length = 2048 > ssl_key = ssl_protocols = !SSLv2 !TLSv1.2 > > The certificate is from Comodo using sha256. > > Any idea? > > Oliver > there is no "Outlook", please do a exact debug what Outlook and Windows Version, disable TLSv1.2 is a bad idea, my bet goes on your ssl_cipher_list, try this # SSL ciphers to use ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL or search list archive and www for other better solutions and general dovecot ssl configs Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From h.reindl at thelounge.net Sun Jan 18 11:07:52 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 18 Jan 2015 12:07:52 +0100 Subject: Outlook and TLSv.1 In-Reply-To: <54B8F4D6.9090500@oliwel.de> References: <54B8F4D6.9090500@oliwel.de> Message-ID: <54BB9408.30001@thelounge.net> Am 16.01.2015 um 12:24 schrieb Oliver Welter: > after adding TLSv1.2 to by TLS options how did you do that? there is no need to add it as long you did not break your configuration intentional the time before > a lot of Outlook users complaint about connection errors, > openssl s_client and Thunderbird works fine. no > I found some posts about this but none of them had a real solution on > this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. > > I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 > > ssl_cert = ssl_cipher_list = ALL:!EXPORT:!LOW:!MEDIUM:!aNULL:+RC4:@STRENGTH !MEDIUM likely is the reason > ssl_dh_parameters_length = 2048 > ssl_key = ssl_protocols = !SSLv2 !TLSv1.2 > > The certificate is from Comodo using sha256 the confiig below works with every known Outlook version down to Outlook 2003 on Windows XP in combination with a RSA4096/SHA256 key as well as with all other reasonable mail clients ssl_protocols = !SSLv2 !SSLv3 ssl_prefer_server_ciphers = yes ssl_cipher_list = ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA:AES256-SHA:DHE-RSA-CAMELLIA128-SHA:DHE-RSA-CAMELLIA256-SHA:CAMELLIA128-SHA:CAMELLIA256-SHA:ECDHE-RSA-DES-CBC3-SHA:DES-CBC3-SHA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From jerry at seibercom.net Sun Jan 18 11:07:46 2015 From: jerry at seibercom.net (Jerry) Date: Sun, 18 Jan 2015 06:07:46 -0500 Subject: Outlook and TLSv.1 In-Reply-To: <54BB729C.1000106@sys4.de> References: <54B8F4D6.9090500@oliwel.de> <54BB729C.1000106@sys4.de> Message-ID: <20150118060746.654d81d6@scorpio> On Sun, 18 Jan 2015 09:45:16 +0100, Robert Schetterer stated: >Am 16.01.2015 um 12:24 schrieb Oliver Welter: >> Hi Folks, >> >> after adding TLSv1.2 to by TLS options a lot of Outlook users complaint >> about connection errors, openssl s_client and Thunderbird works fine. >> >> I found some posts about this but none of them had a real solution on >> this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. >> >> I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 >> >> ssl_cert = > ssl_cipher_list = ALL:!EXPORT:!LOW:!MEDIUM:!aNULL:+RC4:@STRENGTH >> ssl_dh_parameters_length = 2048 >> ssl_key = > ssl_protocols = !SSLv2 !TLSv1.2 >> >> The certificate is from Comodo using sha256. >> >> Any idea? >> >> Oliver >> >there is no "Outlook", please do a exact debug what Outlook and Windows >Version, disable TLSv1.2 is a bad idea, my bet goes on your >ssl_cipher_list, try this > ># SSL ciphers to use >ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > > >or search list archive and www for other better solutions and general >dovecot ssl configs I have: ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLv3:!EXP:!aNULL and Outlook 2013 works fine. -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Sun Jan 18 11:16:18 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 18 Jan 2015 12:16:18 +0100 Subject: Outlook and TLSv.1 In-Reply-To: <20150118060746.654d81d6@scorpio> References: <54B8F4D6.9090500@oliwel.de> <54BB729C.1000106@sys4.de> <20150118060746.654d81d6@scorpio> Message-ID: <54BB9602.3020507@thelounge.net> Am 18.01.2015 um 12:07 schrieb Jerry: > I have: > ssl_cipher_list = ALL:!LOW:!SSLv2:!SSLv3:!EXP:!aNULL > and Outlook 2013 works fine but you break *for sure* older clients and should *not* recommend that broken setup untested and believe you are helping with it !SSLv3 has no business in the cipher list you disable ciphers still valid for TLS that way that was all discussed dozenz of times here ssl_protocols = !SSLv2 !SSLv3 *remove !SSLv3 from teh cipher list* and if you now say your dovecot version don't support "ssl_protocols" then upgrade or just accept that you can't have outdated software and state of the art protovol support -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tim.odriscoll at lambrookschool.co.uk Sun Jan 18 14:21:44 2015 From: tim.odriscoll at lambrookschool.co.uk (Tim ODriscoll) Date: Sun, 18 Jan 2015 14:21:44 +0000 Subject: Shared mailboxes not shown or accessible In-Reply-To: <20150117172929.Horde.BDo2dEUERzW0MfTvnC-HFQ1@mail1.lambrookschool.co.uk> Message-ID: <20150118142144.Horde.7lANnl29IPGWXLeSmbNX8A1@mail1.lambrookschool.co.uk> On Sat, 17 Jan 2015, Tim ODriscoll wrote: > I've gone through all the documentation I can find but still can't > sort this out. Any help would be greatly appreciated. Obviously not *all* the documentation because I've recently found a piece about file/group/g+s permissions and that seems to have solved it. Tim From mlnospam at yahoo.com Sun Jan 18 21:34:27 2015 From: mlnospam at yahoo.com (ML mail) Date: Sun, 18 Jan 2015 21:34:27 +0000 (UTC) Subject: Still receiving Quota warning: mailbox 95% full altough now 20% full Message-ID: <1222593338.2997183.1421616867634.JavaMail.yahoo@jws106135.mail.bf1.yahoo.com> Hello I am using the quota warnings e-mail feature of dovecot to warn users at 80% and 95%. The issue here is that I have one use who his mailbox is now back down to 20% but he still receives the 95% mailbox full warning mail. Any ideas what could be wrong or what to do? My dovecot config of the mailbox server can be found here: http://article.gmane.org/gmane.mail.imap.dovecot/80823 Regards ML From bjlockie at lockie.ca Mon Jan 19 04:18:47 2015 From: bjlockie at lockie.ca (James) Date: Sun, 18 Jan 2015 23:18:47 -0500 Subject: move imap mail to new server Message-ID: <54BC85A7.5060703@lockie.ca> I tarred up my Maildir from one server and untarred it on another, I had hoped it would work, It partially does. It has all my messages but doesn't show the subdirectories in Thunderbird. Maildir# ls -l .INBOX.car total 44 drwx------ 2 rjl 1000 4096 Feb 15 2014 cur -rw------- 1 rjl users 17408 Jan 18 02:10 dovecot.index.cache -rw------- 1 rjl users 1040 Jan 18 02:10 dovecot.index.log -rw------- 1 rjl 1000 23 Aug 26 2011 dovecot-keywords -rw------- 1 rjl users 1683 Jan 18 00:20 dovecot-uidlist -rw------- 1 rjl 1000 0 Aug 4 2011 maildirfolder drwx------ 2 rjl 1000 4096 Aug 4 2011 new drwx------ 2 rjl 1000 4096 Feb 15 2014 tmp Maildir# ls -l .INBOX.car.Enterprise/ total 40 drwx------ 2 rjl 1000 4096 Jul 3 2013 cur -rw------- 1 rjl 1000 20480 Mar 20 2014 dovecot.index.cache -rw------- 1 rjl 1000 1824 Mar 20 2014 dovecot.index.log -rw------- 1 rjl 1000 212 Jul 10 2013 dovecot-uidlist -rw------- 1 rjl 1000 0 Jul 2 2013 maildirfolder drwx------ 2 rjl 1000 4096 Jul 2 2013 new drwx------ 2 rjl 1000 4096 Jul 5 2013 tmp Thunderbird displays the files under: .INBOX.car/cur/ but not .INBOX.car.Enterprise/cur Is there any way to rebuild the subdirectories? From skdovecot at smail.inf.fh-brs.de Mon Jan 19 07:56:32 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 19 Jan 2015 08:56:32 +0100 (CET) Subject: pigeonhole - how to whitelist In-Reply-To: <54B95C4F.9050906@afo.net> References: <54B84242.6090907@afo.net> <54B95C4F.9050906@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 16 Jan 2015, Cliff Hayes wrote: > That's exactly what I needed. > However I have a permission problem. > I added the parameter to 90-sieve.conf and created the directory but now I > get the following permissions errors in maillog even though I have the file > and directory wide open with 777 permissions: > > Error: yY/0JHtauVQfPgAAU+Cu/Q: sieve: failed to open sieve dir: > stat(/var/lib/dovecot/sieve/after.d/) failed: Permission denied > (euid=526(cliffhayes) egid=12(mail) missing +x perm: /var/lib/dovecot, euid ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > is not dir owner) missing x-perm on /var/lib/dovecot - --> chmod a+x /var/lib/dovecot if the group of /var/lib/dovecot is "mail" --> chmod g+x /var/lib/dovecot > > On 1/16/2015 1:33 AM, Steffen Kaiser wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Thu, 15 Jan 2015, Cliff Hayes wrote: >> >>> When new users are added we start them with a spam rule that routes >>> spam to their junk folder. I don't see a way to assign priority ... >>> so how does a user whitelist a spam-flagged email? Are the rules >>> applied in some order? Alphabetically perhaps? If so I can name the >>> spam rule z-spam. >> >> rules do have exactly one order, in which they appear in the Sieve script. >> >> But you certainly mean something different. Maybe a particular Sieve >> front-end, that assembles the Sieve script together? >> >> See, http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#multiscript >> >> There is one personal script the user may change and you can define one >> or more scripts to be executed before or after the personal script. So, >> if this would be pigeonhole problem, you define the spam processing in >> an "after" global script and let have the user whitelist a message, the >> personal script must file the message somewhere and stop script >> processing, see the paragraphe after "sieve_after = ". >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVLi+yHz1H7kL/d9rAQL7VwgAnJyDcjCccum3681zpRl7wwm7BWgJq/9D >> jYGTOg162a/MO1nCcJTV+D0jETe4eaLe7QLLbYrhHyjdOoeHk32w9fMmNtrFsDQS >> PnddE8o0xIxEquuabBbY5grx9KWKBoriZvaN6XbBh+kC+GxAQWkZ8P+4WA5NHZCc >> /FbwD/3Nf5C7rZbujgkxLdhaGD+pb9EfE9+fq6WZD8+/avU/Gfm91N1H0a/I5vGf >> OgeErUwBH35iA0Z++cCv7tT7i4stwHAyF12LVnr9uQQE4XtDXAgQjzzeC/eY008b >> iyB0+i3edeR6peCh+MJ7NIn3ptNEilf8jHAfv5WrnCtRM9uSZvJPmQ== >> =8CZj >> -----END PGP SIGNATURE----- >> > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLy4sHz1H7kL/d9rAQK3Vgf9Et+Nf5llzV1za9EFYTCIXMSrVOg8kEvP dPhmMCaaoQNzwhnwKIqUK3RAtwc+rjgzXyVvWm8K1oQ8rcDtB7MOJ0cWzz9/IMtT f430R7xsHd+Vp0tNk2WraaTLqYCWDBkgtyRPpUot56+O+c8AiehMgca3hfY8WTtk AFU+JeG28/eCdOnpC5Gm7oyj7OGrQNKIkhs7kwvpkun2ujhb0/lN3B229Og7nf2G 0xni44F5qQ6LWu7JrGQZRm36BF3Qa/eLB7UxDzUkzfuOJoZAwSHQi+KBFYlcsLOh 2dlqxogYdSJytfUtfAQNrmL/XVuhPFMNQujrIxB8tWky4xmbjOTTZA== =1xPS -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Jan 19 08:02:11 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 19 Jan 2015 09:02:11 +0100 (CET) Subject: Still receiving Quota warning: mailbox 95% full altough now 20% full In-Reply-To: <1222593338.2997183.1421616867634.JavaMail.yahoo@jws106135.mail.bf1.yahoo.com> References: <1222593338.2997183.1421616867634.JavaMail.yahoo@jws106135.mail.bf1.yahoo.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 18 Jan 2015, ML mail wrote: > I am using the quota warnings e-mail feature of dovecot to warn users at > 80% and 95%. The issue here is that I have one use who his mailbox is > now back down to 20% but he still receives the 95% mailbox full warning > mail. Any ideas what could be wrong or what to do? very that Dovecot uses the same values as you think: doveadm quota get -u "user" Also double check every limit. :-) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVLy6A3z1H7kL/d9rAQJB+gf+OT542n5ejRsrpX5iO8HmN6ZJQdk4s+p6 KjgotXnN3P9IJMQ0nM1Mvd6C4xeSqjP+DXtH08tHIxMZ+jRtXIpgq31+wr/Qs+FE l7oQtOrnycIucYMSwSNCVLejxcLLkFNKgG7ZpT109aY6Kb3mKGqDrDpo1xUEQg6c EHc5hXwAbe8bewFIpKNAnZXdKRuraIS/G4dwJU5v+nJXI2n1jYRqimZOB1W0l7Fh zxYOKTlKgxlBVX75v6rDCK+VRbdtoryyKUIEd+Lv/60XqDhpMRPvvcb96GgfeIpg 8HPesc9PPxVp4htFcoY9IteRdiVXpI1M8fq7/arXbYBvLCYQVA9LTA== =3cmu -----END PGP SIGNATURE----- From list_dovecot at bluerosetech.com Mon Jan 19 21:55:31 2015 From: list_dovecot at bluerosetech.com (Darren Pilgrim) Date: Mon, 19 Jan 2015 13:55:31 -0800 Subject: Outlook and TLSv.1 In-Reply-To: <54BB729C.1000106@sys4.de> References: <54B8F4D6.9090500@oliwel.de> <54BB729C.1000106@sys4.de> Message-ID: <54BD7D53.7010508@bluerosetech.com> On 1/18/2015 12:45 AM, Robert Schetterer wrote: > Am 16.01.2015 um 12:24 schrieb Oliver Welter: >> Hi Folks, >> >> after adding TLSv1.2 to by TLS options a lot of Outlook users complaint >> about connection errors, openssl s_client and Thunderbird works fine. >> >> I found some posts about this but none of them had a real solution on >> this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. >> >> I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 >> >> ssl_cert = > ssl_cipher_list = ALL:!EXPORT:!LOW:!MEDIUM:!aNULL:+RC4:@STRENGTH >> ssl_dh_parameters_length = 2048 >> ssl_key = > ssl_protocols = !SSLv2 !TLSv1.2 >> >> The certificate is from Comodo using sha256. >> >> Any idea? >> >> Oliver >> > > there is no "Outlook", please do a exact debug what Outlook and Windows > Version, disable TLSv1.2 is a bad idea, my bet goes on your > ssl_cipher_list, try this > > # SSL ciphers to use > ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL > > > or search list archive and www for other better solutions and general > dovecot ssl configs I have this in production: ssl_cipher_list = HIGH+kEECDH:HIGH+kEDH:!aNULL:-3DES:+AES256:+SHA:AES128-SHA:DES-CBC3-SHA ssl_protocols = !SSLv2 !SSLv3 TLSv1 TLSv1.1 TLSv1.2 - AES128-SHA & TLSv1 for some Android v4.3 and earlier - DES-CBC3-SHA & TLSv1 for Outlook 2003 on Windows XP - TLSv1 for Thunderbird prior to v27 - TLSv1 for Outlook on Windows Vista/2008 - TLSv1 for Outlook on Windows 7 or 8 without IE 11 installed Everything else supports at least DHE-AES on TLSv1.1 or 1.2. The cipherspec provides AES128, AES256 and Camellia; with AES128 and Camellia preferred over AES256, and SHA2 preferred over SHA1. From tss at iki.fi Tue Jan 20 05:42:01 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 19 Jan 2015 21:42:01 -0800 Subject: Changeset c02f29ca104d badly breaks LMTP/mdbox In-Reply-To: <20150117095127.GA55916@nihlus.leuxner.net> References: <20150117095127.GA55916@nihlus.leuxner.net> Message-ID: <4006587A-7DB4-4245-854D-2414171B39CB@iki.fi> Should be fixed now. I think I'm now done for a while with the new core development work that was needed, so things shouldn't break so easily for a while. Although the body.snippet stuff still needs some more work. From mail at oliwel.de Tue Jan 20 07:14:39 2015 From: mail at oliwel.de (Oliver Welter) Date: Tue, 20 Jan 2015 08:14:39 +0100 Subject: Outlook and TLSv.1 In-Reply-To: <54BD7D53.7010508@bluerosetech.com> References: <54B8F4D6.9090500@oliwel.de> <54BB729C.1000106@sys4.de> <54BD7D53.7010508@bluerosetech.com> Message-ID: <54BE005F.3030508@oliwel.de> Hi All, Am 19.01.2015 um 22:55 schrieb Darren Pilgrim: > On 1/18/2015 12:45 AM, Robert Schetterer wrote: >> Am 16.01.2015 um 12:24 schrieb Oliver Welter: >>> Hi Folks, >>> >>> after adding TLSv1.2 to by TLS options a lot of Outlook users complaint >>> about connection errors, openssl s_client and Thunderbird works fine. >>> >>> I found some posts about this but none of them had a real solution on >>> this - I meanwhile disabled TLSv1.2 which made the Outlook users happy. >>> >>> I run dovecot 2.2.13, OpenSSL 1.0.1j 15 Oct 2014 >>> >>> ssl_cert = >> ssl_cipher_list = ALL:!EXPORT:!LOW:!MEDIUM:!aNULL:+RC4:@STRENGTH >>> ssl_dh_parameters_length = 2048 >>> ssl_key = >> ssl_protocols = !SSLv2 !TLSv1.2 >>> >>> The certificate is from Comodo using sha256. >>> I was not able to track it down exactly but it looks like the problem was a sha1 signed certificate in the chain. After exchanging that with the matching 384bit one, the problems are gone. Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From tlx at leuxner.net Tue Jan 20 08:41:04 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 20 Jan 2015 09:41:04 +0100 Subject: Changeset c02f29ca104d badly breaks LMTP/mdbox In-Reply-To: <4006587A-7DB4-4245-854D-2414171B39CB@iki.fi> References: <20150117095127.GA55916@nihlus.leuxner.net> <4006587A-7DB4-4245-854D-2414171B39CB@iki.fi> Message-ID: <20150120084104.GA598@nihlus.leuxner.net> * Timo Sirainen 2015.01.20 06:42: > Should be fixed now. I think I'm now done for a while with the new core development work that was needed, so things shouldn't break so easily for a while. Although the body.snippet stuff still needs some more work. Looks good now after some testing. On a related note: Is there some tool/script available which can rebuild indexes in such a scenario? Since a whole 'm.' file has been corrupted the only way to mitigate this was restoring from a backup. As I don't want to do high frequency snapshots I wonder how to cope with mdbox corruptions? Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Warning: fscking index file /var/vmail/public/storage/dovecot.map.index Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Warning: mdbox /var/vmail/public/storage: rebuilding indexes Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: Corrupted dbox file /var/vmail/public/storage/m.132 (around offset=1): Invalid dbox version Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: mdbox rebuild: Failed to fix file /var/vmail/public/storage/m.132 Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: Corrupted dbox file /var/vmail/public/storage/m.131 (around offset=1): Invalid dbox version Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: mdbox rebuild: Failed to fix file /var/vmail/public/storage/m.131 Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: Corrupted dbox file /var/vmail/public/storage/m.130 (around offset=1): Invalid dbox version Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: mdbox rebuild: Failed to fix file /var/vmail/public/storage/m.130 Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: Corrupted dbox file /var/vmail/public/storage/m.127 (around offset=1): Invalid dbox version Jan 17 10:15:45 nihlus dovecot: lmtp(tlx at leuxner.net): Error: mdbox rebuild: Failed to fix file /var/vmail/public/storage/m.127 Jan 17 10:15:48 nihlus dovecot: lmtp(48904): Disconnect from local: Successful quit -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From davide.marchi at mail.cgilfe.it Tue Jan 20 09:12:47 2015 From: davide.marchi at mail.cgilfe.it (Davide) Date: Tue, 20 Jan 2015 10:12:47 +0100 Subject: Help backup mails from one dovecot server to another dovecot server Message-ID: <54BE1C0F.7000302@mail.cgilfe.it> Hi to all i'm running two Dovecot servers version 2.2.15 both Server A and Server B Server A is in production instead Server B is ready to start as a replacement My goal is replicate through doveadm backup command mails from Server A to Server B but when i execute following command in Server B error happens doveadm -vvv -D backup -R -u davide.marchi at mail.cgilfe.it imapc: dsync(davide.marchi at mail.cgilfe.it): Error: Couldn't delete mailbox INBOX: INBOX can't be deleted. Below are servers' config Server A conf (dovecot -n) # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.10 auth_master_user_separator = * debug_log_path = /var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 imapc_features = rfc822.size fetch-headers imapc_host = 192.168.10.53 imapc_master_user = master imapc_password = AbC100Vale100 imapc_user = %u info_log_path = /var/log/dovecot/dovecot.log last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 0.0.0.0 log_path = /var/log/dovecot/dovecot-err.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = never mail_gid = 89 mail_location = maildir:~/Maildir:INDEX=/mnt/ramdisk/%u mail_plugins = " notify zlib quota fts fts_solr" mail_prefetch_count = 20 mail_uid = 89 mailbox_list_index = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mmap_disable = yes namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { driver = shadow } passdb { args = webmail=127.0.0.1 cache_key=%u%s driver = vpopmail } plugin { antispam_backend = crm114 antispam_crm_args = -u;/opt/crm114;--fileprefix=/opt/crm114/;--config=/usr/share/crm114/mailfilter.cf antispam_crm_binary = /usr/share/crm114/mailreaver.crm antispam_crm_notspam_arg = --good antispam_crm_spam_arg = --spam antispam_signature = X-CRM114-CacheID antispam_spam_pattern_ignorecase = crm-spam* antispam_trash_pattern_ignorecase = Trash* antispam_unsure_pattern_ignorecase = crm-unsure* fts = solr fts_autoindex = yes fts_decoder = decode2text fts_solr = url=http://192.168.119.5:8983/solr/ break-imap-search mail_log_events = delete undelete expunge copy mail_log_fields = uid box msgid size mail_log_group_events = quota = dict:::redis:host=127.0.0.1:prefix=user/ quota_rule2 = Trash:storage=+20%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/lib/dovecot/sieve/default.sieve sieve_default = /usr/local/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_global_dir = /usr/local/lib/dovecot/sieve/ zlib_save = gz zlib_save_level = 9 } protocols = imap sieve service auth { unix_listener auth-master { group = vchkpw mode = 0600 user = vpopmail } unix_listener auth-userdb { group = vchkpw mode = 0600 user = vpopmail } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 user = dovecot } } service imap-login { executable = /usr/local/libexec/dovecot/imap-login inet_listener imaps { address = 192.168.10.50 } process_limit = 500 process_min_avail = 2 service_count = 0 } service imap { executable = /usr/local/libexec/dovecot/imap } service managesieve-login { executable = managesieve-login inet_listener sieve { address = 127.0.0.1 port = 4190 } process_min_avail = 2 service_count = 0 } service managesieve { process_limit = 10 } service quota-warning { executable = script /usr/local/libexec/dovecot/quota-warning unix_listener quota-warning { mode = 0660 user = vpopmail } user = vpopmail } ssl_cert = -<%s>-%m: %$ hostname = mail.cgilfe.it info_log_path = /var/log/dovecot/dovecot-deliver.log log_path = /var/log/dovecot/dovecot-deliver.log mail_fsync = optimized mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = " notify zlib quota fts fts_solr mail_log sieve" postmaster_address = postmaster at mail.cgilfe.it rejection_reason = Il messaggio destinato a <%t> ? stato automaticamente rifiutato:%n%r rejection_subject = Messaggio Rifiutato: %s submission_host = 127.0.0.1 } Server B conf (dovecot -n) # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 auth_master_user_separator = * debug_log_path = /var/log/dovecot/dovecot-debug.log disable_plaintext_auth = no first_valid_gid = 89 first_valid_uid = 89 imapc_features = rfc822.size fetch-headers imapc_host = 192.168.10.50 imapc_master_user = master imapc_password = AbC100Vale100 imapc_user = %u info_log_path = /var/log/dovecot/dovecot.log last_valid_gid = 89 last_valid_uid = 89 lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = 0.0.0.0 log_path = /var/log/dovecot/dovecot-err.log log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = never mail_gid = 89 mail_location = maildir:~/Maildir:INDEX=/var/index/%u mail_plugins = " notify zlib quota fts fts_solr" mail_prefetch_count = 20 mail_uid = 89 mailbox_list_index = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mmap_disable = yes namespace { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:/var/mail/public:CONTROL=~/Maildir/public:INDEXPVT=~/Maildir/public prefix = Public/ separator = / subscriptions = yes type = public } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { driver = shadow } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { driver = shadow } passdb { args = webmail=127.0.0.1 cache_key=%u%s driver = vpopmail } plugin { fts = solr fts_autoindex = yes fts_decoder = decode2text fts_solr = url=http://192.168.119.5:8983/solr/ break-imap-search mail_log_events = delete undelete expunge copy mail_log_fields = uid box msgid size mail_log_group_events = quota = maildir quota_rule = ?:storage=0 quota_rule2 = Trash:storage=+20%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap sieve service auth { unix_listener auth-master { group = vchkpw mode = 0600 user = vpopmail } unix_listener auth-userdb { group = vchkpw mode = 0600 user = vpopmail } } service decode2text { executable = script /usr/local/libexec/dovecot/decode2text.sh unix_listener decode2text { mode = 0666 user = dovecot } } service imap-login { executable = /usr/local/libexec/dovecot/imap-login inet_listener imaps { address = 192.168.10.53 } process_limit = 500 process_min_avail = 2 service_count = 0 } service imap { executable = /usr/local/libexec/dovecot/imap } service managesieve-login { executable = managesieve-login inet_listener sieve { address = 127.0.0.1 port = 4190 } process_min_avail = 2 service_count = 0 } service managesieve { process_limit = 10 } service quota-warning { executable = script /usr/local/libexec/dovecot/quota-warning unix_listener quota-warning { mode = 0660 user = vpopmail } user = vpopmail } ssl_cert = -<%s>-%m: %$ hostname = mail.cgilfe.it info_log_path = /var/log/dovecot/dovecot-deliver.log log_path = /var/log/dovecot/dovecot-deliver.log mail_fsync = optimized mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = " notify zlib quota fts fts_solr mail_log sieve" postmaster_address = postmaster at mail.cgilfe.it rejection_reason = Il messaggio destinato a <%t> ? stato automaticamente rifiutato:%n%r rejection_subject = Messaggio Rifiutato: %s submission_host = 127.0.0.1 } -- *Davide Marchi* *T*eorema *F*errara *S*rl *V*ia *S*pronello, 7 - *F*errara - 44121 *Tel*: 0532783161 - *Fax*: 0532783368 *E-mail*: davide.marchi at mail.cgilfe.it *Skype*: davide.marchi73 *Web*: http://www.cgilfe.it From adam.przybylski89 at gmail.com Tue Jan 20 11:04:03 2015 From: adam.przybylski89 at gmail.com (Adam Przybylski) Date: Tue, 20 Jan 2015 12:04:03 +0100 Subject: UID changes every few seconds Message-ID: Hi, I'm trying to use Gmail-style conversation view, as described in http://wiki2.dovecot.org/Plugins/Virtual#Examples >From a client based on Roundcube Framework I'm fetching message list from "virtual/INBOX" folder and that works fine to some degree, but the problem is that every few seconds or so (probably when new message is delivered or bounced on the mail server) UID of those messages changes. When I'm trying to display the details of given message (fetch body by UID), I'm getting an error because message with given UID do not exist on the server. When I only refresh the list (fetch messages one more time) and compare UIDs of the same messages between fetches, there are completely different numbers - growing every fetch. When I switch from virtual folder to real INBOX everything works fine, UID stays unchanged over time. Is this the intended behavior for virtual folders or some kind of bug? Or am I doing something wrong? I'm using Dovecot 2.1.7 on Debian GNU/Linux 7 (wheezy). The output of dovecot -n is attached. Thanks in advance, Adam -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot_n.log Type: application/octet-stream Size: 895 bytes Desc: not available URL: From nick.z.edwards at gmail.com Tue Jan 20 12:52:40 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 20 Jan 2015 22:52:40 +1000 Subject: Moving or "upgrading" from MD5 to SSHA512 In-Reply-To: <1573149518.20150114000807@turvamies.fi> References: <67761249.20150110020717@turvamies.fi> <54B0D649.4010104@bluerosetech.com> <906443151.20150110124525@turvamies.fi> <54B595FA.1040600@bluerosetech.com> <1573149518.20150114000807@turvamies.fi> Message-ID: On 1/14/15, Jyri Hovila [Turvamies.fi] wrote: > Hi! > > My "no" referred to the fact that I had not read the article before. =D > > Thanks for linking it - I'm sure it will be very useful. > > I've been terribly busy during the last couple of days and have therefore > not been able to comment much on the list. I'll come back there as soon as > things calm down a bit. =) > So you have this sorted now? (Sorry, was in transit, I'm back home in Australia for next few weeks on leave YAY) From nick.z.edwards at gmail.com Tue Jan 20 12:57:47 2015 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Tue, 20 Jan 2015 22:57:47 +1000 Subject: UID changes every few seconds In-Reply-To: References: Message-ID: On 1/20/15, Adam Przybylski wrote: > Hi, > > I'm trying to use Gmail-style conversation view, as described in > http://wiki2.dovecot.org/Plugins/Virtual#Examples > From a client based on Roundcube Framework I'm fetching message list from > "virtual/INBOX" folder and that works fine to some degree, but the problem > is that every few seconds or so (probably when new message is delivered or > bounced on the mail server) UID of those messages changes. When I'm trying > to display the details of given message (fetch body by UID), I'm getting an > error because message with given UID do not exist on the server. When I > only refresh the list (fetch messages one more time) and compare UIDs of > the same messages between fetches, there are completely different numbers - > growing every fetch. > When I switch from virtual folder to real INBOX everything works fine, UID > stays unchanged over time. Is this the intended behavior for virtual > folders or some kind of bug? Or am I doing something wrong? > > I'm using Dovecot 2.1.7 on Debian GNU/Linux 7 (wheezy). The output of > dovecot -n is attached. > > Thanks in advance, > Adam > 2.1.7 is an antique Try it with current set 2.2.15.. I'd try it for you but I'm in Australia for a few weeks and have no backend access to our network from here. From petehodur at gmail.com Wed Jan 21 05:25:40 2015 From: petehodur at gmail.com (Peter Hodur) Date: Wed, 21 Jan 2015 06:25:40 +0100 Subject: Shared folders - Namespace definition Message-ID: Hello, I'm trying to configure shared mailboxes with ACL. My problem is FS layout. Our maildirs is completely outside of home dirs (home dirs is on pure SSD zpool, maildirs on separate HDD zpool). We are using checkpassword auth, which sets mailbox_location for each user. Layout is following: maildirs: /dpool/mail/maldirs/user-uuid/ home is: /dpool/mail/home/user-uuid/ index & control is under home: /dpool/mail/home/user-uuid/[index|control] the problem is how to specify path in NAMESPACE definition. I can use %% variables, but maildirs is not under home and also does not have "fixed" structure (so I can not use %%u for example). Is it possible to for example set some special env variable from checkpassword and than use it in namespace definition like home variable? Or can someone help to find the code in dovecot, which evaluates namespace location patter so I can patch it? thanks Pete From skdovecot at smail.inf.fh-brs.de Wed Jan 21 06:54:19 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 21 Jan 2015 07:54:19 +0100 (CET) Subject: Shared folders - Namespace definition In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 21 Jan 2015, Peter Hodur wrote: > > maildirs: > > /dpool/mail/maldirs/user-uuid/ > > > home is: > > /dpool/mail/home/user-uuid/ > > > index & control is under home: > > /dpool/mail/home/user-uuid/[index|control] > > > the problem is how to specify path in NAMESPACE definition. I can use %% > variables, but maildirs is not under home and also does not have "fixed" > structure (so I can not use %%u for example). If your home is always prefixed by the same lengthed string, cur the user-uuid: http://wiki2.dovecot.org/Variables "You can take a substring of the variable by giving optional offset followed by '.' and width after the '%' character. For example %2u gives first two characters of the username. %2.1u gives third character of the username. " You could get somewhere, if you rewrite the login user by user-uuid, then %u is the user's uuid. > Is it possible to for example set some special env variable from > checkpassword and than use it in namespace definition like home variable? Is the checkpassword tool called for the users in a shared namespace??? This would cost lots of peformance, because you need to call that script for each user, the current user gets shares from. But if so, return MAIL: http://wiki2.dovecot.org/AuthDatabase/CheckPassword "Dovecot Extensions If you wish to return extra fields for Dovecot, set them in environment variables and then list them in EXTRA environment variable. The userdb extra fields can be returned by prefixing them with userdb_. For example: " - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVL9NHHz1H7kL/d9rAQItfAgAn7LKz8PFdcgNKwcIpe6/ZW4cbY51IvZ7 Y0SL1tHWDfkyIk7ogqePo/JjbtnXGXgupWeHS05Gt2gz20sXcbqb+ethbpO2Ziwg lLNYdcxZByk7KWWdsj+X4P2pYLn4JroYp8G50AKUfvMC4iUKil8/sQ05wwuaw25h L4J6HM6v54cwPIxcpRmLBVHnxD0/8Bo78LRMcs5sPHtaMDPblbqy+ugtaZ9ZGSsu RqJpSf7HG483olm1chmozdkZH66lpkyMcgs5tDb0mz6KKIRnjSURLThc0B6I6pJL d6Ru5rDso4QR9CUvz+ZttFBBMwgzCMv+0AJCE4VNBZrqSHLvGXxS1g== =37S3 -----END PGP SIGNATURE----- From mail at oliwel.de Wed Jan 21 12:24:25 2015 From: mail at oliwel.de (Oliver Welter) Date: Wed, 21 Jan 2015 13:24:25 +0100 Subject: Corruption of index files Message-ID: <54BF9A79.8040702@oliwel.de> Hi All, after upgrading my mail server (dovecot 1.1.7 -> 2.2.13) I get tons of messages about corrupted index files in the syslog ("Error: Corrupted transaction log" and "Warning: fscking index file .. dovecot.index". I tried flock and even dotlock, but the problems persist. The system is a VPS with an overlay filesystem which did work flawlessly with the old dovecot install and is not reporting any errors in the host machine. From time to time I get warnings about an exceeded inotify limit for the dovecot user - might this be a problem? Anybody has an idea what might be the cause those problems and how to track them down. Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From petehodur at gmail.com Wed Jan 21 14:26:27 2015 From: petehodur at gmail.com (Peter Hodur) Date: Wed, 21 Jan 2015 15:26:27 +0100 Subject: Shared folders - Namespace definition In-Reply-To: References: Message-ID: > > >> > If your home is always prefixed by the same lengthed string, cur the > user-uuid: > http://wiki2.dovecot.org/Variables > "You can take a substring of the variable by giving optional offset > followed by '.' and width after the '%' character. For example %2u gives > first two characters of the username. %2.1u gives third character of the > username. " > great! this is exactly what will solve my issue. > > You could get somewhere, if you rewrite the login user by user-uuid, then > %u is the user's uuid. > > Is it possible to for example set some special env variable from >> checkpassword and than use it in namespace definition like home variable? >> > > Is the checkpassword tool called for the users in a shared namespace??? > This would cost lots of peformance, because you need to call that script > for each user, the current user gets shares from. > What is "lot"? I hope, it will be evaluated one once, when user login and only once per each shared folder according to dictionary. So for example, if user have access to two other mailboxes, Dovecot will exec two times checkpassword as userdb lookup. Right? > > But if so, return MAIL: > http://wiki2.dovecot.org/AuthDatabase/CheckPassword > "Dovecot Extensions > > If you wish to return extra fields for Dovecot, set them in environment > variables and then list them in EXTRA environment variable. The userdb > extra fields can be returned by prefixing them with userdb_. For example: " > yes, i do it this way, for "sieve_default" for example. I set userdb_sieve_default env and add its name to userdb_extra. But how can I later access it on the right side for proper evalueating? When I use home dir, there is a "%" shortcut. From cs+dovecot at koch-aplsystems.de Wed Jan 21 14:56:39 2015 From: cs+dovecot at koch-aplsystems.de (Claudius) Date: Wed, 21 Jan 2015 15:56:39 +0100 Subject: Trying to get lda to deliver into public sub-sub-folder Message-ID: <54BFBE27.2040607@koch-aplsystems.de> Hi, I have this setup: recipient_delimiter = + mail_location = sdbox:~/sdbox namespace { type = public prefix = public/ location = sdbox:/home/vmail/shared/sdbox subscriptions = no } Postfix delivers via dovecot-lda: dovecot_shared unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/bin/env HOME=/home/vmail/shared /usr/lib/dovecot/dovecot-lda -f ${sender} -m ${user}/${extension} This works fine, ACL are manually defined and also work great. If I want to deliver into a subfolder I tell postfix to rewrite it to: folder+subfolder at dovecot_shared This is how it looks like: /home/vmail/shared/sdbox/mailboxes/{user}/{extension}/dbox-Mails But with this setup I couldn't get it to deliver into a subfolder of "subfolder/extension". Is it at all possible? Changing the HOME directory does not work because of the sdbox folder structure. I tried to generate a service for each subfolder like that: -m ${user}/subfolder/${extension} But when I did not have an extension I got: save failed to open mailbox {user}/subfolder/: Invalid mailbox name even though it exists. I think it doesn't like the trailing slash, but I don't know how to get rid of it. Hope I explained it well enough :) Regards, Claudius From michael.h.williamson at gmail.com Thu Jan 22 19:11:51 2015 From: michael.h.williamson at gmail.com (Michael Williamson) Date: Thu, 22 Jan 2015 12:11:51 -0700 Subject: sieve filter not working Message-ID: Hi, I have a question. I have dovecot 2.0.9 running on a CentOS 6.6 email server for a small department, ~15 users. amavis and postfix are also enabled. It appears that amavis invokes spamassassin, which tags incoming spam email. All email is then put into users local inbox directory, regardless of spam tag X-Spam_Flag value of YES or NO. I want instead to redirect spam to a special directory. I put a sieve filter in users home directory, "/home/user/.dovecot.sieve": require ["fileinto"]; # Move spam to spam folder if header :contains "X-Spam-Flag" ["YES"] { fileinto "Maildir/.SPAM"; stop; } I try different target directory for fileinto, ".SPAM", "/home/user/Maildir/.SPAM/cur", etc.. I know that those directories exist. The configure file "/etc/dovecot/dovecot.conf" is in original state. I changed one line in configure file "/etc/dovecot/conf.d/15-lda.conf": protocol lda { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = sieve $mail_plugins } I changed one line in configure file "/etc/dovecot/conf.d/20-lmtp.conf": protocol lmtp { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = sieve $mail_plugins } The configure file "/etc/dovecot/conf.d/90-sieve.conf" is unchanged, the relevant part is: plugin { # The path to the user's main active script. sieve = ~/.dovecot.sieve ... I restarted dovecot. I checked that it is running. The configuration command # doveconf -a output is attached. It is not working. How do I diagnose the problem? Thanks, -Mike -------------- next part -------------- A non-text attachment was scrubbed... Name: t Type: application/octet-stream Size: 12633 bytes Desc: not available URL: From tlx at leuxner.net Thu Jan 22 19:49:59 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 22 Jan 2015 20:49:59 +0100 Subject: sieve filter not working In-Reply-To: References: Message-ID: <20150122194959.GA1479@nihlus.leuxner.net> * Michael Williamson 2015.01.22 20:11: > I have dovecot 2.0.9 running on a CentOS 6.6 email server for a small This is a notorious version. Not only is it hopelessly outdated, it used to contain broken features way back... > require ["fileinto"]; > # Move spam to spam folder > if header :contains "X-Spam-Flag" ["YES"] { > fileinto "Maildir/.SPAM"; > stop; > } The mail is very awkwardly structured, I had trouble spotting the actual question FTR. You may set mail_debug to yes: $ grep mail_debug * 10-logging.conf:#mail_debug = no This will then produce verbose logging and most likely will help spotting the reason why the mail is not filed. Inject a mail with logging raised and post the relevant log excerpt here for further analysis. I have a feeling that it should just say fileinto "SPAM", but difficult to say without seeing log output. Current Dovecot versions also have the benefit of the sieve-test tool which lets you apply rules to mailboxes and see what would happen, or refilter mail if you're happy with the outcome of the dry-run. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From info at netocean.de Thu Jan 22 21:16:22 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Thu, 22 Jan 2015 22:16:22 +0100 Subject: Postfix Dovecot LDAP setup Message-ID: <54C168A6.3050902@netocean.de> Hi, since days I'm trying to substitude my SQL based Postfix/Dovecot mail server by a LDAP based setup. Everything seems to go smooth so far - thanks to the many how to and Dovecot wiki. Unfortunately I'm struggeling, since I have to create individual a LDAP schema which suits the currently used LDAP structure. The current tructure looks like this: => dc=MyDomain,dc=TLD ==> ou=People ===> uid=User-1 ====> ou=mail ===> uid=User-2 ====> ou=mail ... and so on ... Within ou=mail should be the individual mail account(s) information of a user. So in the end I want to add a(nother) mail account by something like this: cat << EOF > ./newUser.ldif dn: mailAddress=Test at Domain.TLD,ou=mail,uid=User-1,ou=people,dc=MyDomain,dc=TLD objectclass: top objectclass: mailAccount mailAddress: Test at Domain.TLD noMailPassword: {SSHA}SomePassword noMailAccountStatus: active [...] EOF Therefore I setup a LDAP schema like the following, but it seems to ignore the attributes "MailPassword" and "noMailAccountStatus". Why? I don't understand what I'm missing here on my objectclass? ;/ # ====================== LDAP schema ======================= # # # OID Macros (10001 should be IANA-registered) # objectidentifier nameSpace 1.3.6.1.4.1.10001 objectidentifier mail nameSpace:1 objectidentifier objectClassAccount mail:1 objectidentifier objectClassAccountInfo mail:2 # # Attributes: objectClass[NAME]:1.[SERIAL] # attributetype ( objectClassAccount:1.1 NAME 'mailAddress' DESC 'The hosted mail addresses' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} SINGLE-VALUE ) attributetype ( objectClassAccount:1.2 NAME 'MailPassword' DESC 'The hosted mail password' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40{128} ) attributetype ( objectClassAccount:1.3 NAME 'MailAccountStatus' DESC 'The status of a user account: active, noaccess, disabled, deleted' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE ) [...] # # Objects: objectClass[NAME]:2.[SERIAL] # objectclass ( objectClassAccount:2.1 NAME 'mailAccount' SUP ( top ) STRUCTURAL DESC 'Mail account' MUST ( mailAddress ) MAY ( MailPassword $ MailAccountStatus ) ) # ====================================================== # Thanks Best Regards, Leander From info at netocean.de Thu Jan 22 21:19:01 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Thu, 22 Jan 2015 22:19:01 +0100 Subject: Postfix Dovecot LDAP setup Message-ID: <54C16945.9010305@netocean.de> [EDIT]: Sorry about the second mail. ignore the first one - it had try & and error mistakes in. Problem is still the very same as explaned below. Thanks a lot. Hi, since days I'm trying to substitude my SQL based Postfix/Dovecot mail server by a LDAP based setup. Everything seems to go smooth so far - thanks to the many how to and Dovecot wiki. Unfortunately I'm struggeling, since I have to create individual a LDAP schema which suits the currently used LDAP structure. The current tructure looks like this: => dc=MyDomain,dc=TLD ==> ou=People ===> uid=User-1 ====> ou=mail ===> uid=User-2 ====> ou=mail ... and so on ... Within ou=mail should be the individual mail account(s) information of a user. So in the end I want to add a(nother) mail account by something like this: cat << EOF > ./newUser.ldif dn: mailAddress=Test at Domain.TLD,ou=mail,uid=User-1,ou=people,dc=MyDomain,dc=TLD objectclass: top objectclass: mailAccount mailAddress: Test at Domain.TLD MailPassword: {SSHA}SomePassword MailAccountStatus: active [...] EOF Therefore I setup a LDAP schema like the following, but it seems to ignore the attributes "MailPassword" and "noMailAccountStatus". Why? I don't understand what I'm missing here on my objectclass? ;/ # ====================== LDAP schema ======================= # # # OID Macros (10001 should be IANA-registered) # objectidentifier nameSpace 1.3.6.1.4.1.10001 objectidentifier mail nameSpace:1 objectidentifier objectClassAccount mail:1 objectidentifier objectClassAccountInfo mail:2 # # Attributes: objectClass[NAME]:1.[SERIAL] # attributetype ( objectClassAccount:1.1 NAME 'mailAddress' DESC 'The hosted mail addresses' EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} SINGLE-VALUE ) attributetype ( objectClassAccount:1.2 NAME 'MailPassword' DESC 'The hosted mail password' EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40{128} ) attributetype ( objectClassAccount:1.3 NAME 'MailAccountStatus' DESC 'The status of a user account: active, noaccess, disabled, deleted' EQUALITY caseIgnoreIA5Match SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE ) [...] # # Objects: objectClass[NAME]:2.[SERIAL] # objectclass ( objectClassAccount:2.1 NAME 'mailAccount' SUP ( top ) STRUCTURAL DESC 'Mail account' MUST ( mailAddress ) MAY ( MailPassword $ MailAccountStatus ) ) # ====================================================== # Thanks Best Regards, Leander From michael.h.williamson at gmail.com Thu Jan 22 21:41:39 2015 From: michael.h.williamson at gmail.com (Michael Williamson) Date: Thu, 22 Jan 2015 14:41:39 -0700 Subject: sieve filter not working In-Reply-To: <20150122194959.GA1479@nihlus.leuxner.net> References: <20150122194959.GA1479@nihlus.leuxner.net> Message-ID: Hi, OK. I tried your suggestion. I modified the dovecot config file "10-logging.conf", like so: log_path = syslog and mail_debug = yes It appears that the logging goes to "/var/log/maillog", not "messages" as I expected. Restarting service dovecot produces info in the "maillog" file showing the restart: ... Jan 22 15:20:14 coe dovecot: imap: Server shutting down. bytes=3383/61998 Jan 22 15:20:15 coe dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled) However, when I send a spam email to a user, the log has no dovecot messages: Jan 22 15:11:15 coe postfix/pickup[27525]: E3CA72578F: uid=0 from= Jan 22 15:11:15 coe postfix/cleanup[27819]: E3CA72578F: message-id=<20150122211115.E3CA72578F at coe.tsuniv.edu> Jan 22 15:11:15 coe postfix/qmgr[10863]: E3CA72578F: from=, size=10173, nrcpt=1 (queue active) Jan 22 15:11:16 coe amavis[27456]: (27456-07) LMTP:[127.0.0.1]:10024 /var/spool/amavisd/tmp/amavis-20150122T150549-27456-7GX8WObe: -> SIZE=10173 Recei$ Jan 22 15:11:16 coe amavis[27456]: (27456-07) Checking: nS2V0oL2RKlj -> Jan 22 15:11:18 coe postfix/smtpd[27825]: connect from localhost.localdomain[127.0.0.1] Jan 22 15:11:18 coe postfix/smtpd[27825]: 86B2223871: client=localhost.localdomain[127.0.0.1] Jan 22 15:11:18 coe postfix/cleanup[27819]: 86B2223871: message-id=<20150122211115.E3CA72578F at coe.tsuniv.edu> Jan 22 15:11:18 coe postfix/smtpd[27825]: disconnect from localhost.localdomain[127.0.0.1] Jan 22 15:11:18 coe postfix/qmgr[10863]: 86B2223871: from=, size=10891, nrcpt=1 (queue active) Jan 22 15:11:18 coe amavis[27456]: (27456-07) nS2V0oL2RKlj FWD from -> , BODY=7BIT 250 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as $ Jan 22 15:11:18 coe amavis[27456]: (27456-07) Passed SPAMMY {RelayedTaggedInbound}, -> , Message-ID: <20150122211115.E3CA72578F at coe.tsuniv.edu>, mail_i$ Jan 22 15:11:18 coe postfix/lmtp[27821]: E3CA72578F: to=, relay=127.0.0.1[127.0.0.1]:10024, delay=2.8, delays=0.11/0.04/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 from MTA(smt$ Jan 22 15:11:18 coe postfix/qmgr[10863]: E3CA72578F: removed Jan 22 15:11:18 coe postfix/local[27827]: 86B2223871: to=, relay=local, delay=0.14, delays=0.05/0.04/0/0.04, dsn=2.0.0, status=sent (delivered to maildir) Jan 22 15:11:18 coe postfix/qmgr[10863]: 86B2223871: removed I attached the above snippet of log file lines to this email. However, just watching the maillog file, dovecot messages do appear, like these: Jan 22 15:37:12 coe dovecot: imap-login: Login: user=, method=PLAIN, rip=::1, lip=::1, mpid=29204, TLS Jan 22 15:37:12 coe dovecot: imap(burks): Disconnected: Logged out bytes=94/856 So, is it postfix doing the local mail delivery, not dovecot? Could dovecot debug messages be going to a different log file? (They are not in "/var/log/messages"). Thanks, -Mike On 1/22/15, Thomas Leuxner wrote: > * Michael Williamson 2015.01.22 20:11: > >> I have dovecot 2.0.9 running on a CentOS 6.6 email server for a small > > This is a notorious version. Not only is it hopelessly outdated, it used to > contain broken features way back... > >> require ["fileinto"]; >> # Move spam to spam folder >> if header :contains "X-Spam-Flag" ["YES"] { >> fileinto "Maildir/.SPAM"; >> stop; >> } > > The mail is very awkwardly structured, I had trouble spotting the actual > question FTR. You may set mail_debug to yes: > > $ grep mail_debug * > 10-logging.conf:#mail_debug = no > > This will then produce verbose logging and most likely will help spotting > the reason why the mail is not filed. Inject a mail with logging raised and > post the relevant log excerpt here for further analysis. I have a feeling > that it should just say fileinto "SPAM", but difficult to say without seeing > log output. > > Current Dovecot versions also have the benefit of the sieve-test tool which > lets you apply rules to mailboxes and see what would happen, or refilter > mail if you're happy with the outcome of the dry-run. > > Regards > Thomas > -------------- next part -------------- A non-text attachment was scrubbed... Name: t1 Type: application/octet-stream Size: 2361 bytes Desc: not available URL: From lists at i-sommer.de Thu Jan 22 21:50:30 2015 From: lists at i-sommer.de (Kai Sommer) Date: Thu, 22 Jan 2015 22:50:30 +0100 Subject: "error: number of redirect actions exceeds policy limit" Message-ID: <54C170A6.4020303@i-sommer.de> Greetings, after testing a lot with my system I have to write to this list to ask for help from the cummunity. My system: * Ubuntu 12.04.5 LTS * Dovecot 2.0.19 * Dovecot config (extract) ????? 8< ????? plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_max_redirects = 10 } ????? >8?????? * Sieve filter rules ????? 8< ????? require ["copy"]; # rule:[Alles weiterleiten] if true { redirect "mail_1 at mydomain"; redirect "mail_2 at mydomain"; redirect "mail_3 at mydomain"; redirect "mail_4 at mydomain"; redirect "mail_5 at mydomain"; redirect "mail_6 at mydomain"; keep; stop; } ????? >8?????? Problem scenario: With the redirect-filter (see above) Dovecot redirects a incoming email to the first address and stops withe the message "error: number of redirect actions exceeds policy limit" (.dovecot.sieve.log). In my mail.log I see the message "Error: sieve: execution of script $PATH/.dovecot.sieve failed, but implicit keep was successful (user logfile $PATH/.dovecot.sieve.log may reveal additional details)". If I reduce the count of recipients to four ? which is the Dovecot default number ? everything will redireced as expected. I only could find an old problem report here [0] where as the posted solution doesn't work for me (as you can see above). (Also I tried the rule "redirect :copy" that doesn't changed something ?) [0] ? Is this problem known? ? Is there a solution for that problem? Thanks a lot and best regards! ? Kai -- Kai Sommer sig: 5E2E FA58 43BD 5599 4068 | pubkey: http://ma.ximi.se/mekey -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From redflag237 at googlemail.com Thu Jan 22 22:25:02 2015 From: redflag237 at googlemail.com (Jonas Plitt) Date: Thu, 22 Jan 2015 23:25:02 +0100 Subject: doveadm sync different users Message-ID: Hi all, The command below works to Sync a specific user between Server A and Server B. > doveadm sync -u user at mydomain.me tcp:new-server.domain.de How do i need to modify the command, to sync user1 at domain.me at ServerA and user2domain.me at ServerB? In my case the usernames needed to sync, are not the same. thanks in advance, Jonas From info at netocean.de Thu Jan 22 23:10:17 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Fri, 23 Jan 2015 00:10:17 +0100 Subject: Postfix Dovecot LDAP setup In-Reply-To: <54C16945.9010305@netocean.de> References: <54C16945.9010305@netocean.de> Message-ID: <54C18359.9010708@netocean.de> ... Silly me ... turns out I had a newline in my LDIF .... that's why it ignored all the attributes below. wasted 1.5 days on this flippn thing. Crap ;( Anyway .. happy to move on now ;) Am 22.01.15 um 22:19 schrieb Leander Sch?fer: > [EDIT]: Sorry about the second mail. ignore the first one - it had try > & and error mistakes in. Problem is still the very same as explaned > below. Thanks a lot. > > > > Hi, > > since days I'm trying to substitude my SQL based Postfix/Dovecot mail > server by a LDAP based setup. Everything seems to go smooth so far - > thanks to the many how to and Dovecot wiki. Unfortunately I'm > struggeling, since I have to create individual a LDAP schema which > suits the currently used LDAP structure. The current tructure looks > like this: > > => dc=MyDomain,dc=TLD > ==> ou=People > ===> uid=User-1 > ====> ou=mail > ===> uid=User-2 > ====> ou=mail > > ... and so on ... > > > Within ou=mail should be the individual mail account(s) information of > a user. So in the end I want to add a(nother) mail account by > something like this: > > cat << EOF > ./newUser.ldif > dn: > mailAddress=Test at Domain.TLD,ou=mail,uid=User-1,ou=people,dc=MyDomain,dc=TLD > objectclass: top > objectclass: mailAccount > > mailAddress: Test at Domain.TLD > MailPassword: {SSHA}SomePassword > MailAccountStatus: active > [...] > EOF > > > Therefore I setup a LDAP schema like the following, but it seems to > ignore the attributes "MailPassword" and "noMailAccountStatus". Why? I > don't understand what I'm missing here on my objectclass? ;/ > > > > # ====================== LDAP schema ======================= # > > > # > # OID Macros (10001 should be IANA-registered) > # > > objectidentifier nameSpace 1.3.6.1.4.1.10001 > objectidentifier mail nameSpace:1 > objectidentifier objectClassAccount mail:1 > objectidentifier objectClassAccountInfo mail:2 > > > # > # Attributes: objectClass[NAME]:1.[SERIAL] > # > > attributetype ( objectClassAccount:1.1 > NAME 'mailAddress' > DESC 'The hosted mail addresses' > EQUALITY caseIgnoreIA5Match > SUBSTR caseIgnoreIA5SubstringsMatch > SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} > SINGLE-VALUE ) > > attributetype ( objectClassAccount:1.2 > NAME 'MailPassword' > DESC 'The hosted mail password' > EQUALITY octetStringMatch > SYNTAX 1.3.6.1.4.1.1466.115.121.1.40{128} ) > > attributetype ( objectClassAccount:1.3 > NAME 'MailAccountStatus' > DESC 'The status of a user account: active, noaccess, disabled, > deleted' > EQUALITY caseIgnoreIA5Match > SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 > SINGLE-VALUE ) > > [...] > > # > # Objects: objectClass[NAME]:2.[SERIAL] > # > > objectclass ( objectClassAccount:2.1 > NAME 'mailAccount' > SUP ( top ) > STRUCTURAL > DESC 'Mail account' > MUST ( mailAddress ) > MAY ( MailPassword $ MailAccountStatus ) ) > > # ====================================================== # > > > Thanks > > > Best Regards, > Leander From nbensa+dovecot at gmail.com Thu Jan 22 23:46:30 2015 From: nbensa+dovecot at gmail.com (Norberto Bensa) Date: Thu, 22 Jan 2015 20:46:30 -0300 Subject: from Cyrus to Dovecot, seen status on cyrus.index files Message-ID: Hello, Using cyrus2dovecot (https://github.com/a-schild/cyrus2dovecot) I have migrated some test mailboxes from Cyrus (2.4.17) to Dovecot (2.2.9). The problem I'm having is that the seen status is in cyrus.index files, which cyrus2dovecot doesn't understand. Does anyone use cyrus2dovecot for migrations? If so, how do you deal with seen status when it is stored in the .index files? Many thanks in advance, Norberto From ml+dovecot at valo.at Fri Jan 23 05:21:36 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Fri, 23 Jan 2015 06:21:36 +0100 Subject: sieve filter not working In-Reply-To: References: <20150122194959.GA1479@nihlus.leuxner.net> Message-ID: <50a83e3e19bdca78adde70dabfbfe0de@valo.at> On 2015-01-22 22:41, Michael Williamson wrote: > Hi, Hello, > > OK. I tried your suggestion. I modified the dovecot config file > "10-logging.conf", like so: > > log_path = syslog > > and > > mail_debug = yes > You could set syslog_facility = local5 and have all the log messages in the messages file. Also take a look at the logging dovecot wiki page http://wiki2.dovecot.org/Logging > It appears that the logging goes to "/var/log/maillog", not "messages" > as I expected. > Restarting service dovecot produces info in the "maillog" file showing > the restart: > ... > Jan 22 15:20:14 coe dovecot: imap: Server shutting down. > bytes=3383/61998 > Jan 22 15:20:15 coe dovecot: master: Dovecot v2.0.9 starting up (core > dumps disabled) > > However, when I send a spam email to a user, the log has no dovecot > messages: > > Jan 22 15:11:15 coe postfix/pickup[27525]: E3CA72578F: uid=0 > from= > Jan 22 15:11:15 coe postfix/cleanup[27819]: E3CA72578F: > message-id=<20150122211115.E3CA72578F at coe.tsuniv.edu> > Jan 22 15:11:15 coe postfix/qmgr[10863]: E3CA72578F: > from=, size=10173, nrcpt=1 (queue active) > Jan 22 15:11:16 coe amavis[27456]: (27456-07) LMTP:[127.0.0.1]:10024 > /var/spool/amavisd/tmp/amavis-20150122T150549-27456-7GX8WObe: > -> SIZE=10173 > Recei$ > Jan 22 15:11:16 coe amavis[27456]: (27456-07) Checking: nS2V0oL2RKlj > -> > Jan 22 15:11:18 coe postfix/smtpd[27825]: connect from > localhost.localdomain[127.0.0.1] > Jan 22 15:11:18 coe postfix/smtpd[27825]: 86B2223871: > client=localhost.localdomain[127.0.0.1] > Jan 22 15:11:18 coe postfix/cleanup[27819]: 86B2223871: > message-id=<20150122211115.E3CA72578F at coe.tsuniv.edu> > Jan 22 15:11:18 coe postfix/smtpd[27825]: disconnect from > localhost.localdomain[127.0.0.1] > Jan 22 15:11:18 coe postfix/qmgr[10863]: 86B2223871: > from=, size=10891, nrcpt=1 (queue active) > Jan 22 15:11:18 coe amavis[27456]: (27456-07) nS2V0oL2RKlj FWD from > -> , BODY=7BIT 250 > 2.0.0 from MTA(smtp:[127.0.0.1]:10025): 250 2.0.0 Ok: queued as $ > Jan 22 15:11:18 coe amavis[27456]: (27456-07) Passed SPAMMY > {RelayedTaggedInbound}, -> > , Message-ID: > <20150122211115.E3CA72578F at coe.tsuniv.edu>, mail_i$ > Jan 22 15:11:18 coe postfix/lmtp[27821]: E3CA72578F: > to=, relay=127.0.0.1[127.0.0.1]:10024, > delay=2.8, delays=0.11/0.04/0/2.6, dsn=2.0.0, status=sent (250 2.0.0 > from MTA(smt$ > Jan 22 15:11:18 coe postfix/qmgr[10863]: E3CA72578F: removed > Jan 22 15:11:18 coe postfix/local[27827]: 86B2223871: > to=, relay=local, delay=0.14, > delays=0.05/0.04/0/0.04, dsn=2.0.0, status=sent (delivered to maildir) > Jan 22 15:11:18 coe postfix/qmgr[10863]: 86B2223871: removed > > I attached the above snippet of log file lines to this email. > > However, just watching the maillog file, dovecot messages do appear, > like these: > > Jan 22 15:37:12 coe dovecot: imap-login: Login: user=, > method=PLAIN, rip=::1, lip=::1, mpid=29204, TLS > Jan 22 15:37:12 coe dovecot: imap(burks): Disconnected: Logged out > bytes=94/856 > > So, is it postfix doing the local mail delivery, not dovecot? Could > dovecot debug messages be > going to a different log file? (They are not in "/var/log/messages"). > To answer this question please post relevant parts of your postfix main.cf - mailbox_command - virtual_transport - mailbox_transport I use lmtp delivery from postfix to dovecot with virtual users as described in http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP and that works very well. > Thanks, > -Mike - christian > > > > On 1/22/15, Thomas Leuxner wrote: >> * Michael Williamson 2015.01.22 >> 20:11: >> >>> I have dovecot 2.0.9 running on a CentOS 6.6 email server for a small >> >> This is a notorious version. Not only is it hopelessly outdated, it >> used to >> contain broken features way back... >> >>> require ["fileinto"]; >>> # Move spam to spam folder >>> if header :contains "X-Spam-Flag" ["YES"] { >>> fileinto "Maildir/.SPAM"; >>> stop; >>> } >> >> The mail is very awkwardly structured, I had trouble spotting the >> actual >> question FTR. You may set mail_debug to yes: >> >> $ grep mail_debug * >> 10-logging.conf:#mail_debug = no >> >> This will then produce verbose logging and most likely will help >> spotting >> the reason why the mail is not filed. Inject a mail with logging >> raised and >> post the relevant log excerpt here for further analysis. I have a >> feeling >> that it should just say fileinto "SPAM", but difficult to say without >> seeing >> log output. >> >> Current Dovecot versions also have the benefit of the sieve-test tool >> which >> lets you apply rules to mailboxes and see what would happen, or >> refilter >> mail if you're happy with the outcome of the dry-run. >> >> Regards >> Thomas >> From superinterstellar at gmail.com Fri Jan 23 05:31:20 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 23 Jan 2015 12:31:20 +0700 Subject: [serious help request!] Desperately trying to get Apache Solr working with Dovecot. Message-ID: Hello, I am desperately trying to get Apache Solr to work with Dovecot FTS. I would really appreciate if someone could please help me! I have already done the following:- 1. I can ssh into my server and see that Apache Solr is up and running. ssh -t -L 8983:localhost:8983 user at mydomain.com 2. In the collection1 core selector I have the following files:- solrconfig.xml and schema.xml The schema.xml output is as follows(see link):- http://pastebin.com/thGw2pQj 3. I have installed the dovecot-solr already. 4. Configured dovecot to run solr-fts as follows:- In 10-mail.conf: # Space separated list of plugins to load for all services. Plugins specific to # IMAP, LDA, etc. are added to this list in their own .conf files. #mail_plugins = mail_plugins = fts fts_solr In 90-plugin.conf:- plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8983/solr/ } *Desperately looking for some help! Please help!!!* Thanks Kevin From skdovecot at smail.inf.fh-brs.de Fri Jan 23 07:17:41 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 23 Jan 2015 08:17:41 +0100 (CET) Subject: sieve filter not working In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 22 Jan 2015, Michael Williamson wrote: > > require ["fileinto"]; > # Move spam to spam folder > if header :contains "X-Spam-Flag" ["YES"] { > fileinto "Maildir/.SPAM"; > stop; > } > > It is not working. How do I diagnose the problem? There should be an .err file in the location of the binary form of the Sieve script. Also, the argument of fileinto looks wrong; maybe it was correct in v2.0, but current versions would require just "SPAM"; - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMH1lXz1H7kL/d9rAQLzLQgAsfIvOkAB4zP9WfIplriW9d/KlQa5wdOU Fq4ZjHHD0huKedxAJeMH9HMpYFGhtZqtFFfrSIeKR3lfFNGHjjN9n4QE2z3aEmcQ XHBAkR0DmcruAsGumRZJJK6SUeocdWCj9NrlMliz2nga+hJFHH6iTSuNwkjfz1hZ R+/9h+Dq9jKot1jNPREo4dEM3lKi1sG5002xgxmCZ3hTz1ook/uT4WR0IMWMmcKW LRDbaXExvYlb5nLC33q/xM81avgK2dVUgDL+NJS13beMKUJerap3cCSrYi5iLNiZ S7mzABZZDm0V0kXC4bvILfNBM87L8Ru69lFH+gDpjO6Yjg9FYintBg== =13MG -----END PGP SIGNATURE----- From mail at marc-stuermer.de Fri Jan 23 07:20:52 2015 From: mail at marc-stuermer.de (Marc =?utf-8?b?U3TDvHJtZXI=?=) Date: Fri, 23 Jan 2015 08:20:52 +0100 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: Message-ID: <20150123082052.Horde.3L_3IUHEteEzyNFl1a5bCQ1@webmail.no-carrier.info> Zitat von Norberto Bensa : > Does anyone use cyrus2dovecot for migrations? If so, how do you deal with > seen status when it is stored in the .index files? Why don't you use the migration tool that comes with Dovecot bundled instead, namely dsync? It claims to preserve almost virtually everything from the old server. From superinterstellar at gmail.com Fri Jan 23 07:36:29 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 23 Jan 2015 14:36:29 +0700 Subject: Easiest way to compile dovecot on Ubuntu 14.04 Message-ID: Hello, I need to recompile dovecot on ubuntu 14.04 What would be the easiest way to do this? I am trying to follow this tutorial:- https://blog.vbonhomme.fr/add-full-text-search-fts-to-your-dovecot-using-solr-4-10-on-ubuntu-14-04/ When I run :- apt-get source dovecot Reading package lists... Done Building dependency tree Reading state information... Done NOTICE: 'dovecot' packaging is maintained in the 'Git' version control system at: git://git.debian.org/git/collab-maint/dovecot.git Need to get 5759 kB of source archives. Get:1 http://mirrors.gandi.net/ubuntu/ trusty-updates/main dovecot 1:2.2.9-1ubuntu2.1 (dsc) [3136 B] Get:2 http://mirrors.gandi.net/ubuntu/ trusty-updates/main dovecot 1:2.2.9-1ubuntu2.1 (tar) [4541 kB] Get:3 http://mirrors.gandi.net/ubuntu/ trusty-updates/main dovecot 1:2.2.9-1ubuntu2.1 (diff) [1215 kB] Fetched 5759 kB in 0s (8141 kB/s) sh: 1: dpkg-source: not found Unpack command 'dpkg-source -x dovecot_2.2.9-1ubuntu2.1.dsc' failed. Check if the 'dpkg-dev' package is installed. E: Child process failed Any faster means? Also cant get dpkg-dev installed on my system. Thanks Kevin From mail at marc-stuermer.de Fri Jan 23 07:42:08 2015 From: mail at marc-stuermer.de (Marc =?utf-8?b?U3TDvHJtZXI=?=) Date: Fri, 23 Jan 2015 08:42:08 +0100 Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: Message-ID: <20150123084208.Horde.eSutHfmVdFWgdqmbbZHOiA5@webmail.no-carrier.info> Zitat von Kevin Laurie : > Hello, > > I need to recompile dovecot on ubuntu 14.04 > What would be the easiest way to do this? > > I am trying to follow this tutorial:- > https://blog.vbonhomme.fr/add-full-text-search-fts-to-your-dovecot-using-solr-4-10-on-ubuntu-14-04/ Yes, there is a much, much faster way: apt-get install dovecot-solr 14.04 LTS already provides a nifty binary package for it, no need to compile it yourself. The blogger should have done his home work better. Regards, Marc From skdovecot at smail.inf.fh-brs.de Fri Jan 23 07:45:46 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 23 Jan 2015 08:45:46 +0100 (CET) Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 23 Jan 2015, Kevin Laurie wrote: > sh: 1: dpkg-source: not found > Unpack command 'dpkg-source -x dovecot_2.2.9-1ubuntu2.1.dsc' failed. > Check if the 'dpkg-dev' package is installed. > E: Child process failed > > Any faster means? Also cant get dpkg-dev installed on my system. Well, this is the Debian way, so why would dpkg-dev balk at installation? On Debian based machines, this should be the easiest way ;-) ===== variant 2) Pull the sources from dovecot.org via tarball or mercurial. If you need Sieve and other plugins, I would not call this variant easier. Then you have to option to 2a) try to apply the configuration from the Debian package to the source, in order to have a replacement for the original package, or 2b) create a new configuration from scratch with help from the Debian package, but to install into /usr/local or /opt. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMH8Knz1H7kL/d9rAQJpNgf/eziZXNla+YGFd6JxC3OAarRsi22o+WYJ r0ElldVnpKsKlZrY3aHfXFSL0/Ij8sW9RQSRSTs1KvnY23F6UmDnZmU7+sfQBHXO MaSxVjPCNyI6CmLHPPQ4iKXJTWGAFCrfYDRTvHT0cra4UKqNPhUttWiYFUqo6s+a P42j5sax4RPG7Hb4MUXEaey4il183Fwm7Z5QjyuMKb0C8gjiK5GGPEXcQ4Tutauq 3uoZ5ScuNvExgL8teEk+JBPeKZmOQzyu1NRrLzqxFYx521CQgU3ttEP8x0PxG3Bz /ya2dCpnB1MZbDOXBhssqu5lKvzWKltjVHoCzJKrF0vP4x+EEunO3A== =k/gi -----END PGP SIGNATURE----- From superinterstellar at gmail.com Fri Jan 23 07:47:03 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 23 Jan 2015 14:47:03 +0700 Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: <20150123084208.Horde.eSutHfmVdFWgdqmbbZHOiA5@webmail.no-carrier.info> References: <20150123084208.Horde.eSutHfmVdFWgdqmbbZHOiA5@webmail.no-carrier.info> Message-ID: Dear Marc. Thanks =) Already have dovecot-solr installed. Is there a way to see if dovecot-solr is actually working? Thanks Kevin On Fri, Jan 23, 2015 at 2:42 PM, Marc St?rmer wrote: > > Zitat von Kevin Laurie : > > Hello, >> >> I need to recompile dovecot on ubuntu 14.04 >> What would be the easiest way to do this? >> >> I am trying to follow this tutorial:- >> https://blog.vbonhomme.fr/add-full-text-search-fts-to-your- >> dovecot-using-solr-4-10-on-ubuntu-14-04/ >> > > Yes, there is a much, much faster way: > > apt-get install dovecot-solr > > 14.04 LTS already provides a nifty binary package for it, no need to > compile it yourself. The blogger should have done his home work better. > > Regards, > Marc > From superinterstellar at gmail.com Fri Jan 23 07:49:45 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 23 Jan 2015 14:49:45 +0700 Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: References: Message-ID: Dear Steffen, Variant 2 does look more challenging but I don't think its necessary if dovecot-solr is already available and installed. I guess the problem is with something else. Thanks Kevin On Fri, Jan 23, 2015 at 2:45 PM, Steffen Kaiser < skdovecot at smail.inf.fh-brs.de> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 23 Jan 2015, Kevin Laurie wrote: > > sh: 1: dpkg-source: not found >> Unpack command 'dpkg-source -x dovecot_2.2.9-1ubuntu2.1.dsc' failed. >> Check if the 'dpkg-dev' package is installed. >> E: Child process failed >> >> Any faster means? Also cant get dpkg-dev installed on my system. >> > > Well, this is the Debian way, so why would dpkg-dev balk at installation? > On Debian based machines, this should be the easiest way ;-) > > ===== > > variant 2) > Pull the sources from dovecot.org via tarball or mercurial. If you need > Sieve and other plugins, I would not call this variant easier. > > Then you have to option to > > 2a) try to apply the configuration from the Debian package to the source, > in order to have a replacement for the original package, or > > 2b) create a new configuration from scratch with help from the Debian > package, but to install into /usr/local or /opt. > > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVMH8Knz1H7kL/d9rAQJpNgf/eziZXNla+YGFd6JxC3OAarRsi22o+WYJ > r0ElldVnpKsKlZrY3aHfXFSL0/Ij8sW9RQSRSTs1KvnY23F6UmDnZmU7+sfQBHXO > MaSxVjPCNyI6CmLHPPQ4iKXJTWGAFCrfYDRTvHT0cra4UKqNPhUttWiYFUqo6s+a > P42j5sax4RPG7Hb4MUXEaey4il183Fwm7Z5QjyuMKb0C8gjiK5GGPEXcQ4Tutauq > 3uoZ5ScuNvExgL8teEk+JBPeKZmOQzyu1NRrLzqxFYx521CQgU3ttEP8x0PxG3Bz > /ya2dCpnB1MZbDOXBhssqu5lKvzWKltjVHoCzJKrF0vP4x+EEunO3A== > =k/gi > -----END PGP SIGNATURE----- > From mail at marc-stuermer.de Fri Jan 23 08:18:47 2015 From: mail at marc-stuermer.de (=?UTF-8?B?TWFyYyBTdMO8cm1lcg==?=) Date: Fri, 23 Jan 2015 09:18:47 +0100 Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: References: <20150123084208.Horde.eSutHfmVdFWgdqmbbZHOiA5@webmail.no-carrier.info> Message-ID: <54C203E7.7020905@marc-stuermer.de> Am 23.01.2015 um 08:47 schrieb Kevin Laurie: > Dear Marc. > Thanks =) > Already have dovecot-solr installed. > Is there a way to see if dovecot-solr is actually working? Yes. http://wiki2.dovecot.org/Plugins/FTS/Solr - look there at "Testing." Use a mailbox for it with at least some megabytes of data. If you don't see then upgrade notices, it it not working yet. The notices may look like this: * OK Indexed 11% of the mailbox, ETA 1:12 * OK Indexed 30% of the mailbox, ETA 1:07 * OK Indexed 49% of the mailbox, ETA 0:52 * OK Indexed 61% of the mailbox, ETA 0:43 * OK Indexed 80% of the mailbox, ETA 0:22 * OK Mailbox indexing finished * SEARCH 857 1486 1526 1544 1589 1590 1591 1608 1737 1753 2660 2864 3967 4124 4127 4129 4159 4235 4254 4273 4682 4683 4684 4685 4770 5132 5467 5470 5735 5788 5838 5903 5904 5905 5906 5907 5996 6024 6113 6144 1 OK Search completed (114.650 secs) If you don't see those, enable debugging in dovecot.conf and then you need to further investigate on where there's the problem. From dovecot-l at fu-berlin.de Fri Jan 23 08:36:01 2015 From: dovecot-l at fu-berlin.de (Heiko Schlichting) Date: Fri, 23 Jan 2015 09:36:01 +0100 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: References: Message-ID: <20150123083601.GB433908@zedat.fu-berlin.de> Hi > Using cyrus2dovecot (https://github.com/a-schild/cyrus2dovecot) I have > migrated some test mailboxes from Cyrus (2.4.17) to Dovecot (2.2.9). > > The problem I'm having is that the seen status is in cyrus.index files, > which cyrus2dovecot doesn't understand. > > Does anyone use cyrus2dovecot for migrations? Yes, but this software which is not related to the one you mention: http://www.cyrus2dovecot.sw.fu-berlin.de/ Heiko From superinterstellar at gmail.com Fri Jan 23 08:50:47 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 23 Jan 2015 15:50:47 +0700 Subject: Easiest way to compile dovecot on Ubuntu 14.04 In-Reply-To: <54C203E7.7020905@marc-stuermer.de> References: <20150123084208.Horde.eSutHfmVdFWgdqmbbZHOiA5@webmail.no-carrier.info> <54C203E7.7020905@marc-stuermer.de> Message-ID: Dear Marc, Noted. I just found out on iRedmail the plugins need to be specified in dovecot.conf instead of dovecot/conf.d/90-plugin.conf . Making the changes now. Hopefully it should work. Will check out the testing page. Thanks Kevin On Fri, Jan 23, 2015 at 3:18 PM, Marc St?rmer wrote: > Am 23.01.2015 um 08:47 schrieb Kevin Laurie: > > Dear Marc. >> Thanks =) >> Already have dovecot-solr installed. >> Is there a way to see if dovecot-solr is actually working? >> > > Yes. > > http://wiki2.dovecot.org/Plugins/FTS/Solr - look there at "Testing." > > Use a mailbox for it with at least some megabytes of data. If you don't > see then upgrade notices, it it not working yet. > > The notices may look like this: > > * OK Indexed 11% of the mailbox, ETA 1:12 > * OK Indexed 30% of the mailbox, ETA 1:07 > * OK Indexed 49% of the mailbox, ETA 0:52 > * OK Indexed 61% of the mailbox, ETA 0:43 > * OK Indexed 80% of the mailbox, ETA 0:22 > * OK Mailbox indexing finished > * SEARCH 857 1486 1526 1544 1589 1590 1591 1608 1737 1753 2660 2864 3967 > 4124 4127 4129 4159 4235 4254 4273 4682 4683 4684 4685 4770 5132 5467 5470 > 5735 5788 5838 5903 5904 5905 5906 5907 5996 6024 6113 6144 > 1 OK Search completed (114.650 secs) > > If you don't see those, enable debugging in dovecot.conf and then you need > to further investigate on where there's the problem. > From jogi at mur.at Fri Jan 23 09:52:28 2015 From: jogi at mur.at (=?UTF-8?B?Sm9naSBIb2Ztw7xsbGVy?=) Date: Fri, 23 Jan 2015 10:52:28 +0100 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: <20150123083601.GB433908@zedat.fu-berlin.de> References: <20150123083601.GB433908@zedat.fu-berlin.de> Message-ID: <54C219DC.7040103@mur.at> Hi, Am 2015-01-23 um 09:36 schrieb Heiko Schlichting: >> Does anyone use cyrus2dovecot for migrations? > > Yes, but this software which is not related to the one you mention: > > http://www.cyrus2dovecot.sw.fu-berlin.de/ The two are identical. Apparently someone put cyrus2dovecot on github. Maybe it will change in the future. However, we recently migrated from cyrus to dovecot using this piece of software. The thing you are looking for is probably the switch -S, --cyrus-seen=PATH with which you tell the script where to find the cyrus seen.db. Using this we had no problems with seen flags for regular mailboxes. But migrating shared mailboxes was a PITA ... so if you have these, be prepared for troubles. Regards, -- j.hofm?ller mur.sat -- a space art project http://sat.mur.at/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 213 bytes Desc: OpenPGP digital signature URL: From nbensa+dovecot at gmail.com Fri Jan 23 13:25:46 2015 From: nbensa+dovecot at gmail.com (Norberto Bensa) Date: Fri, 23 Jan 2015 10:25:46 -0300 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: <20150123082052.Horde.3L_3IUHEteEzyNFl1a5bCQ1@webmail.no-carrier.info> References: <20150123082052.Horde.3L_3IUHEteEzyNFl1a5bCQ1@webmail.no-carrier.info> Message-ID: 2015-01-23 4:20 GMT-03:00 Marc St?rmer : > > Zitat von Norberto Bensa : > > Does anyone use cyrus2dovecot for migrations? If so, how do you deal with >> seen status when it is stored in the .index files? >> > > Why don't you use the migration tool that comes with Dovecot bundled > instead, namely dsync? It claims to preserve almost virtually everything > from the old server. > I've used dsync but it's too slow. Also, I have some users with shared mailboxes. Those mailboxes get duplicated. Either this is the way it works or I'm doing something wrong. If I could get dsync to not duplicate shared mailboxes, I think I could use it. Thanks for the suggestion. Regards, Norberto From delrio at mie.utoronto.ca Fri Jan 23 14:21:39 2015 From: delrio at mie.utoronto.ca (Oscar del Rio) Date: Fri, 23 Jan 2015 09:21:39 -0500 Subject: sieve filter not working In-Reply-To: References: Message-ID: <54C258F3.5040601@mie.utoronto.ca> On 22/01/2015 2:11 PM, Michael Williamson wrote: > I put a sieve filter in users home directory, "/home/user/.dovecot.sieve": > > require ["fileinto"]; > # Move spam to spam folder > if header :contains "X-Spam-Flag" ["YES"] { > fileinto "Maildir/.SPAM"; > stop; > } http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } From jogi at mur.at Fri Jan 23 14:38:29 2015 From: jogi at mur.at (=?UTF-8?B?Sm9naSBIb2Ztw7xsbGVy?=) Date: Fri, 23 Jan 2015 15:38:29 +0100 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: References: <20150123082052.Horde.3L_3IUHEteEzyNFl1a5bCQ1@webmail.no-carrier.info> Message-ID: <54C25CE5.5010705@mur.at> Hi, Am 2015-01-23 um 14:25 schrieb Norberto Bensa: > I've used dsync but it's too slow. Yes, that's also what we experienced and why we went to cyrus2dovecot. > Also, I have some users with shared mailboxes. Those mailboxes get > duplicated. Either this is the way it works or I'm doing something wrong. > If I could get dsync to not duplicate shared mailboxes, I think I could use > it. Duplication was not a problem when we migrated. The side effect we had was that all mail in a shared mailbox was marked new for all the sharers. We could not get cyrus2dovecot to use user's flags for shared mailboxes. Apart from that everything worked really well. Regards, -- J.Hofm?ller Ein literarisches Meisterwerk ist nur ein W?rterbuch in Unordnung. - Jean Cocteau -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 213 bytes Desc: OpenPGP digital signature URL: From stephane.cottin at vixns.com Fri Jan 23 14:49:56 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Fri, 23 Jan 2015 15:49:56 +0100 Subject: LDA input validation Message-ID: Hi, I'm using qmail + dspam + dovecot-lda. A typical .qmail file : | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" When dspam segfault or is killed, preline receive an empty content and only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). Then dovecot-lda delivers successfully and the original message contents are lost. Should dovecot-lda returns an error in such conditions ( invalid contents ) ? Qmail will keep it in the queue and retry delivery later. St?phane From john.hendrich at airhop.com Fri Jan 23 15:06:22 2015 From: john.hendrich at airhop.com (John Hendrich) Date: Fri, 23 Jan 2015 10:06:22 -0500 Subject: Client shows null Sender & date Message-ID: <1422025582.9698.11.camel@asus-ub> I'm using Postfix and Dovecot 2.0.19 and Virtual domains & users (mysql). Incoming mail is handled by Postfix and then handed off to Dovecot LMTP for delivery. However, the Sender and Date are essentially null when viewing the email with either the POP3 or IMAP client. The logs (below) show this. I changed the sender and recipient addresses for privacy. Jan 23 08:41:48 klsrv postfix/smtpd[10836]: 1F907F00276: client=unknown[10.0.0.19] Jan 23 08:42:07 klsrv postfix/cleanup[10842]: 1F907F00276: message-id=<> Jan 23 08:42:07 klsrv postfix/qmgr[4889]: 1F907F00276: from=, size=217, nrcpt=1 (queue active) Jan 23 08:42:07 klsrv dovecot: lmtp(10845): Connect from local Jan 23 08:42:07 klsrv dovecot: lmtp(10845, recipient at example.net): wvU2Gq9PwlRdKgAAjuDoqw: subject=test to client: from=: msgid=unspecified: status=saved mail to INBOX: Jan 23 08:42:07 klsrv postfix/lmtp[10844]: 1F907F00276: to=, relay=klsrv.airhop.net[private/dovecot-lmtp], delay=28, delays=28/0.01/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0 wvU2Gq9PwlRdKgAAjuDoqw Saved) Jan 23 08:42:07 klsrv dovecot: lmtp(10845): Disconnect from local: Client quit (in reset) Jan 23 08:42:07 klsrv postfix/qmgr[4889]: 1F907F00276: removed You can see that Postfix accepts the incoming mail (from sender at example.com) and then the message is handed to Dovecot LMTP. But the from=: should be "from=sender at example.com:". Postfix / qmgr has the sender listed (from=) but no sender (from=) is seen in LMTP log output. My LMTP is setup to use UNIX sockets, not the INET socket. Any help is appreciated. From skdovecot at smail.inf.fh-brs.de Fri Jan 23 15:14:10 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 23 Jan 2015 16:14:10 +0100 (CET) Subject: LDA input validation In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 23 Jan 2015, St?phane Cottin wrote: > I'm using qmail + dspam + dovecot-lda. > > A typical .qmail file : > > | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" > > When dspam segfault or is killed, preline receive an empty content and only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). Those two lines are generated by dovecot-lda or preline. > Then dovecot-lda delivers successfully and the original message contents are lost. > > Should dovecot-lda returns an error in such conditions ( invalid contents ) ? Qmail will keep it in the queue and retry delivery later. That's the problem of the used Unix pipe and broken by design. The right member of the pipe (preline) runs independendly of dspam and recieves no content via pipe. Why should preline or dovecot-lda believe there is an error? If preline does not have no nifty option to prevent this, replace the pipe by a shell script: | /usr/local/bin/dspam-lda-wrapper.sh "$EXT@$USER" "$EXT2" ===== /usr/local/bin/dspam-lda-wrapper.sh #!/bin/bash tmpf="/tmp/wrapper.$$.log" trap 'rc=$?; rm -f $tmpf; exit $rc' EXIT if /usr/bin/dspam --client --deliver=stdout --user "$1" >$tmpf; then test -s $tmpf && /usr/bin/preline -f \ /usr/lib/dovecot/dovecot-lda -d "$1" -a "$1" -m "$2" <$tmpf fi ====== - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMJlQnz1H7kL/d9rAQK90Af9Fs+uZ6P0VxQL0r8PpR30hYt1MnExozpz QXbhKMCrXv5fZwAZ6paAbQQ999Emjj0fyBEBKodgYoi3rQm5claJwb1/NzSotgxx Kayc0WpPP2cPMR1QZBA1FmR6+XPyEQO1h8B6aYxNNVNELCMxzotQ5hBPsqrR8ajO 86TL7FYaz6EM5nqW7w/VwwqsPDiM3mlkgckQudMzxeupP/jGZiRAkWODznSRg4Wr yHx257i8jWj3mV2vcew16tyBuwmc3g1renPSwjaUZoLsb2A9vyJjnr0qmBdfrkDz LE6WICAJcZuxQtyy8naTExEGKyQUhBc0nOLVljZ6ZnVC15b8gHWYRw== =WdEr -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Fri Jan 23 15:18:35 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 23 Jan 2015 16:18:35 +0100 (CET) Subject: Client shows null Sender & date In-Reply-To: <1422025582.9698.11.camel@asus-ub> References: <1422025582.9698.11.camel@asus-ub> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 23 Jan 2015, John Hendrich wrote: > I'm using Postfix and Dovecot 2.0.19 and Virtual domains & users > (mysql). Incoming mail is handled by Postfix and then handed off to > Dovecot LMTP for delivery. However, the Sender and Date are essentially > null when viewing the email with either the POP3 or IMAP client. The > logs (below) show this. I changed the sender and recipient addresses > for privacy. > > Jan 23 08:41:48 klsrv postfix/smtpd[10836]: 1F907F00276: > client=unknown[10.0.0.19] > Jan 23 08:42:07 klsrv postfix/cleanup[10842]: 1F907F00276: message-id=<> > Jan 23 08:42:07 klsrv postfix/qmgr[4889]: 1F907F00276: > from=, size=217, nrcpt=1 (queue active) > Jan 23 08:42:07 klsrv dovecot: lmtp(10845): Connect from local > Jan 23 08:42:07 klsrv dovecot: lmtp(10845, recipient at example.net): > wvU2Gq9PwlRdKgAAjuDoqw: subject=test to client: from=: > msgid=unspecified: status=saved mail to INBOX: > Jan 23 08:42:07 klsrv postfix/lmtp[10844]: 1F907F00276: > to=, > relay=klsrv.airhop.net[private/dovecot-lmtp], delay=28, > delays=28/0.01/0.01/0.01, dsn=2.0.0, status=sent (250 2.0.0 > wvU2Gq9PwlRdKgAAjuDoqw Saved) > Jan 23 08:42:07 klsrv dovecot: lmtp(10845): Disconnect from local: > Client quit (in reset) > Jan 23 08:42:07 klsrv postfix/qmgr[4889]: 1F907F00276: removed > > You can see that Postfix accepts the incoming mail (from > sender at example.com) and then the message is handed to Dovecot LMTP. But > the from=: should be "from=sender at example.com:". > > Postfix / qmgr has the sender listed (from=) but no > sender (from=) is seen in LMTP log output. My LMTP is setup to use UNIX > sockets, not the INET socket. > > Any help is appreciated. 1) Where is your Postfix conf, one can comment on? 2) The message-id is empty or absent, did you _verified_, that the message contains "From" and "Date" headers at all? I guess it does not contain them either. So, no client can display missing information. Postfix's "from" is the envelope from, that is storred in the Return-Path header in the message the client retrieves. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMJmS3z1H7kL/d9rAQJg/Qf9EOuwdzwfJrJZ2qqUhFHiq5PlbHoi8N4m 88LH3YrSsGFc30xZ6BCEPywdRJHDOtOkd8GiiV4q+T+wLopSrLpVqCUBgJaSsKsJ 2wMD7Nx4QYQZz8PHalhwR9Tnl5m9A6w8j8gemJnhvweLKFQyxLw0Z6NIzQEUNo14 XFTeJVeTHGBNC6BviX+PgAVQkk50gQd/ZH9FVB5FiNnE0TJWcm5Y1nMg1vzWZpBy IvfAPzgXDnwkAocKAhqjxPTlLkg4JQ7vDcG4w3BQPQj9EeMT8bm3lDix0juLznvX hu5sE3LJY6JDiBogAplR+TwDe7GXg+ndYKsIzUrIdxVYsROc46vf+g== =M+qD -----END PGP SIGNATURE----- From h.reindl at thelounge.net Fri Jan 23 16:04:11 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 23 Jan 2015 17:04:11 +0100 Subject: Client shows null Sender & date In-Reply-To: <1422025582.9698.11.camel@asus-ub> References: <1422025582.9698.11.camel@asus-ub> Message-ID: <54C270FB.1030907@thelounge.net> Am 23.01.2015 um 16:06 schrieb John Hendrich: > I'm using Postfix and Dovecot 2.0.19 and Virtual domains & users > (mysql). Incoming mail is handled by Postfix and then handed off to > Dovecot LMTP for delivery. However, the Sender and Date are essentially > null when viewing the email with either the POP3 or IMAP client. The > logs (below) show this. I changed the sender and recipient addresses > for privacy. > > Jan 23 08:42:07 klsrv postfix/cleanup[10842]: 1F907F00276: message-id=<> > Jan 23 08:42:07 klsrv postfix/qmgr[4889]: 1F907F00276: > from=, size=217, nrcpt=1 (queue active) > > You can see that Postfix accepts the incoming mail (from > sender at example.com) and then the message is handed to Dovecot LMTP. But > the from=: should be "from=sender at example.com:". > > Postfix / qmgr has the sender listed (from=) but no > sender (from=) is seen in LMTP log output. My LMTP is setup to use UNIX > sockets, not the INET socket you need to understand e-mail basics what you see in the postfix logs is the *envelope sender* what you miss in the message is the non-existent To-HEADER -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From michael.h.williamson at gmail.com Fri Jan 23 17:04:25 2015 From: michael.h.williamson at gmail.com (Michael Williamson) Date: Fri, 23 Jan 2015 10:04:25 -0700 Subject: sieve filter not working In-Reply-To: <50a83e3e19bdca78adde70dabfbfe0de@valo.at> References: <20150122194959.GA1479@nihlus.leuxner.net> <50a83e3e19bdca78adde70dabfbfe0de@valo.at> Message-ID: HI, > You could set > > syslog_facility = local5 > > and have all the log messages in the messages file. According to the output of command # doveadm log find every type of message goes to the file I was looking at, "/var/spool/maillog". >> So, is it postfix doing the local mail delivery, not dovecot? >> > To answer this question please post relevant parts of your postfix > main.cf > - mailbox_command > - virtual_transport > - mailbox_transport Those three do not appear to be assigned in "main.cf" (two are commented out and 'virtual_transport' not there): # The mailbox_command parameter specifies the optional external # command to use instead of mailbox delivery. The command is run as # the recipient with proper HOME, SHELL and LOGNAME environment settings. # Exception: delivery for root is done as $default_user. # # Other environment variables of interest: USER (recipient username), # EXTENSION (address extension), DOMAIN (domain part of address), # and LOCAL (the address localpart). # # Unlike other Postfix configuration parameters, the mailbox_command # parameter is not subjected to $parameter substitutions. This is to # make it easier to specify shell syntax (see example below). # # Avoid shell meta characters because they will force Postfix to run # an expensive shell process. Procmail alone is expensive enough. # # IF YOU USE THIS TO DELIVER MAIL SYSTEM-WIDE, YOU MUST SET UP AN # ALIAS THAT FORWARDS MAIL FOR ROOT TO A REAL USER. # #mailbox_command = /some/where/procmail #mailbox_command = /some/where/procmail -a "$EXTENSION" # The mailbox_transport specifies the optional transport in master.cf # to use after processing aliases and .forward files. This parameter # has precedence over the mailbox_command, fallback_transport and # luser_relay parameters. # # Specify a string of the form transport:nexthop, where transport is # the name of a mail delivery transport defined in master.cf. The # :nexthop part is optional. For more details see the sample transport # configuration file. # # NOTE: if you use this feature for accounts not in the UNIX password # file, then you must update the "local_recipient_maps" setting in # the main.cf file, otherwise the SMTP server will reject mail for # non-UNIX accounts with "User unknown in local recipient table". # #mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp Thanks, -Mike From ml+dovecot at valo.at Fri Jan 23 18:38:42 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Fri, 23 Jan 2015 19:38:42 +0100 Subject: sieve filter not working In-Reply-To: References: <20150122194959.GA1479@nihlus.leuxner.net> <50a83e3e19bdca78adde70dabfbfe0de@valo.at> Message-ID: On 2015-01-23 18:04, Michael Williamson wrote: > HI, Hello, > >> You could set >> >> syslog_facility = local5 >> >> and have all the log messages in the messages file. > > According to the output of command > > # doveadm log find > > every type of message goes to the file I was looking at, > "/var/spool/maillog". > Ok, then the information should be in /var/log/messages when you up the verbosity in 10-logging.conf by setting mail_debug=yes and restart dovecot. This should work as expected and show what dovecot is doing. Another question: Local mail delivery of received mail from external sources does indeed work? If so, was that log excerpt in one of the former mails an example of such delivery? > >>> So, is it postfix doing the local mail delivery, not dovecot? >>> >> To answer this question please post relevant parts of your postfix >> main.cf >> - mailbox_command >> - virtual_transport >> - mailbox_transport > > Those three do not appear to be assigned in "main.cf" (two are > commented out > and 'virtual_transport' not there): > > # The mailbox_command parameter specifies the optional external > # command to use instead of mailbox delivery. The command is run as > # the recipient with proper HOME, SHELL and LOGNAME environment > settings. > # Exception: delivery for root is done as $default_user. > # > # Other environment variables of interest: USER (recipient username), > # EXTENSION (address extension), DOMAIN (domain part of address), > # and LOCAL (the address localpart). > # > # Unlike other Postfix configuration parameters, the mailbox_command > # parameter is not subjected to $parameter substitutions. This is to > # make it easier to specify shell syntax (see example below). > # > # Avoid shell meta characters because they will force Postfix to run > # an expensive shell process. Procmail alone is expensive enough. > # > # IF YOU USE THIS TO DELIVER MAIL SYSTEM-WIDE, YOU MUST SET UP AN > # ALIAS THAT FORWARDS MAIL FOR ROOT TO A REAL USER. > # > #mailbox_command = /some/where/procmail > #mailbox_command = /some/where/procmail -a "$EXTENSION" > > # The mailbox_transport specifies the optional transport in master.cf > # to use after processing aliases and .forward files. This parameter > # has precedence over the mailbox_command, fallback_transport and > # luser_relay parameters. > # > # Specify a string of the form transport:nexthop, where transport is > # the name of a mail delivery transport defined in master.cf. The > # :nexthop part is optional. For more details see the sample transport > # configuration file. > # > # NOTE: if you use this feature for accounts not in the UNIX password > # file, then you must update the "local_recipient_maps" setting in > # the main.cf file, otherwise the SMTP server will reject mail for > # non-UNIX accounts with "User unknown in local recipient table". > # > #mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp > Ok, none set. Could you post the output of postconf to the list so we are able to see your running postfix config. I believe dovecot has actually nothing to do with delivering your mails so no sieve filters are applied... > Thanks, > -Mike - christian From michael.h.williamson at gmail.com Fri Jan 23 21:02:06 2015 From: michael.h.williamson at gmail.com (Michael Williamson) Date: Fri, 23 Jan 2015 14:02:06 -0700 Subject: sieve filter not working In-Reply-To: References: <20150122194959.GA1479@nihlus.leuxner.net> <50a83e3e19bdca78adde70dabfbfe0de@valo.at> Message-ID: On 1/23/15, Christian Kivalo wrote: > On 2015-01-23 18:04, Michael Williamson wrote: >> HI, > Hello, > >> >>> You could set >>> >>> syslog_facility = local5 >>> >>> and have all the log messages in the messages file. >> >> According to the output of command >> >> # doveadm log find >> >> every type of message goes to the file I was looking at, >> "/var/spool/maillog". >> > Ok, then the information should be in /var/log/messages when you up the > verbosity in 10-logging.conf by setting > > mail_debug=yes > > and restart dovecot. This should work as expected and show what dovecot > is doing. I tried this again (set mail_debug=yes and restart dovecot) but no dovecot messages appear in "/var/log/messages". I sent spam email, too. Also, no change appears in the output of # doveadm log find > Another question: > Local mail delivery of received mail from external sources does indeed > work? Yes. > If so, was that log excerpt in one of the former mails an example of > such delivery? There are dovecot messages in "/var/log/maillog" showing only imap-login & disconnected. When I monitor "/var/log/maillog" while sending spam, there are no messages containing 'dovecot'. I attached that output to a previous post. It has only messages from postfix and amavis. > >> >>>> So, is it postfix doing the local mail delivery, not dovecot? >>>> >>> To answer this question please post relevant parts of your postfix >>> main.cf >>> - mailbox_command >>> - virtual_transport >>> - mailbox_transport >> >> Those three do not appear to be assigned in "main.cf" (two are >> commented out >> and 'virtual_transport' not there): >> >> # The mailbox_command parameter specifies the optional external >> # command to use instead of mailbox delivery. The command is run as >> # the recipient with proper HOME, SHELL and LOGNAME environment >> settings. >> # Exception: delivery for root is done as $default_user. >> # >> # Other environment variables of interest: USER (recipient username), >> # EXTENSION (address extension), DOMAIN (domain part of address), >> # and LOCAL (the address localpart). >> # >> # Unlike other Postfix configuration parameters, the mailbox_command >> # parameter is not subjected to $parameter substitutions. This is to >> # make it easier to specify shell syntax (see example below). >> # >> # Avoid shell meta characters because they will force Postfix to run >> # an expensive shell process. Procmail alone is expensive enough. >> # >> # IF YOU USE THIS TO DELIVER MAIL SYSTEM-WIDE, YOU MUST SET UP AN >> # ALIAS THAT FORWARDS MAIL FOR ROOT TO A REAL USER. >> # >> #mailbox_command = /some/where/procmail >> #mailbox_command = /some/where/procmail -a "$EXTENSION" >> >> # The mailbox_transport specifies the optional transport in master.cf >> # to use after processing aliases and .forward files. This parameter >> # has precedence over the mailbox_command, fallback_transport and >> # luser_relay parameters. >> # >> # Specify a string of the form transport:nexthop, where transport is >> # the name of a mail delivery transport defined in master.cf. The >> # :nexthop part is optional. For more details see the sample transport >> # configuration file. >> # >> # NOTE: if you use this feature for accounts not in the UNIX password >> # file, then you must update the "local_recipient_maps" setting in >> # the main.cf file, otherwise the SMTP server will reject mail for >> # non-UNIX accounts with "User unknown in local recipient table". >> # >> #mailbox_transport = lmtp:unix:/var/lib/imap/socket/lmtp >> > Ok, none set. Could you post the output of postconf to the list so we > are able to see your running postfix config. OK. It is long (631 lines). 2bounce_notice_recipient = postmaster access_map_defer_code = 450 access_map_reject_code = 554 address_verify_default_transport = $default_transport address_verify_local_transport = $local_transport address_verify_map = address_verify_negative_cache = yes address_verify_negative_expire_time = 3d address_verify_negative_refresh_time = 3h address_verify_poll_count = ${stress?1}${stress:3} address_verify_poll_delay = 3s address_verify_positive_expire_time = 31d address_verify_positive_refresh_time = 7d address_verify_relay_transport = $relay_transport address_verify_relayhost = $relayhost address_verify_sender = $double_bounce_sender address_verify_sender_dependent_relayhost_maps = $sender_dependent_relayhost_maps address_verify_service_name = verify address_verify_transport_maps = $transport_maps address_verify_virtual_transport = $virtual_transport alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases allow_mail_to_commands = alias, forward allow_mail_to_files = alias, forward allow_min_user = no allow_percent_hack = yes allow_untrusted_routing = no alternate_config_directories = always_add_missing_headers = no always_bcc = anvil_rate_time_unit = 60s anvil_status_update_time = 600s append_at_myorigin = yes append_dot_mydomain = yes application_event_drain_time = 100s authorized_flush_users = static:anyone authorized_mailq_users = static:anyone authorized_submit_users = static:anyone backwards_bounce_logfile_compatibility = yes berkeley_db_create_buffer_size = 16777216 berkeley_db_read_buffer_size = 131072 best_mx_transport = biff = yes body_checks = body_checks_size_limit = 51200 bounce_notice_recipient = postmaster bounce_queue_lifetime = 5d bounce_service_name = bounce bounce_size_limit = 50000 bounce_template_file = broken_sasl_auth_clients = yes canonical_classes = envelope_sender, envelope_recipient, header_sender, header_recipient canonical_maps = cleanup_service_name = cleanup command_directory = /usr/sbin command_execution_directory = command_expansion_filter = 1234567890!@%-_=+:,./abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ command_time_limit = 1000s config_directory = /etc/postfix connection_cache_protocol_timeout = 5s connection_cache_service_name = scache connection_cache_status_update_time = 600s connection_cache_ttl_limit = 2s content_filter = amavisfeed:[127.0.0.1]:10024 cyrus_sasl_config_path = daemon_directory = /usr/libexec/postfix daemon_timeout = 18000s data_directory = /var/lib/postfix debug_peer_level = 2 debug_peer_list = default_database_type = hash default_delivery_slot_cost = 5 default_delivery_slot_discount = 50 default_delivery_slot_loan = 3 default_destination_concurrency_failed_cohort_limit = 1 default_destination_concurrency_limit = 20 default_destination_concurrency_negative_feedback = 1 default_destination_concurrency_positive_feedback = 1 default_destination_rate_delay = 0s default_destination_recipient_limit = 50 default_extra_recipient_limit = 1000 default_minimum_delivery_slots = 3 default_privs = nobody default_process_limit = 100 default_rbl_reply = $rbl_code Service unavailable; $rbl_class [$rbl_what] blocked using $rbl_domain${rbl_reason?; $rbl_reason} default_recipient_limit = 20000 default_recipient_refill_delay = 5s default_recipient_refill_limit = 100 default_transport = smtp default_verp_delimiters = += defer_code = 450 defer_service_name = defer defer_transports = delay_logging_resolution_limit = 2 delay_notice_recipient = postmaster delay_warning_time = 0h deliver_lock_attempts = 20 deliver_lock_delay = 1s destination_concurrency_feedback_debug = no detect_8bit_encoding_header = yes disable_dns_lookups = no disable_mime_input_processing = no disable_mime_output_conversion = no disable_verp_bounces = no disable_vrfy_command = no dont_remove = 0 double_bounce_sender = double-bounce duplicate_filter_limit = 1000 empty_address_recipient = MAILER-DAEMON empty_address_relayhost_maps_lookup_key = <> enable_original_recipient = yes error_notice_recipient = postmaster error_service_name = error execution_directory_expansion_filter = 1234567890!@%-_=+:,./abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ expand_owner_alias = no export_environment = TZ MAIL_CONFIG LANG fallback_transport = fallback_transport_maps = fast_flush_domains = $relay_domains fast_flush_purge_time = 7d fast_flush_refresh_time = 12h fault_injection_code = 0 flush_service_name = flush fork_attempts = 5 fork_delay = 1s forward_expansion_filter = 1234567890!@%-_=+:,./abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ forward_path = $home/.forward${recipient_delimiter}${extension}, $home/.forward frozen_delivered_to = yes hash_queue_depth = 1 hash_queue_names = deferred, defer header_address_token_limit = 10240 header_checks = header_size_limit = 102400 helpful_warnings = yes home_mailbox = Maildir/ hopcount_limit = 50 html_directory = no ignore_mx_lookup_error = no import_environment = MAIL_CONFIG MAIL_DEBUG MAIL_LOGTAG TZ XAUTHORITY DISPLAY LANG=C in_flow_delay = 1s inet_interfaces = all inet_protocols = all initial_destination_concurrency = 5 internal_mail_filter_classes = invalid_hostname_reject_code = 501 ipc_idle = 5s ipc_timeout = 3600s ipc_ttl = 1000s line_length_limit = 2048 lmtp_assume_final = no lmtp_bind_address = lmtp_bind_address6 = lmtp_body_checks = lmtp_cname_overrides_servername = no lmtp_connect_timeout = 0s lmtp_connection_cache_destinations = lmtp_connection_cache_on_demand = yes lmtp_connection_cache_time_limit = 2s lmtp_connection_reuse_time_limit = 300s lmtp_data_done_timeout = 600s lmtp_data_init_timeout = 120s lmtp_data_xfer_timeout = 180s lmtp_defer_if_no_mx_address_found = no lmtp_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit lmtp_destination_concurrency_limit = $default_destination_concurrency_limit lmtp_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback lmtp_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback lmtp_destination_rate_delay = $default_destination_rate_delay lmtp_destination_recipient_limit = $default_destination_recipient_limit lmtp_discard_lhlo_keyword_address_maps = lmtp_discard_lhlo_keywords = lmtp_enforce_tls = no lmtp_generic_maps = lmtp_header_checks = lmtp_host_lookup = dns lmtp_initial_destination_concurrency = $initial_destination_concurrency lmtp_lhlo_name = $myhostname lmtp_lhlo_timeout = 300s lmtp_line_length_limit = 990 lmtp_mail_timeout = 300s lmtp_mime_header_checks = lmtp_mx_address_limit = 5 lmtp_mx_session_limit = 2 lmtp_nested_header_checks = lmtp_pix_workaround_delay_time = 10s lmtp_pix_workaround_maps = lmtp_pix_workaround_threshold_time = 500s lmtp_pix_workarounds = disable_esmtp,delay_dotcrlf lmtp_quit_timeout = 300s lmtp_quote_rfc821_envelope = yes lmtp_randomize_addresses = yes lmtp_rcpt_timeout = 300s lmtp_rset_timeout = 20s lmtp_sasl_auth_cache_name = lmtp_sasl_auth_cache_time = 90d lmtp_sasl_auth_enable = no lmtp_sasl_auth_soft_bounce = yes lmtp_sasl_mechanism_filter = lmtp_sasl_password_maps = lmtp_sasl_path = lmtp_sasl_security_options = noplaintext, noanonymous lmtp_sasl_tls_security_options = $lmtp_sasl_security_options lmtp_sasl_tls_verified_security_options = $lmtp_sasl_tls_security_options lmtp_sasl_type = cyrus lmtp_send_xforward_command = no lmtp_sender_dependent_authentication = no lmtp_skip_5xx_greeting = yes lmtp_skip_quit_response = no lmtp_starttls_timeout = 300s lmtp_tcp_port = 24 lmtp_tls_CAfile = lmtp_tls_CApath = lmtp_tls_cert_file = lmtp_tls_ciphers = export lmtp_tls_dcert_file = lmtp_tls_dkey_file = $lmtp_tls_dcert_file lmtp_tls_eccert_file = lmtp_tls_eckey_file = $lmtp_tls_eccert_file lmtp_tls_enforce_peername = yes lmtp_tls_exclude_ciphers = lmtp_tls_fingerprint_cert_match = lmtp_tls_fingerprint_digest = md5 lmtp_tls_key_file = $lmtp_tls_cert_file lmtp_tls_loglevel = 0 lmtp_tls_mandatory_ciphers = medium lmtp_tls_mandatory_exclude_ciphers = lmtp_tls_mandatory_protocols = SSLv3, TLSv1 lmtp_tls_note_starttls_offer = no lmtp_tls_per_site = lmtp_tls_policy_maps = lmtp_tls_protocols = !SSLv2 lmtp_tls_scert_verifydepth = 9 lmtp_tls_secure_cert_match = nexthop lmtp_tls_security_level = lmtp_tls_session_cache_database = lmtp_tls_session_cache_timeout = 3600s lmtp_tls_verify_cert_match = hostname lmtp_use_tls = no lmtp_xforward_timeout = 300s local_command_shell = local_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit local_destination_concurrency_limit = 2 local_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback local_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback local_destination_rate_delay = $default_destination_rate_delay local_destination_recipient_limit = 1 local_header_rewrite_clients = permit_inet_interfaces local_initial_destination_concurrency = $initial_destination_concurrency local_recipient_maps = proxy:unix:passwd.byname $alias_maps local_transport = local:$myhostname luser_relay = mail_name = Postfix mail_owner = postfix mail_release_date = 20100319 mail_spool_directory = /var/mail mail_version = 2.6.6 mailbox_command = mailbox_command_maps = mailbox_delivery_lock = fcntl, dotlock mailbox_size_limit = 1000000000 mailbox_transport = mailbox_transport_maps = mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man maps_rbl_domains = maps_rbl_reject_code = 554 masquerade_classes = envelope_sender, header_sender, header_recipient masquerade_domains = masquerade_exceptions = master_service_disable = max_idle = 100s max_use = 100 maximal_backoff_time = 4000s maximal_queue_lifetime = 5d message_reject_characters = message_size_limit = 30000000 message_strip_characters = milter_command_timeout = 30s milter_connect_macros = j {daemon_name} v milter_connect_timeout = 30s milter_content_timeout = 300s milter_data_macros = i milter_default_action = tempfail milter_end_of_data_macros = i milter_end_of_header_macros = i milter_helo_macros = {tls_version} {cipher} {cipher_bits} {cert_subject} {cert_issuer} milter_macro_daemon_name = $myhostname milter_macro_v = $mail_name $mail_version milter_mail_macros = i {auth_type} {auth_authen} {auth_author} {mail_addr} {mail_host} {mail_mailer} milter_protocol = 6 milter_rcpt_macros = i {rcpt_addr} {rcpt_host} {rcpt_mailer} milter_unknown_command_macros = mime_boundary_length_limit = 2048 mime_header_checks = $header_checks mime_nesting_limit = 100 minimal_backoff_time = 300s multi_instance_directories = multi_instance_enable = no multi_instance_group = multi_instance_name = multi_instance_wrapper = multi_recipient_bounce_reject_code = 550 mydestination = $myhostname, localhost.$mydomain, localhost mydomain = coe.tsuniv.edu myhostname = coe.tsuniv.edu mynetworks = 127.0.0.0/8, 206.23.240.0/24 mynetworks_style = subnet myorigin = $mydomain nested_header_checks = $header_checks newaliases_path = /usr/bin/newaliases.postfix non_fqdn_reject_code = 504 non_smtpd_milters = notify_classes = resource, software owner_request_special = yes parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks,permit_mx_backup_networks,qmqpd_authorized_clients,relay_domains,smtpd_access_maps permit_mx_backup_networks = pickup_service_name = pickup plaintext_reject_code = 450 postmulti_control_commands = reload flush postmulti_start_commands = start postmulti_stop_commands = stop abort drain quick-stop prepend_delivered_header = command, file, forward process_id_directory = pid propagate_unmatched_extensions = canonical, virtual proxy_interfaces = proxy_read_maps = $local_recipient_maps $mydestination $virtual_alias_maps $virtual_alias_domains $virtual_mailbox_maps $virtual_mailbox_domains $relay_recipient_maps $relay_domains $canonical_maps $sender_canonical_maps $recipient_canonical_maps $relocated_maps $transport_maps $mynetworks $sender_bcc_maps $recipient_bcc_maps $smtp_generic_maps $lmtp_generic_maps proxy_write_maps = $smtp_sasl_auth_cache_name $lmtp_sasl_auth_cache_name proxymap_service_name = proxymap proxywrite_service_name = proxywrite qmgr_clog_warn_time = 300s qmgr_fudge_factor = 100 qmgr_message_active_limit = 20000 qmgr_message_recipient_limit = 20000 qmgr_message_recipient_minimum = 10 qmqpd_authorized_clients = qmqpd_client_port_logging = no qmqpd_error_delay = 1s qmqpd_timeout = 300s queue_directory = /var/spool/postfix queue_file_attribute_count_limit = 100 queue_minfree = 0 queue_run_delay = 300s queue_service_name = qmgr rbl_reply_maps = readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES receive_override_options = recipient_bcc_maps = recipient_canonical_classes = envelope_recipient, header_recipient recipient_canonical_maps = recipient_delimiter = reject_code = 554 reject_tempfail_action = defer_if_permit relay_clientcerts = relay_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit relay_destination_concurrency_limit = $default_destination_concurrency_limit relay_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback relay_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback relay_destination_rate_delay = $default_destination_rate_delay relay_destination_recipient_limit = $default_destination_recipient_limit relay_domains = relay_domains_reject_code = 554 relay_initial_destination_concurrency = $initial_destination_concurrency relay_recipient_maps = relay_transport = relay relayhost = relocated_maps = remote_header_rewrite_domain = require_home_directory = no resolve_dequoted_address = yes resolve_null_domain = no resolve_numeric_domain = no rewrite_service_name = rewrite sample_directory = /usr/share/doc/postfix-2.6.6/samples send_cyrus_sasl_authzid = no sender_bcc_maps = sender_canonical_classes = envelope_sender, header_sender sender_canonical_maps = sender_dependent_relayhost_maps = sendmail_path = /usr/sbin/sendmail.postfix service_throttle_time = 60s setgid_group = postdrop show_user_unknown_table_name = yes showq_service_name = showq smtp_always_send_ehlo = yes smtp_bind_address = smtp_bind_address6 = smtp_body_checks = smtp_cname_overrides_servername = no smtp_connect_timeout = 30s smtp_connection_cache_destinations = smtp_connection_cache_on_demand = yes smtp_connection_cache_time_limit = 2s smtp_connection_reuse_time_limit = 300s smtp_data_done_timeout = 600s smtp_data_init_timeout = 120s smtp_data_xfer_timeout = 180s smtp_defer_if_no_mx_address_found = no smtp_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit smtp_destination_concurrency_limit = $default_destination_concurrency_limit smtp_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback smtp_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback smtp_destination_rate_delay = $default_destination_rate_delay smtp_destination_recipient_limit = $default_destination_recipient_limit smtp_discard_ehlo_keyword_address_maps = smtp_discard_ehlo_keywords = smtp_enforce_tls = no smtp_fallback_relay = $fallback_relay smtp_generic_maps = smtp_header_checks = smtp_helo_name = $myhostname smtp_helo_timeout = 300s smtp_host_lookup = dns smtp_initial_destination_concurrency = $initial_destination_concurrency smtp_line_length_limit = 990 smtp_mail_timeout = 300s smtp_mime_header_checks = smtp_mx_address_limit = 5 smtp_mx_session_limit = 2 smtp_nested_header_checks = smtp_never_send_ehlo = no smtp_pix_workaround_delay_time = 10s smtp_pix_workaround_maps = smtp_pix_workaround_threshold_time = 500s smtp_pix_workarounds = disable_esmtp,delay_dotcrlf smtp_quit_timeout = 300s smtp_quote_rfc821_envelope = yes smtp_randomize_addresses = yes smtp_rcpt_timeout = 300s smtp_rset_timeout = 20s smtp_sasl_auth_cache_name = smtp_sasl_auth_cache_time = 90d smtp_sasl_auth_enable = no smtp_sasl_auth_soft_bounce = yes smtp_sasl_mechanism_filter = smtp_sasl_password_maps = smtp_sasl_path = smtp_sasl_security_options = noplaintext, noanonymous smtp_sasl_tls_security_options = $smtp_sasl_security_options smtp_sasl_tls_verified_security_options = $smtp_sasl_tls_security_options smtp_sasl_type = cyrus smtp_send_xforward_command = no smtp_sender_dependent_authentication = no smtp_skip_5xx_greeting = yes smtp_skip_quit_response = yes smtp_starttls_timeout = 300s smtp_tls_CAfile = smtp_tls_CApath = smtp_tls_cert_file = smtp_tls_ciphers = export smtp_tls_dcert_file = smtp_tls_dkey_file = $smtp_tls_dcert_file smtp_tls_eccert_file = smtp_tls_eckey_file = $smtp_tls_eccert_file smtp_tls_enforce_peername = yes smtp_tls_exclude_ciphers = smtp_tls_fingerprint_cert_match = smtp_tls_fingerprint_digest = md5 smtp_tls_key_file = $smtp_tls_cert_file smtp_tls_loglevel = 0 smtp_tls_mandatory_ciphers = medium smtp_tls_mandatory_exclude_ciphers = smtp_tls_mandatory_protocols = SSLv3, TLSv1 smtp_tls_note_starttls_offer = no smtp_tls_per_site = smtp_tls_policy_maps = smtp_tls_protocols = !SSLv2 smtp_tls_scert_verifydepth = 9 smtp_tls_secure_cert_match = nexthop, dot-nexthop smtp_tls_security_level = smtp_tls_session_cache_database = smtp_tls_session_cache_timeout = 3600s smtp_tls_verify_cert_match = hostname smtp_use_tls = no smtp_xforward_timeout = 300s smtpd_authorized_verp_clients = $authorized_verp_clients smtpd_authorized_xclient_hosts = smtpd_authorized_xforward_hosts = smtpd_banner = $myhostname ESMTP $mail_name smtpd_client_connection_count_limit = 50 smtpd_client_connection_rate_limit = 0 smtpd_client_event_limit_exceptions = ${smtpd_client_connection_limit_exceptions:$mynetworks} smtpd_client_message_rate_limit = 0 smtpd_client_new_tls_session_rate_limit = 0 smtpd_client_port_logging = no smtpd_client_recipient_rate_limit = 0 smtpd_client_restrictions = smtpd_data_restrictions = smtpd_delay_open_until_valid_rcpt = yes smtpd_delay_reject = yes smtpd_discard_ehlo_keyword_address_maps = smtpd_discard_ehlo_keywords = smtpd_end_of_data_restrictions = smtpd_enforce_tls = no smtpd_error_sleep_time = 1s smtpd_etrn_restrictions = smtpd_expansion_filter = \t\40!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~ smtpd_forbidden_commands = CONNECT GET POST smtpd_hard_error_limit = ${stress?1}${stress:20} smtpd_helo_required = no smtpd_helo_restrictions = smtpd_history_flush_threshold = 100 smtpd_junk_command_limit = ${stress?1}${stress:100} smtpd_milters = smtpd_noop_commands = smtpd_null_access_lookup_key = <> smtpd_peername_lookup = yes smtpd_policy_service_max_idle = 300s smtpd_policy_service_max_ttl = 1000s smtpd_policy_service_timeout = 100s smtpd_proxy_ehlo = $myhostname smtpd_proxy_filter = smtpd_proxy_timeout = 100s smtpd_recipient_limit = 1000 smtpd_recipient_overshoot_limit = 1000 smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_reject_unlisted_recipient = yes smtpd_reject_unlisted_sender = no smtpd_restriction_classes = smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = no smtpd_sasl_exceptions_networks = smtpd_sasl_local_domain = smtpd_sasl_path = private/auth smtpd_sasl_security_options = noanonymous smtpd_sasl_tls_security_options = $smtpd_sasl_security_options smtpd_sasl_type = dovecot smtpd_sender_login_maps = smtpd_sender_restrictions = smtpd_soft_error_limit = 10 smtpd_starttls_timeout = 300s smtpd_timeout = ${stress?10}${stress:300}s smtpd_tls_CAfile = smtpd_tls_CApath = smtpd_tls_always_issue_session_ids = yes smtpd_tls_ask_ccert = no smtpd_tls_auth_only = yes smtpd_tls_ccert_verifydepth = 9 smtpd_tls_cert_file = /etc/pki/tls/certs/coe.tsuniv.edu.crt smtpd_tls_ciphers = export smtpd_tls_dcert_file = smtpd_tls_dh1024_param_file = smtpd_tls_dh512_param_file = smtpd_tls_dkey_file = $smtpd_tls_dcert_file smtpd_tls_eccert_file = smtpd_tls_eckey_file = $smtpd_tls_eccert_file smtpd_tls_eecdh_grade = none smtpd_tls_exclude_ciphers = smtpd_tls_fingerprint_digest = md5 smtpd_tls_key_file = /etc/pki/tls/private/coe.tsuniv.edu.key smtpd_tls_loglevel = 1 smtpd_tls_mandatory_ciphers = medium smtpd_tls_mandatory_exclude_ciphers = smtpd_tls_mandatory_protocols = SSLv3, TLSv1 smtpd_tls_protocols = smtpd_tls_received_header = no smtpd_tls_req_ccert = no smtpd_tls_security_level = may smtpd_tls_session_cache_database = btree:/var/spool/postfix/smtpd_tls_cache smtpd_tls_session_cache_timeout = 3600s smtpd_tls_wrappermode = no smtpd_use_tls = no soft_bounce = no stale_lock_time = 500s stress = strict_7bit_headers = no strict_8bitmime = no strict_8bitmime_body = no strict_mailbox_ownership = yes strict_mime_encoding_domain = no strict_rfc821_envelopes = no sun_mailtool_compatibility = no swap_bangpath = yes syslog_facility = mail syslog_name = ${multi_instance_name:postfix}${multi_instance_name?$multi_instance_name} tcp_windowsize = 0 tls_daemon_random_bytes = 32 tls_eecdh_strong_curve = prime256v1 tls_eecdh_ultra_curve = secp384r1 tls_export_cipherlist = ALL:+RC4:@STRENGTH tls_high_cipherlist = ALL:!EXPORT:!LOW:!MEDIUM:+RC4:@STRENGTH tls_low_cipherlist = ALL:!EXPORT:+RC4:@STRENGTH tls_medium_cipherlist = ALL:!EXPORT:!LOW:+RC4:@STRENGTH tls_null_cipherlist = eNULL:!aNULL tls_random_bytes = 32 tls_random_exchange_name = ${data_directory}/prng_exch tls_random_prng_update_period = 3600s tls_random_reseed_period = 3600s tls_random_source = dev:/dev/urandom trace_service_name = trace transport_maps = transport_retry_time = 60s trigger_timeout = 10s undisclosed_recipients_header = To: undisclosed-recipients:; unknown_address_reject_code = 450 unknown_address_tempfail_action = $reject_tempfail_action unknown_client_reject_code = 450 unknown_helo_hostname_tempfail_action = $reject_tempfail_action unknown_hostname_reject_code = 450 unknown_local_recipient_reject_code = 550 unknown_relay_recipient_reject_code = 550 unknown_virtual_alias_reject_code = 550 unknown_virtual_mailbox_reject_code = 550 unverified_recipient_defer_code = 450 unverified_recipient_reject_code = 450 unverified_recipient_reject_reason = unverified_recipient_tempfail_action = $reject_tempfail_action unverified_sender_defer_code = 450 unverified_sender_reject_code = 450 unverified_sender_reject_reason = unverified_sender_tempfail_action = $reject_tempfail_action verp_delimiter_filter = -=+ virtual_alias_domains = $virtual_alias_maps virtual_alias_expansion_limit = 1000 virtual_alias_maps = $virtual_maps virtual_alias_recursion_limit = 1000 virtual_destination_concurrency_failed_cohort_limit = $default_destination_concurrency_failed_cohort_limit virtual_destination_concurrency_limit = $default_destination_concurrency_limit virtual_destination_concurrency_negative_feedback = $default_destination_concurrency_negative_feedback virtual_destination_concurrency_positive_feedback = $default_destination_concurrency_positive_feedback virtual_destination_rate_delay = $default_destination_rate_delay virtual_destination_recipient_limit = $default_destination_recipient_limit virtual_gid_maps = virtual_initial_destination_concurrency = $initial_destination_concurrency virtual_mailbox_base = virtual_mailbox_domains = $virtual_mailbox_maps virtual_mailbox_limit = 51200000 virtual_mailbox_lock = fcntl, dotlock virtual_mailbox_maps = virtual_minimum_uid = 100 virtual_transport = virtual virtual_uid_maps = > I believe dovecot has actually nothing to do with delivering your mails > so no sieve filters are applied... Is it postfix? >> Thanks, >> -Mike > - christian > From alvin.sm at Mail.Linux-Consulting.com Sun Jan 18 19:48:36 2015 From: alvin.sm at Mail.Linux-Consulting.com (alvin) Date: Sun, 18 Jan 2015 11:48:36 -0800 Subject: sendmail not invoking dovecot-lda Message-ID: <20150118194836.GA17728@Mail.Linux-Consulting.com> hi dovecot mailinglist - Configuration FreeBSD-9.3 sendmail -d0.1 == sendmail-8.14.9 <<-- dovecot --version == dovecot-2.2.15 # =================================================================== # I'm trying to get sendmail to invoke dovecot.m4 ( dovecot-lda ) to # deliver emails to dovecot's virtual users ( /etc/dovecot/passwd ) # or mysql/postgresql virtual users # # # sendmail -bv dovecot.VirtualUser at localhost --> returns "User Unknown" # # cat test.msg.txt | dovecot-lda -d dovecot.VirtualUser ===> works # email is delivered to dovecot.VirtualUser inbox # # =================================================================== - there are tons of howto and comments for dovecot + postfix and very little for dovecot + sendmail ... therefore, it'd be great if we/i can get this dovecot.m4 problem resolved ------------------------------------------------------------------------------ - status of what works and what does NOT work while testing dovecot's LDA ... ------------------------------------------------------------------------------ # # fyi.. recepient user is in the form of user, user at localhost and user at domain.com # sendmail -bv user.in/etc/passwd = deliverable sendmail -bv user.in/etc/mail/virtusertable = deliverable to sendmail virtual user -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown dovecot user user.in/etc/passwd == returns UID/GID/home/mailbox ( deliverable ) dovecot user dovecot.user == returns UID/GID/home/mailbox ( deliverable ) # -------------------------------------------------------------------- # dovecot's LDA delivers the test email to the dovecot/mysql recepient # -------------------------------------------------------------------- cat test-email-with-headers.txt | dovecot-lda -d dovecot.user ( email delivered ) # ---------------------------------------------------------------- -->> # mail/mutt/sendmail canNOT deliver to dovecot recepient == FAILS # ---------------------------------------------------------------- -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown 3 echo "testing mail to dovecot" | mail -v -s "testing mail to dovecot" doveccot.user == user unknown == ------------------------------------------------------- - my (relevant to dovecot ) sendmail.mc file looks like: ------------------------------------------------------- dnl # dnl # local mail delivery dnl # define(`PROCMAIL_MAILER_PATH', `/usr/local/bin/procmail')dnl FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl dnl # dnl # I'm trying to get sendmail to use dovecot.m4 to deliver local mail dnl # to dovecot's LDA ( dovecot-lda ) for mysql virtual users dnl # dnl uncomment and use either feature(...dovecot-lda) or mailer(dovecot.m4) dnl dnl FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u') dnl MAILER(local) MAILER(smtp) dnl dnl MAILER(dovecot)dnl # did NOT work either dnl dnl # trying to use procmail to deliver local mail to system users ( /etc/passwd ) MAILER(procmail) dnl dnl # is the ordering of dovecot.m4 important relative to (local) and (procmail) dnl dnl # trying to use dovecot-lda to deliver email to dovecot's virtual users dnl # defined in /etc/dovecot/password dnl MAILER(dovecot)dnl # see below dnl dnl End of File ------------------------------------------------------- - my (relevant to dovecot ) sendmail.cf file looks like: ------------------------------------------------------- # # lots of deleted ... ??not?? important until Mlocal ---> MAILER(local) # Mlocal, P=/usr/libexec/mail.local, F=lsDFMAw5:/|@qPSXmnz9, S=EnvFromSMTP/HdrFromL, R=EnvToL/ HdrToL, T=DNS/RFC822/SMTP, A=mail.local -l Mprog, P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/, T=X-Unix/X-Unix/X-Unix, A=sh -c $u ##################################### ### SMTP Mailer specification ### ##################################### # # lots of deleted ... ??not?? important until procmail stuff ---> MAILER(procmail) # ######################*****############## ### PROCMAIL Mailer specification ### ##################*****################## ##### $Id: procmail.m4,v 8.23 2013-11-22 20:51:14 ca Exp $ ##### Mprocmail, P=/usr/local/bin/procmail, F=DFMSPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFr omSMTP, T=DNS/RFC822/X-Unix, A=procmail -Y -m $h $f $u ######################*****######################### ### ### ### 10-Jan-15 amo Dovecot.m4 ### ### ### ### dovecot.m4 stuff ### ### ### ##############################*****################# Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u # # End of sendmail.cf ===================================================================== - dovecot has an example dovecot.m4 ... but it doesn't allow sendmail to call dovecot to deliver local mail with dovecot-lda ===================================================================== http://wiki2.dovecot.org/LDA/Sendmail ######################################### ### DOVECOT Mailer specification ### ######################################### # # # dovecot virtual users defined in: /etc/dovecot/password # # # # cp /etc/mail/dovecot.m4 /usr/share/sendmail/cf/mailer # # # # cd /etc/mail && make # # /etc/rc.d/sendmail restart # # # # the dovecot.m4 stanza are in the resulting sendmail.cf # # # ######################################### Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u # # i've tried using these other options in lieu of the above : # # F=DFMPShnu9 # added "S" option # F=lsDFMAw5:/|@qSPfhn9, # # S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, # # # # tried adding U= between S= and T= lines # U=root:mail U=dovecot:mail U=dovenull:mail # # End of dovecot.m4 http://wiki2.dovecot.org/LDA/Sendmail also states that one could use these 3 statements in lieu of dovecot.m4 # # this replaces sendmail.cf Mlocal P=mail.local with P=dovecot-lda # FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda', \ `/usr/local/libexec/dovecot/dovecot-lda -d $u') dnl MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail) vi /etc/mail/mailertable # # cd /etc/mail && make && /etc/rc.d/sendmail restart # # make and restart sendmail after editing # virtualdomain.example.com vmail:vmail # other options where sendmail still wasn't able to execute dovecot-lda # virtualdomain.example.com dovecot:local # virtualdomain.example.com local:mail # virtualdomain.example.com root:mail ------------------------------------ - my relevant dovecot config options ------------------------------------ # # are there any other major options i'm missing in the dovecot config files # doveconf dovecot -a # # for /etc/passwd passdb { args = driver = pam .. } # # ------------------------------------------------------------------------ # need to get these dovecot's virtual users to be able send/receive emails # ------------------------------------------------------------------------ # sendmail -bv dovecot.VirtualUser at localhost # dovecot.VirtuaUser at localhost... User unknown # passdb { args = /etc/dovecot/passwd # # dovecot.VirtualUser:{PLAIN}password::::: # # end of file driver = passwd-file .. } # passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql .. } ... ... ... userdb { args = driver = passwd .. } # userdb { args = /etc/dovecot/password driver = passwd-file .. } # userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql .. } # # --------------------------------------------------------------------- # -- How do i get sendmail to execute dovecot-lda for incoming emails # -- directed to dovecot virtual users and/or mysql users ?? # --------------------------------------------------------------------- # # end of txt From galmague at almaguer.com.mx Thu Jan 22 03:32:06 2015 From: galmague at almaguer.com.mx (Gabriel Almaguer) Date: Wed, 21 Jan 2015 21:32:06 -0600 Subject: Fatal: master: service(imap): child 19549 killed with signal 11 (core dumped) Message-ID: <54C06F36.3060300@almaguer.com.mx> hello. Today I had a problem (core dumped) in a dovecot installation update yesterday. After many hours I found that when one settings "protocol imap" the option "replication" causes (core dumped). In this server settings "replication" was removed leaving only the "protocol imap" (my error) I send my documentation error so they can add a validation and not generate a Fatal Error end users. Any other info you need please email me Thank You. dovecot-2.2.15 ---------------------/src/plugins/replication/replication-plugin.c replication_mail_transaction_commit(void *txn, struct mail_transaction_commit_changes *changes) { struct replication_mail_txn_context *ctx = (struct replication_mail_txn_context *)txn; struct replication_user *ruser = REPLICATION_USER_CONTEXT(ctx->ns->user); enum replication_priority priority; if (ctx->new_messages || changes->changed) { priority = !ctx->new_messages ? REPLICATION_PRIORITY_LOW : <== linea 258 look dgb ruser->sync_secs == 0 ? REPLICATION_PRIORITY_HIGH : REPLICATION_PRIORITY_SYNC; replication_notify(ctx->ns, priority); } i_free(ctx); } --------------------------------------------------------------------------------------- dovecot -n protocol imap { mail_plugins = quota notify fts fts_squat zlib acl imap_zlib imap_acl quota imap_quota replication } ------------------------------------------------------------- /var/log/maillog Jan 21 20:38:17 xxxxxx dovecot: imap(etrevino at xxxxxxx.com.mx): Fatal: master: service(imap): child 19549 killed with signal 11 (core dumped) --------------------------------------------- /var/log/messages Jan 21 20:38:17 xxxxxx kernel: imap[19549]: segfault at 0000000000000018 rip 00002aacb45ea645 rsp 00007fffc2db7350 error 4 -------------------------------------------------------------------- strace [pid 18265] fstat(4, {st_mode=S_IFREG|0600, st_size=32712, ...}) = 0 [pid 18265] write(4, "\200\200\200\205\4\0\0\0R\2\0\0R\2\0\0\10\0\0\0", 20) = 20 [pid 18265] fcntl(4, F_SETLK, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 [pid 18265] stat("/var/spool/vdovecot/xxxxxx.com.mx/cduran/dovecot.index.log", {st_mode=S_IFREG|0600, st_size=32732, ...}) = 0 [pid 18265] fstat(4, {st_mode=S_IFREG|0600, st_size=32732, ...}) = 0 [pid 18265] munmap(0x2ab13f335000, 32712) = 0 [pid 18265] pread(4, "\200\200\200\205\4\0\0\0R\2\0\0R\2\0\0\10\0\0\0", 8192, 32712) = 20 [pid 18265] pread(4, "", 8192, 32732) = 0 [pid 18265] fstat(4, {st_mode=S_IFREG|0600, st_size=32732, ...}) = 0 [pid 18265] mmap(NULL, 32732, PROT_READ, MAP_SHARED, 4, 0) = 0x2ab13f335000 [pid 18265] madvise(0x2ab13f335000, 32732, MADV_SEQUENTIAL) = 0 [pid 18265] fstat(4, {st_mode=S_IFREG|0600, st_size=32732, ...}) = 0 [pid 18265] --- SIGSEGV (Segmentation fault) @ 0 (0) --- Process 18265 detached --------------------------------------------------------------------------------- gdb Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 replication_mail_transaction_commit (txn=0x18610d70, changes=0x7fff80ac1440) at replication-plugin.c:258 258 replication-plugin.c: No such file or directory. in replication-plugin.c (gdb) bt full #0 replication_mail_transaction_commit (txn=0x18610d70, changes=0x7fff80ac1440) at replication-plugin.c:258 ctx = priority = #1 0x00002ab29a975749 in notify_contexts_mail_transaction_commit (t=0x18610000, changes=0x7fff80ac1440) at notify-plugin.c:132 ctx = 0x1859af70 mail_txn = 0x18610800 #2 0x00002ab29a975d60 in notify_transaction_commit (t=0x18610000, changes_r=0x7fff80ac1440) at notify-storage.c:182 lt = 0x18610d50 #3 0x00002ab29a569beb in quota_mailbox_transaction_commit (ctx=0x18610000, changes_r=0x7fff80ac1440) at quota-storage.c:134 qt = 0x18610d90 #4 0x00002ab299bce134 in mailbox_transaction_commit_get_changes (_t=, changes_r=0x7fff80ac1440) at mail-storage.c:1916 _data_stack_cur_id = 3 t = 0x18610000 box = 0x18609610 save_count = 1 ret = __FUNCTION__ = "mailbox_transaction_commit_get_changes" #5 0x000000000040b85c in cmd_append_parse_new_msg () No symbol table info available. #6 0x000000000040b65b in cmd_append_continue_message () No symbol table info available. #7 0x0000000000416aa7 in command_exec () No symbol table info available. #8 0x000000000040c191 in client_input_append () No symbol table info available. #9 0x00002ab299ee3689 in io_loop_call_io (io=0x1859c010) at ioloop.c:498 ioloop = 0x18585740 t_id = 2 __FUNCTION__ = "io_loop_call_io" #10 0x00002ab299ee4a45 in io_loop_handler_run_internal (ioloop=) at ioloop-epoll.c:220 ctx = event = 0x18587240 list = 0x18587e30 io = 0x20 tv = {tv_sec = 1799, tv_usec = 999036} msecs = ret = 1 i = 0 call = false __FUNCTION__ = "io_loop_handler_run_internal" #11 0x00002ab299ee3729 in io_loop_handler_run (ioloop=0x20) at ioloop.c:545 No locals. #12 0x00002ab299ee39a8 in io_loop_run (ioloop=0x18585740) at ioloop.c:522 __FUNCTION__ = "io_loop_run" #13 0x00002ab299e8a233 in master_service_run (service=0x185855d0, callback=0x28) at master-service.c:566 No locals. #14 0x000000000041f3ce in main () From ltirkkon at niksula.hut.fi Thu Jan 22 12:01:24 2015 From: ltirkkon at niksula.hut.fi (Lauri Tirkkonen) Date: Thu, 22 Jan 2015 14:01:24 +0200 Subject: [PATCH] increase fd_limit to max_client_limit automatically Message-ID: <20150122120123.GK16405@kekkonen.niksula.hut.fi> Hi, with a low soft limit on file descriptors, dovecot 2.2.15 warns on startup: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit It could try increasing the limit first, and only report the warning if that fails. I'm attaching a patch that does just this. Without the patch, the soft fd limit is kept at whatever it was on dovecot startup: % pfexec plimit $(pgrep dovecot) 18737: /opt/niksula/sbin/dovecot resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 10240 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 256 65536 vmemory(kbytes) unlimited unlimited and with patch applied, it's increased to max_client_limit: % pfexec plimit $(pgrep dovecot) 18775: /opt/niksula/sbin/dovecot resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 10240 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 1000 65536 vmemory(kbytes) unlimited unlimited It should probably be increased to higher than that to account for fds other than client sockets, but I don't have insights into that. -- Lauri Tirkkonen Niksula systems specialist -------------- next part -------------- diff -r e30e9b2b0e37 src/master/master-settings.c --- a/src/master/master-settings.c Wed Jan 21 02:21:35 2015 +0200 +++ b/src/master/master-settings.c Thu Jan 22 13:39:41 2015 +0200 @@ -18,6 +18,7 @@ #include #include #include +#include static bool master_settings_verify(void *_set, pool_t pool, const char **error_r); @@ -418,7 +419,7 @@ #ifdef CONFIG_BINARY const struct service_settings *default_service; #else - rlim_t fd_limit; + struct rlimit fd_limit; const char *max_client_limit_source = "default_client_limit"; unsigned int max_client_limit = set->default_client_limit; #endif @@ -591,12 +592,18 @@ client_limit, max_anvil_client_processes); } #ifndef CONFIG_BINARY - if (restrict_get_fd_limit(&fd_limit) == 0 && - fd_limit < (rlim_t)max_client_limit) { - i_warning("fd limit (ulimit -n) is lower than required " - "under max. load (%u < %u), because of %s", - (unsigned int)fd_limit, max_client_limit, - max_client_limit_source); + if (getrlimit(RLIMIT_NOFILE, &fd_limit) == 0) { +#ifdef HAVE_SETRLIMIT + rlim_t old = fd_limit.rlim_cur; + fd_limit.rlim_cur = I_MAX(old, max_client_limit); + if (setrlimit(RLIMIT_NOFILE, &fd_limit) < 0) + fd_limit.rlim_cur = old; +#endif + if (fd_limit.rlim_cur < (rlim_t)max_client_limit) + i_warning("fd limit (ulimit -n) is lower than required " + "under max. load (%u < %u), because of %s", + (unsigned int)fd_limit.rlim_cur, max_client_limit, + max_client_limit_source); } #endif From sanvila at unex.es Fri Jan 23 22:13:44 2015 From: sanvila at unex.es (Santiago Vila) Date: Fri, 23 Jan 2015 23:13:44 +0100 (CET) Subject: Mailbox corruption (dovecot 2.2.13, Debian 8) Message-ID: Hello. The attached mbox folder becomes corrupted in the server when being retrieved by fetchmail using the --folder option. This happens on a system running Debian jessie (i.e. dovecot 2.2.13) using the default configuration. I've reported this to Debian here: https://bugs.debian.org/776094 but I thought it would be good to post it here as well in case it is still a bug in the latest version (well, if it's not anymore a bug in the latest version, a patch against 2.2.13 will surely help the Debian maintainer in either case). Thanks. -------------- next part -------------- A non-text attachment was scrubbed... Name: inbox-b.gz Type: application/gzip Size: 840 bytes Desc: URL: From jtam.home at gmail.com Fri Jan 23 22:52:58 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 23 Jan 2015 14:52:58 -0800 (PST) Subject: LDA input validation In-Reply-To: References: Message-ID: St?phane Cottin writes: > A typical .qmail file : > > | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" > > When dspam segfault or is killed, preline receive an empty content and > only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). > Then dovecot-lda delivers successfully and the original message > contents are lost. Steffen Kaiser rightly points out: > That's the problem of the used Unix pipe and broken by design. The right > member of the pipe (preline) runs independendly of dspam and recieves no > content via pipe. Why should preline or dovecot-lda believe there is an > error? [... and suggests a wrapper script that aborts delivery on empty > dspam output.] I'm not sure if it's appropriate under your circumstances, but you can also pass stderr to stdout as well, and the recipient will get some diagnostic message, which may or may not be helpful. | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" 2>&1 | ... Joseph Tam From stephane.cottin at vixns.com Sat Jan 24 11:07:35 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Sat, 24 Jan 2015 12:07:35 +0100 Subject: LDA input validation In-Reply-To: References: Message-ID: <76329708-69DF-4156-9AE4-C8421E83B273@vixns.com> Hello, > Le 23 janv. 2015 ? 16:14, Steffen Kaiser a ?crit : > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Fri, 23 Jan 2015, St?phane Cottin wrote: > >> I'm using qmail + dspam + dovecot-lda. >> >> A typical .qmail file : >> >> | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" >> >> When dspam segfault or is killed, preline receive an empty content and only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). > > Those two lines are generated by dovecot-lda or preline. preline > >> Then dovecot-lda delivers successfully and the original message contents are lost. >> >> Should dovecot-lda returns an error in such conditions ( invalid contents ) ? Qmail will keep it in the queue and retry delivery later. > > That's the problem of the used Unix pipe and broken by design. The right member of the pipe (preline) runs independendly of dspam and recieves no content via pipe. Why should preline or dovecot-lda believe there is an error? You're right about my mistake using pipes, but there may be some cases where validating input make sense. Anyway, many thanks for your wrapper, I've improved it to fail safely on dspam errors. ==== #!/bin/sh tmpf="/var/tmp/wrapper.$$.log" trap 'rc=$?; rm -f $tmpf; exit $rc' EXIT /usr/bin/dspam --client --deliver=stdout --user "$1" >$tmpf if [ $? -eq 0 ] ; then test -s $tmpf && /usr/bin/preline -f \ /usr/lib/dovecot/dovecot-lda -d "$1" -a "$1" -m "$2" <$tmpf else rm -f $tmpf exit 75 fi ==== St?phane > > If preline does not have no nifty option to prevent this, replace the pipe by a shell script: > > | /usr/local/bin/dspam-lda-wrapper.sh "$EXT@$USER" "$EXT2" > > ===== /usr/local/bin/dspam-lda-wrapper.sh > #!/bin/bash > > tmpf="/tmp/wrapper.$$.log" > trap 'rc=$?; rm -f $tmpf; exit $rc' EXIT > > if /usr/bin/dspam --client --deliver=stdout --user "$1" >$tmpf; then > test -s $tmpf && /usr/bin/preline -f \ > /usr/lib/dovecot/dovecot-lda -d "$1" -a "$1" -m "$2" <$tmpf > fi > ====== > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVMJlQnz1H7kL/d9rAQK90Af9Fs+uZ6P0VxQL0r8PpR30hYt1MnExozpz > QXbhKMCrXv5fZwAZ6paAbQQ999Emjj0fyBEBKodgYoi3rQm5claJwb1/NzSotgxx > Kayc0WpPP2cPMR1QZBA1FmR6+XPyEQO1h8B6aYxNNVNELCMxzotQ5hBPsqrR8ajO > 86TL7FYaz6EM5nqW7w/VwwqsPDiM3mlkgckQudMzxeupP/jGZiRAkWODznSRg4Wr > yHx257i8jWj3mV2vcew16tyBuwmc3g1renPSwjaUZoLsb2A9vyJjnr0qmBdfrkDz > LE6WICAJcZuxQtyy8naTExEGKyQUhBc0nOLVljZ6ZnVC15b8gHWYRw== > =WdEr > -----END PGP SIGNATURE----- From stephane.cottin at vixns.com Sat Jan 24 11:27:47 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Sat, 24 Jan 2015 12:27:47 +0100 Subject: LDA input validation In-Reply-To: References: Message-ID: <2413B895-79A6-4BAA-AF60-122C693A7BD1@vixns.com> > Le 23 janv. 2015 ? 23:52, Joseph Tam a ?crit : > > St?phane Cottin writes: > >> A typical .qmail file : >> >> | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" | /usr/bin/preline -f /usr/lib/dovecot/dovecot-lda -d "$EXT@$USER" -a "$EXT@$USER" -m "$EXT2" >> >> When dspam segfault or is killed, preline receive an empty content and >> only pass to dovecot-lda two headers ( Return-Path and Delivered-To ). >> Then dovecot-lda delivers successfully and the original message >> contents are lost. > > Steffen Kaiser rightly points out: > >> That's the problem of the used Unix pipe and broken by design. The right >> member of the pipe (preline) runs independendly of dspam and recieves no >> content via pipe. Why should preline or dovecot-lda believe there is an >> error? [... and suggests a wrapper script that aborts delivery on empty >> dspam output.] > > I'm not sure if it's appropriate under your circumstances, but you can also > pass stderr to stdout as well, and the recipient will get some diagnostic > message, which may or may not be helpful. > > | /usr/bin/dspam --client --deliver=stdout --user "$EXT@$USER" 2>&1 | ... > > Joseph Tam dspam already send errors to syslog, the point here is to never loose email contents. This was a wrong design, i'm now use a wrapper instead ( see my previous post for details ). St?phane From ml+dovecot at valo.at Sat Jan 24 14:31:22 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 24 Jan 2015 15:31:22 +0100 Subject: sieve filter not working In-Reply-To: <9E0CE520-E99C-49A3-8649-6621734315A3@valo.at> References: <9E0CE520-E99C-49A3-8649-6621734315A3@valo.at> Message-ID: <4638E862-A278-4C40-80B1-670BCB1B08BC@valo.at> Am 23. J?nner 2015 22:02:06 MEZ, schrieb Michael Williamson Hello, >> Another question: >> Local mail delivery of received mail from external sources does >indeed >> work? > >Yes. > That is good. >> If so, was that log excerpt in one of the former mails an example of >> such delivery? > >There are dovecot messages in "/var/log/maillog" showing only >imap-login & disconnected. >When I monitor "/var/log/maillog" while sending spam, there are no >messages containing 'dovecot'. I attached that output to a previous >post. It has only messages from postfix and amavis. > Ther should be messages from dovecot when receiving mail at least when dovecot lda has something to do with delivery. >home_mailbox = Maildir/ This tells postfix where to deliver the mails. dovecot is not configured to delivet mails. Take a look at dovecot wiki there are examples of how to incorporate dovecot lda into thendelivery process. >> I believe dovecot has actually nothing to do with delivering your >mails >> so no sieve filters are applied... > >Is it postfix? > yes, seems so >>> Thanks, >>> -Mike - christian From michael.h.williamson at gmail.com Sat Jan 24 16:13:59 2015 From: michael.h.williamson at gmail.com (Michael Williamson) Date: Sat, 24 Jan 2015 09:13:59 -0700 Subject: sieve filter not working In-Reply-To: <4638E862-A278-4C40-80B1-670BCB1B08BC@valo.at> References: <9E0CE520-E99C-49A3-8649-6621734315A3@valo.at> <4638E862-A278-4C40-80B1-670BCB1B08BC@valo.at> Message-ID: On 1/24/15, Christian Kivalo wrote: >>home_mailbox = Maildir/ > This tells postfix where to deliver the mails. dovecot is not configured to > delivet mails. > > Take a look at dovecot wiki there are examples of how to incorporate dovecot > lda into thendelivery process. > >>> I believe dovecot has actually nothing to do with delivering your >>mails >>> so no sieve filters are applied... >> >>Is it postfix? >> > yes, seems so > OK, thanks. Now I ask, am I better off enabling dovecot lda with the sieve spam filter, or using postfix for filter? -Mike From dave.mehler at gmail.com Sat Jan 24 16:16:00 2015 From: dave.mehler at gmail.com (David Mehler) Date: Sat, 24 Jan 2015 11:16:00 -0500 Subject: sieve filter not working In-Reply-To: References: <9E0CE520-E99C-49A3-8649-6621734315A3@valo.at> <4638E862-A278-4C40-80B1-670BCB1B08BC@valo.at> Message-ID: Hello, I'd use Dovecot Lda for delivery if you want to use sieve. Hth Dave. On 1/24/15, Michael Williamson wrote: > On 1/24/15, Christian Kivalo wrote: > >>>home_mailbox = Maildir/ >> This tells postfix where to deliver the mails. dovecot is not configured >> to >> delivet mails. >> >> Take a look at dovecot wiki there are examples of how to incorporate >> dovecot >> lda into thendelivery process. >> >>>> I believe dovecot has actually nothing to do with delivering your >>>mails >>>> so no sieve filters are applied... >>> >>>Is it postfix? >>> >> yes, seems so >> > > OK, thanks. > > Now I ask, am I better off enabling dovecot lda with the sieve spam > filter, or using postfix for filter? > > -Mike > From mail at marc-stuermer.de Sat Jan 24 18:17:19 2015 From: mail at marc-stuermer.de (=?UTF-8?B?TWFyYyBTdMO8cm1lcg==?=) Date: Sat, 24 Jan 2015 19:17:19 +0100 Subject: from Cyrus to Dovecot, seen status on cyrus.index files In-Reply-To: References: <20150123082052.Horde.3L_3IUHEteEzyNFl1a5bCQ1@webmail.no-carrier.info> Message-ID: <54C3E1AF.6040706@marc-stuermer.de> Am 23.01.2015 um 14:25 schrieb Norberto Bensa: > I've used dsync but it's too slow. I see your point, if you need to be able to migrate many big inboxes in a short time span. That's why in Dovecot's mail_location-settings directive you can use the type "auto:" -, like e.g. "mail_location = auto:/var/customers/%u". If there's an /var/customers/%u/Maildir, it will use Maildir for storage, if there's an /var/customers/%u/mdbox Mdbox and so on. From superinterstellar at gmail.com Sat Jan 24 18:41:19 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Sun, 25 Jan 2015 01:41:19 +0700 Subject: Indexing Mail faster Message-ID: Hi, I am trying to get faster search results on our webmail client(Roundcube). Besides using Lucene for FTS are there other options? Would having all mails indexed give fast results? Currently the time it takes to search 25,000mails is 4mins. If indexed how much faster are we looking at? Really appreciate if someone could advise about this. Thanks Kevin From tlx at leuxner.net Sat Jan 24 19:04:08 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 24 Jan 2015 20:04:08 +0100 Subject: Indexing Mail faster In-Reply-To: References: Message-ID: <20150124190408.GA436@nihlus.leuxner.net> * Kevin Laurie 2015.01.24 19:41: > Currently the time it takes to search 25,000mails is 4mins. If indexed how > much faster are we looking at? With a current version of Dovecot a search is pretty fast _without_ using external indexes. I have a view defined (virtual plugin) with around 22.000 messages in it, and searching the full view only takes 2.5 seconds: $ /usr/lib/dovecot/imap -u tlx at leuxner.net * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL RIGHTS=texk] Logged in as tlx at leuxner.net a select "Virtual/Postfix-Users" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Old) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Old \*)] Flags permitted. * 22094 EXISTS * 0 RECENT * OK [UIDVALIDITY 1420123174] UIDs valid * OK [UIDNEXT 44036] Predicted next UID * OK [HIGHESTMODSEQ 679] Highest a OK [READ-WRITE] Select completed (0.028 secs). b search charset utf-8 body "SQL" * SEARCH 51 52 54 83 87 102 108 126 172 173 195 223 234 238 249 250 252 254 260 263 268 274 293 294 316 317 324 343 412 414 431 432 434 435 436 438 439 440 441 442 443 450 451 454 457 458 464 478 493 496 497 505 590 617 619 625 628 632 636 637 638 639 644 645 646 650 655 657 659 671 676 677 699 702 757 759 762 767 769 770 790 791 792 793 796 799 839 840 841 843 844 845 846 847 848 849 850 864 868 882 905 907 929 953 955 959 1129 1133 1136 1137 1138 1139 1140 1143 1145 1168 1174 1175 1176 1177 1178 1179 1182 1205 1208 1244 1247 1248 1251 1316 1320 1334 1336 1352 1356 1392 1394 1395 1397 1398 1403 1407 1409 1410 1411 1433 1434 1435 1451 1468 1469 1470 1511 1512 1546 1547 1548 1560 1561 1562 1563 1567 1569 1570 1574 1578 1587 1588 1590 1601 1606 1648 1661 1669 1670 1671 1726 1748 1749 1750 1751 1752 1753 1755 1771 1773 1774 1792 1800 1808 1819 1822 1850 1885 1913 1932 1933 1980 1982 1999 2004 2026 2102 2104 2144 2145 2162 2184 2192 2193 2194 2204 2205 2217 2237 2272 2336 2337 2339 2341 2342 2343 2344 2346 2348 2368 2369 2380 2409 2426 2429 2455 2461 2499 2605 2607 2609 2611 2615 2618 2620 2635 2638 2639 2640 2642 2673 2784 2797 2799 2800 2821 2830 2849 2857 2863 2894 2895 2943 2944 2953 2954 2955 3049 3050 3052 3195 3244 3418 3419 3420 3421 3425 3465 3470 3474 3530 3543 3544 3546 3549 3550 3569 3572 3593 3602 3643 3647 3648 3649 3650 3654 3655 3656 3673 3674 3721 3722 3723 3724 3728 3740 3741 3772 3795 3797 3799 3803 3804 3806 3830 3832 3837 3838 3839 3840 3841 3859 3865 3866 3867 3875 3896 3898 3907 3915 3919 3924 3925 3927 3949 3951 4029 4031 4043 4073 4083 4084 4085 4087 4089 4091 4094 4096 4185 4186 4350 4354 4400 4401 4403 4404 4405 4406 4417 4419 4422 4423 4424 4425 4427 4431 4463 4501 4554 4556 4583 4615 4640 4642 4650 4683 4684 4686 4687 4688 4690 4691 4696 4697 4698 4699 4702 4705 4713 4714 4724 4725 4726 4727 4728 4729 4730 4734 4746 4754 4770 4772 4785 4787 4792 4794 4798 4800 4912 4922 4955 4956 4958 4966 4972 4973 4974 4975 4978 4979 4988 4990 4991 4992 4998 5009 5021 5025 5026 5027 5029 5030 5031 5036 5037 5109 5110 5111 5112 5149 5150 5151 5156 5159 5202 5248 5249 5365 5366 5367 5402 5428 5441 5442 5447 5449 5483 5484 5486 5487 5489 5493 5494 5495 5496 5497 5498 5499 5501 5502 5503 5504 5506 5508 5604 5681 5721 5722 5723 5724 5725 5727 5728 5729 5730 5731 5737 5738 5761 5778 5794 5796 5817 5818 5860 5863 5864 5865 5867 5989 5990 5991 6006 6028 6075 6078 6080 6081 6082 6083 6090 6092 6093 6103 6104 6115 6122 6193 6194 6197 6198 6235 6243 6258 6259 6260 6268 6312 6317 6321 6408 6409 6471 6472 6485 6546 6551 6620 6622 6623 6624 6630 6637 6652 6694 6698 6710 6768 6770 6772 6797 6851 6853 6868 6899 6900 6949 7092 7093 7095 7156 7219 7226 7227 7265 7278 7280 7281 7282 7283 7294 7295 7305 7307 7310 7317 7319 7327 7338 7341 7342 7358 7363 7365 7370 7378 7381 7382 7434 7447 7448 7449 7450 7452 7509 7516 7517 7518 7520 7521 7522 7523 7524 7525 7526 7544 7585 7614 7616 7617 7637 7644 7670 7671 7794 7795 7796 7885 7886 7887 7889 7895 7898 7900 7953 8016 8052 8054 8055 8056 8057 8059 8063 8064 8066 8069 8079 8119 8132 8156 8175 8232 8233 8234 8235 8236 8242 8289 8354 8420 8506 8507 8508 8511 8546 8622 8666 8667 8709 8873 8874 8875 8876 8877 8879 8887 8892 8898 8899 8927 8956 8957 8984 8988 8993 9056 9166 9207 9225 9226 9227 9229 9230 9231 9232 9262 9263 9264 9265 9266 9267 9268 9269 9272 9357 9366 9369 9371 9378 9381 9390 9392 9393 9402 9403 9409 9416 9417 9426 9430 9431 9432 9455 9458 9540 9553 9554 9574 9575 9577 9593 9599 9601 9610 9611 9612 9613 9618 9619 9625 9627 9641 9642 9659 9685 9694 9735 9737 9772 9773 9774 9775 9776 9777 9781 9795 9802 9803 9806 9827 9829 9844 9866 9867 9875 9914 9949 9954 9957 9958 9959 9960 9961 9962 9967 9968 9969 9970 9986 9987 9988 9996 10003 10004 10006 10037 10040 10041 10043 10044 10046 10048 10049 10053 10072 10107 10121 10132 10136 10138 10140 10142 10143 10144 10146 10158 10276 10306 10336 10343 10353 10366 10389 10390 10400 10401 10413 10421 10429 10511 10709 10710 10711 10712 10713 10714 10715 10717 10719 10720 10728 10778 10779 10799 10800 10801 10819 10823 10829 10830 10850 10853 10857 10859 10861 10863 10874 10883 10884 10916 10932 11009 11010 11011 11083 11084 11089 11100 11118 11121 11147 11148 11161 11162 11163 11168 11171 11226 11228 11229 11255 11342 11359 11369 11375 11378 11461 11463 11464 11465 11466 11543 11548 11549 11552 11557 11558 11559 11673 11704 11721 11756 11812 11826 11827 11828 11829 11830 11832 11836 11838 11840 11841 11843 11844 11867 11868 11871 11872 11876 11885 11892 11985 11988 11992 12143 12154 12182 12183 12185 12198 12287 12329 12330 12334 12338 12345 12387 12391 12393 12394 12443 12502 12586 12589 12590 12645 12652 12653 12654 12656 12701 12702 12709 12711 12718 12719 12720 12722 12745 12790 12795 12837 12847 12914 12915 13098 13099 13108 13111 13124 13131 13132 13146 13157 13159 13160 13161 13170 13175 13176 13181 13258 13264 13265 13271 13277 13290 13291 13410 13431 13432 13454 13455 13456 13458 13459 13469 13478 13548 13549 13550 13557 13558 13567 13570 13590 13591 13594 13664 13679 13683 13684 13685 13686 13714 13715 13718 13735 13766 13788 13790 13800 13801 13805 13848 13855 13865 13866 13893 13911 13912 13914 13915 13918 13919 13942 13943 13944 13945 13946 13967 13968 13969 13975 13980 13986 13989 13999 14001 14007 14008 14010 14043 14044 14065 14077 14080 14081 14082 14149 14151 14156 14157 14183 14186 14305 14308 14359 14366 14455 14477 14479 14481 14490 14491 14566 14622 14640 14644 14653 14661 14662 14663 14664 14708 14712 14732 14733 14734 14755 14781 14805 14807 14808 14809 14825 14832 14833 14853 14860 14902 14905 14907 14908 14929 14931 14940 14954 14955 15010 15021 15026 15037 15039 15057 15058 15062 15065 15066 15142 15143 15145 15154 15169 15172 15206 15220 15294 15398 15424 15435 15436 15512 15558 15569 15616 15634 15649 15653 15776 15778 15827 15868 15870 15899 15900 15920 15926 15927 15928 15929 15944 15946 15948 16013 16014 16015 16074 16075 16078 16080 16103 16104 16105 16108 16109 16111 16112 16115 16116 16117 16118 16121 16122 16124 16125 16128 16161 16162 16165 16166 16190 16192 16272 16327 16351 16355 16382 16393 16409 16410 16467 16492 16493 16494 16497 16510 16511 16513 16514 16515 16519 16527 16587 16618 16632 16680 16769 16770 16771 16773 16776 16777 16778 16779 16830 16833 16839 16859 16863 16871 16872 16873 16875 16876 16877 16878 16879 16880 16885 16886 16919 16951 16967 16971 16974 16979 16986 16996 17040 17041 17052 17059 17060 17077 17081 17092 17196 17198 17203 17211 17212 17258 17302 17365 17367 17374 17391 17398 17424 17445 17447 17474 17476 17479 17481 17551 17568 17569 17570 17571 17572 17573 17575 17577 17585 17664 17665 17677 17678 17712 17713 17715 17755 17756 17757 17762 17784 17830 17831 17897 17906 17951 18024 18040 18042 18129 18136 18137 18138 18161 18162 18163 18167 18169 18222 18234 18275 18284 18285 18286 18287 18291 18342 18349 18356 18358 18361 18362 18363 18373 18375 18396 18436 18437 18438 18440 18441 18457 18460 18462 18467 18474 18475 18477 18479 18485 18486 18488 18489 18491 18492 18493 18494 18499 18503 18514 18576 18585 18586 18602 18603 18604 18605 18664 18769 18770 18772 18773 18774 18811 18813 18944 18952 18953 18981 18982 18983 18998 19124 19127 19129 19130 19136 19171 19174 19176 19180 19181 19182 19217 19218 19241 19243 19245 19257 19259 19276 19358 19360 19361 19362 19363 19375 19388 19390 19391 19393 19394 19421 19422 19423 19437 19492 19579 19594 19660 19668 19918 19920 19921 20016 20032 20034 20036 20038 20041 20053 20055 20149 20150 20159 20210 20211 20258 20282 20301 20355 20356 20357 20358 20360 20371 20376 20467 20468 20469 20475 20477 20481 20482 20483 20485 20487 20488 20489 20490 20491 20492 20493 20498 20504 20510 20533 20534 20535 20536 20537 20538 20541 20543 20546 20576 20580 20583 20585 20626 20634 20655 20724 20725 20726 20727 20728 20729 20730 20731 20732 20733 20962 20967 20968 20970 20974 21007 21008 21009 21010 21012 21019 21028 21043 21047 21056 21103 21108 21130 21145 21185 21253 21474 21482 21484 21488 21489 21523 21543 21544 21545 21548 21554 21566 21583 21584 21585 21586 21610 21631 21685 21717 21724 21731 21859 21860 21885 b OK Search completed (2.493 secs). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From ml+dovecot at valo.at Sat Jan 24 20:49:56 2015 From: ml+dovecot at valo.at (Christian Kivalo) Date: Sat, 24 Jan 2015 21:49:56 +0100 Subject: sieve filter not working In-Reply-To: References: <9E0CE520-E99C-49A3-8649-6621734315A3@valo.at> <4638E862-A278-4C40-80B1-670BCB1B08BC@valo.at> Message-ID: <78474df3fda6a7968d910f8174f687e6@valo.at> On 2015-01-24 17:13, Michael Williamson wrote: > > OK, thanks. > > Now I ask, am I better off enabling dovecot lda with the sieve spam > filter, or using postfix for filter? > I'd definitly go for dovecot lda, that way you can use sieve to filter your mails. I am using lmtp from postfix to dovecot following the directions in: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP The howto's linked to from the dovecot wiki contain much valuable information and helped me allot in configuring dovecot/postfix. It depends on your distribution if postfix is chrooted or not and what path to the dovecot lmtp socket needs to be configured. In the wiki the postfix 'mailbox_transport' configuration is for a chrooted postfix instance as for example seen on Debian. > -Mike - christian From mail at oliwel.de Sat Jan 24 23:07:38 2015 From: mail at oliwel.de (Oliver Welter) Date: Sun, 25 Jan 2015 00:07:38 +0100 Subject: Corruption of index files In-Reply-To: <54BF9A79.8040702@oliwel.de> References: <54BF9A79.8040702@oliwel.de> Message-ID: <54C425BA.30508@oliwel.de> Hi List, Am 21.01.2015 um 13:24 schrieb Oliver Welter: > Hi All, > > after upgrading my mail server (dovecot 1.1.7 -> 2.2.13) I get tons of > messages about corrupted index files in the syslog ("Error: Corrupted > transaction log" and "Warning: fscking index file .. dovecot.index". > > I tried flock and even dotlock, but the problems persist. > > The system is a VPS with an overlay filesystem which did work flawlessly > with the old dovecot install and is not reporting any errors in the host > machine. > > From time to time I get warnings about an exceeded inotify limit for the > dovecot user - might this be a problem? Anybody has an idea what might > be the cause those problems and how to track them down. Some more debugging - I did a "fuser" on a broken dovecot.index file and see a lot of stale file handles, looks like something is not cleaning up. Any ideas are still welcome.... Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From sca at andreasschulze.de Sun Jan 25 11:41:08 2015 From: sca at andreasschulze.de (Andreas Schulze) Date: Sun, 25 Jan 2015 12:41:08 +0100 Subject: Corruption of index files In-Reply-To: <54C425BA.30508@oliwel.de> References: <54BF9A79.8040702@oliwel.de> <54C425BA.30508@oliwel.de> Message-ID: <20150125114108.GA26116@dimos.andreasschulze.de> Oliver Welter: > >after upgrading my mail server (dovecot 1.1.7 -> 2.2.13) I get tons of > >messages about corrupted index files in the syslog ("Error: Corrupted > >transaction log" and "Warning: fscking index file .. dovecot.index". > > > Some more debugging - I did a "fuser" on a broken dovecot.index file > and see a lot of stale file handles, looks like something is not > cleaning up. Any ideas are still welcome.... if your using classic maildir format I would try to completly remove any dovecot index files. my guess: the existing index files are /so old/ dovecot is unable to repair them in a propper way. Andreas From mail at oliwel.de Sun Jan 25 11:45:25 2015 From: mail at oliwel.de (Oliver Welter) Date: Sun, 25 Jan 2015 12:45:25 +0100 Subject: Corruption of index files In-Reply-To: <20150125114108.GA26116@dimos.andreasschulze.de> References: <54BF9A79.8040702@oliwel.de> <54C425BA.30508@oliwel.de> <20150125114108.GA26116@dimos.andreasschulze.de> Message-ID: <54C4D755.7090602@oliwel.de> Hi Andreas, Am 25.01.2015 um 12:41 schrieb Andreas Schulze: > Oliver Welter: >>> after upgrading my mail server (dovecot 1.1.7 -> 2.2.13) I get tons of >>> messages about corrupted index files in the syslog ("Error: Corrupted >>> transaction log" and "Warning: fscking index file .. dovecot.index". >>> >> Some more debugging - I did a "fuser" on a broken dovecot.index file >> and see a lot of stale file handles, looks like something is not >> cleaning up. Any ideas are still welcome.... > > if your using classic maildir format I would try to completly remove > any dovecot index files. > > my guess: the existing index files are /so old/ > dovecot is unable to repair them in a propper way. I already removed them (when they broke) but the problem comes up again - it seems to be an issue with concurrent mailbox access and locking but I am unable to track it down further. It would be helpful to know what changed in the handling of locks/indexes from 1.1 to 2.2.... Oliver -- Protect your environment - close windows and adopt a penguin! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4074 bytes Desc: S/MIME Cryptographic Signature URL: From sca at andreasschulze.de Sun Jan 25 11:47:59 2015 From: sca at andreasschulze.de (Andreas Schulze) Date: Sun, 25 Jan 2015 12:47:59 +0100 Subject: Indexing Mail faster In-Reply-To: <20150124190408.GA436@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: <20150125114759.GB26116@dimos.andreasschulze.de> Thomas Leuxner: > I have a view defined (virtual plugin) with around 22.000 messages in it interesting. I assume you drop multiple mailing lists together in a mailbox and separate them using the virtual plugin, right? could publish how you configured dovecot virtual plugin? I never had success on using the plugin in a useful way :-/ Thanks Andreas From tlx at leuxner.net Sun Jan 25 12:05:07 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 25 Jan 2015 13:05:07 +0100 Subject: Indexing Mail faster In-Reply-To: <20150125114759.GB26116@dimos.andreasschulze.de> References: <20150124190408.GA436@nihlus.leuxner.net> <20150125114759.GB26116@dimos.andreasschulze.de> Message-ID: <20150125120506.GA20278@nihlus.leuxner.net> * Andreas Schulze 2015.01.25 12:47: > I assume you drop multiple mailing lists together in a mailbox and separate them using the virtual plugin, right? > could publish how you configured dovecot virtual plugin? The namespace "Virtual" will look for its configuration in the user's home directory: namespace { location = virtual:~/mdbox/virtual prefix = Virtual/ separator = / } Inside the ~/mdbox/virtual directory the actual query is defined in directory 'Flagged' (that will show up in the MUA as sub-dir). The actual search/query is defined in file 'dovecot-virtual' which will create the "magic": $ cat virtual/Flagged/dovecot-virtual * Public/* flagged Another example, the one I used in the original reply, is 'gluing' together archives as a single view: $ cat virtual/Postfix-Users/dovecot-virtual Public/Archive/Mailing-Lists/Postfix-Users/* Public/Mailing-Lists/Postfix-Users all HTH -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From sca at andreasschulze.de Sun Jan 25 15:58:58 2015 From: sca at andreasschulze.de (A. Schulze) Date: Sun, 25 Jan 2015 16:58:58 +0100 Subject: Indexing Mail faster In-Reply-To: <20150125120506.GA20278@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150125114759.GB26116@dimos.andreasschulze.de> <20150125120506.GA20278@nihlus.leuxner.net> Message-ID: <20150125165858.Horde.edmPdPq-um-pvftvpA8QKw2@horde.andreasschulze.de> Thomas Leuxner: > namespace { > location = virtual:~/mdbox/virtual > prefix = Virtual/ > separator = / > } > > $ cat virtual/Flagged/dovecot-virtual > * > Public/* > flagged once setup correctly it works like expected :-) > Another example, the one I used in the original reply, is 'gluing' > together archives as a single view: > > $ cat virtual/Postfix-Users/dovecot-virtual > Public/Archive/Mailing-Lists/Postfix-Users/* > Public/Mailing-Lists/Postfix-Users > all how do you separate older postings into the Archive? something like "doveadm search + move ? Andreas From tlx at leuxner.net Sun Jan 25 16:19:17 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 25 Jan 2015 17:19:17 +0100 Subject: Indexing Mail faster In-Reply-To: <20150125165858.Horde.edmPdPq-um-pvftvpA8QKw2@horde.andreasschulze.de> References: <20150124190408.GA436@nihlus.leuxner.net> <20150125114759.GB26116@dimos.andreasschulze.de> <20150125120506.GA20278@nihlus.leuxner.net> <20150125165858.Horde.edmPdPq-um-pvftvpA8QKw2@horde.andreasschulze.de> Message-ID: <20150125161917.GA24799@nihlus.leuxner.net> * A. Schulze 2015.01.25 16:58: > once setup correctly it works like expected :-) Good! > how do you separate older postings into the Archive? > something like "doveadm search + move ? Yes basically with move. I'd like to have them archived by year e.g. Public/Archive/Mailing-Lists/Dovecot/2014. Being lazy when it comes to repetitive tasks, I wrote a little script for it. It gives a good idea how to do it (hopefully). Please note that this only has minimal error handling as I'm setting most parameters in the script rather than the shell - butterfingers... No bashism comments please :) $ cat dovearchive.sh #!/bin/bash # Archive old posts before certain date to 'Public/Archive/Mailbox/Year' # Set basic parameters accordingly: # archive=2011, before_date=$year-mm-dd # source_mailbox_base=Public/Newsletters # dest_mailbox_base=Public/Archive/Newsletters # Actual Mailbox is read from command line set -e archive=2014 let year=$archive+1 before_date=$year-01-01 mailbox_owner=tlx at leuxner.net source_mailbox_base='Public/Mailing-Lists' dest_mailbox_base='Public/Archive/Mailing-Lists' #acl_admin_group=owner acl_admin_group='group=PublicMailboxAdmins' acl_unlock_seq="$acl_admin_group delete expunge insert lookup post read write write-seen write-deleted" acl_lock_seq="$acl_admin_group insert lookup post read write write-seen" acl_lock_archive="$acl_admin_group insert lookup read write write-seen" debug_acl_mailbox_path='path: ' msg_formatted() { echo "$(date "+%b %d %H:%M:%S") $*" } if [ $# -eq 0 ]; then echo "usage: $0 mailbox" >&2 exit 1 fi # Mailbox exists? doveadm acl get -u $mailbox_owner "$source_mailbox_base/$1" || { echo 'Mailbox not found.'; exit 1; } # Create New Archive Mailbox doveadm mailbox create -u $mailbox_owner "$dest_mailbox_base/$1/$archive" # Modify ACL, expunge mail and revert ACL msg_formatted "[>] Archiving \"$dest_mailbox_base/$1/$archive\"" doveadm acl set -u $mailbox_owner "$source_mailbox_base/$1" $acl_unlock_seq doveadm move -u $mailbox_owner "$dest_mailbox_base/$1/$archive" mailbox "$source_mailbox_base/$1" before $before_date doveadm acl set -u $mailbox_owner "$source_mailbox_base/$1" $acl_lock_seq # Get actual path of dovecot-acl file and remove it as it is handled by global ACL acl_path=$(doveadm acl debug -u $mailbox_owner "$source_mailbox_base/$1" 2>&1 | sed -n "s/\(.*\)$debug_acl_mailbox_path//p") [ -f $acl_path/dovecot-acl ] && rm $acl_path/dovecot-acl msg_formatted '[ Complete ]' -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From kremels at kreme.com Sun Jan 25 18:32:46 2015 From: kremels at kreme.com (LuKreme) Date: Sun, 25 Jan 2015 11:32:46 -0700 Subject: Indexing Mail faster In-Reply-To: <20150125120506.GA20278@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150125114759.GB26116@dimos.andreasschulze.de> <20150125120506.GA20278@nihlus.leuxner.net> Message-ID: <5F7591D8-F089-4787-85B0-D30523CFDDB5@kreme.com> On Jan 25, 2015, at 5:05 AM, Thomas Leuxner wrote: > The namespace "Virtual" will look for its configuration in the user's home directory: > > namespace { > location = virtual:~/mdbox/virtual > prefix = Virtual/ > separator = / > } > > Inside the ~/mdbox/virtual directory the actual query is defined in directory 'Flagged' (that will show up in the MUA as sub-dir). The actual search/query is defined in file 'dovecot-virtual' which will create the "magic": > > $ cat virtual/Flagged/dovecot-virtual > * > Public/* > flagged > > Another example, the one I used in the original reply, is 'gluing' together archives as a single view: > > $ cat virtual/Postfix-Users/dovecot-virtual > Public/Archive/Mailing-Lists/Postfix-Users/* > Public/Mailing-Lists/Postfix-Users > all This all sounds interesting. Is there anything i can read to understand what the purpose of Virtual is and why this would help? It sounds like you are using a shared space for mailing lists (which I can see would be useful if you had many people subscribing to the same lists). If so, doesn?t each user lose their indications of what messages are read or replied to? -- 'But you ain't part of it, are you?' said Granny conversationally. 'You try, but you always find yourself watchin' yourself watchin' people, eh? Never quite believin' anything? Thinkin' the wrong thoughts?' From tlx at leuxner.net Sun Jan 25 18:59:00 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 25 Jan 2015 19:59:00 +0100 Subject: Dovecot Virtual Plugin Use Cases (was Re: Indexing Mail faster) In-Reply-To: <5F7591D8-F089-4787-85B0-D30523CFDDB5@kreme.com> References: <20150124190408.GA436@nihlus.leuxner.net> <20150125114759.GB26116@dimos.andreasschulze.de> <20150125120506.GA20278@nihlus.leuxner.net> <5F7591D8-F089-4787-85B0-D30523CFDDB5@kreme.com> Message-ID: <20150125185859.GA28022@nihlus.leuxner.net> * LuKreme 2015.01.25 19:32: > > Another example, the one I used in the original reply, is 'gluing' together archives as a single view: > > > > $ cat virtual/Postfix-Users/dovecot-virtual > > Public/Archive/Mailing-Lists/Postfix-Users/* > > Public/Mailing-Lists/Postfix-Users > > all > > This all sounds interesting. Is there anything i can read to understand what the purpose of Virtual is and why this would help? It sounds like you are using a shared space for mailing lists (which I can see would be useful if you had many people subscribing to the same lists). If so, doesn?t each user lose their indications of what messages are read or replied to? http://wiki2.dovecot.org/Plugins/Virtual It's a bit hard to wrap your head around it in the beginning, but once you understand what it can do, I'm sure you'll find a use case. Think of it as special view for all messages in Dovecot. You define which set of messages you want to display as a mailbox/folder. This view (or multiple views) is being updated dynamically going forward. For instance you can set up a certain filter/selection like you can with a modern MUA. The difference however is: your Virtual view(s) will be available regardless which client you are using, whether they have integrated filters capabilities - or not. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From dave.mehler at gmail.com Sun Jan 25 21:29:45 2015 From: dave.mehler at gmail.com (David Mehler) Date: Sun, 25 Jan 2015 16:29:45 -0500 Subject: Dovecot antispam Message-ID: Hello, This is not specifically a dovecot issue, but to it's antispam plugin, that being the case if private responses are deemed so please do so. I've got a Postfix, amavisd-new, dovecot (delivery and sieve), setup with spamassassin. It's for the most part working fine, here's the problem, false positives and false negatives. The sieve moves spamassassin-tagged messages in to the Spam folder, sometimes it misses either a false positive or negative. What I'm wanting to do and this is where the dovecot-antispam plugin comes in is automatically retrain messages, a false positive or negative needs to be moved and reclassified accordingly. I've found a lot of google results indicating dspam, when I used this the results were terrible after three months of usage I gave up on it. I've thought about using spamc with dovecot-antispam to handle retraining or crm114, but not much on getting this going. If anyone has any information on this or any other non-dspam setup that would work I'd appreciate knowing about it. Thanks. Dave. From info at netocean.de Mon Jan 26 00:50:11 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 01:50:11 +0100 Subject: imap-login: Fatal: pipe() failed: Too many open files Message-ID: <54C58F43.7020803@netocean.de> Hi I keep on getting errors and can't connect/login to Dovecot. I did my research but unfortunately without success. It is for sure not ulimit because ulimit is set to unlimited per default already. Still , it complains about "Too many open files" but this is a test system and the service dovecot and postfix have just been started. No one except me is testing on this system. This is very weired ;/ root at WM-01 [~]$ service dovecot stop; service postfix stop ; echo "" > /var/log/maillog ; echo "" > /var/log/slapd.log ; service postfix start ; service dovecot start Stopping dovecot. Waiting for PIDS: 3245. postfix/postfix-script: stopping the Postfix mail system postfix/postfix-script: starting the Postfix mail system Starting dovecot. root at WM-01 [~]$ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. Connection closed by foreign host. root at WM-01 [~]$ cat /var/log/mail Jan 26 01:39:12 WM-01 dovecot: imap-login: Fatal: pipe() failed: Too many open files Jan 26 01:39:12 WM-01 dovecot: master: Error: service(imap-login): command startup failed, throttling for 4 secs If I uncoment port = 0 in POP and IMAP then it looks even worse then above. Telnet is not even closed by remote host and three of the "Too many open files" errors are showing up in maillog: root at WM-01 [~]$ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. ^] telnet> Connection closed. root at WM-01 [~]$ cat /var/log/maillog Jan 26 01:45:19 WM-01 postfix/postfix-script[60560]: starting the Postfix mail system Jan 26 01:45:19 WM-01 postfix/master[60562]: daemon started -- version 1.0, configuration /usr/local/etc/postfix Jan 26 01:45:19 WM-01 dovecot: master: Dovecot v2.2.15 starting up for imap, pop3, lmtp Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: socket(login) failed: Too many open files Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: auth: connect(login) failed: Too many open files Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: net_accept() failed: Too many open files root at WM-01 [~]$ Does anyone have an idea what could be wrong here? I attached the relevant configuration below. I use OpenLDAP as backend. Please let me know if you need more information. Thank you very much. Best Regards, Leander # ========================== dovecot -n ============================== # # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE amd64 ufs auth_debug = yes auth_debug_passwords = yes auth_failure_delay = 3 secs auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Welcome to Mail Server. login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_debug = yes mail_gid = vmail mail_location = maildir:/var/mail/%d/%n mail_plugins = quota expire mail_log notify mail_privileged_group = mail mail_temp_dir = /tmp/Dovecot mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { expire = Trash mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=1G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh postmaster_address = info at netocean.de sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imaps { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } user = vmail } service pop3-login { inet_listener pop3s { port = 0 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = userdb { driver = prefetch } userdb { driver = ldap args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext # Default fields can be used to specify defaults that LDAP may override #default_fields = home=/home/virtual/%u } # ======================== dovecot/dovecot-ldap.conf.ext =================== # hosts = 127.0.0.1:389 dn = uid=dovecot,ou=systemuser,ou=mail,dc=MyDomain,dc=TLD dnpass = TopSecret tls = yes tls_cert_file = /etc/ssl/RootCA/certs/192.168.50.101.pem tls_key_file = /etc/ssl/RootCA/certs/192.168.50.101.key debug_level = -1 auth_bind = yes ldap_version = 3 base = ou=accounts,ou=mail,dc=MyDomain,dc=TLD scope = subtree user_attrs = mailStorageDirectory=home, mailUidNumber=uid, mailGidNumber=gid, mailQuotaSize=quota_rule=*:bytes=%$, mailQuotaCount=quota_rule2 user_filter = (&(objectClass=mailAccount)(mailAddress=%u)) pass_attrs = mailAddress=user,mailPassword=password pass_filter = (&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=%u)) default_pass_scheme = SSHA From info at netocean.de Mon Jan 26 01:13:20 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 02:13:20 +0100 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C58F43.7020803@netocean.de> References: <54C58F43.7020803@netocean.de> Message-ID: <54C594B0.4030204@netocean.de> I just checked my ulimit again and it really seems like it hhas more than enough - so I still don't understand what I've configured wrong here ;/ root at WM-01 [~]$ su -m dovecot -c "ulimit -a" socket buffer size (bytes, -b) unlimited core file size (blocks, -c) unlimited data seg size (kbytes, -d) 33554432 file size (blocks, -f) unlimited max locked memory (kbytes, -l) 131072 max memory size (kbytes, -m) 7067352 open files (-n) 205587 pipe size (512 bytes, -p) 1 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 11278 virtual memory (kbytes, -v) unlimited swap size (kbytes, -w) unlimited Best Regards, Leander Am 26.01.15 um 01:50 schrieb Leander Sch?fer: > Hi > > I keep on getting errors and can't connect/login to Dovecot. I did my > research but unfortunately without success. It is for sure not ulimit > because ulimit is set to unlimited per default already. Still , it > complains about "Too many open files" but this is a test system and > the service dovecot and postfix have just been started. No one except > me is testing on this system. This is very weired ;/ > > > root at WM-01 [~]$ service dovecot stop; service postfix stop ; echo "" > > /var/log/maillog ; echo "" > /var/log/slapd.log ; service postfix > start ; service dovecot start > Stopping dovecot. > Waiting for PIDS: 3245. > postfix/postfix-script: stopping the Postfix mail system > postfix/postfix-script: starting the Postfix mail system > Starting dovecot. > > root at WM-01 [~]$ telnet 127.0.0.1 143 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > Connection closed by foreign host. > > root at WM-01 [~]$ cat /var/log/mail > Jan 26 01:39:12 WM-01 dovecot: imap-login: Fatal: pipe() failed: Too > many open files > Jan 26 01:39:12 WM-01 dovecot: master: Error: service(imap-login): > command startup failed, throttling for 4 secs > > > > If I uncoment port = 0 in POP and IMAP then it looks even worse then > above. Telnet is not even closed by remote host and three of the "Too > many open files" errors are showing up in maillog: > > > > root at WM-01 [~]$ telnet 127.0.0.1 143 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > ^] > telnet> Connection closed. > root at WM-01 [~]$ cat /var/log/maillog > > Jan 26 01:45:19 WM-01 postfix/postfix-script[60560]: starting the > Postfix mail system > Jan 26 01:45:19 WM-01 postfix/master[60562]: daemon started -- version > 1.0, configuration /usr/local/etc/postfix > Jan 26 01:45:19 WM-01 dovecot: master: Dovecot v2.2.15 starting up for > imap, pop3, lmtp > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: socket(login) > failed: Too many open files > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: auth: connect(login) > failed: Too many open files > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: net_accept() failed: > Too many open files > root at WM-01 [~]$ > > > > > > > Does anyone have an idea what could be wrong here? I attached the > relevant configuration below. I use OpenLDAP as backend. > Please let me know if you need more information. Thank you very much. > > Best Regards, > Leander > > > > > > > > > > # ========================== dovecot -n ============================== # > > > # 2.2.15: /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.6 (3e924b1b6c5c+) > # OS: FreeBSD 10.1-RELEASE amd64 ufs > auth_debug = yes > auth_debug_passwords = yes > auth_failure_delay = 3 secs > auth_mechanisms = plain login > auth_verbose = yes > auth_verbose_passwords = yes > base_dir = /var/run/dovecot/ > disable_plaintext_auth = no > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep > tb-lsub-flags > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > login_greeting = Welcome to Mail Server. > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e > %c %k > mail_debug = yes > mail_gid = vmail > mail_location = maildir:/var/mail/%d/%n > mail_plugins = quota expire mail_log notify > mail_privileged_group = mail > mail_temp_dir = /tmp/Dovecot > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave duplicate > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > expire = Trash > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > quota = maildir:User quota > quota_exceeded_message = Storage quota for this account has been > exceeded, please try again later. > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+30%% > quota_rule3 = Sent:storage=+30%% > quota_warning = storage=90%% quota-warning 90 %u > quota_warning2 = storage=75%% quota-warning 75 %u > } > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > postmaster_address = info at netocean.de > sendmail_path = /usr/local/sbin/sendmail > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service imap-login { > inet_listener imaps { > port = 0 > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > user = vmail > } > service pop3-login { > inet_listener pop3s { > port = 0 > } > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = vmail > } > ssl = required > ssl_cert = ssl_cipher_list = > EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 > userdb { > driver = prefetch > } > userdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > verbose_ssl = yes > protocol lda { > mail_plugins = sieve > } > protocol imap { > mail_max_userip_connections = 3 > mail_plugins = imap_quota imap_zlib > } > protocol pop3 { > mail_max_userip_connections = 3 > } > > > > > > > > > # ======================== dovecot/conf.d/auth-ldap.conf.ext > =================== # > > passdb { > driver = ldap > > # Path for LDAP configuration file, see > example-config/dovecot-ldap.conf.ext > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > } > > # "prefetch" user database means that the passdb already provided the > # needed information and there's no need to do a separate userdb lookup. > # > userdb { > driver = prefetch > } > > userdb { > driver = ldap > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > > # Default fields can be used to specify defaults that LDAP may override > #default_fields = home=/home/virtual/%u > } > > > > > > # ======================== dovecot/dovecot-ldap.conf.ext > =================== # > > hosts = 127.0.0.1:389 > dn = uid=dovecot,ou=systemuser,ou=mail,dc=MyDomain,dc=TLD > dnpass = TopSecret > tls = yes > tls_cert_file = /etc/ssl/RootCA/certs/192.168.50.101.pem > tls_key_file = /etc/ssl/RootCA/certs/192.168.50.101.key > debug_level = -1 > auth_bind = yes > ldap_version = 3 > base = ou=accounts,ou=mail,dc=MyDomain,dc=TLD > scope = subtree > user_attrs = mailStorageDirectory=home, mailUidNumber=uid, > mailGidNumber=gid, mailQuotaSize=quota_rule=*:bytes=%$, > mailQuotaCount=quota_rule2 > user_filter = (&(objectClass=mailAccount)(mailAddress=%u)) > pass_attrs = mailAddress=user,mailPassword=password > pass_filter = > (&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=%u)) > default_pass_scheme = SSHA From edgar at pettijohn-web.com Mon Jan 26 01:18:57 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Sun, 25 Jan 2015 19:18:57 -0600 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C58F43.7020803@netocean.de> References: <54C58F43.7020803@netocean.de> Message-ID: <54C59601.9080205@pettijohn-web.com> What is the OS? On 01/25/15 18:50, Leander Sch?fer wrote: > Hi > > I keep on getting errors and can't connect/login to Dovecot. I did my > research but unfortunately without success. It is for sure not ulimit > because ulimit is set to unlimited per default already. Still , it > complains about "Too many open files" but this is a test system and > the service dovecot and postfix have just been started. No one except > me is testing on this system. This is very weired ;/ > > > root at WM-01 [~]$ service dovecot stop; service postfix stop ; echo "" > > /var/log/maillog ; echo "" > /var/log/slapd.log ; service postfix > start ; service dovecot start > Stopping dovecot. > Waiting for PIDS: 3245. > postfix/postfix-script: stopping the Postfix mail system > postfix/postfix-script: starting the Postfix mail system > Starting dovecot. > > root at WM-01 [~]$ telnet 127.0.0.1 143 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > Connection closed by foreign host. > > root at WM-01 [~]$ cat /var/log/mail > Jan 26 01:39:12 WM-01 dovecot: imap-login: Fatal: pipe() failed: Too > many open files > Jan 26 01:39:12 WM-01 dovecot: master: Error: service(imap-login): > command startup failed, throttling for 4 secs > > > > If I uncoment port = 0 in POP and IMAP then it looks even worse then > above. Telnet is not even closed by remote host and three of the "Too > many open files" errors are showing up in maillog: > > > > root at WM-01 [~]$ telnet 127.0.0.1 143 > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > ^] > telnet> Connection closed. > root at WM-01 [~]$ cat /var/log/maillog > > Jan 26 01:45:19 WM-01 postfix/postfix-script[60560]: starting the > Postfix mail system > Jan 26 01:45:19 WM-01 postfix/master[60562]: daemon started -- version > 1.0, configuration /usr/local/etc/postfix > Jan 26 01:45:19 WM-01 dovecot: master: Dovecot v2.2.15 starting up for > imap, pop3, lmtp > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: socket(login) > failed: Too many open files > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: auth: connect(login) > failed: Too many open files > Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: net_accept() failed: > Too many open files > root at WM-01 [~]$ > > > > > > > Does anyone have an idea what could be wrong here? I attached the > relevant configuration below. I use OpenLDAP as backend. > Please let me know if you need more information. Thank you very much. > > Best Regards, > Leander > > > > > > > > > > # ========================== dovecot -n ============================== # > > > # 2.2.15: /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.6 (3e924b1b6c5c+) > # OS: FreeBSD 10.1-RELEASE amd64 ufs > auth_debug = yes > auth_debug_passwords = yes > auth_failure_delay = 3 secs > auth_mechanisms = plain login > auth_verbose = yes > auth_verbose_passwords = yes > base_dir = /var/run/dovecot/ > disable_plaintext_auth = no > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep > tb-lsub-flags > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > login_greeting = Welcome to Mail Server. > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e > %c %k > mail_debug = yes > mail_gid = vmail > mail_location = maildir:/var/mail/%d/%n > mail_plugins = quota expire mail_log notify > mail_privileged_group = mail > mail_temp_dir = /tmp/Dovecot > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave duplicate > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > } > passdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > expire = Trash > mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > mail_log_fields = uid box msgid size > quota = maildir:User quota > quota_exceeded_message = Storage quota for this account has been > exceeded, please try again later. > quota_rule = *:storage=1G > quota_rule2 = Trash:storage=+30%% > quota_rule3 = Sent:storage=+30%% > quota_warning = storage=90%% quota-warning 90 %u > quota_warning2 = storage=75%% quota-warning 75 %u > } > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > postmaster_address = info at netocean.de > sendmail_path = /usr/local/sbin/sendmail > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service imap-login { > inet_listener imaps { > port = 0 > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0660 > user = postfix > } > user = vmail > } > service pop3-login { > inet_listener pop3s { > port = 0 > } > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = vmail > } > ssl = required > ssl_cert = ssl_cipher_list = > EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA > ssl_key = ssl_prefer_server_ciphers = yes > ssl_protocols = !SSLv2 !SSLv3 > userdb { > driver = prefetch > } > userdb { > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > verbose_ssl = yes > protocol lda { > mail_plugins = sieve > } > protocol imap { > mail_max_userip_connections = 3 > mail_plugins = imap_quota imap_zlib > } > protocol pop3 { > mail_max_userip_connections = 3 > } > > > > > > > > > # ======================== dovecot/conf.d/auth-ldap.conf.ext > =================== # > > passdb { > driver = ldap > > # Path for LDAP configuration file, see > example-config/dovecot-ldap.conf.ext > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > } > > # "prefetch" user database means that the passdb already provided the > # needed information and there's no need to do a separate userdb lookup. > # > userdb { > driver = prefetch > } > > userdb { > driver = ldap > args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > > # Default fields can be used to specify defaults that LDAP may override > #default_fields = home=/home/virtual/%u > } > > > > > > # ======================== dovecot/dovecot-ldap.conf.ext > =================== # > > hosts = 127.0.0.1:389 > dn = uid=dovecot,ou=systemuser,ou=mail,dc=MyDomain,dc=TLD > dnpass = TopSecret > tls = yes > tls_cert_file = /etc/ssl/RootCA/certs/192.168.50.101.pem > tls_key_file = /etc/ssl/RootCA/certs/192.168.50.101.key > debug_level = -1 > auth_bind = yes > ldap_version = 3 > base = ou=accounts,ou=mail,dc=MyDomain,dc=TLD > scope = subtree > user_attrs = mailStorageDirectory=home, mailUidNumber=uid, > mailGidNumber=gid, mailQuotaSize=quota_rule=*:bytes=%$, > mailQuotaCount=quota_rule2 > user_filter = (&(objectClass=mailAccount)(mailAddress=%u)) > pass_attrs = mailAddress=user,mailPassword=password > pass_filter = > (&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=%u)) > default_pass_scheme = SSHA From h.reindl at thelounge.net Mon Jan 26 01:20:47 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 26 Jan 2015 02:20:47 +0100 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C594B0.4030204@netocean.de> References: <54C58F43.7020803@netocean.de> <54C594B0.4030204@netocean.de> Message-ID: <54C5966F.9050601@thelounge.net> Am 26.01.2015 um 02:13 schrieb Leander Sch?fer: > I just checked my ulimit again and it really seems like it has more > than enough - so I still don't understand what I've configured wrong > here ;/ > > root at WM-01 [~]$ su -m dovecot -c "ulimit -a" > socket buffer size (bytes, -b) unlimited > core file size (blocks, -c) unlimited > data seg size (kbytes, -d) 33554432 > file size (blocks, -f) unlimited > max locked memory (kbytes, -l) 131072 > max memory size (kbytes, -m) 7067352 > open files (-n) 205587 > pipe size (512 bytes, -p) 1 > stack size (kbytes, -s) 8192 > cpu time (seconds, -t) unlimited > max user processes (-u) 11278 > virtual memory (kbytes, -v) unlimited > swap size (kbytes, -w) unlimited besides that "imap-login" typically does *not* run under the same user (here dovenull versus dovecot) who tells you that 205587 is "more than enough" just because it is a high value? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From edgar at pettijohn-web.com Mon Jan 26 01:24:32 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Sun, 25 Jan 2015 19:24:32 -0600 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C59601.9080205@pettijohn-web.com> References: <54C58F43.7020803@netocean.de> <54C59601.9080205@pettijohn-web.com> Message-ID: <54C59750.7030809@pettijohn-web.com> Sorry didn't scroll to the bottom to see the dovecot -n. I'm assuming freebsd has an /etc/login.conf similiar to openbsd. If so you may need to do something similiar to this: dovecot:\ :openfiles-cur=512:\ :openfiles-max=2048:\ :tc=daemon: Rebuild the login.conf.db file if necessary: # [ -f /etc/login.conf.db ] && cap_mkdb /etc/login.conf Hope this helps. On 01/25/15 19:18, Edgar Pettijohn wrote: > What is the OS? > > On 01/25/15 18:50, Leander Sch?fer wrote: >> Hi >> >> I keep on getting errors and can't connect/login to Dovecot. I did my >> research but unfortunately without success. It is for sure not ulimit >> because ulimit is set to unlimited per default already. Still , it >> complains about "Too many open files" but this is a test system and >> the service dovecot and postfix have just been started. No one except >> me is testing on this system. This is very weired ;/ >> >> >> root at WM-01 [~]$ service dovecot stop; service postfix stop ; echo "" >> > /var/log/maillog ; echo "" > /var/log/slapd.log ; service postfix >> start ; service dovecot start >> Stopping dovecot. >> Waiting for PIDS: 3245. >> postfix/postfix-script: stopping the Postfix mail system >> postfix/postfix-script: starting the Postfix mail system >> Starting dovecot. >> >> root at WM-01 [~]$ telnet 127.0.0.1 143 >> Trying 127.0.0.1... >> Connected to localhost. >> Escape character is '^]'. >> Connection closed by foreign host. >> >> root at WM-01 [~]$ cat /var/log/mail >> Jan 26 01:39:12 WM-01 dovecot: imap-login: Fatal: pipe() failed: Too >> many open files >> Jan 26 01:39:12 WM-01 dovecot: master: Error: service(imap-login): >> command startup failed, throttling for 4 secs >> >> >> >> If I uncoment port = 0 in POP and IMAP then it looks even worse then >> above. Telnet is not even closed by remote host and three of the "Too >> many open files" errors are showing up in maillog: >> >> >> >> root at WM-01 [~]$ telnet 127.0.0.1 143 >> Trying 127.0.0.1... >> Connected to localhost. >> Escape character is '^]'. >> ^] >> telnet> Connection closed. >> root at WM-01 [~]$ cat /var/log/maillog >> >> Jan 26 01:45:19 WM-01 postfix/postfix-script[60560]: starting the >> Postfix mail system >> Jan 26 01:45:19 WM-01 postfix/master[60562]: daemon started -- >> version 1.0, configuration /usr/local/etc/postfix >> Jan 26 01:45:19 WM-01 dovecot: master: Dovecot v2.2.15 starting up >> for imap, pop3, lmtp >> Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: socket(login) >> failed: Too many open files >> Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: auth: >> connect(login) failed: Too many open files >> Jan 26 01:45:24 WM-01 dovecot: imap-login: Error: net_accept() >> failed: Too many open files >> root at WM-01 [~]$ >> >> >> >> >> >> >> Does anyone have an idea what could be wrong here? I attached the >> relevant configuration below. I use OpenLDAP as backend. >> Please let me know if you need more information. Thank you very much. >> >> Best Regards, >> Leander >> >> >> >> >> >> >> >> >> >> # ========================== dovecot -n ============================== # >> >> >> # 2.2.15: /usr/local/etc/dovecot/dovecot.conf >> # Pigeonhole version 0.4.6 (3e924b1b6c5c+) >> # OS: FreeBSD 10.1-RELEASE amd64 ufs >> auth_debug = yes >> auth_debug_passwords = yes >> auth_failure_delay = 3 secs >> auth_mechanisms = plain login >> auth_verbose = yes >> auth_verbose_passwords = yes >> base_dir = /var/run/dovecot/ >> disable_plaintext_auth = no >> imap_client_workarounds = delay-newmail tb-extra-mailbox-sep >> tb-lsub-flags >> lda_mailbox_autocreate = yes >> lda_mailbox_autosubscribe = yes >> login_greeting = Welcome to Mail Server. >> login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e >> %c %k >> mail_debug = yes >> mail_gid = vmail >> mail_location = maildir:/var/mail/%d/%n >> mail_plugins = quota expire mail_log notify >> mail_privileged_group = mail >> mail_temp_dir = /tmp/Dovecot >> mail_uid = vmail >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave duplicate >> namespace inbox { >> inbox = yes >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> } >> passdb { >> args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext >> driver = ldap >> } >> plugin { >> expire = Trash >> mail_log_events = delete undelete expunge copy mailbox_delete >> mailbox_rename >> mail_log_fields = uid box msgid size >> quota = maildir:User quota >> quota_exceeded_message = Storage quota for this account has been >> exceeded, please try again later. >> quota_rule = *:storage=1G >> quota_rule2 = Trash:storage=+30%% >> quota_rule3 = Sent:storage=+30%% >> quota_warning = storage=90%% quota-warning 90 %u >> quota_warning2 = storage=75%% quota-warning 75 %u >> } >> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >> postmaster_address = info at netocean.de >> sendmail_path = /usr/local/sbin/sendmail >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> unix_listener auth-userdb { >> group = vmail >> mode = 0600 >> user = vmail >> } >> } >> service imap-login { >> inet_listener imaps { >> port = 0 >> } >> } >> service lmtp { >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0660 >> user = postfix >> } >> user = vmail >> } >> service pop3-login { >> inet_listener pop3s { >> port = 0 >> } >> } >> service quota-warning { >> executable = script /usr/local/bin/quota-warning.sh >> unix_listener quota-warning { >> user = vmail >> } >> user = vmail >> } >> ssl = required >> ssl_cert = > ssl_cipher_list = >> EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!ECDSA:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA >> ssl_key = > ssl_prefer_server_ciphers = yes >> ssl_protocols = !SSLv2 !SSLv3 >> userdb { >> driver = prefetch >> } >> userdb { >> args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext >> driver = ldap >> } >> verbose_ssl = yes >> protocol lda { >> mail_plugins = sieve >> } >> protocol imap { >> mail_max_userip_connections = 3 >> mail_plugins = imap_quota imap_zlib >> } >> protocol pop3 { >> mail_max_userip_connections = 3 >> } >> >> >> >> >> >> >> >> >> # ======================== dovecot/conf.d/auth-ldap.conf.ext >> =================== # >> >> passdb { >> driver = ldap >> >> # Path for LDAP configuration file, see >> example-config/dovecot-ldap.conf.ext >> args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext >> } >> >> # "prefetch" user database means that the passdb already provided the >> # needed information and there's no need to do a separate userdb lookup. >> # >> userdb { >> driver = prefetch >> } >> >> userdb { >> driver = ldap >> args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext >> >> # Default fields can be used to specify defaults that LDAP may >> override >> #default_fields = home=/home/virtual/%u >> } >> >> >> >> >> >> # ======================== dovecot/dovecot-ldap.conf.ext >> =================== # >> >> hosts = 127.0.0.1:389 >> dn = uid=dovecot,ou=systemuser,ou=mail,dc=MyDomain,dc=TLD >> dnpass = TopSecret >> tls = yes >> tls_cert_file = /etc/ssl/RootCA/certs/192.168.50.101.pem >> tls_key_file = /etc/ssl/RootCA/certs/192.168.50.101.key >> debug_level = -1 >> auth_bind = yes >> ldap_version = 3 >> base = ou=accounts,ou=mail,dc=MyDomain,dc=TLD >> scope = subtree >> user_attrs = mailStorageDirectory=home, mailUidNumber=uid, >> mailGidNumber=gid, mailQuotaSize=quota_rule=*:bytes=%$, >> mailQuotaCount=quota_rule2 >> user_filter = (&(objectClass=mailAccount)(mailAddress=%u)) >> pass_attrs = mailAddress=user,mailPassword=password >> pass_filter = >> (&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=%u)) >> default_pass_scheme = SSHA From jtam.home at gmail.com Mon Jan 26 04:31:48 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Sun, 25 Jan 2015 20:31:48 -0800 (PST) Subject: LDA input validation In-Reply-To: References: Message-ID: St?phane Cottin writes: > dspam already send errors to syslog, the point here is to never loose > email contents. This was a wrong design, i'm now use a wrapper instead > ( see my previous post for details ). You're stilling going to lose contents. If dspam fails, the mail is dumped, the LDA returns exit code 75, and the MTA will probably issue a bounce Email to the sender. If you really don't want the recipient to lose Email, you should buffer the input into a file, run dspam on it, and if the output is not-null, pipe it to dovecot-lda, otherwise pipe the original input. Joseph Tam From skdovecot at smail.inf.fh-brs.de Mon Jan 26 07:52:14 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 26 Jan 2015 08:52:14 +0100 (CET) Subject: LDA input validation In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 25 Jan 2015, Joseph Tam wrote: > St?phane Cottin writes: > >> dspam already send errors to syslog, the point here is to never loose >> email contents. This was a wrong design, i'm now use a wrapper instead >> ( see my previous post for details ). > > You're stilling going to lose contents. If dspam fails, the mail > is dumped, the LDA returns exit code 75, and the MTA will probably > issue a bounce Email to the sender. which would be OK, if "never loose email contents" means "no message is discarded silently". > If you really don't want the recipient to lose Email, you should > buffer the input into a file, run dspam on it, and if the output > is not-null, pipe it to dovecot-lda, otherwise pipe the original > input. (Y) - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMXyLnz1H7kL/d9rAQJMfgf8ChF+8YSwDbGHribJO7fyOkrXVL81aiMG PXQr3NYEpz4WF1sQT+4lWeTfh+5FJ2HuNXLd2erN8KY3rdcwnxPrYdiZDia02JUx tA3QTMlmgOmPGc9rFqqob5TxuCzgYd+98dn4EokkNBUOK6J60J9uHmNKGkaZRXEd KD49FEJ8MYx7O+NryspJkr7jZELxp8L0wrLHUSZ/QQkqJRj6CqLKDP+InpdcIKqI Sp6sUF6BngFSS3x9+1xygjDOFWfc7KgmNu4gs+hsdPio+skY16QX8jhcKVCsYn/m 2Jv2wNB9vREe+2uDDc345Ssi5wnorhnn+fCrVKx7cDgziCRADDjPQA== =RGxT -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Jan 26 08:02:46 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 26 Jan 2015 09:02:46 +0100 (CET) Subject: Dovecot antispam In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 25 Jan 2015, David Mehler wrote: > I've got a Postfix, amavisd-new, dovecot (delivery and sieve), setup > with spamassassin. It's for the most part working fine, here's the > problem, false positives and false negatives. The sieve moves > spamassassin-tagged messages in to the Spam folder, sometimes it > misses either a false positive or negative. What I'm wanting to do and > this is where the dovecot-antispam plugin comes in is automatically > retrain messages, a false positive or negative needs to be moved and > reclassified accordingly. I've found a lot of google results > indicating dspam, when I used this the results were terrible after > three months of usage I gave up on it. I've thought about using spamc > with dovecot-antispam to handle retraining or crm114, but not much on > getting this going. If anyone has any information on this or any other > non-dspam setup that would work I'd appreciate knowing about it. What kind of information you are seeking? In what regards are the results of dspam _terrible_? Installation / Configuration? :- http://wiki2.dovecot.org/Plugins/Antispam - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMX0pnz1H7kL/d9rAQKDxQgAtPdGTRX3mbO/VQUQC2HIF0FlIt4197M5 agAKg6PhGnwOiQ7hxfm97JXQ6BICdgboZLmE1Tgz/EVPnv5joHY3zP6nJL1J+AQX H4elEeFB1Cw8zMgb/cYJ+kNt8tHlIISztLF6qzK8nvQeZHvNaE5o6y0rn6i7+8nn b7cVVJSebjia113CN0duRFNEJyNEkU9S94TsSFceQPAMyWlxBbpt7kakDOREznSU zjhC7BqDMtcZYy3K/lRSTPDFXickI2fmlal1C5oJFNuSqVjCtm7veh6uL9eBcPvw OYeyrPctnTFth/m1mzTqv0S3NKiswk5YqnR3C+5hJpNEmp/hanYE8w== =zEhj -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Mon Jan 26 08:28:30 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 26 Jan 2015 09:28:30 +0100 (CET) Subject: sendmail not invoking dovecot-lda In-Reply-To: <20150118194836.GA17728@Mail.Linux-Consulting.com> References: <20150118194836.GA17728@Mail.Linux-Consulting.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 18 Jan 2015, alvin wrote: > > - Configuration > FreeBSD-9.3 > sendmail -d0.1 == sendmail-8.14.9 <<-- > dovecot --version == dovecot-2.2.15 > > # =================================================================== > # I'm trying to get sendmail to invoke dovecot.m4 ( dovecot-lda ) to > # deliver emails to dovecot's virtual users ( /etc/dovecot/passwd ) > # or mysql/postgresql virtual users > # > # > # sendmail -bv dovecot.VirtualUser at localhost --> returns "User Unknown" > # > # cat test.msg.txt | dovecot-lda -d dovecot.VirtualUser ===> works > # email is delivered to dovecot.VirtualUser inbox > # > # =================================================================== > > - there are tons of howto and comments for dovecot + postfix > and very little for dovecot + sendmail ... therefore, it'd be > great if we/i can get this dovecot.m4 problem resolved > > ------------------------------------------------------------------------------ > - status of what works and what does NOT work while testing dovecot's LDA ... > ------------------------------------------------------------------------------ > # > # fyi.. recepient user is in the form of user, user at localhost and user at domain.com > # > sendmail -bv user.in/etc/passwd = deliverable > sendmail -bv user.in/etc/mail/virtusertable = deliverable to sendmail virtual user > > -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown 1) is the virtual domain a local name? http://compgroups.net/comp.mail.sendmail/sendmail-dovecot-mailertable-and-passwd-li/1316451 http://www.dovecot.org/list/dovecot/2010-June/049500.html Take note of the remark "reject unknown users early". 2) your mailer is named "dovecot", therefore the mailertable entry in the Wiki: virtualdomain.example.com vmail:vmail should spell, IMHO: virtualdomain.example.com dovecot:dovecot (2nd dovecot has no meaning, but there must be present something) 3) Post a sensable output: echo '3,0 dovecotuser at dovecot.domain' | sendmail -bt -d21.4 4) there had been a post about trying to improve virtual user support in sendmail incl. some ongoing talk. Maybe you find it and get some info back into this list. > > dovecot user user.in/etc/passwd == returns UID/GID/home/mailbox ( deliverable ) > dovecot user dovecot.user == returns UID/GID/home/mailbox ( deliverable ) > > > # -------------------------------------------------------------------- > # dovecot's LDA delivers the test email to the dovecot/mysql recepient > # -------------------------------------------------------------------- > cat test-email-with-headers.txt | dovecot-lda -d dovecot.user ( email delivered ) > > > # ---------------------------------------------------------------- > -->> # mail/mutt/sendmail canNOT deliver to dovecot recepient == FAILS > # ---------------------------------------------------------------- > -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown > 3 > echo "testing mail to dovecot" | mail -v -s "testing mail to dovecot" doveccot.user > == user unknown == > > ------------------------------------------------------- > - my (relevant to dovecot ) sendmail.mc file looks like: > ------------------------------------------------------- > > dnl # > dnl # local mail delivery > dnl # > define(`PROCMAIL_MAILER_PATH', `/usr/local/bin/procmail')dnl > > FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl > > dnl # > dnl # I'm trying to get sendmail to use dovecot.m4 to deliver local mail > dnl # to dovecot's LDA ( dovecot-lda ) for mysql virtual users > dnl # > dnl uncomment and use either feature(...dovecot-lda) or mailer(dovecot.m4) > dnl > dnl FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u') > dnl > > MAILER(local) > MAILER(smtp) > dnl > dnl MAILER(dovecot)dnl # did NOT work either > dnl > dnl # trying to use procmail to deliver local mail to system users ( /etc/passwd ) > MAILER(procmail) > dnl > dnl # is the ordering of dovecot.m4 important relative to (local) and (procmail) > dnl > dnl # trying to use dovecot-lda to deliver email to dovecot's virtual users > dnl # defined in /etc/dovecot/password > dnl > MAILER(dovecot)dnl # see below > dnl > dnl End of File > > ------------------------------------------------------- > - my (relevant to dovecot ) sendmail.cf file looks like: > ------------------------------------------------------- > > # > # lots of deleted ... ??not?? important until Mlocal ---> MAILER(local) > # > > Mlocal, P=/usr/libexec/mail.local, F=lsDFMAw5:/|@qPSXmnz9, S=EnvFromSMTP/HdrFromL, R=EnvToL/ > HdrToL, > T=DNS/RFC822/SMTP, > A=mail.local -l > Mprog, P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/, > T=X-Unix/X-Unix/X-Unix, > A=sh -c $u > > ##################################### > ### SMTP Mailer specification ### > ##################################### > > # > # lots of deleted ... ??not?? important until procmail stuff ---> MAILER(procmail) > # > > ######################*****############## > ### PROCMAIL Mailer specification ### > ##################*****################## > ##### $Id: procmail.m4,v 8.23 2013-11-22 20:51:14 ca Exp $ ##### > > Mprocmail, P=/usr/local/bin/procmail, F=DFMSPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFr > omSMTP, > T=DNS/RFC822/X-Unix, > A=procmail -Y -m $h $f $u > > ######################*****######################### > ### ### > ### 10-Jan-15 amo Dovecot.m4 ### > ### ### > ### dovecot.m4 stuff ### > ### ### > ##############################*****################# > Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, > S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, > T=DNS/RFC822/X-Unix, > A=/usr/local/libexec/dovecot/dovecot-lda -d $u > # > # End of sendmail.cf > > > ===================================================================== > - dovecot has an example dovecot.m4 ... but it doesn't allow sendmail > to call dovecot to deliver local mail with dovecot-lda > ===================================================================== > > http://wiki2.dovecot.org/LDA/Sendmail > > ######################################### > ### DOVECOT Mailer specification ### > ######################################### > # # > # dovecot virtual users defined in: /etc/dovecot/password # > # # > # cp /etc/mail/dovecot.m4 /usr/share/sendmail/cf/mailer # > # # > # cd /etc/mail && make # > # /etc/rc.d/sendmail restart # > # # > # the dovecot.m4 stanza are in the resulting sendmail.cf # > # # > ######################################### > Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, > S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP, > T=DNS/RFC822/X-Unix, > A=/usr/local/libexec/dovecot/dovecot-lda -d $u > # > # i've tried using these other options in lieu of the above : > # > # F=DFMPShnu9 # added "S" option > # F=lsDFMAw5:/|@qSPfhn9, > # > # S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, > # # > # # tried adding U= between S= and T= lines > # U=root:mail U=dovecot:mail U=dovenull:mail > # > # End of dovecot.m4 > > http://wiki2.dovecot.org/LDA/Sendmail > also states that one could use these 3 statements in lieu of dovecot.m4 > > # > # this replaces sendmail.cf Mlocal P=mail.local with P=dovecot-lda > # > FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda', \ > `/usr/local/libexec/dovecot/dovecot-lda -d $u') > dnl MODIFY_MAILER_FLAGS(`LOCAL', `-f') > MAILER(procmail) > > vi /etc/mail/mailertable > # > # cd /etc/mail && make && /etc/rc.d/sendmail restart > # > # make and restart sendmail after editing > # > virtualdomain.example.com vmail:vmail > > # other options where sendmail still wasn't able to execute dovecot-lda > # virtualdomain.example.com dovecot:local > # virtualdomain.example.com local:mail > # virtualdomain.example.com root:mail > > ------------------------------------ > - my relevant dovecot config options > ------------------------------------ > # > # are there any other major options i'm missing in the dovecot config files > # > doveconf > dovecot -a > # > # for /etc/passwd > passdb { > args = > driver = pam > .. > } > # > # ------------------------------------------------------------------------ > # need to get these dovecot's virtual users to be able send/receive emails > # ------------------------------------------------------------------------ > # sendmail -bv dovecot.VirtualUser at localhost > # dovecot.VirtuaUser at localhost... User unknown > # > passdb { > args = /etc/dovecot/passwd > # > # dovecot.VirtualUser:{PLAIN}password::::: > # > # end of file > driver = passwd-file > .. > } > # > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > .. > } > ... > ... > ... > userdb { > args = > driver = passwd > .. > } > # > userdb { > args = /etc/dovecot/password > driver = passwd-file > .. > } > # > userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > .. > } > > # > # --------------------------------------------------------------------- > # -- How do i get sendmail to execute dovecot-lda for incoming emails > # -- directed to dovecot virtual users and/or mysql users ?? > # --------------------------------------------------------------------- > # > # end of txt > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMX6r3z1H7kL/d9rAQKGPwf9Ei/dGybYnAMxHkdYt/GAtD2oMkbwtx6t frX0GC3Np1cfeiYWGGcOlV7RYXlBp/J1wxWYz3gzE4g9NhZjbIrvlIDBARD6WGzi R3OCvW/eCnJoKwNSe4lY3e2PnnFB2pxiB4ybSbTb5h1+/GQ4+IsCXoxb0cy/41q1 zgdpUTBo4gmt8DsnbViTTsaQbkEPlUsI73nSEFqOlKPDsa0Cuh/EnFRZc1RnDwT4 I2hXHLSlfJyZi2wslRdMeFQCtkx/81E1hsiPr+9K3ozXrgCp6BLUNCbTsdy8zfCw JDjoh1DX2toErElJMyQWITcKZyIn3jzxR67VRnN3Bf6ZjUTwfFwBhw== =qufZ -----END PGP SIGNATURE----- From h.reindl at thelounge.net Mon Jan 26 09:09:02 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 26 Jan 2015 10:09:02 +0100 Subject: LDA input validation In-Reply-To: References: Message-ID: <54C6042E.6010800@thelounge.net> Am 26.01.2015 um 08:52 schrieb Steffen Kaiser: > On Sun, 25 Jan 2015, Joseph Tam wrote: >> St?phane Cottin writes: > >>> dspam already send errors to syslog, the point here is to never loose >>> email contents. This was a wrong design, i'm now use a wrapper instead >>> ( see my previous post for details ). > >> You're stilling going to lose contents. If dspam fails, the mail >> is dumped, the LDA returns exit code 75, and the MTA will probably >> issue a bounce Email to the sender. > > which would be OK, if "never loose email contents" means "no message is > discarded silently". no, it is not OK to backscatter because the spamfilter fails realize that 99% auf junk is using forged senders recently i got each day some hundret such bounces from mailservers configured by fools reply to spam with forged senders and if i could i would have gone out for beat every responsible admin straight in the face -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From stephane.cottin at vixns.com Mon Jan 26 09:42:26 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Mon, 26 Jan 2015 10:42:26 +0100 Subject: LDA input validation In-Reply-To: References: Message-ID: > Le 26 janv. 2015 ? 05:31, Joseph Tam a ?crit : > > St?phane Cottin writes: > >> dspam already send errors to syslog, the point here is to never loose >> email contents. This was a wrong design, i'm now use a wrapper instead >> ( see my previous post for details ). > > You're stilling going to lose contents. If dspam fails, the mail > is dumped, the LDA returns exit code 75, and the MTA will probably > issue a bounce Email to the sender. from dovecot-lda man page : 75 A temporary failure. This is returned for almost all failures. See the log file for details. (EX_TEMPFAIL) The mta keep the mail and retry delivery later, which is what I need. > > If you really don't want the recipient to lose Email, you should > buffer the input into a file, run dspam on it, and if the output > is not-null, pipe it to dovecot-lda, otherwise pipe the original > input. I don't want to deliver email on dspam error, I prefer TEMPFAIL and future retries, so I have a chance to fix the issue later without loosing or deliver unprocessed mails > > Joseph Tam From stephane.cottin at vixns.com Mon Jan 26 09:52:38 2015 From: stephane.cottin at vixns.com (=?windows-1252?Q?St=E9phane_Cottin?=) Date: Mon, 26 Jan 2015 10:52:38 +0100 Subject: LDA input validation In-Reply-To: <54C6042E.6010800@thelounge.net> References: <54C6042E.6010800@thelounge.net> Message-ID: <0E8E113E-01E0-4F11-AF79-D2A0C845DC77@vixns.com> > Le 26 janv. 2015 ? 10:09, Reindl Harald a ?crit : > > > > Am 26.01.2015 um 08:52 schrieb Steffen Kaiser: >> On Sun, 25 Jan 2015, Joseph Tam wrote: >>> St?phane Cottin writes: >> >>>> dspam already send errors to syslog, the point here is to never loose >>>> email contents. This was a wrong design, i'm now use a wrapper instead >>>> ( see my previous post for details ). >> >>> You're stilling going to lose contents. If dspam fails, the mail >>> is dumped, the LDA returns exit code 75, and the MTA will probably >>> issue a bounce Email to the sender. >> >> which would be OK, if "never loose email contents" means "no message is >> discarded silently". > > no, it is not OK to backscatter because the spamfilter fails > > realize that 99% auf junk is using forged senders > > recently i got each day some hundret such bounces from mailservers configured by fools reply to spam with forged senders and if i could i would have gone out for beat every responsible admin straight in the face > I may discard emails based on RBLs, but I don't want to discard emails based on statistical fllters, I prefer deliver them in the Junk folder and let the user have a chance to reclassify using dovecot_antispam. And yes, bounce spams to (forged or not) sender is useless. From h.reindl at thelounge.net Mon Jan 26 10:21:14 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 26 Jan 2015 11:21:14 +0100 Subject: LDA input validation In-Reply-To: <0E8E113E-01E0-4F11-AF79-D2A0C845DC77@vixns.com> References: <54C6042E.6010800@thelounge.net> <0E8E113E-01E0-4F11-AF79-D2A0C845DC77@vixns.com> Message-ID: <54C6151A.3040105@thelounge.net> Am 26.01.2015 um 10:52 schrieb St?phane Cottin: >> Le 26 janv. 2015 ? 10:09, Reindl Harald a ?crit : >>>> You're stilling going to lose contents. If dspam fails, the mail >>>> is dumped, the LDA returns exit code 75, and the MTA will probably >>>> issue a bounce Email to the sender. >>> >>> which would be OK, if "never loose email contents" means "no message is >>> discarded silently". >> >> no, it is not OK to backscatter because the spamfilter fails >> >> realize that 99% auf junk is using forged senders >> >> recently i got each day some hundret such bounces from mailservers configured by fools reply to spam with forged senders and if i could i would have gone out for beat every responsible admin straight in the face > > I may discard emails based on RBLs, but I don't want to discard emails based on statistical fllters, I prefer deliver them in the Junk folder and let the user have a chance to reclassify using dovecot_antispam. > And yes, bounce spams to (forged or not) sender is useless you *must not* discard mails - in no context - period that's why milters exist to tag between let say 5.0 and 8.0 spam points and REJECT pre-queue based on SpamAssassin and/or ClamAV maybe dspam can't do that, but it's *abandonware* anyways http://comments.gmane.org/gmane.mail.spam.dspam.user/19136 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From info at netocean.de Mon Jan 26 10:27:16 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 11:27:16 +0100 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C5966F.9050601@thelounge.net> References: <54C58F43.7020803@netocean.de> <54C594B0.4030204@netocean.de> <54C5966F.9050601@thelounge.net> Message-ID: <54C61684.1000405@netocean.de> Am 26.01.15 um 02:24 schrieb Edgar Pettijohn: > Sorry didn't scroll to the bottom to see the dovecot -n. I'm assuming > freebsd has an /etc/login.conf similiar to openbsd. If so you may > need to do something similiar to this: > > dovecot:\ > :openfiles-cur=512:\ > :openfiles-max=2048:\ > :tc=daemon: > > Rebuild the login.conf.db file if necessary: > > # [ -f /etc/login.conf.db ] && cap_mkdb /etc/login.conf > > Hope this helps. > Thanks for the hint. FreeBSD takes sysctl(8) settings as a base to ulimit(3). I even tried tuning on those values. sysctl kern.maxfilesperproc=405587 sysctl kern.maxfiles=428433 Unfortunately this does not solve the problem. I don't think it is the root of the problem. The current value - even with Dovecot and Postfix already running is: kern.openfiles: 675 It is just, that Dovecot is kind of in a frozen condition when trying to login to it via telnet(1). Am 26.01.15 um 02:20 schrieb Reindl Harald: > > Am 26.01.2015 um 02:13 schrieb Leander Sch?fer: >> I just checked my ulimit again and it really seems like it has more >> than enough - so I still don't understand what I've configured wrong >> here ;/ >> >> root at WM-01 [~]$ su -m dovecot -c "ulimit -a" >> socket buffer size (bytes, -b) unlimited >> core file size (blocks, -c) unlimited >> data seg size (kbytes, -d) 33554432 >> file size (blocks, -f) unlimited >> max locked memory (kbytes, -l) 131072 >> max memory size (kbytes, -m) 7067352 >> open files (-n) 205587 >> pipe size (512 bytes, -p) 1 >> stack size (kbytes, -s) 8192 >> cpu time (seconds, -t) unlimited >> max user processes (-u) 11278 >> virtual memory (kbytes, -v) unlimited >> swap size (kbytes, -w) unlimited > > besides that "imap-login" typically does *not* run under the same user > (here dovenull versus dovecot) who tells you that 205587 is "more than > enough" just because it is a high value? Yes, because I am monitoring those values via Munin. I can clearly see that my system does not require a value even close to this. By average my system has 673 files open. In FreeBSD sysctl(8) is the base for ulimit(3) - meaning since I don't maintain user specific ulimit(3) - it is also the same values for dovenull, postfix, vmail and every other user on the system. My current sysctl(8) values should provide way enough space to grow super large: sysctl kern.maxfilesperproc=205587 sysctl kern.maxfiles=228433 And this is where it becomes so absurd to me. I don't see where I should tune anymore. I more and more get the feeling it is a configuration error of dovecot itself?! Thanks & Best Regards, Leander > > > From stephane.cottin at vixns.com Mon Jan 26 10:58:05 2015 From: stephane.cottin at vixns.com (=?windows-1252?Q?St=E9phane_Cottin?=) Date: Mon, 26 Jan 2015 11:58:05 +0100 Subject: LDA input validation In-Reply-To: <54C6151A.3040105@thelounge.net> References: <54C6042E.6010800@thelounge.net> <0E8E113E-01E0-4F11-AF79-D2A0C845DC77@vixns.com> <54C6151A.3040105@thelounge.net> Message-ID: > Le 26 janv. 2015 ? 11:21, Reindl Harald a ?crit : > > > Am 26.01.2015 um 10:52 schrieb St?phane Cottin: >>> Le 26 janv. 2015 ? 10:09, Reindl Harald a ?crit : >>>>> You're stilling going to lose contents. If dspam fails, the mail >>>>> is dumped, the LDA returns exit code 75, and the MTA will probably >>>>> issue a bounce Email to the sender. >>>> >>>> which would be OK, if "never loose email contents" means "no message is >>>> discarded silently". >>> >>> no, it is not OK to backscatter because the spamfilter fails >>> >>> realize that 99% auf junk is using forged senders >>> >>> recently i got each day some hundret such bounces from mailservers configured by fools reply to spam with forged senders and if i could i would have gone out for beat every responsible admin straight in the face >> >> I may discard emails based on RBLs, but I don't want to discard emails based on statistical fllters, I prefer deliver them in the Junk folder and let the user have a chance to reclassify using dovecot_antispam. >> And yes, bounce spams to (forged or not) sender is useless > > you *must not* discard mails - in no context - period right, s/discard/reject/ > > that's why milters exist to tag between let say 5.0 and 8.0 spam points and REJECT pre-queue based on SpamAssassin and/or ClamAV > > maybe dspam can't do that, but it's *abandonware* anyways > http://comments.gmane.org/gmane.mail.spam.dspam.user/19136 dspam is fast and lightweight, ideal for low memory virtual servers, and AFAIK the only spamassasin alternative. From jtam.home at gmail.com Mon Jan 26 11:03:18 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 26 Jan 2015 03:03:18 -0800 (PST) Subject: LDA input validation In-Reply-To: References: Message-ID: St?phane Cottin writes: >> You're stilling going to lose contents. If dspam fails, the mail >> is dumped, the LDA returns exit code 75, and the MTA will probably >> issue a bounce Email to the sender. > > from dovecot-lda man page : > > 75 A temporary failure. This is returned for almost all failures. See the log file for > details. (EX_TEMPFAIL) > > The mta keep the mail and retry delivery later, which is what I need. Ah, my mistake. But ... > I don't want to deliver email on dspam error, I prefer TEMPFAIL and > future retries, so I have a chance to fix the issue later without > loosing or deliver unprocessed mails ... you only have a limited time to fix it. Depending on the sending MTA's retry schedule, you might have only a few days or less to to deal with it before the sending MTA stops trying. If it's important not to lose mail, then the deliver on failure strategy would be more appropriate. You can even copy the input message that caused dpam to crash to an inspection queue for later perusal. Joseph Tam From info at netocean.de Mon Jan 26 11:16:59 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 12:16:59 +0100 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C61684.1000405@netocean.de> References: <54C58F43.7020803@netocean.de> <54C594B0.4030204@netocean.de> <54C5966F.9050601@thelounge.net> <54C61684.1000405@netocean.de> Message-ID: <54C6222B.9080100@netocean.de> I also keep on getting funny errors of dovecot EVEN THOUGH I authenticated successfully through SMTP (Dovecto SASL) ?!?! Just for debugging this entire issue I set those parameters: debug_level = -1 auth_verbose = yes auth_verbose_passwords = yes auth_debug = yes auth_debug_passwords = yes mail_debug = yes verbose_ssl = yes cat /var/loca/maillog Jan 26 12:08:22 WM-01 postfix/postfix-script[97931]: starting the Postfix mail system Jan 26 12:08:22 WM-01 postfix/master[97933]: daemon started -- version 1.0, configuration /usr/local/etc/postfix Jan 26 12:08:22 WM-01 dovecot: master: Dovecot v2.2.15 starting up for imap, pop3, lmtp Jan 26 12:08:59 WM-01 postfix/smtpd[99014]: connect from localhost[127.0.0.1] Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_extended_operation_s Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_extended_operation Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_initial_request Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_new_connection 1 1 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_open_connection Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_connect_to_host: TCP 127.0.0.1:389 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_new_socket: 32 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_prepare_socket: 32 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_connect_to_host: Trying 127.0.0.1:389 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_pvt_connect: fd: 32 tm: -1 async: 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: attempting to connect: Jan 26 12:08:59 WM-01 dovecot: auth: Error: connect success Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_open_defconn: successful Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_server_request Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid 1 (infinite timeout) Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid 1 all 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:08:59 2015 Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:08:59 WM-01 dovecot: auth: Error: * msgid 1, origid 1, status InProgress Jan 26 12:08:59 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid 1 all 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 1 all 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 1 message type extended-result Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 new referrals Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: mark request completed, ld 0x1245e100 msgid 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: request done: ld 0x1245e100 msgid 1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: res_errno: 0, res_error: <>, res_matched: <> Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_free_request (origid 1, msgid 1) Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_extended_result Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_result Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_msgfree Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:before/connect initialization Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv2/v3 write client hello A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read server hello A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS certificate verification: depth: 1, err: 0, subject: [...] Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS certificate verification: depth: 0, err: 0, subject: [...] Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read server certificate A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read server certificate request A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read server done A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 write client certificate A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 write client key exchange A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 write certificate verify A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 write change cipher spec A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 write finished A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 flush data Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read server session ticket A Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: SSL_connect:SSLv3 read finished A Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_bind Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_simple_bind Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_sasl_bind Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_initial_request Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_server_request Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:08:59 2015 Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:08:59 WM-01 dovecot: auth: Error: * msgid 2, origid 2, status InProgress Jan 26 12:08:59 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid -1 all 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 2 message type bind Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 new referrals Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: mark request completed, ld 0x1245e100 msgid 2 Jan 26 12:08:59 WM-01 dovecot: auth: Error: request done: ld 0x1245e100 msgid 2 Jan 26 12:08:59 WM-01 dovecot: auth: Error: res_errno: 0, res_error: <>, res_matched: <> Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_free_request (origid 2, msgid 2) Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_result Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_msgfree Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 1 status: Connected Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:08:59 2015 Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 0 (abandoned 0) Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:08:59 WM-01 postfix/smtpd[99014]: Anonymous TLS connection established from localhost[127.0.0.1]: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_search Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: "(&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=postmaster at mydomain.tld))" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: AND Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter_list "(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=postmaster at mydomain.tld)" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: "(objectClass=mailAccount)" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: "objectClass=mailAccount" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: "(mailAccountStatus=active)" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: "mailAccountStatus=active" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: "(mailAddress=postmaster at mydomain.tld)" Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: "mailAddress=postmaster at mydomain.tld" Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_build_search_req ATTRS: mailAddress Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_initial_request Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_server_request Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:09:09 2015 Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 3, origid 3, status InProgress Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 3 message type search-entry Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:09:09 2015 Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 3, origid 3, status InProgress Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 3 message type search-result Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 new referrals Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: mark request completed, ld 0x1245e100 msgid 3 Jan 26 12:09:09 WM-01 dovecot: auth: Error: request done: ld 0x1245e100 msgid 3 Jan 26 12:09:09 WM-01 dovecot: auth: Error: res_errno: 0, res_error: <>, res_matched: <> Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_free_request (origid 3, msgid 3) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_first_attribute Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_values Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_next_attribute Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_first_attribute Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_values Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_next_attribute Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_dn Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_bind Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_simple_bind Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_sasl_bind Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_initial_request Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_server_request Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:09:09 2015 Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 4, origid 4, status InProgress Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: Connected Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:09:09 2015 Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 4, origid 4, status InProgress Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals 0, parent count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 1 (abandoned 0) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 msgid 4 message type bind Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 new referrals Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: mark request completed, ld 0x1245e100 msgid 4 Jan 26 12:09:09 WM-01 dovecot: auth: Error: request done: ld 0x1245e100 msgid 4 Jan 26 12:09:09 WM-01 dovecot: auth: Error: res_errno: 0, res_error: <>, res_matched: <> Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_free_request (origid 4, msgid 4) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 msgid -1 Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 msgid -1 (timeout 0 usec) Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: 389 (default) Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 1 status: Connected Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 12:09:09 2015 Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Outstanding Requests: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request count 0 (abandoned 0) Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response Queue: Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response count 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld 0x1245e100 msgid -1 all 0 Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList returns ld 0x1245e100 NULL Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select Jan 26 12:09:20 WM-01 postfix/smtpd[99014]: lost connection after AUTH from localhost[127.0.0.1] Jan 26 12:09:20 WM-01 postfix/smtpd[99014]: disconnect from localhost[127.0.0.1] Am 26.01.15 um 11:27 schrieb Leander Sch?fer: > Am 26.01.15 um 02:24 schrieb Edgar Pettijohn: >> Sorry didn't scroll to the bottom to see the dovecot -n. I'm >> assuming freebsd has an /etc/login.conf similiar to openbsd. If so >> you may need to do something similiar to this: >> >> dovecot:\ >> :openfiles-cur=512:\ >> :openfiles-max=2048:\ >> :tc=daemon: >> >> Rebuild the login.conf.db file if necessary: >> >> # [ -f /etc/login.conf.db ] && cap_mkdb /etc/login.conf >> >> Hope this helps. >> > Thanks for the hint. FreeBSD takes sysctl(8) settings as a base to > ulimit(3). I even tried tuning on those values. > > sysctl kern.maxfilesperproc=405587 > sysctl kern.maxfiles=428433 > > Unfortunately this does not solve the problem. I don't think it is the > root of the problem. > > The current value - even with Dovecot and Postfix already running is: > kern.openfiles: 675 > It is just, that Dovecot is kind of in a frozen condition when trying > to login to it via telnet(1). > > > Am 26.01.15 um 02:20 schrieb Reindl Harald: >> >> Am 26.01.2015 um 02:13 schrieb Leander Sch?fer: >>> I just checked my ulimit again and it really seems like it has more >>> than enough - so I still don't understand what I've configured wrong >>> here ;/ >>> >>> root at WM-01 [~]$ su -m dovecot -c "ulimit -a" >>> socket buffer size (bytes, -b) unlimited >>> core file size (blocks, -c) unlimited >>> data seg size (kbytes, -d) 33554432 >>> file size (blocks, -f) unlimited >>> max locked memory (kbytes, -l) 131072 >>> max memory size (kbytes, -m) 7067352 >>> open files (-n) 205587 >>> pipe size (512 bytes, -p) 1 >>> stack size (kbytes, -s) 8192 >>> cpu time (seconds, -t) unlimited >>> max user processes (-u) 11278 >>> virtual memory (kbytes, -v) unlimited >>> swap size (kbytes, -w) unlimited >> >> besides that "imap-login" typically does *not* run under the same >> user (here dovenull versus dovecot) who tells you that 205587 is >> "more than enough" just because it is a high value? > Yes, because I am monitoring those values via Munin. I can clearly see > that my system does not require a value even close to this. By average > my system has 673 files open. In FreeBSD sysctl(8) is the base for > ulimit(3) - meaning since I don't maintain user specific ulimit(3) - > it is also the same values for dovenull, postfix, vmail and every > other user on the system. My current sysctl(8) values should provide > way enough space to grow super large: > > sysctl kern.maxfilesperproc=205587 > sysctl kern.maxfiles=228433 > > > And this is where it becomes so absurd to me. I don't see where I > should tune anymore. I more and more get the feeling it is a > configuration error of dovecot itself?! > > > Thanks & Best Regards, > Leander >> >> >> From stephane.cottin at vixns.com Mon Jan 26 11:22:51 2015 From: stephane.cottin at vixns.com (=?utf-8?Q?St=C3=A9phane_Cottin?=) Date: Mon, 26 Jan 2015 12:22:51 +0100 Subject: LDA input validation In-Reply-To: References: Message-ID: <23383AC0-FA63-47A4-9C35-4E0421CFAA4C@vixns.com> > Le 26 janv. 2015 ? 12:03, Joseph Tam a ?crit : > > St?phane Cottin writes: > >>> You're stilling going to lose contents. If dspam fails, the mail >>> is dumped, the LDA returns exit code 75, and the MTA will probably >>> issue a bounce Email to the sender. >> >> from dovecot-lda man page : >> >> 75 A temporary failure. This is returned for almost all failures. See the log file for >> details. (EX_TEMPFAIL) >> >> The mta keep the mail and retry delivery later, which is what I need. > > Ah, my mistake. But ... > >> I don't want to deliver email on dspam error, I prefer TEMPFAIL and >> future retries, so I have a chance to fix the issue later without >> loosing or deliver unprocessed mails > > ... you only have a limited time to fix it. Depending on the sending > MTA's retry schedule, you might have only a few days or less to to deal > with it before the sending MTA stops trying. We hopfully are notified whitin a few minutes when something's wrong in mail delivery, having a few days to fix is far more than needed. > > If it's important not to lose mail, then the deliver on failure strategy > would be more appropriate. You can even copy the input message that > caused dpam to crash to an inspection queue for later perusal. We do not have dspam crashes caused per email contents, they always are system( disk errors, oom kill,?) , clamav or database issues. Anyways, having a copy of the input message somewhere, whatever it's unrelated to the error is a good advice, we'll think about it. > > Joseph Tam From info at netocean.de Mon Jan 26 14:22:07 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 15:22:07 +0100 Subject: Thunderbird: improper command pipelining after EHLO Message-ID: <54C64D8F.2090406@netocean.de> I couldn't find working solutions for this anomalie on the net. What does this mean and does someone know how to fix this? postfix/smtpd[18757]: improper command pipelining after EHLO from unknown[192.168.10.233]: QUIT\r\n Thanks Best Regards, Leander From h.reindl at thelounge.net Mon Jan 26 14:29:10 2015 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 26 Jan 2015 15:29:10 +0100 Subject: Thunderbird: improper command pipelining after EHLO In-Reply-To: <54C64D8F.2090406@netocean.de> References: <54C64D8F.2090406@netocean.de> Message-ID: <54C64F36.3060902@thelounge.net> Am 26.01.2015 um 15:22 schrieb Leander Sch?fer: > I couldn't find working solutions for this anomalie on the net. What > does this mean and does someone know how to fix this? > > postfix/smtpd[18757]: improper command pipelining after EHLO from > unknown[192.168.10.233]: QUIT\r\n that's hardly a dovecot topic and without "postconf -n", in doubt with content of "master.cf" and more informations nobody can help you at all -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From Olaf.Hopp at kit.edu Mon Jan 26 14:43:08 2015 From: Olaf.Hopp at kit.edu (Olaf Hopp) Date: Mon, 26 Jan 2015 15:43:08 +0100 Subject: Sieve permissions issue following update In-Reply-To: <54A5742A.8040207@rename-it.nl> References: <548732CB.4070606@blackrosetech.com> <5487351C.7030303@localhost.localdomain.org> <5487528D.6030105@blackrosetech.com> <54A54734.4010805@rename-it.nl> <59183495-00BF-4C02-B56E-E118A50386DF@inoc.net> <54A55E9F.8030905@rename-it.nl> <54A5742A.8040207@rename-it.nl> Message-ID: <54C6527C.1010902@kit.edu> On 01/01/2015 05:22 PM, Stephan Bosch wrote: > On 1/1/2015 4:17 PM, Robert Blayzor wrote: >> On Jan 1, 2015, at 9:58 AM, Robert Blayzor wrote: >>>> Hmm. This smells like a bug. I notice that your modification times of >>>> the .sieve and .svbin file are exactly the same (that is somewhat >>>> unusual). I'm looking at a potential bug that would explain your problem. >>>> >>>> To confirm, could you try running sievec again, so that the .svbin is >>>> actually newer than the .sieve? >> >> If it makes any difference at all... I only see this using "dovecot-lda". If I change my Exim transport to use Dovecot's LMTP, I do not see this problem. > > That is odd. > Hi Stephan and Robert, the same issue here and I'm using Exim with dovecot-lmtp and not with dovecot-lda. So it doesn't seem to be a problem of LDA vs. lmtp Pigeonhole 0.4.5 Dovecot 2.2.15 CentOS 6.6 Regards, Olaf -- Karlsruher Institut f?r Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakult?t f?r Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Geb?ude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp at kit.edu atis.informatik.kit.edu www.kit.edu KIT - Universit?t des Landes Baden-W?rttemberg und nationales Forschungszentrum in der Helmholtz-Gemeinschaft -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5214 bytes Desc: S/MIME Cryptographic Signature URL: From raubvogel at gmail.com Mon Jan 26 14:44:46 2015 From: raubvogel at gmail.com (Mauricio Tavares) Date: Mon, 26 Jan 2015 09:44:46 -0500 Subject: Thunderbird: improper command pipelining after EHLO In-Reply-To: <54C64F36.3060902@thelounge.net> References: <54C64D8F.2090406@netocean.de> <54C64F36.3060902@thelounge.net> Message-ID: On Mon, Jan 26, 2015 at 9:29 AM, Reindl Harald wrote: > > Am 26.01.2015 um 15:22 schrieb Leander Sch?fer: >> >> I couldn't find working solutions for this anomalie on the net. What >> does this mean and does someone know how to fix this? >> >> postfix/smtpd[18757]: improper command pipelining after EHLO from >> unknown[192.168.10.233]: QUIT\r\n > > that's hardly a dovecot topic and without "postconf -n", in doubt with > content of "master.cf" and more informations nobody can help you at all > Agreed. smtpd means mail going out. Unless you setup dovecot to help with the authentication, it could not care less about how your email leaves your server. If you control your postfix server, crank up debugging and see if that helps. If you can't, try the thunderbird list/forum; it too has a debugging mode (https://wiki.mozilla.org/MailNews:Logging). From info at netocean.de Mon Jan 26 15:01:49 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 16:01:49 +0100 Subject: Thunderbird: improper command pipelining after EHLO In-Reply-To: <54C64F36.3060902@thelounge.net> References: <54C64D8F.2090406@netocean.de> <54C64F36.3060902@thelounge.net> Message-ID: <54C656DD.7060501@netocean.de> I'll check my master.cf again. I'm also more and more sure the mistake is hiding in there. Thanks Am 26.01.15 um 15:29 schrieb Reindl Harald: > > Am 26.01.2015 um 15:22 schrieb Leander Sch?fer: >> I couldn't find working solutions for this anomalie on the net. What >> does this mean and does someone know how to fix this? >> >> postfix/smtpd[18757]: improper command pipelining after EHLO from >> unknown[192.168.10.233]: QUIT\r\n > > that's hardly a dovecot topic and without "postconf -n", in doubt with > content of "master.cf" and more informations nobody can help you at all > > On Mon, Jan 26, 2015 at 9:29 AM, Reindl Harald wrote: >> Am 26.01.2015 um 15:22 schrieb Leander Sch?fer: >>> I couldn't find working solutions for this anomalie on the net. What >>> does this mean and does someone know how to fix this? >>> >>> postfix/smtpd[18757]: improper command pipelining after EHLO from >>> unknown[192.168.10.233]: QUIT\r\n >> that's hardly a dovecot topic and without "postconf -n", in doubt with >> content of "master.cf" and more informations nobody can help you at all >> > Agreed. smtpd means mail going out. Unless you setup dovecot to > help with the authentication, it could not care less about how your > email leaves your server. If you control your postfix server, crank up > debugging and see if that helps. If you can't, try the thunderbird > list/forum; it too has a debugging mode > (https://wiki.mozilla.org/MailNews:Logging). From dave.mehler at gmail.com Mon Jan 26 15:58:04 2015 From: dave.mehler at gmail.com (David Mehler) Date: Mon, 26 Jan 2015 10:58:04 -0500 Subject: Dovecot antispam In-Reply-To: References: Message-ID: Hi, Can I get a look at your antispam configs? I'd like to compare them against mine. Thanks. Dave. On 1/26/15, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Sun, 25 Jan 2015, David Mehler wrote: > >> I've got a Postfix, amavisd-new, dovecot (delivery and sieve), setup >> with spamassassin. It's for the most part working fine, here's the >> problem, false positives and false negatives. The sieve moves >> spamassassin-tagged messages in to the Spam folder, sometimes it >> misses either a false positive or negative. What I'm wanting to do and >> this is where the dovecot-antispam plugin comes in is automatically >> retrain messages, a false positive or negative needs to be moved and >> reclassified accordingly. I've found a lot of google results >> indicating dspam, when I used this the results were terrible after >> three months of usage I gave up on it. I've thought about using spamc >> with dovecot-antispam to handle retraining or crm114, but not much on >> getting this going. If anyone has any information on this or any other >> non-dspam setup that would work I'd appreciate knowing about it. > > What kind of information you are seeking? > In what regards are the results of dspam _terrible_? > > Installation / Configuration? :- http://wiki2.dovecot.org/Plugins/Antispam > > - -- > Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVMX0pnz1H7kL/d9rAQKDxQgAtPdGTRX3mbO/VQUQC2HIF0FlIt4197M5 > agAKg6PhGnwOiQ7hxfm97JXQ6BICdgboZLmE1Tgz/EVPnv5joHY3zP6nJL1J+AQX > H4elEeFB1Cw8zMgb/cYJ+kNt8tHlIISztLF6qzK8nvQeZHvNaE5o6y0rn6i7+8nn > b7cVVJSebjia113CN0duRFNEJyNEkU9S94TsSFceQPAMyWlxBbpt7kakDOREznSU > zjhC7BqDMtcZYy3K/lRSTPDFXickI2fmlal1C5oJFNuSqVjCtm7veh6uL9eBcPvw > OYeyrPctnTFth/m1mzTqv0S3NKiswk5YqnR3C+5hJpNEmp/hanYE8w== > =zEhj > -----END PGP SIGNATURE----- > From edgar at pettijohn-web.com Mon Jan 26 22:20:33 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Mon, 26 Jan 2015 16:20:33 -0600 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C61684.1000405@netocean.de> References: <54C58F43.7020803@netocean.de> <54C594B0.4030204@netocean.de> <54C5966F.9050601@thelounge.net> <54C61684.1000405@netocean.de> Message-ID: <54C6BDB1.6060806@pettijohn-web.com> On 01/26/15 04:27, Leander Sch?fer wrote: > Am 26.01.15 um 02:24 schrieb Edgar Pettijohn: >> Sorry didn't scroll to the bottom to see the dovecot -n. I'm >> assuming freebsd has an /etc/login.conf similiar to openbsd. If so >> you may need to do something similiar to this: >> >> dovecot:\ >> :openfiles-cur=512:\ >> :openfiles-max=2048:\ >> :tc=daemon: >> >> Rebuild the login.conf.db file if necessary: >> >> # [ -f /etc/login.conf.db ] && cap_mkdb /etc/login.conf >> >> Hope this helps. >> > Thanks for the hint. FreeBSD takes sysctl(8) settings as a base to > ulimit(3). I even tried tuning on those values. > > sysctl kern.maxfilesperproc=405587 > sysctl kern.maxfiles=428433 > > Unfortunately this does not solve the problem. I don't think it is the > root of the problem. > > The current value - even with Dovecot and Postfix already running is: > kern.openfiles: 675 > It is just, that Dovecot is kind of in a frozen condition when trying > to login to it via telnet(1). > > > Am 26.01.15 um 02:20 schrieb Reindl Harald: >> >> Am 26.01.2015 um 02:13 schrieb Leander Sch?fer: >>> I just checked my ulimit again and it really seems like it has more >>> than enough - so I still don't understand what I've configured wrong >>> here ;/ >>> >>> root at WM-01 [~]$ su -m dovecot -c "ulimit -a" >>> socket buffer size (bytes, -b) unlimited >>> core file size (blocks, -c) unlimited >>> data seg size (kbytes, -d) 33554432 >>> file size (blocks, -f) unlimited >>> max locked memory (kbytes, -l) 131072 >>> max memory size (kbytes, -m) 7067352 >>> open files (-n) 205587 >>> pipe size (512 bytes, -p) 1 >>> stack size (kbytes, -s) 8192 >>> cpu time (seconds, -t) unlimited >>> max user processes (-u) 11278 >>> virtual memory (kbytes, -v) unlimited >>> swap size (kbytes, -w) unlimited >> >> besides that "imap-login" typically does *not* run under the same >> user (here dovenull versus dovecot) who tells you that 205587 is >> "more than enough" just because it is a high value? > Yes, because I am monitoring those values via Munin. I can clearly see > that my system does not require a value even close to this. By average > my system has 673 files open. In FreeBSD sysctl(8) is the base for > ulimit(3) - meaning since I don't maintain user specific ulimit(3) - > it is also the same values for dovenull, postfix, vmail and every > other user on the system. My current sysctl(8) values should provide > way enough space to grow super large: > > sysctl kern.maxfilesperproc=205587 > sysctl kern.maxfiles=228433 > > > And this is where it becomes so absurd to me. I don't see where I > should tune anymore. I more and more get the feeling it is a > configuration error of dovecot itself?! > > > Thanks & Best Regards, > Leander >> >> >> I know thats the error it gave me until I changed the settings for dovecot in login.conf. The only other thing I would suggest trying is install dovecot from the opposite of what you installed it from either packages or ports, and see if its a bug in the build. From edgar at pettijohn-web.com Mon Jan 26 22:27:18 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn) Date: Mon, 26 Jan 2015 16:27:18 -0600 Subject: Thunderbird: improper command pipelining after EHLO In-Reply-To: <54C64F36.3060902@thelounge.net> References: <54C64D8F.2090406@netocean.de> <54C64F36.3060902@thelounge.net> Message-ID: <54C6BF46.20502@pettijohn-web.com> On 01/26/15 08:29, Reindl Harald wrote: > > Am 26.01.2015 um 15:22 schrieb Leander Sch?fer: >> I couldn't find working solutions for this anomalie on the net. What >> does this mean and does someone know how to fix this? >> >> postfix/smtpd[18757]: improper command pipelining after EHLO from >> unknown[192.168.10.233]: QUIT\r\n > > that's hardly a dovecot topic and without "postconf -n", in doubt with > content of "master.cf" and more informations nobody can help you at all > Looks like its from an internal machine. Maybe someone is playing with telnet. From stephan at rename-it.nl Mon Jan 26 23:03:00 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 27 Jan 2015 00:03:00 +0100 Subject: "error: number of redirect actions exceeds policy limit" In-Reply-To: <54C170A6.4020303@i-sommer.de> References: <54C170A6.4020303@i-sommer.de> Message-ID: <54C6C7A4.7070301@rename-it.nl> On 1/22/2015 10:50 PM, Kai Sommer wrote: > Greetings, > > after testing a lot with my system I have to write to this list to ask > for help from the cummunity. > > My system: > * Ubuntu 12.04.5 LTS > * Dovecot 2.0.19 > * Dovecot config (extract) This is a relic of ancient times. I checked and I cannot reproduce this problem with the latest version. I can't exactly remember what this bug actually was or when it was fixed, but it is a long time ago. There is not much I can do for you if you don't upgrade. Regards, Stephan. From superinterstellar at gmail.com Tue Jan 27 02:24:08 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Tue, 27 Jan 2015 10:24:08 +0800 Subject: Indexing Mail faster In-Reply-To: <20150124190408.GA436@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: Hi Thomas, That's very interesting. Below is my search result. Any idea why is my result so slow:- Appreciate if you could advise. Thanks Kevin . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA] Logged in . select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded \*)] Flags permitted. * 49225 EXISTS * 73 RECENT * OK [UNSEEN 46791] First unseen. * OK [UIDVALIDITY 1414214135] UIDs valid * OK [UIDNEXT 106460] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed (0.548 secs). . search text dear * OK Searched 9% of the mailbox, ETA 1:39 * OK Searched 20% of the mailbox, ETA 1:18 * OK Searched 29% of the mailbox, ETA 1:12 * OK Searched 38% of the mailbox, ETA 1:03 * OK Searched 50% of the mailbox, ETA 0:49 * OK Searched 60% of the mailbox, ETA 0:39 * OK Searched 69% of the mailbox, ETA 0:30 * OK Searched 79% of the mailbox, ETA 0:20 On Sun, Jan 25, 2015 at 3:04 AM, Thomas Leuxner wrote: > * Kevin Laurie 2015.01.24 19:41: > > > Currently the time it takes to search 25,000mails is 4mins. If indexed > how > > much faster are we looking at? > > With a current version of Dovecot a search is pretty fast _without_ using > external indexes. I have a view defined (virtual plugin) with around 22.000 > messages in it, and searching the full view only takes 2.5 seconds: > > $ /usr/lib/dovecot/imap -u tlx at leuxner.net > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN > CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL > RIGHTS=texk] Logged in as tlx at leuxner.net > a select "Virtual/Postfix-Users" > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Old) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Old \*)] > Flags permitted. > * 22094 EXISTS > * 0 RECENT > * OK [UIDVALIDITY 1420123174] UIDs valid > * OK [UIDNEXT 44036] Predicted next UID > * OK [HIGHESTMODSEQ 679] Highest > a OK [READ-WRITE] Select completed (0.028 secs). > b search charset utf-8 body "SQL" > * SEARCH 51 52 54 83 87 102 108 126 172 173 195 223 234 238 249 250 252 > 254 260 263 268 274 293 294 316 317 324 343 412 414 431 432 434 435 436 438 > 439 440 441 442 443 450 451 454 457 458 464 478 493 496 497 505 590 617 619 > 625 628 632 636 637 638 639 644 645 646 650 655 657 659 671 676 677 699 702 > 757 759 762 767 769 770 790 791 792 793 796 799 839 840 841 843 844 845 846 > 847 848 849 850 864 868 882 905 907 929 953 955 959 1129 1133 1136 1137 > 1138 1139 1140 1143 1145 1168 1174 1175 1176 1177 1178 1179 1182 1205 1208 > 1244 1247 1248 1251 1316 1320 1334 1336 1352 1356 1392 1394 1395 1397 1398 > 1403 1407 1409 1410 1411 1433 1434 1435 1451 1468 1469 1470 1511 1512 1546 > 1547 1548 1560 1561 1562 1563 1567 1569 1570 1574 1578 1587 1588 1590 1601 > 1606 1648 1661 1669 1670 1671 1726 1748 1749 1750 1751 1752 1753 1755 1771 > 1773 1774 1792 1800 1808 1819 1822 1850 1885 1913 1932 1933 1980 1982 1999 > 2004 2026 2102 2104 2144 2145 2162 2184 2192 2193 2194 2204 2205 2217 2237 > 2272 2336 2337 2339 2341 2342 2343 2344 2346 2348 2368 2369 2380 2409 2426 > 2429 2455 2461 2499 2605 2607 2609 2611 2615 2618 2620 2635 2638 2639 2640 > 2642 2673 2784 2797 2799 2800 2821 2830 2849 2857 2863 2894 2895 2943 2944 > 2953 2954 2955 3049 3050 3052 3195 3244 3418 3419 3420 3421 3425 3465 3470 > 3474 3530 3543 3544 3546 3549 3550 3569 3572 3593 3602 3643 3647 3648 3649 > 3650 3654 3655 3656 3673 3674 3721 3722 3723 3724 3728 3740 3741 3772 3795 > 3797 3799 3803 3804 3806 3830 3832 3837 3838 3839 3840 3841 3859 3865 3866 > 3867 3875 3896 3898 3907 3915 3919 3924 3925 3927 3949 3951 4029 4031 4043 > 4073 4083 4084 4085 4087 4089 4091 4094 4096 4185 4186 4350 4354 4400 4401 > 4403 4404 4405 4406 4417 4419 4422 4423 4424 4425 4427 4431 4463 4501 4554 > 4556 4583 4615 4640 4642 4650 4683 4684 4686 4687 4688 4690 4691 4696 4697 > 4698 4699 4702 4705 4713 4714 4724 4725 4726 4727 4728 4729 4730 4734 4746 > 4754 4770 4772 4785 4787 4792 4794 4798 4800 4912 4922 4955 4956 4958 4966 > 4972 4973 4974 4975 4978 4979 4988 4990 4991 4992 4998 5009 5021 5025 5026 > 5027 5029 5030 5031 5036 5037 5109 5110 5111 5112 5149 5150 5151 5156 5159 > 5202 5248 5249 5365 5366 5367 5402 5428 5441 5442 5447 5449 5483 5484 5486 > 5487 5489 5493 5494 5495 5496 5497 5498 5499 5501 5502 5503 5504 5506 5508 > 5604 5681 5721 5722 5723 5724 5725 5727 5728 5729 5730 5731 5737 5738 5761 > 5778 5794 5796 5817 5818 5860 5863 5864 5865 5867 5989 5990 5991 6006 6028 > 6075 6078 6080 6081 6082 6083 6090 6092 6093 6103 6104 6115 6122 6193 6194 > 6197 6198 6235 6243 6258 6259 6260 6268 6312 6317 6321 6408 6409 6471 6472 > 6485 6546 6551 6620 6622 6623 6624 6630 6637 6652 6694 6698 6710 6768 6770 > 6772 6797 6851 6853 6868 6899 6900 6949 7092 7093 7095 7156 7219 7226 7227 > 7265 7278 7280 7281 7282 7283 7294 7295 7305 7307 7310 7317 7319 7327 7338 > 7341 7342 7358 7363 7365 7370 7378 7381 7382 7434 7447 7448 7449 7450 7452 > 7509 7516 7517 7518 7520 7521 7522 7523 7524 7525 7526 7544 7585 7614 7616 > 7617 7637 7644 7670 7671 7794 7795 7796 7885 7886 7887 7889 7895 7898 7900 > 7953 8016 8052 8054 8055 8056 8057 8059 8063 8064 8066 8069 8079 8119 8132 > 8156 8175 8232 8233 8234 8235 8236 8242 8289 8354 8420 8506 8507 8508 8511 > 8546 8622 8666 8667 8709 8873 8874 8875 8876 8877 8879 8887 8892 8898 8899 > 8927 8956 8957 8984 8988 8993 9056 9166 9207 9225 9226 9227 9229 9230 9231 > 9232 9262 9263 9264 9265 9266 9267 9268 9269 9272 9357 9366 9369 9371 9378 > 9381 9390 9392 9393 9402 9403 9409 9416 9417 9426 9430 9431 9432 9455 9458 > 9540 9553 9554 9574 9575 9577 9593 9599 9601 9610 9611 9612 9613 9618 9619 > 9625 9627 9641 9642 9659 9685 9694 9735 9737 9772 9773 9774 9775 9776 9777 > 9781 9795 9802 9803 9806 9827 9829 9844 9866 9867 9875 9914 9949 9954 9957 > 9958 9959 9960 9961 9962 9967 9968 9969 9970 9986 9987 9988 9996 10003 > 10004 10006 10037 10040 10041 10043 10044 10046 10048 10049 10053 10072 > 10107 10121 10132 10136 10138 10140 10142 10143 10144 10146 10158 10276 > 10306 10336 10343 10353 10366 10389 10390 10400 10401 10413 10421 10429 > 10511 10709 10710 10711 10712 10713 10714 10715 10717 10719 10720 10728 > 10778 10779 10799 10800 10801 10819 10823 10829 10830 10850 10853 10857 > 10859 10861 10863 10874 10883 10884 10916 10932 11009 11010 11011 11083 > 11084 11089 11100 11118 11121 11147 11148 11161 11162 11163 11168 11171 > 11226 11228 11229 11255 11342 11359 11369 11375 11378 11461 11463 11464 > 11465 11466 11543 11548 11549 11552 11557 11558 11559 11673 11704 11721 > 11756 11812 11826 11827 11828 11829 11830 11832 11836 11838 11840 11841 > 11843 11844 11867 11868 11871 11872 11876 11885 11892 11985 11988 11992 > 12143 12154 12182 12183 12185 12198 12287 12329 12330 12334 12338 12345 > 12387 12391 12393 12394 12443 12502 12586 12589 12590 12645 12652 12653 > 12654 12656 12701 12702 12709 12711 12718 12719 12720 12722 12745 12790 > 12795 12837 12847 12914 12915 13098 13099 13108 13111 13124 13131 13132 > 13146 13157 13159 13160 13161 13170 13175 13176 13181 13258 13264 13265 > 13271 13277 13290 13291 13410 13431 13432 13454 13455 13456 13458 13459 > 13469 13478 13548 13549 13550 13557 13558 13567 13570 13590 13591 13594 > 13664 13679 13683 13684 13685 13686 13714 13715 13718 13735 13766 13788 > 13790 13800 13801 13805 13848 13855 13865 13866 13893 13911 13912 13914 > 13915 13918 13919 13942 13943 13944 13945 13946 13967 13968 13969 13975 > 13980 13986 13989 13999 14001 14007 14008 14010 14043 14044 14065 14077 > 14080 14081 14082 14149 14151 14156 14157 14183 14186 14305 14308 14359 > 14366 14455 14477 14479 14481 14490 14491 14566 14622 14640 14644 14653 > 14661 14662 14663 14664 14708 14712 14732 14733 14734 14755 14781 14805 > 14807 14808 14809 14825 14832 14833 14853 14860 14902 14905 14907 14908 > 14929 14931 14940 14954 14955 15010 15021 15026 15037 15039 15057 15058 > 15062 15065 15066 15142 15143 15145 15154 15169 15172 15206 15220 15294 > 15398 15424 15435 15436 15512 15558 15569 15616 15634 15649 15653 15776 > 15778 15827 15868 15870 15899 15900 15920 15926 15927 15928 15929 15944 > 15946 15948 16013 16014 16015 16074 16075 16078 16080 16103 16104 16105 > 16108 16109 16111 16112 16115 16116 16117 16118 16121 16122 16124 16125 > 16128 16161 16162 16165 16166 16190 16192 16272 16327 16351 16355 16382 > 16393 16409 16410 16467 16492 16493 16494 16497 16510 16511 16513 16514 > 16515 16519 16527 16587 16618 16632 16680 16769 16770 16771 16773 16776 > 16777 16778 16779 16830 16833 16839 16859 16863 16871 16872 16873 16875 > 16876 16877 16878 16879 16880 16885 16886 16919 16951 16967 16971 16974 > 16979 16986 16996 17040 17041 17052 17059 17060 17077 17081 17092 17196 > 17198 17203 17211 17212 17258 17302 17365 17367 17374 17391 17398 17424 > 17445 17447 17474 17476 17479 17481 17551 17568 17569 17570 17571 17572 > 17573 17575 17577 17585 17664 17665 17677 17678 17712 17713 17715 17755 > 17756 17757 17762 17784 17830 17831 17897 17906 17951 18024 18040 18042 > 18129 18136 18137 18138 18161 18162 18163 18167 18169 18222 18234 18275 > 18284 18285 18286 18287 18291 18342 18349 18356 18358 18361 18362 18363 > 18373 18375 18396 18436 18437 18438 18440 18441 18457 18460 18462 18467 > 18474 18475 18477 18479 18485 18486 18488 18489 18491 18492 18493 18494 > 18499 18503 18514 18576 18585 18586 18602 18603 18604 18605 18664 18769 > 18770 18772 18773 18774 18811 18813 18944 18952 18953 18981 18982 18983 > 18998 19124 19127 19129 19130 19136 19171 19174 19176 19180 19181 19182 > 19217 19218 19241 19243 19245 19257 19259 19276 19358 19360 19361 19362 > 19363 19375 19388 19390 19391 19393 19394 19421 19422 19423 19437 19492 > 19579 19594 19660 19668 19918 19920 19921 20016 20032 20034 20036 20038 > 20041 20053 20055 20149 20150 20159 20210 20211 20258 20282 20301 20355 > 20356 20357 20358 20360 20371 20376 20467 20468 20469 20475 20477 20481 > 20482 20483 20485 20487 20488 20489 20490 20491 20492 20493 20498 20504 > 20510 20533 20534 20535 20536 20537 20538 20541 20543 20546 20576 20580 > 20583 20585 20626 20634 20655 20724 20725 20726 20727 20728 20729 20730 > 20731 20732 20733 20962 20967 20968 20970 20974 21007 21008 21009 21010 > 21012 21019 21028 21043 21047 21056 21103 21108 21130 21145 21185 21253 > 21474 21482 21484 21488 21489 21523 21543 21544 21545 21548 21554 21566 > 21583 21584 21585 21586 21610 21631 21685 21717 21724 21731 21859 21860 > 21885 > b OK Search completed (2.493 secs). > From mkawada at redhat.com Tue Jan 27 02:54:57 2015 From: mkawada at redhat.com (mkawada at redhat.com) Date: Tue, 27 Jan 2015 11:54:57 +0900 Subject: dovecot migration Message-ID: <54C6FE01.3040500@redhat.com> Dear List, To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I wonder if the UIDL format "pop3_uidl_format" setting is required in the configuration file for clients not to re-download the same mails from the migrated mail server. Currently, no UIDL format is set in /etc/dovecot file of the one using dovecot-1.x server which means it is commented out as default like below. ]# grep pop3_uidl_format /etc/dovecot.conf #pop3_uidl_format = %08Xu%08Xv <---- Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one using dovecot-2.x, and have not been able to find any information saying it's the defualt setting. Here are the questions: 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default setting? Or any ways to check the default? 2.Do they need to specify anything for the "pop3_uidl_format =" field for their migration in this case? Env infomation: dovecot-1.x dovecot-2.x mail_location = mbox Any information/clues would be greatly appreciated. Thanks, From tlx at leuxner.net Tue Jan 27 08:02:21 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 27 Jan 2015 09:02:21 +0100 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: <20150127080221.GA38114@nihlus.leuxner.net> * Kevin Laurie 2015.01.27 03:24: > . search text dear > * OK Searched 9% of the mailbox, ETA 1:39 > * OK Searched 20% of the mailbox, ETA 1:18 > * OK Searched 29% of the mailbox, ETA 1:12 > * OK Searched 38% of the mailbox, ETA 1:03 > * OK Searched 50% of the mailbox, ETA 0:49 > * OK Searched 60% of the mailbox, ETA 0:39 > * OK Searched 69% of the mailbox, ETA 0:30 > * OK Searched 79% of the mailbox, ETA 0:20 I guess we need to start looking at your version and the format/mail_location configuration, best to post your 'doveconf -n'. Emulating the same search on my box looks a lot better: $ /usr/lib/dovecot/imap -u tlx at leuxner.net * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL RIGHTS=texk] Logged in as tlx at leuxner.net a select "Virtual/Postfix-Users" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Old) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Old \*)] Flags permitted. * 22120 EXISTS * 0 RECENT * OK [UNSEEN 22109] First unseen. * OK [UIDVALIDITY 1420123174] UIDs valid * OK [UIDNEXT 44062] Predicted next UID * OK [HIGHESTMODSEQ 702] Highest a OK [READ-WRITE] Select completed (0.031 secs). b search text dear * SEARCH 111 187 189 237 266 293 294 339 842 853 882 883 929 946 1346 1348 1349 1433 1434 1435 1474 1475 1483 1500 1504 1509 1512 1514 1639 1642 1766 2364 2366 2367 2635 2638 2639 2640 2642 2644 2664 2738 2750 2830 2849 2857 3264 3265 3266 3267 3268 3271 3274 3301 3302 3303 3305 3486 3489 3494 3499 3500 3752 3899 4013 4135 4136 4209 4421 4460 4464 4465 4469 4477 4478 4479 4480 4482 4483 4522 4535 4536 4538 4751 4769 5168 5169 5178 5179 5219 5254 5257 5300 5301 5304 5333 5336 5363 5364 5384 5406 5476 5479 5482 5493 5494 5499 5507 5516 5521 5768 6241 6243 7470 7471 7861 7862 7864 7874 7875 7876 7877 7879 8389 8399 8428 8429 8907 8937 8938 8940 9469 9495 9610 9688 9735 9881 9882 9883 9885 9913 9914 9915 9996 10009 10091 10094 10218 10229 10230 10332 10335 10340 10341 10517 10610 10612 10826 10829 10830 11081 11082 11087 11089 11097 11144 11145 11212 11213 11359 11365 11366 11367 11369 11375 11600 11673 11710 11941 11942 11946 11948 12046 12205 12208 12209 12407 12409 12411 13104 13551 13589 13806 13815 13818 13881 13883 13885 13980 14453 14578 14617 14618 14620 14666 14667 14668 14677 14685 14783 14790 14964 14966 14967 14972 15050 15436 15476 15949 15959 15964 15965 15971 16106 16110 16283 16284 16293 16310 16318 16331 16372 16373 16374 16785 16786 16787 16985 16987 17050 17855 17857 17871 17875 17876 17877 17959 17978 17979 18044 18807 18814 18821 18822 18823 18917 18918 18936 19005 19121 19291 19292 20058 20372 20467 20468 20469 20475 20489 20510 20516 20521 20522 20523 20524 20541 20560 20562 20582 20584 20653 20654 20904 21532 21595 21596 21716 21717 21719 21724 21725 * 7 RECENT b OK Search completed (4.468 secs). -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From joseba.torre at ehu.es Tue Jan 27 09:06:48 2015 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 27 Jan 2015 10:06:48 +0100 Subject: dovecot migration In-Reply-To: <54C6FE01.3040500@redhat.com> References: <54C6FE01.3040500@redhat.com> Message-ID: <54C75528.8010608@ehu.es> El 27/01/15 a las 03:54, mkawada at redhat.com escribi?: > Dear List, > > To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I > wonder if the UIDL format "pop3_uidl_format" setting is required in the > configuration file for clients not to re-download the same mails from > the migrated mail server. Currently, no UIDL format is set in > /etc/dovecot file of the one using dovecot-1.x server which means it is > commented out as default like below. > > ]# grep pop3_uidl_format /etc/dovecot.conf > > #pop3_uidl_format = %08Xu%08Xv <---- > > Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by > default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one > using dovecot-2.x, and have not been able to find any information saying > it's the defualt setting. > > Here are the questions: > > 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default > setting? Or any ways to check the default? You can get the current configuration with dovecot -a In order to get only diferences from the defaults, dovecot -n. > > 2.Do they need to specify anything for the "pop3_uidl_format =" field > for their migration in this case? Better safe than sorry: get the current value and add the pop3_uidl_format line in your conf. > > Env infomation: > dovecot-1.x > dovecot-2.x > mail_location = mbox > > Any information/clues would be greatly appreciated. > > Thanks, > From tm at del.bg Tue Jan 27 09:22:33 2015 From: tm at del.bg (Teodor Milkov) Date: Tue, 27 Jan 2015 11:22:33 +0200 Subject: How to quote the special wildcard characters '*' and '?' In-Reply-To: <54B7C95E.2080702@icdsoft.com> References: <54B7C95E.2080702@icdsoft.com> Message-ID: <54C758D9.5070807@del.bg> On 15/01/15 11:17, David Davidov wrote: > Hello, > > I am searching for a way to execute doveadm command against mailbox > containing a special wildcard character in its name. For example: > > # doveadm expunge -u 'me?@example.com' mailbox 'INBOX.Junk Mail' > SAVEDBEFORE 7 > > The result is: > doveadm(root): Error: User listing returned failure > doveadm: Error: Failed to iterate through some users > > with exit code 75 > > In the mail log: > dovecot: auth: Error: Trying to iterate users, but userdbs don't support it > > To be clear: the question mark is a part of the mailbox's name. Not an > attempt to use it as wildcard. > > Is there a way > 1. To quote the special characters > or > 2. To disable wildcard matcing in the -u parameter Hello, I'm struggling with the same problem. Is there a way to escape wildcard characters or dovecot must be patched? Best regards, Teodor From superinterstellar at gmail.com Tue Jan 27 10:15:02 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Tue, 27 Jan 2015 18:15:02 +0800 Subject: Indexing Mail faster In-Reply-To: <20150127080221.GA38114@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> Message-ID: Dear Thomas, Below is the output of my 'dovecot -n' . The slow speed is really bad, imagine waiting 4 mins for every search. . Something is terribly wrong in my config. Please have a look and advise. Thanks Kevin # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } first_valid_uid = 2000 last_valid_uid = 2000 listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 2000 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = quota fts mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users-password driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl auth_socket_path = /var/run/dovecot/auth-master quota = dict:user::proxy::quotadict quota_rule = *:storage=1G quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = /%Lh/sieve/dovecot.sieve sieve_default = /var/vmail/sieve/dovecot.sieve sieve_dir = /%Lh/sieve sieve_global_dir = /var/vmail/sieve } protocols = pop3 imap sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl = required ssl_cert = wrote: > * Kevin Laurie 2015.01.27 03:24: > > > . search text dear > > * OK Searched 9% of the mailbox, ETA 1:39 > > * OK Searched 20% of the mailbox, ETA 1:18 > > * OK Searched 29% of the mailbox, ETA 1:12 > > * OK Searched 38% of the mailbox, ETA 1:03 > > * OK Searched 50% of the mailbox, ETA 0:49 > > * OK Searched 60% of the mailbox, ETA 0:39 > > * OK Searched 69% of the mailbox, ETA 0:30 > > * OK Searched 79% of the mailbox, ETA 0:20 > > I guess we need to start looking at your version and the > format/mail_location configuration, best to post your 'doveconf -n'. > > Emulating the same search on my box looks a lot better: > > $ /usr/lib/dovecot/imap -u tlx at leuxner.net > * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE > IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN > CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE NOTIFY QUOTA ACL > RIGHTS=texk] Logged in as tlx at leuxner.net > a select "Virtual/Postfix-Users" > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Old) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Old \*)] > Flags permitted. > * 22120 EXISTS > * 0 RECENT > * OK [UNSEEN 22109] First unseen. > * OK [UIDVALIDITY 1420123174] UIDs valid > * OK [UIDNEXT 44062] Predicted next UID > * OK [HIGHESTMODSEQ 702] Highest > a OK [READ-WRITE] Select completed (0.031 secs). > b search text dear > * SEARCH 111 187 189 237 266 293 294 339 842 853 882 883 929 946 1346 1348 > 1349 1433 1434 1435 1474 1475 1483 1500 1504 1509 1512 1514 1639 1642 1766 > 2364 2366 2367 2635 2638 2639 2640 2642 2644 2664 2738 2750 2830 2849 2857 > 3264 3265 3266 3267 3268 3271 3274 3301 3302 3303 3305 3486 3489 3494 3499 > 3500 3752 3899 4013 4135 4136 4209 4421 4460 4464 4465 4469 4477 4478 4479 > 4480 4482 4483 4522 4535 4536 4538 4751 4769 5168 5169 5178 5179 5219 5254 > 5257 5300 5301 5304 5333 5336 5363 5364 5384 5406 5476 5479 5482 5493 5494 > 5499 5507 5516 5521 5768 6241 6243 7470 7471 7861 7862 7864 7874 7875 7876 > 7877 7879 8389 8399 8428 8429 8907 8937 8938 8940 9469 9495 9610 9688 9735 > 9881 9882 9883 9885 9913 9914 9915 9996 10009 10091 10094 10218 10229 10230 > 10332 10335 10340 10341 10517 10610 10612 10826 10829 10830 11081 11082 > 11087 11089 11097 11144 11145 11212 11213 11359 11365 11366 11367 11369 > 11375 11600 11673 11710 11941 11942 11946 11948 12046 12205 12208 12209 > 12407 12409 12411 13104 13551 13589 13806 13815 13818 13881 13883 13885 > 13980 14453 14578 14617 14618 14620 14666 14667 14668 14677 14685 14783 > 14790 14964 14966 14967 14972 15050 15436 15476 15949 15959 15964 15965 > 15971 16106 16110 16283 16284 16293 16310 16318 16331 16372 16373 16374 > 16785 16786 16787 16985 16987 17050 17855 17857 17871 17875 17876 17877 > 17959 17978 17979 18044 18807 18814 18821 18822 18823 18917 18918 18936 > 19005 19121 19291 19292 20058 20372 20467 20468 20469 20475 20489 20510 > 20516 20521 20522 20523 20524 20541 20560 20562 20582 20584 20653 20654 > 20904 21532 21595 21596 21716 21717 21719 21724 21725 > * 7 RECENT > b OK Search completed (4.468 secs). > From tlx at leuxner.net Tue Jan 27 10:47:55 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 27 Jan 2015 11:47:55 +0100 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> Message-ID: <20150127104755.GB38114@nihlus.leuxner.net> * Kevin Laurie 2015.01.27 11:15: Hi Kevin, > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS Although this was released back in 2013, let's assume the core search is not substantially broken for this release. > mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ > mail_plugins = quota fts As the mail storage looks okay, could you try with the parameter 'fts' removed? We would then have the search hit a vanilla Dovecot without plugins involved... Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From skdovecot at smail.inf.fh-brs.de Tue Jan 27 13:46:22 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 27 Jan 2015 14:46:22 +0100 (CET) Subject: dovecot migration In-Reply-To: <54C6FE01.3040500@redhat.com> References: <54C6FE01.3040500@redhat.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 27 Jan 2015, mkawada at redhat.com wrote: > To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I > wonder if the UIDL format "pop3_uidl_format" setting is required in the > configuration file for clients not to re-download the same mails from > the migrated mail server. Currently, no UIDL format is set in > /etc/dovecot file of the one using dovecot-1.x server which means it is > commented out as default like below. > > ]# grep pop3_uidl_format /etc/dovecot.conf > > #pop3_uidl_format = %08Xu%08Xv <---- > > Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by > default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one > using dovecot-2.x, and have not been able to find any information saying > it's the defualt setting. > > Here are the questions: > > 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default > setting? Or any ways to check the default? It is the default, you can verify it by doveconf -a (Note the -a, not -n) > 2.Do they need to specify anything for the "pop3_uidl_format =" field > for their migration in this case? No, usually not. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMeWrnz1H7kL/d9rAQJvXAf/Y+qpqEpCwqXCYJyhm5SvtC/XazZXL/rg XBjIEF/TGryk62rN6W6F9e4JbrLgZY90/OwmzGyFoUq4qOwsUVxgI5lQaPrTqIGO MeMJieen5kwrLfaaVkbFhz069n58c6cp5BmFqJVp8CTF7e3cQ/1KM4qTntzDJGB6 uFmZSOTwVgZshbTlC5KvYEdMskhSg+lCEmo8FbpRGxQoMGaHOGz40KECe9DbKxHJ ieTPtbDIZJlBIYZ1mc2olvFwtRFGT5ymFir3RB16GG8zVQZnN3Cn1LIrIJ6LwiIU fRsww2BfNxH8cPZtqq9eOouRqNjPQJMH0VsaSQPF/lPG1DVCysqUiQ== =yx0J -----END PGP SIGNATURE----- From alvin.sm at Mail.Linux-Consulting.com Tue Jan 27 19:29:18 2015 From: alvin.sm at Mail.Linux-Consulting.com (alvin) Date: Tue, 27 Jan 2015 11:29:18 -0800 (PST) Subject: sendmail not invoking dovecot-lda In-Reply-To: Message-ID: <201501271929.t0RJTJDH009421@Mail.Linux-Consulting.com> hi steffen - thanx for your reply sendmail-8.15.1 + dovecot-2.2.15 + OpenSSL is working .. my sendmail is also configured for smtp-auth and saslauth and seems to work with dovecot one other thing to note, sendmail requires CA signed *.pem certs vs self-signed ... in my case, dovecot uses the same *.pem certs http://www.sendmail.org/~ca/email/other/cagreg.html # # ================ # # testing dovecot to fetch the emails # # ================ # # dovecot.user is only defined in /etc/dovecot/passwd # # echo "test mail" | mail -v dovecot.user at vmail.example.com # echo "test mutt" | mutt -s mutt dovecot.user at vmail.example.com # # echo "sendmail" | sendmail -d60.5 -d27.2 -bv dovecot.user at vmail.example.com # # telnet vmail.example.com 110 # openssl s_client -connect vmail.example.com:995 # pop3s # user dovecot.user # pass passwd # list # retr 1 # quit # # telnet vmail.example.com 143 # openssl s_client -connect vmail.example.com:993 # imaps # a1 login dovecot.user passwd # a2 LIST "" "*" # a3 select INBOX # a4 FETCH 1 BODY[] # a5 logout # # # replace gmail.com with your ssl-enabled dovecot server # openssl s_client -connect pop.gmail.com:995 -showcerts # openssl s_client -connect imap.gmail.com:993 -showcerts # # # replace gmail.com with your SASL enabled STMPT server # openssl s_client -connect smtp.gmail.com:587 -starttls smtp # > On Sun, 18 Jan 2015, alvin wrote: > > > > ------------------------------------------------------------------------------ > > - status of what works and what does NOT work while testing dovecot's LDA ... > > ------------------------------------------------------------------------------ > > # > > # fyi.. recepient user is in the form of user, user at localhost and user at domain.com > > # > > sendmail -bv user.in/etc/passwd = deliverable > > sendmail -bv user.in/etc/mail/virtusertable = deliverable to sendmail virtual user > > > > -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown > > 1) is the virtual domain a local name? > http://compgroups.net/comp.mail.sendmail/sendmail-dovecot-mailertable-and-passwd-li/1316451 > http://www.dovecot.org/list/dovecot/2010-June/049500.html it turns out, the hostname must NOT be listed in /etc/mail/local-host-names and that it still must resolve ( /etc/hosts ) > 2) your mailer is named "dovecot", therefore the mailertable entry in the > Wiki: > > virtualdomain.example.com vmail:vmail and again, virtualdomain -- should NOT be listed in /etc/mail/local-host-names - when i removed it from local-host-names, things started be more debuggable and fixed yup > virtualdomain.example.com dovecot:dovecot > (2nd dovecot has no meaning, but there must be present something) yup > 3) Post a sensable output: > echo '3,0 dovecotuser at dovecot.domain' | sendmail -bt -d21.4 it's working ... other sendmail tests ... # sendmail -d60.5 -d27.2 -bv dovecot.user # dovecot.user... User unknown # # sendmail -d60.5 -d27.2 -bv dovecot.user at fake # dovecot.yser at fake... deliverable: mailer esmtp, host fake, user dovecot.plain at fake # # sendmail -d60.5 -d27.2 -bv dovecot.user # # -d60.5 should trace map lookups (including mailertable) # # -d27.2 should trace alias expansion # # echo '3,0 dovecot.user at localhost ' | sendmail -bt -d21.12 -d60.5 # # -d60.5 tracking maps (virtusertable) lookups # # -d21.12 tracking processing of R lines in sendmail.cf # # sendmail -d27.2 -bv dovecot.user # # -d27.2 tracking processing of aliases # # sendmail -Am -bv dovecot.user at vmail < /dev/null # echo '3,0 dovecot.user at vmail.example.com' | sendmail -Am -bv dovecot.user at vmail.example.com # dovecot.user at vmail.example.com.. deliverable: mailer dovecot, host vmail, user dovecot.user at vmail.example.com > 4) there had been a post about trying to improve virtual user support in > sendmail incl. some ongoing talk. Maybe you find it and get some info back > into this list. few posts showed another sendmail file that needs to be configured /etc/mail/access # vmail must be defined in /etc/hosts ... NOT listed in local-host-named To: vmail.example.com RELAY > > dovecot user dovecot.user == returns UID/GID/home/mailbox ( deliverable ) these are good tests to show dovecot is able to deliver its emails in my case, dovecot resolves users in /etc/passwd, /etc/mail/virtusertable, /etc/dovecot/password and hopefully, it will also resolve /usr/local/mysql database users > > # -------------------------------------------------------------------- > > # dovecot's LDA delivers the test email to the dovecot/mysql recepient > > # -------------------------------------------------------------------- > > cat test-email-with-headers.txt | dovecot-lda -d dovecot.user ( email delivered ) these are good tests to show dovecot is able to deliver its emails > > # ---------------------------------------------------------------- > > -->> # mail/mutt/sendmail canNOT deliver to dovecot recepient == FAILS > > # ---------------------------------------------------------------- > > -->> sendmail -bv dovecot.user.in/dovecot/virtual.passwd.file == user unknown > > 3 > > echo "testing mail to dovecot" | mail -v -s "testing mail to dovecot" doveccot.user > > == user unknown == "user unknown" comes from misconfigured since sendmail is still confused - /etc/mail/mailertable - /etc/mail/local-host-names - /etc/mail/access my final sendmail config .... only import stuff shown .. > > ------------------------------------------------------- > > - my (relevant to dovecot ) sendmail.mc file looks like: > > ------------------------------------------------------- > > > > dnl # > > dnl # local mail delivery > > dnl # > > define(`PROCMAIL_MAILER_PATH', `/usr/local/bin/procmail')dnl > > > > FEATURE(local_procmail,`',`procmail -t -Y -a $h -d $u')dnl REMOVE define/feature to do with procmail ----------------------------------------- > > dnl # > > dnl # I'm trying to get sendmail to use dovecot.m4 to deliver local mail > > dnl # to dovecot's LDA ( dovecot-lda ) for mysql virtual users > > dnl # > > dnl uncomment and use either feature(...dovecot-lda) or mailer(dovecot.m4) > > dnl > > dnl FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u') use Mailer(dovecot) instead .... Feature and Mailer is mutually exclusive in this case > > MAILER(local) > > MAILER(smtp) > > dnl > > dnl # trying to use procmail to deliver local mail to system users ( /etc/passwd ) > > MAILER(procmail) > > dnl > > dnl > > MAILER(dovecot)dnl # see below > > dnl > > dnl End of File > > > > ------------------------------------------------------- > > - my (relevant to dovecot ) sendmail.cf file looks like: > > ------------------------------------------------------- take out procmail stuff > > ##################################### > > ### SMTP Mailer specification ### > > ##################################### > > > > # > > # lots of deleted ... ??not?? important until procmail stuff ---> MAILER(procmail) > > # > > > > ######################*****############## > > ### PROCMAIL Mailer specification ### > > ##################*****################## > > ##### $Id: procmail.m4,v 8.23 2013-11-22 20:51:14 ca Exp $ ##### > > > > Mprocmail, P=/usr/local/bin/procmail, F=DFMSPhnu9, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFr > > omSMTP, > > T=DNS/RFC822/X-Unix, > > A=procmail -Y -m $h $f $u > > > > ######################*****######################### > > ### ### > > ### 10-Jan-15 amo Dovecot.m4 ### > > ### ### > > ### dovecot.m4 stuff ### > > ### ### > > ##############################*****################# > > Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, > > S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, > > T=DNS/RFC822/X-Unix, > > A=/usr/local/libexec/dovecot/dovecot-lda -d $u note i'm using "S=EnvFromL/HdrFromL" NOT "S=EnvFromSMTP/HdrFromSMTP" and similarly for R= > > # > > # End of sendmail.cf > > http://wiki2.dovecot.org/LDA/Sendmail > > > > ######################################### > > ### DOVECOT Mailer specification ### > > ######################################### > > Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=DFMPhnu9, > > S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP, should wiki2.dovecot.org be updated to show S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, > > T=DNS/RFC822/X-Unix, > > A=/usr/local/libexec/dovecot/dovecot-lda -d $u for debugging purproses .... it is very helpful to use a wrapper script ( wraplda.sh ) around dovecot-lda to get UID/GID info of who is calling dovecot-lda http://dovecot.org/pipermail/dovecot/2013-January/087658.html > > http://wiki2.dovecot.org/LDA/Sendmail > > also states that one could use these 3 statements in lieu of dovecot.m4 > > > > # > > # this replaces sendmail.cf Mlocal P=mail.local with P=dovecot-lda > > # > > FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda', \ > > `/usr/local/libexec/dovecot/dovecot-lda -d $u') > > dnl MODIFY_MAILER_FLAGS(`LOCAL', `-f') > > MAILER(procmail) didn't work for me > > ------------------------------------ > > - my relevant dovecot config options > > ------------------------------------ > > # > > # are there any other major options i'm missing in the dovecot config files > > # > > doveconf > > dovecot -a > > # > > # for /etc/passwd > > passdb { > > args = > > driver = pam > > .. > > } > > # > > # ------------------------------------------------------------------------ > > # need to get these dovecot's virtual users to be able send/receive emails > > # ------------------------------------------------------------------------ > > # sendmail -bv dovecot.VirtualUser at localhost > > # dovecot.VirtuaUser at localhost... User unknown > > # > > passdb { > > args = /etc/dovecot/passwd > > # > > # dovecot.VirtualUser:{PLAIN}password::::: > > # > > # end of file > > driver = passwd-file > > .. > > } > > # > > passdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > .. > > } > > ... > > ... > > ... > > userdb { > > args = > > driver = passwd > > .. > > } > > # > > userdb { > > args = /etc/dovecot/password > > driver = passwd-file > > .. > > } > > # > > userdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > .. > > } ..... > > # end of txt > > > > - -- > Steffen Kaiser From superinterstellar at gmail.com Wed Jan 28 02:11:52 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Wed, 28 Jan 2015 10:11:52 +0800 Subject: Indexing Mail faster In-Reply-To: <20150127104755.GB38114@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> Message-ID: Dear Thomas, I have removed the fts plugin. See below:- # dovecot -n # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN dict { acl = mysql:/etc/dovecot/dovecot-share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } first_valid_uid = 2000 last_valid_uid = 2000 listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 2000 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = quota mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users-password driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl auth_socket_path = /var/run/dovecot/auth-master quota = dict:user::proxy::quotadict quota_rule = *:storage=1G quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = /%Lh/sieve/dovecot.sieve sieve_default = /var/vmail/sieve/dovecot.sieve sieve_dir = /%Lh/sieve sieve_global_dir = /var/vmail/sieve } protocols = pop3 imap sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl = required ssl_cert = wrote: > * Kevin Laurie 2015.01.27 11:15: > > Hi Kevin, > > > # 2.2.9: /etc/dovecot/dovecot.conf > > # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS > > Although this was released back in 2013, let's assume the core search is > not substantially broken for this release. > > > mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ > > mail_plugins = quota fts > > As the mail storage looks okay, could you try with the parameter 'fts' > removed? We would then have the search hit a vanilla Dovecot without > plugins involved... > > Regards > Thomas > From superinterstellar at gmail.com Wed Jan 28 02:31:22 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Wed, 28 Jan 2015 10:31:22 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> Message-ID: Also tried to reload the config but still getting slow search. Just tried to reindex the box by running dovecot index -u user at email.net inbox I am still getting searches close to 2:45 minutes. It seems strange. A smaller inbox took less time:- . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA] Logged in . search text hello . BAD No mailbox selected. . select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 4825 EXISTS * 66 RECENT * OK [UNSEEN 3667] First unseen. * OK [UIDVALIDITY 1414486440] UIDs valid * OK [UIDNEXT 4826] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed (0.068 secs). . search text hello * OK Searched 50% of the mailbox, ETA 0:09 * OK Searched 95% of the mailbox, ETA 0:00 * SEARCH 20 21 22 23 24 32 39 40 41 42 45 48 49 50 59 60 270 373 428 576 585 624 749 756 770 776 798 829 849 917 927 928 934 971 981 983 984 1049 1062 1084 1095 1108 1186 1202 1203 1204 1212 1214 1278 1320 1345 1364 1365 1366 1367 1368 1384 1431 1456 1463 1480 1484 1485 1486 1507 1510 1525 1549 1550 1567 1574 1607 1608 1612 1613 1649 1701 1705 1711 1722 1765 1777 1826 1846 1856 1885 1889 1895 1983 1985 1986 2020 2022 2025 2064 2171 2175 2176 2177 2190 2224 2273 2274 2287 2299 2300 2313 2329 2338 2433 2434 2437 2440 2499 2574 2590 2595 2617 2618 2624 2625 2626 2627 2650 2652 2657 2701 2735 2785 2791 2792 2858 2872 2873 2875 2878 2929 3196 3242 3279 3325 3326 3327 3328 3345 3406 3408 3455 3456 3655 3679 3722 3723 3731 3792 3798 3812 3841 4085 4137 4192 4379 4708 . OK Search completed (21.478 secs). On Wed, Jan 28, 2015 at 10:11 AM, Kevin Laurie wrote: > Dear Thomas, > > I have removed the fts plugin. > See below:- > > # dovecot -n > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS > auth_master_user_separator = * > auth_mechanisms = PLAIN LOGIN > dict { > acl = mysql:/etc/dovecot/dovecot-share-folder.conf > quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf > } > first_valid_uid = 2000 > last_valid_uid = 2000 > listen = * > log_path = /var/log/dovecot.log > mail_debug = yes > mail_gid = 2000 > mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ > mail_plugins = quota > mail_uid = 2000 > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character > vacation subaddress comparator-i;ascii-numeric relational regex imap4flags > copy include variables body enotify environment mailbox date ihave > namespace { > inbox = yes > location = > mailbox Drafts { > auto = subscribe > special_use = \Drafts > } > mailbox Junk { > auto = subscribe > special_use = \Junk > } > mailbox Sent { > auto = subscribe > special_use = \Sent > } > mailbox "Sent Messages" { > auto = no > special_use = \Sent > } > mailbox Spam { > auto = no > special_use = \Junk > } > mailbox Trash { > auto = subscribe > special_use = \Trash > } > prefix = > separator = / > type = private > } > namespace { > list = children > location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u > prefix = Shared/%%u/ > separator = / > subscriptions = yes > type = shared > } > passdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > passdb { > args = /etc/dovecot/dovecot-master-users-password > driver = passwd-file > master = yes > } > plugin { > acl = vfile > acl_shared_dict = proxy::acl > auth_socket_path = /var/run/dovecot/auth-master > quota = dict:user::proxy::quotadict > quota_rule = *:storage=1G > quota_warning = storage=85%% quota-warning 85 %u > quota_warning2 = storage=90%% quota-warning 90 %u > quota_warning3 = storage=95%% quota-warning 95 %u > sieve = /%Lh/sieve/dovecot.sieve > sieve_default = /var/vmail/sieve/dovecot.sieve > sieve_dir = /%Lh/sieve > sieve_global_dir = /var/vmail/sieve > } > protocols = pop3 imap sieve lmtp > service auth { > unix_listener /var/spool/postfix/private/dovecot-auth { > group = postfix > mode = 0666 > user = postfix > } > unix_listener auth-master { > group = vmail > mode = 0666 > user = vmail > } > unix_listener auth-userdb { > group = vmail > mode = 0660 > user = vmail > } > } > service dict { > unix_listener dict { > group = vmail > mode = 0660 > user = vmail > } > } > service imap-login { > process_limit = 500 > service_count = 1 > } > service lmtp { > executable = lmtp -L > inet_listener lmtp { > port = 24 > } > process_min_avail = 5 > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > user = vmail > } > service pop3-login { > service_count = 1 > } > service quota-warning { > executable = script /usr/local/bin/dovecot-quota-warning.sh > unix_listener quota-warning { > group = vmail > mode = 0660 > user = vmail > } > } > ssl = required > ssl_cert = ssl_key = userdb { > args = /etc/dovecot/dovecot-mysql.conf > driver = sql > } > protocol lda { > auth_socket_path = /var/run/dovecot/auth-master > lda_mailbox_autocreate = yes > log_path = /var/log/dovecot-sieve.log > mail_plugins = quota sieve > postmaster_address = root > } > protocol lmtp { > info_log_path = /var/log/dovecot-lmtp.log > lmtp_save_to_detail_mailbox = yes > mail_plugins = quota sieve > postmaster_address = postmaster > recipient_delimiter = + > } > protocol imap { > imap_client_workarounds = tb-extra-mailbox-sep > mail_plugins = quota imap_quota > } > protocol pop3 { > mail_plugins = quota > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > pop3_uidl_format = %08Xu%08Xv > } > > > > > > > > The search is still the same even after removal. Do you think I have a > broken search dovecot? > > . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE > SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT > MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS > LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN > CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE QUOTA] Logged in > . select inbox > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded > \*)] Flags permitted. > * 49255 EXISTS > * 30 RECENT > * OK [UNSEEN 46791] First unseen. > * OK [UIDVALIDITY 1414214135] UIDs valid > * OK [UIDNEXT 106490] Predicted next UID > * OK [NOMODSEQ] No permanent modsequences > . OK [READ-WRITE] Select completed (0.110 secs). > . search dear > . BAD Error in IMAP command SEARCH: Unknown argument DEAR > . search text hello > * OK Searched 8% of the mailbox, ETA 1:53 > * > > On Tue, Jan 27, 2015 at 6:47 PM, Thomas Leuxner wrote: > >> * Kevin Laurie 2015.01.27 11:15: >> >> Hi Kevin, >> >> > # 2.2.9: /etc/dovecot/dovecot.conf >> > # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS >> >> Although this was released back in 2013, let's assume the core search is >> not substantially broken for this release. >> >> > mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ >> > mail_plugins = quota fts >> >> As the mail storage looks okay, could you try with the parameter 'fts' >> removed? We would then have the search hit a vanilla Dovecot without >> plugins involved... >> >> Regards >> Thomas >> > > From mkawada at redhat.com Wed Jan 28 05:23:34 2015 From: mkawada at redhat.com (mkawada at redhat.com) Date: Wed, 28 Jan 2015 14:23:34 +0900 Subject: dovecot migration In-Reply-To: <54C75528.8010608@ehu.es> References: <54C6FE01.3040500@redhat.com> <54C75528.8010608@ehu.es> Message-ID: <54C87256.7040602@redhat.com> Hi, Joseba-san, I very much appreciate your help! Thanks, (2015?01?27? 18:06), Joseba Torre wrote: > El 27/01/15 a las 03:54, mkawada at redhat.com escribi?: >> Dear List, >> >> To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I >> wonder if the UIDL format "pop3_uidl_format" setting is required in the >> configuration file for clients not to re-download the same mails from >> the migrated mail server. Currently, no UIDL format is set in >> /etc/dovecot file of the one using dovecot-1.x server which means it is >> commented out as default like below. >> >> ]# grep pop3_uidl_format /etc/dovecot.conf >> >> #pop3_uidl_format = %08Xu%08Xv <---- >> >> Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by >> default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one >> using dovecot-2.x, and have not been able to find any information saying >> it's the defualt setting. >> >> Here are the questions: >> >> 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default >> setting? Or any ways to check the default? > > You can get the current configuration with > > dovecot -a > > In order to get only diferences from the defaults, dovecot -n. > >> >> 2.Do they need to specify anything for the "pop3_uidl_format =" field >> for their migration in this case? > > Better safe than sorry: get the current value and add the > pop3_uidl_format line in your conf. >> >> Env infomation: >> dovecot-1.x >> dovecot-2.x >> mail_location = mbox >> >> Any information/clues would be greatly appreciated. >> >> Thanks, >> From mkawada at redhat.com Wed Jan 28 05:24:37 2015 From: mkawada at redhat.com (mkawada at redhat.com) Date: Wed, 28 Jan 2015 14:24:37 +0900 Subject: dovecot migration In-Reply-To: References: <54C6FE01.3040500@redhat.com> Message-ID: <54C87295.9090708@redhat.com> Hi, Steffen-san, I very much appreciate your help! Thanks, (2015?01?27? 22:46), Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 27 Jan 2015, mkawada at redhat.com wrote: > >> To migrate a dovecot server from dovecot-1.x to dovecot-2.x, then I >> wonder if the UIDL format "pop3_uidl_format" setting is required in the >> configuration file for clients not to re-download the same mails from >> the migrated mail server. Currently, no UIDL format is set in >> /etc/dovecot file of the one using dovecot-1.x server which means it is >> commented out as default like below. >> >> ]# grep pop3_uidl_format /etc/dovecot.conf >> >> #pop3_uidl_format = %08Xu%08Xv <---- >> >> Does this mean the "pop3_uidl_format" is in use as "%08Xu%08Xv" by >> default? I see the same in /etc/dovecot/conf.d/20-imap.conf of the one >> using dovecot-2.x, and have not been able to find any information saying >> it's the defualt setting. >> >> Here are the questions: >> >> 1.Can we say the "pop3_uidl_format = %08Xu%08Xv" is the enabled default >> setting? Or any ways to check the default? > > It is the default, you can verify it by > > doveconf -a > > (Note the -a, not -n) > >> 2.Do they need to specify anything for the "pop3_uidl_format =" field >> for their migration in this case? > > No, usually not. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVMeWrnz1H7kL/d9rAQJvXAf/Y+qpqEpCwqXCYJyhm5SvtC/XazZXL/rg > XBjIEF/TGryk62rN6W6F9e4JbrLgZY90/OwmzGyFoUq4qOwsUVxgI5lQaPrTqIGO > MeMJieen5kwrLfaaVkbFhz069n58c6cp5BmFqJVp8CTF7e3cQ/1KM4qTntzDJGB6 > uFmZSOTwVgZshbTlC5KvYEdMskhSg+lCEmo8FbpRGxQoMGaHOGz40KECe9DbKxHJ > ieTPtbDIZJlBIYZ1mc2olvFwtRFGT5ymFir3RB16GG8zVQZnN3Cn1LIrIJ6LwiIU > fRsww2BfNxH8cPZtqq9eOouRqNjPQJMH0VsaSQPF/lPG1DVCysqUiQ== > =yx0J > -----END PGP SIGNATURE----- From christian.binder at freilassing.de Wed Jan 28 08:10:54 2015 From: christian.binder at freilassing.de (Christian Binder Stadt Freilassing) Date: Wed, 28 Jan 2015 09:10:54 +0100 Subject: Uniqueness of dovecot mailbox-guids Message-ID: <7f87f3d9e2ca521b91d4f96d7c31051d@egroupware.freilassing.de> Dovecot version: 2.2.9 I'm using doveadm backup to migrate about hundred user mailboxes from cyrus to dovecot. I defined a master user for the old cyrus mailserver in imapc backend to get access to all the mailboxes. Now I realized that after migration every user's "Inbox" or "Trash" on the new dovecot server has the same mailbox-guid, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox dd9456... trash 5549cc... Is this a wanted behaviour? If i do not migrate with doveadm backup and just manually create mailboxes, the mailbox-guids are globally unique, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox c543fe... trash t49ccc... Thanks! Christian From tlx at leuxner.net Wed Jan 28 08:45:02 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 28 Jan 2015 09:45:02 +0100 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> Message-ID: <20150128084502.GA7518@nihlus.leuxner.net> * Kevin Laurie 2015.01.28 03:31: > . OK Search completed (21.478 secs). Really hard to say what commit probably fixed that already. Dovecot uses a very dynamic development model. There are old versions like 2.0.9 which people still try to get help with since they are in some dusted distro repository, but known to have partially broken features... Looking at the CVS of Dovecot there are so many changes it's really hard to say there was an issue or not. http://hg.dovecot.org/dovecot-2.2/shortlog One changelog (2.2.11) however indicates the search was not working as it should: - IMAP: SEARCH/SORT PARTIAL was handled completely wrong in v2.2.11+ You may want to look at the official Enterprise repository which does support Ubuntu. This one does have the current stable releases in it and is not undergoing so many changes compared to the CVS auto-compiles also offered: http://shop.dovecot.fi/home/8-dovecot-ee-repository-access.html Give that one a try with vanilla search... -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From superinterstellar at gmail.com Wed Jan 28 10:53:56 2015 From: superinterstellar at gmail.com (superinterstellar at gmail.com) Date: Wed, 28 Jan 2015 10:53:56 +0000 Subject: Indexing Mail faster In-Reply-To: <20150128084502.GA7518@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> Message-ID: <20150128105356.6004882.88854.347@gmail.com> ?Dear Mr.Thomas, Thanks for your feedback. Will try dovecot-ee.? Also could you advise which dovecot are you using? Thanks Kevin ? Original Message ? From: Thomas Leuxner Sent: Wednesday, 28 January 2015 08:45 To: dovecot at dovecot.org Reply To: dovecot at dovecot.org Subject: Re: Indexing Mail faster * Kevin Laurie 2015.01.28 03:31: > . OK Search completed (21.478 secs). Really hard to say what commit probably fixed that already. Dovecot uses a very dynamic development model. There are old versions like 2.0.9 which people still try to get help with since they are in some dusted distro repository, but known to have partially broken features... Looking at the CVS of Dovecot there are so many changes it's really hard to say there was an issue or not. http://hg.dovecot.org/dovecot-2.2/shortlog One changelog (2.2.11) however indicates the search was not working as it should: - IMAP: SEARCH/SORT PARTIAL was handled completely wrong in v2.2.11+ You may want to look at the official Enterprise repository which does support Ubuntu. This one does have the current stable releases in it and is not undergoing so many changes compared to the CVS auto-compiles also offered: http://shop.dovecot.fi/home/8-dovecot-ee-repository-access.html Give that one a try with vanilla search... From tlx at leuxner.net Wed Jan 28 10:59:40 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 28 Jan 2015 11:59:40 +0100 Subject: Indexing Mail faster In-Reply-To: <20150128105356.6004882.88854.347@gmail.com> References: <20150124190408.GA436@nihlus.leuxner.net> <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> Message-ID: <20150128105939.GB7518@nihlus.leuxner.net> * superinterstellar at gmail.com 2015.01.28 11:53: > Also could you advise which dovecot are you using? Bleeding edge (latest and hopefully greatest compiled from HG): # 2.2.15 (7557234ac0f4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From superinterstellar at gmail.com Wed Jan 28 11:07:36 2015 From: superinterstellar at gmail.com (superinterstellar at gmail.com) Date: Wed, 28 Jan 2015 11:07:36 +0000 Subject: Indexing Mail faster In-Reply-To: <20150128105939.GB7518@nihlus.leuxner.net> References: <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> <20150128105939.GB7518@nihlus.leuxner.net> Message-ID: <20150128110736.6004882.8269.350@gmail.com> Dear Thomas, Do you think I should try 2.2.15? Or stick to EE? Tempted to try the latest dovecot. Thanks Kevin ? Original Message ? From: Thomas Leuxner Sent: Wednesday, 28 January 2015 11:00 To: dovecot at dovecot.org Reply To: dovecot at dovecot.org Subject: Re: Indexing Mail faster * superinterstellar at gmail.com 2015.01.28 11:53: > Also could you advise which dovecot are you using? Bleeding edge (latest and hopefully greatest compiled from HG): # 2.2.15 (7557234ac0f4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 From tlx at leuxner.net Wed Jan 28 13:16:28 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 28 Jan 2015 14:16:28 +0100 Subject: Indexing Mail faster In-Reply-To: <20150128110736.6004882.8269.350@gmail.com> References: <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> <20150128105939.GB7518@nihlus.leuxner.net> <20150128110736.6004882.8269.350@gmail.com> Message-ID: <20150128131628.GE7518@nihlus.leuxner.net> * superinterstellar at gmail.com 2015.01.28 12:07: > Do you think I should try 2.2.15? Or stick to EE? This is in the repo. My version just has *all* the latest changes applied which sometime can be untested. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From superinterstellar at gmail.com Wed Jan 28 14:08:41 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Wed, 28 Jan 2015 22:08:41 +0800 Subject: Indexing Mail faster In-Reply-To: <20150128131628.GE7518@nihlus.leuxner.net> References: <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> <20150128105939.GB7518@nihlus.leuxner.net> <20150128110736.6004882.8269.350@gmail.com> <20150128131628.GE7518@nihlus.leuxner.net> Message-ID: Dear Thomas, Could you advise(or provide a link to) for a safe way to upgrade the Dovecot version. I am a little worried about testing and making trial-and-error approach as it could be risky. Appreciate if you could help with this. I have been trying to address this "slow search" issue for a while with very limited success(I was trying to implement FTS also), so I will appreciate if you could support. Apologies for my noobness. Thank You Kevin Do you have much ideas on how to upgrade dovecot. I am little worried about upgrading dovecot as there could be risks to data. I am trying to upgrade from 2.2.9 to 2.2.15 or Dovecot-EE. On Wed, Jan 28, 2015 at 9:16 PM, Thomas Leuxner wrote: > * superinterstellar at gmail.com 2015.01.28 12 > :07: > > > Do you think I should try 2.2.15? Or stick to EE? > > This is in the repo. My version just has *all* the latest changes applied > which sometime can be untested. > From superinterstellar at gmail.com Wed Jan 28 15:02:24 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Wed, 28 Jan 2015 23:02:24 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> <20150128105939.GB7518@nihlus.leuxner.net> <20150128110736.6004882.8269.350@gmail.com> <20150128131628.GE7518@nihlus.leuxner.net> Message-ID: Dear Thomas, >From what I have read is that I cannot automatically install dovecot, I would need to add the repos to my repo list and then use pining to install the latest version. Kindly advise if I am on correct track? Thanks Kevin On Wed, Jan 28, 2015 at 10:08 PM, Kevin Laurie wrote: > Dear Thomas, > Could you advise(or provide a link to) for a safe way to upgrade the > Dovecot version. > I am a little worried about testing and making trial-and-error approach as > it could be risky. > Appreciate if you could help with this. I have been trying to address this > "slow search" issue for a while with very limited success(I was trying to > implement FTS also), so I will appreciate if you could support. > Apologies for my noobness. > Thank You > Kevin > > > Do you have much ideas on how to upgrade dovecot. I am little worried > about upgrading dovecot as there could be risks to data. > I am trying to upgrade from 2.2.9 to 2.2.15 or Dovecot-EE. > > > On Wed, Jan 28, 2015 at 9:16 PM, Thomas Leuxner wrote: > >> * superinterstellar at gmail.com 2015.01.28 12 >> :07: >> >> > Do you think I should try 2.2.15? Or stick to EE? >> >> This is in the repo. My version just has *all* the latest changes applied >> which sometime can be untested. >> > > From tss at iki.fi Wed Jan 28 15:37:36 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 28 Jan 2015 15:37:36 +0000 Subject: Dovecot & gethostbyname() vulnerability Message-ID: <125B6C03-ED51-481B-93AF-E08A2E6A9E7D@iki.fi> So just a quick note: Dovecot in general doesn't do DNS lookups, except the ones in configuration files and such. So I don't think there are any Dovecot setups which do DNS lookups for untrusted hostnames. Also, gethostbyname() is used only if IPv6 support isn't compiled into Dovecot. And IPv6 support is enabled by default if the libc is detected to support it, so pretty much all systems use it. From superinterstellar at gmail.com Wed Jan 28 16:35:45 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 00:35:45 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150127080221.GA38114@nihlus.leuxner.net> <20150127104755.GB38114@nihlus.leuxner.net> <20150128084502.GA7518@nihlus.leuxner.net> <20150128105356.6004882.88854.347@gmail.com> <20150128105939.GB7518@nihlus.leuxner.net> <20150128110736.6004882.8269.350@gmail.com> <20150128131628.GE7518@nihlus.leuxner.net> Message-ID: Hi guys, Please correct me if I am adding repos to dovecot correctly. I hope I am doing it correctly Step 1. Add the following repos to the following file /etc/apt/sources.list. deb http://xi.rename-it.nl/debian/pool/stable-auto/dovecot-2.2/ trusty universe deb-src http://xi.rename-it.nl/debian/pool/stable-auto/dovecot-2.2/ trusty universe Step 2. apt-get update Step 3. apt-get build-dep dovecot-2.2 Step 4. apt-get -b source -t trusty dovecot-2.2 I hope the process listed above is correct and that if I execute it I wont end up in a mess. Please comment. Thanks Kevin On Wed, Jan 28, 2015 at 11:02 PM, Kevin Laurie wrote: > Dear Thomas, > From what I have read is that I cannot automatically install dovecot, I > would need to add the repos to my repo list and then use pining to install > the latest version. Kindly advise if I am on correct track? > > Thanks > Kevin > > > > > On Wed, Jan 28, 2015 at 10:08 PM, Kevin Laurie < > superinterstellar at gmail.com> wrote: > >> Dear Thomas, >> Could you advise(or provide a link to) for a safe way to upgrade the >> Dovecot version. >> I am a little worried about testing and making trial-and-error approach >> as it could be risky. >> Appreciate if you could help with this. I have been trying to address >> this "slow search" issue for a while with very limited success(I was trying >> to implement FTS also), so I will appreciate if you could support. >> Apologies for my noobness. >> Thank You >> Kevin >> >> >> Do you have much ideas on how to upgrade dovecot. I am little worried >> about upgrading dovecot as there could be risks to data. >> I am trying to upgrade from 2.2.9 to 2.2.15 or Dovecot-EE. >> >> >> On Wed, Jan 28, 2015 at 9:16 PM, Thomas Leuxner wrote: >> >>> * superinterstellar at gmail.com 2015.01.28 >>> 12:07: >>> >>> > Do you think I should try 2.2.15? Or stick to EE? >>> >>> This is in the repo. My version just has *all* the latest changes >>> applied which sometime can be untested. >>> >> >> > From superinterstellar at gmail.com Thu Jan 29 03:59:56 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 11:59:56 +0800 Subject: Troubleshooting doveadm for Solr Configuration Message-ID: Hey guys, I am trying to pinpoint where the problem with my dovecot-FTS installation . I am trying to rule out between the Dovecot and the Solr Module. I think if the output below looks OK then the dovecot is fine, and will focus more on the Solr. Appreciate if someone could check and confirm. Thanks root at mail:/etc/dovecot/conf.d# doveadm -D -v fts rescan -u admin at email.net doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_sieve_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so doveadm(admin at email.net): Debug: auth input: admin at email.net master_user= admin at email.net home=/var/vmail/vmail1/ email.net/t/h/a/admin-2014.10.25.07.15.15/ quota_rule=*:bytes=26214400000 doveadm(admin at email.net): Debug: Added userdb setting: plugin/master_user= admin at email.net doveadm(admin at email.net): Debug: Added userdb setting: plugin/quota_rule=*:bytes=26214400000 doveadm(admin at email.net): Debug: Effective uid=2000, gid=2000, home=/var/vmail/vmail1/email.net/t/h/a/thai-2014.10.25.07.15.15/ doveadm(admin at email.net): Debug: Quota root: name=user backend=dict args=:proxy::quotadict doveadm(admin at email.net): Debug: Quota rule: root=user mailbox=* bytes=26214400000 messages=0 doveadm(admin at email.net): Debug: Quota warning: bytes=22282240000 (85%) messages=0 reverse=no command=quota-warning 85 admin at email.net doveadm(admin at email.net): Debug: Quota warning: bytes=23592960000 (90%) messages=0 reverse=no command=quota-warning 90 admin at email.net doveadm(admin at email.net): Debug: Quota warning: bytes=24903680000 (95%) messages=0 reverse=no command=quota-warning 95 admin at email.net doveadm(admin at email.net): Debug: Quota grace: root=user bytes=2621440000 (10%) doveadm(admin at email.net): Debug: dict quota: user=admin at email.net, uri=proxy::quotadict, noenforcing=0 doveadm(admin at email.net): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir://var/vmail/vmail1/ email.net/t/h/a/thai-admin.10.25.07.15.15//Maildir/:INDEX=//var/vmail/vmail1/email.net/t/h/a/admin-2014.10.25.07.15.15//Maildir/ doveadm(admin at email.net): Debug: maildir++: root=//var/vmail/vmail1/ email.net/t/h/a/admin-2014.10.25.07.15.15//Maildir, index=, indexpvt=, control=, inbox=//var/vmail/vmail1/ sicl.net/t/h/a/thai-2014.10.25.07.15.15//Maildir, alt= doveadm(admin at email.net): Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/Shared/%u doveadm(admin at email.net): Debug: shared: root=/var/run/dovecot, index=, indexpvt=, control=, inbox=, alt= doveadm(admin at email.net): Debug: fts: Indexes disabled for namespace 'Shared/%u/' From petehodur at gmail.com Thu Jan 29 04:25:30 2015 From: petehodur at gmail.com (Peter Hodur) Date: Thu, 29 Jan 2015 05:25:30 +0100 Subject: Indexing Mail faster In-Reply-To: <20150124190408.GA436@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: > * Kevin Laurie 2015.01.24 19:41: > > > Currently the time it takes to search 25,000mails is 4mins. If indexed > how > > much faster are we looking at? > > With a current version of Dovecot a search is pretty fast _without_ using > external indexes. I have a view defined (virtual plugin) with around 22.000 > messages in it, and searching the full view only takes 2.5 seconds: > > hmmm, could you please tell me more about your setup? What storage format do you use? Maildir or? What is the cumulative size of your messages? My results without FTS on ZFS FS with SSD L2ARC are not so good: a4 select INBOX._OLD-OUTLOOK * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk JunkRecorded $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. ** 7748 EXISTS* * 0 RECENT * OK [UIDVALIDITY 1421198037] UIDs valid * OK [UIDNEXT 11509] Predicted next UID * OK [HIGHESTMODSEQ 12204] Highest a4 OK [READ-WRITE] Select completed (0.001 secs). a5 search charset utf-8 body "mall" * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 240 249 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 385 389 390 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 531 532 533 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 672 677 692 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 821 833 855 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 1010 1011 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 1137 1140 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 1282 1286 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 1471 1494 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 1606 1611 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 1735 1781 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 2002 2006 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 2208 2210 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 2377 2378 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 2539 2551 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 2754 2780 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 2936 2938 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 3161 3178 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 3386 3428 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 3801 3808 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 3953 3954 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 4059 4063 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 4145 4146 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 4258 4259 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 4502 4538 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 4594 4604 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 4787 4788 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 4901 4905 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 5129 5131 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 5276 5277 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 5391 5392 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 5642 5647 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 5866 5867 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 5958 5990 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 6233 6271 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 6482 6492 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 6852 6859 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 7042 7108 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 7317 7322 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 7417 7418 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 7502 7503 7558 7588 7589 7628 7685 7695 7699 7703 7723 *a5 OK Search completed (7.846 secs).* Searching against "subject" is pretty fast, few miliseconds ... From superinterstellar at gmail.com Thu Jan 29 04:37:30 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 12:37:30 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: Dear Peter, My inbox is MDA_external Storage: 17GB of 24GB Subject / From / To is fast but FTS(Full Text Search) for body is horrible. I suppose this is where we need Apache Solr. Do you think my mail storage format is bad? Do I need to change for better performance? Please advise Kevin On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur wrote: > > * Kevin Laurie 2015.01.24 19:41: > > > > > Currently the time it takes to search 25,000mails is 4mins. If indexed > > how > > > much faster are we looking at? > > > > With a current version of Dovecot a search is pretty fast _without_ using > > external indexes. I have a view defined (virtual plugin) with around > 22.000 > > messages in it, and searching the full view only takes 2.5 seconds: > > > > > hmmm, could you please tell me more about your setup? What storage format > do you use? Maildir or? What is the cumulative size of your messages? > > My results without FTS on ZFS FS with SSD L2ARC are not so good: > > a4 select INBOX._OLD-OUTLOOK > * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk > JunkRecorded $Forwarded) > * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk > $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. > ** 7748 EXISTS* > * 0 RECENT > * OK [UIDVALIDITY 1421198037] UIDs valid > * OK [UIDNEXT 11509] Predicted next UID > * OK [HIGHESTMODSEQ 12204] Highest > a4 OK [READ-WRITE] Select completed (0.001 secs). > > a5 search charset utf-8 body "mall" > * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 240 249 > 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 385 389 390 > 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 531 532 533 > 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 672 677 692 > 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 821 833 855 > 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 1010 1011 > 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 1137 1140 > 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 1282 1286 > 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 1471 1494 > 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 1606 1611 > 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 1735 1781 > 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 2002 2006 > 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 2208 2210 > 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 2377 2378 > 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 2539 2551 > 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 2754 2780 > 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 2936 2938 > 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 3161 3178 > 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 3386 3428 > 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 3801 3808 > 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 3953 3954 > 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 4059 4063 > 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 4145 4146 > 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 4258 4259 > 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 4502 4538 > 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 4594 4604 > 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 4787 4788 > 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 4901 4905 > 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 5129 5131 > 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 5276 5277 > 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 5391 5392 > 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 5642 5647 > 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 5866 5867 > 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 5958 5990 > 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 6233 6271 > 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 6482 6492 > 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 6852 6859 > 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 7042 7108 > 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 7317 7322 > 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 7417 7418 > 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 7502 7503 > 7558 7588 7589 7628 7685 7695 7699 7703 7723 > *a5 OK Search completed (7.846 secs).* > > > Searching against "subject" is pretty fast, few miliseconds ... > From superinterstellar at gmail.com Thu Jan 29 04:43:14 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 12:43:14 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: Hi Peter, Sorry, I think its maildir Output of my dovecot -n is listed below:- # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS auth_master_user_separator = * auth_mechanisms = PLAIN LOGIN dict { acl = mysql:/etc/dovecot/dovecot- share-folder.conf quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf } first_valid_uid = 2000 last_valid_uid = 2000 listen = * log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 2000 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = quota fts mail_uid = 2000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { auto = no special_use = \Sent } mailbox Spam { auto = no special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } namespace { list = children location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } passdb { args = /etc/dovecot/dovecot-master-users-password driver = passwd-file master = yes } plugin { acl = vfile acl_shared_dict = proxy::acl auth_socket_path = /var/run/dovecot/auth-master quota = dict:user::proxy::quotadict quota_rule = *:storage=1G quota_warning = storage=85%% quota-warning 85 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = /%Lh/sieve/dovecot.sieve sieve_default = /var/vmail/sieve/dovecot.sieve sieve_dir = /%Lh/sieve sieve_global_dir = /var/vmail/sieve } protocols = pop3 imap sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { group = vmail mode = 0666 user = vmail } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { process_limit = 500 service_count = 1 } service lmtp { executable = lmtp -L inet_listener lmtp { port = 24 } process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service pop3-login { service_count = 1 } service quota-warning { executable = script /usr/local/bin/dovecot-quota-warning.sh unix_listener quota-warning { group = vmail mode = 0660 user = vmail } } ssl = required ssl_cert = wrote: > Dear Peter, > > My inbox is MDA_external > Storage: 17GB of 24GB > > Subject / From / To is fast but FTS(Full Text Search) for body is > horrible. I suppose this is where we need Apache Solr. > > Do you think my mail storage format is bad? Do I need to change for better > performance? > Please advise > Kevin > > > > On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur wrote: > >> > * Kevin Laurie 2015.01.24 19:41: >> > >> > > Currently the time it takes to search 25,000mails is 4mins. If indexed >> > how >> > > much faster are we looking at? >> > >> > With a current version of Dovecot a search is pretty fast _without_ >> using >> > external indexes. I have a view defined (virtual plugin) with around >> 22.000 >> > messages in it, and searching the full view only takes 2.5 seconds: >> > >> > >> hmmm, could you please tell me more about your setup? What storage format >> do you use? Maildir or? What is the cumulative size of your messages? >> >> My results without FTS on ZFS FS with SSD L2ARC are not so good: >> >> a4 select INBOX._OLD-OUTLOOK >> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk >> JunkRecorded $Forwarded) >> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk >> $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. >> ** 7748 EXISTS* >> * 0 RECENT >> * OK [UIDVALIDITY 1421198037] UIDs valid >> * OK [UIDNEXT 11509] Predicted next UID >> * OK [HIGHESTMODSEQ 12204] Highest >> a4 OK [READ-WRITE] Select completed (0.001 secs). >> >> a5 search charset utf-8 body "mall" >> * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 240 249 >> 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 385 389 >> 390 >> 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 531 532 >> 533 >> 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 672 677 >> 692 >> 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 821 833 >> 855 >> 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 1010 1011 >> 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 1137 1140 >> 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 1282 1286 >> 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 1471 1494 >> 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 1606 1611 >> 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 1735 1781 >> 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 2002 2006 >> 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 2208 2210 >> 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 2377 2378 >> 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 2539 2551 >> 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 2754 2780 >> 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 2936 2938 >> 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 3161 3178 >> 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 3386 3428 >> 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 3801 3808 >> 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 3953 3954 >> 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 4059 4063 >> 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 4145 4146 >> 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 4258 4259 >> 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 4502 4538 >> 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 4594 4604 >> 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 4787 4788 >> 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 4901 4905 >> 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 5129 5131 >> 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 5276 5277 >> 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 5391 5392 >> 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 5642 5647 >> 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 5866 5867 >> 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 5958 5990 >> 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 6233 6271 >> 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 6482 6492 >> 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 6852 6859 >> 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 7042 7108 >> 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 7317 7322 >> 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 7417 7418 >> 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 7502 7503 >> 7558 7588 7589 7628 7685 7695 7699 7703 7723 >> *a5 OK Search completed (7.846 secs).* >> >> >> Searching against "subject" is pretty fast, few miliseconds ... >> > > From superinterstellar at gmail.com Thu Jan 29 05:29:43 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 13:29:43 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: Dear Peter, Noted. Thanks for your input. Appreciate it. At this point my urgent most priority is to get FTS. Waiting 3 mins for a body search email is bad(but of course my email box is large) I need to have this sorted out by today as I have been putting this off for too long mostly because of lack of troubleshooting documentation online(if i do figure this out will create some for guidance) Regards Kevin On Thu, Jan 29, 2015 at 1:18 PM, Peter Hodur wrote: > > > On Thursday, January 29, 2015, Kevin Laurie > wrote: > >> Dear Peter, >> Oh. Sorry(didnt know you were addressing someone else) my apologies. But >> as you can see , I am desperately trying to address this issue. >> > > No problem ;))) i wrote because my search result is good but not so > good as someone wrote here couple of days ago. > > > >> I have 1 disk with 200GB running on a VPS. >> The file system is ext4. >> > > Im not dovecot expert, but i think this is your problem. You need more > iops. If you need performance, VPS may be (if connected to fast SAN) but > basicly is not a good choice. > > >> I think I will need to implement FTS to fix this as I will need body >> searches. >> > > FTS could be a solution. You are right. > > >> Have you tried FTS before? >> > > Im sorry, never :( > > But may be someone more skilled than me could answer the main question, > how much dovecot index helps in case imap search agains body. > > Im not sure, but my opinion is that it helps little (may be not at all) > > Ps: may be you use it, but do not forget to setup delivering from mail > server via LDA or LMTP instead of direct to maildirs. This is better > because dovecot updates your indexes at the time of message arrival. Later > accesses should be faster. > > Peter > > > >> >> >> Regards >> Kevin >> > >> >> On Thu, Jan 29, 2015 at 12:55 PM, Peter Hodur >> wrote: >> >>> Kevin, >>> >>> My message was not addressed to you ;) because my results are NOT so >>> good as someone wrote it here. I can fulltext cca 8k messages in 7-8 >>> seconds. >>> >>> Someone wrote that he has results cca 22k messages in 4 seconds :( >>> >>> Im not sure, but the answer is may be in the index and disk subsystem. >>> >>> If and only if dovecot index does not have saved in the index keywords >>> from BODY of messages the problem is your/my disk subsystem. >>> >>> Generally, messages from mailing list like this are pretty small. The >>> problem is that dovecot must read all messages. If you have maildir its >>> mean open and read many files. >>> >>> But my test search was agains archive of my personal inbox - so not onoy >>> small messages like this but also messages with big attachments etc. >>> >>> If dovecot index only headers, than it in case of full text search read >>> whole messages - mime parts allows that plain text body can be after >>> attachment etc. >>> >>> And here is it all about iops and throughput. >>> >>> How many disks you have? And what setup? >>> >>> Generally, the only good setup is many smaller disks in RAID 10 >>> (stripped mirrors) and if your fs allows it, with added read cache - in >>> case of zfs: l2arc on ssd. >>> >>> >>> >>> Pete >>> >>> >>> >>> On Thursday, January 29, 2015, Kevin Laurie >>> wrote: >>> >>>> Hi Peter, >>>> Sorry, >>>> >>>> I think its maildir >>>> Output of my dovecot -n is listed below:- >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> # 2.2.9: /etc/dovecot/dovecot.conf >>>> # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS >>>> auth_master_user_separator = * >>>> auth_mechanisms = PLAIN LOGIN >>>> dict { >>>> acl = mysql:/etc/dovecot/dovecot- >>>> share-folder.conf >>>> quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf >>>> } >>>> first_valid_uid = 2000 >>>> last_valid_uid = 2000 >>>> listen = * >>>> log_path = /var/log/dovecot.log >>>> mail_debug = yes >>>> mail_gid = 2000 >>>> mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ >>>> mail_plugins = quota fts >>>> mail_uid = 2000 >>>> managesieve_notify_capability = mailto >>>> managesieve_sieve_capability = fileinto reject envelope >>>> encoded-character vacation subaddress comparator-i;ascii-numeric relational >>>> regex imap4flags copy include variables body enotify environment mailbox >>>> date ihave >>>> namespace { >>>> inbox = yes >>>> location = >>>> mailbox Drafts { >>>> auto = subscribe >>>> special_use = \Drafts >>>> } >>>> mailbox Junk { >>>> auto = subscribe >>>> special_use = \Junk >>>> } >>>> mailbox Sent { >>>> auto = subscribe >>>> special_use = \Sent >>>> } >>>> mailbox "Sent Messages" { >>>> auto = no >>>> special_use = \Sent >>>> } >>>> mailbox Spam { >>>> auto = no >>>> special_use = \Junk >>>> } >>>> mailbox Trash { >>>> auto = subscribe >>>> special_use = \Trash >>>> } >>>> prefix = >>>> separator = / >>>> type = private >>>> } >>>> namespace { >>>> list = children >>>> location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u >>>> prefix = Shared/%%u/ >>>> separator = / >>>> subscriptions = yes >>>> type = shared >>>> } >>>> passdb { >>>> args = /etc/dovecot/dovecot-mysql.conf >>>> driver = sql >>>> } >>>> passdb { >>>> args = /etc/dovecot/dovecot-master-users-password >>>> driver = passwd-file >>>> master = yes >>>> } >>>> plugin { >>>> acl = vfile >>>> acl_shared_dict = proxy::acl >>>> auth_socket_path = /var/run/dovecot/auth-master >>>> quota = dict:user::proxy::quotadict >>>> quota_rule = *:storage=1G >>>> quota_warning = storage=85%% quota-warning 85 %u >>>> quota_warning2 = storage=90%% quota-warning 90 %u >>>> quota_warning3 = storage=95%% quota-warning 95 %u >>>> sieve = /%Lh/sieve/dovecot.sieve >>>> sieve_default = /var/vmail/sieve/dovecot.sieve >>>> sieve_dir = /%Lh/sieve >>>> sieve_global_dir = /var/vmail/sieve >>>> } >>>> protocols = pop3 imap sieve lmtp >>>> service auth { >>>> unix_listener /var/spool/postfix/private/dovecot-auth { >>>> group = postfix >>>> mode = 0666 >>>> user = postfix >>>> } >>>> unix_listener auth-master { >>>> group = vmail >>>> mode = 0666 >>>> user = vmail >>>> } >>>> unix_listener auth-userdb { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> service dict { >>>> unix_listener dict { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> service imap-login { >>>> process_limit = 500 >>>> service_count = 1 >>>> } >>>> service lmtp { >>>> executable = lmtp -L >>>> inet_listener lmtp { >>>> port = 24 >>>> } >>>> process_min_avail = 5 >>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>>> group = postfix >>>> mode = 0600 >>>> user = postfix >>>> } >>>> user = vmail >>>> } >>>> service pop3-login { >>>> service_count = 1 >>>> } >>>> service quota-warning { >>>> executable = script /usr/local/bin/dovecot-quota-warning.sh >>>> unix_listener quota-warning { >>>> group = vmail >>>> mode = 0660 >>>> user = vmail >>>> } >>>> } >>>> ssl = required >>>> ssl_cert = >>> ssl_key = >>> userdb { >>>> args = /etc/dovecot/dovecot-mysql.conf >>>> driver = sql >>>> } >>>> protocol lda { >>>> auth_socket_path = /var/run/dovecot/auth-master >>>> lda_mailbox_autocreate = yes >>>> log_path = /var/log/dovecot-sieve.log >>>> mail_plugins = quota fts sieve >>>> postmaster_address = root >>>> } >>>> protocol lmtp { >>>> info_log_path = /var/log/dovecot-lmtp.log >>>> lmtp_save_to_detail_mailbox = yes >>>> mail_plugins = quota sieve >>>> postmaster_address = postmaster >>>> recipient_delimiter = + >>>> } >>>> protocol imap { >>>> imap_client_workarounds = tb-extra-mailbox-sep >>>> mail_plugins = quota fts imap_quota >>>> } >>>> protocol pop3 { >>>> mail_plugins = quota fts >>>> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >>>> pop3_uidl_format = %08Xu%08Xv >>>> } >>>> >>>> >>>> On Thu, Jan 29, 2015 at 12:37 PM, Kevin Laurie < >>>> superinterstellar at gmail.com> wrote: >>>> >>>>> Dear Peter, >>>>> >>>>> My inbox is MDA_external >>>>> Storage: 17GB of 24GB >>>>> >>>>> Subject / From / To is fast but FTS(Full Text Search) for body is >>>>> horrible. I suppose this is where we need Apache Solr. >>>>> >>>>> Do you think my mail storage format is bad? Do I need to change for >>>>> better performance? >>>>> Please advise >>>>> Kevin >>>>> >>>>> >>>>> >>>>> On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur >>>>> wrote: >>>>> >>>>>> > * Kevin Laurie 2015.01.24 19:41: >>>>>> > >>>>>> > > Currently the time it takes to search 25,000mails is 4mins. If >>>>>> indexed >>>>>> > how >>>>>> > > much faster are we looking at? >>>>>> > >>>>>> > With a current version of Dovecot a search is pretty fast _without_ >>>>>> using >>>>>> > external indexes. I have a view defined (virtual plugin) with >>>>>> around 22.000 >>>>>> > messages in it, and searching the full view only takes 2.5 seconds: >>>>>> > >>>>>> > >>>>>> hmmm, could you please tell me more about your setup? What storage >>>>>> format >>>>>> do you use? Maildir or? What is the cumulative size of your messages? >>>>>> >>>>>> My results without FTS on ZFS FS with SSD L2ARC are not so good: >>>>>> >>>>>> a4 select INBOX._OLD-OUTLOOK >>>>>> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk >>>>>> JunkRecorded $Forwarded) >>>>>> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk >>>>>> $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. >>>>>> ** 7748 EXISTS* >>>>>> * 0 RECENT >>>>>> * OK [UIDVALIDITY 1421198037] UIDs valid >>>>>> * OK [UIDNEXT 11509] Predicted next UID >>>>>> * OK [HIGHESTMODSEQ 12204] Highest >>>>>> a4 OK [READ-WRITE] Select completed (0.001 secs). >>>>>> >>>>>> a5 search charset utf-8 body "mall" >>>>>> * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 240 >>>>>> 249 >>>>>> 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 385 >>>>>> 389 390 >>>>>> 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 531 >>>>>> 532 533 >>>>>> 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 672 >>>>>> 677 692 >>>>>> 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 821 >>>>>> 833 855 >>>>>> 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 1010 >>>>>> 1011 >>>>>> 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 1137 >>>>>> 1140 >>>>>> 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 1282 >>>>>> 1286 >>>>>> 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 1471 >>>>>> 1494 >>>>>> 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 1606 >>>>>> 1611 >>>>>> 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 1735 >>>>>> 1781 >>>>>> 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 2002 >>>>>> 2006 >>>>>> 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 2208 >>>>>> 2210 >>>>>> 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 2377 >>>>>> 2378 >>>>>> 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 2539 >>>>>> 2551 >>>>>> 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 2754 >>>>>> 2780 >>>>>> 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 2936 >>>>>> 2938 >>>>>> 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 3161 >>>>>> 3178 >>>>>> 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 3386 >>>>>> 3428 >>>>>> 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 3801 >>>>>> 3808 >>>>>> 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 3953 >>>>>> 3954 >>>>>> 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 4059 >>>>>> 4063 >>>>>> 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 4145 >>>>>> 4146 >>>>>> 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 4258 >>>>>> 4259 >>>>>> 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 4502 >>>>>> 4538 >>>>>> 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 4594 >>>>>> 4604 >>>>>> 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 4787 >>>>>> 4788 >>>>>> 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 4901 >>>>>> 4905 >>>>>> 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 5129 >>>>>> 5131 >>>>>> 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 5276 >>>>>> 5277 >>>>>> 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 5391 >>>>>> 5392 >>>>>> 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 5642 >>>>>> 5647 >>>>>> 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 5866 >>>>>> 5867 >>>>>> 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 5958 >>>>>> 5990 >>>>>> 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 6233 >>>>>> 6271 >>>>>> 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 6482 >>>>>> 6492 >>>>>> 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 6852 >>>>>> 6859 >>>>>> 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 7042 >>>>>> 7108 >>>>>> 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 7317 >>>>>> 7322 >>>>>> 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 7417 >>>>>> 7418 >>>>>> 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 7502 >>>>>> 7503 >>>>>> 7558 7588 7589 7628 7685 7695 7699 7703 7723 >>>>>> *a5 OK Search completed (7.846 secs).* >>>>>> >>>>>> >>>>>> Searching against "subject" is pretty fast, few miliseconds ... >>>>>> >>>>> >>>>> >>>> >> From jtam.home at gmail.com Thu Jan 29 07:34:42 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 28 Jan 2015 23:34:42 -0800 (PST) Subject: Indexing Mail faster In-Reply-To: References: Message-ID: Kevin writes: > Appreciate if you could help with this. I have been trying to address this > "slow search" issue for a while with very limited success(I was trying to > implement FTS also), so I will appreciate if you could support. When I'm stumped, one of the diagnostic tools I use is process tracing. Connect via IMAP, in another window/session process trace the IMAP worker process (Linux: strace -r -otrace.dump {pid})), initiate the search and quit. Then you can look through the trace and see whether it gets hung on a particular operation. Joseph Tam From superinterstellar at gmail.com Thu Jan 29 07:56:48 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 15:56:48 +0800 Subject: Indexing Mail faster In-Reply-To: References: Message-ID: Hey Joseph Thanks. Strace seems like a very useful tool Only problem is that I dont think it is maintained on ubuntu. Tried to run:- apt-get install strace but could not download it. Might need to download and build it. Do you know any other way of getting it? Thanks Kevin A. On Thu, Jan 29, 2015 at 3:34 PM, Joseph Tam wrote: > Kevin writes: > > Appreciate if you could help with this. I have been trying to address this >> "slow search" issue for a while with very limited success(I was trying to >> implement FTS also), so I will appreciate if you could support. >> > > When I'm stumped, one of the diagnostic tools I use is process tracing. > Connect via IMAP, in another window/session process trace the IMAP worker > process (Linux: strace -r -otrace.dump {pid})), initiate the search and > quit. Then you can look through the trace and see whether it gets hung on > a particular operation. > > Joseph Tam > From thom at cagroups.com Thu Jan 29 08:02:29 2015 From: thom at cagroups.com (Thom Miller) Date: Thu, 29 Jan 2015 01:02:29 -0700 Subject: Indexing Mail faster In-Reply-To: References: Message-ID: <1422518549.22437.1.camel@cagroups.com> On Thu, 2015-01-29 at 15:56 +0800, Kevin Laurie wrote: > Hey Joseph > Thanks. Strace seems like a very useful tool > Only problem is that I dont think it is maintained on ubuntu. > Tried to run:- > apt-get install strace but could not download it. > Might need to download and build it. Do you know any other way of getting > it? > Thanks > Kevin A. try apt-cache search strace On Jessie (I'm not running Ubuntu) I get: devscripts - scripts to make the life of a Debian Package maintainer easier dnstracer - trace DNS queries to the source ioapps - IO profiler and IO traces replayer netsniff-ng - Linux network packet sniffer toolkit python-ptrace - Python bindings for ptrace strace - System call tracer subversion-tools - Assorted tools related to Apache Subversion xtrace - trace communication between X client and server You should be able to find the appropriate package to install to get strace. -Thom From superinterstellar at gmail.com Thu Jan 29 08:09:02 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:09:02 +0800 Subject: Indexing Mail faster In-Reply-To: <1422518549.22437.1.camel@cagroups.com> References: <1422518549.22437.1.camel@cagroups.com> Message-ID: Hi Thom, No results. I think I am missing some repositories. I might add some from ubuntu. On Thu, Jan 29, 2015 at 4:02 PM, Thom Miller wrote: > On Thu, 2015-01-29 at 15:56 +0800, Kevin Laurie wrote: > > Hey Joseph > > Thanks. Strace seems like a very useful tool > > Only problem is that I dont think it is maintained on ubuntu. > > Tried to run:- > > apt-get install strace but could not download it. > > Might need to download and build it. Do you know any other way of getting > > it? > > Thanks > > Kevin A. > > try apt-cache search strace > > On Jessie (I'm not running Ubuntu) I get: > > devscripts - scripts to make the life of a Debian Package maintainer > easier > dnstracer - trace DNS queries to the source > ioapps - IO profiler and IO traces replayer > netsniff-ng - Linux network packet sniffer toolkit > python-ptrace - Python bindings for ptrace > strace - System call tracer > subversion-tools - Assorted tools related to Apache Subversion > xtrace - trace communication between X client and server > > You should be able to find the appropriate package to install to get > strace. > > -Thom > From superinterstellar at gmail.com Thu Jan 29 08:13:47 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:13:47 +0800 Subject: Indexing Mail faster In-Reply-To: References: <1422518549.22437.1.camel@cagroups.com> Message-ID: Yes. I was missing due to repositories. Added more sources and now its fine. Thanks! Kevin On Thu, Jan 29, 2015 at 4:09 PM, Kevin Laurie wrote: > Hi Thom, > No results. I think I am missing some repositories. > I might add some from ubuntu. > > On Thu, Jan 29, 2015 at 4:02 PM, Thom Miller wrote: > >> On Thu, 2015-01-29 at 15:56 +0800, Kevin Laurie wrote: >> > Hey Joseph >> > Thanks. Strace seems like a very useful tool >> > Only problem is that I dont think it is maintained on ubuntu. >> > Tried to run:- >> > apt-get install strace but could not download it. >> > Might need to download and build it. Do you know any other way of >> getting >> > it? >> > Thanks >> > Kevin A. >> >> try apt-cache search strace >> >> On Jessie (I'm not running Ubuntu) I get: >> >> devscripts - scripts to make the life of a Debian Package maintainer >> easier >> dnstracer - trace DNS queries to the source >> ioapps - IO profiler and IO traces replayer >> netsniff-ng - Linux network packet sniffer toolkit >> python-ptrace - Python bindings for ptrace >> strace - System call tracer >> subversion-tools - Assorted tools related to Apache Subversion >> xtrace - trace communication between X client and server >> >> You should be able to find the appropriate package to install to get >> strace. >> >> -Thom >> > > From tlx at leuxner.net Thu Jan 29 08:20:24 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Thu, 29 Jan 2015 09:20:24 +0100 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: <20150129082023.GA14698@nihlus.leuxner.net> * Peter Hodur 2015.01.29 05:25: > > With a current version of Dovecot a search is pretty fast _without_ using > > external indexes. I have a view defined (virtual plugin) with around 22.000 > > messages in it, and searching the full view only takes 2.5 seconds: > > > > > hmmm, could you please tell me more about your setup? What storage format > do you use? Maildir or? What is the cumulative size of your messages? Although the question was not formatted/quoted correctly I guess it is more towards myself as it explicitely speaks of a non-FTS setup: $ dmesg | grep smpboot [ 0.000000] smpboot: Allowing 64 CPUs, 56 hotplug CPUs [ 0.148600] smpboot: CPU0: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz (fam: 06, model: 3c, stepping: 03) [ 0.300030] smpboot: Total of 8 processors activated (52677.72 BogoMIPS) The volume providing the mail root is using ext4 on an MD mirror (with CRYPT and LVM overhead): $ lsblk [...] ??sda3 8:3 0 ... 0 part ??md2 9:2 0 ... 0 raid1 ??crypt1 (dm-0) 253:0 0 ... 0 crypt ??vg0-vmail (dm-11) 253:11 0 ... 0 lvm /var/vmail $ doveconf -n | head -3 # 2.2.15 (7557234ac0f4): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 # OS: Linux 3.16.0-0.bpo.4-amd64 x86_64 Debian 7.8 I'm using unspectacular 7,200rpm SATA drives and mdbox. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From mail at marc-stuermer.de Thu Jan 29 08:23:28 2015 From: mail at marc-stuermer.de (=?UTF-8?B?TWFyYyBTdMO8cm1lcg==?=) Date: Thu, 29 Jan 2015 09:23:28 +0100 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: <54C9EE00.8080206@marc-stuermer.de> Am 27.01.2015 um 03:24 schrieb Kevin Laurie: > Hi Thomas, > That's very interesting. > Below is my search result. Any idea why is my result so slow:- > Appreciate if you could advise. It depends on a couple of facts. Full text searches must go through a number of bottle necks, depending on your installation, namely: a) type of disk drive(s), b) way of installation of those drives, c) the underlying file system itself, d) the storage format being chosen to save the mails, e) how many users you've got on your system, f) memory of your system, g) CPU power of your system and h) how busy your system is. If you got for example a lonely box with let's sax four gigs of RAM and you are using it by yourself only, chances are high enough that even with a big enough up time most of your mails are in the file system cache of your OS. Meaning a full text search would happen mostly in RAM and therefor of course would be blazingly fast. If you take the same machine and let it serve let's say about 2000 mailboxes, this would be a very different kind of matter. Your file system cache would be flushed frequently and full text searches without index would be way slower, because now those search actually mean I/O operations for Dovecot on your type of storage. If you really want consistent, fast full text search speeds on a busy box, incremental full text indexing is the only way to go. If you are the only user of a big box chances are high you get similar results without. From superinterstellar at gmail.com Thu Jan 29 08:24:05 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:24:05 +0800 Subject: Indexing Mail faster In-Reply-To: <20150129082023.GA14698@nihlus.leuxner.net> References: <20150124190408.GA436@nihlus.leuxner.net> <20150129082023.GA14698@nihlus.leuxner.net> Message-ID: Dear Thomas, You do seem to have some computing power. I think that does help. Thanks Kevin A. On Thu, Jan 29, 2015 at 4:20 PM, Thomas Leuxner wrote: > * Peter Hodur 2015.01.29 05:25: > > > > With a current version of Dovecot a search is pretty fast _without_ > using > > > external indexes. I have a view defined (virtual plugin) with around > 22.000 > > > messages in it, and searching the full view only takes 2.5 seconds: > > > > > > > > hmmm, could you please tell me more about your setup? What storage format > > do you use? Maildir or? What is the cumulative size of your messages? > > > Although the question was not formatted/quoted correctly I guess it is > more towards myself as it explicitely speaks of a non-FTS setup: > > $ dmesg | grep smpboot > [ 0.000000] smpboot: Allowing 64 CPUs, 56 hotplug CPUs > [ 0.148600] smpboot: CPU0: Intel(R) Xeon(R) CPU E3-1230 v3 @ 3.30GHz > (fam: 06, model: 3c, stepping: 03) > [ 0.300030] smpboot: Total of 8 processors activated (52677.72 BogoMIPS) > > The volume providing the mail root is using ext4 on an MD mirror (with > CRYPT and LVM overhead): > > > $ lsblk > [...] > ??sda3 8:3 0 ... 0 part > ??md2 9:2 0 ... 0 raid1 > ??crypt1 (dm-0) 253:0 0 ... 0 crypt > ??vg0-vmail (dm-11) 253:11 0 ... 0 lvm /var/vmail > > $ doveconf -n | head -3 > # 2.2.15 (7557234ac0f4): /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.6 > # OS: Linux 3.16.0-0.bpo.4-amd64 x86_64 Debian 7.8 > > I'm using unspectacular 7,200rpm SATA drives and mdbox. > From christian.binder at freilassing.de Thu Jan 29 08:24:12 2015 From: christian.binder at freilassing.de (Christian Binder Stadt Freilassing) Date: Thu, 29 Jan 2015 09:24:12 +0100 Subject: Uniqueness of dovecot mailbox-guids Message-ID: <19be7073210b3a37aa30b79fcaf796b0@egroupware.freilassing.de> Sorry if this is a duplicate message. I'm new in this mailing list and not sure if the first message really got posted ... Dovecot version: 2.2.9 We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I defined a master user for the old cyrus mailserver in imapc backend to get access to all the mailboxes. Now I realized that after migration every user's "Inbox" or "Trash" on the new dovecot server has the same mailbox-guid, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox dd9456... trash 5549cc... Is this a wanted behaviour? If i do not migrate with doveadm backup and just manually create mailboxes, the mailbox-guids are globally unique, e.g: User a: inbox dd9456... trash 5549cc... User b: inbox c543fe... trash t49ccc... Thanks! Christian From superinterstellar at gmail.com Thu Jan 29 08:28:48 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:28:48 +0800 Subject: Indexing Mail faster In-Reply-To: <54C9EE00.8080206@marc-stuermer.de> References: <20150124190408.GA436@nihlus.leuxner.net> <54C9EE00.8080206@marc-stuermer.de> Message-ID: Dear Marc, Noted. This is starting to make a lot of sense. Of course, more computing power, hence more performance. Also filesystems and such all have and effect on the outcome. I might experiment with adding more computing power and seeing the result. My mailbox does not have many users so I might try that. Currently running a 2gb RAM with 2core processors. On Thu, Jan 29, 2015 at 4:23 PM, Marc St?rmer wrote: > Am 27.01.2015 um 03:24 schrieb Kevin Laurie: > > Hi Thomas, >> That's very interesting. >> Below is my search result. Any idea why is my result so slow:- >> Appreciate if you could advise. >> > > It depends on a couple of facts. Full text searches must go through a > number of bottle necks, depending on your installation, namely: > > a) type of disk drive(s), > b) way of installation of those drives, > c) the underlying file system itself, > d) the storage format being chosen to save the mails, > e) how many users you've got on your system, > f) memory of your system, > g) CPU power of your system and > h) how busy your system is. > > If you got for example a lonely box with let's sax four gigs of RAM and > you are using it by yourself only, chances are high enough that even with a > big enough up time most of your mails are in the file system cache of your > OS. Meaning a full text search would happen mostly in RAM and therefor of > course would be blazingly fast. > > If you take the same machine and let it serve let's say about 2000 > mailboxes, this would be a very different kind of matter. Your file system > cache would be flushed frequently and full text searches without index > would be way slower, because now those search actually mean I/O operations > for Dovecot on your type of storage. > > If you really want consistent, fast full text search speeds on a busy box, > incremental full text indexing is the only way to go. > > If you are the only user of a big box chances are high you get similar > results without. > From superinterstellar at gmail.com Thu Jan 29 08:45:04 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:45:04 +0800 Subject: Indexing Mail faster In-Reply-To: References: Message-ID: Dear Joseph, Just to clarify, the pid you are referring to is of dovecot? Correct? Thanks Kevin On Thu, Jan 29, 2015 at 3:34 PM, Joseph Tam wrote: > Kevin writes: > > Appreciate if you could help with this. I have been trying to address this >> "slow search" issue for a while with very limited success(I was trying to >> implement FTS also), so I will appreciate if you could support. >> > > When I'm stumped, one of the diagnostic tools I use is process tracing. > Connect via IMAP, in another window/session process trace the IMAP worker > process (Linux: strace -r -otrace.dump {pid})), initiate the search and > quit. Then you can look through the trace and see whether it gets hung on > a particular operation. > > Joseph Tam > From jtam.home at gmail.com Thu Jan 29 09:56:45 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 29 Jan 2015 01:56:45 -0800 (PST) Subject: Indexing Mail faster In-Reply-To: References: Message-ID: On Thu, 29 Jan 2015, Kevin Laurie wrote: > Just to clarify, the pid you are referring to is of dovecot? Yes, the dovecot/imap worker process. If there is no system call that is using up a lot of time, but the cumulative open/read time for going through many files in your Maildir, then you have some sort of filesystem metadata performance problem. Joseph Tam From skdovecot at smail.inf.fh-brs.de Thu Jan 29 10:24:10 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 29 Jan 2015 11:24:10 +0100 (CET) Subject: Dovecot antispam In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 26 Jan 2015, David Mehler wrote: > Can I get a look at your antispam configs? I'd like to compare them > against mine. I use spool2dir in conjuntion with a custom inotify listener, that feeds the messages via socket to spamd, so it's highly unique. But post you config and what you want to achieve with it. > On 1/26/15, Steffen Kaiser wrote: >> On Sun, 25 Jan 2015, David Mehler wrote: >> >>> I've got a Postfix, amavisd-new, dovecot (delivery and sieve), setup >>> with spamassassin. It's for the most part working fine, here's the >>> problem, false positives and false negatives. The sieve moves >>> spamassassin-tagged messages in to the Spam folder, sometimes it >>> misses either a false positive or negative. What I'm wanting to do and >>> this is where the dovecot-antispam plugin comes in is automatically >>> retrain messages, a false positive or negative needs to be moved and >>> reclassified accordingly. I've found a lot of google results >>> indicating dspam, when I used this the results were terrible after >>> three months of usage I gave up on it. I've thought about using spamc >>> with dovecot-antispam to handle retraining or crm114, but not much on >>> getting this going. If anyone has any information on this or any other >>> non-dspam setup that would work I'd appreciate knowing about it. >> >> What kind of information you are seeking? >> In what regards are the results of dspam _terrible_? >> >> Installation / Configuration? :- http://wiki2.dovecot.org/Plugins/Antispam - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMoKSnz1H7kL/d9rAQIJ4Af+IRhtUD36LBxJx7XpU7jM+U5VOxjkux0c f4ypPa6CJzpqQ60UBOpWT3EiME1dEnvw/y0U3V7rZPXFntXVAditynTDIiGsN97V OTB0Z7QCKq4AJ13kUoDNrCpAUaNhw5TzR2LwkEcAZSr9IE1aAWD1SbV8FFoxec2c p+L+wYA7FLgF/dZrmG1USjyqPyXT1ODOmESRwRZ5HJZ4io44nois5j3/psMLNwOE xqBy3dSz9uVBlGu4OeZEcWkd+2fWtf83bkyfb7ZM4zSgOe8qFE8YJ86HBq007PAN qPc6o6Q8uylkCZekNzKY3MF9AyI1XO0IsneC9/eLsGVpddu3oPxYhw== =y+NZ -----END PGP SIGNATURE----- From berni at birkenwald.de Thu Jan 29 12:53:07 2015 From: berni at birkenwald.de (Bernhard Schmidt) Date: Thu, 29 Jan 2015 12:53:07 +0000 (UTC) Subject: quota_full_tempfail = yes broken with lmtp_rcpt_check_quota (was: Re: lmtp_rcpt_check_quota not working) References: <7DE209CE-2F9D-404C-8215-A705ACCC84A0@swing.be> Message-ID: Bernhard Schmidt wrote: Hi, > Axel Luttgens wrote: > > Hi Axel, > >> Le 11 f?vr. 2014 ? 08:34, Bernhard Schmidt a ?crit : >> >>> [...] >>> so Dovecot should reject in RCPT TO phase, but according to the logs of the >>> upstream Postfix it does so after END-OF-DATA >>> >>> (host xxx[2001:xxx] said: 452 4.2.2 >>> Quota exceeded (mailbox for user is >>> full) (in reply to end of DATA command)) >>> [...] >> >> Hello Bernhard, >> >> Could you exercise your quota-status service with variations of: >> >> # telnet xxx 12340 >> Trying xxx... >> Connected to xxx. >> Escape character is '^]'. >> recipient=john.doe at example.com >> size=1000000 >> >> action=OK >> >> ^] >> >> and see whether the replies are the expected one? >> >> If yes, this would point to some error in Postfix' configuration. >> >> If not, well... could be you are facing the same kind of problem I'm >> having here (never managed to have the service reply anything else >> than "OK" or "REJECT Unknown user"...) > > quota-status works absolutely fine, our frontend MXes defer mails for > mailboxes over quota. But when we try to send a mail to the mailbox > nevertheless through LMTP (i.e. because it was generated on a server > that doesn't check quota-status), it is deferred in LMTP in the > END-OF-DATA phase, not in RCPT TO as expected. To follow up here: Dovecot 2.2.13 (from Debian testing compiled on wheezy) is still showing the same behaviour as 2.2.9. lmtp_rcpt_check_quota is apparently ignored, the over-quota defer happens after the DATA phase. # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 [...] lmtp_rcpt_check_quota = yes quota_full_tempfail = yes protocol lmtp { mail_plugins = quota sieve } -> MAIL FROM: <- 250 2.1.0 OK -> RCPT TO: <- 250 2.1.5 OK -> DATA <- 354 OK -> Date: Thu, 29 Jan 2015 13:35:51 +0100 -> To: recipient at domain.de -> From: sender at domain.de -> Subject: test Thu, 29 Jan 2015 13:35:51 +0100 -> X-Mailer: swaks v20120320.0 jetmore.org/john/code/swaks/ -> -> This is a test mailing -> -> . <** 452 4.2.2 Quota exceeded (mailbox for user is full) With 2.2.15 from dovecot-ee it is different. The quota is now checked at RCPT stage # 2.2.15.8 (912dd04d7a14+): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (4b9b9a88ac9b) # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.8 -> MAIL FROM: <- 250 2.1.0 OK -> RCPT TO: <** 552 5.2.2 Quota exceeded (mailbox for user is full) But as you can see, the setting for quota_full_tempfail is now ignored. It should be deferring the message, not rejecting it. When I disable lmtp_rcpt_check_quota it behaves like 2.2.13. Summary: - In 2.2.9 and 2.2.13 lmtp_rcpt_check_quota is ignored - In 2.2.15 lmtp_rcpt_check_quota works, but quota_full_tempfail is ignored at the RCPT TO stage. It still works at the DATA stage, so disabling lmtp_rcpt_check_quota is a workaround falling back to 2.2.13 behaviour Bernhard From devurandom at gmx.net Thu Jan 29 19:39:08 2015 From: devurandom at gmx.net (Dennis Schridde) Date: Thu, 29 Jan 2015 20:39:08 +0100 Subject: mdbox: Broken virtual size for mail UID ... In-Reply-To: <74B04B4F-1AFE-4127-8C1A-EF6AFF3389EC@iki.fi> References: <1549264.4KWotaRmrS@talib> <74B04B4F-1AFE-4127-8C1A-EF6AFF3389EC@iki.fi> Message-ID: <2629778.v0CbVYbaPU@ernie> Hello everyone! I originally asked Timo for help in this case, but I understand he is very busy these days, so I am now posting my questions to this mailinglist. On Sunday 28 Dec 2014 10:02:08 Timo Sirainen wrote: > On 28 Dec 2014, at 07:06, Dennis Schridde wrote: > > My harddrives crashed, and now I get "Broken virtual size for mail UID ?" > > messages in the dovecot logs from some mdbox folders. I assume the logged > > emails are destroyed forever, but I would like to restore as much as > > possible of them and keep them in their corrupted / incomplete state. How > > do I do that? > > > > I guess what I need is some kind of "doveadm fsck" command, that tries its > > best to limit the damage to a minimum: Inform me about the broken emails > > and then ask me whether they should be stored in truncated form, be > > deleted, etc. Is there such a thing? > > http://dovecot.org/tools/mdbox-recover.pl Thank you! >From what I read in the Wiki and how I understand the script, I should proceed like this: #!/bin/bash broken_storage=$PWD/storage recovered_storage=$PWD/storage-recovered cd ${recovered_storage} n=1 for f in ${broken_storage}/m.* ; do mdbox-recover.pl ${f} for f in msg.* ; do mv ${f} m.${n} $((n++)) done done mv ${broken_storage} storage-backup mv ${recovered_storage} storage exit 0 Afterwards dovecot will now automatically regenerate the dovecot.map.index files from the now bare m.* files, and as the dovecot.index files in the actual mailboxes reference the GUID, and the messages in the m.* files have the GUID embedded, nothing will be lost. Is that correct? To create proper assessment of the damage, I need to know a bit more: 1. In the script on line 43, the comment says: # end of metadata block missing, finish the previous mail That seems to be wrong, since idx2!=-1 and state==STATE_BODY on that line, while I understand it should be -1 and STATE_META if we were still reading the metadata block. I guess it should be something like: # still in data block, finish the previous mail 2. Line 47 reads: # truncated / broken data? just keep writing to previous file That is the only error which could be detected in that script, right? 3. A message could also be corrupted by loosing its complete data block from \001\003 to \001\002, which cannot be detected in the script, right? 4. The only case to detect this and find other unexpected oddities would be to compare each msg.* in size and offset with the messages referenced in the dovecot.map.index file, correct? How do I do that, if the dovecot.map.index does not contain the GUID of the message? Do I need to fuzzy match the file_id, offset and size with my current values, or is there a more reliable way to do this? 5. Metadata refers to the RFC822 mail headers? Or are they also included in the body, and the mentioned metadata is dovecot-only metadata? If the latter, do they also contain the size of the body and similar information? Best regards, Dennis From bobber at kc0dxf.net Thu Jan 29 22:15:34 2015 From: bobber at kc0dxf.net (Bobber) Date: Thu, 29 Jan 2015 16:15:34 -0600 Subject: Info on shared folders Message-ID: <54CAB106.2080000@kc0dxf.net> I'm interested in setting up shared folders in a small office environment.? Can a folder be created and shared with other users from an IMAP client such as Thunderbird? -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/ From rs at sys4.de Thu Jan 29 22:42:34 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 29 Jan 2015 23:42:34 +0100 Subject: Info on shared folders In-Reply-To: <54CAB106.2080000@kc0dxf.net> References: <54CAB106.2080000@kc0dxf.net> Message-ID: <54CAB75A.5080006@sys4.de> Am 29.01.2015 um 23:15 schrieb Bobber: > I'm interested in setting up shared folders in a small office > environment.? Can a folder be created and shared with other users from > an IMAP client such as Thunderbird? > > yes https://addons.mozilla.org/de/thunderbird/addon/imap-acl-extension/ Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From mlnospam at yahoo.com Fri Jan 30 08:28:40 2015 From: mlnospam at yahoo.com (ML mail) Date: Fri, 30 Jan 2015 08:28:40 +0000 (UTC) Subject: auth: Error: auth worker: Aborted request: Lookup timed out Message-ID: <82795088.2832314.1422606520115.JavaMail.yahoo@mail.yahoo.com> Hello, I was checking the errors on my mailbox server using "doveadm log errors" and found that around 2-3 times a day I get the following error: Jan 29 22:14:43 auth: Error: auth worker: Aborted request: Lookup timed out Any ideas if this is a bad sign? For example some timeout on my side which is too low or is it just an aborted connection from the side of the IMAP client? The configuration on my Dovecot mailbox server can be found here: http://article.gmane.org/gmane.mail.imap.dovecot/80823 It is a Debian 7 box with standard debian packages. Best regards ML From skdovecot at smail.inf.fh-brs.de Fri Jan 30 08:40:50 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 30 Jan 2015 09:40:50 +0100 (CET) Subject: auth: Error: auth worker: Aborted request: Lookup timed out In-Reply-To: <82795088.2832314.1422606520115.JavaMail.yahoo@mail.yahoo.com> References: <82795088.2832314.1422606520115.JavaMail.yahoo@mail.yahoo.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 30 Jan 2015, ML mail wrote: > I was checking the errors on my mailbox server using "doveadm log errors" and found that around 2-3 times a day I get the following error: > > Jan 29 22:14:43 auth: Error: auth worker: Aborted request: Lookup timed out > > Any ideas if this is a bad sign? For example some timeout on my side which is too low or is it just an aborted connection from the side of the IMAP client? 1st bet is that the SQL query timed out. It is bad sign. > > The configuration on my Dovecot mailbox server can be found here: > > http://article.gmane.org/gmane.mail.imap.dovecot/80823 you should place userdb prefetch before userdb sql. The current order makes no sense. But this is not the source of your problem. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMtDknz1H7kL/d9rAQIgHwf7BluhPtKfdZR31VoVsfyEgteQT0rAXE6r bsv96ToibGO+9gFjk3LM745FFJuFwiBsSCqRIrF4KxXBoI4wBY072JlLwnBjOh/C 57wwvVTw8DIPEkMin0cCOo/OLHXXaGs0h9vcQ86F/m4wnQTyMD1l2YYEdAITpoQM s+nqfzPiYfTSsHCOovEdb2L54VYpgJvTE8BXehdvd2sSpygaCZEV91R1ToxGZypy Y1gdd6YR6AIPiBQXdw0rlLJABiSHGwcV1s7g1xBz2exZguxfBDDuP97OPCU+D+c6 efVVqVoedJMi1311/jIYdtyxUjYUGYZXzsKcBCYuyYPWjQ82MdKyLw== =rbCo -----END PGP SIGNATURE----- From christian.binder at freilassing.de Fri Jan 30 09:26:22 2015 From: christian.binder at freilassing.de (Christian Binder Stadt Freilassing) Date: Fri, 30 Jan 2015 10:26:22 +0100 Subject: Uniqueness of dovecot mailbox-guids In-Reply-To: <17447_1422519994_t0T8QX63004644_19be7073210b3a37aa30b79fcaf796b0@egroupware.freilassing.de> Message-ID: <0199dd697ff34d39d013ab479cd5fed7@egroupware.freilassing.de> Update: seems to me that the old cyrus mailserver already uses these non-unique mailbox guids. Is it necessary to have them unique over all private namespaces? Thanks Christian ----------------urspr?ngliche Nachricht----------------- Von:Christian Binder Stadt Freilassing [christian.binder at freilassing.de ] An:"dovecot dovecot.org" [dovecot at dovecot.org ] Datum:Thu, 29 Jan 2015 09:24:12 +0100 ------------------------------------------------- > Sorry if this is a duplicate message. I'm new in this mailing list and not sure if > the first message really got posted ... > > Dovecot version:2.2.9 > > We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I > defined a master user for the old cyrus mailserver in imapc backend to get access > to all the mailboxes. Now I realized that after migration every user's "Inbox" or > "Trash" on the new dovecot server has the same mailbox-guid, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox dd9456... > trash 5549cc... > > Is this a wanted behaviour? If i do not migrate with doveadm backup and just > manually create mailboxes, the mailbox-guids are globally unique, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox c543fe... > trash t49ccc... > > Thanks! > Christian > From mail at marc-stuermer.de Fri Jan 30 11:15:44 2015 From: mail at marc-stuermer.de (Marc Stuermer) Date: Fri, 30 Jan 2015 11:15:44 +0000 Subject: Troubleshooting doveadm for Solr Configuration In-Reply-To: References: Message-ID: <54CB67E0.3010208@marc-stuermer.de> Am 29.01.2015 um 03:59 schrieb Kevin Laurie: > I am trying to pinpoint where the problem with my dovecot-FTS installation > . I am trying to rule out between the Dovecot and the Solr Module. > I think if the output below looks OK then the dovecot is fine, and will > focus more on the Solr. doveadm(admin at email.net): Debug: fts: Indexes disabled for namespace 'Shared/%u/' Any luck in trying other mailboxes? From christian.binder at freilassing.de Fri Jan 30 10:28:22 2015 From: christian.binder at freilassing.de (Christian Binder Stadt Freilassing) Date: Fri, 30 Jan 2015 11:28:22 +0100 Subject: Uniqueness of dovecot mailbox-guids In-Reply-To: <17447_1422519994_t0T8QX63004644_19be7073210b3a37aa30b79fcaf796b0@egroupware.freilassing.de> Message-ID: <7050ab8b81bf382b498ee0d2c49485dd@egroupware.freilassing.de> If mailbox guids should be unique systemwide, would it be safe to delete all 'dovecot-uidlist' files of the user's mailboxes and then do a doveadm force-resync after mailbox migration? Thanks! Christian ----------------urspr?ngliche Nachricht----------------- Von: Christian Binder Stadt Freilassing [christian.binder at freilassing.de ] An: "dovecot dovecot.org" [dovecot at dovecot.org ] Datum: Thu, 29 Jan 2015 09:24:12 +0100 ------------------------------------------------- > Sorry if this is a duplicate message. I'm new in this mailing list and not sure if > the first message really got posted ... > > Dovecot version: 2.2.9 > > We're using doveadm backup to migrate user mailboxes from cyrus to dovecot. I > defined a master user for the old cyrus mailserver in imapc backend to get access > to all the mailboxes. Now I realized that after migration every user's "Inbox" or > "Trash" on the new dovecot server has the same mailbox-guid, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox dd9456... > trash 5549cc... > > Is this a wanted behaviour? If i do not migrate with doveadm backup and just > manually create mailboxes, the mailbox-guids are globally unique, e.g: > > User a: > inbox dd9456... > trash 5549cc... > > User b: > inbox c543fe... > trash t49ccc... > > Thanks! > Christian > From mlnospam at yahoo.com Fri Jan 30 10:36:21 2015 From: mlnospam at yahoo.com (ML mail) Date: Fri, 30 Jan 2015 10:36:21 +0000 (UTC) Subject: auth: Error: auth worker: Aborted request: Lookup timed out In-Reply-To: References: Message-ID: <781536044.2880842.1422614181597.JavaMail.yahoo@mail.yahoo.com> Do you know what is Dovecot's default SQL query time out and with which parameter can I maybe increase that timeout? Thanks for the hint regarding the userdb prefetch, that makes sense and I have now put the userdb prefetch before the userdb SQL as recommended. Doing that I also noticed that in my /etc/dovecot/conf.d/10-auth.conf I still have the system user authentication enabled with this line: !include auth-system.conf.ext although I am using SQL only (PostgreSQL). Can I safely disable this line you think? On Friday, January 30, 2015 9:45 AM, Steffen Kaiser wrote: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 30 Jan 2015, ML mail wrote: > I was checking the errors on my mailbox server using "doveadm log errors" and found that around 2-3 times a day I get the following error: > > Jan 29 22:14:43 auth: Error: auth worker: Aborted request: Lookup timed out > > Any ideas if this is a bad sign? For example some timeout on my side which is too low or is it just an aborted connection from the side of the IMAP client? 1st bet is that the SQL query timed out. It is bad sign. > > The configuration on my Dovecot mailbox server can be found here: > > http://article.gmane.org/gmane.mail.imap.dovecot/80823 you should place userdb prefetch before userdb sql. The current order makes no sense. But this is not the source of your problem. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVMtDknz1H7kL/d9rAQIgHwf7BluhPtKfdZR31VoVsfyEgteQT0rAXE6r bsv96ToibGO+9gFjk3LM745FFJuFwiBsSCqRIrF4KxXBoI4wBY072JlLwnBjOh/C 57wwvVTw8DIPEkMin0cCOo/OLHXXaGs0h9vcQ86F/m4wnQTyMD1l2YYEdAITpoQM s+nqfzPiYfTSsHCOovEdb2L54VYpgJvTE8BXehdvd2sSpygaCZEV91R1ToxGZypy Y1gdd6YR6AIPiBQXdw0rlLJABiSHGwcV1s7g1xBz2exZguxfBDDuP97OPCU+D+c6 efVVqVoedJMi1311/jIYdtyxUjYUGYZXzsKcBCYuyYPWjQ82MdKyLw== =rbCo -----END PGP SIGNATURE----- From info at netocean.de Fri Jan 30 14:03:09 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Fri, 30 Jan 2015 15:03:09 +0100 Subject: LDAP Quota Limits & Warnings Message-ID: <54CB8F1D.5030903@netocean.de> I found this on the dovecot wiki: The following limit names are supported: * *storage*: Quota limit in kilobytes, 0 means unlimited. * *bytes*: Quota limit in bytes, 0 means unlimited. * *messages*: Quota limit in number of messages, 0 means unlimited. This probably isn't very useful. * *backend*: Quota backend-specific limit configuration. * *ignore*: Don't include the specified mailbox in quota at all. As I understood it, all 5 limit categories are valid to use with maildir format while only "storage" and "messages" are valid to use with dmbox format, right? Also the "ignore" limit may appear more than one time for a user - like Trash:ignore and Spam:ignore, right? So I assumed following should work - and it does - except for "ignore" - even though LDAP does not hold multiple values for it - only a single one for each limit: user_attrs = [...] mailQuotaStorage=quota_rule=*:storage=%$, mailQuotaMessages=quota_rule2=*:messages=%$, mailQuotaIgnore=quota_rule3=*:ignore=%$ pass_attrs = [...] mailQuotaStorage=userdb_quota_rule=*:storage=%$, mailQuotaMessages=userdb_quota_rule2=*:messages=%$, mailQuotaIgnore=userdb_quota_rule3=*:ignore=%$ And unfortunately /var/log/maillog tells me: dovecot: imap(abuse at netocean.local): Error: user abuse at netocean.local: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:ignore=Trash: Unknown rule limit name: ignore Yet /var/log/debug.log sounds a bit more promising ... but then again not really ... dovecot: imap: Debug: Added userdb setting: mail=maildir:/usr/local/www/default/mail/user at mydomain.tld dovecot: imap: Debug: Added userdb setting: plugin/quota_rule=*:storage=1048576 dovecot: imap: Debug: Added userdb setting: plugin/quota_rule2=*:messages=10 dovecot: imap: Debug: Added userdb setting: plugin/quota_rule3=*:ignore=Trash dovecot: imap(abuse at netocean.local): Debug: Effective uid=2110, gid=2110, home=/var/vmail/user at mydomain.tld/mail dovecot: imap(abuse at netocean.local): Debug: Quota root: name=User quota backend=maildir args= dovecot: imap(abuse at netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 dovecot: imap(abuse at netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=10 dovecot: imap(abuse at netocean.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=10 Additionally, if one of the two working quotas (storage & messages) is exceeded than Dovecot sends out a warning from this setting: plugin { quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u # What message to send to IMAP clients (and SMTP senders) when quota is exceeded? quota_exceeded_message = Quota exceeded, please go to http://MyDomain.TLD for instructions on how to fix this. } No my question is how to provide two different warnings. One for storage like: Sorry, you're mail storage is full. Please delete some. And a second for messages like: Sorry, you've too many messages. Please delete some. Thanks & Best Regards Leander From ya.mwork at yandex.ru Fri Jan 30 15:03:03 2015 From: ya.mwork at yandex.ru (Evgeny Basov) Date: Fri, 30 Jan 2015 18:03:03 +0300 Subject: Domain quota Message-ID: <54CB9D27.6060907@yandex.ru> Hello. I'm tried to use domain quota: plugin { quota = dict:Domain quota:%d:redis:host=127.0.0.1:prefix=domain/ quota_rule = *:storage=0 quota2 = dict:User quota::redis:host=127.0.0.1:prefix=user/ quota2_rule = *:storage=0 quota3_rule = Trash:storage=+5%% } but got next problem with it. Init state: # doveadm quota get -u "testacc at m.tld" Quota name Type Value Limit % Domain quota STORAGE 27070 204800 13 Domain quota MESSAGE 21 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 # doveadm quota get -u "testacc2 at m.tld" Quota name Type Value Limit % Domain quota STORAGE 27070 204800 13 Domain quota MESSAGE 21 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 Make recalc: # doveadm quota recalc -u "testacc2 at m.tld" and got: # doveadm quota get -u "testacc2 at m.tld" Quota name Type Value Limit % Domain quota STORAGE 2 204800 0 Domain quota MESSAGE 3 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 # doveadm quota get -u "testacc at m.tld" Quota name Type Value Limit % Domain quota STORAGE 2 204800 0 Domain quota MESSAGE 3 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 All values of all domain quota are equals. Execute recalc one more time: # doveadm quota recalc -u "testacc at m.tld" # doveadm quota get -u "testacc at m.tld" Quota name Type Value Limit % Domain quota STORAGE 27068 204800 13 Domain quota MESSAGE 18 - 0 User quota STORAGE 27068 - 0 User quota MESSAGE 18 - 0 # doveadm quota get -u "testacc2 at m.tld" Quota name Type Value Limit % Domain quota STORAGE 27068 204800 13 Domain quota MESSAGE 18 - 0 User quota STORAGE 2 - 0 User quota MESSAGE 3 - 0 How can I get fast realtime changed domain quota? Used dovecot 2.2.13. From listas at adminlinux.com.br Fri Jan 30 15:58:45 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Fri, 30 Jan 2015 13:58:45 -0200 Subject: Upstart script for Poolmon Message-ID: <54CBAA35.3090207@adminlinux.com.br> Hi My servers are Ubuntu 14.04 and I'm needing to make a Upstart script for Poolmon. Does someone already made this and could to share with us ? Thanks! -- Thiago henrique From info at netocean.de Fri Jan 30 21:21:29 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Fri, 30 Jan 2015 22:21:29 +0100 Subject: LDAP: "unused" & "uid missing" Message-ID: <54CBF5D9.4010103@netocean.de> Does someone know what this means? It looks to me like it got the UID, yet it complains ... ? Also, why does it complain about attributes being unused, yet I use prefetch, so they will be used?! Log: [...] dovecot: auth: Debug: ldap(test at mydomain.tld,192.168.10.233,): result: mailStorageDirectory=/var/vmail mailUidNumber=2110 mailGidNumber=2110 mailQuotaStorage=10240 mailQuotaMessages=20; mailQuotaMessages,mailStorageDirectory,mailUidNumber,mailQuotaStorage,mailGidNumber unused dovecot: auth: Debug: ldap(test at mydomain.tld,192.168.10.233,): result: mailStorageDirectory=/var/vmail mailUidNumber=2110 mailGidNumber=2110 mailQuotaStorage=10240 mailQuotaMessages=20; uid missing [...] As mentioned, I use prefetch, so this is how my dovecot/dovecot-ldap.conf.ext is looking: [...] #user_attrs = mailStorageDirectory=home=%$/%u, mailStorageDirectory=mail=maildir:%$/%u/maildir, mailUidNumber=uid, mailGidNumber=gid, mailQuotaStorage=quota_rule=*:storage=%$, mailQuotaMessages=quota_rule2=*:messages=%$ pass_attrs = uid=user, userPassword=password, mailStorageDirectory=userdb_home=%$/%u, mailStorageDirectory=userdb_mail=maildir:%$/%u/maildir, mailUidNumber=userdb_uid, mailGidNumber=userdb_gid, mailQuotaStorage=userdb_quota_rule=*:storage=%$, mailQuotaMessages=userdb_quota_rule2=*:messages=%$ [...] Thanks From mlnospam at yahoo.com Sat Jan 31 14:21:00 2015 From: mlnospam at yahoo.com (ML mail) Date: Sat, 31 Jan 2015 14:21:00 +0000 (UTC) Subject: auth: Error: auth worker: Aborted request: Lookup timed out In-Reply-To: <82795088.2832314.1422606520115.JavaMail.yahoo@mail.yahoo.com> References: <82795088.2832314.1422606520115.JavaMail.yahoo@mail.yahoo.com> Message-ID: <1071171846.199501.1422714061075.JavaMail.yahoo@mail.yahoo.com> I don't know if this is related but I also get quite a few of these error messages: Jan 31 14:10:46 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer Jan 31 14:13:20 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer Jan 31 14:13:20 auth: Warning: auth client 0 disconnected with 2 pending requests: Connection reset by peer Jan 31 14:13:22 auth: Warning: auth client 0 disconnected with 2 pending requests: EOF Jan 31 14:13:26 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer maybe around 20-30 per day. On Friday, January 30, 2015 9:28 AM, ML mail wrote: Hello, I was checking the errors on my mailbox server using "doveadm log errors" and found that around 2-3 times a day I get the following error: Jan 29 22:14:43 auth: Error: auth worker: Aborted request: Lookup timed out Any ideas if this is a bad sign? For example some timeout on my side which is too low or is it just an aborted connection from the side of the IMAP client? The configuration on my Dovecot mailbox server can be found here: http://article.gmane.org/gmane.mail.imap.dovecot/80823 It is a Debian 7 box with standard debian packages. Best regards ML From alvin.sm at Mail.Linux-Consulting.com Sat Jan 31 19:40:18 2015 From: alvin.sm at Mail.Linux-Consulting.com (alvin) Date: Sat, 31 Jan 2015 11:40:18 -0800 (PST) Subject: auth: Error: auth worker: Aborted request: Lookup timed out In-Reply-To: <1071171846.199501.1422714061075.JavaMail.yahoo@mail.yahoo.com> Message-ID: <201501311940.t0VJeIsm015171@Mail.Linux-Consulting.com> hi > I don't know if this is related but I also get quite a few of these error messages: > > Jan 31 14:10:46 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer my interpretation: a) you have a very very bz network ... the other end cannot respond to incoming requests even if your network is NOT bz, eg, no collision blinking on your hubs/switches, you are still having network problems b) if all of your dovecot tests is on one host ... disconnect it from the network and see if dovecot's auth finishes its tasks c) to clean up your network ... - use switches ... not hubs .... even inexpensive netgear switches is good enuff - use good 3-6' cat6e cables ... we'll assume the bldg's wiring is done to bldg specs - my guess, you're probably having cabling problems ) - separate slow devices from faster devices eg. separate printers onto its own network with a switch in between printers and everybody else - separate 10/100 devices from gigE devices ... do not mix them up on the same switch/hub c ya alvin > Jan 31 14:13:20 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer > Jan 31 14:13:20 auth: Warning: auth client 0 disconnected with 2 pending requests: Connection reset by peer > Jan 31 14:13:22 auth: Warning: auth client 0 disconnected with 2 pending requests: EOF > Jan 31 14:13:26 auth: Warning: auth client 0 disconnected with 1 pending requests: Connection reset by peer > > maybe around 20-30 per day. > From info at netocean.de Mon Jan 26 12:58:52 2015 From: info at netocean.de (=?UTF-8?B?TGVhbmRlciBTY2jDpGZlcg==?=) Date: Mon, 26 Jan 2015 12:58:52 -0000 Subject: imap-login: Fatal: pipe() failed: Too many open files In-Reply-To: <54C6222B.9080100@netocean.de> References: <54C58F43.7020803@netocean.de> <54C594B0.4030204@netocean.de> <54C5966F.9050601@thelounge.net> <54C61684.1000405@netocean.de> <54C6222B.9080100@netocean.de> Message-ID: <54C639C3.3060806@netocean.de> It turns out, that this errors "too many open files ..." occurred because a) missing "index ATTRIBUTE eq" for username and password in slapd.conf as well as b) password attribute was not read properly by dovecot from LDAP - I had to remove it from my schema and instead use "userPassword" from "objectclass: simpleSecurityObject" This two changes made the "too many open files ..." errors disapear. BUT, I still have the problem of receiving all those "dovecot: auth: Error:" lines in my maillog. I know I could turn them off by reducing debug parameters - but why is it declared as error in the first place - even though user authentication is working fine? Thanks & Best Regards, Leander Am 26.01.15 um 12:16 schrieb Leander Sch?fer: > I also keep on getting funny errors of dovecot EVEN THOUGH I > authenticated successfully through SMTP (Dovecto SASL) ?!?! > > Just for debugging this entire issue I set those parameters: > > debug_level = -1 > auth_verbose = yes > auth_verbose_passwords = yes > auth_debug = yes > auth_debug_passwords = yes > mail_debug = yes > verbose_ssl = yes > > > > > cat /var/loca/maillog > > > Jan 26 12:08:22 WM-01 postfix/postfix-script[97931]: starting the > Postfix mail system > Jan 26 12:08:22 WM-01 postfix/master[97933]: daemon started -- version > 1.0, configuration /usr/local/etc/postfix > Jan 26 12:08:22 WM-01 dovecot: master: Dovecot v2.2.15 starting up for > imap, pop3, lmtp > Jan 26 12:08:59 WM-01 postfix/smtpd[99014]: connect from > localhost[127.0.0.1] > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_extended_operation_s > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_extended_operation > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_initial_request > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_new_connection 1 1 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_open_connection > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_connect_to_host: TCP > 127.0.0.1:389 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_new_socket: 32 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_prepare_socket: 32 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_connect_to_host: > Trying 127.0.0.1:389 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_pvt_connect: fd: 32 > tm: -1 async: 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: attempting to connect: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: connect success > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_open_defconn: successful > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_server_request > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid 1 (infinite timeout) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid 1 all 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:08:59 2015 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: * msgid 1, origid 1, > status InProgress > Jan 26 12:08:59 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid 1 all 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 1 all 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 1 message type extended-result > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 > new referrals > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: mark request > completed, ld 0x1245e100 msgid 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: request done: ld > 0x1245e100 msgid 1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: res_errno: 0, res_error: > <>, res_matched: <> > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_free_request (origid > 1, msgid 1) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_extended_result > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_result > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_msgfree > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:before/connect initialization > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv2/v3 write client hello A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read server hello A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS certificate > verification: depth: 1, err: 0, subject: [...] > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS certificate > verification: depth: 0, err: 0, subject: [...] > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read server certificate A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read server certificate request A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read server done A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 write client certificate A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 write client key exchange A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 write certificate verify A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 write change cipher spec A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 write finished A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 flush data > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read server session ticket A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: TLS trace: > SSL_connect:SSLv3 read finished A > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_bind > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_simple_bind > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_sasl_bind > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_initial_request > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_send_server_request > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:08:59 2015 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: * msgid 2, origid 2, > status InProgress > Jan 26 12:08:59 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid -1 all 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 2 message type bind > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 > new referrals > Jan 26 12:08:59 WM-01 dovecot: auth: Error: read1msg: mark request > completed, ld 0x1245e100 msgid 2 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: request done: ld > 0x1245e100 msgid 2 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: res_errno: 0, res_error: > <>, res_matched: <> > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_free_request (origid > 2, msgid 2) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_parse_result > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_msgfree > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: refcnt: 1 status: > Connected > Jan 26 12:08:59 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:08:59 2015 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 0 (abandoned 0) > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:08:59 WM-01 dovecot: auth: Error: Empty > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:08:59 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:08:59 WM-01 postfix/smtpd[99014]: Anonymous TLS connection > established from localhost[127.0.0.1]: TLSv1.2 with cipher > ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_search > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: > "(&(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=postmaster at mydomain.tld))" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: AND > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter_list > "(objectClass=mailAccount)(mailAccountStatus=active)(mailAddress=postmaster at mydomain.tld)" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: > "(objectClass=mailAccount)" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: > "objectClass=mailAccount" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: > "(mailAccountStatus=active)" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: > "mailAccountStatus=active" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: > "(mailAddress=postmaster at mydomain.tld)" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_filter: simple > Jan 26 12:09:09 WM-01 dovecot: auth: Error: put_simple_filter: > "mailAddress=postmaster at mydomain.tld" > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_build_search_req > ATTRS: mailAddress > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_initial_request > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_server_request > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:09:09 2015 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 3, origid 3, > status InProgress > Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 3 message type search-entry > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:09:09 2015 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 3, origid 3, > status InProgress > Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 3 message type search-result > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 > new referrals > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: mark request > completed, ld 0x1245e100 msgid 3 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: request done: ld > 0x1245e100 msgid 3 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: res_errno: 0, res_error: > <>, res_matched: <> > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_free_request (origid > 3, msgid 3) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_first_attribute > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_values > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_next_attribute > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_first_attribute > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_values > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_next_attribute > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_get_dn > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_bind > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_simple_bind > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_sasl_bind > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_initial_request > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_send_server_request > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:09:09 2015 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 4, origid 4, > status InProgress > Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 2 status: > Connected > Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:09:09 2015 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * msgid 4, origid 4, > status InProgress > Jan 26 12:09:09 WM-01 dovecot: auth: Error: outstanding referrals > 0, parent count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 1 (abandoned 0) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 > msgid 4 message type bind > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: ld 0x1245e100 0 > new referrals > Jan 26 12:09:09 WM-01 dovecot: auth: Error: read1msg: mark request > completed, ld 0x1245e100 msgid 4 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: request done: ld > 0x1245e100 msgid 4 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: res_errno: 0, res_error: > <>, res_matched: <> > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_free_request (origid > 4, msgid 4) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_parse_result > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_msgfree > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_result ld 0x1245e100 > msgid -1 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg ld 0x1245e100 > msgid -1 (timeout 0 usec) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: wait4msg continue ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Connections: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: * host: 127.0.0.1 port: > 389 (default) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: refcnt: 1 status: > Connected > Jan 26 12:09:09 WM-01 dovecot: auth: Error: last used: Mon Jan 26 > 12:09:09 2015 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 > Outstanding Requests: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 request > count 0 (abandoned 0) > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ** ld 0x1245e100 Response > Queue: > Jan 26 12:09:09 WM-01 dovecot: auth: Error: Empty > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ld 0x1245e100 response > count 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList ld > 0x1245e100 msgid -1 all 0 > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_chkResponseList > returns ld 0x1245e100 NULL > Jan 26 12:09:09 WM-01 dovecot: auth: Error: ldap_int_select > Jan 26 12:09:20 WM-01 postfix/smtpd[99014]: lost connection after AUTH > from localhost[127.0.0.1] > Jan 26 12:09:20 WM-01 postfix/smtpd[99014]: disconnect from > localhost[127.0.0.1] > > > > > > > > > > > Am 26.01.15 um 11:27 schrieb Leander Sch?fer: >> Am 26.01.15 um 02:24 schrieb Edgar Pettijohn: >>> Sorry didn't scroll to the bottom to see the dovecot -n. I'm >>> assuming freebsd has an /etc/login.conf similiar to openbsd. If so >>> you may need to do something similiar to this: >>> >>> dovecot:\ >>> :openfiles-cur=512:\ >>> :openfiles-max=2048:\ >>> :tc=daemon: >>> >>> Rebuild the login.conf.db file if necessary: >>> >>> # [ -f /etc/login.conf.db ] && cap_mkdb /etc/login.conf >>> >>> Hope this helps. >>> >> Thanks for the hint. FreeBSD takes sysctl(8) settings as a base to >> ulimit(3). I even tried tuning on those values. >> >> sysctl kern.maxfilesperproc=405587 >> sysctl kern.maxfiles=428433 >> >> Unfortunately this does not solve the problem. I don't think it is >> the root of the problem. >> >> The current value - even with Dovecot and Postfix already running is: >> kern.openfiles: 675 >> It is just, that Dovecot is kind of in a frozen condition when trying >> to login to it via telnet(1). >> >> >> Am 26.01.15 um 02:20 schrieb Reindl Harald: >>> >>> Am 26.01.2015 um 02:13 schrieb Leander Sch?fer: >>>> I just checked my ulimit again and it really seems like it has more >>>> than enough - so I still don't understand what I've configured wrong >>>> here ;/ >>>> >>>> root at WM-01 [~]$ su -m dovecot -c "ulimit -a" >>>> socket buffer size (bytes, -b) unlimited >>>> core file size (blocks, -c) unlimited >>>> data seg size (kbytes, -d) 33554432 >>>> file size (blocks, -f) unlimited >>>> max locked memory (kbytes, -l) 131072 >>>> max memory size (kbytes, -m) 7067352 >>>> open files (-n) 205587 >>>> pipe size (512 bytes, -p) 1 >>>> stack size (kbytes, -s) 8192 >>>> cpu time (seconds, -t) unlimited >>>> max user processes (-u) 11278 >>>> virtual memory (kbytes, -v) unlimited >>>> swap size (kbytes, -w) unlimited >>> >>> besides that "imap-login" typically does *not* run under the same >>> user (here dovenull versus dovecot) who tells you that 205587 is >>> "more than enough" just because it is a high value? >> Yes, because I am monitoring those values via Munin. I can clearly >> see that my system does not require a value even close to this. By >> average my system has 673 files open. In FreeBSD sysctl(8) is the >> base for ulimit(3) - meaning since I don't maintain user specific >> ulimit(3) - it is also the same values for dovenull, postfix, vmail >> and every other user on the system. My current sysctl(8) values >> should provide way enough space to grow super large: >> >> sysctl kern.maxfilesperproc=205587 >> sysctl kern.maxfiles=228433 >> >> >> And this is where it becomes so absurd to me. I don't see where I >> should tune anymore. I more and more get the feeling it is a >> configuration error of dovecot itself?! >> >> >> Thanks & Best Regards, >> Leander >>> >>> >>> From mfoley at novatec-inc.com Tue Jan 27 03:30:51 2015 From: mfoley at novatec-inc.com (Mark Foley) Date: Tue, 27 Jan 2015 03:30:51 -0000 Subject: Howto NTML Message-ID: <201501270329.t0R3TDh4016706@server.novatec-inc.com> Has anyone gotten NTLM working with Dovecot and Outlook? I have a Samba4 domain controller / active directory running just fine on Linux Slackware64 14.1. PLAIN authenticiation works just fine if I create /etc/passwd accounts for the domain users. I've tried for over two weeks to get NTLM working from Outlook 2007 on a domain workstation without success. I end up with various permission errors and ultimately procmail won't deliver if I "adjust" mailbox folder permissions too liberally. Also it seems I have to log in each time in Outlook. If someone has made this work, I'd be very interesting in seeing the doveconf -n output, the permissions on the mailbox directories, the nsswitch.conf, and anything else you had to tweak to get this working. I'd really love to have AD authentication working on this setup! --Mark From superinterstellar at gmail.com Thu Jan 29 08:43:32 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 29 Jan 2015 16:43:32 +0800 Subject: Indexing Mail faster In-Reply-To: References: <20150124190408.GA436@nihlus.leuxner.net> Message-ID: Hi Micheal, Here is how it should look:- ~# telnet localhost imap Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. . login user at domain.net password . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE BINARY MOVE SEARCH=FUZZY QUOTA] Logged in . select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft $Forwarded \*)] Flags permitted. * 49319 EXISTS * 12 RECENT * OK [UNSEEN 46791] First unseen. * OK [UIDVALIDITY 1414214135] UIDs valid * OK [UIDNEXT 106554] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed (0.128 secs). . search text hello * OK Searched 6% of the mailbox, ETA 2:34 Finally at the end you logout by . logout On Thu, Jan 29, 2015 at 4:39 PM, Kevin Laurie wrote: > Dear Michael, > You will need to telnet locally to imap. So run the following commands in > your server > > 1. telnet localhost imap > 2. . login email at domain.com password > 3. . select inbox > 4. . search text "passport" (This command here will search your inbox and > give you the time needed) > 5. . logout > > Notice the period(.) You need to use it . > > Hope this is clear. > Regards > Kevin > > > On Thu, Jan 29, 2015 at 4:32 PM, M. Koehler > wrote: > >> Hi Kevin, >> >> I want to test it with my dovecot but I don?t know how to measure it. >> Could you please provide my the configuration/terminal commands to get an >> reliable output? I just have mailboxes up to 4GB in VM on a proxmox server >> (Debian Wheezy at the moment). >> >> regards, >> Michael >> >> >> >> > Am 29.01.2015 um 06:29 schrieb Kevin Laurie < >> superinterstellar at gmail.com>: >> > >> > Dear Peter, >> > Noted. Thanks for your input. Appreciate it. >> > At this point my urgent most priority is to get FTS. Waiting 3 mins for >> a >> > body search email is bad(but of course my email box is large) >> > I need to have this sorted out by today as I have been putting this off >> for >> > too long mostly because of lack of troubleshooting documentation >> online(if >> > i do figure this out will create some for guidance) >> > Regards >> > Kevin >> > >> > On Thu, Jan 29, 2015 at 1:18 PM, Peter Hodur >> wrote: >> > >> >> >> >> >> >> On Thursday, January 29, 2015, Kevin Laurie < >> superinterstellar at gmail.com> >> >> wrote: >> >> >> >>> Dear Peter, >> >>> Oh. Sorry(didnt know you were addressing someone else) my apologies. >> But >> >>> as you can see , I am desperately trying to address this issue. >> >>> >> >> >> >> No problem ;))) i wrote because my search result is good but not so >> >> good as someone wrote here couple of days ago. >> >> >> >> >> >> >> >>> I have 1 disk with 200GB running on a VPS. >> >>> The file system is ext4. >> >>> >> >> >> >> Im not dovecot expert, but i think this is your problem. You need more >> >> iops. If you need performance, VPS may be (if connected to fast SAN) >> but >> >> basicly is not a good choice. >> >> >> >> >> >>> I think I will need to implement FTS to fix this as I will need body >> >>> searches. >> >>> >> >> >> >> FTS could be a solution. You are right. >> >> >> >> >> >>> Have you tried FTS before? >> >>> >> >> >> >> Im sorry, never :( >> >> >> >> But may be someone more skilled than me could answer the main question, >> >> how much dovecot index helps in case imap search agains body. >> >> >> >> Im not sure, but my opinion is that it helps little (may be not at all) >> >> >> >> Ps: may be you use it, but do not forget to setup delivering from mail >> >> server via LDA or LMTP instead of direct to maildirs. This is better >> >> because dovecot updates your indexes at the time of message arrival. >> Later >> >> accesses should be faster. >> >> >> >> Peter >> >> >> >> >> >> >> >>> >> >>> >> >>> Regards >> >>> Kevin >> >>> >> >> >> >>> >> >>> On Thu, Jan 29, 2015 at 12:55 PM, Peter Hodur >> >>> wrote: >> >>> >> >>>> Kevin, >> >>>> >> >>>> My message was not addressed to you ;) because my results are NOT so >> >>>> good as someone wrote it here. I can fulltext cca 8k messages in 7-8 >> >>>> seconds. >> >>>> >> >>>> Someone wrote that he has results cca 22k messages in 4 seconds :( >> >>>> >> >>>> Im not sure, but the answer is may be in the index and disk >> subsystem. >> >>>> >> >>>> If and only if dovecot index does not have saved in the index >> keywords >> >>>> from BODY of messages the problem is your/my disk subsystem. >> >>>> >> >>>> Generally, messages from mailing list like this are pretty small. The >> >>>> problem is that dovecot must read all messages. If you have maildir >> its >> >>>> mean open and read many files. >> >>>> >> >>>> But my test search was agains archive of my personal inbox - so not >> onoy >> >>>> small messages like this but also messages with big attachments etc. >> >>>> >> >>>> If dovecot index only headers, than it in case of full text search >> read >> >>>> whole messages - mime parts allows that plain text body can be after >> >>>> attachment etc. >> >>>> >> >>>> And here is it all about iops and throughput. >> >>>> >> >>>> How many disks you have? And what setup? >> >>>> >> >>>> Generally, the only good setup is many smaller disks in RAID 10 >> >>>> (stripped mirrors) and if your fs allows it, with added read cache - >> in >> >>>> case of zfs: l2arc on ssd. >> >>>> >> >>>> >> >>>> >> >>>> Pete >> >>>> >> >>>> >> >>>> >> >>>> On Thursday, January 29, 2015, Kevin Laurie < >> superinterstellar at gmail.com> >> >>>> wrote: >> >>>> >> >>>>> Hi Peter, >> >>>>> Sorry, >> >>>>> >> >>>>> I think its maildir >> >>>>> Output of my dovecot -n is listed below:- >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> # 2.2.9: /etc/dovecot/dovecot.conf >> >>>>> # OS: Linux 3.10.62-xenU-25-0e6777a-x86_64 x86_64 Ubuntu 14.04.1 LTS >> >>>>> auth_master_user_separator = * >> >>>>> auth_mechanisms = PLAIN LOGIN >> >>>>> dict { >> >>>>> acl = mysql:/etc/dovecot/dovecot- >> >>>>> share-folder.conf >> >>>>> quotadict = mysql:/etc/dovecot/dovecot-used-quota.conf >> >>>>> } >> >>>>> first_valid_uid = 2000 >> >>>>> last_valid_uid = 2000 >> >>>>> listen = * >> >>>>> log_path = /var/log/dovecot.log >> >>>>> mail_debug = yes >> >>>>> mail_gid = 2000 >> >>>>> mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ >> >>>>> mail_plugins = quota fts >> >>>>> mail_uid = 2000 >> >>>>> managesieve_notify_capability = mailto >> >>>>> managesieve_sieve_capability = fileinto reject envelope >> >>>>> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational >> >>>>> regex imap4flags copy include variables body enotify environment >> mailbox >> >>>>> date ihave >> >>>>> namespace { >> >>>>> inbox = yes >> >>>>> location = >> >>>>> mailbox Drafts { >> >>>>> auto = subscribe >> >>>>> special_use = \Drafts >> >>>>> } >> >>>>> mailbox Junk { >> >>>>> auto = subscribe >> >>>>> special_use = \Junk >> >>>>> } >> >>>>> mailbox Sent { >> >>>>> auto = subscribe >> >>>>> special_use = \Sent >> >>>>> } >> >>>>> mailbox "Sent Messages" { >> >>>>> auto = no >> >>>>> special_use = \Sent >> >>>>> } >> >>>>> mailbox Spam { >> >>>>> auto = no >> >>>>> special_use = \Junk >> >>>>> } >> >>>>> mailbox Trash { >> >>>>> auto = subscribe >> >>>>> special_use = \Trash >> >>>>> } >> >>>>> prefix = >> >>>>> separator = / >> >>>>> type = private >> >>>>> } >> >>>>> namespace { >> >>>>> list = children >> >>>>> location = maildir:/%%Lh/Maildir/:INDEX=/%%Lh/Maildir/Shared/%%u >> >>>>> prefix = Shared/%%u/ >> >>>>> separator = / >> >>>>> subscriptions = yes >> >>>>> type = shared >> >>>>> } >> >>>>> passdb { >> >>>>> args = /etc/dovecot/dovecot-mysql.conf >> >>>>> driver = sql >> >>>>> } >> >>>>> passdb { >> >>>>> args = /etc/dovecot/dovecot-master-users-password >> >>>>> driver = passwd-file >> >>>>> master = yes >> >>>>> } >> >>>>> plugin { >> >>>>> acl = vfile >> >>>>> acl_shared_dict = proxy::acl >> >>>>> auth_socket_path = /var/run/dovecot/auth-master >> >>>>> quota = dict:user::proxy::quotadict >> >>>>> quota_rule = *:storage=1G >> >>>>> quota_warning = storage=85%% quota-warning 85 %u >> >>>>> quota_warning2 = storage=90%% quota-warning 90 %u >> >>>>> quota_warning3 = storage=95%% quota-warning 95 %u >> >>>>> sieve = /%Lh/sieve/dovecot.sieve >> >>>>> sieve_default = /var/vmail/sieve/dovecot.sieve >> >>>>> sieve_dir = /%Lh/sieve >> >>>>> sieve_global_dir = /var/vmail/sieve >> >>>>> } >> >>>>> protocols = pop3 imap sieve lmtp >> >>>>> service auth { >> >>>>> unix_listener /var/spool/postfix/private/dovecot-auth { >> >>>>> group = postfix >> >>>>> mode = 0666 >> >>>>> user = postfix >> >>>>> } >> >>>>> unix_listener auth-master { >> >>>>> group = vmail >> >>>>> mode = 0666 >> >>>>> user = vmail >> >>>>> } >> >>>>> unix_listener auth-userdb { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> service dict { >> >>>>> unix_listener dict { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> service imap-login { >> >>>>> process_limit = 500 >> >>>>> service_count = 1 >> >>>>> } >> >>>>> service lmtp { >> >>>>> executable = lmtp -L >> >>>>> inet_listener lmtp { >> >>>>> port = 24 >> >>>>> } >> >>>>> process_min_avail = 5 >> >>>>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> >>>>> group = postfix >> >>>>> mode = 0600 >> >>>>> user = postfix >> >>>>> } >> >>>>> user = vmail >> >>>>> } >> >>>>> service pop3-login { >> >>>>> service_count = 1 >> >>>>> } >> >>>>> service quota-warning { >> >>>>> executable = script /usr/local/bin/dovecot-quota-warning.sh >> >>>>> unix_listener quota-warning { >> >>>>> group = vmail >> >>>>> mode = 0660 >> >>>>> user = vmail >> >>>>> } >> >>>>> } >> >>>>> ssl = required >> >>>>> ssl_cert = > >>>>> ssl_key = > >>>>> userdb { >> >>>>> args = /etc/dovecot/dovecot-mysql.conf >> >>>>> driver = sql >> >>>>> } >> >>>>> protocol lda { >> >>>>> auth_socket_path = /var/run/dovecot/auth-master >> >>>>> lda_mailbox_autocreate = yes >> >>>>> log_path = /var/log/dovecot-sieve.log >> >>>>> mail_plugins = quota fts sieve >> >>>>> postmaster_address = root >> >>>>> } >> >>>>> protocol lmtp { >> >>>>> info_log_path = /var/log/dovecot-lmtp.log >> >>>>> lmtp_save_to_detail_mailbox = yes >> >>>>> mail_plugins = quota sieve >> >>>>> postmaster_address = postmaster >> >>>>> recipient_delimiter = + >> >>>>> } >> >>>>> protocol imap { >> >>>>> imap_client_workarounds = tb-extra-mailbox-sep >> >>>>> mail_plugins = quota fts imap_quota >> >>>>> } >> >>>>> protocol pop3 { >> >>>>> mail_plugins = quota fts >> >>>>> pop3_client_workarounds = outlook-no-nuls oe-ns-eoh >> >>>>> pop3_uidl_format = %08Xu%08Xv >> >>>>> } >> >>>>> >> >>>>> >> >>>>> On Thu, Jan 29, 2015 at 12:37 PM, Kevin Laurie < >> >>>>> superinterstellar at gmail.com> wrote: >> >>>>> >> >>>>>> Dear Peter, >> >>>>>> >> >>>>>> My inbox is MDA_external >> >>>>>> Storage: 17GB of 24GB >> >>>>>> >> >>>>>> Subject / From / To is fast but FTS(Full Text Search) for body is >> >>>>>> horrible. I suppose this is where we need Apache Solr. >> >>>>>> >> >>>>>> Do you think my mail storage format is bad? Do I need to change for >> >>>>>> better performance? >> >>>>>> Please advise >> >>>>>> Kevin >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> On Thu, Jan 29, 2015 at 12:25 PM, Peter Hodur > > >> >>>>>> wrote: >> >>>>>> >> >>>>>>>> * Kevin Laurie 2015.01.24 19:41: >> >>>>>>>> >> >>>>>>>>> Currently the time it takes to search 25,000mails is 4mins. If >> >>>>>>> indexed >> >>>>>>>> how >> >>>>>>>>> much faster are we looking at? >> >>>>>>>> >> >>>>>>>> With a current version of Dovecot a search is pretty fast >> _without_ >> >>>>>>> using >> >>>>>>>> external indexes. I have a view defined (virtual plugin) with >> >>>>>>> around 22.000 >> >>>>>>>> messages in it, and searching the full view only takes 2.5 >> seconds: >> >>>>>>>> >> >>>>>>>> >> >>>>>>> hmmm, could you please tell me more about your setup? What storage >> >>>>>>> format >> >>>>>>> do you use? Maildir or? What is the cumulative size of your >> messages? >> >>>>>>> >> >>>>>>> My results without FTS on ZFS FS with SSD L2ARC are not so good: >> >>>>>>> >> >>>>>>> a4 select INBOX._OLD-OUTLOOK >> >>>>>>> * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk >> >>>>>>> JunkRecorded $Forwarded) >> >>>>>>> * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft >> $Junk >> >>>>>>> $NotJunk JunkRecorded $Forwarded \*)] Flags permitted. >> >>>>>>> ** 7748 EXISTS* >> >>>>>>> * 0 RECENT >> >>>>>>> * OK [UIDVALIDITY 1421198037] UIDs valid >> >>>>>>> * OK [UIDNEXT 11509] Predicted next UID >> >>>>>>> * OK [HIGHESTMODSEQ 12204] Highest >> >>>>>>> a4 OK [READ-WRITE] Select completed (0.001 secs). >> >>>>>>> >> >>>>>>> a5 search charset utf-8 body "mall" >> >>>>>>> * SEARCH 2 49 101 117 158 171 185 192 197 202 207 223 228 234 236 >> 240 >> >>>>>>> 249 >> >>>>>>> 279 280 281 287 288 289 290 297 321 327 337 344 351 360 370 373 >> 385 >> >>>>>>> 389 390 >> >>>>>>> 391 398 405 413 424 444 458 463 470 474 480 482 505 513 520 530 >> 531 >> >>>>>>> 532 533 >> >>>>>>> 543 559 560 561 562 563 566 588 593 597 625 630 639 644 656 671 >> 672 >> >>>>>>> 677 692 >> >>>>>>> 720 723 734 738 741 745 752 755 757 765 775 777 784 791 818 820 >> 821 >> >>>>>>> 833 855 >> >>>>>>> 863 864 868 881 896 910 917 922 926 928 931 991 996 997 998 1000 >> 1010 >> >>>>>>> 1011 >> >>>>>>> 1012 1014 1018 1019 1026 1047 1068 1077 1095 1101 1105 1122 1136 >> 1137 >> >>>>>>> 1140 >> >>>>>>> 1155 1160 1166 1171 1179 1180 1197 1208 1229 1239 1258 1263 1271 >> 1282 >> >>>>>>> 1286 >> >>>>>>> 1290 1298 1319 1364 1365 1370 1386 1408 1410 1429 1463 1465 1470 >> 1471 >> >>>>>>> 1494 >> >>>>>>> 1518 1522 1529 1530 1536 1541 1548 1571 1581 1585 1588 1594 1605 >> 1606 >> >>>>>>> 1611 >> >>>>>>> 1612 1619 1620 1625 1652 1666 1667 1729 1730 1731 1732 1733 1734 >> 1735 >> >>>>>>> 1781 >> >>>>>>> 1782 1817 1818 1897 1900 1921 1940 1946 1960 1972 1981 1995 1998 >> 2002 >> >>>>>>> 2006 >> >>>>>>> 2028 2049 2057 2095 2100 2157 2168 2181 2185 2192 2203 2204 2207 >> 2208 >> >>>>>>> 2210 >> >>>>>>> 2220 2225 2255 2273 2282 2283 2288 2289 2317 2320 2340 2367 2374 >> 2377 >> >>>>>>> 2378 >> >>>>>>> 2379 2384 2389 2402 2409 2436 2459 2475 2476 2488 2504 2519 2538 >> 2539 >> >>>>>>> 2551 >> >>>>>>> 2566 2572 2597 2599 2603 2617 2629 2664 2698 2716 2731 2733 2753 >> 2754 >> >>>>>>> 2780 >> >>>>>>> 2805 2808 2815 2818 2850 2861 2862 2867 2886 2896 2900 2914 2931 >> 2936 >> >>>>>>> 2938 >> >>>>>>> 2939 2950 2969 2990 3017 3019 3062 3075 3094 3101 3115 3138 3159 >> 3161 >> >>>>>>> 3178 >> >>>>>>> 3185 3190 3204 3217 3218 3248 3263 3265 3266 3273 3282 3288 3295 >> 3386 >> >>>>>>> 3428 >> >>>>>>> 3453 3476 3478 3479 3511 3548 3606 3629 3693 3694 3737 3793 3799 >> 3801 >> >>>>>>> 3808 >> >>>>>>> 3812 3814 3815 3834 3849 3860 3862 3880 3910 3917 3930 3932 3952 >> 3953 >> >>>>>>> 3954 >> >>>>>>> 3957 3959 3968 3971 3973 3978 3979 3980 4008 4022 4040 4057 4058 >> 4059 >> >>>>>>> 4063 >> >>>>>>> 4064 4066 4069 4070 4075 4096 4112 4131 4132 4133 4141 4143 4144 >> 4145 >> >>>>>>> 4146 >> >>>>>>> 4147 4167 4174 4199 4201 4202 4203 4206 4211 4217 4218 4226 4229 >> 4258 >> >>>>>>> 4259 >> >>>>>>> 4267 4287 4357 4359 4363 4364 4365 4367 4390 4391 4462 4475 4497 >> 4502 >> >>>>>>> 4538 >> >>>>>>> 4540 4552 4557 4558 4561 4563 4567 4571 4572 4573 4575 4577 4593 >> 4594 >> >>>>>>> 4604 >> >>>>>>> 4611 4619 4628 4638 4639 4662 4672 4678 4679 4692 4696 4785 4786 >> 4787 >> >>>>>>> 4788 >> >>>>>>> 4789 4792 4793 4794 4802 4817 4818 4819 4820 4836 4857 4874 4887 >> 4901 >> >>>>>>> 4905 >> >>>>>>> 4906 4907 4908 4911 4925 4928 4940 4941 4953 5060 5103 5116 5118 >> 5129 >> >>>>>>> 5131 >> >>>>>>> 5136 5158 5163 5182 5184 5203 5212 5216 5269 5270 5271 5272 5273 >> 5276 >> >>>>>>> 5277 >> >>>>>>> 5278 5286 5301 5302 5303 5304 5320 5341 5358 5371 5385 5389 5390 >> 5391 >> >>>>>>> 5392 >> >>>>>>> 5395 5409 5412 5424 5425 5437 5544 5587 5600 5602 5613 5615 5620 >> 5642 >> >>>>>>> 5647 >> >>>>>>> 5666 5668 5687 5696 5700 5736 5740 5749 5764 5783 5809 5814 5853 >> 5866 >> >>>>>>> 5867 >> >>>>>>> 5877 5888 5895 5896 5897 5898 5899 5908 5910 5911 5912 5939 5950 >> 5958 >> >>>>>>> 5990 >> >>>>>>> 6000 6059 6074 6095 6097 6112 6137 6141 6189 6193 6212 6228 6229 >> 6233 >> >>>>>>> 6271 >> >>>>>>> 6273 6275 6285 6310 6317 6335 6383 6384 6397 6427 6430 6459 6463 >> 6482 >> >>>>>>> 6492 >> >>>>>>> 6506 6565 6585 6620 6670 6673 6675 6705 6715 6716 6741 6812 6826 >> 6852 >> >>>>>>> 6859 >> >>>>>>> 6895 6896 6907 6913 6919 6935 6943 6948 6979 7023 7025 7035 7039 >> 7042 >> >>>>>>> 7108 >> >>>>>>> 7131 7145 7163 7171 7172 7194 7198 7199 7203 7256 7257 7294 7303 >> 7317 >> >>>>>>> 7322 >> >>>>>>> 7343 7344 7347 7348 7352 7386 7390 7391 7392 7393 7407 7408 7409 >> 7417 >> >>>>>>> 7418 >> >>>>>>> 7419 7420 7421 7426 7432 7437 7462 7467 7468 7473 7474 7475 7488 >> 7502 >> >>>>>>> 7503 >> >>>>>>> 7558 7588 7589 7628 7685 7695 7699 7703 7723 >> >>>>>>> *a5 OK Search completed (7.846 secs).* >> >>>>>>> >> >>>>>>> >> >>>>>>> Searching against "subject" is pretty fast, few miliseconds ... >> >>>>>>> >> >>>>>> >> >>>>>> >> >>>>> >> >>> >> >> >