From kabelpada at kabelmail.de Fri Aug 1 02:40:56 2014 From: kabelpada at kabelmail.de (Daniel Parthey) Date: Fri, 1 Aug 2014 04:40:56 +0200 Subject: LMTP during dsync migration In-Reply-To: <53D8B68E.5080603@mur.at> References: <53D7A647.30602@mur.at> <53D7ABF7.8000405@thinline.cz> <53D8B68E.5080603@mur.at> Message-ID: <20140801024056.GA24109@daniel.localdomain> Jogi Hofm?ller wrote: > We are not planning on multiple runs of doveadm backup. The question is > what to do with LMTP _while_ the doveadm backup migration is happening? > > At the time when mailbox X get's migrated, the settings used for the > proxy will be removed, so user X cannot access the mailbox on the old > server anymore. Given the size of some of the mailboxes chances are > quite high that there will be delivery attempts _during_ the migration > process. That's what I am thinking about. What we did was to install a "retry" transport in addition to "lmtp" in postfix master.cf: retry unix - - n - - error See http://www.postfix.org/error.8.html Before migrating each user, we instructed postfix to route the affected user to the retry transport. This puts all incoming messages for this use into the queue while the migration is running. When the migration of this user has proven to be successful, you can revert the user to the default lmtp transport. Good luck, Daniel From larryrtx at gmail.com Fri Aug 1 05:40:56 2014 From: larryrtx at gmail.com (Larry Rosenman) Date: Fri, 1 Aug 2014 00:40:56 -0500 Subject: Mail Archiving/Lucene Indexes/mbox Message-ID: I have a script (available on demand) that moves all my mail for a given month to a separate NAMESPACE (#ARCHIVE). When I do that, the lucene-indexes folder in the PRIMARY namespace seems to get cleaned out, and NOTHING gets put in the lucene-indexes folder (although it gets created) in the #ARCHIVE namespace. Is this to spec? the archiving is done via doveadm move commands. Thanks! -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From hmoreno at gmv.com Fri Aug 1 08:17:42 2014 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Fri, 1 Aug 2014 08:17:42 +0000 Subject: vacation rule addresses Message-ID: Hello everyone, We have Dovecot 2.1.6 and Pigeonhole 0.3.0. We would like Pigeonhole not to send the vacation response to noreply at whatever.com. In this wiki http://wiki2.dovecot.org/Pigeonhole/Sieve/Usage#Vacation_auto-reply are showed the different exceptions, but the noreply is not in there. How can I add this address, or another one, to this exception list? Thank you very much in advanced. Kind regards. H?ctor Moreno. P Please consider the environment before printing this e-mail. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From skdovecot at smail.inf.fh-brs.de Fri Aug 1 08:49:29 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 1 Aug 2014 10:49:29 +0200 (CEST) Subject: vacation rule addresses In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 1 Aug 2014, H?ctor Moreno Blanco wrote: > We would like Pigeonhole not to send the vacation response to > noreply at whatever.com. In this wiki > http://wiki2.dovecot.org/Pigeonhole/Sieve/Usage#Vacation_auto-reply are > showed the different exceptions, but the noreply is not in there. > > How can I add this address, or another one, to this exception list? You call vacation conditionally by excluding those addresses using if (http://tools.ietf.org/html/rfc5228#section-3.1) allof ( ) and the tests address (http://tools.ietf.org/html/rfc5228#section-5.1), envelope (http://tools.ietf.org/html/rfc5228#section-5.4), or header (http://tools.ietf.org/html/rfc5228#section-5.7). http://tools.ietf.org/html/rfc5230#section-4.8 - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU9tUmnz1H7kL/d9rAQLY7gf/b3lbQBCXrTi90ILCbhx6ZLAbjTjKExFn Mt7J/d3vFx3Qj8mbisTYF3+IJ3qROjqXSEaRB0oiTyiaHu73/p/0NVRCt23ee4t4 AmwvyQLrrrJtDKiMXbpY47aUINNDU4oOTYyiZJPl7jqPJGHmbhGCit7SZrE7h3J0 oFi6X0JI7bEvgtOnNKJBSUZcfTPzDscDr7PuwLKL5cHu8P0jjGw4t3vBIAFXGYrM NEEeEzINpY3XVUZC9sgYczvh8x47FhVBIWHT8DhJCP5c+y+g0D65nbWeOAYWDIRU hyPrcuyb2U4daiBB8/28GJ/f91a6REPDIUwIkbnbOFd8azak6bVKaw== =ZSP0 -----END PGP SIGNATURE----- From www-1223 at mail.ru Fri Aug 1 12:02:31 2014 From: www-1223 at mail.ru (=?UTF-8?B?d3d3LTEyMjM=?=) Date: Fri, 01 Aug 2014 16:02:31 +0400 Subject: =?UTF-8?B?dXNpbmcgZmxhdCBmaWxlIGZvciBkaWN0aW9uYXJ5IHF1b3RhIGJhY2tlbmQ=?= Message-ID: <1406894551.911569376@f436.i.mail.ru> Hi! I want use flat file for dictionary quota backend, but I didn't find the description of that he has to contain. Somebody can give an example of such file? PS: sorry for my English. Regards, Oleg Ivanov. From tss at iki.fi Fri Aug 1 14:02:19 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:02:19 +0300 Subject: solr_add_definite_query quotes queries In-Reply-To: <1406825211.53599.YahooMailNeo@web161401.mail.bf1.yahoo.com> References: <1406825211.53599.YahooMailNeo@web161401.mail.bf1.yahoo.com> Message-ID: <7C8C4D44-92D7-4D9C-8B55-DA9627790461@iki.fi> On 31 Jul 2014, at 19:46, G H wrote: > So Solr 4.0 and higher supports regular expressions in searching. However, Dovecot is quoting search queries in solr_add_definite_query (via solr_add_str_arg). This breaks regex searching. I do a sample search for emails with "merged" in their body via the query /merg.*/. If you look at the Solr logs, it is sending: > q=body:"/merg.*/" > which is converted to body:\"/merg.*/\" inside Solr. > > > but if it had sent only: > q=body:/merg.*/ > > > then regex would have worked. > > Is regular expression searching on the dashboard? I'm considering patching fts-backend-solr.c to not quote-enclose the search, but am looking for any better suggestions. Or perhaps not quote-enclose if the search is prefixed with RX: (ie: "RX:/merg.*/")... I think if such feature is added it should definitely be optional and disabled by default. Who knows what kind of problems users could cause by getting direct access to that functionality. I guess the possibilities are to either have some kind of a prefix (maybe that's the configurability, fts_solr_passthrough_search_prefix=SOLR:) or maybe have a whole new search parameter that specifies that this is now a direct Solr query (probably too much trouble). From tss at iki.fi Fri Aug 1 14:07:16 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:07:16 +0300 Subject: LMTP during dsync migration In-Reply-To: <53D7A647.30602@mur.at> References: <53D7A647.30602@mur.at> Message-ID: On 29 Jul 2014, at 16:48, Jogi Hofm?ller wrote: > Another question regarding migration. While migrating a mailbox with > dsync is it safe to deliver mail via LMTP to the new (target) mailbox or > is it wiser to deactivate LMTP delivery to this mailbox until it's fully > migrated? > > And what methods could I use to stop delivery to a mailbox during > migration? Our user data is stored on an LDAP server. Temporarily disabling mail delivery for the user being migrated is the safest choice, but it's not a requirement. You could run "doveadm sync -1" for the user and it'll merge the changes without deleting any changes. There is some problem with "doveadm sync -1" though that if it's ran multiple times (more than once) and there are large enough changes it'll start doing some stupid things (can't remember the details - either duplicates mails or just changes their UIDs unnecessarily). Running it once should be safe though. From tss at iki.fi Fri Aug 1 14:09:36 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:09:36 +0300 Subject: Mail Archiving/Lucene Indexes/mbox In-Reply-To: References: Message-ID: <8EFAB478-7E83-4381-8A47-EE7D22133D21@iki.fi> On 01 Aug 2014, at 08:40, Larry Rosenman wrote: > I have a script (available on demand) that moves all my mail for a given > month to a separate NAMESPACE (#ARCHIVE). > > When I do that, the lucene-indexes folder in the PRIMARY namespace seems to > get cleaned out, and NOTHING gets put in the lucene-indexes folder > (although it gets created) in the #ARCHIVE namespace. > > Is this to spec? > > the archiving is done via doveadm move commands. Do you have fts_autoindex=yes? If not, the Lucene index update is delayed until the first search is issued. From tss at iki.fi Fri Aug 1 14:13:43 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:13:43 +0300 Subject: [Dovecot] Converting old emails to compressed format In-Reply-To: References: <530FC7CB.1040200@allard.it> <53D7BA2D.8040206@netcologne.de> Message-ID: <62ED858D-E383-4CD8-9FFD-C3816AC6D680@iki.fi> On 29 Jul 2014, at 18:22, Wolfgang Rosenauer wrote: > Hi Christian, > > On Tue, Jul 29, 2014 at 5:13 PM, Christian Rohmann > wrote: >> >> You have to set the compression type with the zlib_save option. >> >> i.e.: "-o plugin/zlib_save=gz" > > been there: > dsync -o plugin/zlib_save=gz backup -u testy > maildir:/srv/dovecot/testy/maildir.new > > doesn't make a difference unfortunately. > My mailboxes are in maildir format and besides enabling zlib I do not > change the format. > My testmailbox has only one message but this still is uncompressed after dsync. > > Where is Peer who wrote in his book that this should just work? Maybe it's trying to copy the mails by hard linking instead of physically copying the data. See if -o maildir_copy_with_hardlinks=no helps? From tss at iki.fi Fri Aug 1 14:16:48 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:16:48 +0300 Subject: Maybe error in login-common/login-proxy.c/proxy_client_input In-Reply-To: <409CEB1649811749B2E36961BEE2DDBC45F2D306@srv-mbx4.rambler.ramblermedia.com> References: <409CEB1649811749B2E36961BEE2DDBC45F2D306@srv-mbx4.rambler.ramblermedia.com> Message-ID: <5711D095-2F0B-4318-A4FE-723A2E24E9E4@iki.fi> On 23 Jul 2014, at 12:54, Potapov Sergey wrote: > It seems that o_stream_cork should be done on proxy->server_output but not on proxy->client_output. > > o_stream_cork(proxy->client_output); > ret2 = o_stream_send(proxy->server_output, buf, ret); > o_stream_uncork(proxy->server_output); Yes. I fixed this a while ago. Just need to get v2.2.14 released once I can get through reading mails in this list.. http://hg.dovecot.org/dovecot-2.2/rev/6a9508d28d34 From tss at iki.fi Fri Aug 1 14:18:40 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:18:40 +0300 Subject: [PATCH] Fix typo in src/imap-urlauth/imap-urlauth-worker.c In-Reply-To: <20140719230035.GA11996@jelmer.co.uk> References: <20140719230035.GA11996@jelmer.co.uk> Message-ID: <562BF32E-7665-48B2-A6BB-7F28A609AC1B@iki.fi> On 20 Jul 2014, at 02:00, Jelmer Vernooij wrote: > The attached trivial patch fixes a typo in src/imap-urlauth/imap-urlauth-worker.c. Thanks, committed: http://hg.dovecot.org/dovecot-2.2/rev/417a984e91f3 From larryrtx at gmail.com Fri Aug 1 14:22:42 2014 From: larryrtx at gmail.com (Larry Rosenman) Date: Fri, 1 Aug 2014 09:22:42 -0500 Subject: Mail Archiving/Lucene Indexes/mbox In-Reply-To: <8EFAB478-7E83-4381-8A47-EE7D22133D21@iki.fi> References: <8EFAB478-7E83-4381-8A47-EE7D22133D21@iki.fi> Message-ID: Yes, I do. thebighonker.lerctr.org /home/ler $ doveconf -n # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.0-STABLE amd64 auth_default_realm = lerctr.org auth_mechanisms = plain login auth_realms = lerctr.org thebighonker.lerctr.org tbh.lerctr.org auth_username_format = %Ln lda_mailbox_autocreate = yes lmtp_save_to_detail_mailbox = yes login_access_sockets = tcpwrap mail_debug = yes mail_location = mbox:~/mail:INBOX=~/mail/INBOX mail_plugins = fts fts_lucene stats mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate vacation-seconds editheader namespace archive { hidden = no inbox = no list = no location = mbox:~/MAILARCHIVE prefix = "#ARCHIVE/" separator = / } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox INBOX { auto = create } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = failure_show_msg=yes session=yes max_requests=20 driver = pam } plugin { fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. normalize no_snowball recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +editheader +vacation-seconds stats_command_min_time = 1 mins stats_domain_min_time = 12 hours stats_ip_min_time = 12 hours stats_memory_limit = 16 M stats_refresh = 5s stats_session_min_time = 15 mins stats_track_cmds = yes stats_user_min_time = 1 hours } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-client { mode = 0666 } unix_listener auth-master { mode = 0666 } } service stats { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = stats extra_groups = fifo_listener stats-mail { group = mode = 0666 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener stats { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service tcpwrap { unix_listener login/tcpwrap { group = $default_login_user mode = 0600 user = $default_login_user } } ssl_cert = : stored mail into mailbox 'INBOX' Aug 1 00:05:45 thebighonker exim[70321]: 1XD51Y-000ICE-2D => ler < ler at lerctr.org> R=localuser T=local_delivery_dovecot S=11327 QT=45s DT=0s Aug 1 00:05:45 thebighonker exim[70321]: 1XD51Y-000ICE-2D Completed QT=45s Aug 1 00:05:46 thebighonker dovecot: indexer-worker(ler): Indexed 1 messages in INBOX the script: thebighonker.lerctr.org /home/ler $ cat bin/archive-mail #!/bin/sh PATH=$PATH:/usr/local/bin #Expects to be run after midnight on the first of the month # to archive all the previous months mail #Date Run: TODAY=`date "+%Y-%m-%d"` #last month in YYYY/MM YEAR_LAST_MONTH=`date -v-1d "+%Y/%m"` #1st of last month as 01-Mon-YYYY FIRST_LAST_MONTH=`date -v-1d "+01-%b-%Y"` echo 'TODAY=' ${TODAY} echo 'YEAR_LAST_MONTH=' ${YEAR_LAST_MONTH} echo 'FIRST_LAST_MONTH=' ${FIRST_LAST_MONTH} # get a list of all the mailboxes with at least one real message doveadm -f table mailbox status vsize \* 2>/dev/null | \ awk '{if ($2 > 0) print $1}' >/tmp/ler-mailbox-list.$$ 2>/dev/null # archive all the mails for i in `cat /tmp/ler-mailbox-list.$$` do echo `date` start ${i} doveadm mailbox create \#ARCHIVE/${YEAR_LAST_MONTH}/${i} doveadm -f tab mailbox status messages ${i} doveadm move \#ARCHIVE/${YEAR_LAST_MONTH}/${i} mailbox \ ${i} BEFORE ${TODAY} SINCE ${FIRST_LAST_MONTH} doveadm -f tab mailbox status messages ${i} echo `date` done ${i} done rm /tmp/ler-mailbox-list.$$ thebighonker.lerctr.org /home/ler $ I am running the hg patch that fixes the filenames to be in the .imap space, but that is the only patch I have applied on top of 2.2.13. On Fri, Aug 1, 2014 at 9:09 AM, Timo Sirainen wrote: > On 01 Aug 2014, at 08:40, Larry Rosenman wrote: > > > I have a script (available on demand) that moves all my mail for a given > > month to a separate NAMESPACE (#ARCHIVE). > > > > When I do that, the lucene-indexes folder in the PRIMARY namespace seems > to > > get cleaned out, and NOTHING gets put in the lucene-indexes folder > > (although it gets created) in the #ARCHIVE namespace. > > > > Is this to spec? > > > > the archiving is done via doveadm move commands. > > Do you have fts_autoindex=yes? If not, the Lucene index update is delayed > until the first search is issued. > > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From tss at iki.fi Fri Aug 1 14:29:42 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:29:42 +0300 Subject: Differences in "Delivered-To" header between deliver and LMTP In-Reply-To: <53BFAD1F.3020406@Media-Brokers.com> References: <53BD87D3.9070108@jrice.me> <665E335C-289C-4390-A7B9-9B2F3D2E72A2@iki.fi> <53BFAD1F.3020406@Media-Brokers.com> Message-ID: <68C9FF17-C1F4-453E-8A3A-FF27CCB97B70@iki.fi> On 11 Jul 2014, at 12:23, Charles Marcus wrote: >> Oh, that's annoying. Dovecot LDA doesn't actually add this header, it was your MTA that added it. But looks like I hadn't checked what the MTAs actually write to the header when I added this feature to LMTP, and nobody had mentioned this before either. I wonder if anybody's system breaks if I just change it now.. > > If you do this Timo, would you please also consider going ahead and adding the 20 or so lines of code you said it would take to also add support for the x-original-to header (this specific was in a private email to me while our support contract was active), as we discussed on this list - and you then discussed with Wietse, back in April or so of 2012: > > http://dovecot.org/list/dovecot/2012-April/082585.html The problem is still that it would require Dovecot LMTP to start advertising the DSN extension, which brings it more responsibilities. I'm still a bit scared of doing such change as it would need to be thought out well and to see what its impact would be for current MTAs. I'm especially worried that different MTAs would behave differently and we might need to add a setting to control what Dovecot needs to do. From tss at iki.fi Fri Aug 1 14:33:20 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 17:33:20 +0300 Subject: Dovecot EE on SciLinux 6.5: No Config Files in /etc In-Reply-To: <53C69913.1020306@globalchangemusic.org> References: <53C69913.1020306@globalchangemusic.org> Message-ID: <7D1B63CF-8C92-4DA3-A487-FAE8EA1FD2DE@iki.fi> On 16 Jul 2014, at 18:24, Asai wrote: > I tried installing Dovecot EE on Scientific 6.5 yesterday, but there was no repo under that release version, so I hardcoded the release version to 6 in yum.repos.d/dovecot.repo. > > That worked, but when I installed, it didn't install any default configuration files. /etc/dovecot was empty. > > Is that to be expected for EE? It's expected, although I guess maybe there could be some README file. The main problem I've noticed with some other RPMs have been that if I add a new file to /etc/dovecot/conf.d/ it gets included when upgrading the RPM. But if that new file depends on changes that were made to other config files, then adding the file breaks the whole configuration completely. This is what happened in v2.0 -> v2.1 atrpms RPM upgrade. I don't want that ever to happen with dovecot-ee. Maybe there would be some other ways to solve that, but the simplest solution was to have no config files. From tss at iki.fi Fri Aug 1 15:13:31 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 1 Aug 2014 18:13:31 +0300 Subject: HIGHESTMODSEQ tracking In-Reply-To: <20140715141359.Horde.dexbWi1OkSM04YZc37jcmQ3@bigworm.curecanti.org> References: <20140709153745.Horde.75rqVkqynWzjwvzd3953jw1@bigworm.curecanti.org> <20140709180537.Horde.YGyQtaBeFXNmQ89BJlSi7Q1@bigworm.curecanti.org> <20140714174719.Horde.px3Mk2olwJghwWJ-q5T5kw6@bigworm.curecanti.org> <20140715141359.Horde.dexbWi1OkSM04YZc37jcmQ3@bigworm.curecanti.org> Message-ID: <294F875F-1161-4DF9-A54F-0D411356F1F7@iki.fi> On 15 Jul 2014, at 23:13, Michael M Slusarz wrote: >>> For the "more than 50% market share" of Dovecot / Cyrus, do you have a >>>> breakdown by version number? At least in terms of 1.* vs 2.0 and higher? >>>> >>> >>> I do not. >> >> >> And without being able to get a version number from a Dovecot session (or >> so it seems to me -- nothing returned from ID...).... it looks kind of sad. > > ID extension is pretty much worthless for version identification. It is trivially spoofed -- and some servers do exactly this in the real world. All it takes is one server/version to be spoofed to make that data worthless. > > It's possible to do some level of basic version sniffing by things like banner messages, Human-readable responses, CAPABILITY lists, and ordering of responses to various commands. However, this is really only useful for broad statistical surveys and not precise version determination. Here are some interesting results based on such survey: http://openemailsurvey.org/dovecot-versions.png From jean-louis at dupond.be Fri Aug 1 20:00:03 2014 From: jean-louis at dupond.be (Jean-Louis Dupond) Date: Fri, 01 Aug 2014 22:00:03 +0200 Subject: Multiple Passdb's - Order Message-ID: <53DBF1C3.9000307@dupond.be> Hi, Today I upgraded my dovecot to a newer version (2.2.9 debian wheezy-backports) At that moment I also reconfigured the whole thing a bit nicer. Now I stumbled on the following issue. In my configuration I use 2 backends (LDAP & passwd-file). passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } userdb { args = uid=vmail gid=vmail home=/var/vmail/%d/%n driver = static } Now this worked fine before. But if I tested it, LDAP logins worked, and passwd-file logins failed. Failed log (user is only in the passwd-file): Aug 01 21:31:55 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 01 21:31:55 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Aug 01 21:31:55 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_pgsql.so Aug 01 21:31:55 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_sqlite.so Aug 01 21:31:55 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 01 21:31:55 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Aug 01 21:31:55 auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Aug 01 21:31:55 auth: Debug: passwd-file /etc/dovecot/users: Read 10 users in 0 secs Aug 01 21:31:55 auth: Debug: auth client connected (pid=5516) Aug 01 21:31:55 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=c34kcZb/uwBbtKLu lip=xxx rip=xxxx lport=993 rport=52411 Aug 01 21:31:55 auth: Debug: client passdb out: CONT 1 Aug 01 21:31:55 auth: Debug: client in: CONT 1 xxxxx (previous base64 data may contain sensitive data) Aug 01 21:31:55 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:31:55 auth: Debug: ldap(info at xxx.com,xxx,): pass search: base=uid=info at xxx.com,ou=mail,dc=dupie,dc=be scope=subtree filter=(&(objectClass=posixAccount)(uid=info at xxx.com)) fields=uid,userPassword Aug 01 21:31:55 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:31:55 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:31:55 auth: Debug: passwd-file(info at xxx.com,xxx,): lookup: user=info at xxx.com file=/etc/dovecot/users Aug 01 21:31:57 auth: Debug: client passdb out: FAIL 1 user=info at xxx.com temp Aug 01 21:31:57 auth: Debug: client in: AUTH 2 PLAIN service=imap secured session=c34kcZb/uwBbtKLu lip=xxx rip=xxx lport=993 rport=52411 resp=xxx (previous base64 data may contain sensitive data) Aug 01 21:32:01 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:32:01 auth: Debug: ldap(info at xxx.com,xxx,): pass search: base=uid=info at xxx.com,ou=mail,dc=dupie,dc=be scope=subtree filter=(&(objectClass=posixAccount)(uid=info at xxx.com)) fields=uid,userPassword Aug 01 21:32:01 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:32:01 auth: Debug: cache(info at xxx.com,xxx,): hit: {CRAM-MD5}19192922..... Aug 01 21:32:03 auth: Debug: client passdb out: FAIL 2 user=info at xxx.com temp As you see it returned a FAIL, but the password was retrieved from the users file (see the cache hit). Now I changed the order of the passdb's in my config, to make passwd-file come first, and then LDAP. And what happend now, it started working ... Logs: Aug 01 21:30:23 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 01 21:30:23 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Aug 01 21:30:23 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_pgsql.so Aug 01 21:30:23 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_sqlite.so Aug 01 21:30:23 auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 01 21:30:23 auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libauthdb_ldap.so Aug 01 21:30:23 auth: Debug: Read auth token secret from /var/run/dovecot/auth-token-secret.dat Aug 01 21:30:23 auth: Debug: passwd-file /etc/dovecot/users: Read 10 users in 0 secs Aug 01 21:30:23 auth: Debug: auth client connected (pid=5459) Aug 01 21:30:24 auth: Debug: client in: AUTH 1 PLAIN service=imap secured session=TTm6a5b/swBbtKLu lip=xxx rip=xxx lport=993 rport=52403 Aug 01 21:30:24 auth: Debug: client passdb out: CONT 1 Aug 01 21:30:24 auth: Debug: client in: CONT 1 xxxx (previous base64 data may contain sensitive data) Aug 01 21:30:24 auth: Debug: cache(info at xxx.com,xxx,): miss Aug 01 21:30:24 auth: Debug: passwd-file(info at xxx.com,xxx,): lookup: user=info at xxx.com file=/etc/dovecot/users Aug 01 21:30:24 auth: Debug: client passdb out: OK 1 user=info at xxx.com Aug 01 21:30:24 auth: Debug: master in: REQUEST 1973682177 5459 1 9976303da4c6eb55a008b0cae6ab12a1 session_pid=5462 request_auth_token Aug 01 21:30:24 auth: Debug: master userdb out: USER 1973682177 info at xxx.com uid=1001 gid=1001 home=/var/vmail/xxx.com/info auth_token=31afcf6ff0c5ed5150beb96ad3718cff452badae The only thing I changed here was the order the passdb backends. It works fine now, but it not clear to me why the order is so important. Could somebody explain this? Or is this some bug? Thanks Jean-Louis From mike at mikepultz.com Fri Aug 1 20:54:21 2014 From: mike at mikepultz.com (Mike Pultz) Date: Fri, 1 Aug 2014 16:54:21 -0400 Subject: autocreate mailboxes Message-ID: <026c01cfadca$c4ff04c0$4efd0e40$@mikepultz.com> Hello, I just upgraded to Dovecot 2.2.13 from the 2.0 version branch- so far everything is working great. I wanted to take advantage of the mailbox option to auto create and auto subscribe users to a Sent and Trash folder (using the support for the special use flags), but ideally, I only want to do this for new mailboxes, and not for all the mailboxes that already exist on the system. The reason for this is because all the existing users likely already have a sent-mail folder configured, and I don't want to implement this, and have their sent-mail folder switched to this new one. When I set this up on my test system, I noticed that the new sent and trash folders appeared in my existing mailbox, and when I checked the account on my iPhone, it switched my sent-mail folder away from my "normal" sent-mail folder, and I couldn't change it back. Is there a way to do this ONLY when dovecot creates a new mailbox? Thanks, Mike From gedalya at gedalya.net Fri Aug 1 21:26:25 2014 From: gedalya at gedalya.net (Gedalya) Date: Fri, 01 Aug 2014 17:26:25 -0400 Subject: autocreate mailboxes In-Reply-To: <026c01cfadca$c4ff04c0$4efd0e40$@mikepultz.com> References: <026c01cfadca$c4ff04c0$4efd0e40$@mikepultz.com> Message-ID: <53DC0601.7010401@gedalya.net> On 08/01/2014 04:54 PM, Mike Pultz wrote: > I wanted to take advantage of the mailbox option to auto create and auto > subscribe users to a Sent and Trash folder (using the support for the > special use flags), but ideally, I only want to do this for new mailboxes, > and not for all the mailboxes that already exist on the system. I guess this could be done with extra fields: http://wiki2.dovecot.org/UserDatabase/ExtraFields The syntax being something like namespace/inbox/mailbox/Sent/special_use=\Sent ..? Or something like that. Note the userdb_import facility for passing multiple options. **I wonder if it wouldn't be useful to be able to create configuration sections and activate them with a single, simple extra field rather than pass the entire section itself? In a similar manner to protocol imap { }, maybe we can have $extra_options1 == "yes" { } and pass extra_options1 = yes as an extra field? From asai at globalchangemusic.org Fri Aug 1 22:08:02 2014 From: asai at globalchangemusic.org (Asai) Date: Fri, 01 Aug 2014 15:08:02 -0700 Subject: Dovecot EE on SciLinux 6.5: No Config Files in /etc In-Reply-To: <7D1B63CF-8C92-4DA3-A487-FAE8EA1FD2DE@iki.fi> References: <53C69913.1020306@globalchangemusic.org> <7D1B63CF-8C92-4DA3-A487-FAE8EA1FD2DE@iki.fi> Message-ID: <53DC0FC2.1090708@globalchangemusic.org> > On 16 Jul 2014, at 18:24, Asai wrote: > >> I tried installing Dovecot EE on Scientific 6.5 yesterday, but there was no repo under that release version, so I hardcoded the release version to 6 in yum.repos.d/dovecot.repo. >> >> That worked, but when I installed, it didn't install any default configuration files. /etc/dovecot was empty. >> >> Is that to be expected for EE? > It's expected, although I guess maybe there could be some README file. The main problem I've noticed with some other RPMs have been that if I add a new file to /etc/dovecot/conf.d/ it gets included when upgrading the RPM. But if that new file depends on changes that were made to other config files, then adding the file breaks the whole configuration completely. This is what happened in v2.0 -> v2.1 atrpms RPM upgrade. I don't want that ever to happen with dovecot-ee. Maybe there would be some other ways to solve that, but the simplest solution was to have no config files. Thanks for the info. Perhaps you could put a README in /etc/dovecot where the configs would normally be? From gayleard at eircom.net Fri Aug 1 22:18:07 2014 From: gayleard at eircom.net (Timothy Murphy) Date: Sat, 02 Aug 2014 00:18:07 +0200 Subject: Configuring pigeonhole/sieve Message-ID: <8476964.OVhcKYWKVt@rose.gayleard.eu> I've been trying for a few days to get pigeonhole/sieve working in order to filter email with spamassassin. Unfortunately I have had no success; and I have found no document which simply tells me what changes I need to make to which files, and what other action I have to take. I'm running CentOS-7 on my home-server, and have postfix + dovecot working perfectly; moreover spam is marked with ***Spam*** added to the Subject header. The only problem is that this spam is getting through to my inbox. It's clear that I'm overlooking some action I should be taking, and I'm sure there must be a document somewhere that gives simple and complete instructions on setting up pigeonhole/sieve to work with SpamAssassin. If anyone knows of such a document I should be eternally grateful. I was told at one point that "You need to pass your mail through dovecot-lda or lmtp", but I have no idea what this means. I see in /var/log/dovecot when I re-start dovecot "Aug 02 00:13:04 master: Info: Dovecot v2.2.10 starting up for imap, pop3, lmtp, sieve, sieve (core dumps disabled", which seems to imply that lmtp is enabled. Any suggestions or advice gratefully received. -- Timothy Murphy e-mail: gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin 2, Ireland From kabelpada at kabelmail.de Fri Aug 1 23:06:28 2014 From: kabelpada at kabelmail.de (Daniel Parthey) Date: Sat, 2 Aug 2014 01:06:28 +0200 Subject: Configuring pigeonhole/sieve In-Reply-To: <8476964.OVhcKYWKVt@rose.gayleard.eu> References: <8476964.OVhcKYWKVt@rose.gayleard.eu> Message-ID: <20140801230628.GA31858@daniel.localdomain> Timothy Murphy wrote: > I've been trying for a few days to get pigeonhole/sieve working > in order to filter email with spamassassin. > It's clear that I'm overlooking some action I should be taking, > and I'm sure there must be a document somewhere > that gives simple and complete instructions > on setting up pigeonhole/sieve to work with SpamAssassin. > If anyone knows of such a document I should be eternally grateful. Postfix delivers your mail to the dovecot mailbox, this is configured in postfix main.cf/master.cf, not in dovecot. > I was told at one point that > "You need to pass your mail through dovecot-lda or lmtp", > but I have no idea what this means. It means that you need to configure a LMTP (lightweight message transfer protocol) transport in postfix, for example as described in the following documents: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP http://wiki2.dovecot.org/LMTP If using INET sockets, an entry in postfix master.cf would look like this: dovecot-lmtp unix - - n - - lmtp Postfix main.cf might look like this: virtual_transport = dovecot-lmtp:dovecotserver:24 In this example the dovecot LMTP service is listening on Host "dovecotserver" on Port 24 TCP. > I see in /var/log/dovecot when I re-start dovecot > "Aug 02 00:13:04 master: Info: Dovecot v2.2.10 starting up > for imap, pop3, lmtp, sieve, sieve (core dumps disabled", > which seems to imply that lmtp is enabled. Yes, dovecot provides the LMTP service and you should be able to connect to it using telnet, but you need to route incoming mails from postfix via LMTP into Dovecot's LMTP service instead of writing the mail directly to the filesystem. If postfix writes directly to the filesystem without involving Dovecot/Pigeonhole, it circumvents the SIEVE service and therefore your SIEVE rules won't apply, even though the service itself is running. Regards Daniel From wrosenauer at gmail.com Sat Aug 2 07:48:36 2014 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Sat, 2 Aug 2014 09:48:36 +0200 Subject: [Dovecot] Converting old emails to compressed format In-Reply-To: <62ED858D-E383-4CD8-9FFD-C3816AC6D680@iki.fi> References: <530FC7CB.1040200@allard.it> <53D7BA2D.8040206@netcologne.de> <62ED858D-E383-4CD8-9FFD-C3816AC6D680@iki.fi> Message-ID: On Fri, Aug 1, 2014 at 4:13 PM, Timo Sirainen wrote: > On 29 Jul 2014, at 18:22, Wolfgang Rosenauer wrote: > >> been there: >> dsync -o plugin/zlib_save=gz backup -u testy >> maildir:/srv/dovecot/testy/maildir.new >> >> doesn't make a difference unfortunately. >> My mailboxes are in maildir format and besides enabling zlib I do not >> change the format. >> My testmailbox has only one message but this still is uncompressed after dsync. > > Maybe it's trying to copy the mails by hard linking instead of physically copying the data. See if -o maildir_copy_with_hardlinks=no helps? That's it indeed. I haven't thought about hardlinks but with that option the mails actually get compressed. Thanks a lot, Wolfgang From jerry at seibercom.net Sat Aug 2 10:57:29 2014 From: jerry at seibercom.net (Jerry) Date: Sat, 2 Aug 2014 06:57:29 -0400 Subject: Advantage of LMTP Message-ID: <20140802065729.2a2d7ebc@scorpio> Sat, 2 Aug 2014 06:53:58 -0400 Presently, I have: "virtual_transport = dovecot" in my Postfix main.cf file and dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient} in the master.cf file. Is there any advantage to using LMPT instead? All of my users are configured as virtual users. If I change it, do I have to eliminate the above from my master.cf file? -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: not available URL: From bourek at thinline.cz Sat Aug 2 11:09:18 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Sat, 02 Aug 2014 13:09:18 +0200 Subject: Advantage of LMTP In-Reply-To: <20140802065729.2a2d7ebc@scorpio> References: <20140802065729.2a2d7ebc@scorpio> Message-ID: <53DCC6DE.30203@thinline.cz> On 2.8.2014 12:57, Jerry wrote: > Sat, 2 Aug 2014 06:53:58 -0400 > > Presently, I have: "virtual_transport = dovecot" in my Postfix main.cf file > and > dovecot unix - n n - - pipe > flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient} > in the master.cf file. > > Is there any advantage to using LMPT instead? All of my users are configured > as virtual users. If I change it, do I have to eliminate the above from my > master.cf file? > As far as I know, LMTP process in Dovecot is long-living one, so on busy server you can save quite a few CPU cycles that go into spawning dovecot-lda for each incoming message. Also error handling could be (not sure) a bit better, with pipe service you only have dovecot-lda's return code indicating success/failure whereas LMTP can tell you what went wrong. (Well, Postfix actually scans executed program's output for status codes but I don't think dovecot-lda emits that.) You can keep your dovecot service in the configuration - it does no harm especially if it's not used. (OTOH there's no reason to keep it there if you switch to LMTP.) From tim at titan21.co.uk Sat Aug 2 18:37:50 2014 From: tim at titan21.co.uk (Tim) Date: Sat, 02 Aug 2014 19:37:50 +0100 Subject: INBOX subfolders not showing up In-Reply-To: <53DA5F6A.9000304@familie-sitte.org> References: <53DA5F6A.9000304@familie-sitte.org> Message-ID: <53DD2FFE.8060503@titan21.co.uk> I came across something like this recently and it turned out it was a permissions issue on a users maildir/ folder. Check that - took me hours to figure it out and 2 secs to fix. Have you also checked your logs to see if there are any errors? On 31/07/14 16:23, Matthias Sitte wrote: > Hi, > > I have a problem with INBOX subfolders and subscriptions which I can't > solve... Maybe somebody has seen this, too... > > I can create regular subfolders (not in INBOX) and (un-)subscribe to > them with Thunderbird. If I create subfolders in the INBOX with TB > this seems to work now in v31 (just found this out), but with v24.6 > neither INBOX subfolder creation nor subscriptions worked. When I > modified the `subscriptions' file manually on the server, the folders > would show up in TB, though. > > Maybe this was a TB bug, but quite similarly INBOX subfolders don't > show up on iPhones (iOS up-to-date), but all other subfolders do. > > Any ideas what's going on? > > Best, > Matthias > > ====================================================================== > > # dovecot -n > # 2.1.7: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4 > hostname = familie-sitte.org > log_path = /var/log/dovecot.log > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/srv/mail/%d/%n/Maildir:LAYOUT=fs > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace inbox { > inbox = yes > list = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > separator = / > subscriptions = yes > type = private > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > plugin { > quota = maildir:User quota > quota_warning = storage=95%% quota-warning 95 %u > quota_warning2 = storage=80%% quota-warning 80 %u > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > postmaster_address = postmaster at familie-sitte.org > protocols = " imap sieve pop3" > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > } > service imap-login { > inet_listener imaps { > port = 993 > ssl = yes > } > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > } > service quota-warning { > executable = script /usr/local/bin/quota-warning.sh > unix_listener quota-warning { > user = vmail > } > user = dovecot > } > ssl_cert = ssl_cipher_list = > SSDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA:EECDH:RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS > ssl_key = userdb { > args = uid=vmail gid=vmail home=/srv/mail/%d/%n > driver = static > } > protocol lda { > mail_plugins = " quota sieve" > } > protocol imap { > mail_plugins = " imap_quota quota" > } -- Tim Smith Tel: 01423 564078 Mob: 07984 398 299 Email: tim at titan21.co.uk Web: www.titan21.co.uk From tss at iki.fi Sat Aug 2 20:45:05 2014 From: tss at iki.fi (Timo Sirainen) Date: Sat, 2 Aug 2014 23:45:05 +0300 Subject: autocreate mailboxes In-Reply-To: <53DC0601.7010401@gedalya.net> References: <026c01cfadca$c4ff04c0$4efd0e40$@mikepultz.com> <53DC0601.7010401@gedalya.net> Message-ID: On 02 Aug 2014, at 00:26, Gedalya wrote: > On 08/01/2014 04:54 PM, Mike Pultz wrote: >> I wanted to take advantage of the mailbox option to auto create and auto >> subscribe users to a Sent and Trash folder (using the support for the >> special use flags), but ideally, I only want to do this for new mailboxes, >> and not for all the mailboxes that already exist on the system. > > I guess this could be done with extra fields: http://wiki2.dovecot.org/UserDatabase/ExtraFields > > The syntax being something like namespace/inbox/mailbox/Sent/special_use=\Sent ..? Or something like that. Note the userdb_import facility for passing multiple options. > > **I wonder if it wouldn't be useful to be able to create configuration sections and activate them with a single, simple extra field rather than pass the entire section itself? > In a similar manner to protocol imap { }, maybe we can have $extra_options1 == "yes" { } and pass extra_options1 = yes as an extra field? Maybe, but could be a lot of work. For this case however it's easy. Create two namespaces: namespace inbox { prefix = inbox = yes mailbox Sent { auto = create } # ..etc.. } namespace old { prefix = inbox = yes disabled = yes } Then in the extra field you can just set namespace/inbox/disabled=yes and namespace/old/disabled=no From lazy404 at gmail.com Sun Aug 3 13:13:38 2014 From: lazy404 at gmail.com (Lazy) Date: Sun, 3 Aug 2014 15:13:38 +0200 Subject: [PATH] expire plugins segfault with misconfigured dict In-Reply-To: References: Message-ID: 2014-07-07 15:18 GMT+02:00 Lazy : > dovecot 2.2.13 > > With configuration like > > plugin { > > expire_dict = blblbla > } > > expiere plugins segfaults during deinit. It might also segfault if the > dict service was unavaiable. > > attached bellow is a diff against 2.2.13 > > Jul 7 14:51:11 dovecot1 dovecot: lmtp(10548, lazy): Error: expire > plugin: dict_init(blblbla) failed: Dictionary URI is missing ':': > blblbla > > Jul 7 14:51:11 dovecot1 dovecot: lmtp(10548, lazy): Fatal: master: > service(lmtp): child 10548 killed with signal 11 (core dumps disabled) > > > Program received signal SIGSEGV, Segmentation fault. > > 0x00007f337d8af8ae in dict_deinit (_dict=0x8) at dict.c:87 > > 87 struct dict *dict = *_dict; > > (gdb) bt > > #0 0x00007f337d8af8ae in dict_deinit (_dict=0x8) at dict.c:87 > > #1 0x00007f337c88f647 in expire_mail_user_deinit (user=0x1b60e80) at > expire-plugin.c:299 > > #2 0x00007f337cca3e7f in quota_user_deinit (user=0x1b60e80) at > quota-storage.c:491 > > #3 0x00007f337dbcb0f1 in mail_user_unref (_user=0x1b2d070) at mail-user.c:163 > > #4 0x000000000040832b in client_input_data_write_local > (client=0x1b2cf60, input=0x1b43f10) at commands.c:830 > > #5 0x0000000000408755 in client_input_data_write (client=0x1b2cf60) > at commands.c:939 > > #6 0x0000000000408b18 in client_input_data_handle (client=0x1b2cf60) > at commands.c:1033 > > #7 0x0000000000408b52 in client_input_data (client=0x1b2cf60) at > commands.c:1042 > > #8 0x00007f337d8fc723 in io_loop_call_io (io=0x1b2c360) at ioloop.c:441 > > #9 0x00007f337d8fe6e5 in io_loop_handler_run_internal > (ioloop=0x1b27740) at ioloop-epoll.c:220 > > #10 0x00007f337d8fc8be in io_loop_handler_run (ioloop=0x1b27740) at ioloop.c:488 > > #11 0x00007f337d8fc817 in io_loop_run (ioloop=0x1b27740) at ioloop.c:465 > > #12 0x00007f337d888d40 in master_service_run (service=0x1b275d0, > callback=0x404b5c ) at master-service.c:566 > > #13 0x0000000000404db2 in main (argc=1, argv=0x1b27390) at main.c:122 > > (gdb) p _dict > > $1 = (struct dict **) 0x8 > > (gdb) bt full > > #0 0x00007f337d8af8ae in dict_deinit (_dict=0x8) at dict.c:87 > > dict = 0x10 > > #1 0x00007f337c88f647 in expire_mail_user_deinit (user=0x1b60e80) at > expire-plugin.c:299 > > euser = 0x0 > > #2 0x00007f337cca3e7f in quota_user_deinit (user=0x1b60e80) at > quota-storage.c:491 > > quser = 0x1b62958 > > quota_set = 0x1b656a0 > > #3 0x00007f337dbcb0f1 in mail_user_unref (_user=0x1b2d070) at mail-user.c:163 > > user = 0x1b60e80 > > __FUNCTION__ = "mail_user_unref" > > #4 0x000000000040832b in client_input_data_write_local > (client=0x1b2cf60, input=0x1b43f10) at commands.c:830 > > session = 0x1b60a70 > > src_mail = 0x1b5c250 > > old_uid = 300 > > first_uid = 4294967295 > > __FUNCTION__ = "client_input_data_write_local" > > #5 0x0000000000408755 in client_input_data_write (client=0x1b2cf60) > at commands.c:939 > > input = 0x1b43f10 > > ret = true > > #6 0x0000000000408b18 in client_input_data_handle (client=0x1b2cf60) > at commands.c:1033 > > data = 0x1b556b8 "\330\001" > > size = 2 > > ret = -1 > > #7 0x0000000000408b52 in client_input_data (client=0x1b2cf60) at > commands.c:1042 > reposting -- Michal Grzedzicki --- src/plugins/expire/expire-plugin.c (wersja 1343) +++ src/plugins/expire/expire-plugin.c (kopia robocza) @@ -296,10 +296,12 @@ { struct expire_mail_user *euser = EXPIRE_USER_CONTEXT(user); - dict_deinit(&euser->db); - expire_set_deinit(&euser->set); + if (euser != NULL) { + dict_deinit(&euser->db); + expire_set_deinit(&euser->set); - euser->module_ctx.super.deinit(user); + euser->module_ctx.super.deinit(user); + } } static const char *const *expire_get_patterns(struct mail_user *user) From jreid at vnet.net Sun Aug 3 22:32:21 2014 From: jreid at vnet.net (Joe Reid) Date: Sun, 03 Aug 2014 17:32:21 -0500 Subject: permission denied Message-ID: <53DEB875.6080801@vnet.net> I'm getting 2 errors which are probably related, the second of which I haven't actually seen an issue with yet (I just found it looking at the error log) The issue is that I cannot change my subscriptions through Thunderbird (version 31.0). I created the subscriptions through Thunderbird, but now it won't let me change them. dovecot responds with sutff like this when I sniff network traffic or configure Thunderbird to log IMAP traffic: 10 NO [NOPERM] No permission to modify subscriptions The time stamp on the .subscriptions.lock file gets updated every time I try to make a change. To test I edited .subscriptions by hand and forced Thunderbird to try to subscribe something and I got access to the folder I added, so I presume it is reading the file correctly. Permissions on~jreid/mail is 0700, the existing .subscriptions file is 0640 and .subscriptions.lock file is 0600. all the files are owned by me and their group is my primary group (not mail). As for the second error, the dovecot.index.log.newlock file gets created but no dovecot.index.log file gets created. And dovecot would have created the directory tree to those files itself... I'm new to dovecot but not Unix or IMAP so I'm sure this is just some configuration option I've missed, any help would be most appreciated. $ /opt/csw/bin/doveconf -n # 2.2.12: /etc/opt/csw/dovecot/dovecot.conf # OS: SunOS 5.11 i86pc disable_plaintext_auth = no first_valid_uid = 100 mail_access_groups = mail mail_location = mbox:~/mail:INBOX=/var/share/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = imap ssl_cert = References: <53DA5F6A.9000304@familie-sitte.org> <53DD2FFE.8060503@titan21.co.uk> Message-ID: <53DED786.3080109@familie-sitte.org> Well, I created a test user to check what kind of permissions Dovecot gives (after reading the docs I was kinda confused a bit). Except for the INBOX folder (which had some group/other read flags) everything was the same. I've corrected that. Folders created with Thunderbird (24.5) now show up, but I have to restart it. Well, I can live with that. On the other hand, the logs don't show anything unusual, no immediate warnings or errors about permissions. I'm still confused about it. I remember vaguely that it worked with the previous major version of Dovecot before upgrading to Wheezy... On 08/02/2014 01:37 PM, Tim wrote: > I came across something like this recently and it turned out it was a > permissions issue on a users maildir/ folder. Check that - took me hours > to figure it out and 2 secs to fix. Have you also checked your logs to > see if there are any errors? > > On 31/07/14 16:23, Matthias Sitte wrote: >> Hi, >> >> I have a problem with INBOX subfolders and subscriptions which I can't >> solve... Maybe somebody has seen this, too... >> >> I can create regular subfolders (not in INBOX) and (un-)subscribe to >> them with Thunderbird. If I create subfolders in the INBOX with TB >> this seems to work now in v31 (just found this out), but with v24.6 >> neither INBOX subfolder creation nor subscriptions worked. When I >> modified the `subscriptions' file manually on the server, the folders >> would show up in TB, though. >> >> Maybe this was a TB bug, but quite similarly INBOX subfolders don't >> show up on iPhones (iOS up-to-date), but all other subfolders do. >> >> Any ideas what's going on? >> >> Best, >> Matthias >> >> ====================================================================== >> >> # dovecot -n >> # 2.1.7: /etc/dovecot/dovecot.conf >> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4 >> hostname = familie-sitte.org >> log_path = /var/log/dovecot.log >> log_timestamp = "%Y-%m-%d %H:%M:%S " >> mail_location = maildir:/srv/mail/%d/%n/Maildir:LAYOUT=fs >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace inbox { >> inbox = yes >> list = yes >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> separator = / >> subscriptions = yes >> type = private >> } >> passdb { >> args = /etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> quota = maildir:User quota >> quota_warning = storage=95%% quota-warning 95 %u >> quota_warning2 = storage=80%% quota-warning 80 %u >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> postmaster_address = postmaster at familie-sitte.org >> protocols = " imap sieve pop3" >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service imap-login { >> inet_listener imaps { >> port = 993 >> ssl = yes >> } >> } >> service managesieve-login { >> inet_listener sieve { >> port = 4190 >> } >> } >> service quota-warning { >> executable = script /usr/local/bin/quota-warning.sh >> unix_listener quota-warning { >> user = vmail >> } >> user = dovecot >> } >> ssl_cert = > ssl_cipher_list = >> SSDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA:EECDH:RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS >> >> ssl_key = > userdb { >> args = uid=vmail gid=vmail home=/srv/mail/%d/%n >> driver = static >> } >> protocol lda { >> mail_plugins = " quota sieve" >> } >> protocol imap { >> mail_plugins = " imap_quota quota" >> } > -- Dr. Matthias Sitte Email: matthias at familie-sitte.org 1003 Justin Lane matthias.sitte at gmail.com Apt. 1037 matthias.sitte at gmx.de Austin, TX 78757-2644 UNITED STATES OF AMERICA From matthias at familie-sitte.org Mon Aug 4 01:01:08 2014 From: matthias at familie-sitte.org (Dr. Matthias Sitte) Date: Sun, 03 Aug 2014 20:01:08 -0500 Subject: INBOX subfolders not showing up In-Reply-To: <53DD2FFE.8060503@titan21.co.uk> References: <53DA5F6A.9000304@familie-sitte.org> <53DD2FFE.8060503@titan21.co.uk> Message-ID: <53DEDB54.3030800@familie-sitte.org> Ok, just googled some more and came across this post [1]. It's basically a very similar to mine, and describes the same problem that INBOX has no subfolders. So, I've logged in with telnet to list the INBOX subfolders as well: === SNIP === 2 list "" INBOX * LIST (\HasNoChildren) "/" "INBOX" 2 OK List completed. === SNAP === Duh. But everything else shows up: === SNIP === 4 list "" "*" [...] * LIST (\HasChildren) "/" "Archives" [...] 4 OK List completed. === SNAP === So, what just happened? According to [1], there's a bug in lib-storage with LAYOUT=fs and non-mbox maildir format [2]. Is that correct? I have to assume that it's not fixed in Debian's packaged version 2.1.7, right? Can someone confirm that? [1] http://dovecot.2317879.n4.nabble.com/Subfolders-of-Inbox-with-LAYOUT-fs-td43659.html [2] http://hg.dovecot.org/dovecot-2.1/rev/589d2f485a6e On 08/02/2014 01:37 PM, Tim wrote: > I came across something like this recently and it turned out it was a > permissions issue on a users maildir/ folder. Check that - took me hours > to figure it out and 2 secs to fix. Have you also checked your logs to > see if there are any errors? > > On 31/07/14 16:23, Matthias Sitte wrote: >> Hi, >> >> I have a problem with INBOX subfolders and subscriptions which I can't >> solve... Maybe somebody has seen this, too... >> >> I can create regular subfolders (not in INBOX) and (un-)subscribe to >> them with Thunderbird. If I create subfolders in the INBOX with TB >> this seems to work now in v31 (just found this out), but with v24.6 >> neither INBOX subfolder creation nor subscriptions worked. When I >> modified the `subscriptions' file manually on the server, the folders >> would show up in TB, though. >> >> Maybe this was a TB bug, but quite similarly INBOX subfolders don't >> show up on iPhones (iOS up-to-date), but all other subfolders do. >> >> Any ideas what's going on? >> >> Best, >> Matthias >> >> ====================================================================== >> >> # dovecot -n >> # 2.1.7: /etc/dovecot/dovecot.conf >> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4 >> hostname = familie-sitte.org >> log_path = /var/log/dovecot.log >> log_timestamp = "%Y-%m-%d %H:%M:%S " >> mail_location = maildir:/srv/mail/%d/%n/Maildir:LAYOUT=fs >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace inbox { >> inbox = yes >> list = yes >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> prefix = >> separator = / >> subscriptions = yes >> type = private >> } >> passdb { >> args = /etc/dovecot/dovecot-sql.conf.ext >> driver = sql >> } >> plugin { >> quota = maildir:User quota >> quota_warning = storage=95%% quota-warning 95 %u >> quota_warning2 = storage=80%% quota-warning 80 %u >> sieve = ~/.dovecot.sieve >> sieve_dir = ~/sieve >> } >> postmaster_address = postmaster at familie-sitte.org >> protocols = " imap sieve pop3" >> service auth { >> unix_listener /var/spool/postfix/private/auth { >> group = postfix >> mode = 0660 >> user = postfix >> } >> } >> service imap-login { >> inet_listener imaps { >> port = 993 >> ssl = yes >> } >> } >> service managesieve-login { >> inet_listener sieve { >> port = 4190 >> } >> } >> service quota-warning { >> executable = script /usr/local/bin/quota-warning.sh >> unix_listener quota-warning { >> user = vmail >> } >> user = dovecot >> } >> ssl_cert = > ssl_cipher_list = >> SSDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA:EECDH:RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS >> >> ssl_key = > userdb { >> args = uid=vmail gid=vmail home=/srv/mail/%d/%n >> driver = static >> } >> protocol lda { >> mail_plugins = " quota sieve" >> } >> protocol imap { >> mail_plugins = " imap_quota quota" >> } > -- Dr. Matthias Sitte Email: matthias at familie-sitte.org 1003 Justin Lane matthias.sitte at gmail.com Apt. 1037 matthias.sitte at gmx.de Austin, TX 78757-2644 UNITED STATES OF AMERICA From tech at frontrowcomputer.com Mon Aug 4 01:45:09 2014 From: tech at frontrowcomputer.com (Tech Support Department) Date: Sun, 03 Aug 2014 21:45:09 -0400 Subject: Certificates Message-ID: <53DEE5A5.8080809@frontrowcomputer.com> When running Dovecot with Postfix under Virtualmin on my VPS, I was wondering if there was a way for it to handle a separate certificate for each domain and IP address? Thank you. From gedalya at gedalya.net Mon Aug 4 02:31:07 2014 From: gedalya at gedalya.net (Gedalya) Date: Sun, 03 Aug 2014 22:31:07 -0400 Subject: Certificates In-Reply-To: <53DEE5A5.8080809@frontrowcomputer.com> References: <53DEE5A5.8080809@frontrowcomputer.com> Message-ID: <53DEF06B.9050107@gedalya.net> On 08/03/2014 09:45 PM, Tech Support Department wrote: > When running Dovecot with Postfix under Virtualmin on my VPS, I was > wondering if there was a way for it to handle a separate certificate > for each domain and IP address? > > Thank you. Yes. http://wiki2.dovecot.org/SSL/DovecotConfiguration http://wiki2.dovecot.org/SSL/SNIClientSupport If you dedicate an IP address per domain, it's easy. The scenario of multiple domains per IP address is called SNI, and it has limited support by clients, and therefore may not be a realistic solution for you. From skdovecot at smail.inf.fh-brs.de Mon Aug 4 07:57:05 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Aug 2014 09:57:05 +0200 (CEST) Subject: permission denied In-Reply-To: <53DEB875.6080801@vnet.net> References: <53DEB875.6080801@vnet.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sun, 3 Aug 2014, Joe Reid wrote: > The time stamp on the .subscriptions.lock file gets updated every time I try > to make a change. To test I edited .subscriptions by hand and forced > Thunderbird to try to subscribe something and I got access to the folder I > added, so I presume it is reading the file correctly. > > Permissions on~jreid/mail is 0700, the existing .subscriptions file is 0640 > and .subscriptions.lock file is 0600. all the files are owned by me and > their group is my primary group (not mail). > > $ /opt/csw/bin/doveconf -n > # 2.2.12: /etc/opt/csw/dovecot/dovecot.conf > # OS: SunOS 5.11 i86pc > disable_plaintext_auth = no > first_valid_uid = 100 > mail_access_groups = mail > mail_location = mbox:~/mail:INBOX=/var/share/mail/%u [cut] > passdb { > driver = pam > } > userdb { > driver = passwd > } > $ doveadm log errors > [...] > Aug 03 16:47:02 imap(jreid): Error: > rename(/home/jreid/mail/.subscriptions.lock, /home/jreid/mail/.subscriptions) > failed: Permission denied > Aug 03 16:52:49 imap(jreid): Error: > rename(/home/jreid/mail/2012/.imap/Received/dovecot.index.log.newlock, > /home/jreid/mail/2012/.imap/Received/dovecot.index.log) failed: Permission > denied Who owns /home/jreid/mail, .subscriptions.lock file and /home/jreid/mail/2012/.imap/Received ? Please send a ls -la of /home/jreid/mail. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU9880Xz1H7kL/d9rAQI4kgf7B7nn7mzp8WHqYkD7azYToI7oYxIpoO1e bi29AjlWNqTkwrxmCyhKbUqwaGan+ZbxN1xRDxsh8q7SoN3p6uF3ot1os2vBrX/1 UuSsxKEohi7/9S6x/eN9a/gKbRsKfb8wMUwBIW4HlWJgLuaywCqwl1WOk78RpddG zUPsH3W63X/8lx84n6ZBqwddCYp/R4IYsQdyQMpMCjpj0DzTqniWhtd2i3ZiyaaF dOhaWgcjTZ1cjr4iJnZniylHkuS4NsFZejSpM1An41aq4b0b51FHSGG8R3ze+sop LVTctxCK7U+/sO296Lv5Lu49ndAlx0H7KViD9RKFWzaPtgQ+vDADZw== =k6IU -----END PGP SIGNATURE----- From lgroups at waagmeester.co.za Mon Aug 4 08:00:32 2014 From: lgroups at waagmeester.co.za (Coert) Date: Mon, 04 Aug 2014 10:00:32 +0200 Subject: Dovecot with dbox indexing weirdness Message-ID: <1828537.izTKSc659U@penguin> Hello all, I run a local dovecot imap server, with dbox storage in my home directory. It works well, but everytime I restart the machine, it seems according to lsof to reindex everything (and I am sitting with 250k mails). Is this the intended behaviour? I would expect it to happen only once or on update of a folder? regards, Coert From greg.sullivan at sullivang.net Mon Aug 4 08:44:43 2014 From: greg.sullivan at sullivang.net (Greg Sullivan) Date: Mon, 4 Aug 2014 18:44:43 +1000 Subject: Is atomic MOVING of messages between IMAP folders possible? Message-ID: I would like to use a shared IMAP account, with multiple users accessing it simultaneously. The users would take ownership of messages by first attempting to MOVE the messages from the Inbox, into their private IMAP folder, still within the same account. Now, since there will be multiple users competing for the same messages, I naturally want only ONE of the simultaneous moves to be successful at a time. So far, this isn't working. If I do the move from two clients, simultaneously, the messages can go to *both *destination folders - duplicates can result. Is it possible to configure Dovecot and/or an IMAP client to behave the way I want it to? If the answer to this is YES, then I'll offer my config details. If the answer is NO, the next question is - do any email systems at all behave the way I want? (I tried a hosted Exchange/OWA service - it has the same problem) Thanks & regards, Greg. From gayleard at alice.it Mon Aug 4 10:01:36 2014 From: gayleard at alice.it (Timothy Murphy) Date: Mon, 04 Aug 2014 12:01:36 +0200 Subject: postfix-dovecot via lmtp Message-ID: <1846988.K2TEy98DxC@rose.gayleard.eu> I've been trying for some days to get postfix + dovecot + pigeonhole/sieve working on a CentOS-7 home server. I think I may have detected the problem; I read in /var/log/maillog Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: to=, orig_to=, relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 Internal error occurred. Refer to server log for more information. (in reply to RCPT TO command)) (I've split one long line here.) 1) I see that the permissions on /var/spool/postfix/private/ are [tim at alfred ~]$ ls -ls /var/spool/postfix/ 4 drwx------. 2 postfix root 4096 Aug 4 11:30 private The socket dovecot-lmtp setup by postfix is in this directory; does dovecot need permission to access this? 2) I see that this mail (a test email from me) was deferred. What exactly does this mean? Where is the mail stored? I couldn't find it anywhere. (It has already been brought to the server by fetchmail.) 3) Am I right in thinking that there are 2 alternatives to dovecot/sieve a) amavisd-new with postfix + spamassassin b) postfix + procmail ? 4) I remember seeing a posting (probably not in this mailing list) describing how one can trace the progress of email through the server. Unfortunately I didn't note the URL, and now cannot find it. Does anyone have a pointer to such a document? As will probably be obvious, I am not at all clear how postfix and dovecot fit together if one introduces lmtp. I take it that without lmtp (or similar) postfix just delivers email to ~/Maildir/cur (in my case), and dovecot is not involved at all until a client (KMail in my case) looks at email? -- Timothy Murphy e-mail: gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin 2, Ireland From h.reindl at thelounge.net Mon Aug 4 10:14:44 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 04 Aug 2014 12:14:44 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: <1846988.K2TEy98DxC@rose.gayleard.eu> References: <1846988.K2TEy98DxC@rose.gayleard.eu> Message-ID: <53DF5D14.4000302@thelounge.net> Am 04.08.2014 um 12:01 schrieb Timothy Murphy: > I've been trying for some days to get postfix + dovecot + pigeonhole/sieve > working on a CentOS-7 home server. > I think I may have detected the problem; I read in /var/log/maillog > Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: > to=, orig_to=, > relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, > delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred > (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 > Internal error occurred. > Refer to server log for more information. (in reply to RCPT TO command)) > (I've split one long line here.) "Refer to server log for more information" comes from dovecot lmtpd so look there and post the output of "doveconf -n" > 1) I see that the permissions on /var/spool/postfix/private/ are > [tim at alfred ~]$ ls -ls /var/spool/postfix/ > 4 drwx------. 2 postfix root 4096 Aug 4 11:30 private > The socket dovecot-lmtp setup by postfix is in this directory; > does dovecot need permission to access this? they talk to each other since dovecot replies over ltmpd -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 246 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Mon Aug 4 10:54:44 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 04 Aug 2014 12:54:44 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: <4621613.Zgn1ePRBzZ@rose.gayleard.eu> References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53DF5D14.4000302@thelounge.net> <4621613.Zgn1ePRBzZ@rose.gayleard.eu> Message-ID: <53DF6674.2060700@thelounge.net> STOP to post HTML and convert even plaintext messages Am 04.08.2014 um 12:50 schrieb Timothy Murphy: > On Monday, August 04, 2014 12:14:44 PM Reindl Harald wrote: > > > >> > I read in /var/log/maillog > >> > > >> > Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: > >> > to=, orig_to=, > >> > relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, > >> > delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred > >> > (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 > >> > Internal error occurred. > >> > Refer to server log for more information. (in reply to RCPT TO > >> > command)) > > > >> "Refer to server log for more information" comes from dovecot lmtpd > >> so look there and post the output of "doveconf -n" > > > > Thanks very much for your response. > > > > The only information in /var/log/dovecot > > (where I send dovecot debug and info logs) is > > Aug 04 11:29:13 lmtp(14872): > > Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= > > Aug 04 11:29:13 lmtp(14872): > > Info: Connect from local > > Aug 04 11:29:13 lmtp(14872): > > Debug: auth input: > > Aug 04 11:29:13 lmtp(14872): > > Info: Disconnect from local: Successful quit > > > > The output of "doveconf -n" is > > ------------------------- > > # 2.2.10: /etc/dovecot/dovecot.conf > > # OS: Linux 3.10.0-123.4.4.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) > > debug_log_path = /var/log/dovecot > > info_log_path = /var/log/dovecot > > listen = * > > log_path = /var/log/dovecot > > login_trusted_networks = 192.168.2.0/24 > > mail_debug = yes > > mail_gid = 1000 > > mail_location = maildir:~/Maildir > > mail_uid = 1000 > > managesieve_notify_capability = mailto > > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress > comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date > ihave > > mbox_write_locks = fcntl > > namespace inbox { > > inbox = yes > > location = > > mailbox Drafts { > > special_use = \Drafts > > } > > mailbox Junk { > > special_use = \Junk > > } > > mailbox Sent { > > special_use = \Sent > > } > > mailbox "Sent Messages" { > > special_use = \Sent > > } > > mailbox Trash { > > special_use = \Trash > > } > > prefix = > > } > > passdb { > > driver = pam > > } > > plugin { > > sieve = ~/.sieve/dovecot.sieve > > sieve_dir = ~/.sieve > > } > > protocols = imap pop3 lmtp sieve > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > inet_listener imaps { > > port = 993 > > ssl = yes > > } > > } > > service lmtp { > > unix_listener /var/spool/postfix/private/dovecot-lmtp { > > group = postfix > > mode = 0777 > > user = postfix > > } > > } > > service managesieve-login { > > inet_listener sieve { > > port = 4190 > > } > > } > > ssl = required > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > protocol lmtp { > > mail_plugins = " sieve" > > } > > protocol lda { > > mail_plugins = " sieve" > > } > > protocol sieve { > > managesieve_notify_capability = fileinto reject envelope environment mailbox date > > } > > ------------------------- > > > > > > -- > > Timothy Murphy > > e-mail: gayleard /at/ eircom.net > > School of Mathematics, Trinity College, Dublin 2, Ireland -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 246 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Mon Aug 4 11:06:25 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 04 Aug 2014 13:06:25 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: <4005594.s5t1RB772I@rose.gayleard.eu> References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53DF5D14.4000302@thelounge.net> <4005594.s5t1RB772I@rose.gayleard.eu> Message-ID: <53DF6931.3000901@thelounge.net> the same HTML crap, even after 4 medical operations i am not blind, here the thread ends for me, not interested in discuss about mailserves if even handling the mail-client fails P.S.: the same applies for "reply-all" Am 04.08.2014 um 13:00 schrieb Timothy Murphy: > On Monday, August 04, 2014 12:14:44 PM Reindl Harald wrote: > > > >> > I read in /var/log/maillog > >> > > >> > Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: > >> > to=, orig_to=, > >> > relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, > >> > delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred > >> > (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 > >> > Internal error occurred. > >> > Refer to server log for more information. (in reply to RCPT TO > >> > command)) > >> > >> "Refer to server log for more information" comes from dovecot lmtpd > >> so look there and post the output of "doveconf -n" > > > > Thanks very much for your response. > > > > The only output in /var/log/dovecot (where I send dovecot logs) is > > -------------------------- > > Aug 04 11:29:13 lmtp(14872): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= > > Aug 04 11:29:13 lmtp(14872): Info: Connect from local > > Aug 04 11:29:13 lmtp(14872): Debug: auth input: > > Aug 04 11:29:13 lmtp(14872): Info: Disconnect from local: Successful quit > > -------------------------- > > > > The output of "doveconf -n" is > > -------------------------- > > # 2.2.10: /etc/dovecot/dovecot.conf > > # OS: Linux 3.10.0-123.4.4.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) > > debug_log_path = /var/log/dovecot > > info_log_path = /var/log/dovecot > > listen = * > > log_path = /var/log/dovecot > > login_trusted_networks = 192.168.2.0/24 > > mail_debug = yes > > mail_gid = 1000 > > mail_location = maildir:~/Maildir > > mail_uid = 1000 > > managesieve_notify_capability = mailto > > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress > comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date > ihave > > mbox_write_locks = fcntl > > namespace inbox { > > inbox = yes > > location = > > mailbox Drafts { > > special_use = \Drafts > > } > > mailbox Junk { > > special_use = \Junk > > } > > mailbox Sent { > > special_use = \Sent > > } > > mailbox "Sent Messages" { > > special_use = \Sent > > } > > mailbox Trash { > > special_use = \Trash > > } > > prefix = > > } > > passdb { > > driver = pam > > } > > plugin { > > sieve = ~/.sieve/dovecot.sieve > > sieve_dir = ~/.sieve > > } > > protocols = imap pop3 lmtp sieve > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > inet_listener imaps { > > port = 993 > > ssl = yes > > } > > } > > service lmtp { > > unix_listener /var/spool/postfix/private/dovecot-lmtp { > > group = postfix > > mode = 0777 > > user = postfix > > } > > } > > service managesieve-login { > > inet_listener sieve { > > port = 4190 > > } > > } > > ssl = required > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > protocol lmtp { > > mail_plugins = " sieve" > > } > > protocol lda { > > mail_plugins = " sieve" > > } > > protocol sieve { > > managesieve_notify_capability = fileinto reject envelope environment mailbox date > > } > > -------------------------- > > > > -- > > Timothy Murphy > > e-mail: gayleard /at/ eircom.net > > School of Mathematics, Trinity College, Dublin 2, Ireland -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 246 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Mon Aug 4 12:00:38 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Aug 2014 14:00:38 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: Message-ID: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> On 04 Aug 2014, at 10:44, Greg Sullivan wrote: > I would like to use a shared IMAP account, with multiple users accessing it > simultaneously. The users would take ownership of messages by first > attempting to MOVE the messages from the Inbox, into their private IMAP > folder, still within the same account. Now, since there will be multiple > users competing for the same messages, I naturally want only ONE of the > simultaneous moves to be successful at a time. > > So far, this isn't working. If I do the move from two clients, > simultaneously, the messages can go to *both *destination folders - > duplicates can result. > > Is it possible to configure Dovecot and/or an IMAP client to behave the way > I want it to? If the answer to this is YES, then I'll offer my config > details. If the answer is NO, the next question is - do any email systems > at all behave the way I want? (I tried a hosted Exchange/OWA service - it > has the same problem) Dovecot doesn't even attempt to do atomic MOVEs. I don't think any server will. If you can change the client code, you could use CONDSTORE instead, which does give atomic STOREs. From skdovecot at smail.inf.fh-brs.de Mon Aug 4 12:04:20 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 4 Aug 2014 14:04:20 +0200 (CEST) Subject: not able to compile deleted_to_trash Plugin In-Reply-To: <490991D6720B3F44BC4230600E34A546018EAB984C@srv01.sma.local> References: <490991D6720B3F44BC4230600E34A546018EAB984C@srv01.sma.local> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 28 Jul 2014, Pimminger Alfred wrote: > root at ABCD:/CavE/download/delete_to_trash# dovecot --version > 2.2.9 > ---------------------------------------------------------------------------------------- > > I get the following output after doing make > > root at ABCD:/CavE/download/delete_to_trash# make I've justP compiled https://github.com/lexbrugman/dovecot_deleted_to_trash/archive/master.zip with v2.2.13+ successfully. It also did something, I mean, there are some log messages about moving/not moving into trash. Maybe, you check that you have the correct / most uptodate version; or you check in which release the problematic files have been changed. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU992xXz1H7kL/d9rAQKGngf/clNFaQbPvwKPugvotyLYZTlYDo1ewExZ DWKu09JOw3LUaD3Mi28vlp2QVOQkot0AhSejG4mn1rErBzqnE3LnfLN6SWfteTGA YZzocixDslEWM5kDyaDPmAOmMfSvJQ8Pori0ECAKz82AUxx+fAFyF1LvBO0srD9q zv6miZrNSl3ipZE+BDwJG5g9sKmKh7Ro3S6hQtY2qfvkjWREexx7ILbBJWkcnN0q z2FxxuFtg9/7lMgnKoS9DXCbqT+xBm7MTeyJvYtnZtTC6PAWQagklJVXLinrUnjf nf8Z3aHCvaFrJqae6zbrwOaGvyVIbBNEVNqNvPCh7kDRfQ9aAqhSWA== =scg4 -----END PGP SIGNATURE----- From h.reindl at thelounge.net Mon Aug 4 12:03:09 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 04 Aug 2014 14:03:09 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> Message-ID: <53DF767D.4060008@thelounge.net> Am 04.08.2014 um 14:00 schrieb Timo Sirainen: > On 04 Aug 2014, at 10:44, Greg Sullivan wrote: > >> I would like to use a shared IMAP account, with multiple users accessing it >> simultaneously. The users would take ownership of messages by first >> attempting to MOVE the messages from the Inbox, into their private IMAP >> folder, still within the same account. Now, since there will be multiple >> users competing for the same messages, I naturally want only ONE of the >> simultaneous moves to be successful at a time. >> >> So far, this isn't working. If I do the move from two clients, >> simultaneously, the messages can go to *both *destination folders - >> duplicates can result. >> >> Is it possible to configure Dovecot and/or an IMAP client to behave the way >> I want it to? If the answer to this is YES, then I'll offer my config >> details. If the answer is NO, the next question is - do any email systems >> at all behave the way I want? (I tried a hosted Exchange/OWA service - it >> has the same problem) > > Dovecot doesn't even attempt to do atomic MOVEs. I don't think any server will. If you can change the client code, you could use CONDSTORE instead, which does give atomic STOREs. Internet Message Access Protocol (IMAP) - MOVE Extension http://tools.ietf.org/html/rfc6851 well, both, client and server would need to support it rely on that is unlikely for many years -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 246 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Mon Aug 4 12:07:14 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Aug 2014 14:07:14 +0200 Subject: [PATH] expire plugins segfault with misconfigured dict In-Reply-To: References: Message-ID: On 03 Aug 2014, at 15:13, Lazy wrote: >> plugin { >> >> expire_dict = blblbla >> } >> >> expiere plugins segfaults during deinit. It might also segfault if the >> dict service was unavaiable. >> >> attached bellow is a diff against 2.2.13 Thanks! I fixed it differently though, better to handle the dict_init() failure the same as other failures: http://hg.dovecot.org/dovecot-2.2/rev/cac32684b3d6 From greg.sullivan at sullivang.net Mon Aug 4 12:12:46 2014 From: greg.sullivan at sullivang.net (Greg Sullivan) Date: Mon, 4 Aug 2014 22:12:46 +1000 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <53DF767D.4060008@thelounge.net> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> Message-ID: Yes, both client and server support IMAP MOVE, and both also support CONDSTORE. I have tried both with and without CONDSTORE enabled in the client, with the same result. I am very confident IMAP MOVE is actually being invoked, because intra-account moves occur extremely rapidly. (much faster than inter-account moves, which of course is a copy & delete) Thanks so far. Client is Postbox & Thunderbird on Windows. (I realise Postbox is based on Thunderbird) Greg. On 4 August 2014 22:03, Reindl Harald wrote: > > > Am 04.08.2014 um 14:00 schrieb Timo Sirainen: > > On 04 Aug 2014, at 10:44, Greg Sullivan > wrote: > > > >> I would like to use a shared IMAP account, with multiple users > accessing it > >> simultaneously. The users would take ownership of messages by first > >> attempting to MOVE the messages from the Inbox, into their private IMAP > >> folder, still within the same account. Now, since there will be multiple > >> users competing for the same messages, I naturally want only ONE of the > >> simultaneous moves to be successful at a time. > >> > >> So far, this isn't working. If I do the move from two clients, > >> simultaneously, the messages can go to *both *destination folders - > >> duplicates can result. > >> > >> Is it possible to configure Dovecot and/or an IMAP client to behave the > way > >> I want it to? If the answer to this is YES, then I'll offer my config > >> details. If the answer is NO, the next question is - do any email > systems > >> at all behave the way I want? (I tried a hosted Exchange/OWA service - > it > >> has the same problem) > > > > Dovecot doesn't even attempt to do atomic MOVEs. I don't think any > server will. If you can change the client code, you could use CONDSTORE > instead, which does give atomic STOREs. > > Internet Message Access Protocol (IMAP) - MOVE Extension > http://tools.ietf.org/html/rfc6851 > > well, both, client and server would need to support it > rely on that is unlikely for many years > > From jreid at vnet.net Mon Aug 4 12:27:46 2014 From: jreid at vnet.net (Joe Reid) Date: Mon, 04 Aug 2014 07:27:46 -0500 Subject: permission denied In-Reply-To: References: <53DEB875.6080801@vnet.net> Message-ID: <53DF7C42.5020408@vnet.net> jreid owns everything under ~jreid/mail: $ ls -la total 104 drwx------ 20 jreid staff 23 Aug 4 07:22 ./ drwxr-xr-x 12 jreid staff 24 Aug 4 07:21 ../ drwxrwx--- 4 jreid staff 7 Jul 29 14:47 .imap/ -rw------- 1 jreid staff 94 Aug 3 12:24 .subscriptions -rw------- 1 jreid staff 108 Aug 3 22:16 .subscriptions.lock [...] drwxr-x--- 3 jreid staff 6 Aug 4 07:19 2012/ drwxr-x--- 3 jreid staff 6 Jul 29 16:12 2013/ drwxr-x--- 3 jreid staff 6 Aug 3 17:43 2014/ drwxr-x--- 6 jreid staff 7 May 21 2012 Mail_archive/ -rw-r----- 1 jreid staff 536 Aug 4 07:22 Trash $ ls -la 2012/.imap/Received/ total 8 drwxr-x--- 2 jreid staff 3 Aug 4 07:19 ./ drwxr-x--- 3 jreid staff 3 Aug 3 12:25 ../ -rw-r----- 1 jreid staff 40 Aug 4 07:19 dovecot.index.log.newlock On 8/4/2014 2:57 AM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Sun, 3 Aug 2014, Joe Reid wrote: > >> The time stamp on the .subscriptions.lock file gets updated every >> time I try to make a change. To test I edited .subscriptions by hand >> and forced Thunderbird to try to subscribe something and I got access >> to the folder I added, so I presume it is reading the file correctly. >> >> Permissions on~jreid/mail is 0700, the existing .subscriptions file >> is 0640 and .subscriptions.lock file is 0600. all the files are >> owned by me and their group is my primary group (not mail). >> >> $ /opt/csw/bin/doveconf -n >> # 2.2.12: /etc/opt/csw/dovecot/dovecot.conf >> # OS: SunOS 5.11 i86pc >> disable_plaintext_auth = no >> first_valid_uid = 100 >> mail_access_groups = mail >> mail_location = mbox:~/mail:INBOX=/var/share/mail/%u > [cut] >> passdb { >> driver = pam >> } >> userdb { >> driver = passwd >> } > >> $ doveadm log errors >> [...] >> Aug 03 16:47:02 imap(jreid): Error: >> rename(/home/jreid/mail/.subscriptions.lock, >> /home/jreid/mail/.subscriptions) failed: Permission denied >> Aug 03 16:52:49 imap(jreid): Error: >> rename(/home/jreid/mail/2012/.imap/Received/dovecot.index.log.newlock, /home/jreid/mail/2012/.imap/Received/dovecot.index.log) >> failed: Permission denied > > Who owns /home/jreid/mail, .subscriptions.lock file and > /home/jreid/mail/2012/.imap/Received ? > > Please send a ls -la of /home/jreid/mail. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.11 (GNU/Linux) > > iQEVAwUBU9880Xz1H7kL/d9rAQI4kgf7B7nn7mzp8WHqYkD7azYToI7oYxIpoO1e > bi29AjlWNqTkwrxmCyhKbUqwaGan+ZbxN1xRDxsh8q7SoN3p6uF3ot1os2vBrX/1 > UuSsxKEohi7/9S6x/eN9a/gKbRsKfb8wMUwBIW4HlWJgLuaywCqwl1WOk78RpddG > zUPsH3W63X/8lx84n6ZBqwddCYp/R4IYsQdyQMpMCjpj0DzTqniWhtd2i3ZiyaaF > dOhaWgcjTZ1cjr4iJnZniylHkuS4NsFZejSpM1An41aq4b0b51FHSGG8R3ze+sop > LVTctxCK7U+/sO296Lv5Lu49ndAlx0H7KViD9RKFWzaPtgQ+vDADZw== > =k6IU > -----END PGP SIGNATURE----- -- joe From tss at iki.fi Mon Aug 4 12:44:43 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Aug 2014 14:44:43 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> Message-ID: <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> On 04 Aug 2014, at 14:12, Greg Sullivan wrote: > Yes, both client and server support IMAP MOVE, and both also support > CONDSTORE. > > I have tried both with and without CONDSTORE enabled in the client, with > the same result. With CONDSTORE I was thinking you could do it something like: 1 FETCH 1 (FLAGS MODSEQ) * 1 FLAGS () MODSEQ 12345 2 STORE (UNCHANGEDSINCE 12345) 1 +FLAGS $AtomicMove 3 MOVE 1 elsewhere If another client attempts the same, either 1 will return $AtomicMove in flags -> abort or 2 will fail with NO. But you should still handle failures if the client/connection dies between 2 and 3 or 3 fails for some reason. But, of course if you can't change the client code to do this then it doesn't help. > I am very confident IMAP MOVE is actually being invoked, because > intra-account moves occur extremely rapidly. (much faster than > inter-account moves, which of course is a copy & delete) Inter-account physically copies the data (FETCH + APPEND + EXPUNGE). Alternative to MOVE is COPY + EXPUNGE, which is just as fast as MOVE. Dovecot actually implements MOVE by internally doing a COPY + EXPUNGE. From tss at iki.fi Mon Aug 4 13:03:02 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 Aug 2014 15:03:02 +0200 Subject: Multiple Passdb's - Order In-Reply-To: <53DBF1C3.9000307@dupond.be> References: <53DBF1C3.9000307@dupond.be> Message-ID: <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> On 01 Aug 2014, at 22:00, Jean-Louis Dupond wrote: > Today I upgraded my dovecot to a newer version (2.2.9 debian > wheezy-backports) > At that moment I also reconfigured the whole thing a bit nicer. > > Now I stumbled on the following issue. .. > The only thing I changed here was the order the passdb backends. > > It works fine now, but it not clear to me why the order is so important. > Could somebody explain this? Or is this some bug? I can't reproduce it. Also did your log contain also the errors/warnings? It appeared that it was failing with some internal error, which you didn't show in the log. Also does it work if you disable auth cache? Full doveconf -n output may have been helpful also. From jean-louis at dupond.be Mon Aug 4 13:11:24 2014 From: jean-louis at dupond.be (Jean-Louis Dupond) Date: Mon, 04 Aug 2014 15:11:24 +0200 Subject: Multiple Passdb's - Order In-Reply-To: <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> References: <53DBF1C3.9000307@dupond.be> <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> Message-ID: <6fb75bf2cf5a93842d38610ab3bcbf1d@dupond.be> Hi, Thanks for your response. The error/warning log was indeed not included. I looked it up now, and dovecot only printed the following error: Aug 1 21:31:55 xcx dovecot: auth: Error: ldap(info at xxx.com,xxxxx,): ldap_search(base=uid=info at xxx.com,ou=mail,dc=xxx,dc=com filter=(&(objectClass=posixAccount)(uid=info at dupie.be))) failed: No such object This shouldn't really be a problem, because the user does indeed not exist in ldap, but only in the passwd-file. The problem also exists with auth cache disabled. And my doveconf (the passdb order is the working order here): # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 auth_cache_size = 2 M mail_location = maildir:~/Maildir mail_plugins = virtual managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } mailbox virtual/Flagged { special_use = \Flagged } prefix = } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { sieve = /var/vmail/%d/%n/.dovecot.sieve sieve_dir = /var/vmail/%d/%n/sieve } postmaster_address = postmaster at xxx.com protocols = " imap lmtp sieve pop3" service auth { unix_listener /var/spool/postfix-mailing/private/auth { mode = 0666 } unix_listener /var/spool/postfix/private/auth { mode = 0666 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_ca = On 01 Aug 2014, at 22:00, Jean-Louis Dupond > wrote: > >> Today I upgraded my dovecot to a newer version (2.2.9 debian >> wheezy-backports) >> At that moment I also reconfigured the whole thing a bit nicer. >> >> Now I stumbled on the following issue. > .. >> The only thing I changed here was the order the passdb backends. >> >> It works fine now, but it not clear to me why the order is so >> important. >> Could somebody explain this? Or is this some bug? > > I can't reproduce it. Also did your log contain also the > errors/warnings? It appeared that it was failing with some internal > error, which you didn't show in the log. Also does it work if you > disable auth cache? Full doveconf -n output may have been helpful > also. From uothrawn at yahoo.com Mon Aug 4 13:17:17 2014 From: uothrawn at yahoo.com (G H) Date: Mon, 4 Aug 2014 06:17:17 -0700 Subject: Solr indexing appends domain to users Message-ID: <1407158237.29293.YahooMailNeo@web161403.mail.bf1.yahoo.com> My users are set up as "admin", "john", etc -- as opposed to john at example.com. When I run test command "/usr/libexec/dovecot/dovecot-lda -d admin < /tmp/mymail.txt", the mail is successfully imported with Solr, Tika (yay). However, results are never found when I search within IMAP. Why? Dovecot is appending the domain name to the user and admin fields of the Solr record ("admin at example.com" instead of just "admin") during index. But during searching, Dovecot is only searching for the user directly, from the Solr logs: ? q=body:"TESTCONTENT"&fq=%2Bbox:7dff0b0d4f84df53f25300003d823887+%2Buser:"admin" If I rerun the query via the Solr web gui with user:"admin at example.com", the message is found. How do I disable appending the domain name to the solr index names? Thanks, G H From christoph.lechleitner at iteg.at Mon Aug 4 23:47:30 2014 From: christoph.lechleitner at iteg.at (Christoph Lechleitner) Date: Tue, 05 Aug 2014 01:47:30 +0200 Subject: dovecot-lda 2.2.13 crashing when sieve_before used Message-ID: <53E01B92.5030003@iteg.at> Hi together! We use dovecot under Debian wheezy amd64, using the repository ... deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.2 main Yesterday's routine upgrade to version 2:2.2.13-1~auto+130 (or some other upgrade that came along from Debian?) introduced a problem with mail delivery, which I was able to track down to a crash of dovecot-lda. Whenever I do something like ... /usr/lib/dovecot/dovecot-lda -d christoph.lechleitner at iteg.at , storage_r=0x7fffe2dbec00) at lda-sieve-plugin.c:973 #7 0x00007f30d5e05464 in mail_deliver (ctx=ctx at entry=0x7fffe2dbec70, storage_r=storage_r at entry=0x7fffe2dbec00) at mail-deliver.c:400 #8 0x0000000000402a92 in main (argc=3, argv=0x2402390) at main.c:437 The problem vanishes once I remove the sieve_before script, in our case /etc/dovecot/sieve/discard-viruses.sieve The sieve script does not contain anything fancy, and it works perfectly when used as sieve_default. It seems the mere presence of a sieve_before script triggers the problem. Some environment info: As of dovecot -n, I guess only the sieve related parts are relevant, if at all: plugin { sieve = /var/vmail/%d/%n/sieve sieve_before = /etc/dovecot/sieve/discard-viruses.sieve sieve_default = /etc/dovecot/sieve/default.sieve sieve_dir = /var/vmail/%d/%n/sieve-scripts } I also tried version 2:2.2.13-1~auto+138 from the dovecot-2.2.patched repository that I stumbled over while browsing the repository server. No difference regarding our problem. Our filesystem is ext4 disguised as simfs by the OpenVZ virtualization kernel. I'm pretty sure the filesystem does not matter with our problem. I'd be happy to re-test with testing releases, but I'd like to avoid setting up a full-blown test environment where unstable and bleeding edge versions would be ok. Thanks in advance for any help, Regards, Christoph -- Christoph Lechleitner ------------------------------------------------------------------------ ITEG IT-Engineers GmbH | Conradstr. 5, A-6020 Innsbruck FN 365826f | Handelsgericht Innsbruck | Mobiltelefon: +43 699 12090423 Mail: christoph.lechleitner at iteg.at | Web: http://www.iteg.at/ ------------------------------------------------------------------------ From stephan at rename-it.nl Tue Aug 5 02:01:35 2014 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 05 Aug 2014 04:01:35 +0200 Subject: dovecot-lda 2.2.13 crashing when sieve_before used In-Reply-To: <53E01B92.5030003@iteg.at> References: <53E01B92.5030003@iteg.at> Message-ID: <53E03AFF.9010600@rename-it.nl> Hi, On 8/5/2014 1:47 AM, Christoph Lechleitner wrote: > Hi together! > > We use dovecot under Debian wheezy amd64, using the repository ... > deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.2 main > > Yesterday's routine upgrade to version 2:2.2.13-1~auto+130 (or some > other upgrade that came along from Debian?) introduced a problem with > mail delivery, which I was able to track down to a crash of dovecot-lda. > Yes, my bad. I hadn't actually tested sieve_before pointing to a single regular file (so not a directory). Last two commits should fix this. > I also tried version 2:2.2.13-1~auto+138 from the dovecot-2.2.patched > repository that I stumbled over while browsing the repository server. No > difference regarding our problem. Ehmm. Don't do that. :) It contains all kinds of highly experimental code, i.e. it includes the dovecot-2.2-patches and pigeonhole-0.4-patches patch queues. > I'd be happy to re-test with testing releases, but I'd like to avoid > setting up a full-blown test environment where unstable and bleeding > edge versions would be ok. You can re-test when a new package is released by Xi. Should happen within the hour after this message was sent. > Thanks in advance for any help, Thanks for the complete bug report. Regards, Stephan. From dovecot.org at veggiechinese.net Tue Aug 5 06:31:42 2014 From: dovecot.org at veggiechinese.net (Will Yardley) Date: Mon, 4 Aug 2014 23:31:42 -0700 Subject: prefix behavior with Dovecot / Squirrelmail Message-ID: <20140805063141.GA39405@aura.veggiechinese.net> Old: Dovecot 1.1.18 + Squirrelmail 1.4.8 + Imapproxy New: Dovecot 2.2.10 + Squirrelmail 1.4.22 (no Imapproxy) In both, we have: [dovecot config] namespace private { separator = . prefix = Mail. inbox = yes } (The 'Mail' prefix is set this way for compatibility reasons) $imap_server_type = 'dovecot'; $default_folder_prefix = 'Mail.'; $trash_folder = 'Mail.Trash'; $sent_folder = 'Mail.Sent'; $draft_folder = 'Mail.Drafts'; $show_contain_subfolders_option = true; $default_sub_of_inbox = false; (I know some of these are redundant with $imap_server_type, but that's how it's setup now). This works as we expect with the old setup - folders are set in, e.g., $maildir/.Sent, and we only see one set of each. With the newer one, it auto-creates folders as Mail.{Trash,Sent,Drafts}, which translates into Mail.Mail.{Trash,Sent,Drafts}. If I set $foo_folder to just 'Foo' (e.g., $trash_folder = 'Trash';), the auto-creation works as expected, however, now I have two sets of folders (one indented) in the folder list, and the set linking to Mail.{Trash,Sent,Drafts} (the indented ones), don't work. What's the easiest way to get the new setup working in a way that will cause the least change (preferably no change) to end-users. TIA w From felixrubiodalmau at gmail.com Tue Aug 5 08:04:22 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Tue, 05 Aug 2014 10:04:22 +0200 Subject: Set an archive folder for every user Message-ID: <2556456.mhAvvuxn9H@jarvis> Hi everybody, I have a running postfix+dovecot installation, running flawlessly. The machine this setup is running onto has 2 mirrored SSD disks (in which dovecot stores the mails) and 2 mirrored regular HD. I'd like to keep the fresh emails in the SSD, and move them to an Archive folder after some days/weeks. Is there any way in dovecot to set up a folder for every user, that points to an external disk rather than the default one? Thank you, Felix -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From dovecot.org at veggiechinese.net Tue Aug 5 08:24:44 2014 From: dovecot.org at veggiechinese.net (Will Yardley) Date: Tue, 5 Aug 2014 01:24:44 -0700 Subject: prefix behavior with Dovecot / Squirrelmail In-Reply-To: <20140805063141.GA39405@aura.veggiechinese.net> References: <20140805063141.GA39405@aura.veggiechinese.net> Message-ID: <20140805082444.GB39405@aura.veggiechinese.net> On Mon, Aug 04, 2014 at 11:31:42PM -0700, Will Yardley wrote: > namespace private { > separator = . > prefix = Mail. > inbox = yes > } ps - Will simply making the Mail. namespace "hidden" fix the problem, while keeping things backwards compatible for clients who have the prefix set to Mail already? w From bind at enas.net Tue Aug 5 08:25:23 2014 From: bind at enas.net (Urban Loesch) Date: Tue, 05 Aug 2014 10:25:23 +0200 Subject: Set an archive folder for every user In-Reply-To: <2556456.mhAvvuxn9H@jarvis> References: <2556456.mhAvvuxn9H@jarvis> Message-ID: <53E094F3.1050601@enas.net> Hi, you can try the ALT Storage. http://wiki2.dovecot.org/MailboxFormat/dbox Scroll down to "Alternate storage". Regards, Urban Am 05.08.2014 10:04, schrieb Felix Rubio Dalmau: > Hi everybody, > > I have a running postfix+dovecot installation, running flawlessly. The machine this setup is running onto has 2 mirrored SSD disks (in which dovecot stores the mails) and 2 mirrored regular HD. I'd like to keep the fresh emails in the SSD, and move them to an Archive folder after some days/weeks. Is there any way in dovecot to set up a folder for every user, that points to an external disk rather than the default one? > > Thank you, > Felix > From felixrubiodalmau at gmail.com Tue Aug 5 08:26:17 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Tue, 05 Aug 2014 10:26:17 +0200 Subject: Set an archive folder for every user In-Reply-To: <53E094F3.1050601@enas.net> References: <2556456.mhAvvuxn9H@jarvis> <53E094F3.1050601@enas.net> Message-ID: <3095314.XDO60FiOvh@jarvis> Ok! Thank you! Felix On Tuesday 05 August 2014 10:25:23 Urban Loesch wrote: > Hi, > > you can try the ALT Storage. > > http://wiki2.dovecot.org/MailboxFormat/dbox > > Scroll down to "Alternate storage". > > Regards, > Urban > > Am 05.08.2014 10:04, schrieb Felix Rubio Dalmau: > > Hi everybody, > > > > I have a running postfix+dovecot installation, running flawlessly. The machine this setup is running onto has 2 mirrored SSD disks (in which dovecot stores the mails) and 2 mirrored regular HD. I'd like to keep the fresh emails in the SSD, and move them to an Archive folder after some days/weeks. Is there any way in dovecot to set up a folder for every user, that points to an external disk rather than the default one? > > > > Thank you, > > Felix > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From christoph.lechleitner at iteg.at Tue Aug 5 09:08:05 2014 From: christoph.lechleitner at iteg.at (Christoph Lechleitner) Date: Tue, 05 Aug 2014 11:08:05 +0200 Subject: dovecot-lda 2.2.13 crashing when sieve_before used In-Reply-To: <53E03AFF.9010600@rename-it.nl> References: <53E01B92.5030003@iteg.at> <53E03AFF.9010600@rename-it.nl> Message-ID: <53E09EF5.5070707@iteg.at> Am 2014-08-05 04:01, schrieb Stephan Bosch: > >> Yesterday's routine upgrade to version 2:2.2.13-1~auto+130 (or some >> other upgrade that came along from Debian?) introduced a problem with >> mail delivery, which I was able to track down to a crash of dovecot-lda. > > Yes, my bad. I hadn't actually tested sieve_before pointing to a single > regular file (so not a directory). Who doesn't hate writing testcases ;-) > Last two commits should fix this. > [...] > You can re-test when a new package is released by Xi. Should happen > within the hour after this message was sent. Just got 2:2.2.13-1~auto+131 from the Debian repos, and the problem is fixed. Thanks for the ultra-fast fix! > Thanks for the complete bug report. Always. My main job is software development, I know how frustrating incomplete bug reports can be. Regards, Christoph From felixrubiodalmau at gmail.com Tue Aug 5 09:45:03 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Tue, 05 Aug 2014 11:45:03 +0200 Subject: Migrating from maildir to mdbox Message-ID: <8635832.tsE0d8qolL@jarvis> Hi everybody, I have a virtual user setup that is using maildir, and I'd like to migrate it to mdbox. Following directions from http://wiki2.dovecot.org/Migration/MailFormat and keeping in mind that my maildir is in maildir:/mnt/vmail/%d/%u, after having updated 10-mail.conf I have tried to execute: dsync -u "user at domain.org" mirror maildir:/mnt/vmail/domain.org/user But it replies this answer, that I do not understand: dsync(user at domain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) What am I doing wrong? Thank you, Felix -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From skdovecot at smail.inf.fh-brs.de Tue Aug 5 10:07:16 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 5 Aug 2014 12:07:16 +0200 (CEST) Subject: Migrating from maildir to mdbox In-Reply-To: <8635832.tsE0d8qolL@jarvis> References: <8635832.tsE0d8qolL@jarvis> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: > But it replies this answer, that I do not understand: > > dsync(user at domain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) > http://wiki2.dovecot.org/Namespaces "You should use the same hierarchy separator for all namespaces. All list=yes namespaces must use the same separator, but if you find it necessary (e.g. for backwards compatibility namespaces) you may use different separators for list=no namespaces." - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+Cs1Hz1H7kL/d9rAQKrUwf/VK45CoNwZ15jnUPTvHD3QlHRzJQGu98k uhfAom3gUGJeUXzZ52/yaWrW4NExCM7GCs+vIO0fABib3lSTiGV6ocDlWiyPgLQk wK6gV8GYpOo6ODM03taw1NN/lw+Z+UiN4sMVEw971Zhv6V5yEzlWx+D77n2U12BA KE5Xu85xnNxEjLBNuELsZCWJo1qOHQZyUCoP2yk3gSSzheUhrfOQghwJDWWfl3ua g2M7NWWde24ap/rnYhC/WNfFdSg2F110Hq1Y5f8BmDeVoIK/ZXdPumfUICXk+vHR pG4rHrbHPU0702kvTdbrFlvkuMURF4LM7XYjhsHDU3k5uhxrehn40w== =7/n9 -----END PGP SIGNATURE----- From felixrubiodalmau at gmail.com Tue Aug 5 10:14:03 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Tue, 05 Aug 2014 12:14:03 +0200 Subject: Migrating from maildir to mdbox In-Reply-To: <53e0ac99.e1dcc20a.5778.3e2aSMTPIN_ADDED_BROKEN@mx.google.com> References: <8635832.tsE0d8qolL@jarvis> <53e0ac99.e1dcc20a.5778.3e2aSMTPIN_ADDED_BROKEN@mx.google.com> Message-ID: <10064191.0q9U0cL4EU@jarvis> Hi Steffen, but I have not set-up any namespace separator, so I understand that all namespaces use the default one, right? :-S Do you think I should better use the next command? dsync -u "user at domain.org" mirror maildir:/mnt/vmail/domain.org/user:LAYOUT=fs Felix On Tuesday 05 August 2014 12:07:16 Steffen Kaiser wrote: > On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: > > > But it replies this answer, that I do not understand: > > > > dsync(user at domain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) > > > > http://wiki2.dovecot.org/Namespaces > > "You should use the same hierarchy separator for all namespaces. All > list=yes namespaces must use the same separator, but if you find it > necessary (e.g. for backwards compatibility namespaces) you may use > different separators for list=no namespaces." > > -- > Steffen Kaiser > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From greg.sullivan at sullivang.net Tue Aug 5 11:19:46 2014 From: greg.sullivan at sullivang.net (Greg Sullivan) Date: Tue, 5 Aug 2014 21:19:46 +1000 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> Message-ID: Thanks Timo, and no, I can't (easily) change the code of the client. I must say I am extremely disappointed that intra-account moves are not atomic. As far as I can tell, IMAP was designed to allow shared access, so in my opinion this operation should be atomic. Heaven FORBID that I should ask for entire conversation moves to be atomic as well. (which is really what I want) Looks like a bloatware - sorry - helpdesk system - is what I will need to use. Greg. On 4 August 2014 22:44, Timo Sirainen wrote: > On 04 Aug 2014, at 14:12, Greg Sullivan > wrote: > > > Yes, both client and server support IMAP MOVE, and both also support > > CONDSTORE. > > > > I have tried both with and without CONDSTORE enabled in the client, with > > the same result. > > With CONDSTORE I was thinking you could do it something like: > > 1 FETCH 1 (FLAGS MODSEQ) > * 1 FLAGS () MODSEQ 12345 > 2 STORE (UNCHANGEDSINCE 12345) 1 +FLAGS $AtomicMove > 3 MOVE 1 elsewhere > > If another client attempts the same, either 1 will return $AtomicMove in > flags -> abort or 2 will fail with NO. But you should still handle failures > if the client/connection dies between 2 and 3 or 3 fails for some reason. > > But, of course if you can't change the client code to do this then it > doesn't help. > > > I am very confident IMAP MOVE is actually being invoked, because > > intra-account moves occur extremely rapidly. (much faster than > > inter-account moves, which of course is a copy & delete) > > Inter-account physically copies the data (FETCH + APPEND + EXPUNGE). > Alternative to MOVE is COPY + EXPUNGE, which is just as fast as MOVE. > Dovecot actually implements MOVE by internally doing a COPY + EXPUNGE. > > From crohmann at netcologne.de Tue Aug 5 11:45:31 2014 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 05 Aug 2014 13:45:31 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53CF7FC5.2040301@netcologne.de> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> Message-ID: <53E0C3DB.6010108@netcologne.de> Hey Timo and all, may I PING this subject once again to maybe get Timo's opinion. On 23.07.2014 11:26, Christian Rohmann wrote: > Bounced / rejected messages for something that will be usually be > resolved very quickly and the messages can then be delivered after all > is just not very nice for users. The admin made a mistake and the users > have to deal with the problems is just not my approach. > > But in the end I don't even want to argue that rejecting the messages > might not be a valid behavior for some. That's why I suggested to make > this configurable, just like the quota behavior. I'd really like to hear Timo's view on having lmtp do a (configurable) DEFER when the disk is full which is, most likely, a "temporary" error. Storage is cheap nowadays, it just has to be configured to the right volume and at the right time. Would be nice of Dovecot would not punish users (by bouncing their mail) for mistakes the admin made ;-) Thanks, Christian From vitaliy.tokarev at gmail.com Tue Aug 5 12:02:19 2014 From: vitaliy.tokarev at gmail.com (Vitaliy T) Date: Tue, 5 Aug 2014 16:02:19 +0400 Subject: [Patch] mail-log plugin supports remote, local ip addresses Message-ID: -- With Best Regards, Vitaliy T. From vitaliy.tokarev at gmail.com Tue Aug 5 12:07:36 2014 From: vitaliy.tokarev at gmail.com (Vitaliy T) Date: Tue, 5 Aug 2014 16:07:36 +0400 Subject: [Patch] mail_log plugin supports remote, local ip addresses Message-ID: Hello, In attached file you will find patch for mail_log plugin to output local, remote ip addresses. It may be usefull for somebody. To enable support for dovecot 2.2.x versions change the line 'include "net.h"' instead of 'include "network.h"'. Thank you. P.S. Sorry, for previous empty message. -- With Best Regards, Vitaliy T. -------------- next part -------------- --- src/plugins/mail-log/mail-log-plugin.c.orig 2014-08-05 13:12:21.000000000 +0400 +++ src/plugins/mail-log/mail-log-plugin.c 2014-08-05 13:15:30.000000000 +0400 @@ -13,6 +13,8 @@ #include +#include "network.h" + #define MAILBOX_NAME_LOG_LEN 64 #define HEADER_LOG_LEN 80 @@ -27,7 +29,9 @@ MAIL_LOG_FIELD_VSIZE = 0x10, MAIL_LOG_FIELD_FLAGS = 0x20, MAIL_LOG_FIELD_FROM = 0x40, - MAIL_LOG_FIELD_SUBJECT = 0x80 + MAIL_LOG_FIELD_SUBJECT = 0x80, + MAIL_LOG_FIELD_REMOTE_IP = 0x100, + MAIL_LOG_FIELD_LOCAL_IP = 0x200 }; #define MAIL_LOG_DEFAULT_FIELDS \ (MAIL_LOG_FIELD_UID | MAIL_LOG_FIELD_BOX | \ @@ -58,6 +62,8 @@ "flags", "from", "subject", + "rip", + "lip", NULL }; @@ -265,6 +271,14 @@ mail_get_keywords(mail)); str_append(text, "), "); } + if ((muser->fields & MAIL_LOG_FIELD_REMOTE_IP) != 0) { + str_printfa(text, "rip=%s", net_ip2addr(mail->box->storage->user->remote_ip)); + str_append(text, ", "); + } + if ((muser->fields & MAIL_LOG_FIELD_LOCAL_IP) != 0) { + str_printfa(text, "lip=%s", net_ip2addr(mail->box->storage->user->local_ip)); + str_append(text, ", "); + } str_truncate(text, str_len(text)-2); msg->event = event; From alanm at sics.se Tue Aug 5 12:49:22 2014 From: alanm at sics.se (Alan McGinlay) Date: Tue, 05 Aug 2014 14:49:22 +0200 Subject: replicator only starts for users who log in at least once Message-ID: Hi! I recently setup a second with replication to the other. Both machines run ubuntu 12.04 with dovecot 2.2.13 (6bb26098a45c). All users mail accounts are currently synced manually to the first (mb1) using a dsync script, they are synchronised from a cyrus imap server. Users are still logging into the cyrus server. It appears that the only accounts that are replicated are ones who have logged into either of the dovecot servers at least once, is this how it is supposed to work? I was hoping to get full replication of all mail accounts before starting to move users into the new system. Thanks, Alan From akbwiz+dovecot at gmail.com Mon Aug 4 17:16:23 2014 From: akbwiz+dovecot at gmail.com (Akash) Date: Mon, 4 Aug 2014 22:46:23 +0530 Subject: Dovecot v2.2 FTS is not indexing "text/html" emails... Message-ID: <13af324581f2f903952d676bf3f8b3aa.squirrel@> Hi, I am not sure its intended or a fault in the newest Dovecot versions. I have been using Dovecot v1.2.15 on Debian squeeze and FTS is working as expected. When I search a quoted string "very good", I get 107 results including plain and HTML emails which have this phrase. In order to compare the benefits of lucene over squat, I recently started testing dovecot v2.2.13 on Debian Sid with the same maildir content. But now the same search "very good" yielded just 8 results. I thought it could be some problem with lucene so I tried switching to squat and got 107 results again. After this I deleted the old squat search index files created by v1.2.15 and re-indexed the mail-box by using doveadm index command. Now the same squat search is giving 8 results just as lucene. So I have realized that its not a problem with just lucene but FTS in newer dovecot isn't indexing those emails which have Content-type as text/html. Thus if a mail is like this: Content-Type: text/html He is very good. It isn't shown in search by the squat indexes created using dovecot v2.2.13. I have done further testing on some sample emails which confirmed this behavior. Why is this so? -Regards, Akash From tss at iki.fi Tue Aug 5 16:11:36 2014 From: tss at iki.fi (Timo Sirainen) Date: Tue, 5 Aug 2014 18:11:36 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> Message-ID: <014F7BEC-A62B-47FA-ADB1-2B958BE9AD4D@iki.fi> Note that MOVE isn't atomic even between moving within one user's folders. The MOVE RFC itself also doesn't say anything about it ever having to be atomic. Although if by atomicity you mean that you simply want to make sure that the same source mail can't be MOVEd twice, that would be doable with some work I think. Even for full conversations (without partial failures). On 05 Aug 2014, at 13:19, Greg Sullivan wrote: > Thanks Timo, and no, I can't (easily) change the code of the client. > > I must say I am extremely disappointed that intra-account moves are not > atomic. As far as I can tell, IMAP was designed to allow shared access, so > in my opinion this operation should be atomic. Heaven FORBID that I should > ask for entire conversation moves to be atomic as well. (which is really > what I want) > > Looks like a bloatware - sorry - helpdesk system - is what I will need to > use. > > Greg. > > > On 4 August 2014 22:44, Timo Sirainen wrote: > >> On 04 Aug 2014, at 14:12, Greg Sullivan >> wrote: >> >>> Yes, both client and server support IMAP MOVE, and both also support >>> CONDSTORE. >>> >>> I have tried both with and without CONDSTORE enabled in the client, with >>> the same result. >> >> With CONDSTORE I was thinking you could do it something like: >> >> 1 FETCH 1 (FLAGS MODSEQ) >> * 1 FLAGS () MODSEQ 12345 >> 2 STORE (UNCHANGEDSINCE 12345) 1 +FLAGS $AtomicMove >> 3 MOVE 1 elsewhere >> >> If another client attempts the same, either 1 will return $AtomicMove in >> flags -> abort or 2 will fail with NO. But you should still handle failures >> if the client/connection dies between 2 and 3 or 3 fails for some reason. >> >> But, of course if you can't change the client code to do this then it >> doesn't help. >> >>> I am very confident IMAP MOVE is actually being invoked, because >>> intra-account moves occur extremely rapidly. (much faster than >>> inter-account moves, which of course is a copy & delete) >> >> Inter-account physically copies the data (FETCH + APPEND + EXPUNGE). >> Alternative to MOVE is COPY + EXPUNGE, which is just as fast as MOVE. >> Dovecot actually implements MOVE by internally doing a COPY + EXPUNGE. >> >> From greg.sullivan at sullivang.net Tue Aug 5 16:29:22 2014 From: greg.sullivan at sullivang.net (Greg Sullivan) Date: Wed, 6 Aug 2014 02:29:22 +1000 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <014F7BEC-A62B-47FA-ADB1-2B958BE9AD4D@iki.fi> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <014F7BEC-A62B-47FA-ADB1-2B958BE9AD4D@iki.fi> Message-ID: That's promising that it should be doable. (yes, all I want is for the move to only occur once - duplicate messages is not a "move" at all) I'll forward your suggestions to the Thunderbird & Postbox teams. In the meantime I'll continue to evaluate helpdesk systems and "collaborative inbox" products. Greg. On 06/08/2014 2:11 am, "Timo Sirainen" wrote: > Note that MOVE isn't atomic even between moving within one user's folders. > The MOVE RFC itself also doesn't say anything about it ever having to be > atomic. Although if by atomicity you mean that you simply want to make sure > that the same source mail can't be MOVEd twice, that would be doable with > some work I think. Even for full conversations (without partial failures). > > On 05 Aug 2014, at 13:19, Greg Sullivan > wrote: > > > Thanks Timo, and no, I can't (easily) change the code of the client. > > > > I must say I am extremely disappointed that intra-account moves are not > > atomic. As far as I can tell, IMAP was designed to allow shared access, > so > > in my opinion this operation should be atomic. Heaven FORBID that I > should > > ask for entire conversation moves to be atomic as well. (which is really > > what I want) > > > > Looks like a bloatware - sorry - helpdesk system - is what I will need to > > use. > > > > Greg. > > > > > > On 4 August 2014 22:44, Timo Sirainen wrote: > > > >> On 04 Aug 2014, at 14:12, Greg Sullivan > >> wrote: > >> > >>> Yes, both client and server support IMAP MOVE, and both also support > >>> CONDSTORE. > >>> > >>> I have tried both with and without CONDSTORE enabled in the client, > with > >>> the same result. > >> > >> With CONDSTORE I was thinking you could do it something like: > >> > >> 1 FETCH 1 (FLAGS MODSEQ) > >> * 1 FLAGS () MODSEQ 12345 > >> 2 STORE (UNCHANGEDSINCE 12345) 1 +FLAGS $AtomicMove > >> 3 MOVE 1 elsewhere > >> > >> If another client attempts the same, either 1 will return $AtomicMove in > >> flags -> abort or 2 will fail with NO. But you should still handle > failures > >> if the client/connection dies between 2 and 3 or 3 fails for some > reason. > >> > >> But, of course if you can't change the client code to do this then it > >> doesn't help. > >> > >>> I am very confident IMAP MOVE is actually being invoked, because > >>> intra-account moves occur extremely rapidly. (much faster than > >>> inter-account moves, which of course is a copy & delete) > >> > >> Inter-account physically copies the data (FETCH + APPEND + EXPUNGE). > >> Alternative to MOVE is COPY + EXPUNGE, which is just as fast as MOVE. > >> Dovecot actually implements MOVE by internally doing a COPY + EXPUNGE. > >> > >> > > From Jochen.Bern at LINworks.de Tue Aug 5 18:00:37 2014 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Tue, 05 Aug 2014 20:00:37 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> Message-ID: <53E11BC5.5060707@LINworks.de> On -10.01.-28163 20:59, Greg Sullivan wrote: > I must say I am extremely disappointed that intra-account moves are not > atomic. As far as I can tell, IMAP was designed to allow shared access, so > in my opinion this operation should be atomic. Heaven FORBID that I should > ask for entire conversation moves to be atomic as well. (which is really > what I want) How would it be of any use to the passive client that the *operation* is atomic when (as far as I can see, which admittedly mightn't be much) there is no way defined in the IMAP protocol to atomically *notify* it of said change? IMAP IDLE, for example, may inform it that one message disappeared from mailbox X and one popped up in mailbox Y - not that these two are actually the same message, still have the same set of flags set, etc.. That's for the client to find out by specific requests - which already breaks the atomicity and allows for a race condition between clients. Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From felixrubiodalmau at gmail.com Tue Aug 5 18:36:37 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Tue, 05 Aug 2014 20:36:37 +0200 Subject: Migrating from maildir to mdbox Message-ID: <4472774.NR6asGpeta@jarvis> Hi all, I have (I think) migrated my dovecot mail format from maildir to mdbox. I have changed in 10-mail.conf the format from maildir to mdbox, and I have executed this command for all my users: dsync -u user at domain.org mirror maildir:/mnt/vmail/domain/user The only output of the command has been, for some users, this: dsync(user at domain.org): Warning: quota: Namespace '' is not Maildir, skipping for Maildir++ quota Is this important? How can I check that I am effectively using the new format? Thank you, Felix -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From skdovecot at smail.inf.fh-brs.de Wed Aug 6 06:02:38 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 Aug 2014 08:02:38 +0200 (CEST) Subject: Migrating from maildir to mdbox In-Reply-To: <10064191.0q9U0cL4EU@jarvis> References: <8635832.tsE0d8qolL@jarvis> <53e0ac99.e1dcc20a.5778.3e2aSMTPIN_ADDED_BROKEN@mx.google.com> <10064191.0q9U0cL4EU@jarvis> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: > but I have not set-up any namespace separator, so I understand that all namespaces use the default one, right? :-S Post your doveconf -n, then. > > Do you think I should better use the next command? > > dsync -u "user at domain.org" mirror maildir:/mnt/vmail/domain.org/user:LAYOUT=fs > > Felix > > On Tuesday 05 August 2014 12:07:16 Steffen Kaiser wrote: >> On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: >> >>> But it replies this answer, that I do not understand: >>> >>> dsync(user at domain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) >>> >> >> http://wiki2.dovecot.org/Namespaces >> >> "You should use the same hierarchy separator for all namespaces. All >> list=yes namespaces must use the same separator, but if you find it >> necessary (e.g. for backwards compatibility namespaces) you may use >> different separators for list=no namespaces." >> >> -- >> Steffen Kaiser >> > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+HE/3z1H7kL/d9rAQL0wQgAkKfCkg3mNX3mwbXGa7+KEonO7Etbr38i 4Hw+0HlJlpmYd7E5HNrfzg/ENwt/M8Ljq6JqjXzoxm53UpuhVFwf7su+mc92peYE 9o54n8CzyY/1syX71bIjCPmUDl6oV79gJLj9HBvhI2CB1F2C6N8ttDTQOFTmbpad Wv3O36pGb2A+YnP1IqMaxU/RASwlpj2y4GGNuCwJXy3z+aKsdxORfBiHwbeDQvk9 Ukon1RG+n4TLk+xwwrjLNGympD883tEJxibzmYexp+FmC3Jz70zwslTMNwFF8v1n 46HBHtesWZSnxo8rmB3sd1y04M5ekq6zjWis02Sn4HH5QmGPK2keaw== =cCX0 -----END PGP SIGNATURE----- From felixrubiodalmau at gmail.com Wed Aug 6 06:21:40 2014 From: felixrubiodalmau at gmail.com (Felix Rubio Dalmau) Date: Wed, 06 Aug 2014 08:21:40 +0200 Subject: Migrating from maildir to mdbox In-Reply-To: <53e1c4bb.420dc30a.4eb1.4ae5SMTPIN_ADDED_BROKEN@mx.google.com> References: <8635832.tsE0d8qolL@jarvis> <10064191.0q9U0cL4EU@jarvis> <53e1c4bb.420dc30a.4eb1.4ae5SMTPIN_ADDED_BROKEN@mx.google.com> Message-ID: <1412576.QfrB4Q6YJF@jarvis> Hi Steffan, Solved! Thank you very much for you help. I think there was a configuration error and when I used the "separator" keyword was not working properly. Yesterday night, I tried again, and everything worked :-) Regards! Felix On Wednesday 06 August 2014 08:02:38 you wrote: > On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: > > > but I have not set-up any namespace separator, so I understand that all namespaces use the default one, right? :-S > > Post your doveconf -n, then. > > > > > Do you think I should better use the next command? > > > > dsync -u "user at domain.org" mirror maildir:/mnt/vmail/domain.org/user:LAYOUT=fs > > > > Felix > > > > On Tuesday 05 August 2014 12:07:16 Steffen Kaiser wrote: > >> On Tue, 5 Aug 2014, Felix Rubio Dalmau wrote: > >> > >>> But it replies this answer, that I do not understand: > >>> > >>> dsync(user at domain.org): Error: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) > >>> > >> > >> http://wiki2.dovecot.org/Namespaces > >> > >> "You should use the same hierarchy separator for all namespaces. All > >> list=yes namespaces must use the same separator, but if you find it > >> necessary (e.g. for backwards compatibility namespaces) you may use > >> different separators for list=no namespaces." > >> > >> -- > >> Steffen Kaiser > >> > > > > -- > Steffen Kaiser > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: This is a digitally signed message part. URL: From sca at andreasschulze.de Wed Aug 6 10:01:09 2014 From: sca at andreasschulze.de (A. Schulze) Date: Wed, 06 Aug 2014 12:01:09 +0200 Subject: maildir: could lda/lmtp log filenames? Message-ID: <20140806120109.Horde.LGcC7qB0_m8T0PGPME5b9g7@horde.andreasschulze.de> hello, on a mailstorage server receiving messages per lda/lmtp I like to pimp my logfile. when starting to search for a specific message I have a queueid. I like to combine that id with the filename the message has after delivery to a maildir. I know, that will not work with other formats then maildir. Aug 6 11:37:30 mailserver postfix/lmtp[17884]: 3hSnmj208Zz259d: to= relay=$lmtp-server status=sent (250 2.0.0 $dovecot-id) Aug 6 11:37:29 $lmtp-server dovecot: lmtp(1440, rcpt at example.org): $dovecot-id: sieve: msgid=<..>: stored mail into mailbox 'INBOX/folder' and exactly there I would like to see "stored mail into mailbox 'INBOX/folder/new/FILENAME'" yes, I know, I /should/ user doveadm and avoid access messages via filesystem but in that case it would be much faster to access the information I need. Any ideas? Andreas From greg.sullivan at sullivang.net Wed Aug 6 10:15:23 2014 From: greg.sullivan at sullivang.net (Greg Sullivan) Date: Wed, 6 Aug 2014 20:15:23 +1000 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <53E11BC5.5060707@LINworks.de> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <53E11BC5.5060707@LINworks.de> Message-ID: Jochen, I don't have any in-depth knowledge of the IMAP protocol. I'm just saying that given that IMAP is designed for concurrent access from multiple clients, I would have expected it to behave much better when more than one person attempts to move a message, that's all. I was gobsmacked when I discovered that duplicates could easily occur! Quote from the IMAP wikipedia page: Internet Message Access Protocol (IMAP) is a protocol for e-mail retrieval and storage developed by Mark Crispin in 1986 at Stanford University as an alternative to POP. IMAP unlike POP, specifically allows multiple clients simultaneously connected to the same mailbox, and through flags stored on the server, different clients accessing the same mailbox at the same or different times can detect state changes made by other clients. Regards, Greg. On 6 August 2014 04:00, Jochen Bern wrote: > On -10.01.-28163 20:59, Greg Sullivan wrote: > > I must say I am extremely disappointed that intra-account moves are not > > atomic. As far as I can tell, IMAP was designed to allow shared access, > so > > in my opinion this operation should be atomic. Heaven FORBID that I > should > > ask for entire conversation moves to be atomic as well. (which is really > > what I want) > > How would it be of any use to the passive client that the *operation* is > atomic when (as far as I can see, which admittedly mightn't be much) > there is no way defined in the IMAP protocol to atomically *notify* it > of said change? > > IMAP IDLE, for example, may inform it that one message disappeared from > mailbox X and one popped up in mailbox Y - not that these two are > actually the same message, still have the same set of flags set, etc.. > That's for the client to find out by specific requests - which already > breaks the atomicity and allows for a race condition between clients. > > Regards, > J. Bern > -- > *NEU* - NEC IT-Infrastruktur-Produkte im : > Server--Storage--Virtualisierung--Management SW--Passion for Performance > Jochen Bern, Systemingenieur --- LINworks GmbH > Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt > PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 > Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 > Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel > From h.reindl at thelounge.net Wed Aug 6 10:21:29 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 06 Aug 2014 12:21:29 +0200 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <53E11BC5.5060707@LINworks.de> Message-ID: <53E201A9.60804@thelounge.net> Am 06.08.2014 um 12:15 schrieb Greg Sullivan: > Jochen, > I don't have any in-depth knowledge of the IMAP protocol. I'm just saying > that given that IMAP is designed for concurrent access from multiple > clients, I would have expected it to behave much better when more than one > person attempts to move a message, that's all. I was gobsmacked when I > discovered that duplicates could easily occur! > > Quote from the IMAP wikipedia page: > Internet Message Access Protocol (IMAP) is a protocol for e-mail retrieval > and storage developed by Mark Crispin in 1986 at Stanford University as an > alternative to POP. IMAP unlike POP, specifically allows multiple clients > simultaneously connected to the same mailbox, and through flags stored on > the server, different clients accessing the same mailbox at the same or > different times can detect state changes made by other clients. that's the theory "can detect" -> when and how in case of concurrency things are not that easy because you have anyways network latency as part of the game > On 6 August 2014 04:00, Jochen Bern wrote: > >> On -10.01.-28163 20:59, Greg Sullivan wrote: >>> I must say I am extremely disappointed that intra-account moves are not >>> atomic. As far as I can tell, IMAP was designed to allow shared access, >> so >>> in my opinion this operation should be atomic. Heaven FORBID that I >> should >>> ask for entire conversation moves to be atomic as well. (which is really >>> what I want) >> >> How would it be of any use to the passive client that the *operation* is >> atomic when (as far as I can see, which admittedly mightn't be much) >> there is no way defined in the IMAP protocol to atomically *notify* it >> of said change? >> >> IMAP IDLE, for example, may inform it that one message disappeared from >> mailbox X and one popped up in mailbox Y - not that these two are >> actually the same message, still have the same set of flags set, etc.. >> That's for the client to find out by specific requests - which already >> breaks the atomicity and allows for a race condition between clients. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 246 bytes Desc: OpenPGP digital signature URL: From lists at wildgooses.com Wed Aug 6 13:16:33 2014 From: lists at wildgooses.com (Ed W) Date: Wed, 06 Aug 2014 14:16:33 +0100 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <014F7BEC-A62B-47FA-ADB1-2B958BE9AD4D@iki.fi> Message-ID: <53E22AB1.1060101@wildgooses.com> On 05/08/2014 17:29, Greg Sullivan wrote: > That's promising that it should be doable. (yes, all I want is for the move > to only occur once - duplicate messages is not a "move" at all) I'll > forward your suggestions to the Thunderbird & Postbox teams. > > In the meantime I'll continue to evaluate helpdesk systems and > "collaborative inbox" products. > > Greg. > I agree with the goal though. I have extremely simple needs for a helpdesk/CMS type system, and some plugins to Thunderbird would be quite satisfactory for my needs Really I need: - Enhanced addressbook, possibly reading via vcard from my main business system (bring in customer details and links to their orders on main system) - Ability to force breaking and rejoin of specific message threading (because customers find an old invoice and hit reply to it to send us a support request + other customers who send you 15 emails (without hitting reply to trigger threading) to describe a single problem). Note I believe this requires rewriting the message, so it couldn't be atomic with current IMAP? - Enhanced use of flags to mark whether thread needs further input or is closed Nice to have would be: - Logging these state changes to somewhere else so that you can get statistics (can probably be done by polling the state of the IMAP server though?) - Atomic "locking" of threads so that we don't get two people answer something. Could be handled through use of flags perhaps? Thunderbird is helpful in that in theory all one needs to do is write the above in javascript and drop appropriate display buttons on the email inbox, so even if some external lock manager is needed to arbitrate access, then this is no great problem. In practice I lack the time to work on this, but I'm vaguely interested to find out if there is a way to hire "plugin developers" for Thunderbird? Good luck Ed W From fischerb at fischer-ing.de Wed Aug 6 13:23:24 2014 From: fischerb at fischer-ing.de (Fischer, Bernhard) Date: Wed, 06 Aug 2014 15:23:24 +0200 Subject: Panic on alway_bcc mail with attachment while using sdbox Message-ID: I do always get a panic on service lmtp, while receiving a mail by postfix through always_bcc, containing an attachment. If the bcc recipient would be supplied by a small delay, I would say that the panic wouldn't occur, but I don't know how to verify this. dovecot is configured with 'mail_attachment_dir' and the public namespace location is sdbox dovecot log: Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Panic: file fs-api.c: line 615 (fs_copy): assertion failed: (src->fs == dest->fs) Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x66676) [0x443676] -> /usr/lib/dovecot/libdovecot.so.0(+0x66700) [0x443700] -> /usr/lib/dovecot/libdovecot.so.0(+0x18c23) [0x3f5c23] -> /usr/lib/dovecot/libdovecot.so.0(+0x2278b) [0x3ff78b] -> /usr/lib/dovecot/libdovecot-storage.so.0(sdbox_copy+0x4ad) [0x23c6ad] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x62) [0x27da72] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_using_mail+0x2b) [0x27db4b] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1a1) [0x5dd3c1] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x143) [0x5dd7c3] -> dovecot/lmtp() [0x804d090] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x43) [0x455973] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xdb) [0x456c4b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0x4558b8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2e) [0x3fb0ee] -> dovecot/lmtp(main+0x1b3) [0x804b463] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x5f9d26] -> dovecot/lmtp() [0x804b1e1] Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Fatal: master: service(lmtp): child 20367 killed with signal 6 (core dumps disabled) From andre at rodier.me Wed Aug 6 15:50:21 2014 From: andre at rodier.me (=?UTF-8?Q?Andr=C3=A9_Rodier?=) Date: Wed, 06 Aug 2014 16:50:21 +0100 Subject: Important emails Message-ID: <48be1bed681852796de3e88fd8a1a03b@webmail2.rodier.me> Hi, Sorry if I am not posting on the right mailing list, but I need email and especially IMAP experts opinion. I have developed a small perl script for Amavis that tries to recognise business emails from bulk & mailing lists, in order to promote 'important' emails, ala google. This is a feature I found very useful in corporate environments, where a lot of users are receiving advertisements, but not consider it as spam. (For info, the code is here: https://github.com/arodier/emclass) My script add an email header (X-Email-Type), and dovecot uses a virtual folder called 'Important' to present them. I know there is IMAP labels, and they are server side stored. I have successfully played with Thunderbird, but IMHO, it's not very reliable. The name of the label is the responsibility of the email client. As a result, if one client decided that $Label1 is 'Important', and $Label2 is 'Business', other clients should be configured exactly the same. I wonder if there is some norm - or convention - to attach labels to email headers, or if such a header exists, for instance a header called e.g. 'X-Tags', 'X-Keywords' or 'X-Labels'. So, once stored, an email could contains - for instance - a header like this: X-Tags: Travel,Advertisement,France. Then, it would be easy to have server side search for labels. Thanks for your feedback. Kind regards, Andr?. From gowen-dovecot at swynwyr.com Wed Aug 6 15:56:10 2014 From: gowen-dovecot at swynwyr.com (gowen (dovecot)) Date: Wed, 06 Aug 2014 11:56:10 -0400 Subject: Sieve filter extracting wrong header value on certain mail messages Message-ID: # 2.2.12: /etc/dovecot/dovecot.conf # OS: Linux 3.15.4-x86_64-linode45 x86_64 openSUSE 12.3 (x86_64) Short version: 1) I filter out spam by using a dovecot sieve filter to act based on a header inserted by bogofilter 2) In some cases, sieve fails to correctly file based on this header 3) Testing with sieve-test indicates that it's testing the wrong header 4) Suggestions? Am I doing something wrong? Is there a problem with these headers that bends test-sieve out of shape? 5) Test files from below are also available via http://www.swynwyr.com/sieve-test/ Any help is appreciated!!! Long version with full details: Here is the simple sieve filter I'm using to test, and two different variants of an email which triggers this problem. I basically deleted things one by one from an original email (1) to get minimal testcases that triggered (2) or did not trigger (3) the problem. The filter: === begin test.sieve === require ["fileinto", "body", "regex"]; if header :contains "X-Bogosity" "Spam," { fileinto "SPAM.SPAM"; stop; } elsif header :contains "X-Bogosity" "Unsure," { fileinto "SPAM.UNSURE"; stop; } keep; stop; === end test.sieve === Here is an example email that will cause this filter to not function as expected: === begin message.2 === Return-Path: Subject: =?utf-8?B?QUhTIEtpdGNoZW4gTWFrZW92ZXIgR2l2ZWF3YXkg?= Subject: ?HS Kit?hen ?ak??ov??r Giv??away Subject: =?ISO-8859-1?Q?AHS Kitchen Makeover Giveaway =A0=A0=A0?= Subject: Subject: AHS Kitchen Makeover Giveaway X-Bogosity: Spam, tests=bogofilter, spamicity=1.000000, version=1.2.3 Home Warranty Summer Giveaway === end message.2 === If I run test-sieve on this, we can clearly see in the trace log that when it's supposed to test for header "X-Bogosity" it in fact looks at one of the multiple "Subject" headers. Note that the result is storing in INBOX rather than SPAM.SPAM, and that the "matching value" for "X-Bogosity headers" is actually the contents of the final "Subject" header in the test on line 3. In the second test, on line 6, it uses a different and correct value for the same header! But too late to help as the first test was the one it should have had a match on. === begin session === $ sieve-test -t trace2.log -Tlevel=matching test.sieve message.2 Performed actions: * store message in folder: INBOX Implicit keep: (none) sieve-test(gowen): Info: final result: success $ cat trace2.log ## Started executing script 'test' 3: header test 3: starting `:contains' match with `i;ascii-casemap' comparator: 3: extracting `X-Bogosity' headers from message 3: matching value `AHS Kitchen Makeover Giveaway' 3: with key `Spam,' => 0 3: finishing match with result: not matched 3: jump if result is false 3: jumping to line 6 6: header test 6: starting `:contains' match with `i;ascii-casemap' comparator: 6: extracting `X-Bogosity' headers from message 6: matching value `Spam, tests=bogofilter, spamicity=1.000000, version=1.2.3' 6: with key `Unsure,' => 0 6: finishing match with result: not matched 6: jump if result is false 6: jumping to line 11 11: keep action; store message in default mailbox 12: stop command; end all script execution ## Finished executing script 'test' $ === end session === Now lets remove the "Return-Path" from that message. (Of the original 26 headers, all but the 7 shown in message.2 could be removed and still have this message break the filter). This alters thing enough that sieve starts working as expected (in fact, removing any one of the 6 non-X-Bogosity headers will cause this to start working). === begin message.3 === Subject: =?utf-8?B?QUhTIEtpdGNoZW4gTWFrZW92ZXIgR2l2ZWF3YXkg?= Subject: ?HS Kit?hen ?ak??ov??r Giv??away Subject: =?ISO-8859-1?Q?AHS Kitchen Makeover Giveaway =A0=A0=A0?= Subject: Subject: AHS Kitchen Makeover Giveaway X-Bogosity: Spam, tests=bogofilter, spamicity=1.000000, version=1.2.3 Home Warranty Summer Giveaway === end message.3 === Now when we run test-sieve it works as expected, filing into SPAM.SPAM and showing the correct matching value for line 3: === begin session === $ sieve-test -t trace3.log -Tlevel=matching test.sieve message.3 Performed actions: * store message in folder: SPAM.SPAM Implicit keep: (none) sieve-test(gowen): Info: final result: success $ cat trace3.log ## Started executing script 'test' 3: header test 3: starting `:contains' match with `i;ascii-casemap' comparator: 3: extracting `X-Bogosity' headers from message 3: matching value `Spam, tests=bogofilter, spamicity=1.000000, version=1.2.3' 3: with key `Spam,' => 1 3: finishing match with result: matched 3: jump if result is false 3: not jumping 4: fileinto action 4: store message in mailbox `SPAM.SPAM' 5: stop command; end all script execution ## Finished executing script 'test' $ === end session === From zsturgeon at pccc.com Wed Aug 6 16:59:55 2014 From: zsturgeon at pccc.com (Zachary Sturgeon) Date: Wed, 06 Aug 2014 12:59:55 -0400 Subject: Multiple userdb on specific ports Message-ID: <53E25F0B.3040107@pccc.com> Hello everyone, Is it possible to, from one instance, bind a single userdb to a specific listen port in Dovecot? For example, requiring passwd or PAM authentication on IMAP port 143, but using an SQL passdb only on port 8143? Regards, ZWS -- *Zachary Sturgeon* Programmer Peregrine Computer Consultants Corporation 3927 Old Lee Highway, Suite 102-C Fairfax, VA 22030-2422 http://www.pccc.com/ 703-359-9700 x44 / 800-823-8402 (Toll-Free) 703-359-8451 (fax) zsturgeon at pccc.com From slusarz at curecanti.org Wed Aug 6 18:33:33 2014 From: slusarz at curecanti.org (Michael M Slusarz) Date: Wed, 06 Aug 2014 12:33:33 -0600 Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <53E11BC5.5060707@LINworks.de> Message-ID: <20140806123333.Horde.qA1sLPQruK08VkkbXDKJcg1@bigworm.curecanti.org> Quoting Greg Sullivan : > I was gobsmacked when I discovered that duplicates could easily occur! > > Quote from the IMAP wikipedia page: > Internet Message Access Protocol (IMAP) is a protocol for e-mail retrieval > and storage developed by Mark Crispin in 1986 at Stanford University as an > alternative to POP. IMAP unlike POP, specifically allows multiple clients > simultaneously connected to the same mailbox, and through flags stored on > the server, different clients accessing the same mailbox at the same or > different times can detect state changes made by other clients. Disagree. I'm not "gobsmacked" due to the fact that IMAP was designed to ensure that no unintentional DESTRUCTIVE actions take place. I'd be "gobsmacked" if it was the opposite - preferring duplicate prevention over message loss. What's worse: 1. Concurrent users where one accidentally deletes (i.e. EXPUNGE) a message due to inconsistent mailbox state between the two sessions. 2. Concurrent users where one accidentally creates a duplicate message during a move operation by the other user. 1 is worse. (Although 1 becomes mostly irrelevant once UIDs are used and UIDPLUS is available.) But 2, in real-world practice, simply doesn't happen enough to make it a critical issue. michael From HFlor at gmx.de Thu Aug 7 07:13:28 2014 From: HFlor at gmx.de (Hardy Flor) Date: Thu, 07 Aug 2014 09:13:28 +0200 Subject: call postlogin script with doveadm exec Message-ID: <53E32718.6080501@gmx.de> Hello, how is a call to the imap-postloginscriptes possible when starting with doveadm exec imap? In the imap-login-post script will set the ACL_GROUP. Hardy From skdovecot at smail.inf.fh-brs.de Thu Aug 7 07:25:09 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 Aug 2014 09:25:09 +0200 (CEST) Subject: Is atomic MOVING of messages between IMAP folders possible? In-Reply-To: <20140806123333.Horde.qA1sLPQruK08VkkbXDKJcg1@bigworm.curecanti.org> References: <31378934-D212-488F-A8B2-E8110D00E7D9@iki.fi> <53DF767D.4060008@thelounge.net> <8610572E-2643-4B78-A3C2-955E0179DF07@iki.fi> <53E11BC5.5060707@LINworks.de> <20140806123333.Horde.qA1sLPQruK08VkkbXDKJcg1@bigworm.curecanti.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Quoting Greg Sullivan : > >> I was gobsmacked when I discovered that duplicates could easily occur! "move" is not atomic in a filesystem either. Under circumstances it has to be implemented by "cp && rm". Unless you protect the move's source by locking - as Timo suggested with the message at Tue, 5 Aug 2014 21:19:46 - the same duplication can occur. In your reply you've wrote: "Heaven FORBID that I should ask for entire conversation moves to be atomic as well. (which is really what I want)" Actually you want to use database-like transactions for IMAP commands or sessions. Actually, what should happen _during_ the MOVE, if another client queries for the source message or launches commands on it? Now, for a short period of time the copy takes the message is available at the source location only, then it disappears if the move was successful. IMHO, you could write a plugin that monitors or wraps "MOVE" commands and denies second simultaneous move attempts. In a way, that plugin would do the locking Timo suggested as a server-side implementation. How many simultaneous MOVE do you expect? Maybe you could lock the source mailbox wholly during a move. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+Mp13z1H7kL/d9rAQLiAAgAqryldRY7xyLxhULj+nmABbspsXsauip5 KNGYx1lPmTh1JqhvGZ/1mxn80ImidXa2U3poa8ltHPthjHW1NvqsDCMFXTzpUC36 l6zIP7VOEwMSLAq0/KmsbvJIh13h9IiZRzG3zZDO7AaDyxru171qctuEwOKO3EWp W4kicYne3fAj/FXEI4+Dnm7a6cJabIRL2DTzkU33JC0S+/seq/8ZXj9XO99ocwJV 86xjdtaD90H3akIA1G7/Q1Ielw5tEPGMD7Ye4RF+ra1DYOsj9RYXisVqNhYLDh84 qUsBPXm7Ap4rgp07iar10ex3UYQbGEq0VR9bxUxna4IMS67EtBwLpQ== =7Rkz -----END PGP SIGNATURE----- From fischerb at fischer-ing.de Thu Aug 7 07:24:52 2014 From: fischerb at fischer-ing.de (Fischer, Bernhard) Date: Thu, 07 Aug 2014 09:24:52 +0200 Subject: Panic on alway_bcc mail with attachment while using sdbox In-Reply-To: References: Message-ID: <7499c3c15ec075568625bd9575cd8d23@go.fischer.ing> I missed to mention, it's dovecot version 2.2.10 under CentOS 6.5. I'm using the original package from http://dl.atrpms.net/all/dovecot-2.2.10-1_14.el6.i686.rpm Anyone aware of this panic in combination of attachment storage and sdbox? -bernhard At Wednesday, 06-08-2014 on 15:23 Fischer, Bernhard wrote: > I do always get a panic on service lmtp, while receiving a mail by postfix through always_bcc, containing an attachment. If the bcc recipient would be supplied by a small delay, I would say that the panic wouldn't occur, but I don't know how to verify this. > dovecot is configured with 'mail_attachment_dir' and the public namespace location is sdbox > > dovecot log: > Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Panic: file fs-api.c: line 615 (fs_copy): assertion failed: (src->fs == dest->fs) > Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x66676) [0x443676] -> /usr/lib/dovecot/libdovecot.so.0(+0x66700) [0x443700] -> /usr/lib/dovecot/libdovecot.so.0(+0x18c23) [0x3f5c23] -> /usr/lib/dovecot/libdovecot.so.0(+0x2278b) [0x3ff78b] -> /usr/lib/dovecot/libdovecot-storage.so.0(sdbox_copy+0x4ad) [0x23c6ad] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_copy+0x62) [0x27da72] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_save_using_mail+0x2b) [0x27db4b] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver_save+0x1a1) [0x5dd3c1] -> /usr/lib/dovecot/libdovecot-lda.so.0(mail_deliver+0x143) [0x5dd7c3] -> dovecot/lmtp() [0x804d090] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x43) [0x455973] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xdb) [0x456c4b] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x48) [0x4558b8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x2e) [0x3fb0ee] -> dovecot/lmtp(main+0x1b3) [0x804b463] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x5f9d26] -> dovecot/lmtp() [0x804b1e1] > Aug 6 14:18:02 dovecot dovecot: lmtp(20367, vscanner at fischer-ing.de): Fatal: master: service(lmtp): child 20367 killed with signal 6 (core dumps disabled) > From jerry at seibercom.net Thu Aug 7 10:24:31 2014 From: jerry at seibercom.net (Jerry) Date: Thu, 7 Aug 2014 06:24:31 -0400 Subject: Pigeonhole version 0.4.3 on FreeBSD-10 Message-ID: <20140807062431.2f4c6365@scorpio> Thu, 7 Aug 2014 06:22:01 -0400 I am running Pigeonhole version 0.4.3 on a FreeBSD 10 machine. How do I actually trace which rule in my sieve file is causing a specific action on a specific email. Other then eliminating each rule one by one and rerunning the script to isolate it, how can I discover what rule is causing the problem? Thanks! -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: not available URL: From stephan at rename-it.nl Thu Aug 7 10:39:56 2014 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 07 Aug 2014 12:39:56 +0200 Subject: Pigeonhole version 0.4.3 on FreeBSD-10 In-Reply-To: <20140807062431.2f4c6365@scorpio> References: <20140807062431.2f4c6365@scorpio> Message-ID: <53E3577C.2060807@rename-it.nl> On 8/7/2014 12:24 PM, Jerry wrote: > Thu, 7 Aug 2014 06:22:01 -0400 > > I am running Pigeonhole version 0.4.3 on a FreeBSD 10 machine. How do I > actually trace which rule in my sieve file is causing a specific action on a > specific email. Other then eliminating each rule one by one and rerunning the > script to isolate it, how can I discover what rule is causing the problem? You can use the sieve-test tool for that. man sieve-test Regards, Stephan. From jerry at seibercom.net Thu Aug 7 11:07:09 2014 From: jerry at seibercom.net (Jerry) Date: Thu, 7 Aug 2014 07:07:09 -0400 Subject: Pigeonhole version 0.4.3 on FreeBSD-10 In-Reply-To: <53E3577C.2060807@rename-it.nl> References: <20140807062431.2f4c6365@scorpio> <53E3577C.2060807@rename-it.nl> Message-ID: <20140807070709.4a904d1b@scorpio> On Thu, 07 Aug 2014 12:39:56 +0200, Stephan Bosch stated: >On 8/7/2014 12:24 PM, Jerry wrote: >> Thu, 7 Aug 2014 06:22:01 -0400 >> >> I am running Pigeonhole version 0.4.3 on a FreeBSD 10 machine. How do I >> actually trace which rule in my sieve file is causing a specific action on >> a specific email. Other then eliminating each rule one by one and >> rerunning the script to isolate it, how can I discover what rule is >> causing the problem? > >You can use the sieve-test tool for that. > >man sieve-test I have, but I cannot tell which specific rule is catching it. EXAMPLE: Performed actions: * store message in folder: SPAM.Drugs + create mailbox if it does not exist Implicit keep: (none) I have numerous rules to catch SPAM. One of them is catching an email that is NOT Drug related spam. I do not know which one it is. Only the final result is printed out. I could comment out them out one by one, but there should be an easier method. -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: not available URL: From stephan at rename-it.nl Thu Aug 7 11:24:11 2014 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 07 Aug 2014 13:24:11 +0200 Subject: Pigeonhole version 0.4.3 on FreeBSD-10 In-Reply-To: <20140807070709.4a904d1b@scorpio> References: <20140807062431.2f4c6365@scorpio> <53E3577C.2060807@rename-it.nl> <20140807070709.4a904d1b@scorpio> Message-ID: <53E361DB.5090800@rename-it.nl> On 8/7/2014 1:07 PM, Jerry wrote: > On Thu, 07 Aug 2014 12:39:56 +0200, Stephan Bosch stated: > >> On 8/7/2014 12:24 PM, Jerry wrote: >>> Thu, 7 Aug 2014 06:22:01 -0400 >>> >>> I am running Pigeonhole version 0.4.3 on a FreeBSD 10 machine. How do I >>> actually trace which rule in my sieve file is causing a specific action on >>> a specific email. Other then eliminating each rule one by one and >>> rerunning the script to isolate it, how can I discover what rule is >>> causing the problem? >> You can use the sieve-test tool for that. >> >> man sieve-test > I have, but I cannot tell which specific rule is catching it. > > EXAMPLE: > > Performed actions: > > * store message in folder: SPAM.Drugs > + create mailbox if it does not exist > > Implicit keep: > > (none) > > I have numerous rules to catch SPAM. One of them is catching an email that > is NOT Drug related spam. I do not know which one it is. Only the final > result is printed out. I could comment out them out one by one, but there > should be an easier method. sieve-test also has trace support using -t and -T options. Regards, Stephan. From ben at indietorrent.org Thu Aug 7 15:04:22 2014 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 07 Aug 2014 11:04:22 -0400 Subject: Exit status code 134; what is it, in the context of Dovecot Antispam plug-in? In-Reply-To: <53D7BBD5.7080106@indietorrent.org> References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> Message-ID: <53E39576.3000705@indietorrent.org> On 7/29/2014 11:20 AM, Ben Johnson wrote: > On 7/29/2014 3:13 AM, Steffen Kaiser wrote: >> On Mon, 28 Jul 2014, Ben Johnson wrote: >> >>> I have some debugging output in my pipe script; the output looks >> >> How does your script looks like? >> > > http://pastebin.com/nh8SwQtw > >>> Copying message contents to temporary file for debugging >>> purposes; file is: /tmp/sendmail-msg-7662.txt Checking if the >>> command-line input argument string (--spam) contains the string >>> "ham" or "spam" Mode is "SPAM" Calling (as user vmail) >>> '/usr/lib/dovecot/deliver -d "sa-training at example.com" -m >>> "Training.SPAM" -p "/tmp/sendmail-msg-7662.txt"' Exit status was >>> 134 >> >> Check out your local /usr/include/sysexits.h, if the exit code is >> defined there. It's not in mine. >> > > Exit code 134 is not defined in /usr/include/sysexits.h on my system. > >>> Yet, I'm able to copy the above command and execute it manually, >>> via the command-line, and it works (and by "works", I mean to >>> say that the behavior is correct and exactly as expected; I >>> receive the "Spam" email at the designated mailbox). Here's how >>> I'm calling it when it works perfectly well (as "root"): >> >>> # su -c '/usr/lib/dovecot/deliver -d "sa-training at example.com" -m >>> "Training.HAM" -p "/tmp/sendmail-msg-7460.txt"' vmail >> >>> Any idea what status 134 might be or how to work around it? It >>> looks to be some kind of "temporary failure exception", but that >>> is less than informative in this context. >> >>> # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-32-generic >>> x86_64 Ubuntu 14.04.1 LTS plugin { antispam_backend = pipe >>> antispam_debug_target = syslog antispam_pipe_program = /bin/bash >>> antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh >>> antispam_pipe_program_notspam_arg = --ham >>> antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = >>> /tmp antispam_spam_pattern_ignorecase = SPAM;JUNK >>> antispam_trash_pattern_ignorecase = trash;Deleted * >>> antispam_verbose_debug = 1 } >> >> >> -- Steffen Kaiser > > Is it possible that this is some kind of apparmor restriction? I ask > because apparmor is indeed installed on this machine. > > If you examine the script source (cited above), you will see that I've > had to use "the hammer that is strace" to debug issues with Dovecot + > Antispam before... maybe it's worth trying in this case. > > Happy to hear any further suggestions. > > Thanks again, > > -Ben > Still struggling with this. strace doesn't reveal anything useful, either. In short, dovecot deliver is returning with exit code 134 when I try to execute the following command in the context of my dovecot-antispam pipe script: /usr/lib/dovecot/deliver -d "sa-training at example.com" -m "Training.HAM" -p "/tmp/sendmail-msg-7460.txt"' vmail Yet, if I execute the same exact command after su-ing to the vmail user, it works: # su vmail $ whoami vmail $ /usr/lib/dovecot/deliver -d "sa-training at example.com" -m "Training.HAM" -p "/tmp/sendmail-msg-7460.txt" I have ensured that the pipe script is, in fact, being executed as the vmail user, by inserting the following into my debug output: CURRENT_USER=$(whoami) echo "$CURRENT_USER" This outputs "vmail". I have this working with exactly the same setup (near as I can tell) on a machine with Dovevot 2.0.19 (via Ubuntu 12.04 LTS). This problem machine is running 2.2.9 (via Ubuntu 14.04 LTS). My "doveconf -n" output is at the bottom of my original post. I would love to figure this out; it will be the capstone on an otherwise perfect build. :) Thanks for any ideas! -Ben From skdovecot at smail.inf.fh-brs.de Thu Aug 7 15:14:41 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 Aug 2014 17:14:41 +0200 (CEST) Subject: Exit status code 134; what is it, in the context of Dovecot Antispam plug-in? In-Reply-To: <53E39576.3000705@indietorrent.org> References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 7 Aug 2014, Ben Johnson wrote: > On 7/29/2014 11:20 AM, Ben Johnson wrote: >> On 7/29/2014 3:13 AM, Steffen Kaiser wrote: >>> On Mon, 28 Jul 2014, Ben Johnson wrote: >>> >>>> I have some debugging output in my pipe script; the output looks >>> >>> How does your script looks like? >>> >> >> http://pastebin.com/nh8SwQtw >> >>>> Copying message contents to temporary file for debugging >>>> purposes; file is: /tmp/sendmail-msg-7662.txt Checking if the >>>> command-line input argument string (--spam) contains the string >>>> "ham" or "spam" Mode is "SPAM" Calling (as user vmail) >>>> '/usr/lib/dovecot/deliver -d "sa-training at example.com" -m >>>> "Training.SPAM" -p "/tmp/sendmail-msg-7662.txt"' Exit status was >>>> 134 >>> >>> Check out your local /usr/include/sysexits.h, if the exit code is >>> defined there. It's not in mine. >>> >> >> Exit code 134 is not defined in /usr/include/sysexits.h on my system. >> >>>> Yet, I'm able to copy the above command and execute it manually, >>>> via the command-line, and it works (and by "works", I mean to >>>> say that the behavior is correct and exactly as expected; I >>>> receive the "Spam" email at the designated mailbox). Here's how >>>> I'm calling it when it works perfectly well (as "root"): >>> >>>> # su -c '/usr/lib/dovecot/deliver -d "sa-training at example.com" -m >>>> "Training.HAM" -p "/tmp/sendmail-msg-7460.txt"' vmail >>> >>>> Any idea what status 134 might be or how to work around it? It >>>> looks to be some kind of "temporary failure exception", but that >>>> is less than informative in this context. >>> >>>> # 2.2.9: /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-32-generic >>>> x86_64 Ubuntu 14.04.1 LTS plugin { antispam_backend = pipe >>>> antispam_debug_target = syslog antispam_pipe_program = /bin/bash >>>> antispam_pipe_program_args = /usr/local/bin/sa-learn-pipe.sh >>>> antispam_pipe_program_notspam_arg = --ham >>>> antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = >>>> /tmp antispam_spam_pattern_ignorecase = SPAM;JUNK >>>> antispam_trash_pattern_ignorecase = trash;Deleted * >>>> antispam_verbose_debug = 1 } >>> >>> >>> -- Steffen Kaiser >> >> Is it possible that this is some kind of apparmor restriction? I ask >> because apparmor is indeed installed on this machine. Well, of course apparmor can interject with any operation. You ought to see that in the apparmor logs. I do not have no experience with it though. >> If you examine the script source (cited above), you will see that I've >> had to use "the hammer that is strace" to debug issues with Dovecot + >> Antispam before... maybe it's worth trying in this case. >> > Still struggling with this. strace doesn't reveal anything useful, either. > > In short, dovecot deliver is returning with exit code 134 when I try to > execute the following command in the context of my dovecot-antispam pipe > script: > > /usr/lib/dovecot/deliver -d "sa-training at example.com" -m > "Training.HAM" -p "/tmp/sendmail-msg-7460.txt"' vmail > > Yet, if I execute the same exact command after su-ing to the vmail user, > it works: > > # su vmail > $ whoami > vmail > $ /usr/lib/dovecot/deliver -d "sa-training at example.com" -m > "Training.HAM" -p "/tmp/sendmail-msg-7460.txt" > > I have ensured that the pipe script is, in fact, being executed as the > vmail user, by inserting the following into my debug output: > > CURRENT_USER=$(whoami) > echo "$CURRENT_USER" > > This outputs "vmail". > > I have this working with exactly the same setup (near as I can tell) on > a machine with Dovevot 2.0.19 (via Ubuntu 12.04 LTS). This problem > machine is running 2.2.9 (via Ubuntu 14.04 LTS). My "doveconf -n" output > is at the bottom of my original post. > > I would love to figure this out; it will be the capstone on an otherwise > perfect build. :) > > Thanks for any ideas! one idea: http://www.tldp.org/LDP/abs/html/exitcodes.html exit code 134 would be in bash's meaning (if this website is correct all) some program died off signal 6. This would be Abort in Linux. prepend your script with exec >> /tmp/trace 2>&1 set -vx that will dump anything visible into /tmp/trace - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+OX4Xz1H7kL/d9rAQKMywgAmXzynP+lVcPhKfrQ+O3gih98+6C50lD5 g1jmKuGuiiPxBruD1Z4M9tCajN0t4bBBXQKUdvyNedms+iIi94sTEmC14DUq//+g M/Fu/0FL2RZxS3NaaYcR5vz3jrHcGDBKewffbWauRHMF0PIy4IOCTeTwSvjAFleb dBI51KhHWDqYw7T4ZGAAgZlp2ympG1PA2NU0YaSy87oa2WGoIii7F8AgvSaze/0j kwZZKfg35C5/zrWyRixSompjUJzUAaKc4TmWxggjejGv+yiJHxiTFgpCwBsci2XA KHfSOzAyezfvXTS1ZdC+yXYuqUAERZj6ArtHKmsu/aSCDg9T9w4ZVw== =6cGs -----END PGP SIGNATURE----- From ben at indietorrent.org Thu Aug 7 15:41:05 2014 From: ben at indietorrent.org (Ben Johnson) Date: Thu, 07 Aug 2014 11:41:05 -0400 Subject: Exit status code 134; what is it, in the context of Dovecot Antispam plug-in? In-Reply-To: References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> Message-ID: <53E39E11.90602@indietorrent.org> On 8/7/2014 11:14 AM, Steffen Kaiser wrote: > one idea: http://www.tldp.org/LDP/abs/html/exitcodes.html > > exit code 134 would be in bash's meaning (if this website is > correct all) some program died off signal 6. This would be Abort in > Linux. > > prepend your script with > > exec >> /tmp/trace 2>&1 set -vx > > that will dump anything visible into /tmp/trace > > - -- Steffen Kaiser Thank you for your continued assistance, Steffen. You seem to be exactly correct with the Abort signal. I prepended the values you suggested to the pipe script and here's the relevant output: + /usr/lib/dovecot/deliver -d sa-training at example.com -m Training.SPAM ^A^H5584 prefix=lda: ^A^F5584 io_add(0x1) called twice fd=7, callback=0x7f23489fb6f0 -> 0x7f23489aa530 ^A^D5584 Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e271) [0x7f23489e9271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) [0x7f23489e934e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f23489a4a9e] -> /usr/lib/dovecot/libdovecot.so.0(ioloop_iolist_add+0x83) [0x7f23489f9533] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) [0x7f23489f9cbb] -> /usr/lib/dovecot/libdovecot.so.0(io_add+0x9b) [0x7f23489f89fb] -> /usr/lib/dovecot/libdovecot.so.0(master_service_io_listeners_add+0x69) [0x7f23489a9e49] -> /usr/lib/dovecot/libdovecot.so.0(master_service_init_finish+0xb0) [0x7f23489a9f90] -> /usr/lib/dovecot/deliver(main+0x1cb) [0x7f234939269b] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f23485e6ec5] -> /usr/lib/dovecot/deliver(+0x31de) [0x7f23493931de] /usr/local/bin/sa-learn-pipe.sh: line 52: 5584 Aborted (core dumped) /usr/lib/dovecot/deliver -d "sa-training at example.com" -m "Training.$mode" + echo 'Exit status was 134' Does this tell us anything more? I don't see any indication that apparmor is at play, as there is no profile that should apply in this context (and there is nothing in the relevant log file): # service apparmor status apparmor module is loaded. 8 profiles are loaded. 8 profiles are in enforce mode. /sbin/dhclient /usr/bin/freshclam /usr/lib/NetworkManager/nm-dhcp-client.action /usr/lib/connman/scripts/dhclient-script /usr/sbin/clamd /usr/sbin/mysqld /usr/sbin/ntpd /usr/sbin/tcpdump 0 profiles are in complain mode. 4 processes have profiles defined. 4 processes are in enforce mode. /usr/bin/freshclam (2015) /usr/sbin/clamd (1897) /usr/sbin/mysqld (1239) /usr/sbin/ntpd (2472) 0 processes are in complain mode. 0 processes are unconfined but have a profile defined. I'll try to reproduce this on an identically-configured server. I wonder if it would be worth changing the version of Dovecot. But I hate to play whack-a-mole if a more systematic approach is to be had. Thanks again, -Ben From ben at electricembers.coop Thu Aug 7 19:21:23 2014 From: ben at electricembers.coop (Benjamin Connelly) Date: Thu, 7 Aug 2014 12:21:23 -0700 (PDT) Subject: invalid/unknown path to storage Message-ID: Am I correct: pigeonhole now wants the symlink to the filter file to use an absolute path instead of the relative paths we had? i.e.) sieve-filterfile@ -> /home/mail/domain/user at domain/roundcube.sieve instead of sieve-filterfile@ -> roundcube.sieve Dovecot logs these: dovecot: managesieve(user at domain.org): Warning: sieve-storage: Active sieve script symlink /home/mail/domain.org/user at domain.org/sieve-filterfile is broken: invalid/unknown path to storage (points to roundcube.sieve). and the debug log: Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve: Pigeonhole version 0.4.3 initializing Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: using active sieve script path: /home/mail/domain.org/user at domain.org/sieve-filterfile Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: using sieve script storage directory: /home/mail/domain.org/user at domain.org/ Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: using permissions from /home/mail/domain.org/user at domain.org/: mode=0700 gid=-1 Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: relative path to sieve storage in active link: I found if I manually delete a symlink and replace it using the full path: sieve-filterfile -> /home/mail/domain.org/user at domain.org/roundcube.sieve then everybody is happy. So I'm looking to make a configuration change (and then change all existing symlinks.) I just wanted to check with the list about the two settings we return as part of the user_query: concat(homedir, maildir, 'sieve-filterfile') as sieve, concat(homedir,maildir) as sieve_dir In testing it seems like changing 'sieve' to return not the whole path but just the link name, actuall causes pigeonhole to properly make the symlink with full path! Does this seem right? FreeBSD (8.4) dovecot-pigeonhole (0.4.3_1) dovecot2 (2.2.13_3) Benjamin From stephan at rename-it.nl Fri Aug 8 00:44:22 2014 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 08 Aug 2014 02:44:22 +0200 Subject: invalid/unknown path to storage In-Reply-To: References: Message-ID: <53E41D66.8030309@rename-it.nl> On 8/7/2014 9:21 PM, Benjamin Connelly wrote: > Am I correct: pigeonhole now wants the symlink to the filter file to > use an absolute path instead of the relative paths we had? > > i.e.) sieve-filterfile@ -> /home/mail/domain/user at domain/roundcube.sieve > instead of sieve-filterfile@ -> roundcube.sieve > > Dovecot logs these: > > dovecot: managesieve(user at domain.org): Warning: sieve-storage: Active > sieve script symlink > /home/mail/domain.org/user at domain.org/sieve-filterfile is broken: > invalid/unknown path to storage (points to roundcube.sieve). > > and the debug log: > > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve: Pigeonhole > version 0.4.3 initializing > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve: include: > sieve_global_dir is not set; it is currently not possible to include > `:global' scripts. > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: > using active sieve script path: > /home/mail/domain.org/user at domain.org/sieve-filterfile > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: > using sieve script storage directory: > /home/mail/domain.org/user at domain.org/ > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: > using permissions from /home/mail/domain.org/user at domain.org/: > mode=0700 gid=-1 > Jul 29 15:37:59 managesieve(user at domain.org): Debug: sieve-storage: > relative path to sieve storage in active link: > > I found if I manually delete a symlink and replace it using the full > path: > > sieve-filterfile -> > /home/mail/domain.org/user at domain.org/roundcube.sieve > > then everybody is happy. So I'm looking to make a configuration change > (and then change all existing symlinks.) I just wanted to check with > the list about the two settings we return as part of the user_query: > > concat(homedir, maildir, 'sieve-filterfile') as sieve, > concat(homedir,maildir) as sieve_dir > > In testing it seems like changing 'sieve' to return not the whole path > but just the link name, actuall causes pigeonhole to properly make the > symlink with full path! Does this seem right? This is probably the path comparison bug that I fixed implicitly while I added generic sieve script storage support. (A few '*' are missing here: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/file/1c6130ff5dd6/src/lib-sievestorage/sieve-storage-script.c#l40). I initially wondered why nobody encountered and reported this bug before I found it. Then I realized that most people would heed the advice not to put the active script symlink inside the script storage directory (http://wiki2.dovecot.org/Pigeonhole/Sieve/Configuration#line-18), so the relative path to the sieve storage in the active symlink would not be empty. Looks like you didn't. Anyway, your approach is a good workaround for now. Regards, Stephan. From lists at wiesinger.com Fri Aug 8 05:34:42 2014 From: lists at wiesinger.com (Gerhard Wiesinger) Date: Fri, 08 Aug 2014 07:34:42 +0200 Subject: Dovecot, Thunderbird and messages in wrong folder Message-ID: <53E46172.4090507@wiesinger.com> Hello, Sometimes it happens that messages (e.g. one or two) appear in wrong folders in Thunderbird (latest version, also past versions, different computers, also Earlybird). When I repair the folder (folder/Settings/Repair Folder) in Thunderbird, the folder is correct then. It happens on my new server with Fedora 20, Dovecot 2.2.13 and Maildir but it also happened on my old server with Fedora 1, Dovecot 2.2.13 and mbox. Has anyone the same issue? Any guess whether it might be a thunderbird or dovecot issue? Anyone has the same issues with thunderbird and a different IMAP server (e.g. cyrus)? Thank you. Ciao, Gerhard Config (only relevant parts here): # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.15.8-200.fc20.x86_64 x86_64 Fedora release 20 (Heisenbug) auth_mechanisms = plain login lda_mailbox_autocreate = yes listen = * mail_gid = vmail mail_home = /home/vmail/%n mail_location = maildir:~/Maildir mail_max_userip_connections = 20 mail_plugins = " stats mailbox_alias zlib" mail_uid = vmail protocols = imap lmtp sieve service imap-login { inet_listener imap { port = 0 } process_min_avail = 2 service_count = 0 } service imap { client_limit = 20 executable = imap postlogin process_limit = 20 } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 2 } service managesieve-login { process_min_avail = 1 } service postlogin { executable = script-login -d rawlog } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl_cert = References: <53AFA9CA.6010103@wiesinger.com> <53C68339.2040404@wiesinger.com> Message-ID: <53E468B7.2060009@wiesinger.com> Hello, Anyone interested, I solved it the following way: imap server: dovecot <=> /bin/saslcheckpassword <=> local unix domain socket for saslauthd <=> socat client ^ = Remote TLS connection v auth server: socat server <=> saslauthd <=> /etc/passwd,/etc/shadow saslcheckpassword is based on checkpassword.sh and enhanced for SASL, modified version attached: https://bitbucket.org/vizovitin/dovecot-conf-examples/src/tip/checkpassword-shell/checkpassword.sh Detailed configs below. imap server: touch /var/log/dovecot-saslcheckpassword.log chown dovecot.dovecot /var/log/dovecot-saslcheckpassword.log chmod 750 /var/log/dovecot-saslcheckpassword.log Advantage is single authentication entity. Works well for me for some time now. If you have some questions feel free to ask. Ciao, Gerhard ================================================================================================================================================================ IMAP server, SASL client: cat /etc/systemd/system/multi-user.target.wants/saslclient.service [Unit] Description=SASL remote client for authentication, provides local unix domain socket After=network.target [Service] # Must be running under permissions for the dovecot user! ExecStart=/usr/bin/socat -4 -ly UNIX-LISTEN:/var/run/saslauthd/mux,fork,user=dovecot,group=dovecot 'OPENSSL:myserver:1234,verify=1,method=TLSv1,cipher=DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXP:!LOW:!MD5,cafile=/etc/socat/mycert.crt' [Install] WantedBy=multi-user.target ================================================================================================================================================================ Auth server: cat /etc/systemd/system/multi-user.target.wants/saslserver.service [Unit] Description=SASL remote server for authentication After=network.target [Service] ExecStart=/usr/bin/socat -4 -ly 'OPENSSL-LISTEN:9999,reuseaddr,fork,verify=0,method=TLSv1,cipher=DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXP:!LOW:!MD5,cert=/etc/socat/mycert.pem,cafile=/etc/socat/mycert.crt' /var/run/saslauthd/mux [Install] WantedBy=multi-user.target ================================================================================================================================================================ Relevant dovecot config: userdb { args = username_format=%n /etc/dovecot/users driver = passwd-file } passdb { args = /bin/saslcheckpassword driver = checkpassword } ================================================================================================================================================================ -- http://www.wiesinger.com/ On 16.07.2014 15:50, Gerhard Wiesinger wrote: > > Any comments? > > Thank you. > > Ciao, > Gerhard > > http://www.wiesinger.com/ -------------- next part -------------- #!/bin/bash # Example Dovecot checkpassword script that may be used as both passdb or userdb. # # Originally written by Nikolay Vizovitin, 2013. # Enhanced by Gerhard Wiesinger, 2014. # Assumes authentication DB is in /etc/dovecot/users, each line has ':' format. # Place this script into /etc/dovecot/checkpassword.sh file and make executable. # Implementation guidelines at http://wiki2.dovecot.org/AuthDatabase/CheckPassword # The first and only argument is path to checkpassword-reply binary. # It should be executed at the end if authentication succeeds. CHECKPASSWORD_REPLY_BINARY="$1" # Messages to stderr will end up in mail log (prefixed with "dovecot: auth: Error:") #LOG=/dev/stderr # Will be e.g. /tmp/systemd-dovecot.service-XfZAUy5/tmp/saslcheckpassword.log #LOG=/tmp/saslcheckpassword.log # touch /var/log/dovecot-saslcheckpassword.log # chown dovecot.dovecot /var/log/dovecot-saslcheckpassword.log # chmod 750 /var/log/dovecot-saslcheckpassword.log LOG=/var/log/dovecot-saslcheckpassword.log #export default_userdb_uid=popuser #export default_userdb_gid=popuser export default_userdb_uid=vmail export default_userdb_gid=vmail export SASL=1 # User and password will be supplied on file descriptor 3. INPUT_FD=3 # Error return codes. ERR_PERMFAIL=1 ERR_NOUSER=3 ERR_TEMPFAIL=111 # Make testing this script easy. To check it just run: # printf '%s\0%s\0' | ./checkpassword.sh test; echo "$?" if [ "$CHECKPASSWORD_REPLY_BINARY" = "test" ]; then CHECKPASSWORD_REPLY_BINARY=/bin/true INPUT_FD=0 fi # Credentials lookup function. Given a user name it should output 'user:password' if such # account exists or nothing if it does not. Return non-zero code in case of error. credentials_lookup() { local db="$1" local user="$2" awk -F ':' -v USER="$user" '($1 == USER) {print}' "$db" 2>>$LOG } # Credentials verification function. Given a user name and password it should output non-empty # string (this implementation outputs 'user:password') in case supplied credentials are valid # or nothing if they are not. Return non-zero code in case of error. credentials_verify() { local db="$1" local user="$2" local pass="$3" awk -F ':' -v USER="$user" -v PASS="$pass" '($1 == USER && $2 == PASS) {print}' "$db" 2>>$LOG } credentials_verify_sasl() { local db="$1" local user="$2" local pass="$3" mail_name="`echo \"$user\" | awk -F '@' '{ print $1 }'`" domain_name="`echo \"$user\" | awk -F '@' '{ print $2 }'`" #export HOME="/var/qmail/mailnames/$domain_name/$mail_name/" export HOME="/home/vmail/${user}/" if [ ! -z "${domain_name}" ]; then # Don't log to stdout (otherwise authenticated=yes)!! /usr/sbin/testsaslauthd -s smtp -r "${domain_name}" -u "${mail_name}" -p "${pass}" > /dev/null 2>&1 else # Don't log to stdout (otherwise authenticated=yes)!! /usr/sbin/testsaslauthd -s smtp -u "${mail_name}" -p "${pass}" > /dev/null 2>&1 fi if [ "$?" = "0" ]; then # Success, echo something echo "USER=\"$user\" PASS=\"**********\"" fi } # Just a simple logging helper. log_result() { # echo "$*; Input: $USER:$PASS Home: $HOME; Reply binary: $CHECKPASSWORD_REPLY_BINARY" >>$LOG echo "`date "+%Y.%m.%d %H:%M:%S"` $*; Input: $USER Home: $HOME; Reply binary: $CHECKPASSWORD_REPLY_BINARY" >>$LOG } # Read input data. It is available from $INPUT_FD as "${USER}\0${PASS}\0". # Password may be empty if not available (i.e. if doing credentials lookup). read -d $'\0' -r -u $INPUT_FD USER read -d $'\0' -r -u $INPUT_FD PASS # Both mailbox and domain directories should be in lowercase on file system. # So let's convert login user name to lowercase and tell Dovecot that 'user' and 'home' # (which overrides 'mail_home' global parameter) values should be updated. # Of course, conversion to lowercase may be done in Dovecot configuration as well. export USER="`echo \"$USER\" | tr 'A-Z' 'a-z'`" mail_name="`echo \"$USER\" | awk -F '@' '{ print $1 }'`" domain_name="`echo \"$USER\" | awk -F '@' '{ print $2 }'`" #export HOME="/var/qmail/mailnames/$domain_name/$mail_name/" export HOME="/home/vmail/${USER}/" # Script will be called under dovecot/dovecot user/group #id >>$LOG # CREDENTIALS_LOOKUP is not set in our use case! # CREDENTIALS_LOOKUP=1 environment is set when doing non-plaintext authentication. if [ "$CREDENTIALS_LOOKUP" = 1 ]; then action=credentials_lookup else if [ ! -z "${SASL}" ]; then action=credentials_verify_sasl else action=credentials_verify fi fi # Perform credentials lookup/verification. lookup_result=`$action "/etc/dovecot/users" "$USER" "$PASS"` || { # If it failed, consider it an internal temporary error. # This usually happens due to permission problems. log_result "internal error (ran as `id`)" exit $ERR_TEMPFAIL } if [ -n "$lookup_result" ]; then # Dovecot calls the script with AUTHORIZED=1 environment set when performing a userdb lookup. # The script must acknowledge this by changing the environment to AUTHORIZED=2, # otherwise the lookup fails. [ "$AUTHORIZED" != 1 ] || export AUTHORIZED=2 # And here's how to return extra fields from userdb/passdb lookup, e.g. 'uid' and 'gid'. # All virtual mail users in Plesk actually run under 'popuser'. # See also: # http://wiki2.dovecot.org/PasswordDatabase/ExtraFields # http://wiki2.dovecot.org/UserDatabase/ExtraFields # http://wiki2.dovecot.org/VirtualUsers export userdb_uid=${default_userdb_uid} export userdb_gid=${default_userdb_gid} export EXTRA="userdb_uid userdb_gid $EXTRA" if [ "$CREDENTIALS_LOOKUP" = 1 ]; then # If this is a credentials lookup, return password together with its scheme. # The password scheme that Dovecot wants is available in SCHEME environment variable # (e.g. SCHEME=CRAM-MD5), however 'PLAIN' scheme can be converted to anything internally # by Dovecot, so we'll just return 'PLAIN' password. found_password="`echo \"$lookup_result\" | awk -F ':' '{ print $2 }'`" export password="{PLAIN}$found_password" export EXTRA="password $EXTRA" # log_result "credentials lookup result: '$password' [SCHEME='$SCHEME', EXTRA='$EXTRA']" log_result "credentials lookup result: '**********' [SCHEME='$SCHEME', EXTRA='$EXTRA']" else log_result "lookup result: '$lookup_result'" fi # At the end of successful authentication execute checkpassword-reply binary. exec $CHECKPASSWORD_REPLY_BINARY else # If matching credentials were not found, return proper error code depending on lookup mode. if [ "$AUTHORIZED" = 1 -a "$CREDENTIALS_LOOKUP" = 1 ]; then log_result "lookup failed (user not found)" exit $ERR_NOUSER else log_result "lookup failed (credentials are invalid)" exit $ERR_PERMFAIL fi fi # vim:set ts=4 sts=4 sw=4 ai: From h.reindl at thelounge.net Fri Aug 8 07:47:28 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 08 Aug 2014 09:47:28 +0200 Subject: Dovecot, Thunderbird and messages in wrong folder In-Reply-To: <53E46172.4090507@wiesinger.com> References: <53E46172.4090507@wiesinger.com> Message-ID: <53E48090.10004@thelounge.net> Am 08.08.2014 um 07:34 schrieb Gerhard Wiesinger: > Sometimes it happens that messages (e.g. one or two) appear in wrong folders in Thunderbird (latest version, also > past versions, different computers, also Earlybird). When I repair the folder (folder/Settings/Repair Folder) in > Thunderbird, the folder is correct then. > > It happens on my new server with Fedora 20, Dovecot 2.2.13 and Maildir but it also happened on my old server with > Fedora 1, Dovecot 2.2.13 and mbox. > > Has anyone the same issue? > Any guess whether it might be a thunderbird or dovecot issue? > Anyone has the same issues with thunderbird and a different IMAP server (e.g. cyrus)? it's a TB problem and happens with dbmail-imapd too from time to time in fact similar things are happening for sure with Apple Mail and i would say in the one or another form it affects all clients with local caches which had to deal with connection troubles, program crashes or power outages at the wrong moment and my expierience shows that folders/accounts with a desktop MUA, 2 smartphones and a tablet are more likely affected "repair folder" rebuilds the .msf file once per year i stop TB and remove all .msf files from my profile interesting what like unfinished drafts and such things sometime appear :-) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From skdovecot at smail.inf.fh-brs.de Fri Aug 8 08:38:39 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 8 Aug 2014 10:38:39 +0200 (CEST) Subject: Backtrace io_add(0x1) called twice fd= (was Exit status code 134; what is it, in the context of Dovecot Antispam plug-in?) In-Reply-To: <53E39E11.90602@indietorrent.org> References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> <53E39E11.90602@indietorrent.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 7 Aug 2014, Ben Johnson wrote: > On 8/7/2014 11:14 AM, Steffen Kaiser wrote: >> one idea: http://www.tldp.org/LDP/abs/html/exitcodes.html >> >> exit code 134 would be in bash's meaning (if this website is >> correct all) some program died off signal 6. This would be Abort in >> Linux. >> >> prepend your script with >> >> exec >> /tmp/trace 2>&1 set -vx >> >> that will dump anything visible into /tmp/trace >> >> - -- Steffen Kaiser > > Thank you for your continued assistance, Steffen. > > You seem to be exactly correct with the Abort signal. > > I prepended the values you suggested to the pipe script and here's the > relevant output: > > + /usr/lib/dovecot/deliver -d sa-training at example.com -m Training.SPAM > ^A^H5584 prefix=lda: > ^A^F5584 io_add(0x1) called twice fd=7, callback=0x7f23489fb6f0 -> > 0x7f23489aa530 Unfortunately the only spot found is: http://dovecot.org/pipermail/dovecot/2012-May/135636.html Is it the same fd=# always? Is it already open in your script? You could check with lsof -p $$ However, you should check if Dovecot v2.2.13 already fixes the problem. > ^A^D5584 Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x5e271) > [0x7f23489e9271] -> /usr/lib/dovecot/libdovecot.so.0(+0x5e34e) > [0x7f23489e934e] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) > [0x7f23489a4a9e] -> > /usr/lib/dovecot/libdovecot.so.0(ioloop_iolist_add+0x83) > [0x7f23489f9533] -> > /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_add+0x3b) > [0x7f23489f9cbb] -> /usr/lib/dovecot/libdovecot.so.0(io_add+0x9b) > [0x7f23489f89fb] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_io_listeners_add+0x69) > [0x7f23489a9e49] -> > /usr/lib/dovecot/libdovecot.so.0(master_service_init_finish+0xb0) > [0x7f23489a9f90] -> /usr/lib/dovecot/deliver(main+0x1cb) > [0x7f234939269b] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) > [0x7f23485e6ec5] -> /usr/lib/dovecot/deliver(+0x31de) [0x7f23493931de] > /usr/local/bin/sa-learn-pipe.sh: line 52: 5584 Aborted > (core dumped) /usr/lib/dovecot/deliver -d "sa-training at example.com" > -m "Training.$mode" > + echo 'Exit status was 134' > > Does this tell us anything more? > > I don't see any indication that apparmor is at play, as there is no > profile that should apply in this context (and there is nothing in the > relevant log file): > > # service apparmor status > apparmor module is loaded. > 8 profiles are loaded. > 8 profiles are in enforce mode. > /sbin/dhclient > /usr/bin/freshclam > /usr/lib/NetworkManager/nm-dhcp-client.action > /usr/lib/connman/scripts/dhclient-script > /usr/sbin/clamd > /usr/sbin/mysqld > /usr/sbin/ntpd > /usr/sbin/tcpdump > 0 profiles are in complain mode. > 4 processes have profiles defined. > 4 processes are in enforce mode. > /usr/bin/freshclam (2015) > /usr/sbin/clamd (1897) > /usr/sbin/mysqld (1239) > /usr/sbin/ntpd (2472) > 0 processes are in complain mode. > 0 processes are unconfined but have a profile defined. > > I'll try to reproduce this on an identically-configured server. I > wonder if it would be worth changing the version of Dovecot. But I > hate to play whack-a-mole if a more systematic approach is to be had. > > Thanks again, > > -Ben > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+SMkXz1H7kL/d9rAQL7owf/UoNtkoN02JU/1ODYJCaccrpZFTaW1L98 hUPs40eAkh7XsCCe8ymBlG8PzTushkDlqW1EGY2JP3qr5wFV9ACG9ga1Z2oik7CE R3ELjcB6z4D7j2gIHbSGeF+rAIwP2I8K4tVwd4bfVDm2nv+8fAC2OFo4osark1Z9 +3szhhwYIdlon3droAKkUarppXLX9AiYRHHrIpd3ITI55r1x3D6Ni8ClTmyIqKk2 fuKvrFCzU+OIkBegguLfHjhtU6iG4t6RWgb6X77YfnfNy4jHcbeHc9j0dKL4/bP8 Cy5ro1twUcqtC7qQY2kdI3Ka59+dGFhoewFczEsZ8cVOb+ivpv2MiQ== =VORN -----END PGP SIGNATURE----- From Jochen.Bern at LINworks.de Fri Aug 8 09:07:22 2014 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Fri, 08 Aug 2014 11:07:22 +0200 Subject: Exit status code 134; what is it, in the context of Dovecot Antispam plug-in? In-Reply-To: <53E39576.3000705@indietorrent.org> References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> Message-ID: <53E4934A.50001@LINworks.de> On -10.01.-28163 20:59, Ben Johnson wrote: > I have ensured that the pipe script is, in fact, being executed as the > vmail user, by inserting the following into my debug output: > > CURRENT_USER=$(whoami) > echo "$CURRENT_USER" > > This outputs "vmail". FWIW, if a problem with identities and permissions is still a possibility, you should have a look at primary and secondary groups (e.g., output of "id" command) as well. And maybe also the data reported by "umask", "secon --self", ... Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From gayleard at alice.it Fri Aug 8 09:32:07 2014 From: gayleard at alice.it (Timothy Murphy) Date: Fri, 08 Aug 2014 11:32:07 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: <53DF5D14.4000302@thelounge.net> References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53DF5D14.4000302@thelounge.net> Message-ID: <18768869.sRBPJ0IrFd@rose.gayleard.eu> On Monday, August 04, 2014 12:14:44 PM Reindl Harald wrote: > > I've been trying for some days to get postfix + dovecot + pigeonhole/sieve > > working on a CentOS-7 home server. > > I think I may have detected the problem; I read in /var/log/maillog > > > > Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: > > to=, orig_to=, > > relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, > > delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred > > (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 > > Internal error occurred. > > Refer to server log for more information. (in reply to RCPT TO > > command)) > "Refer to server log for more information" comes from dovecot lmtpd > so look there and post the output of "doveconf -n" Thanks very much for your response. The only output in /var/log/dovecot (where I send dovecot logs) is -------------------------- Aug 04 11:29:13 lmtp(14872): Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= Aug 04 11:29:13 lmtp(14872): Info: Connect from local Aug 04 11:29:13 lmtp(14872): Debug: auth input: Aug 04 11:29:13 lmtp(14872): Info: Disconnect from local: Successful quit -------------------------- The output of "doveconf -n" is -------------------------- # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-123.4.4.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 (Core) debug_log_path = /var/log/dovecot info_log_path = /var/log/dovecot listen = * log_path = /var/log/dovecot login_trusted_networks = 192.168.2.0/24 mail_debug = yes mail_gid = 1000 mail_location = maildir:~/Maildir mail_uid = 1000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.sieve/dovecot.sieve sieve_dir = ~/.sieve } protocols = imap pop3 lmtp sieve service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0777 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } ssl = required ssl_cert = From nick.z.edwards at gmail.com Fri Aug 8 10:08:09 2014 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Fri, 8 Aug 2014 20:08:09 +1000 Subject: postfix-dovecot via lmtp In-Reply-To: <53DF6674.2060700@thelounge.net> References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53DF5D14.4000302@thelounge.net> <4621613.Zgn1ePRBzZ@rose.gayleard.eu> <53DF6674.2060700@thelounge.net> Message-ID: sod off dictator who the fuck do you think you are to tell people how to post On 8/4/14, Reindl Harald wrote: > STOP to post HTML and convert even plaintext messages > > Am 04.08.2014 um 12:50 schrieb Timothy Murphy: >> On Monday, August 04, 2014 12:14:44 PM Reindl Harald wrote: >> >> >> >>> > I read in /var/log/maillog >> >>> > >> >>> > Aug 4 11:29:13 alfred postfix/lmtp[14871]: 9C04C220A99: >> >>> > to=, orig_to=, >> >>> > relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=971, >> >>> > delays=971/0.03/0.02/0.1, dsn=4.3.0, status=deferred >> >>> > (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 >> >>> > Internal error occurred. >> >>> > Refer to server log for more information. (in reply to RCPT TO >> >>> > command)) >> >> >> >>> "Refer to server log for more information" comes from dovecot lmtpd >> >>> so look there and post the output of "doveconf -n" >> >> >> >> Thanks very much for your response. >> >> >> >> The only information in /var/log/dovecot >> >> (where I send dovecot debug and info logs) is >> >> Aug 04 11:29:13 lmtp(14872): >> >> Debug: none: root=, index=, indexpvt=, control=, inbox=, alt= >> >> Aug 04 11:29:13 lmtp(14872): >> >> Info: Connect from local >> >> Aug 04 11:29:13 lmtp(14872): >> >> Debug: auth input: >> >> Aug 04 11:29:13 lmtp(14872): >> >> Info: Disconnect from local: Successful quit >> >> >> >> The output of "doveconf -n" is >> >> ------------------------- >> >> # 2.2.10: /etc/dovecot/dovecot.conf >> >> # OS: Linux 3.10.0-123.4.4.el7.x86_64 x86_64 CentOS Linux release 7.0.1406 >> (Core) >> >> debug_log_path = /var/log/dovecot >> >> info_log_path = /var/log/dovecot >> >> listen = * >> >> log_path = /var/log/dovecot >> >> login_trusted_networks = 192.168.2.0/24 >> >> mail_debug = yes >> >> mail_gid = 1000 >> >> mail_location = maildir:~/Maildir >> >> mail_uid = 1000 >> >> managesieve_notify_capability = mailto >> >> managesieve_sieve_capability = fileinto reject envelope encoded-character >> vacation subaddress >> comparator-i;ascii-numeric relational regex imap4flags copy include >> variables body enotify environment mailbox date >> ihave >> >> mbox_write_locks = fcntl >> >> namespace inbox { >> >> inbox = yes >> >> location = >> >> mailbox Drafts { >> >> special_use = \Drafts >> >> } >> >> mailbox Junk { >> >> special_use = \Junk >> >> } >> >> mailbox Sent { >> >> special_use = \Sent >> >> } >> >> mailbox "Sent Messages" { >> >> special_use = \Sent >> >> } >> >> mailbox Trash { >> >> special_use = \Trash >> >> } >> >> prefix = >> >> } >> >> passdb { >> >> driver = pam >> >> } >> >> plugin { >> >> sieve = ~/.sieve/dovecot.sieve >> >> sieve_dir = ~/.sieve >> >> } >> >> protocols = imap pop3 lmtp sieve >> >> service imap-login { >> >> inet_listener imap { >> >> port = 143 >> >> } >> >> inet_listener imaps { >> >> port = 993 >> >> ssl = yes >> >> } >> >> } >> >> service lmtp { >> >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> >> group = postfix >> >> mode = 0777 >> >> user = postfix >> >> } >> >> } >> >> service managesieve-login { >> >> inet_listener sieve { >> >> port = 4190 >> >> } >> >> } >> >> ssl = required >> >> ssl_cert = > >> ssl_key = > >> userdb { >> >> driver = passwd >> >> } >> >> protocol lmtp { >> >> mail_plugins = " sieve" >> >> } >> >> protocol lda { >> >> mail_plugins = " sieve" >> >> } >> >> protocol sieve { >> >> managesieve_notify_capability = fileinto reject envelope environment >> mailbox date >> >> } >> >> ------------------------- >> >> >> >> >> >> -- >> >> Timothy Murphy >> >> e-mail: gayleard /at/ eircom.net >> >> School of Mathematics, Trinity College, Dublin 2, Ireland > > From bourek at thinline.cz Fri Aug 8 10:27:17 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Fri, 08 Aug 2014 12:27:17 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53DF5D14.4000302@thelounge.net> <4621613.Zgn1ePRBzZ@rose.gayleard.eu> <53DF6674.2060700@thelounge.net> Message-ID: <53E4A605.8070803@thinline.cz> On 8.8.2014 12:08, Nick Edwards wrote: > sod off dictator who the fuck do you think you are to tell people how to post > A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing in e-mail? Posting HTML e-mails to mailing lists is also generally considered bad etiquette. So is calling ppl names by the way. From gayleard at alice.it Fri Aug 8 13:56:34 2014 From: gayleard at alice.it (Timothy Murphy) Date: Fri, 08 Aug 2014 15:56:34 +0200 Subject: postfix-dovecot via lmtp In-Reply-To: <53E4A605.8070803@thinline.cz> References: <1846988.K2TEy98DxC@rose.gayleard.eu> <53E4A605.8070803@thinline.cz> Message-ID: <2098910.plZrzrFxWQ@rose.gayleard.eu> On Friday, August 08, 2014 12:27:17 PM Jiri Bourek wrote: > On 8.8.2014 12:08, Nick Edwards wrote: > > sod off dictator who the fuck do you think you are to tell people how to > > post > Posting HTML e-mails to mailing lists is also generally considered bad > etiquette. > So is calling ppl names by the way. As the OP who received this message from Reindl I must say I had no objection to it - the post was rejected by the mailing list as it was too long, because it contained a huge amount of (pointless) HTML code. I did not mean to use HTML, and it sent me on an exploration to find why this was occurring. (I found in the end there was an obscure setting in the mail client - KMail - that I was using.) But Reindl has been very helpful to me on several occasions, and his occasional acerbity is well worth the price. In my experience he always gets to the bottom of a problem, and gives precise instructions for solving it, while many, perhaps most, posters give vague or ambiguous replies. -- Timothy Murphy e-mail: gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin 2, Ireland From ben at indietorrent.org Fri Aug 8 16:40:00 2014 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 08 Aug 2014 12:40:00 -0400 Subject: Backtrace io_add(0x1) called twice fd= (was Exit status code 134; what is it, in the context of Dovecot Antispam plug-in?) In-Reply-To: References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> <53E39E11.90602@indietorrent.org> Message-ID: <53E4FD60.7040808@indietorrent.org> On 8/8/2014 4:38 AM, Steffen Kaiser wrote: > On Thu, 7 Aug 2014, Ben Johnson wrote: >> On 8/7/2014 11:14 AM, Steffen Kaiser wrote: >>> one idea: http://www.tldp.org/LDP/abs/html/exitcodes.html >>> >>> exit code 134 would be in bash's meaning (if this website is >>> correct all) some program died off signal 6. This would be >>> Abort in Linux. >>> >>> prepend your script with >>> >>> exec >> /tmp/trace 2>&1 set -vx >>> >>> that will dump anything visible into /tmp/trace >>> >>> - -- Steffen Kaiser > >> Thank you for your continued assistance, Steffen. > >> You seem to be exactly correct with the Abort signal. > >> I prepended the values you suggested to the pipe script and >> here's the relevant output: > >> + /usr/lib/dovecot/deliver -d sa-training at example.com -m >> Training.SPAM ^A^H5584 prefix=lda: ^A^F5584 io_add(0x1) called >> twice fd=7, callback=0x7f23489fb6f0 -> 0x7f23489aa530 > > Unfortunately the only spot found is: > http://dovecot.org/pipermail/dovecot/2012-May/135636.html > > Is it the same fd=# always? Is it already open in your script? You > could check with lsof -p $$ > > However, you should check if Dovecot v2.2.13 already fixes the > problem. > > -- Steffen Kaiser So, I upgraded to Dovecot 2.2.13, and had to build the antispam plugin from source (because my distro doesn't provide pre-built binary packages for antispam that meet the dependency requirements for the Dovecot 2.2.13 packages that reside at http://xi.rename-it.nl/debian/). The "make" script complained that "dovecot-config" could not be found. Well, from what I can determine, there is no file by this name in /usr/lib/dovecot/; the file name appears to be "config" (not "dovecot-config"). I created a symlink for the name that antispam was expecting and the build process succeeded. Not sure if there was a better way to deal with that, but it seemed to work, and everything seems to be up-and-running at this point. Also, unless I'm mistaken, the "pipe" back-end for antispam has disappeared; I'm using mailtrain instead, which seems to work the same way. Unfortunately, despite the valiant effort, the behavior is exactly the same; still seeing exit code status 134 whenever the antispam plugin fires. :( And yes, Steffen, the fd=7 is always present and the same. > Dovecot tries to be admin-friendly. Common error messages are made > as easily understandable as possible. Any crash, no matter how it > happened, is considered a bug that will be fixed. Have we reached this point yet? Happy to try any other suggestions... Thanks! -Ben From mysqlstudent at gmail.com Fri Aug 8 18:11:21 2014 From: mysqlstudent at gmail.com (Alex) Date: Fri, 8 Aug 2014 14:11:21 -0400 Subject: Throttling pop3-login connections Message-ID: Hi, I have a fedora20 system with dovecot-2.2.13 running various services, including pop3. I'm noticing some users are frequently hamming pop3, and wondered if this was normal, or something I should be investigating? Aug 8 14:05:20 email dovecot: pop3-login: Login: user=, method=PLAIN, rip=97.77.115.121, lip=192.168.1.1, mpid=30509, session= Aug 8 14:05:21 email dovecot: pop3(user1): Disconnected: Logged out top=0/0, retr=0/0, del=0/15, size=5693601 So it is immediately followed by a logout, but when there are 50 of them successively in a five minute period, I wondered if it is creating unnecessary overhead on the system? I suppose this most likely is how they have their email client configured, but wondered if some throttling would be necessary? Any advice would be most appreciated. Thanks, Alex From gheskett at wdtv.com Fri Aug 8 20:40:18 2014 From: gheskett at wdtv.com (Gene Heskett) Date: Fri, 8 Aug 2014 16:40:18 -0400 Subject: Throttling pop3-login connections In-Reply-To: References: Message-ID: <201408081640.18910.gheskett@wdtv.com> On Friday 08 August 2014 14:11:21 Alex did opine And Gene did reply: > Hi, > > I have a fedora20 system with dovecot-2.2.13 running various services, > including pop3. I'm noticing some users are frequently hamming pop3, > and wondered if this was normal, or something I should be > investigating? > > Aug 8 14:05:20 email dovecot: pop3-login: Login: user=, > method=PLAIN, rip=97.77.115.121, lip=192.168.1.1, mpid=30509, > session= > Aug 8 14:05:21 email dovecot: pop3(user1): Disconnected: Logged out > top=0/0, retr=0/0, del=0/15, size=5693601 > > So it is immediately followed by a logout, but when there are 50 of > them successively in a five minute period, I wondered if it is > creating unnecessary overhead on the system? > > I suppose this most likely is how they have their email client > configured, but wondered if some throttling would be necessary? > > Any advice would be most appreciated. > Thanks, > Alex Depends on how they are accessing it. I use fetchmail here, without any working imap (so I am still a lurker trying to figure out this imap thing), and I have fetchmail set to scan each of 3 ISP accounts, sleeping 3 minutes after the scan is complete before starting the next scan. No ISP has complained in the about 8 years I have been doing it 24/7/365.25 Anybody hitting it at a noticeably higher rate should be encouraged to reconfigure their agent for a friendlier scan interval. If that doesn't work, I'd study up on tar pitting. Many email agents are essentially locked for the user while they scan for new mail, so I'm reasonably sure that would "get their attention". I just noticed the rip address and the local address aren't even in the same network block, that would make me check your network as NO 192.168.xx.xx address is supposed to be accessible from a world wide address beyond your router unless you've enabled a port forward rule in the router. That would make me get out the scanner (I use the clamav kit here) looking for evidence of a "powned" machine. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) Genes Web page US V Castleman, SCOTUS, Mar 2014 is grounds for Impeaching SCOTUS From h.reindl at thelounge.net Fri Aug 8 20:55:08 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 08 Aug 2014 22:55:08 +0200 Subject: Throttling pop3-login connections In-Reply-To: <201408081640.18910.gheskett@wdtv.com> References: <201408081640.18910.gheskett@wdtv.com> Message-ID: <53E5392C.10100@thelounge.net> Am 08.08.2014 um 22:40 schrieb Gene Heskett: > On Friday 08 August 2014 14:11:21 Alex did opine >> I have a fedora20 system with dovecot-2.2.13 running various services, >> including pop3. I'm noticing some users are frequently hamming pop3, >> and wondered if this was normal, or something I should be >> investigating? >> >> Aug 8 14:05:20 email dovecot: pop3-login: Login: user=, >> method=PLAIN, rip=97.77.115.121, lip=192.168.1.1, mpid=30509, >> session= >> Aug 8 14:05:21 email dovecot: pop3(user1): Disconnected: Logged out >> top=0/0, retr=0/0, del=0/15, size=5693601 i would ask the user to change at least to 3 minutes instead 1 44000 loglines per month and user is a lot if everybody would do that > I just noticed the rip address and the local address aren't even in the > same network block, that would make me check your network as NO > 192.168.xx.xx address is supposed to be accessible from a world wide > address beyond your router unless you've enabled a port forward rule in > the router that's why it is the *local* IP just normal in any case of proxying -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From mysqlstudent at gmail.com Fri Aug 8 21:45:29 2014 From: mysqlstudent at gmail.com (Alex) Date: Fri, 8 Aug 2014 17:45:29 -0400 Subject: Throttling pop3-login connections In-Reply-To: <201408081640.18910.gheskett@wdtv.com> References: <201408081640.18910.gheskett@wdtv.com> Message-ID: Hi, > Depends on how they are accessing it. I use fetchmail here, without any > working imap (so I am still a lurker trying to figure out this imap > thing), and I have fetchmail set to scan each of 3 ISP accounts, sleeping > 3 minutes after the scan is complete before starting the next scan. No > ISP has complained in the about 8 years I have been doing it 24/7/365.25 > > Anybody hitting it at a noticeably higher rate should be encouraged to > reconfigure their agent for a friendlier scan interval. If that doesn't > work, I'd study up on tar pitting. Many email agents are essentially > locked for the user while they scan for new mail, so I'm reasonably sure > that would "get their attention". Okay, that makes sense, and is in line with what I was also thinking. This is like 30 concurrent requests, then nothing for a minute or two, then another 30 concurrent requests. > I just noticed the rip address and the local address aren't even in the > same network block, that would make me check your network as NO > 192.168.xx.xx address is supposed to be accessible from a world wide > address beyond your router unless you've enabled a port forward rule in > the router. My apologies; this was my attempt at not disclosing the network range. It's a public range, with legitimate users accessing it from around the world. Reindl Harald wrote: > i would ask the user to change at least to 3 minutes instead 1 > 44000 loglines per month and user is a lot if everybody would do that We're in the process of updating the user docs, so we'll add this to it. Thanks so much. Alex From stephan at rename-it.nl Sat Aug 9 01:43:31 2014 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 09 Aug 2014 03:43:31 +0200 Subject: Sieve filter extracting wrong header value on certain mail messages In-Reply-To: References: Message-ID: <53E57CC3.1080308@rename-it.nl> On 8/6/2014 5:56 PM, gowen (dovecot) wrote: > > # 2.2.12: /etc/dovecot/dovecot.conf > # OS: Linux 3.15.4-x86_64-linode45 x86_64 openSUSE 12.3 (x86_64) > > Short version: > > 1) I filter out spam by using a dovecot sieve filter to act based on a > header inserted by bogofilter > 2) In some cases, sieve fails to correctly file based on this header > 3) Testing with sieve-test indicates that it's testing the wrong header > 4) Suggestions? Am I doing something wrong? Is there a problem with > these headers that bends test-sieve out of shape? > 5) Test files from below are also available via > http://www.swynwyr.com/sieve-test/ This looks like a Dovecot bug. We're working on this one. Regards, Stephan. From gowen-dovecot at swynwyr.com Sat Aug 9 02:49:35 2014 From: gowen-dovecot at swynwyr.com (gowen (dovecot)) Date: Fri, 08 Aug 2014 22:49:35 -0400 Subject: Sieve filter extracting wrong header value on certain mail messages In-Reply-To: <53E57CC3.1080308@rename-it.nl> References: <53E57CC3.1080308@rename-it.nl> Message-ID: <0ee25315b65a8153851ee194b8d46bba@www.swynwyr.com> On 2014-08-08 21:43, Stephan Bosch wrote: > On 8/6/2014 5:56 PM, gowen (dovecot) wrote: >> Short version: >> >> 1) I filter out spam by using a dovecot sieve filter to act based on a >> header inserted by bogofilter >> 2) In some cases, sieve fails to correctly file based on this header >> 3) Testing with sieve-test indicates that it's testing the wrong >> header > > This looks like a Dovecot bug. We're working on this one. I'd be happy to help debug it if you can suggest to me how to compile so that I end up with a libdovecot-sieve that has debugging symbols in it. I've downloaded and built dovecot-2.2.13 and dovecot-2.2-pigeonhole-0.4.3, and I can get most of sieve-test built with -g so that I can run it through gdb and have insight, but I can't seem to get that library built with symbols and the interesting stuff (I'm starting with sieve-message.c) is in the library. Thanks, gowen From rs at sys4.de Sat Aug 9 06:48:41 2014 From: rs at sys4.de (Robert Schetterer) Date: Sat, 09 Aug 2014 08:48:41 +0200 Subject: Throttling pop3-login connections In-Reply-To: References: Message-ID: <53E5C449.4010002@sys4.de> Am 08.08.2014 um 20:11 schrieb Alex: > Hi, > > I have a fedora20 system with dovecot-2.2.13 running various services, > including pop3. I'm noticing some users are frequently hamming pop3, and > wondered if this was normal, or something I should be investigating? > > Aug 8 14:05:20 email dovecot: pop3-login: Login: user=, > method=PLAIN, rip=97.77.115.121, lip=192.168.1.1, mpid=30509, > session= > Aug 8 14:05:21 email dovecot: pop3(user1): Disconnected: Logged out > top=0/0, retr=0/0, del=0/15, size=5693601 > > So it is immediately followed by a logout, but when there are 50 of them > successively in a five minute period, I wondered if it is creating > unnecessary overhead on the system? > > I suppose this most likely is how they have their email client configured, > but wondered if some throttling would be necessary? > > Any advice would be most appreciated. > Thanks, > Alex > depends if this are your users, or if its brute force pop3 has not much overhead, to fight brute force use fail2ban or you may have a look here https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/ but be aware with NAT by blocking ips Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From gayleard at alice.it Sat Aug 9 11:30:37 2014 From: gayleard at alice.it (Timothy Murphy) Date: Sat, 09 Aug 2014 13:30:37 +0200 Subject: postfix-dovecot Auth USER lookup failed Message-ID: <8606745.ZkvkI21qDK@rose.gayleard.eu> I'm running postfix + dovecot on my CentOS-7 home server. When I send myself a message I get this error message in /var/log/maillog: Aug 9 12:59:57 alfred postfix/lmtp[31336]: B0D02220748: to=, orig_to=, relay=alfred.gayleard.eu[private/dovecot-lmtp], delay=475, delays=474/0.03/0.02/0.09, dsn=4.3.0, status=deferred (host alfred.gayleard.eu[private/dovecot-lmtp] said: 451 4.3.0 Internal error occurred. Refer to server log for more information. (in reply to RCPT TO command)) and in /var/log/dovecot I read Aug 09 13:13:03 auth-worker(31472): Error: passwd(tim at localhost.gayleard.eu): getpwnam() failed: Address family not supported by protocol Aug 09 13:13:03 lmtp(31470): Error: user tim at localhost.gayleard.eu: Auth USER lookup failed It seems that USER is set to tim at localhost.gayleard.eu rather than tim . I am including the line mailbox_transport = lmtp:unix:private/dovecot-lmtp in /etc/postfix/main.cf . If I omit this line so that postfix sends email directly to ~/Maildir I have no problem - except that spam is not being filtered. I give the output of postconf -n below. Any advice or enlightenment gratefully received. Output of postconf -n ---------------------- [tim at alfred ~]$ cat /tmp/postconf alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases command_directory = /usr/sbin config_directory = /etc/postfix daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 home_mailbox = Maildir/ html_directory = no inet_interfaces = all inet_protocols = all mail_owner = postfix mailbox_transport = lmtp:unix:private/dovecot-lmtp mailq_path = /usr/bin/mailq.postfix manpage_directory = /usr/share/man mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain mydomain = gayleard.eu myhostname = alfred.gayleard.eu mynetworks = 192.168.0.0/24, 127.0.0.0/8 myorigin = $mydomain newaliases_path = /usr/bin/newaliases.postfix queue_directory = /var/spool/postfix readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES relay_domains = relayhost = out.alice.it sample_directory = /usr/share/doc/postfix-2.6.6/samples sendmail_path = /usr/sbin/sendmail.postfix setgid_group = postdrop smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/password smtp_sasl_security_options = smtpd_milters = unix:/var/run/spamass-milter/postfix/sock unknown_local_recipient_reject_code = 550 ---------------------- -- Timothy Murphy e-mail: gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin 2, Ireland From user+dovecot at localhost.localdomain.org Sat Aug 9 16:17:05 2014 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Sat, 09 Aug 2014 16:17:05 +0000 Subject: postfix-dovecot Auth USER lookup failed In-Reply-To: <8606745.ZkvkI21qDK@rose.gayleard.eu> References: <8606745.ZkvkI21qDK@rose.gayleard.eu> Message-ID: <53E64981.1060307@localhost.localdomain.org> On 08/09/2014 11:30 AM, Timothy Murphy wrote: > I'm running postfix + dovecot on my CentOS-7 home server. > When I send myself a message I get this error message in /var/log/maillog: > Aug 9 12:59:57 alfred postfix/lmtp[31336]: B0D02220748: > to=, orig_to=, > relay=alfred.gayleard.eu[private/dovecot-lmtp], > delay=475, delays=474/0.03/0.02/0.09, dsn=4.3.0, > status=deferred (host alfred.gayleard.eu[private/dovecot-lmtp] said: > 451 4.3.0 Internal error occurred. > Refer to server log for more information. (in reply to RCPT TO command)) > and in /var/log/dovecot I read > Aug 09 13:13:03 auth-worker(31472): > Error: passwd(tim at localhost.gayleard.eu): getpwnam() failed: > Address family not supported by protocol > Aug 09 13:13:03 lmtp(31470): Error: user tim at localhost.gayleard.eu: Auth > USER lookup failed > > It seems that USER is set to tim at localhost.gayleard.eu rather than tim . > > I am including the line > mailbox_transport = lmtp:unix:private/dovecot-lmtp > in /etc/postfix/main.cf . your (missing) `doveconf -n` doesn't contain auth_username_format = %Ln. So, edit your conf.d/10-auth.conf and set auth_username_format to %Ln. Regards, Pascal -- The trapper recommends today: cafefeed.1422118 at localdomain.org From mysqlstudent at gmail.com Sat Aug 9 16:23:23 2014 From: mysqlstudent at gmail.com (Alex) Date: Sat, 9 Aug 2014 12:23:23 -0400 Subject: Throttling pop3-login connections In-Reply-To: <53E5C449.4010002@sys4.de> References: <53E5C449.4010002@sys4.de> Message-ID: Hi, > > I have a fedora20 system with dovecot-2.2.13 running various services, > > including pop3. I'm noticing some users are frequently hamming pop3, and > > wondered if this was normal, or something I should be investigating? > > > > Aug 8 14:05:20 email dovecot: pop3-login: Login: user=, > > method=PLAIN, rip=97.77.115.121, lip=192.168.1.1, mpid=30509, > > session= > > Aug 8 14:05:21 email dovecot: pop3(user1): Disconnected: Logged out > > top=0/0, retr=0/0, del=0/15, size=5693601 > > > > So it is immediately followed by a logout, but when there are 50 of them > > successively in a five minute period, I wondered if it is creating > > unnecessary overhead on the system? > > > > I suppose this most likely is how they have their email client configured, > > but wondered if some throttling would be necessary? > > > > Any advice would be most appreciated. > > Thanks, > > Alex > > > > depends if this are your users, or if its brute force > pop3 has not much overhead, to fight brute force use fail2ban Yes, I've implemented fail2ban, and it's working pretty well. It does now look like brute force. When/if they complain to the helpdesk, we'll deal with it then. > https://sys4.de/de/blog/2014/03/27/fighting-smtp-auth-brute-force-attacks/ This is also helpful, thanks. Thanks, Alex From andrzej.filip at gmail.com Sat Aug 9 16:49:44 2014 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Sat, 09 Aug 2014 18:49:44 +0200 Subject: Converting maildir files from quoted-printable to 8bit Message-ID: <53E65128.2050908@gmail.com> How to easily convert maildir files from (single part) quoted-printable to 8bit encoding? [I would like to ease access to sent/posted archives.] From akbwiz+dovecot at gmail.com Sat Aug 9 18:30:07 2014 From: akbwiz+dovecot at gmail.com (Akash) Date: Sun, 10 Aug 2014 00:00:07 +0530 Subject: Dovecot v2.2 FTS is not indexing "text/html" emails... Message-ID: <566f2dac006f8b1f11c9402a450df9a3@mailjol.info> Hi, I am not sure its intended or a fault in the newest Dovecot versions. I have been using Dovecot v1.2.15 on Debian squeeze and FTS is working as expected. When I search a quoted string "very good", I get 107 results including plain and HTML emails which have this phrase. In order to compare the benefits of lucene over squat, I recently started testing dovecot v2.2.13 on Debian Sid with the same maildir content. But now the same search "very good" yielded just 8 results. I thought it could be some problem with lucene so I tried switching to squat and got 107 results again. After this I deleted the old squat search index files created by v1.2.15 and re-indexed the mail-box by using doveadm index command. Now the same squat search is giving 8 results just as lucene. So I have realized that its not a problem with just lucene but FTS in newer dovecot isn't indexing those emails which have Content-type as text/html. Thus if a mail is like this: Content-Type: text/html He is very good. It isn't shown in search by the squat indexes created using dovecot v2.2.13. I have done further testing on some sample emails which confirmed this behavior. Why is this so? -Regards, Akash From tss at iki.fi Sat Aug 9 22:19:41 2014 From: tss at iki.fi (Timo Sirainen) Date: Sun, 10 Aug 2014 01:19:41 +0300 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E0C3DB.6010108@netcologne.de> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> Message-ID: <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> On 05 Aug 2014, at 14:45, Christian Rohmann wrote: > may I PING this subject once again to maybe get Timo's opinion. > > On 23.07.2014 11:26, Christian Rohmann wrote: >> Bounced / rejected messages for something that will be usually be >> resolved very quickly and the messages can then be delivered after all >> is just not very nice for users. The admin made a mistake and the users >> have to deal with the problems is just not my approach. >> >> But in the end I don't even want to argue that rejecting the messages >> might not be a valid behavior for some. That's why I suggested to make >> this configurable, just like the quota behavior. > > I'd really like to hear Timo's view on having lmtp do a (configurable) > DEFER when the disk is full which is, most likely, a "temporary" error. My opinion: It shouldn't be configurable - it should always cause temporary error. The only thing I'm slightly worried about is if write failures because of "user's filesystem quota full" will always return EDQUOT error for write() instead of ENOSPC, but I suppose they will in any modern OS. And it would require changing MAIL_ERROR_NOSPACE definition a bit inside Dovecot, but that's less of an issue. I'll change this once I have some time&energy. Patches welcome also. From tss at iki.fi Sat Aug 9 22:31:47 2014 From: tss at iki.fi (Timo Sirainen) Date: Sun, 10 Aug 2014 01:31:47 +0300 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> Message-ID: On 10 Aug 2014, at 01:19, Timo Sirainen wrote: >> I'd really like to hear Timo's view on having lmtp do a (configurable) >> DEFER when the disk is full which is, most likely, a "temporary" error. > > My opinion: It shouldn't be configurable - it should always cause temporary error. The only thing I'm slightly worried about is if write failures because of "user's filesystem quota full" will always return EDQUOT error for write() instead of ENOSPC, but I suppose they will in any modern OS. And it would require changing MAIL_ERROR_NOSPACE definition a bit inside Dovecot, but that's less of an issue. And a bit more generic statement about anything related to errors in Dovecot: Problems that admins can solve are temporary errors for users and the'll need an error logged. Problems that are caused by users themselves (like over quota) are usually not temporary errors and they shouldn't have errors logged (since admin can't usually do anything about them anyway). From dovecot.org at veggiechinese.net Sun Aug 10 04:18:23 2014 From: dovecot.org at veggiechinese.net (Will Yardley) Date: Sat, 9 Aug 2014 21:18:23 -0700 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> Message-ID: <20140810041823.GA79430@aura.veggiechinese.net> On Sun, Aug 10, 2014 at 01:31:47AM +0300, Timo Sirainen wrote: > Problems that admins can solve are temporary errors for users and > the'll need an error logged. Problems that are caused by users > themselves (like over quota) are usually not temporary errors and they > shouldn't have errors logged (since admin can't usually do anything > about them anyway). Depends on the environment; in many cases, the admin could, or may even be expected to, raise the quota. Also, you're assuming that users will be able to interpret an error message, even a clear one, correctly, and that the MUA will always convey the error to the user. I am not sure either of these assumptions are always true. Logging is important so when the user calls in with "I can't do X", the admin can see why quickly. w From dovecot at vosslamber.nl Sun Aug 10 16:24:47 2014 From: dovecot at vosslamber.nl (Luuk) Date: Sun, 10 Aug 2014 18:24:47 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <20140810041823.GA79430@aura.veggiechinese.net> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> Message-ID: <53E79CCF.80709@vosslamber.nl> On 10-8-2014 06:18, Will Yardley wrote: > Depends on the environment; in many cases, the admin could, or may even > be expected to, raise the quota. !but should not! Quota should be set according to some rules, and never be raised because of a 'quota reached'. What is the use of 'quota' if the admin raises your quota when things are full? From claus.r at bayern-mail.de Sun Aug 10 17:30:34 2014 From: claus.r at bayern-mail.de (Claus) Date: Sun, 10 Aug 2014 19:30:34 +0200 Subject: doveadm proxy list Message-ID: <53E7AC3A.7090902@bayern-mail.de> Hi, i have configured my dovecot-proxy with "shutdown_clients = no" and "service_count = 1" for a "default_process_limit = 5000" and "default_client_limit = 25000". I don't know if this is a prefered setup for so many users/concurrent sessions but so i can reload my dovecot without disconnecting all sessions. The main problem are pop3 sessions - if i kill them, the mailbox remains locked and the client gets an error. Back to my problem: If i reload dovecot the "doveadm proxy list" shows me the list of the new processes. Is there any possibility to see the stats of the old sessions (user,source,destination) ? greet's Claus PS.: Could someone explain me why i have to set default_client_limit to 5x default_process_limit? From gayleard at alice.it Sun Aug 10 17:40:37 2014 From: gayleard at alice.it (Timothy Murphy) Date: Sun, 10 Aug 2014 19:40:37 +0200 Subject: postfix-dovecot Auth USER lookup failed References: <8606745.ZkvkI21qDK@rose.gayleard.eu> <53E64981.1060307@localhost.localdomain.org> Message-ID: <52E53D4F217EE624@smtp204.alice.it> (added by postmaster@alice.it) Pascal Volk wrote: >> in /var/log/dovecot I read >> Aug 09 13:13:03 auth-worker(31472): >> Error: passwd(tim at localhost.gayleard.eu): getpwnam() failed: >> Address family not supported by protocol >> Aug 09 13:13:03 lmtp(31470): Error: user tim at localhost.gayleard.eu: >> Auth >> USER lookup failed >> >> It seems that USER is set to tim at localhost.gayleard.eu rather than tim . > your (missing) `doveconf -n` doesn't contain auth_username_format = %Ln. > So, edit your conf.d/10-auth.conf and set auth_username_format to %Ln. Thanks very much - that seems to have been the whole problem. After changing %Lu (the default) to %Ln everything seems to be working, including spam being sent to ~/Maildir/.Spam. It does seem slightly odd to me that %u stands for user + domain, while %n stands for user without domain? And why is %Lu the default? Is it just for those using an SQL user database? -- Timothy Murphy e-mail: gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin 2, Ireland From frozenyoghurt2010 at gmail.com Mon Aug 11 14:14:39 2014 From: frozenyoghurt2010 at gmail.com (Michael Ludwig) Date: Mon, 11 Aug 2014 16:14:39 +0200 Subject: Frequent error: unknown namespace Trash Message-ID: Hello to all of you! ;-) My first post on this list. I frequently get this error in dovecot.log every time a user logs in and out: ============================= imap(mailaddress): Error: quota: Unknown namespace: Trash ============================= When I change my conf.d/90-quota.conf from "Trash" to ".Trash" (as this is the imap/maildir folder name, the error stays the same, but with the ".Trash" in dovecot.log This is what my 90-quoato.conf looks like: ============================= plugin { quota_rule = *:storage=2G quota_rule2 = Trash:storage=+100M quota_grace = 10%% } ============================= Where can I start my search, what is going wrong? Just for letting you know: This mailserver is a fresh and clean setup. I am using postfix/dovecotvimbadmin/mysql/spamassassin/clamsmtp. Cheers, Michael From crohmann at netcologne.de Mon Aug 11 14:22:17 2014 From: crohmann at netcologne.de (Christian Rohmann) Date: Mon, 11 Aug 2014 16:22:17 +0200 Subject: Options mail_nfs_storage and mail_nfs_index not in 2.2(.13) example config anymore? Message-ID: <53E8D199.9010908@netcologne.de> Hey dovecot-users, did I miss anything anywhere about the configuration variables "nfs_storage" and "mail_nfs_index" being removed from the example configuration in 2.2? In 2.1 they where both in 10-mail.conf commented out and set to their default "no". But with 2.2 they are not in any of the exmaple config files anymore. doveadm is still showing the variables and they can be used as ever still. Just wondering if I missed something and this should be configured differently on 2.2 now. Regards Christian From frozenyoghurt2010 at gmail.com Mon Aug 11 14:55:48 2014 From: frozenyoghurt2010 at gmail.com (Michael Ludwig) Date: Mon, 11 Aug 2014 16:55:48 +0200 Subject: Frequent error: unknown namespace Trash Message-ID: Hello to all of you! ;-) My first post on this list. I frequently get this error in dovecot.log every time a user logs in and out: ============================= imap(mailaddress): Error: quota: Unknown namespace: Trash ============================= When I change my conf.d/90-quota.conf from "Trash" to ".Trash" (as this is the imap/maildir folder name, the error stays the same, but with the ".Trash" in dovecot.log This is what my 90-quoato.conf looks like: ============================= plugin { quota_rule = *:storage=2G quota_rule2 = Trash:storage=+100M quota_grace = 10%% } ============================= Where can I start my search, what is going wrong? Just for letting you know: This mailserver is a fresh and clean setup. I am using postfix/dovecotvimbadmin/mysql/spamassassin/clamsmtp. Cheers, Michael From list1 at jrice.me Mon Aug 11 15:42:05 2014 From: list1 at jrice.me (Jeff Rice) Date: Mon, 11 Aug 2014 11:42:05 -0400 Subject: Sieve: Saving "pristine" messages for backups and spam training Message-ID: <53E8E44D.4010600@jrice.me> Hello, I'm trying to work out a way to have my Sieve filter save a "pristine" version of email messages as a backup, primarily to use for training the spam filter. I would like is to have every message saved into a single, site-wide directory (in the global sieve) before being processed additionally and delivered. The messages in that directory will be used to train the spam filter without having to worry about removing Spamassassin headers and so forth. I thought fileinto :copy might do what I wanted, but this creates a backup directory individually for each user. That's unmanageable for the spam training process I use. redirect *could* work, but that adds a header during the process so the email saved would not be "pristine". I'm thinking of using the extprograms plugin to pipe to a program that will do a simple copy. That feels very hackish, however, and I'm hoping there is a more elegant solution. Am I missing something obvious here? Thanks! Jeff From rs at sys4.de Mon Aug 11 16:01:34 2014 From: rs at sys4.de (Robert Schetterer) Date: Mon, 11 Aug 2014 18:01:34 +0200 Subject: Sieve: Saving "pristine" messages for backups and spam training In-Reply-To: <53E8E44D.4010600@jrice.me> References: <53E8E44D.4010600@jrice.me> Message-ID: <53E8E8DE.20407@sys4.de> Am 11.08.2014 um 17:42 schrieb Jeff Rice: > Hello, > I'm trying to work out a way to have my Sieve filter save a "pristine" > version of email messages as a backup, primarily to use for training the > spam filter. why , mail passes your smtp server with i.e spamass-milter ( i.e tagged spam will train bayes auto ,depend to setup ) the very rest of false postive or untagged spam, should send from users to i.e a train script etc, spam tagged mail could be filter auto to Junk folder by sieve global rule ( with pop3 use virtual dove setup ) I would like is to have every message saved into a single, > site-wide directory (in the global sieve) before being processed > additionally and delivered. The messages in that directory will be used > to train the spam filter without having to worry about removing > Spamassassin headers and so forth. > > I thought fileinto :copy might do what I wanted, but this creates a > backup directory individually for each user. That's unmanageable for > the spam training process I use. redirect *could* work, but that adds a > header during the process so the email saved would not be "pristine". > > I'm thinking of using the extprograms plugin to pipe to a program that > will do a simple copy. That feels very hackish, however, and I'm hoping > there is a more elegant solution. > > Am I missing something obvious here? keep stuff simple > > Thanks! > Jeff Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From tss at iki.fi Mon Aug 11 16:59:19 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Aug 2014 19:59:19 +0300 Subject: poolmon improvements Message-ID: <270843B8-9D0A-4FC6-95A9-0EDB5CA2F8AE@iki.fi> I've been planning to improve poolmon failure checking for a long time already, but I still haven't managed to get to it. Maybe somebody else has more time, so here's a feature request for anyone to implement: poolmon currently gives up immediately if the first check to any service fails. It really should be trying multiple times over multiple seconds before giving up. I think ideally it would be: - Individual check timeout could still be the default 5 seconds - Add full check time setting, which could be e.g. 15 seconds. If all checks fail during this time then disable the host. - If request fails because connection gets rejected, retry quickly, e.g. after 0,1 seconds - If check fails because of protocol errors, wait for a long time, e.g. 1 second So this would avoid backend being removed in situations where it really shouldn't be removed: - Dovecot restarts - Dovecot reloads - load spikes and other random issues that cause temporary problems Especially the load spike is an annoying issue which my plan doesn't even fully solve. The solution to fix a heavily overloaded cluster isn't really to start removing all of its backends that are busy working.. From tss at iki.fi Mon Aug 11 17:02:08 2014 From: tss at iki.fi (Timo Sirainen) Date: Mon, 11 Aug 2014 20:02:08 +0300 Subject: Options mail_nfs_storage and mail_nfs_index not in 2.2(.13) example config anymore? In-Reply-To: <53E8D199.9010908@netcologne.de> References: <53E8D199.9010908@netcologne.de> Message-ID: On 11 Aug 2014, at 17:22, Christian Rohmann wrote: > Hey dovecot-users, > > did I miss anything anywhere about the configuration variables > "nfs_storage" and "mail_nfs_index" > being removed from the example configuration in 2.2? > > In 2.1 they where both in 10-mail.conf commented out and set to their > default "no". But with 2.2 they are not in any of the exmaple config > files anymore. > > doveadm is still showing the variables and they can be used as ever still. > > Just wondering if I missed something and this should be configured > differently on 2.2 now. They have never fully worked as http://wiki2.dovecot.org/NFS mentions. I removed them from the configs so people wouldn't accidentally try to use them. The only supported solution is to use Dovecot director. If you use only a single Dovecot server then you don't need the mail_nfs_* settings either. From dovecot.org at veggiechinese.net Mon Aug 11 20:44:29 2014 From: dovecot.org at veggiechinese.net (Will Yardley) Date: Mon, 11 Aug 2014 13:44:29 -0700 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E79CCF.80709@vosslamber.nl> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> <53E79CCF.80709@vosslamber.nl> Message-ID: <20140811204429.GE36407@aura.veggiechinese.net> On Sun, Aug 10, 2014 at 06:24:47PM +0200, Luuk wrote: > On 10-8-2014 06:18, Will Yardley wrote: > > Depends on the environment; in many cases, the admin could, or may even > > be expected to, raise the quota. > > !but should not! > > Quota should be set according to some rules, and never be raised because > of a 'quota reached'. A few things on this: 1) "Should" is a matter of opinion, and different environments have different business requirements. I've worked as a sysadmin for almost 15 years, in a variety of different settings (small startup, larger startup, later acquired by a corporation, academic), and I've found that there's always *some* squeaky wheel who is going to make a lot of noise and get their quota raised. In several roles, in fact, that person is often another technical person, who also happened to be my boss. In fact, I've been in plenty of situations where quotas can't be set at all, or are set so high that they're basically useless. I thought (at times) that changing to a different setting, for example, education, might change this, but I have not found this to be the case. In a pure ISP / hosting provider type situation, it is often necessary to have a strict policy about quotas; however, on the corporate side of that same organization, there are often different business requirements. So, if you enjoy the cozy situation of being able to tell your users what quota they can have, in all circumstances, more power to you, but I don't think this is typical in the "real world". And, in a sense, it needen't always be. If more disk space is what whoever the most important users in your organization "need" to get their work done in a way that's comfortable for them, it may well be the case that this is exactly what you'll need to provide for them -- especially if the organization is willing to fund the hardware necessary to support larger quotas for some, or all, users. 2) Again, even if the quota policy is strict, it's not always the case that users understand the error message they get, even if their MUA presents it in a friendly way. 3) In many cases, users aren't able to delete mail to get under their (hard) quota without having the quota raised temporarily. If their mail client deletes by copying messages into the trash, then purging, for example, I've seen cases where the only way for the user to trim down their usage is to temporarily increase the quota long enough for them to get their usage down. 4) Some environments (and some users) require a more "high touch" approach than others. > What is the use of 'quota' if the admin raises your quota when things > are full? One use is to prevent a mail loop or other problem affecting one or two users from filling up a storage volume. Another is so that usage requirements (and exceptions to default quotas) can be tracked. Regardless of how individual organizations handles quotas, I don't see how having Dovecot log an over quota event would be a bad thing. w From h.reindl at thelounge.net Mon Aug 11 20:49:32 2014 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 11 Aug 2014 22:49:32 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <20140811204429.GE36407@aura.veggiechinese.net> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> <53E79CCF.80709@vosslamber.nl> <20140811204429.GE36407@aura.veggiechinese.net> Message-ID: <53E92C5C.90305@thelounge.net> Am 11.08.2014 um 22:44 schrieb Will Yardley: > 3) In many cases, users aren't able to delete mail to get under their > (hard) quota without having the quota raised temporarily. If their mail > client deletes by copying messages into the trash, then purging, for > example, I've seen cases where the only way for the user to trim down > their usage is to temporarily increase the quota long enough for them to > get their usage down that is all fine and true but that don't change the fact that if there is 'no space left on device' someone did not do his job properly and a full disk happens all 10 years do you really think that a mailservers job is to handle that all 10 years happening mistake special? why? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From jtam.home at gmail.com Mon Aug 11 21:41:52 2014 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 11 Aug 2014 14:41:52 -0700 (PDT) Subject: Throttling pop3-login connections In-Reply-To: References: Message-ID: Alex writes: > Okay, that makes sense, and is in line with what I was also thinking. This > is like 30 concurrent requests, then nothing for a minute or two, then > another 30 concurrent requests. 30 concurrent requests at 1 or 2 minute intervals? This seems weird, as I would expect it to be more or less randomly distributed over that interval. Does this happen at the top of each minute? If what you observe is true, the only situation I can think of that would cause this is a bunch of POP3 fetch clients running as a cron job synchronized to a minute grid (i.e seconds=0). Joseph Tam From bourek at thinline.cz Mon Aug 11 21:51:29 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Mon, 11 Aug 2014 23:51:29 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <20140810041823.GA79430@aura.veggiechinese.net> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> Message-ID: <53E93AE1.2050607@thinline.cz> On 10.8.2014 06:18, Will Yardley wrote: > On Sun, Aug 10, 2014 at 01:31:47AM +0300, Timo Sirainen wrote: > >> Problems that admins can solve are temporary errors for users and >> the'll need an error logged. Problems that are caused by users >> themselves (like over quota) are usually not temporary errors and they >> shouldn't have errors logged (since admin can't usually do anything >> about them anyway). > > Depends on the environment; in many cases, the admin could, or may even > be expected to, raise the quota. > If you're expected to raise the quota in case it's exceeded, don't set it in the first place, there is no point to it. Or - if you really want that exercise in futility - use quota_warning and raise it automatically. As for running out of disk space completely - well I assume you're using some kind of monitoring service, so just add a check for free disk space and you're done. There's really no need to cover this in Dovecot. From ben at indietorrent.org Mon Aug 11 21:52:35 2014 From: ben at indietorrent.org (Ben Johnson) Date: Mon, 11 Aug 2014 17:52:35 -0400 Subject: Sieve: Saving "pristine" messages for backups and spam training In-Reply-To: <53E8E44D.4010600@jrice.me> References: <53E8E44D.4010600@jrice.me> Message-ID: <53E93B23.3030403@indietorrent.org> On 8/11/2014 11:42 AM, Jeff Rice wrote: > Hello, > I'm trying to work out a way to have my Sieve filter save a "pristine" > version of email messages as a backup, primarily to use for training the > spam filter. I would like is to have every message saved into a single, > site-wide directory (in the global sieve) before being processed > additionally and delivered. The messages in that directory will be used > to train the spam filter without having to worry about removing > Spamassassin headers and so forth. Provided I understand you correctly, my first thought is that saving a duplicate copy of every single message that arrives on this system seems wasteful. Why not save only the messages that would actually be useful for spam training purposes? > > I thought fileinto :copy might do what I wanted, but this creates a > backup directory individually for each user. That's unmanageable for > the spam training process I use. redirect *could* work, but that adds a > header during the process so the email saved would not be "pristine". > > I'm thinking of using the extprograms plugin to pipe to a program that > will do a simple copy. That feels very hackish, however, and I'm hoping > there is a more elegant solution. > There is; the Dovecot Antispam plug-in. It does exactly what you describe, and it addresses the problem of storing a duplicate copy of all messages. In short, when a user drags a message from any folder to "Junk", you'll receive a "pristine" copy of the message at any local address you specify, delivered to any folder you specify (e.g., "Train as SPAM") within that "training user's" mailbox. Conversely, when a user drags a message from "Junk" to any other folder, you'll receive a copy of the message in your "Train as HAM" folder. Then, you can point your anti-spam solution's training executable to these two "pristine master corpus" folders. If you ever need to reclassify messages, or expunge them, doing so is trivial with this master corpus approach. > Am I missing something obvious here? > > Thanks! > Jeff Happy to provide a sample script for the antispam plugin's mailtrain back-end, as that's the one I use. Cheers, -Ben From crohmann at netcologne.de Mon Aug 11 22:18:44 2014 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 12 Aug 2014 00:18:44 +0200 (CEST) Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E79CCF.80709@vosslamber.nl> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> <53E79CCF.80709@vosslamber.nl> Message-ID: <1609581771.25229.1407795524936.open-xchange@cc-app1.netcologne.de> > Luuk hat am 10. August 2014 um 18:24 geschrieben: > Quota should be set according to some rules, and never be raised because > of a 'quota reached'. > What is the use of 'quota' if the admin raises your quota when things > are full? Guys, you are mixing up user quota and the "disk full" or better "no space left on device" situation. The whole point of my question or rather "feature request" was, to change the behavior of Dovecot when the whole disk (read: affects all users,regardless of their set quota and its current usage) is full and there is no space left to store any new messages. Dovecot already does support to either reject or defer on a reached quota of a particular user. The problem I was talking about is the agreeably rather seldom admin f&$"-up of not properly maintaining the disk space or any other resource. But in the end, I hate to punish users (a.k.a "people") and send them looking for their bounced email because of this very temporary problem. Regards Christian From andrzej.filip at gmail.com Mon Aug 11 22:42:37 2014 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Tue, 12 Aug 2014 00:42:37 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> Message-ID: <53E946DD.8040501@gmail.com> On 08/10/2014 12:31 AM, Timo Sirainen wrote: > On 10 Aug 2014, at 01:19, Timo Sirainen wrote: > >>> I'd really like to hear Timo's view on having lmtp do a (configurable) >>> DEFER when the disk is full which is, most likely, a "temporary" error. >> >> My opinion: It shouldn't be configurable - it should always cause temporary error. The only thing I'm slightly worried about is if write failures because of "user's filesystem quota full" will always return EDQUOT error for write() instead of ENOSPC, but I suppose they will in any modern OS. And it would require changing MAIL_ERROR_NOSPACE definition a bit inside Dovecot, but that's less of an issue. > > And a bit more generic statement about anything related to errors in Dovecot: > > Problems that admins can solve are temporary errors for users and the'll need an error logged. Problems that are caused by users themselves (like over quota) are usually not temporary errors and they shouldn't have errors logged (since admin can't usually do anything about them anyway). Do you plan to support MTA queries about mailbox status (overquota,no space left on device)? It would allow MTA to deploy its own policies before sending "RCPT TO:" reply. "Socket map" protocol may be handy. From list1 at jrice.me Mon Aug 11 23:17:09 2014 From: list1 at jrice.me (Jeff Rice) Date: Mon, 11 Aug 2014 19:17:09 -0400 Subject: Sieve: Saving "pristine" messages for backups and spam training In-Reply-To: <53E93B23.3030403@indietorrent.org> References: <53E8E44D.4010600@jrice.me> <53E93B23.3030403@indietorrent.org> Message-ID: <53E94EF5.5090202@jrice.me> > Ben Johnson > August 11, 2014 at 5:52 PM > On 8/11/2014 11:42 AM, Jeff Rice wrote: >> I'm trying to work out a way to have my Sieve filter save a "pristine" >> version of email messages as a backup, primarily to use for training the >> spam filter. I would like is to have every message saved into a single, >> site-wide directory (in the global sieve) before being processed >> additionally and delivered. The messages in that directory will be used >> to train the spam filter without having to worry about removing >> Spamassassin headers and so forth. > > Provided I understand you correctly, my first thought is that saving a > duplicate copy of every single message that arrives on this system seems > wasteful. > A bit wasteful, but disk space is cheap and it's a limited, rolling backup. The value of retraining goes down significantly as time passes, so I'm not planning on keeping messages there for an extended period of time. Cron will clean out older messages after a set period of time. >> I'm thinking of using the extprograms plugin to pipe to a program that >> will do a simple copy. That feels very hackish, however, and I'm hoping >> there is a more elegant solution. > > There is; the Dovecot Antispam plug-in. It does exactly what you > describe, and it addresses the problem of storing a duplicate copy of > all messages. > > In short, when a user drags a message from any folder to "Junk", you'll > receive a "pristine" copy of the message at any local address you > specify, delivered to any folder you specify (e.g., "Train as SPAM") > within that "training user's" mailbox. > Hmm. Perhaps I'm just dense, but I don't see this behavior documented in the Antispam plugin docs. I'm happy to be corrected if I've misunderstood. I'd rather use an existing tool if possible. What I can see that Antispam will train on the version of the message the user drags into the "Junk" folder. But that message may have had headers added by a sieve filter or Spamassassin, for example. By "pristine", I mean "as received" by the LDA. CRM114's "reaver_cache" is along the lines of what I'm thinking of. Jeff > Jeff Rice > August 11, 2014 at 11:42 AM > Hello, > I'm trying to work out a way to have my Sieve filter save a "pristine" > version of email messages as a backup, primarily to use for training > the spam filter. I would like is to have every message saved into a > single, site-wide directory (in the global sieve) before being > processed additionally and delivered. The messages in that directory > will be used to train the spam filter without having to worry about > removing Spamassassin headers and so forth. > > I thought fileinto :copy might do what I wanted, but this creates a > backup directory individually for each user. That's unmanageable for > the spam training process I use. redirect *could* work, but that adds > a header during the process so the email saved would not be "pristine". > > I'm thinking of using the extprograms plugin to pipe to a program that > will do a simple copy. That feels very hackish, however, and I'm > hoping there is a more elegant solution. > > Am I missing something obvious here? > > Thanks! > Jeff From jean-louis at dupond.be Tue Aug 12 07:53:39 2014 From: jean-louis at dupond.be (Jean-Louis Dupond) Date: Tue, 12 Aug 2014 09:53:39 +0200 Subject: Multiple Passdb's - Order In-Reply-To: <6fb75bf2cf5a93842d38610ab3bcbf1d@dupond.be> References: <53DBF1C3.9000307@dupond.be> <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> <6fb75bf2cf5a93842d38610ab3bcbf1d@dupond.be> Message-ID: <157be37205187883a127c9ac1e41d1c3@dupond.be> Anyone might have a clue? Would like to get the issue solved. Or at least know why its acting this way. Thanks Jean-Louis Op 2014-08-04 15:11, schreef Jean-Louis Dupond: > Hi, > > Thanks for your response. > The error/warning log was indeed not included. > > I looked it up now, and dovecot only printed the following error: > Aug 1 21:31:55 xcx dovecot: auth: Error: > ldap(info at xxx.com,xxxxx,): > ldap_search(base=uid=info at xxx.com,ou=mail,dc=xxx,dc=com > filter=(&(objectClass=posixAccount)(uid=info at dupie.be))) failed: No > such object > > This shouldn't really be a problem, because the user does indeed not > exist in ldap, but only in the passwd-file. > The problem also exists with auth cache disabled. > > And my doveconf (the passdb order is the working order here): > # 2.2.9: /etc/dovecot/dovecot.conf > # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 > auth_cache_size = 2 M > mail_location = maildir:~/Maildir > mail_plugins = virtual > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace inbox { > inbox = yes > location = > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > mailbox virtual/Flagged { > special_use = \Flagged > } > prefix = > } > passdb { > args = scheme=CRYPT username_format=%u /etc/dovecot/users > driver = passwd-file > } > passdb { > args = /etc/dovecot/dovecot-ldap.conf.ext > driver = ldap > } > plugin { > sieve = /var/vmail/%d/%n/.dovecot.sieve > sieve_dir = /var/vmail/%d/%n/sieve > } > postmaster_address = postmaster at xxx.com > protocols = " imap lmtp sieve pop3" > service auth { > unix_listener /var/spool/postfix-mailing/private/auth { > mode = 0666 > } > unix_listener /var/spool/postfix/private/auth { > mode = 0666 > } > } > service lmtp { > unix_listener /var/spool/postfix/private/dovecot-lmtp { > group = postfix > mode = 0600 > user = postfix > } > user = vmail > } > service managesieve-login { > inet_listener sieve { > port = 4190 > } > } > ssl_ca = ssl_cert = ssl_key = userdb { > args = uid=vmail gid=vmail home=/var/vmail/%d/%n > driver = static > } > protocol lmtp { > mail_plugins = virtual sieve > } > > Thanks > Jean-Louis > > > Op 2014-08-04 15:03, schreef Timo Sirainen: >> On 01 Aug 2014, at 22:00, Jean-Louis Dupond >> wrote: >> >>> Today I upgraded my dovecot to a newer version (2.2.9 debian >>> wheezy-backports) >>> At that moment I also reconfigured the whole thing a bit nicer. >>> >>> Now I stumbled on the following issue. >> .. >>> The only thing I changed here was the order the passdb backends. >>> >>> It works fine now, but it not clear to me why the order is so >>> important. >>> Could somebody explain this? Or is this some bug? >> >> I can't reproduce it. Also did your log contain also the >> errors/warnings? It appeared that it was failing with some internal >> error, which you didn't show in the log. Also does it work if you >> disable auth cache? Full doveconf -n output may have been helpful >> also. From skdovecot at smail.inf.fh-brs.de Tue Aug 12 08:15:42 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Tue, 12 Aug 2014 10:15:42 +0200 (CEST) Subject: Sieve: Saving "pristine" messages for backups and spam training In-Reply-To: <53E94EF5.5090202@jrice.me> References: <53E8E44D.4010600@jrice.me> <53E93B23.3030403@indietorrent.org> <53E94EF5.5090202@jrice.me> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 11 Aug 2014, Jeff Rice wrote: >> Ben Johnson >> August 11, 2014 at 5:52 PM >> On 8/11/2014 11:42 AM, Jeff Rice wrote: >>> I'm thinking of using the extprograms plugin to pipe to a program that >>> will do a simple copy. That feels very hackish, however, and I'm hoping >>> there is a more elegant solution. >> >> There is; the Dovecot Antispam plug-in. It does exactly what you >> describe, and it addresses the problem of storing a duplicate copy of >> all messages. >> >> In short, when a user drags a message from any folder to "Junk", you'll >> receive a "pristine" copy of the message at any local address you >> specify, delivered to any folder you specify (e.g., "Train as SPAM") >> within that "training user's" mailbox. >> > Hmm. Perhaps I'm just dense, but I don't see this behavior documented in the > Antispam plugin docs. I'm happy to be corrected if I've misunderstood. I'd > rather use an existing tool if possible. > > What I can see that Antispam will train on the version of the message the > user drags into the "Junk" folder. But that message may have had headers > added by a sieve filter or Spamassassin, for example. By "pristine", I mean > "as received" by the LDA. > > CRM114's "reaver_cache" is along the lines of what I'm thinking of. How about this: Your MTA forwards each message to a central mail account, where they get spooled and purged after n days by cron. If you need a "pristine" copy of a message, you take the message from sieve, e.g. via antispam plugin, determine from Message-Id, recieved headers and whatsoever which "pristine" copy could be meant and use the one from the central store. That will also bypass changes of the message added because of delivery itself, because you know that this accounts has no Sieve and you can remove the last recived header etc.pp. >> I thought fileinto :copy might do what I wanted, but this creates a backup >> directory individually for each user. That's unmanageable for the spam >> training process I use. redirect *could* work, but that adds a header >> during the process so the email saved would not be "pristine". If you think an early "sieve_before" command will do, try a hidden namespace, add write-only ACLs for everyone for one mailbox and "fileinto :copy" there. The hidden namespace shall keep it from the eyes of most users, which would ask questions. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+nNLnz1H7kL/d9rAQK3/Af/XjlnbOXtSBcphjMZifx894IbJnDuCRVq QMWbBLbsk+zsOiww9gKcC+99AA7+dPYyGf5E+33U1UkuFi8EwS2YT/IfkF4FeC4x HZ+ERRHwVW5rJBtkx1BzjxWspuH+0X1R3CerdBGW2vifGZ6vr9uUk1gU1mG+kjB9 qp8cTh1PZxKcye3MR+bnbCH/lPNDAnvFVJtNnNBxweE2Ujd6QG4oepS+OH0QH+8R QqiUF4vSrIDc1pcyLkzQus9oCyRaaveTTnuzq5CdQzTOF4awX3X0Co2HJyXokJHa DToIoqd1Czawn/O2vxYoqTRf4ugxrZGx7oKw2YcGFH+/7SxXpU1fQg== =/38s -----END PGP SIGNATURE----- From bourek at thinline.cz Tue Aug 12 08:14:36 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Tue, 12 Aug 2014 10:14:36 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E946DD.8040501@gmail.com> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <53E946DD.8040501@gmail.com> Message-ID: <53E9CCEC.6080001@thinline.cz> On 12.8.2014 00:42, Andrzej A. Filip wrote: > On 08/10/2014 12:31 AM, Timo Sirainen wrote: >> On 10 Aug 2014, at 01:19, Timo Sirainen wrote: >> >>>> I'd really like to hear Timo's view on having lmtp do a (configurable) >>>> DEFER when the disk is full which is, most likely, a "temporary" error. >>> >>> My opinion: It shouldn't be configurable - it should always cause temporary error. The only thing I'm slightly worried about is if write failures because of "user's filesystem quota full" will always return EDQUOT error for write() instead of ENOSPC, but I suppose they will in any modern OS. And it would require changing MAIL_ERROR_NOSPACE definition a bit inside Dovecot, but that's less of an issue. >> >> And a bit more generic statement about anything related to errors in Dovecot: >> >> Problems that admins can solve are temporary errors for users and the'll need an error logged. Problems that are caused by users themselves (like over quota) are usually not temporary errors and they shouldn't have errors logged (since admin can't usually do anything about them anyway). > > Do you plan to support MTA queries about mailbox status (overquota,no > space left on device)? It would allow MTA to deploy its own policies > before sending "RCPT TO:" reply. "Socket map" protocol may be handy. > This is already supported for Postfix. Look for quota-status policy service. From frozenyoghurt2010 at gmail.com Tue Aug 12 09:20:03 2014 From: frozenyoghurt2010 at gmail.com (Michael Ludwig) Date: Tue, 12 Aug 2014 11:20:03 +0200 Subject: Test / can you see my mail?? Message-ID: Hi! I have to excuse my behave and I am really sorry. Yesterday I wrote two mails to this list and they never showed up in the online archives where I checked it. So that's the reason for this testmail. I hope this mail is readable to all of you? Cheers, Michael From gedalya at gedalya.net Tue Aug 12 09:38:35 2014 From: gedalya at gedalya.net (Gedalya) Date: Tue, 12 Aug 2014 05:38:35 -0400 Subject: Frequent error: unknown namespace Trash In-Reply-To: References: Message-ID: <53E9E09B.9030701@gedalya.net> Please read http://dovecot.org/mailinglists.html Please post your complete (dovecot -n output). On 08/11/2014 10:14 AM, Michael Ludwig wrote: > Hello to all of you! ;-) > > My first post on this list. I frequently get this error in dovecot.log > every time a user logs in and out: > > ============================= > imap(mailaddress): Error: quota: Unknown namespace: Trash > ============================= > > > When I change my conf.d/90-quota.conf from "Trash" to ".Trash" (as this is > the imap/maildir folder name, the error stays the same, but with the > ".Trash" in dovecot.log > This is what my 90-quoato.conf looks like: > > ============================= > plugin { > quota_rule = *:storage=2G > quota_rule2 = Trash:storage=+100M > quota_grace = 10%% > } > ============================= > > Where can I start my search, what is going wrong? > Just for letting you know: This mailserver is a fresh and clean setup. > I am using postfix/dovecotvimbadmin/mysql/spamassassin/clamsmtp. > > Cheers, > Michael From tolga at ozses.net Tue Aug 12 09:50:02 2014 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Tue, 12 Aug 2014 12:50:02 +0300 Subject: Test / can you see my mail?? In-Reply-To: References: Message-ID: I can see it. On 12 Aug 2014 12:20, "Michael Ludwig" wrote: > Hi! > > I have to excuse my behave and I am really sorry. > > Yesterday I wrote two mails to this list and they never showed up in the > online archives where I checked it. > > So that's the reason for this testmail. > > I hope this mail is readable to all of you? > > Cheers, > Michael > From jk at jkart.de Tue Aug 12 10:00:50 2014 From: jk at jkart.de (Jim Knuth) Date: Tue, 12 Aug 2014 12:00:50 +0200 Subject: Test / can you see my mail?? In-Reply-To: References: Message-ID: <53E9E5D2.9090509@jkart.de> am 12.08.14 11:50 schrieb Muzaffer Tolga Ozses : > I can see it. > On 12 Aug 2014 12:20, "Michael Ludwig" wrote: > >> Hi! >> >> I have to excuse my behave and I am really sorry. >> >> Yesterday I wrote two mails to this list and they never showed up in the >> online archives where I checked it. >> >> So that's the reason for this testmail. >> >> I hope this mail is readable to all of you? >> >> Cheers, >> Michael >> me too ;) And I see Tofu ? -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Die gr??te Kunst, die man im Leben lernen mu?, ist die Wiedergutmachung von Irrt?mern. Eines der Mittel ist, sie einzugestehen. (Hermann Melville) From slippyr4 at gmail.com Tue Aug 12 10:23:06 2014 From: slippyr4 at gmail.com (Jon Hulatt) Date: Tue, 12 Aug 2014 11:23:06 +0100 Subject: doveadm pw with SHA512-CRYPT won't roundtrip Message-ID: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> Hi, Not sure if this is a PBKAC or not:- root at ds3:/usr/share/postfixadmin# doveadm pw -s SHA512-CRYPT -p password {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ root at ds3:/usr/share/postfixadmin# doveadm pw -t {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ -p password doveadm(root): Fatal: reverse password verification check failed: Password mismatch root at ds3:/usr/share/postfixadmin# dovecot --version 2.2.9 It works ok with other schemes that i?ve tried (e.g. SSHA-512). Is this a bug, or my mistake? thanks Jon From cristiano.deana at gmail.com Tue Aug 12 10:25:13 2014 From: cristiano.deana at gmail.com (Cristiano Deana) Date: Tue, 12 Aug 2014 12:25:13 +0200 Subject: Test / can you see my mail?? In-Reply-To: References: Message-ID: On Tue, Aug 12, 2014 at 11:20 AM, Michael Ludwig wrote: HI, > I hope this mail is readable to all of you? I don't know. Do you? -- Cris, member of G.U.F.I Italian FreeBSD User Group http://www.gufi.org/ From fumiyas at osstech.jp Tue Aug 12 10:29:47 2014 From: fumiyas at osstech.jp (SATOH Fumiyasu) Date: Tue, 12 Aug 2014 19:29:47 +0900 Subject: doveadm pw with SHA512-CRYPT won't roundtrip In-Reply-To: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> References: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> Message-ID: <87ha1irof8.wl%fumiyas@osstech.jp> At Tue, 12 Aug 2014 11:23:06 +0100, Jon Hulatt wrote: > root at ds3:/usr/share/postfixadmin# doveadm pw -s SHA512-CRYPT -p password > {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ > > root at ds3:/usr/share/postfixadmin# doveadm pw -t {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ -p password > doveadm(root): Fatal: reverse password verification check failed: Password mismatch You must escape shell's meta characters, e.g., "{", "}", "$" and so on. # doveadm pw -t '{SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/' -p password {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ (verified) -- -- Name: SATOH Fumiyasu @ OSS Technology Corp. (fumiyas @ osstech co jp) -- Business Home: http://www.OSSTech.co.jp/ -- GitHub Home: https://GitHub.com/fumiyas/ -- PGP Fingerprint: BBE1 A1C9 525A 292E 6729 CDEC ADC2 9DCA 5E1C CBCA From fischerb at fischer-ing.de Tue Aug 12 10:29:27 2014 From: fischerb at fischer-ing.de (Fischer, Bernhard) Date: Tue, 12 Aug 2014 12:29:27 +0200 Subject: doveadm pw with SHA512-CRYPT won't roundtrip In-Reply-To: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> References: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> Message-ID: <25095ba0dead9acd2bcef81142030889@go.fischer.ing> At Tuesday, 12-08-2014 on 12:23 Jon Hulatt wrote: > Hi, > > Not sure if this is a PBKAC or not:- > > root at ds3:/usr/share/postfixadmin# doveadm pw -s SHA512-CRYPT -p password > {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ > > root at ds3:/usr/share/postfixadmin# doveadm pw -t {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ -p password > doveadm(root): Fatal: reverse password verification check failed: Password mismatch > > root at ds3:/usr/share/postfixadmin# dovecot --version > 2.2.9 > > > It works ok with other schemes that i?ve tried (e.g. SSHA-512). > > Is this a bug, or my mistake? > > thanks > > Jon The simple answer may be to quote the password. -bernhard From bourek at thinline.cz Tue Aug 12 10:40:36 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Tue, 12 Aug 2014 12:40:36 +0200 Subject: doveadm pw with SHA512-CRYPT won't roundtrip In-Reply-To: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> References: <1BBCDAAC-9E14-46C4-945C-9036CC3A7565@gmail.com> Message-ID: <53E9EF24.5060501@thinline.cz> On 12.8.2014 12:23, Jon Hulatt wrote: > Hi, > > Not sure if this is a PBKAC or not:- > > root at ds3:/usr/share/postfixadmin# doveadm pw -s SHA512-CRYPT -p password > {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ > > root at ds3:/usr/share/postfixadmin# doveadm pw -t {SHA512-CRYPT}$6$aUgGXP0UshkMj7hY$9JV4yMRsjIe/98CzmglYrMjf.9NJ.FXzxcLE9B0v3doCRUWo2wRncc6hg6VCs0DCUHQbeC/bRDZdGCge/nB/h/ -p password > doveadm(root): Fatal: reverse password verification check failed: Password mismatch > > root at ds3:/usr/share/postfixadmin# dovecot --version > 2.2.9 > > > It works ok with other schemes that i?ve tried (e.g. SSHA-512). > > Is this a bug, or my mistake? > > thanks > > Jon > Tried quoting the hash in 'single quotes'? My guess is your shell is interpreting the $ character and trying to replace part of the hash with some (undefined) variable From Jochen.Bern at LINworks.de Tue Aug 12 11:04:55 2014 From: Jochen.Bern at LINworks.de (Jochen Bern) Date: Tue, 12 Aug 2014 13:04:55 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E93AE1.2050607@thinline.cz> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> <53E93AE1.2050607@thinline.cz> Message-ID: <53E9F4D7.8050801@LINworks.de> On -10.01.-28163 20:59, Jiri Bourek wrote: > On 10.8.2014 06:18, Will Yardley wrote: >> Depends on the environment; in many cases, the admin could, or may even >> be expected to, raise the quota. > > If you're expected to raise the quota in case it's exceeded, don't set > it in the first place, there is no point to it. Or - if you really want > that exercise in futility - use quota_warning and raise it automatically. Where exactly did you read "automatically"? Users need reminders not to be disk hogs. Managers want to be asked before company resources get allocated. Sysadmins may want to verify that it's actually the *user* using the disk space and not some software or e-mail problem. Quotas and their getting exceeded provide a hook for all these (non computer-only) procedures, even if the quota eventually *does* get raised for all cases of proper need. > As for running out of disk space completely - well I assume you're using > some kind of monitoring service, so just add a check for free disk space > and you're done. There's really no need to cover this in Dovecot. dovecot already has code to detect this situation because catching potential error conditions is what well-written software *does*. The request at hand is about how exactly it should propagate the error back to its client. Regards, J. Bern -- *NEU* - NEC IT-Infrastruktur-Produkte im : Server--Storage--Virtualisierung--Management SW--Passion for Performance Jochen Bern, Systemingenieur --- LINworks GmbH Postfach 100121, 64201 Darmstadt | Robert-Koch-Str. 9, 64331 Weiterstadt PGP (1024D/4096g) FP = D18B 41B1 16C0 11BA 7F8C DCF7 E1D5 FAF4 444E 1C27 Tel. +49 6151 9067-231, Zentr. -0, Fax -299 - Amtsg. Darmstadt HRB 85202 Unternehmenssitz Weiterstadt, Gesch?ftsf?hrer Metin Dogan, Oliver Michel From bourek at thinline.cz Tue Aug 12 11:14:47 2014 From: bourek at thinline.cz (Jiri Bourek) Date: Tue, 12 Aug 2014 13:14:47 +0200 Subject: Defer email via LMTP when there is 'no space left on device' instead of rejecting it In-Reply-To: <53E9F4D7.8050801@LINworks.de> References: <53CE7F09.9070100@netcologne.de> <53CE83C2.80708@thelounge.net> <53CE98AA.9060508@moritz.augsburger.name> <53CF7FC5.2040301@netcologne.de> <53E0C3DB.6010108@netcologne.de> <8974DA6C-5A09-4155-B42D-F1727FD14891@iki.fi> <20140810041823.GA79430@aura.veggiechinese.net> <53E93AE1.2050607@thinline.cz> <53E9F4D7.8050801@LINworks.de> Message-ID: <53E9F727.8060502@thinline.cz> On 12.8.2014 13:04, Jochen Bern wrote: > On -10.01.-28163 20:59, Jiri Bourek wrote: >> On 10.8.2014 06:18, Will Yardley wrote: >>> Depends on the environment; in many cases, the admin could, or may even >>> be expected to, raise the quota. >> >> If you're expected to raise the quota in case it's exceeded, don't set >> it in the first place, there is no point to it. Or - if you really want >> that exercise in futility - use quota_warning and raise it automatically. > > Where exactly did you read "automatically"? > > Users need reminders not to be disk hogs. Managers want to be asked > before company resources get allocated. Sysadmins may want to verify > that it's actually the *user* using the disk space and not some software > or e-mail problem. Quotas and their getting exceeded provide a hook for > all these (non computer-only) procedures, even if the quota eventually > *does* get raised for all cases of proper need. You can do all of those things as a reaction to quota_warning e-mail sent to you _before_ the account exceeds its quota. No need to hit the error path by actually exceeding the quota. From luciano at vespaperitivo.it Tue Aug 12 12:24:10 2014 From: luciano at vespaperitivo.it (Luciano Mannucci) Date: Tue, 12 Aug 2014 14:24:10 +0200 Subject: Test / can you see my mail?? In-Reply-To: References: Message-ID: <20140812122603.3C6DF235ED@talvi.dovecot.org> On Tue, 12 Aug 2014 11:20:03 +0200 Michael Ludwig wrote: > I hope this mail is readable to all of you? Nope, I can't... :-) l. -- /"\ /Via A. Salaino, 7 - 20144 Milano (Italy) \ / ASCII RIBBON CAMPAIGN / PHONE : +39 2 485781 FAX: +39 2 48578250 X AGAINST HTML MAIL / E-MAIL: posthamster at sublink.sublink.ORG / \ AND POSTINGS / WWW: http://www.lesassaie.IT/ From uothrawn at yahoo.com Tue Aug 12 13:35:48 2014 From: uothrawn at yahoo.com (G H) Date: Tue, 12 Aug 2014 06:35:48 -0700 Subject: BUG: Solr FTS engine appends domain name to index records Message-ID: <1407850548.28043.YahooMailNeo@web161403.mail.bf1.yahoo.com> The Solr FTS engine appends domain names to the "user" and "id" fields inside Solr when indexing. However, when an IMAP search is performed, the domain name is not appended, resulting in 0 results, 100% of the time. You can reproduce this by setting up dovecot to have user names like?"admin", "james", etc (as opposed to admin at example.com). Then import some mail (ie: "dovecot-lda -d admin < /tmp/mymail.txt"). Now attempt to search for that mail and it will not be returned as a result. You can see in the Solr GUI (http://xxxx:8983/solr/) that record is indexed as: ????? { ??????? "uid": 1, ??????? "box": "7dff0b0d4f84df53f25300003d823887", ??????? "user": "admin at example.com", ??????? "id": "1/7dff0b0d4f84df53f25300003d823887/admin at example.com", ??????? "_version_": 1475511406809841700 ????? } However, the IMAP search is sent to Solr performed with user:"admin" instead of user:"admin at example.com". Either the Solr FTS plugin needs to not append the domain name or the search does need to do so. From Philipp.Faeustlin at uni-hohenheim.de Tue Aug 12 13:52:29 2014 From: Philipp.Faeustlin at uni-hohenheim.de (Philipp Faeustlin) Date: Tue, 12 Aug 2014 15:52:29 +0200 Subject: Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/' Message-ID: <53EA1C1D.4020805@uni-hohenheim.de> Hello List, with dovecot version 2.2.13.9, I actually have a problem using lazy_expunge and acl together. Shared/Public Folders work fine with lazy_expunge disabled. If lazy_expunge is enabled and userA got a shared folder form userB, then there is the error shown in subject. My test case to reproduce the error. doveadm mailbox list -u userA at domain --> Output Fine. doveadm acl set -u userB at domain INBOX user=userA at domain lookup read write write-seen write-deleted insert expunge create --> Fine. doveadm mailbox list -u userA at domain --> Some output and then: --> doveadm(userA at domain): Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/' I think this is a bug in dovecot, can someone of the dovecot developers or someone else confirm this? If it is not please show me the fault in my configuration. In May 2014 there was a thread with the same issue on this mailing list, but with no solution. http://dovecot.org/pipermail/dovecot/2014-May/096355.html Best regards Philipp F?ustlin --------------------------------------------- # 2.2.13.9 (19befd72d9d8): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 Scientific Linux release 6.5 (Carbon) auth_debug = yes auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %Ln auth_verbose = yes listen = * mail_debug = yes mail_gid = vmail mail_location = maildir:~/maildir mail_plugins = " acl lazy_expunge mail_log notify zlib notify replication" mail_shared_explicit_inbox = yes mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace { hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/maildir:INDEX=%h/Shared/%%u:CONTROL=%h/Shared/%%u prefix = Shared/%%u/ separator = / subscriptions = yes type = shared } namespace { location = maildir:/vmail/%d/public:INDEXPVT=~/index/%d/public prefix = Groups/ separator = / subscriptions = no type = public } namespace { hidden = yes list = no location = maildir:~/expunged prefix = .EXPUNGED/ separator = / subscriptions = no type = private } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile:/etc/dovecot/global-acls acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db lazy_expunge = .EXPUNGED/ lazy_expunge_only_last_instance = yes mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size mail_replica = tcps:mailhost1.rz.uni-hohenheim.de:12345 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve zlib_save = gz zlib_save_level = 6 } protocols = imap pop3 lmtp sieve replication_dsync_parameters = -d -l 30 -U service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service doveadm { inet_listener { port = 12345 ssl = yes } } service imap-login { process_min_avail = 16 service_count = 0 } service imap-postlogin { executable = script-login /usr/local/bin/postlogin.sh user = $default_internal_user } service imap { executable = imap imap-postlogin process_limit = 8192 } service lmtp { unix_listener /var/spool/postfix/private/lmtp-dovecot { group = postfix user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { process_min_avail = 16 service_count = 0 } service pop3 { process_limit = 8192 } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { group = vmail mode = 0660 } } ssl_cert = Hello, since some weeks I'm playing with namespaces. But I still did nod found a solution for all faces of different problems. Current issue: Public namespace. I have users inbox as follow: mail_home = /data/mail/%Ln/ mail_location = maildir:~/Maildir:INDEX=~/.dovecot.index namespace { inbox = yes separator = / prefix = INBOX/ } Public namespace: namespace { location = maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/ prefix = PUBLIC/ separator = / type = public } that way only one subscription file is possible. Any use could subscribe/unsubscribe any folder and that match all users. If I add "subscription=no" the parent namespace' subscription is used. In fact there is no parent namespace an no subscription management is possible at all. On the other side I tried to change the PUBLIC namespace prefix to "INBOX/PUBLIC" but now the namespace is *below* the inbox. That's also not what I want. How do other users organise the namespaces INBOX, PUBLIC, SHARED and VIRTUAL (via virtual plugin) ? Thanks Andreas From django at nausch.org Tue Aug 12 17:25:13 2014 From: django at nausch.org (Django) Date: Tue, 12 Aug 2014 19:25:13 +0200 Subject: Test / can you see my mail?? In-Reply-To: <20140812122603.3C6DF235ED@talvi.dovecot.org> References: <20140812122603.3C6DF235ED@talvi.dovecot.org> Message-ID: <2301620.IFtTtAb3ur@pml010048.intra.nausch.org> HI! On Tuesday 12 August 2014 14:24:10 Luciano Mannucci wrote: > Nope, I can't... :-) me2! cu! From frozenyoghurt2010 at gmail.com Tue Aug 12 20:06:50 2014 From: frozenyoghurt2010 at gmail.com (Michael Ludwig) Date: Tue, 12 Aug 2014 22:06:50 +0200 Subject: Frequent error: unknown namespace Trash In-Reply-To: <53EA4FE2.2000608@gedalya.net> References: <53E9E09B.9030701@gedalya.net> <53EA4FE2.2000608@gedalya.net> Message-ID: Hi Gedalya, I am very sorry for not having replied on-list. That was not my intention! I am not used to this behaviour of mailing lists, normally there is a reply-to field which points to the mailing list mail address. So do I have to use MAILBOX.Trash in 90-quota.conf ?? Or INBOX.Trash ? Cheers, Michael 2014-08-12 19:33 GMT+02:00 Gedalya : > On 08/12/2014 09:28 AM, Michael Ludwig wrote: > >> prefix = INBOX. >> separator = . >> > Maybe that's it? It's therefore MAILBOX.Trash, not just Trash. > > You're supposed to reply on-list! Don't reply to gedalya at gedalya.net, > reply to dovecot at dovecot.org. If you want you can also cc the message > author, some people do that. > > From gedalya at gedalya.net Tue Aug 12 20:20:05 2014 From: gedalya at gedalya.net (Gedalya) Date: Tue, 12 Aug 2014 16:20:05 -0400 Subject: Frequent error: unknown namespace Trash In-Reply-To: References: <53E9E09B.9030701@gedalya.net> <53EA4FE2.2000608@gedalya.net> Message-ID: <53EA76F5.309@gedalya.net> Sorry, my typo!! I guess it's INBOX. My apologies :-) Try it out and let us know. On 08/12/2014 04:06 PM, Michael Ludwig wrote: > Hi Gedalya, > > I am very sorry for not having replied on-list. That was not my intention! > I am not used to this behaviour of mailing lists, normally there is a > reply-to field which points to the mailing list mail address. > > So do I have to use MAILBOX.Trash in 90-quota.conf ?? > Or INBOX.Trash ? > > Cheers, > Michael > > > > 2014-08-12 19:33 GMT+02:00 Gedalya : > >> On 08/12/2014 09:28 AM, Michael Ludwig wrote: >> >>> prefix = INBOX. >>> separator = . >>> >> Maybe that's it? It's therefore MAILBOX.Trash, not just Trash. >> >> You're supposed to reply on-list! Don't reply to gedalya at gedalya.net, >> reply to dovecot at dovecot.org. If you want you can also cc the message >> author, some people do that. >> >> From skdovecot at smail.inf.fh-brs.de Wed Aug 13 06:58:54 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 13 Aug 2014 08:58:54 +0200 (CEST) Subject: namspace management In-Reply-To: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> References: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 12 Aug 2014, A. Schulze wrote: > Current issue: Public namespace. > I have users inbox as follow: > > mail_home = /data/mail/%Ln/ > mail_location = maildir:~/Maildir:INDEX=~/.dovecot.index > namespace { > inbox = yes > separator = / > prefix = INBOX/ > } > > Public namespace: > namespace { > location = > maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/ > prefix = PUBLIC/ > separator = / > type = public > } > > that way only one subscription file is possible. Any use could > subscribe/unsubscribe any folder > and that match all users. Did you tried: location = maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/:CONTROL=~/.dovecot.index.public/ - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+sMrnz1H7kL/d9rAQLKLggAhnecLGjpAEPDRlUOlne+TArb8f0BVFuI 4Yi1I9t0LxTaaL89EV/ipnT36sqGXVYtlSYxXKAULpKHvSdhwg+4Em2sf5f86j8r tRnljbx+Wjh1ptLX46HsyByjhnTNQTJJQCP3IpU64xkUyzeWclZzqI3KrvX2Mkuj FZ8nZZin/p+Hz0YnFXML9ASARCx1cwYNI1BrbKI+foWXky0VLxK802jPpfZRplQw rHg8JdpsvzxV2R7IEu4986ovAtJ4LC2zx8u6TSQURFgYOC2SVI1SJYJt29wdCVp7 RP82elLsqGg5xaKpVZuU/7ueJg7ikGiE1qxpq6gFr9ZwjAZkYEjjAg== =K7ra -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed Aug 13 07:01:32 2014 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 13 Aug 2014 09:01:32 +0200 (CEST) Subject: Frequent error: unknown namespace Trash In-Reply-To: References: <53E9E09B.9030701@gedalya.net> <53EA4FE2.2000608@gedalya.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 12 Aug 2014, Michael Ludwig wrote: > So do I have to use MAILBOX.Trash in 90-quota.conf ?? > Or INBOX.Trash ? > > 2014-08-12 19:33 GMT+02:00 Gedalya : >> On 08/12/2014 09:28 AM, Michael Ludwig wrote: >> >>> prefix = INBOX. >>> separator = . >>> >> Maybe that's it? It's therefore MAILBOX.Trash, not just Trash. I would try: INBOX.Trash - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEVAwUBU+sNTHz1H7kL/d9rAQJSOggAs1p7A46LfGDHqA68Ua5/FsDjkhOeTmir XpXNcPH+hyphDnrJXhJO/BIWG2ajPVRb6jGfRbSVFwYua5E5f/x2WDnzHcnQ6dEb C0np/YruneoiLucOaAp6jqWoF7G8hLthqG+GXNubxkfTx30J0b0prRfZ0RH+Hd7E WQL4JAjcUtcFhNO7tiu2iSgAp6WLwMGzZcOTHDdlBuu34AtDsTGNWBMOsPzek7l5 Rb9NT+EcBDXVG+jQW8YdGK301N0KVVxnbR+LVawn4We9a9+wcGCIOhoxiZs3kvrp P2425dZMrr5Iski4l5s8Vc2Tg+pGQV8m6/x8sNZ8s24tkTXYx/IhoQ== =FjlO -----END PGP SIGNATURE----- From bautista.florent at gmail.com Wed Aug 13 08:24:42 2014 From: bautista.florent at gmail.com (Florent Bautista) Date: Wed, 13 Aug 2014 10:24:42 +0200 Subject: LMTP re-reading all messages Message-ID: Hi all, I have a little problem with Dovecot 2.1.17 and MailDir accounts. I was using a shared file system (MooseFS), and all was pretty stable. Now I'm switching to CephFS, and Dovecot LMTP have problem to deliver mails. Very often, on large accounts, LMTP processes are running for a few hours before delivering the message. When I strace it, I can see that they are reading (open and stat operations) ALL the messages of the account. I was thinking to an index problem, but I think that after hundreds of delivered mails, indexes are rebuilded and problem should be solved. But not. The same way, If user connects to IMAP, a lot of IMAP processes get stuck on "getquotaroot" command (multiple processes for a single account). The previous FS didn't support mmap, so I disabled it. The new one supports it, so I enabled it, but it does not solve the problem. Do you have an idea of why Dovecot (LMTP or IMAP) spend a lot of time on large accounts ? Thank you a lot. Florent From admin at awib.it Wed Aug 13 19:32:37 2014 From: admin at awib.it (admin) Date: Wed, 13 Aug 2014 21:32:37 +0200 Subject: (Again) Some trouble with dovecot 2.1(.7) and SASL Message-ID: <01540C4F-42CF-44E0-8016-E04645F0BA58@awib.it> The parentheses in the subject are due to referral to this discussion on the mailing list: http://dovecot.org/list/dovecot/2012-February/063851.html At first I thought I found the problem, but then realized this was 2012 and the suggested patch already is applied in dovecot. I also have segault messages in my kernel log; not too many, but it was enough to dig for it: > Wed Jun 25 08:29:05 2014 auth[17590]: segfault at 10 ip 00007f02ad5e8e56 sp 00007fff932f9250 error 4 in auth[7f02ad5d3000+40000] > Mon Jun 30 16:11:41 2014 auth[12501]: segfault at 10 ip 00007fc6a1fefe56 sp 00007fff276e6ea0 error 4 in auth[7fc6a1fda000+40000] > Sat Jul 5 13:11:32 2014 auth[15603]: segfault at 10 ip 00007f6447a6be56 sp 00007fff09c48020 error 4 in auth[7f6447a56000+40000] > Tue Jul 8 05:08:21 2014 auth[30071]: segfault at 10 ip 00007ff22656ae56 sp 00007fff26b017f0 error 4 in auth[7ff226555000+40000] > Mon Jul 21 14:25:28 2014 auth[30918]: segfault at 10 ip 00007fe1fc817e56 sp 00007fff7e654f20 error 4 in auth[7fe1fc802000+40000] <--- > Wed Aug 13 18:56:31 2014 auth[27593]: segfault at 10 ip 00007ffbf8ce8e56 sp 00007fffe0157f70 error 4 in auth[7ffbf8cd3000+40000] > Wed Aug 13 18:57:59 2014 auth[24264]: segfault at 10 ip 00007f2df31bde56 sp 00007fff094263c0 error 4 in auth[7f2df31a8000+40000] The last 2 entries are a result of me being able to reproduce the problem, the 3rd last is the only one I was able to track down (due to log rotation) I tracked the problem down to a contact form of a website of one of my customers. whenever used, the segfault gets triggered. Nevertheless, the message always gets delivered. So we are talking about the marked line; the corresponding log entries from mail.info: > Jul 21 14:25:29 mail postfix/smtpd[27590]: connect from unknown[] > Jul 21 14:25:29 mail dovecot: auth: Fatal: master: service(auth): child 30918 killed with signal 11 (core dumps disabled) > Jul 21 14:25:29 mail postfix/smtpd[27590]: warning: unknown[]: SASL DIGEST-MD5 authentication failed: Connection lost to authentication server > Jul 21 14:25:29 mail dovecot: auth-worker(27595): mysql(127.0.0.1): Connected to database mail > Jul 21 14:25:29 mail postfix/smtpd[27590]: D4D8E44752: client=unknown[], sasl_method=CRAM-MD5, sasl_username= > Jul 21 14:25:29 mail postfix/cleanup[27598]: D4D8E44752: message-id= > Jul 21 14:25:29 mail postfix/qmgr[13349]: D4D8E44752: from=, size=2680, nrcpt=1 (queue active) > Jul 21 14:25:29 mail postfix/smtpd[27590]: disconnect from unknown[] > Jul 21 14:25:29 mail dovecot: lmtp(27600): Connect from local > Jul 21 14:25:30 mail postfix/lmtp[27599]: D4D8E44752: to=, orig_to=, relay=[private/dovecot-lmtp], delay=0.19, delays=0.07/0.04/0.04/0.04, dsn=2.0.0, status=sent (250 2.0.0 MrY+OdkizVPQawAAMrkmwA Saved) > Jul 21 14:25:30 mail postfix/qmgr[13349]: D4D8E44752: removed > Jul 21 14:25:30 mail dovecot: lmtp(27600, info at doma.in): MrY+OdkizVPQawAAMrkmwA: msgid=: saved mail to INBOX > Jul 21 14:25:30 mail dovecot: lmtp(27600): Disconnect from local: Client quit (in reset) contact at doma.in is a virtual address which gets forwarded to info at doma.in (/etc/postfix/virtual). The used CMS is PyroCMS (v2.2.3) - the version is not up2date, but nevertheless the CMS might be the one to blame, a service should not segfault. Some packages are to be updated, as well as the CMS; but I am willing to wait to provide information, which leads to the problem, as I cannot switch between different versions on my productive system for the sake of gaining debug information. I hope that someone has any kind of clue and is willing to resolve that issue with me, asking me for the necessary information to do so. Thanks in advance! -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5820 bytes Desc: not available URL: From user7743 at gmail.com Thu Aug 14 12:11:39 2014 From: user7743 at gmail.com (user7743) Date: Thu, 14 Aug 2014 14:11:39 +0200 Subject: Quota Help Message-ID: <1408018299.16623.23.camel@neon.contriveit.local> Hi all, First post here so hopefully I get all the required information correct. I am running a mail server on Debian, its consists of Postfix (2.9.6-2), Dovecot (2.1.7) and MySQL (5.5). Basic stuff is all working find so I decided to try and implement quotas. This is working to an extent but not properly and I am a little lost so looking for some help. In my configuration the default mailbox quota is set to "*:storage=100M". When I create a mailbox with 250M specified in the database, dovecot is applying the default quota rather than that specified in the database. So in the database I have a test user with the following entry password,email,quota_kb [hash], test at example.org, 25000 If I turn debugging on for mail and auth I see the following logged when the user logs in (logs pasted at the end in a bid to maintain some readability) If I check the quota with doveadm I get the limit back of 102400 which is the default mailbox size rather than that from the database. doveadm quota get -u test at example.org Quota, name, Type, Value, Limit, % User quota, STORAGE, 19139, 102400, 18 User quota, MESSAGE, 567, - I can't quite figure out where I am going wrong in terms of getting the database value to supersede the default value set in 90-quota.conf. I did read one thing in the quota documentation on the dovecot website (http://wiki2.dovecot.org/Quota/Configuration) that I didn't get and this might be where I am going wrong but can't check. It says "Check that dovecot-lda is called with -d parameter." Where would this be configured and where could I check? Below are some other inputs - hopefully they help. ### Debug logs at login Aug 14 14:06:24: auth: Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 14 14:06:24: auth: Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Aug 14 14:06:24: auth: Debug: auth client connected (pid=8090) Aug 14 14:06:24: auth: Debug: auth client connected (pid=8093) Aug 14 14:06:24: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=2QDLu5UAKAAuBSdf#011lip=99.99.99.99#011rip=55.55.55.55#011lport=993#011rport=50216 Aug 14 14:06:24: auth: Debug: client out: CONT#0111#011 Aug 14 14:06:24: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=ZBnLu5UAKQAuBSdf#011lip=99.99.99.99#011rip=55.55.55.55#011lport=993#011rport=50217 Aug 14 14:06:24: auth: Debug: client out: CONT#0111#011 Aug 14 14:06:24: auth: Debug: client in: CONT Aug 14 14:06:24: auth-worker(8095): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Aug 14 14:06:24: auth-worker(8095): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Aug 14 14:06:24: auth-worker(8095): mysql(127.0.0.1): Connected to database mailserver Aug 14 14:06:24: auth-worker(8095): Debug: sql(test.example.org,55.55.55.55): query: SELECT email as user, password FROM users WHERE email='test.example.org'; Aug 14 14:06:24: auth: Debug: client out: OK#0111#011user=test.example.org Aug 14 14:06:24: auth: Debug: master in: REQUEST#0111720057857#0118090#0111#0114bf45b372441751900d7d967578b0e03 Aug 14 14:06:24: auth-worker(8095): Debug: sql(test.example.org,55.55.55.55): SELECT CONCAT('/var/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, CONCAT('maildir:storage=+',quota_kb) AS quota_rule FROM users WHERE email='test.example.org'; Aug 14 14:06:24: auth: Debug: master out: USER#0111720057857#011test.example.org#011home=/var/vmail/contriveit.com/test#011uid=5000#011gid=5000#011quota_rule=maildir:storage=+250000 Aug 14 14:06:24: imap-login: Login: user=, method=PLAIN, rip=55.55.55.55, lip=99.99.99.99, mpid=8097, TLS, session=<2QDLu5UAKAAuBSdf> Aug 14 14:06:24: auth: Debug: client in: CONT Aug 14 14:06:24: auth-worker(8095): Debug: sql(test.example.org,55.55.55.55): query: SELECT email as user, password FROM users WHERE email='test.example.org'; Aug 14 14:06:24: auth: Debug: client out: OK#0111#011user=test.example.org Aug 14 14:06:24: imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Aug 14 14:06:24: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Aug 14 14:06:24: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Aug 14 14:06:24: imap: Debug: Added userdb setting: plugin/quota_rule=maildir:storage=+250000 Aug 14 14:06:24: imap(test.example.org): Debug: Effective uid=5000, gid=5000, home=/var/vmail/contriveit.com/test Aug 14 14:06:24: imap(test.example.org): Debug: Quota root: name=User quota backend=maildir args= Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User quota mailbox=maildir bytes=+256000000 messages=0 Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User quota mailbox=Trash bytes=+10485760 messages=0 Aug 14 14:06:24: imap(test.example.org): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/vmail/contriveit.com/test/Maildir Aug 14 14:06:24: auth: Debug: master in: REQUEST#011635830273#0118093#0111#011316aed6aaba731b5f79c78d2664a7c99 Aug 14 14:06:24: imap(test.example.org): Debug: maildir++: root=/var/vmail/contriveit.com/test/Maildir, index=, control=, inbox=/var/vmail/contriveit.com/test/Maildir, alt= Aug 14 14:06:24: auth-worker(8095): Debug: sql(test.example.org,55.55.55.55): SELECT CONCAT('/var/vmail/',CONCAT(SUBSTRING_INDEX(email,'@',-1),'/',SUBSTRING_INDEX(email,'@',1))) AS home, 5000 AS uid, 5000 AS gid, CONCAT('maildir:storage=+',quota_kb) AS quota_rule FROM users WHERE email='test.example.org'; Aug 14 14:06:24: auth: Debug: master out: USER#011635830273#011test.example.org#011home=/var/vmail/contriveit.com/test#011uid=5000#011gid=5000#011quota_rule=maildir:storage=+250000 Aug 14 14:06:24: imap-login: Login: user=, method=PLAIN, rip=55.55.55.55, lip=99.99.99.99, mpid=8098, TLS, session= Aug 14 14:06:24: imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Aug 14 14:06:24: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Aug 14 14:06:24: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Aug 14 14:06:24: imap: Debug: Added userdb setting: plugin/quota_rule=maildir:storage=+250000 Aug 14 14:06:24: imap(test.example.org): Debug: Effective uid=5000, gid=5000, home=/var/vmail/contriveit.com/test Aug 14 14:06:24: imap(test.example.org): Debug: Quota root: name=User quota backend=maildir args= Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User quota mailbox=maildir bytes=+256000000 messages=0 Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User quota mailbox=Trash bytes=+10485760 messages=0 Aug 14 14:06:24: imap(test.example.org): Debug: Namespace inbox: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/var/vmail/contriveit.com/test/Maildir Aug 14 14:06:24: imap(test.example.org): Debug: maildir++: root=/var/vmail/contriveit.com/test/Maildir, index=, control=, inbox=/var/vmail/contriveit.com/test/Maildir, alt= ### doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab084.17 x86_64 Debian 7.6 simfs auth_debug = yes auth_mechanisms = plain login dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } mail_debug = yes mail_location = maildir:/var/vmail/%d/%n/Maildir mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = create special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=100M quota_rule2 = Trash:storage=+10M sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at example.org protocols = " imap sieve" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl_cert = References: <1408018299.16623.23.camel@neon.contriveit.local> Message-ID: <53ECAB1C.5020700@gedalya.net> On 08/14/2014 08:11 AM, user7743 wrote: > CONCAT('maildir:storage=+',quota_kb) AS quota_rule What you're doing is applying this extra storage to a mailbox called maildir. > Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User > quota mailbox=maildir bytes=+256000000 messages=0 You probably want to change this to CONCAT('*:storage=+',quota_kb) or just = rather than =+ From user7743 at gmail.com Thu Aug 14 12:36:59 2014 From: user7743 at gmail.com (user7743) Date: Thu, 14 Aug 2014 14:36:59 +0200 Subject: Quota Help In-Reply-To: <53ECAB1C.5020700@gedalya.net> References: <1408018299.16623.23.camel@neon.contriveit.local> <53ECAB1C.5020700@gedalya.net> Message-ID: <53ECAD6B.6000109@gmail.com> So simple yet so effective. Many thanks, that seems to have done the job. On 14/08/14 14:27, Gedalya wrote: > > On 08/14/2014 08:11 AM, user7743 wrote: >> CONCAT('maildir:storage=+',quota_kb) AS quota_rule > What you're doing is applying this extra storage to a mailbox called > maildir. >> Aug 14 14:06:24: imap(test.example.org): Debug: Quota rule: root=User >> quota mailbox=maildir bytes=+256000000 messages=0 > > You probably want to change this to CONCAT('*:storage=+',quota_kb) or > just = rather than =+ > From uothrawn at yahoo.com Thu Aug 14 13:02:39 2014 From: uothrawn at yahoo.com (G H) Date: Thu, 14 Aug 2014 06:02:39 -0700 Subject: Segfault with indexer-worker using Tika/Solr Message-ID: <1408021359.14038.YahooMailNeo@web161405.mail.bf1.yahoo.com> Using Dovecot 2.2.13 with Tika 1.5 on Centos 5 64bit Tika is throwing an exception for some reason, which seems to be causing Dovecot's indexer to segfault. I'm using mdbox/gzipped compressed mail and private indexes for the shared folder. I'm importing mail via formail/dovecot-lda: formail -s /usr/libexec/dovecot/dovecot-lda -d admin -m Public/TestSubFolder? < /tmp/Inbox.mbox dovecot -n and the GDB backtrace are attached. # find /var/mail/ -maxdepth 2 /var/mail/_public /var/mail/_public/dovecot.mailbox.log /var/mail/_public/dovecot-uidvalidity /var/mail/_public/dovecot-uidvalidity.53ecade1 /var/mail/_public/storage /var/mail/_public/subscriptions /var/mail/_public/mailboxes /var/mail/admin /var/mail/admin/dovecot-uidvalidity.53ecade2 /var/mail/admin/dovecot.mailbox.log /var/mail/admin/public /var/mail/admin/dovecot-uidvalidity /var/mail/admin/storage /var/mail/admin/subscriptions /var/mail/admin/mailboxes -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: backtrace.txt URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: doveconf.txt URL: From uothrawn at yahoo.com Thu Aug 14 13:26:03 2014 From: uothrawn at yahoo.com (G H) Date: Thu, 14 Aug 2014 06:26:03 -0700 Subject: BUG: Solr FTS engine appends domain name to index records In-Reply-To: <1407850548.28043.YahooMailNeo@web161403.mail.bf1.yahoo.com> References: <1407850548.28043.YahooMailNeo@web161403.mail.bf1.yahoo.com> Message-ID: <1408022763.13452.YahooMailNeo@web161403.mail.bf1.yahoo.com> Some more information: Using the doveadm search command, the domain name is included in the user key in the search sent to Solr. IMAP appears to be the odd man out here. ----- Original Message ----- From: G H To: "dovecot at dovecot.org" Cc: Sent: Tuesday, August 12, 2014 9:35 AM Subject: BUG: Solr FTS engine appends domain name to index records The Solr FTS engine appends domain names to the "user" and "id" fields inside Solr when indexing. However, when an IMAP search is performed, the domain name is not appended, resulting in 0 results, 100% of the time. You can reproduce this by setting up dovecot to have user names like?"admin", "james", etc (as opposed to admin at example.com). Then import some mail (ie: "dovecot-lda -d admin < /tmp/mymail.txt"). Now attempt to search for that mail and it will not be returned as a result. You can see in the Solr GUI (http://xxxx:8983/solr/) that record is indexed as: ????? { ??????? "uid": 1, ??????? "box": "7dff0b0d4f84df53f25300003d823887", ??????? "user": "admin at example.com", ??????? "id": "1/7dff0b0d4f84df53f25300003d823887/admin at example.com", ??????? "_version_": 1475511406809841700 ????? } However, the IMAP search is sent to Solr performed with user:"admin" instead of user:"admin at example.com". Either the Solr FTS plugin needs to not append the domain name or the search does need to do so. From alex.handle at gmail.com Thu Aug 14 14:11:12 2014 From: alex.handle at gmail.com (Alex Ha) Date: Thu, 14 Aug 2014 16:11:12 +0200 Subject: panic: dsync and imapc Message-ID: Hi Dovecot list! I try to sync from a remote imap server to my local dovecot server via the dsync tool with imapc. Here is my command: # /usr/local/dovecot/bin/dsync -c /usr/local/dovecot/etc/dovecot/dovecot.conf -D -o imapc_user= test at example.com -o imapc_password=pwd backup -R -u test at example.com imapc: Here the error message: dsync(test at example.com): Panic: file quota-maildir.c: line 827: unreached Please see the attached debug.txt for a full debug messages and stacktrace. System info OS: Ubuntu 14.04 Server 64 Bit Dovecot: 2.2.13 Thanks, Alex -------------- next part -------------- # 2.2.13: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 3.13.0-27-generic x86_64 Ubuntu 14.04 LTS auth_default_realm = example.com auth_mechanisms = plain login auth_verbose = yes default_client_limit = 10000 default_process_limit = 2000 disable_plaintext_auth = no imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags imapc_features = rfc822.size fetch-headers imapc_host = external-imap.example.com imapc_list_prefix = INBOX imapc_user = %u mail_location = maildir:~/Maildir mail_plugins = " quota" mail_prefetch_count = 20 namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam { auto = subscribe special_use = \Junk } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_warning = storage=90%% quota-warning 90 %u } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_save_uidl = yes postmaster_address = postmaster at example.com service anvil { client_limit = 12560 } service auth { inet_listener { port = 12345 } unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl_ca = /usr/local/dovecot/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f514ec0e30a] -> /usr/local/dovecot/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f514ebc8a5e] -> /usr/local/dovecot/lib/dovecot/lib10_quota_plugin.so(+0x9888) [0x7f514e3d4888] -> /usr/local/dovecot/lib/dovecot/lib10_quota_plugin.so(quota_add_user_namespace+0x143) [0x7f514e3d08f3] -> /usr/local/dovecot/lib/dovecot/lib10_quota_plugin.so(quota_mailbox_list_created+0x19a) [0x7f514e3d710a] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(hook_mailbox_list_created+0x61) [0x7f514eed0f51] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(mailbox_list_create+0x345) [0x7f514eed43c5] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_storage_create_full+0x44d) [0x7f514eecd36d] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_storage_create+0x1e) [0x7f514eecd62e] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(+0x4c8f8) [0x7f514eec78f8] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_namespaces_init+0xef) [0x7f514eec859f] -> /usr/local/dovecot/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_next+0x54d) [0x7f514eedac5d] -> /usr/local/dovecot/bin/dsync() [0x429501] -> /usr/local/dovecot/bin/dsync() [0x412123] -> /usr/local/dovecot/bin/dsync(doveadm_mail_try_run+0x257) [0x412dd7] -> /usr/local/dovecot/bin/dsync(main+0x3d8) [0x411d18] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f514e801ec5] -> /usr/local/dovecot/bin/dsync() [0x411f2f] Aborted (core dumped) From listas at adminlinux.com.br Thu Aug 14 19:28:02 2014 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Thu, 14 Aug 2014 16:28:02 -0300 Subject: Trying Dovecot Replication with dsync Message-ID: <53ED0DC2.5080202@adminlinux.com.br> Hi, I have a failover cluster for mail server with: Ubuntu12.04 + DRBD (for block replication) + Ext4 filesystem + Dovecot-2.0.19-2 with Mdbox It works fine with ~50k accounts. My cluster design: http://adminlinux.com.br/cluster_design.txt I plan to test Dovecot Replication with dsync to build an active/active cluster with load balancing. Can anyone direct me to some literature? A tutorial for example. I'm installing a cluster (for tests) with Ubuntu 14.04 + Ext4 filesystem (without Drbd) + Dovecot-2.2.9-1ubuntu2.1. My conf is in: http://adminlinux.com.br/replication-doveconf.txt I'm following this doc http://wiki2.dovecot.org/Replication but I'm getting some errors: 1) Aug 14 15:39:34 ceph1 dovecot: imap(account=domaintest.com): Error: open(/var/run/dovecot/replication-notify-fifo) failed: Permission denied What is the correct perms for /var/run/dovecot/replication-notify-fifo ? Currently: # ll /var/run/dovecot/replication-notify-fifo prw------- 1 root root 0 Aug 14 15:57 /var/run/dovecot/replication-notify-fifo| 2) Aug 14 16:02:34 ceph1 dovecot: dsync-server(account=domaintest.com): Error: Couldn't create lock /var/lib/imap/account=domaintest.com/.dovecot-sync.lock: No such file or directory Aug 14 16:02:34 ceph1 dovecot: dsync-local(account=domaintest.com): Error: Couldn't create lock /var/lib/imap/account=domaintest.com/.dovecot-sync.lock: No such file or directory In my production cluster, it is automatically created by dovecot when the user performs the first login: /var/lib/imap/ for INDEX /var/spool/imap/ for Mdbox Thanks! -- Thiago Henrique From marcio.merlone at a1.ind.br Thu Aug 14 19:59:07 2014 From: marcio.merlone at a1.ind.br (=?windows-1252?Q?M=E1rcio_Merlone?=) Date: Thu, 14 Aug 2014 16:59:07 -0300 Subject: Trying Dovecot Replication with dsync In-Reply-To: <53ED0DC2.5080202@adminlinux.com.br> References: <53ED0DC2.5080202@adminlinux.com.br> Message-ID: <53ED150B.1030303@a1.ind.br> On 14-08-2014 16:28, Listas at Adminlinux wrote: > I have a failover cluster for mail server with: > Ubuntu12.04 + DRBD (for block replication) + Ext4 filesystem > + Dovecot-2.0.19-2 with Mdbox > (...) > I plan to test Dovecot Replication with dsync to build an > active/active cluster with load balancing. Sorry, I can't help you. But out of curiosity, if you already have DRBD, why dsync replication? Regards, -- *Marcio Merlone* From todasminhaslistas at gmail.com Thu Aug 14 20:40:27 2014 From: todasminhaslistas at gmail.com (Minhas Listas) Date: Thu, 14 Aug 2014 17:40:27 -0300 Subject: Trying Dovecot Replication with dsync Message-ID: Hi, I have a failover cluster for mail server with: Ubuntu12.04 + DRBD (for block replication) + Ext4 filesystem + Dovecot-2.0.19-2 with Mdbox It works fine with ~50k accounts. My cluster design: http://adminlinux.com.br/cluster_design.txt I plan to test Dovecot Replication with dsync to build an active/active cluster with load balancing. Can anyone direct me to some literature? A tutorial for example. I'm installing a cluster (for tests) with Ubuntu 14.04 + Ext4 filesystem (without Drbd) + Dovecot-2.2.9-1ubuntu2.1. My conf is in: http://adminlinux.com.br/replication-doveconf.txt I'm following this doc http://wiki2.dovecot.org/Replication but I'm getting some errors: 1) Aug 14 15:39:34 ceph1 dovecot: imap(account=domaintest.com): Error: open(/var/run/dovecot/replication-notify-fifo) failed: Permission denied What is the correct perms for /var/run/dovecot/replication-notify-fifo ? Currently: # ll /var/run/dovecot/replication-notify-fifo prw------- 1 root root 0 Aug 14 15:57 /var/run/dovecot/replication-notify-fifo| 2) Aug 14 16:02:34 ceph1 dovecot: dsync-server(account=domaintest.com): Error: Couldn't create lock */var/lib/imap/account=domaintest.com /*.dovecot-sync.lock: No such file or directory Aug 14 16:02:34 ceph1 dovecot: dsync-local(account=domaintest.com): Error: Couldn't create lock */var/lib/imap/account=domaintest.com /*.dovecot-sync.lock: No such file or directory In my production cluster, it is automatically created by dovecot when the user performs the first login: */var/lib/imap/* for INDEX */var/spool/imap/* for Mdbox Thanks! -- Thiago Henrique From idefix at fechner.net Thu Aug 14 22:18:28 2014 From: idefix at fechner.net (Matthias Fechner) Date: Fri, 15 Aug 2014 00:18:28 +0200 Subject: Cannot use dovecot with vpopmail and sendmail Message-ID: <53ED35B4.2000307@fechner.net> Hi, I setup a configuration that will use sendmail with dovecot as imap/pop3 service and vpopmail as backend to handle virtual users and store email there. For this I created a dovecot.m4 as described in the wiki, it looks like: ######################*****############## ### DOVECOT Mailer specification ### ##################*****################## Mdovecot, P=/usr/local/libexec/dovecot/dovecot-lda, F=l59DFMPhnu, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP, M=51200000, U=vpopmail:vchkpw, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u Then I added to my .mc file: MAILER(dovecot) I added to my mailertable: dovecot.fechner.net dovecot:dovecot I create one entry in the virtusertable to catch a email and forward it to dovecot (all other email have to handled by another imap server till dovecot is running fine): dovecot at fechner.net idefix at dovecot.fechner.net I created a domain and a user in vpopmail to catch idefix at dovecot.fechner.net. If I execute: doveadm auth test idefix at dovecot.fechner.net Password: passdb: idefix at dovecot.fechner.net auth succeeded extra fields: user=idefix at dovecot.fechner.net So I think the first part is fine. But if I try to sent an email to dovecot at fechner.net I get the error message: Aug 14 21:48:34 anny dovecot: lda(idefix at dovecot.fechner.net): Fatal: setuid(89(vpopmail) from userdb lookup) failed with euid=1002(idefix): Operation not permitted (This binary should probably be called with process user set to 89(vpopmail) instead of 1002(idefix)) Aug 14 21:48:34 anny sendmail[94303]: s7EJXgma087740: to=, ctladdr= (1002/0), delay=00:14:52, xdelay=00:00:00, mailer=dovecot, pri=120389, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL It seems to be a problem with dovecot-lda but I have no idea what is wrong. The user is set in the .m4 file and permission/group/user is set in the auth socket correctly. The output of doveconf -n is: # 2.2.13: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 10.0-STABLE amd64 zfs auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes first_valid_uid = 10 mail_debug = yes mail_location = maildir:/usr/local/vpopmail/domains/%d/%n/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = cache_key=%u%r%l webmail=127.0.0.1 driver = vpopmail } service auth-worker { unix_listener auth-worker { group = vchkpw user = vpopmail } } service auth { unix_listener auth-userdb { group = vchkpw mode = 0666 user = vpopmail } user = vpopmail } service imap-login { inet_listener imap { port = 22143 } inet_listener imaps { port = 22993 } } service pop3-login { inet_listener pop3 { port = 22110 } inet_listener pop3s { port = 22995 } } userdb { args = quota_template=quota_rule=*:backend=%q driver = vpopmail } Thanks for any ideas who this could be fixed. Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook From tss at iki.fi Fri Aug 15 11:06:10 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 Aug 2014 14:06:10 +0300 Subject: Multiple Passdb's - Order In-Reply-To: <157be37205187883a127c9ac1e41d1c3@dupond.be> References: <53DBF1C3.9000307@dupond.be> <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> <6fb75bf2cf5a93842d38610ab3bcbf1d@dupond.be> <157be37205187883a127c9ac1e41d1c3@dupond.be> Message-ID: <0BF03DB1-85C1-4C1D-910B-541B5AE9A271@iki.fi> Two problems. These fix them both: http://hg.dovecot.org/dovecot-2.2/rev/14082c2d81dd http://hg.dovecot.org/dovecot-2.2/rev/5e818f1b376d On 12 Aug 2014, at 10:53, Jean-Louis Dupond wrote: > Anyone might have a clue? > Would like to get the issue solved. Or at least know why its acting this way. > > Thanks > Jean-Louis > > Op 2014-08-04 15:11, schreef Jean-Louis Dupond: >> Hi, >> Thanks for your response. >> The error/warning log was indeed not included. >> I looked it up now, and dovecot only printed the following error: >> Aug 1 21:31:55 xcx dovecot: auth: Error: >> ldap(info at xxx.com,xxxxx,): >> ldap_search(base=uid=info at xxx.com,ou=mail,dc=xxx,dc=com >> filter=(&(objectClass=posixAccount)(uid=info at dupie.be))) failed: No >> such object >> This shouldn't really be a problem, because the user does indeed not >> exist in ldap, but only in the passwd-file. >> The problem also exists with auth cache disabled. >> And my doveconf (the passdb order is the working order here): >> # 2.2.9: /etc/dovecot/dovecot.conf >> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 >> auth_cache_size = 2 M >> mail_location = maildir:~/Maildir >> mail_plugins = virtual >> managesieve_notify_capability = mailto >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character vacation subaddress comparator-i;ascii-numeric >> relational regex imap4flags copy include variables body enotify >> environment mailbox date ihave >> namespace inbox { >> inbox = yes >> location = >> mailbox Drafts { >> special_use = \Drafts >> } >> mailbox Junk { >> special_use = \Junk >> } >> mailbox Sent { >> special_use = \Sent >> } >> mailbox "Sent Messages" { >> special_use = \Sent >> } >> mailbox Trash { >> special_use = \Trash >> } >> mailbox virtual/Flagged { >> special_use = \Flagged >> } >> prefix = >> } >> passdb { >> args = scheme=CRYPT username_format=%u /etc/dovecot/users >> driver = passwd-file >> } >> passdb { >> args = /etc/dovecot/dovecot-ldap.conf.ext >> driver = ldap >> } >> plugin { >> sieve = /var/vmail/%d/%n/.dovecot.sieve >> sieve_dir = /var/vmail/%d/%n/sieve >> } >> postmaster_address = postmaster at xxx.com >> protocols = " imap lmtp sieve pop3" >> service auth { >> unix_listener /var/spool/postfix-mailing/private/auth { >> mode = 0666 >> } >> unix_listener /var/spool/postfix/private/auth { >> mode = 0666 >> } >> } >> service lmtp { >> unix_listener /var/spool/postfix/private/dovecot-lmtp { >> group = postfix >> mode = 0600 >> user = postfix >> } >> user = vmail >> } >> service managesieve-login { >> inet_listener sieve { >> port = 4190 >> } >> } >> ssl_ca = > ssl_cert = > ssl_key = > userdb { >> args = uid=vmail gid=vmail home=/var/vmail/%d/%n >> driver = static >> } >> protocol lmtp { >> mail_plugins = virtual sieve >> } >> Thanks >> Jean-Louis >> Op 2014-08-04 15:03, schreef Timo Sirainen: >>> On 01 Aug 2014, at 22:00, Jean-Louis Dupond wrote: >>>> Today I upgraded my dovecot to a newer version (2.2.9 debian >>>> wheezy-backports) >>>> At that moment I also reconfigured the whole thing a bit nicer. >>>> Now I stumbled on the following issue. >>> .. >>>> The only thing I changed here was the order the passdb backends. >>>> It works fine now, but it not clear to me why the order is so important. >>>> Could somebody explain this? Or is this some bug? >>> I can't reproduce it. Also did your log contain also the >>> errors/warnings? It appeared that it was failing with some internal >>> error, which you didn't show in the log. Also does it work if you >>> disable auth cache? Full doveconf -n output may have been helpful >>> also. From tss at iki.fi Fri Aug 15 11:10:02 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 Aug 2014 14:10:02 +0300 Subject: namspace management In-Reply-To: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> References: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> Message-ID: On 12 Aug 2014, at 19:04, A. Schulze wrote: > since some weeks I'm playing with namespaces. But I still did nod found a solution for all faces of different problems. > > Current issue: Public namespace. > I have users inbox as follow: > > mail_home = /data/mail/%Ln/ > mail_location = maildir:~/Maildir:INDEX=~/.dovecot.index > namespace { > inbox = yes > separator = / > prefix = INBOX/ > } > > Public namespace: > namespace { > location = maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/ > prefix = PUBLIC/ > separator = / > type = public > } > > that way only one subscription file is possible. Any use could subscribe/unsubscribe any folder > and that match all users. > If I add "subscription=no" the parent namespace' subscription is used. This is the solution. > In fact there is no parent namespace an no subscription management is possible at all. Add another namespace just for the subscriptions: namespace { list = no hidden = yes } From tss at iki.fi Fri Aug 15 11:10:38 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 Aug 2014 14:10:38 +0300 Subject: namspace management In-Reply-To: References: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> Message-ID: <955FA22B-FD7A-44EB-8746-D326A2FDD035@iki.fi> On 13 Aug 2014, at 09:58, Steffen Kaiser wrote: > location = maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/:CONTROL=~/.dovecot.index.public/ Not recommended, because it also moves dovecot-keywords file so any keywords added to the public folder won't be shared across users. From jean-louis at dupond.be Fri Aug 15 11:12:44 2014 From: jean-louis at dupond.be (Jean-Louis Dupond) Date: Fri, 15 Aug 2014 13:12:44 +0200 Subject: Multiple Passdb's - Order In-Reply-To: <0BF03DB1-85C1-4C1D-910B-541B5AE9A271@iki.fi> References: <53DBF1C3.9000307@dupond.be> <3AFC6691-C030-420B-AEDE-FEB948E63BD6@iki.fi> <6fb75bf2cf5a93842d38610ab3bcbf1d@dupond.be> <157be37205187883a127c9ac1e41d1c3@dupond.be> <0BF03DB1-85C1-4C1D-910B-541B5AE9A271@iki.fi> Message-ID: <53EDEB2C.3000409@dupond.be> Hi, Thanks alot for those changes. A 'bug' killed again :) Will test them out later, but the changes seems logic indeed. Thx Jean-Louis Op 15-08-14 om 13:06 schreef Timo Sirainen: > Two problems. These fix them both: > > http://hg.dovecot.org/dovecot-2.2/rev/14082c2d81dd > http://hg.dovecot.org/dovecot-2.2/rev/5e818f1b376d > > On 12 Aug 2014, at 10:53, Jean-Louis Dupond wrote: > >> Anyone might have a clue? >> Would like to get the issue solved. Or at least know why its acting this way. >> >> Thanks >> Jean-Louis >> >> Op 2014-08-04 15:11, schreef Jean-Louis Dupond: >>> Hi, >>> Thanks for your response. >>> The error/warning log was indeed not included. >>> I looked it up now, and dovecot only printed the following error: >>> Aug 1 21:31:55 xcx dovecot: auth: Error: >>> ldap(info at xxx.com,xxxxx,): >>> ldap_search(base=uid=info at xxx.com,ou=mail,dc=xxx,dc=com >>> filter=(&(objectClass=posixAccount)(uid=info at dupie.be))) failed: No >>> such object >>> This shouldn't really be a problem, because the user does indeed not >>> exist in ldap, but only in the passwd-file. >>> The problem also exists with auth cache disabled. >>> And my doveconf (the passdb order is the working order here): >>> # 2.2.9: /etc/dovecot/dovecot.conf >>> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 >>> auth_cache_size = 2 M >>> mail_location = maildir:~/Maildir >>> mail_plugins = virtual >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope >>> encoded-character vacation subaddress comparator-i;ascii-numeric >>> relational regex imap4flags copy include variables body enotify >>> environment mailbox date ihave >>> namespace inbox { >>> inbox = yes >>> location = >>> mailbox Drafts { >>> special_use = \Drafts >>> } >>> mailbox Junk { >>> special_use = \Junk >>> } >>> mailbox Sent { >>> special_use = \Sent >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } >>> mailbox Trash { >>> special_use = \Trash >>> } >>> mailbox virtual/Flagged { >>> special_use = \Flagged >>> } >>> prefix = >>> } >>> passdb { >>> args = scheme=CRYPT username_format=%u /etc/dovecot/users >>> driver = passwd-file >>> } >>> passdb { >>> args = /etc/dovecot/dovecot-ldap.conf.ext >>> driver = ldap >>> } >>> plugin { >>> sieve = /var/vmail/%d/%n/.dovecot.sieve >>> sieve_dir = /var/vmail/%d/%n/sieve >>> } >>> postmaster_address = postmaster at xxx.com >>> protocols = " imap lmtp sieve pop3" >>> service auth { >>> unix_listener /var/spool/postfix-mailing/private/auth { >>> mode = 0666 >>> } >>> unix_listener /var/spool/postfix/private/auth { >>> mode = 0666 >>> } >>> } >>> service lmtp { >>> unix_listener /var/spool/postfix/private/dovecot-lmtp { >>> group = postfix >>> mode = 0600 >>> user = postfix >>> } >>> user = vmail >>> } >>> service managesieve-login { >>> inet_listener sieve { >>> port = 4190 >>> } >>> } >>> ssl_ca = >> ssl_cert = >> ssl_key = >> userdb { >>> args = uid=vmail gid=vmail home=/var/vmail/%d/%n >>> driver = static >>> } >>> protocol lmtp { >>> mail_plugins = virtual sieve >>> } >>> Thanks >>> Jean-Louis >>> Op 2014-08-04 15:03, schreef Timo Sirainen: >>>> On 01 Aug 2014, at 22:00, Jean-Louis Dupond wrote: >>>>> Today I upgraded my dovecot to a newer version (2.2.9 debian >>>>> wheezy-backports) >>>>> At that moment I also reconfigured the whole thing a bit nicer. >>>>> Now I stumbled on the following issue. >>>> .. >>>>> The only thing I changed here was the order the passdb backends. >>>>> It works fine now, but it not clear to me why the order is so important. >>>>> Could somebody explain this? Or is this some bug? >>>> I can't reproduce it. Also did your log contain also the >>>> errors/warnings? It appeared that it was failing with some internal >>>> error, which you didn't show in the log. Also does it work if you >>>> disable auth cache? Full doveconf -n output may have been helpful >>>> also. From tss at iki.fi Fri Aug 15 11:16:09 2014 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 Aug 2014 14:16:09 +0300 Subject: (Again) Some trouble with dovecot 2.1(.7) and SASL In-Reply-To: <01540C4F-42CF-44E0-8016-E04645F0BA58@awib.it> References: <01540C4F-42CF-44E0-8016-E04645F0BA58@awib.it> Message-ID: <1690805D-9C3E-4FDB-9E6D-6C022BDFA50F@iki.fi> On 13 Aug 2014, at 22:32, admin wrote: > The parentheses in the subject are due to referral to this discussion on the mailing list: http://dovecot.org/list/dovecot/2012-February/063851.html > > At first I thought I found the problem, but then realized this was 2012 and the suggested patch already is applied in dovecot. > > I also have segault messages in my kernel log; not too many, but it was enough to dig for it: > >> Wed Jun 25 08:29:05 2014 auth[17590]: segfault at 10 ip 00007f02ad5e8e56 sp 00007fff932f9250 error 4 in auth[7f02ad5d3000+40000] >> Mon Jun 30 16:11:41 2014 auth[12501]: segfault at 10 ip 00007fc6a1fefe56 sp 00007fff276e6ea0 error 4 in auth[7fc6a1fda000+40000] >> Sat Jul 5 13:11:32 2014 auth[15603]: segfault at 10 ip 00007f6447a6be56 sp 00007fff09c48020 error 4 in auth[7f6447a56000+40000] >> Tue Jul 8 05:08:21 2014 auth[30071]: segfault at 10 ip 00007ff22656ae56 sp 00007fff26b017f0 error 4 in auth[7ff226555000+40000] >> Mon Jul 21 14:25:28 2014 auth[30918]: segfault at 10 ip 00007fe1fc817e56 sp 00007fff7e654f20 error 4 in auth[7fe1fc802000+40000] <--- >> Wed Aug 13 18:56:31 2014 auth[27593]: segfault at 10 ip 00007ffbf8ce8e56 sp 00007fffe0157f70 error 4 in auth[7ffbf8cd3000+40000] >> Wed Aug 13 18:57:59 2014 auth[24264]: segfault at 10 ip 00007f2df31bde56 sp 00007fff094263c0 error 4 in auth[7f2df31a8000+40000] > > > The last 2 entries are a result of me being able to reproduce the problem, the 3rd last is the only one I was able to track down (due to log rotation) Having a gdb backtrace of the crash would be very useful. See http://dovecot.org/bugreport.html > I tracked the problem down to a contact form of a website of one of my customers. whenever used, the segfault gets triggered. > Nevertheless, the message always gets delivered. > > So we are talking about the marked line; the corresponding log entries from mail.info: > >> Jul 21 14:25:29 mail postfix/smtpd[27590]: D4D8E44752: client=unknown[], sasl_method=CRAM-MD5, sasl_username= Looks like some bug related to CRAM-MD5. I don't see any fixes to it since v2.1.7, but I looked through the code and I don't see any obvious bugs at least. It's also possible that the bug is elsewhere in auth and has been fixed since v2.1.7. From malloyj at btinternet.com Fri Aug 15 12:24:38 2014 From: malloyj at btinternet.com (Joseph Malloy) Date: Fri, 15 Aug 2014 13:24:38 +0100 Subject: Server dovecot: auth: Warning: sql: Ignoring changed iterate_quer Message-ID: <1408105478.55366.YahooMailNeo@web87704.mail.ir2.yahoo.com> Hello,? I am having an issue with Dovecot. I am using Roundcube to access my mailboxes, which I have set up using ViMbAdmin.? However when I go to the Roundcube, I get login Failed.? So looking in the mail.log I can see the following:? Aug 15 13:20:09 Server dovecot: auth: Warning: sql: Ignoring changed iterate_query in /etc/dovecot/dovecot-sql.conf.ext, because userdb sql not used. (If this is intentional, set userdb_warning_disable=yes) Aug 15 13:20:11 Server dovecot: imap-login: Disconnected (auth failed, 1 attempts in 2 secs): user=, method=PLAIN, rip=62.49.15.89, lip=192.168.0.90, TLS, session= I am most greatful for the support Joe :) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Configs.txt URL: From ben at indietorrent.org Fri Aug 15 13:57:09 2014 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 15 Aug 2014 09:57:09 -0400 Subject: Backtrace io_add(0x1) called twice fd= (was Exit status code 134; what is it, in the context of Dovecot Antispam plug-in?) In-Reply-To: <53E4FD60.7040808@indietorrent.org> References: <53D6EC93.8000503@indietorrent.org> <53D7BBD5.7080106@indietorrent.org> <53E39576.3000705@indietorrent.org> <53E39E11.90602@indietorrent.org> <53E4FD60.7040808@indietorrent.org> Message-ID: <53EE11B5.3010101@indietorrent.org> On 8/8/2014 12:40 PM, Ben Johnson wrote: > Unfortunately, despite the valiant effort, the behavior is exactly the > same; still seeing exit code status 134 whenever the antispam plugin > fires. :( > > And yes, Steffen, the fd=7 is always present and the same. > >> > Dovecot tries to be admin-friendly. Common error messages are made >> > as easily understandable as possible. Any crash, no matter how it >> > happened, is considered a bug that will be fixed. > Have we reached this point yet? > > Happy to try any other suggestions... > > Thanks! > > -Ben I'll submit a bug report once I'm able to capture the appropriate debugging information. Thanks, -Ben From sca at andreasschulze.de Fri Aug 15 17:14:39 2014 From: sca at andreasschulze.de (A. Schulze) Date: Fri, 15 Aug 2014 19:14:39 +0200 Subject: namspace management In-Reply-To: <955FA22B-FD7A-44EB-8746-D326A2FDD035@iki.fi> References: <20140812180410.Horde.JZp53v0DUjXa6rDUwLpz2g5@horde.andreasschulze.de> <955FA22B-FD7A-44EB-8746-D326A2FDD035@iki.fi> Message-ID: <20140815191439.Horde.dtZXh9Q9egIGqZREP_qCCw1@horde.andreasschulze.de> Timo Sirainen: > On 13 Aug 2014, at 09:58, Steffen Kaiser >> location = >> maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.index.public/:CONTROL=~/.dovecot.index.public/ > > Not recommended, because it also moves dovecot-keywords file so any > keywords added to the public folder won't be shared across users. OK, moved only the subscription file to a place next to the Maildir: /data/mail_public/Maildir contain public mailboxes /data/mail_public/Maildir/dovecot-acl allow access ( contain only "authenticated lrs" ) /data/mail_public/subscriptions contain the subscription files namespace { inbox = yes prefix = INBOX/ separator = / } namespace { location = maildir:/data/mail_public/Maildir/:INDEXPVT=~/.dovecot.public/:SUBSCRIPTIONS=../subscriptions/%Lu prefix = PUBLIC/ separator = / type = public } namespace { list = children location = maildir:%%h/Maildir:INDEXPVT=~/.dovecot.shared/%%u/:CONTROL=~/.dovecot.shared/%%u/ prefix = SHARED/%%u/ separator = / type = shared } Andreas From ben at indietorrent.org Fri Aug 15 19:53:21 2014 From: ben at indietorrent.org (Ben Johnson) Date: Fri, 15 Aug 2014 15:53:21 -0400 Subject: Fatal: master: service(imap): child 28145 killed with signal 11 (core dumped) -- started happening after upgrade last night Message-ID: <53EE6531.9000701@indietorrent.org> Hello, I upgraded Dovecot last night and have been receiving core-dumps any time I attempt to access a mailbox (via IMAP) ever since. Logging-in seems to work, but any attempt to access the mailbox contents (regardless of folder) causes a core-dump. Please let me know if I can provide any additional details. apt Sources Details: # grep "dovecot" /etc/apt/sources.list deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.2 main deb-src http://xi.rename-it.nl/debian/ stable-auto/dovecot-2.2 main Package Upgrade Details: Start-Date: 2014-08-14 19:30:57 Commandline: apt-get upgrade Upgrade: dovecot-sieve:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-managesieved:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-dev:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-mysql:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-core:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-pop3d:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150), dovecot-imapd:amd64 (2.2.13-1~auto+138, 2.2.13-1~auto+150) End-Date: 2014-08-14 19:32:39 Filesystem Details # df -T | grep "xvda1" Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/xvda1 ext3 61796348 8037864 50612756 14% / ================= DOVECONF ================= # doveconf -n # 2.2.13 (c810e288aaed): /etc/dovecot/dovecot.conf # OS: Linux 3.13.0-24-generic x86_64 Ubuntu 14.04.1 LTS auth_mechanisms = plain login disable_plaintext_auth = no listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_max_userip_connections = 20 mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_backend = mailtrain antispam_debug_target = syslog antispam_mail_notspam = --ham antispam_mail_sendmail = /bin/bash antispam_mail_sendmail_args = /usr/local/bin/sa-learn-pipe.sh antispam_mail_spam = --spam antispam_spam_pattern_ignorecase = SPAM;JUNK antispam_trash_pattern_ignorecase = trash;Deleted * antispam_verbose_debug = 1 quota = dict:user::file:/var/vmail/%d/%n/.quotausage quota_rule2 = Trash:storage=+100M quota_rule3 = Junk:ignore quota_rule4 = INBOX:storage=+100M quota_warning = storage=100%% quota-reached 100 %u %d quota_warning2 = storage=95%% quota-warning 95 %u %d quota_warning3 = storage=80%% quota-warning 80 %u %d quota_warning4 = -storage=100%% quota-below below %u %d sieve = /var/vmail/%d/%n/.sieve } postmaster_address = postmaster at example.com protocols = imap pop3 sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service config { unix_listener config { group = vmail mode = 0600 user = vmail } } service imap-login { client_limit = 1000 process_limit = 500 } service quota-below { executable = script /usr/local/bin/quota-below.sh unix_listener quota-below { group = vmail mode = 0666 user = vmail } user = vmail } service quota-reached { executable = script /usr/local/bin/quota-reached.sh unix_listener quota-reached { group = vmail mode = 0666 user = vmail } user = vmail } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0666 user = vmail } user = vmail } ssl_cert = , method=PLAIN, rip=76.179.182.207, lip=192.64.115.178, mpid=29100, TLS, session= Aug 15 19:39:30 xen dovecot: imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Aug 15 19:39:30 xen dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Aug 15 19:39:30 xen dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib11_imap_quota_plugin.so Aug 15 19:39:30 xen dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib90_antispam_plugin.so Aug 15 19:39:30 xen dovecot: imap: Debug: Added userdb setting: mail=maildir:/var/vmail/example.com/ben/Maildir Aug 15 19:39:30 xen dovecot: imap: Debug: Added userdb setting: plugin/quota_rule=*:storage=0B Aug 15 19:39:30 xen dovecot: imap: Debug: Added userdb setting: plugin/sieve=/var/vmail/example.com/ben/.sieve Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Effective uid=5000, gid=5000, home=/var/vmail/example.com/ben Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota root: name=user backend=dict args=:file:/var/vmail/example.com/ben/.quotausage Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota rule: root=user mailbox=* bytes=0 messages=0 Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota rule: root=user mailbox=Trash bytes=+104857600 messages=0 Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota rule: root=user mailbox=Junk ignored Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota rule: root=user mailbox=INBOX bytes=+104857600 messages=0 Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota warning: bytes=0 (100%) messages=0 reverse=no command=quota-reached 100 ben at example.com example.com Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota warning: bytes=0 (95%) messages=0 reverse=no command=quota-warning 95 ben at example.com example.com Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota warning: bytes=0 (80%) messages=0 reverse=no command=quota-warning 80 ben at example.com example.com Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota warning: bytes=0 (100%) messages=0 reverse=yes command=quota-below below ben at example.com example.com Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: Quota grace: root=user bytes=0 (10%) Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: dict quota: user=ben at example.com, uri=file:/var/vmail/example.com/ben/.quotausage, noenforcing=0 Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Debug: maildir++: root=/var/vmail/example.com/ben/Maildir, index=, indexpvt=, control=, inbox=/var/vmail/example.com/ben/Maildir, alt= Aug 15 19:39:30 xen dovecot: imap(ben at example.com): Fatal: master: service(imap): child 29100 killed with signal 11 (core dumped) From arekm at maven.pl Fri Aug 15 21:25:46 2014 From: arekm at maven.pl (Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?=) Date: Fri, 15 Aug 2014 23:25:46 +0200 Subject: auth_username_translation and LTMP problem Message-ID: <201408152325.46688.arekm@maven.pl> auth_username_translation seems to be appliet on RCPT TO address of LTMP transport. Why dovecot is doing that? And better question - is there a way to disable auth_username_translation for LTMP but leave enabled for the rest (imap, pop3 etc) ? Background: I'm doing auth_username_translation = @= to allow logins like aaa at bbb.pl to be internally translated to aaa=bbb.pl. That works fine. Now my exim delivers mail to dovecot using LTMP and it does translation or it own, so it does: RCPT TO: Unfortunately looks like dovecot it doing translation one more time and looking in user database for aaa=bbb.pl=mymbox.pl where such user doesn't exist. Only aaa=bbb.pl user exists. Thanks, -- Arkadiusz Mi?kiewicz, arekm / maven.pl From arekm at maven.pl Fri Aug 15 22:14:10 2014 From: arekm at maven.pl (Arkadiusz =?utf-8?q?Mi=C5=9Bkiewicz?=) Date: Sat, 16 Aug 2014 00:14:10 +0200 Subject: auth_username_translation and LTMP problem In-Reply-To: <201408152325.46688.arekm@maven.pl> References: <201408152325.46688.arekm@maven.pl> Message-ID: <201408160014.11057.arekm@maven.pl> On Friday 15 of August 2014, Arkadiusz Mi?kiewicz wrote: > auth_username_translation seems to be appliet on RCPT TO address of LTMP > transport. > > Why dovecot is doing that? > > And better question - is there a way to disable auth_username_translation > for LTMP but leave enabled for the rest (imap, pop3 etc) ? Note, just tested: auth_username_translation = @= protocol lmtp { auth_username_translation = } This can be set but that doesn't work. dovecot seems to be still doing translation even for lmtp. -- Arkadiusz Mi?kiewicz, arekm / maven.pl From axel.luttgens at skynet.be Sat Aug 16 07:31:57 2014 From: axel.luttgens at skynet.be (Axel Luttgens) Date: Sat, 16 Aug 2014 09:31:57 +0200 Subject: Server dovecot: auth: Warning: sql: Ignoring changed iterate_quer In-Reply-To: <1408105478.55366.YahooMailNeo@web87704.mail.ir2.yahoo.com> References: <1408105478.55366.YahooMailNeo@web87704.mail.ir2.yahoo.com> Message-ID: Le 15 ao?t 2014 ? 14:24, Joseph Malloy a ?crit : > Hello, > > [...] > > Aug 15 13:20:09 Server dovecot: auth: Warning: sql: Ignoring changed iterate_query in /etc/dovecot/dovecot-sql.conf.ext, because userdb sql not used. (If this is intentional, set userdb_warning_disable=yes) > > [...] Hello Joseph, It seems you have defined a passdb { } section (twice!?), but no userdb { } one. You could try by replacing the second occurrence of keyword "passdb" by "userdb". HTH, Axel From matthias at familie-sitte.org Sat Aug 16 18:37:33 2014 From: matthias at familie-sitte.org (Dr. Matthias Sitte) Date: Sat, 16 Aug 2014 13:37:33 -0500 Subject: [SOLVED] Re: INBOX subfolders not showing up In-Reply-To: <53DEDB54.3030800@familie-sitte.org> References: <53DA5F6A.9000304@familie-sitte.org> <53DD2FFE.8060503@titan21.co.uk> <53DEDB54.3030800@familie-sitte.org> Message-ID: <53EFA4ED.4060304@familie-sitte.org> Marking this issue as solved. Solution is below. Matthias On 08/03/2014 08:01 PM, Dr. Matthias Sitte wrote: > Ok, just googled some more and came across this post [1]. It's basically > a very similar to mine, and describes the same problem that INBOX has no > subfolders. So, I've logged in with telnet to list the INBOX subfolders > as well: > > === SNIP === > 2 list "" INBOX > * LIST (\HasNoChildren) "/" "INBOX" > 2 OK List completed. > === SNAP === > > Duh. But everything else shows up: > > === SNIP === > > 4 list "" "*" > [...] > * LIST (\HasChildren) "/" "Archives" > [...] > 4 OK List completed. > === SNAP === > > So, what just happened? According to [1], there's a bug in lib-storage > with LAYOUT=fs and non-mbox maildir format [2]. Is that correct? I have > to assume that it's not fixed in Debian's packaged version 2.1.7, right? > > Can someone confirm that? > > > > [1] > http://dovecot.2317879.n4.nabble.com/Subfolders-of-Inbox-with-LAYOUT-fs-td43659.html > [2] http://hg.dovecot.org/dovecot-2.1/rev/589d2f485a6e > > > > On 08/02/2014 01:37 PM, Tim wrote: >> I came across something like this recently and it turned out it was a >> permissions issue on a users maildir/ folder. Check that - took me hours >> to figure it out and 2 secs to fix. Have you also checked your logs to >> see if there are any errors? >> >> On 31/07/14 16:23, Matthias Sitte wrote: >>> Hi, >>> >>> I have a problem with INBOX subfolders and subscriptions which I can't >>> solve... Maybe somebody has seen this, too... >>> >>> I can create regular subfolders (not in INBOX) and (un-)subscribe to >>> them with Thunderbird. If I create subfolders in the INBOX with TB >>> this seems to work now in v31 (just found this out), but with v24.6 >>> neither INBOX subfolder creation nor subscriptions worked. When I >>> modified the `subscriptions' file manually on the server, the folders >>> would show up in TB, though. >>> >>> Maybe this was a TB bug, but quite similarly INBOX subfolders don't >>> show up on iPhones (iOS up-to-date), but all other subfolders do. >>> >>> Any ideas what's going on? >>> >>> Best, >>> Matthias >>> >>> ====================================================================== >>> >>> # dovecot -n >>> # 2.1.7: /etc/dovecot/dovecot.conf >>> # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.6 ext4 >>> hostname = familie-sitte.org >>> log_path = /var/log/dovecot.log >>> log_timestamp = "%Y-%m-%d %H:%M:%S " >>> mail_location = maildir:/srv/mail/%d/%n/Maildir:LAYOUT=fs >>> managesieve_notify_capability = mailto >>> managesieve_sieve_capability = fileinto reject envelope >>> encoded-character vacation subaddress comparator-i;ascii-numeric >>> relational regex imap4flags copy include variables body enotify >>> environment mailbox date ihave >>> namespace inbox { >>> inbox = yes >>> list = yes >>> location = >>> mailbox Drafts { >>> special_use = \Drafts >>> } >>> mailbox Junk { >>> special_use = \Junk >>> } >>> mailbox Sent { >>> special_use = \Sent >>> } >>> mailbox "Sent Messages" { >>> special_use = \Sent >>> } >>> mailbox Trash { >>> special_use = \Trash >>> } >>> prefix = >>> separator = / >>> subscriptions = yes >>> type = private >>> } >>> passdb { >>> args = /etc/dovecot/dovecot-sql.conf.ext >>> driver = sql >>> } >>> plugin { >>> quota = maildir:User quota >>> quota_warning = storage=95%% quota-warning 95 %u >>> quota_warning2 = storage=80%% quota-warning 80 %u >>> sieve = ~/.dovecot.sieve >>> sieve_dir = ~/sieve >>> } >>> postmaster_address = postmaster at familie-sitte.org >>> protocols = " imap sieve pop3" >>> service auth { >>> unix_listener /var/spool/postfix/private/auth { >>> group = postfix >>> mode = 0660 >>> user = postfix >>> } >>> } >>> service imap-login { >>> inet_listener imaps { >>> port = 993 >>> ssl = yes >>> } >>> } >>> service managesieve-login { >>> inet_listener sieve { >>> port = 4190 >>> } >>> } >>> service quota-warning { >>> executable = script /usr/local/bin/quota-warning.sh >>> unix_listener quota-warning { >>> user = vmail >>> } >>> user = dovecot >>> } >>> ssl_cert = >> ssl_cipher_list = >>> SSDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA256:EECDH+aRSA+RC4:EDH+aRSA:EECDH:RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS >>> >>> ssl_key = >> userdb { >>> args = uid=vmail gid=vmail home=/srv/mail/%d/%n >>> driver = static >>> } >>> protocol lda { >>> mail_plugins = " quota sieve" >>> } >>> protocol imap { >>> mail_plugins = " imap_quota quota" >>> } >> > From akbwiz+dovecot at gmail.com Sun Aug 17 07:43:42 2014 From: akbwiz+dovecot at gmail.com (Akash) Date: Sun, 17 Aug 2014 13:13:42 +0530 Subject: Segmentation fault while indexing a large mailbox using doveadm.. Message-ID: Hi, I am having the same issue as: http://www.dovecot.org/list/dovecot/2014-April/095681.html although I am using the latest dovecot-core and dovecot-lucene packages from the debian sid repo which already has the patch (http://hg.dovecot.org/dovecot-2.2/rev/5e42a8c18b51) mentioned in the above thread implemented. doveadm(admin): Info: Old.All: Caching mails seq=1..300332 9400/300332 Program received signal SIGSEGV, Segmentation fault. (gdb) bt #0 __wcslen ( s=s at entry=0xb93887a8 L" btnToggle: \"Change size\"}\r\n