March 2015 - dovecot - dovecot.org

Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd)
by Santiago Vila 11 May '15

11 May '15

Hello. I wrote about this three weeks ago but got no answer. I'm going to officially "forward" the Debian bug this time, with all the details. The test case is just 840 bytes long. Please give it a try. ---------- Forwarded message ---------- From: Santiago Vila <sanvila(a)unex.es> To: submit(a)bugs.debian.org Date: Fri, 23 Jan 2015 22:32:28 +0100 (CET) Subject: dovecot-imapd: corrupts mailbox after trying to retrieve it Package: dovecot-imapd Version: 1:2.2.13-11 Severity: serious The following mbox folder, when put in $HOME/mail, becomes corrupted after trying to retrieve it with fetchmail. The problem may be reproduced by using the same machine as server and client: * Put "inbox-b" in $HOME/mail * Put this in $HOME/.fetchmailrc server localhost proto imap port 143: user "someuser" pass "thepassword" * Retrieve email using this command line: fetchmail -a localhost --folder inbox-b -m "true" Note: By looking at the "true" above it is clear that whatever fetchmail does with the message is not important at all. You will see something like this: 12 messages for someuser at localhost (folder inbox-b). reading message someuser@localhost:1 of 12 (171 header octets) (3 body octets) flushed reading message someuser@localhost:2 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:3 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:4 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:5 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:6 of 12 (171 header octets) (3 body octets) flushed reading message someuser@localhost:7 of 12 (171 header octets) (3 body octets) flushed reading message someuser@localhost:8 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:9 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:10 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:11 of 12 (245 header octets) (3 body octets) flushed reading message someuser@localhost:12 of 12 (273 header octets)fetchmail: incorrect header line found - see manpage for bad-header option not flushed And in fact "inbox-b" in the server is now like this: [...] >From root(a)example.com Tue Jan 13 10:18:20 2015 rstuvwxyzabcdefghijklmnopqrstuvwxyz(a)example.com To: a(a)example.com Subject: a MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Message-Id: <20150113091737.B5ADA5F8B1(a)example.com> Date: Tue, 13 Jan 2015 10:17:25 +0100 (CET) X-UID: 16035 Status: O a Note how the From: line has been truncated from its original state. I have been suffering from this problem for months. At first I believed it was some misbehaving procmail/formail recipe I had on the server, but that's not the case as this example shows. Thanks.

6 23

SQLite does not depend on zlib, was: Re: [PATCH] Split sql drivers from lib-sql to plugins
by Bernd Kuhls 04 May '15

04 May '15

Tomas Janousek <tjanouse(a)redhat.com> wrote in news:20070413132731.GA8281 @redhat.com: > - SQL_LIBS="$SQL_LIBS -lsqlite3 -lz" > + SQLITE_LIBS="$SQLITE_LIBS -lsqlite3 -lz" Hi, this patch fixes a build error during cross compilation to a system without the libz target package: --- dovecot-2.2.15.org/configure.ac 2014-10-25 05:57:08.000000000 +0200 +++ dovecot-2.2.15/configure.ac 2014-11-08 10:06:23.015570150 +0100 @@ -2293,7 +2293,7 @@ if test $want_sqlite != no; then AC_CHECK_LIB(sqlite3, sqlite3_open, [ AC_CHECK_HEADER(sqlite3.h, [ - SQLITE_LIBS="$SQLITE_LIBS -lsqlite3 -lz" + SQLITE_LIBS="$SQLITE_LIBS -lsqlite3" AC_DEFINE(HAVE_SQLITE,, Build with SQLite3 support) found_sql_drivers="$found_sql_drivers sqlite" Regards, Bernd

2 2

acl and lazy_expunge plugins are incompatible?
by Paolo Cravero 24 Apr '15

24 Apr '15

Hello list. Following two previous unanswered requests (http://www.dovecot.org/list/dovecot/2014-August/097449.html and http://www.dovecot.org/list/dovecot/2014-May/096261.html) I fell into the same problem: ACL and lazy_expunge plugins do not work together. I had lazy_expunge enabled working OK. Then I activated ACL plugin, and gave dovecot.2 visibily over paolo.cravero's Inbox: the IMAP connection fails on server-side. Relevant debug log is: Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: Quota grace: root=User quota bytes=576716800 (10%) Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: dict quota: user=paolo.cravero(a)csi.it, uri=file:/maildata1/home/csi.it/p/paolo.cravero/dovecot-quota, noenforcing=0 Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: fs: root=/maildata1/main/csi.it/p/paolo.cravero, index=/maildata1/indexes/csi.it/p/paolo.cravero, indexpvt=, control=, inbox=, alt=/maildata1/altstorage/csi.it/p/paolo.cravero Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: acl: initializing backend with data: vfile Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: acl: acl username = paolo.cravero(a)csi.it Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: acl: owner = 1 Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Debug: acl vfile: Global ACLs disabled Mar 3 15:59:02 tst-msg03 dovecot: imap(dovecot.2(a)csi.it): Fatal: lazy_expunge: Unknown namespace: '.EXPUNGED/' Removing the lazy_expunge, ACLs do work as expected and I can add/remove/use them. Is there a known compatibility issue? Or there's something wrong in my config and someone else is running with both plugins enabled? Thanks, Paolo PS: output of dovecot -n: # 2.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.5 (Santiago) ext3 auth_mechanisms = plain login disable_plaintext_auth = no imap_client_workarounds = tb-extra-mailbox-sep lmtp_save_to_detail_mailbox = yes mail_attachment_dir = /maildata1/attachments mail_attachment_hash = %{sha256} mail_debug = yes mail_gid = 500 mail_location = mdbox:/maildata1/main/%d/%1n/%n:ALT=/maildata1/altstorage/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n mail_plugins = quota mail_log notify lazy_expunge acl mail_uid = 500 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate mbox_write_locks = fcntl mdbox_rotate_size = 50 M namespace { hidden = no list = yes location = mdbox:/maildata1/main/%d/%1n/%n:INDEX=/maildata1/indexes/%d/%1n/%n:MAILBOXDIR=expunged prefix = .EXPUNGED/ subscriptions = no } namespace { list = children location = mdbox:/maildata1/main/%%d/%%1n/%%n:ALT=/maildata1/altstorage/%%d/%%1n/%%n:INDEX=/maildata1/indexes/%%d/%%1n/%%n prefix = user/%%u/ separator = / type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/maildata1/db/shared-mailboxes.db lazy_expunge = .EXPUNGED/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename flag_change append mail_log_fields = uid box msgid size flags vsize quota = dict:User quota::file:%h/dovecot-quota quota_grace = 10%% quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = vmail user = vmail } } service lmtp { inet_listener lmtp { address = 10.102.42.114 port = 24 } process_min_avail = 2 user = vmail } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota mail_log notify lazy_expunge acl sieve } protocol lda { mail_plugins = quota mail_log notify lazy_expunge acl sieve } protocol imap { mail_plugins = quota mail_log notify lazy_expunge acl imap_quota imap_acl }

2 2

concurrent IMAP connections (plus 2 easy questions)
by Ron Cleven 15 Apr '15

15 Apr '15

I posted a few days back asking about configuration issues with a modestly large number of IMAP connections. Several people were kind enough to respond with various ideas. Armed with those ideas and Google, I was able to determine the underlying configuration issues with CentOS 7 and Dovecot 2.2.10. I did some further benchmarking to ensure that we could properly plan our server configuration requirements as we continue to roll this out, so I thought I would share this, partly to see if anyone has experiences that conflict with my findings. I found that each incremental dovecot IMAP process required about 750k of RAM. Simulations confirmed that 8 gig of RAM was sufficient to support approximately 10,000 concurrent IMAP connections (no POP3). CentOS settings: Added: fs/inotify/max_user_instances = 28000 to: /etc/sysctl.d/99-sysctl.conf Added: * hard nproc 28000 * soft nproc 28000 * hard nofile 28000 * soft nofile 28000 to: /etc/security/limits.d/20-nproc.conf To verify the settings after reboot: cat /proc/sys/fs/inotify/max_user_instances ulimit -Hn ulimit -Sn Dovecot: Added: process_limit = 28000 to service imap{} /etc/dovecot/conf.d/10-master.conf Added (using proxies, would like to just turn off this check): mail_max_userip_connections = 40 to protocol imap {}: /etc/dovecot/conf.d/20-imap.conf Added: process_limit = 28000 to service managesieve {}: /etc/dovecot/conf.d/20-managesieve.conf Hope this helps someone. Feedback would be welcome. It would be great if there was a blessed cookbook for Dovecot scaling. At this point it seems a bit like an art form. Now to my two "easy" questions. 1) TCP replication between our two CentOS boxes has been working great, no complaints. However, I realized I did not know how to check the status of replications, as one might want to do if one of the two boxes was taken offline for maintenance or repair. On the surface, it would appear that the command: doveadm replicator status '*' would likely show me what I want to know. However, when I tried it, I got: doveadm(): Fatal: net_connect_unix(/var/run/dovecot/replicator-doveadm) failed: No such file or directory The documentation says that doveadm assumes the socket /var/run/dovecot/replicator-doveadm and the command format provides a "-a" override. In that directory, there is no "replicator-doveadm" socket, but I do see a "replicator" socket. So, should I be using the command: doveadm replicator status -a /var/run/dovecot/replicator '*' or is the non-existence of the replicator-doveadm socket indicative of something I might have done wrong with the config? I hate experimenting more than I have to with a production box. The 2.2.10 Dovecot I am running was installed via yum, so there are no potential compilation issues. 2) It seems logical to me that: dovecot stop would first lock out any new user connections, then do the equivalent of a: doveadm kick "*" before actually cycling everything else down to ensure the shutdown is as graceful as possible. I suppose I could experiment and find out for certain, but I have the sense that the "stop" command is not quite that elegant, so we have implemented procedures to work around it. Just a point of curiosity. An alternative would be if there was a doveadm command to lock out any new user connections, which could then be followed by kick and stop. I have found that many clients are VERY quick to reconnect after a kick.

2 1

Authcache and user changing
by Lazy 13 Apr '15

13 Apr '15

Hi, I have noticed that during auth cache hits usernames are not updated. (We use ldap backend and change username with user_attrs = uid=user, mailMessageStore=home, mailQuotaSize=quota_rule=*:bytes=%$ cold cache lmtp(14414): Debug: auth input: testmon_testmon home=/vmail/te/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14414): Debug: changed username to testmon_testmon lmtp(14414): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 hot cache lmtp(14715): Debug: auth input: iqmon(a)mon.test.pl home=/vmail/iq/testmon_testmon quota_rule=*:bytes=104857600 lmtp(14715): Debug: Added userdb setting: plugin/quota_rule=*:bytes=104857600 this kills our dictionary based quota (users have multiple quota instances depending on timing and alias the email was delivered to. Is there a way to force addition of user to the auth cache ? -- Michal Grzedzicki

3 5

Need a bit of help with the antispam plugin
by Anthony Papillion 08 Apr '15

08 Apr '15

Hello Everyone, I'm running the antispam plugin on Dovecot 2.0.19 on Ubuntu Server 14.04 and I can't seem to get it to work. In the IMAP section of dovecot.conf I have the following lines: protocol imap { mail_plugins = $mail_plugins imap_quota imap_acl antispam # mail_plugins = $mail_plugins imap_quota imap_acl imap_client_workarounds = tb-extra-mailbox-sep # Maximum number of IMAP connections allowed for a user from each IP address. # NOTE: The username is compared case-sensitively. # Default is 10. # Increase it to avoid issue like below: # "Maximum number of concurrent IMAP connections exceeded" mail_max_userip_connections = 20 plugin { antispam_debug_target = syslog antispam_verbose_debug = 1 antispam_backend = pipe antispam_trash = Trash antispam_spam = Junk antispam_pipe_program_spam_arg = --spam antispam_pipe_program_notspam_arg = --ham antispam_pipe_program = /usr/bin/sa-learn antispam_pipe_program_args = --username=%Lu } } For some reason, when I try to move something to Junk, I get the following in my syslog: Mar 31 18:27:16 anonymail imap: antispam: plugin initialising (2.0-notgit) Mar 31 18:27:16 anonymail imap: antispam: verbose debug enabled Mar 31 18:27:16 anonymail imap: antispam: "Junk" is exact match spam folder Mar 31 18:27:16 anonymail imap: antispam: no unsure folders Mar 31 18:27:16 anonymail imap: antispam: "Trash" is exact match trash folder Mar 31 18:27:16 anonymail imap: antispam: pipe backend spam argument = --spam Mar 31 18:27:16 anonymail imap: antispam: pipe backend not-spam argument = --ham Mar 31 18:27:16 anonymail imap: antispam: pipe backend program = /usr/bin/sa-learn Mar 31 18:27:16 anonymail imap: antispam: pipe backend program arg[0] = --username=anthony(a)cajuntechie.org Mar 31 18:27:16 anonymail imap: antispam: pipe backend tmpdir /tmp Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_unsure(Junk): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_trash(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_trash(Junk): 0 Mar 31 18:27:19 anonymail imap: antispam: mail copy: from trash: 0, to trash: 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_spam(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_spam(Junk): 1 Mar 31 18:27:19 anonymail imap: antispam: mailbox_is_unsure(INBOX): 0 Mar 31 18:27:19 anonymail imap: antispam: mail copy: src spam: 0, dst spam: 1, src unsure: 0 Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program /usr/bin/sa-learn Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program /usr/bin/sa-learn Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program parameter 1 --username=anthony(a)cajuntechie.org Mar 31 18:27:19 anonymail imap: antispam: running mailtrain backend program parameter 2 --spam Mar 31 18:27:21 anonymail imap: antispam: run program failed with exit code -1 This is probably a permissions issue but I'm not able to debug it. Can anyone offer me any clues as to what I might be doing wrong or how I might fix it? Thanks, Anthony -- Anthony Papillion Phone: 1.918.631.7331 VoIP (SIP): 1259010(a)localphone.com XMPP Chat: cypher(a)chat.cpunk.us Fingerprint: 65EF73EC 8B57F6B1 8C475BD4 426088AC FE21B251 PGP Key: http://www.cajuntechie.org/p/my-pgp-key.html

4 16

Deleting empty folders
by Samuel Williams 07 Apr '15

07 Apr '15

So, along with the problem of lots of folders ending in 1 1 1 1, I'd like to have a strategy to delete these. I was trying to understand if it is possible to use http://linux.die.net/man/1/doveadm-expunge to delete empty folders, but without also deleting messages? Is this possible and if so what is the syntax? Kind regards, Samuel

5 10

Dovecot Oy merger with Open-Xchange AG
by Timo Sirainen 03 Apr '15

03 Apr '15

Hi all, Today I can finally announce that Dovecot Oy company has merged with Open-Xchange AG. This helps us to get more Dovecot developers, support people and so on. Most importantly, eventually it should allow me to get back to doing what I like the most: Designing new and interesting stuff for Dovecot and perfecting the old stuff :) OX is a great match to Dovecot going forward. They also really like open source and share our plans for the future. Nothing big will change as a result of this merger: Dovecot will stay Dovecot with its own name and release schedules. We're not going to force OX and Dovecot to be the same product, other than having a somewhat deeper integration between them. Here are the press release links about it: http://www.dovecot.fi/open-xchange-and-dovecot-announce-merger-to-create-wo… http://www.open-xchange.com/dovecot http://www.open-xchange.com/announcements/18

16 32

Install process breaks its self by design
by Merlin at Desktop Masters 02 Apr '15

02 Apr '15

Greetings, I just tried to install Dovecot for the first time. It was on a virgin Ubuntu/Debian server (Version: 1:2.2.9-1ubuntu5) and the install failed. After a bit of struggling I worked out that the install fails BY DESIGN. Spoke about this with a few people and we decided it was best to report this issue and request that you redesign the packing so it does not fail. The reason it fails is as follows... When you install it asks you if you want to use self signed certificates. I have certificates so I said, "No". At the end of the install the script tries to start dovecot. It cannot because the service does not have certificates so it fails. This causes the script to abort and the other packages not to install and breaks the install process. I had to purge dovecot, then install it with the self signed certificates. So we have 2 issues here. 1. Why allow people to not install certificates if it is just going to break the install and confuse them. 2. Why are you starting the service right after install when dovecot clearly requires manual configuration as part of the setup. I recommend you resolve this by NOT starting the service as part of the install unless it is an upgrade and the service was already running. ~ Merlin

2 1

multiple imap servers for high availability
by Robert Fantini 31 Mar '15

31 Mar '15

Hello, We run imap for our domain at our lan. I'd like to set up a backup mirror that has imap storage always duplicated. Then using something like heartbeat to share the imapd IP address . Could someone point me to documents , if any , that give ideas on how to set up the mirror? best regards, Rob

2 2