October 2016 - dovecot

fix SIS attachment errors
by Webert de Souza Lima 13 Oct '16

13 Oct '16

Hi, I've sent some e-mails about this before but since there was no answers I'll write it differently, with different information. I'm using SIS (Single Instance Attachment Storage). For any reason that is not relevant now, many attachments are missing and the messages can't be fetched: Error: read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1))) failed: read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19 b/l]) failed: open(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36) failed: No such file or directory in this specific case, the /dovecot/attach/bar.example/23/ae/ director doesn't exist. In other cases, just one file is missing so I would assume the hardlink could be recreated and it would work. If I create the missing file (with touch or whatever), I get the following errors: Error: read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19 b/l]) failed: Stream is smaller than expected (0 < 483065) Error: read(attachments-connector(zlib(/dovecot/mdbox/bar.example/foo/storage/m.1))) failed: read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19 b/l]) failed: Stream is smaller than expected (0 < 483065) Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error If I try to fill the file with the amount of bytes it complains about with the following command: $ dd if=/dev/zero of=/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36 bs=1 count=483065 then I get the following error: Error: read(/dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19 b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0) Error: read(attachments-connector(zlib(/srv/dovecot/mdbox/bar.example/foo/storage/m.1))) failed: read(//dovecot/attach/bar.example/23/ae/23aed008c1f32f048afd38d9aae68c5aeae2d17a9170e28c60c75a02ec199ef4e7079cd92988ad857bd6e12cd24cdd7619bd29f26edeec842a6911bb14a86944-fb0b6a214dfa63573c1f00009331bd36[base64:19 b/l]) failed: Stream is larger than expected (483928 > 483065, eof=0) Error: fetch(body) failed for box=INBOX uid=15: BUG: Unknown internal error Based on this I have a few questions: 1. Is there a way, or a tool to scan all mailboxes to get all the messages that have compromised attachments? 2. is there a way to "fix" the missing files (even if creating fake files or removing the attachments information from the messages) 3. What I need is to migrate these boxes using doveadm backup/sync, which fails when these errors occur. Is is possible to ignore them or is there another tool that would do it? Thank you. Webert Lima Belo Horizonte, Brasil

1 1

First steps in Dovecot; IMAP not working
by Marnaud 13 Oct '16

13 Oct '16

Hello, I'm new in Dovecot and am having troubles making it working. I'm trying using Outlook and Apple's Mail as the mail clients. Outlook says it can't establish a secured connection to the server (for the IMAP protocol). I'm guessing sending e-mails works but I can't check. This is my current configuration (using dovecot -n): # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-042stab116.1 x86_64 Debian 8.6 mail_location = mbox:~/mail:INBOX=/var/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { driver = pam } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } } service imap-login { inet_listener imaps { port = 993 ssl = yes } } ssl = no ssl_cert = </etc/ssl/certs/mail.crt ssl_key = </etc/ssl/private/mail.key userdb { driver = passwd } userdb { driver = passwd } I find abnormal I'm seeing "ssl = no" in this configuration despite the fact that I have "ssl = required" in the /etc/dovecot/conf.d/10-ssl.conf file, but I'm new to this... I have looked around the web; finally, I'm asking here, hoping it's the correct place to ask. Arnaud

2 1

SSL error
by arnaud gaboury 13 Oct '16

13 Oct '16

I run dovecot + postfix as my email server. Everything is working as expected, but I see an error in the dovecot logs: lmtp(7331): Error: SSL context initialization failed, disabling SSL: ENGINE_init(dynamic) failed Dovecot is running and emails are OK. I wonder why this error and how I can fix it? I see it is a SSL issue but no idea in which direction to look. Thank you for help

2 4

Strange subscriptions added by dsync backup
by Rik Theys 13 Oct '16

13 Oct '16

Hi, We're in the process of migrating our dovecot 1.x mail server to a Dovecot 2.2.25 server. During the migration I'm moving from mbox storage to mdbox. I use the following method to do a one-way sync from our current mail server to our new mail server (command executed on the new server): doveadm -v -o imapc_host=oldserver \ -o imapc_user=$u \ -o imapc_master_user=$masteruser \ -o imapc_password="$masterpass" \ -o imapc_port=993 -o imapc_ssl=imaps \ -o ssl_client_ca_file=/etc/pki/tls/certs/ca-bundle.crt \ -o imapc_ssl_verify=yes \ -o mail_fsync=never \ -o mail_prefetch_count=20 \ backup -R -u $u imapc: $u is replaced by the username I'm migrating. The dsync command runs and exits with code 0 (all OK). However, when I look at the subscriptions file on the new server, there seem to be additional subscriptions that are not in the subscriptions file on the old server: 7b0d681945d0fc5711560000ffff90ca 7c0d681945d0fc5711560000ffff90ca 7d0d681945d0fc5711560000ffff90ca 7e0d681945d0fc5711560000ffff90ca 7f0d681945d0fc5711560000ffff90ca 800d681945d0fc5711560000ffff90ca Where do these come from? Is something wrong with the migration? It also seems that subscriptions for IMAP folders that only contain subfolders[1] are no longer present in the new subscriptions file: # diff -u sub-old.sorted sub-new.sorted --- sub-old.sorted 2016-10-11 13:42:44.175070610 +0200 +++ sub-new.sorted 2016-10-11 13:47:53.973888462 +0200 @@ -1,26 +1,26 @@ -Archive/Administration/ +7b0d681945d0fc5711560000ffff90ca +7c0d681945d0fc5711560000ffff90ca +7d0d681945d0fc5711560000ffff90ca +7e0d681945d0fc5711560000ffff90ca +7f0d681945d0fc5711560000ffff90ca +800d681945d0fc5711560000ffff90ca Archive/Administration/Conferences Archive/Announcements -Archive/Education/ Archive/Education/E02N3A Archive/Education/I0D51A Archive/Politics -Archive/Research/ -Archive/Research/FET/ Archive/Research/FET/VPH - Virtual Physiological Human Archive/Research/Grants/ICON IBBT Call 2011 Archive/Research/Grants/Marie Curie ITN 2011 Archive/Research/Grants/Odysseus Archive/Research/Grants/SymBioSysII/Funding Archive/Research/Grants/SymBioSysII/JobApplications -Archive/Research/Manuscripts/ Archive/Research/Manuscripts/ruby-ensembl-api Archive/Research/Projects/GUNZ Archive/Research/Projects/MIQAS Archive/Research/Projects/MODY Archive/Research/Projects/ruby-ensembl-api Archives -Archive/Service/ Archive/Service/EditorORC Archive/Service/Reviewing Deleted Messages Is this expected behaviour? Regards, Rik [1] On the old server a folder can only contain either messages or subfolders, not both at the same time. -- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>

1 0

Auto-archiving
by ＠lbutlr 13 Oct '16

13 Oct '16

I’d like to know if there is a way to tell dovecot to 1) move messages older than # days to the Archive folder 2) rebuild the indexes 3) remove any folders that are left with no mail Preferably, I’d like this to be a action I an schedule via crontab or something to fire off for any users that want it. So, I do not want it to do this across all users and mailboxes.

3 5

Disappearing message when move to Inbox subfolder
by Simon 13 Oct '16

13 Oct '16

We are using dovecot 2.0.9 on Centos 6.5 and having the following issues for one single user/single mailbox. When an Outlook (2016) user creates a folder under their Inbox, it does not create in their mailbox (checking with webmail the folder is not there). If they drag a message to that folder in Outlook, then it disappears. Ive searched their mailbox using doveadm and its simply gone. Ive tested with webmail doing essentially the same thing and it works with no issues. Any ideas here would be most well received :) Thanks in advance, Simon

1 0

dsync replication quota2 issue
by George Kontostanos 12 Oct '16

12 Oct '16

Hello list, We are testing a configuration with 2 mail servers using dsync replication (dovecot 2.2.25 ). Everything works fine except the quota2 which is calculated wrong only on one server. Quota2 resides on different databases since each server needs to update it. The problem: The local server always updates quota2 twice on each message it receives. This happens only on one server. Updates run fine on the second. SQL Debug: Query UPDATE quota2 SET bytes=bytes+2108,messages=messages+1 WHERE username = 'user(a)domain.org' Query UPDATE quota2 SET bytes=bytes+2108,messages=messages+1 WHERE username = 'user(a)domain.org' The result on the server that runs fine mysql> select * from quota2; +----------------------------+---------+----------+ | username | bytes | messages | +----------------------------+---------+----------+ | | user(a)domain.org | 2917126 | 17 | The result on the server that has the problem: mysql> select * from quota2; +----------------------------+---------+----------+ | username | bytes | messages | +----------------------------+---------+----------+ | | user(a)domain.org | 2920317 | 19 | dovecot -n is the same on both: root@mx2:/var/log # dovecot -n # 2.2.25 (7be1766): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.15 (97b3da0) # OS: FreeBSD 10.3-RELEASE amd64 ufs auth_mechanisms = plain login auth_verbose = yes default_client_limit = 2560 default_process_limit = 512 dict { acl = mysql:/usr/local/etc/dovecot/dovecot-dict-shares-sql.conf.ext quota = mysql:/usr/local/etc/dovecot/dovecot-dict-quota-sql.conf.ext } doveadm_password = # hidden, use -P to show it doveadm_port = 12345 log_path = /var/log/dovecot.log mail_debug = yes mail_home = /usr/local/vhosts/mail/%d/%n mail_location = maildir:/usr/local/vhosts/mail/%d/%n:LAYOUT=fs mail_max_userip_connections = 70 mail_plugins = quota acl notify replication mail_privileged_group = vmail mail_shared_explicit_inbox = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mbox_write_locks = fcntl namespace { inbox = no list = children location = maildir:/usr/local/vhosts/mail/%%d/%%n:LAYOUT=fs:INDEX=/usr/local/vhosts/indexes/%d/%n/shared/%%u:INDEXPVT=/usr/local/vhosts/indexes/%d/%n/shared/%%u prefix = shared/%%d/%%n/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl mail_replica = tcp:beta.sophimail.com:12345 quota = dict:User quota::proxy::quota quota_rule2 = Trash:storage=+100M sieve = /usr/local/vhosts/mail/%d/%n/.dovecot.sieve sieve_before = /usr/local/vhosts/sieve/before.d/ sieve_dir = /usr/local/vhosts/mail/%d/%n sieve_global_dir = /usr/local/vhosts/sieve/%d sieve_global_path = /usr/local/vhosts/sieve/%d/default.sieve } protocols = imap lmtp sieve sieve service aggregator { fifo_listener replication-notify-fifo { mode = 0666 user = vmail } unix_listener replication-notify { mode = 0666 user = vmail } } service auth-worker { user = vmail } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { mode = 0600 user = vmail } user = dovecot } service config { unix_listener config { user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service doveadm { inet_listener { port = 12345 } user = vmail } service imap-login { inet_listener imap { port = 143 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service replicator { unix_listener replicator-doveadm { mode = 0666 } } ssl_cert = </usr/local/etc/postfix/keys/domain.com.crt ssl_key = </usr/local/etc/postfix/keys/domain.com.key userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = quota acl notify replication sieve notify replication } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = quota acl notify replication imap_quota imap_acl notify replication } protocol lda { mail_plugins = quota acl notify replication sieve acl postmaster_address = root } local 192.168.3.6 { protocol imap { ssl_cert = </usr/local/etc/postfix/keys/domain.com.crt ssl_key = </usr/local/etc/postfix/keys/domain.com.key } } dovecot-dict-quota-sql.conf.ext: connect = host=127.0.0.1 dbname=quota user=mailadmin password=********** map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } Sorry for the lengthy email, any help is very much appreciated. -- George Kontostanos ---

1 1

Detect IMAP server domain name in Dovecot IMAP proxy
by KT Walrus 12 Oct '16

12 Oct '16

I’m in the process of setting up a Dovecot IMAP proxy to handle a number of IMAP server domains. At the current time, I have my users divided into 70 different groups of users (call them G1 to G70). I want each group to configure their email client to access their mailboxes at a domain name based on the group they belong to (e.g., g1.example.com <http://g1.example.com/>, g2.example.com <http://g2.example.com/>, …, g70.example.com <http://g70.example.com/>). I will only support TLS encrypted IMAP connections to the Dovecot IMAP proxy (‘ssl=yes’ in the inet_listener). My SSL cert has alternate names for all 70 group domain names. I want the group domain to only support users that have been assigned to the group the domain name represents. That is, a user assigned to G23 would only be allowed to configure their email client for the IMAP server named g23.example.com <http://g23.example.com/>. My solution during testing has been to have the Dovecot IMAP proxy to listen on different ports: 9930-9999. I plan to purchase 70 IPs, one for each group, and redirect traffic on port 993 to the appropriate Dovecot IMAP proxy port based on the IP I assign to the group domain name in the site’s DNS. The SQL for handling the IMAP login uses the port number of the inet_listener I think this could work in production, but it will cost me extra to rent the 70 IPs and might be a pain to manage. Eventually, I would like to have over 5,000 groups so requiring an IP per group is less than ideal. I also think having Dovecot IMAP proxy have 5,000 inet_listeners might not work so well or might create too many threads/processes/ports to fit on a small proxy server. I would rather have 1 public IP for each Dovecot IMAP proxy and somehow communicate to the userdb which group domain name was configured in the email client so only the users assigned to this group can login with that username. Anyone have any ideas? For HTTP traffic, it is easy to query the host in the HTTP Request, but I don’t think IMAP traffic has such host info in it. Does the Dovecot IMAP proxy receive the hostname from the email client when exchanging SSL certs (like SNI for HTTPS)? Or, maybe I should have group domain in the username used to log in with (e.g., username+g23(a)example.com <mailto:username+g23@example.com> or username(a)g23.example.com <mailto:username@g23.example.com>). I don’t like to make the user configure their email client to log in with a name that is different than their mailbox address. It is simpler to just have them configure their email client with username(a)example.com <mailto:username@example.com> for both authorization and for the from/sender headers in the messages. Anyway, any ideas of how to set this up in production?

3 3

Outlook 2010 woes
by Bryan Holloway 12 Oct '16

12 Oct '16

Hello, everyone. We have recently begun migrating folks from an older server to a newer one, and things have been going quite well except for -- you guessed it -- Outlook 2010 users. Specifically it appears to be one customer in particular, and this particular customer has many nested mailboxes and lots of e-mail in general. Not sure if this is a factor. Old server: * Ubuntu 10.04.4 LTS * Dovecot 2.1.13 * Maildir++ * Local auth via passwd/shadow files New server: * Debian GNU/Linux 8.6 * Dovecot 2.2.13 * Maildir++ * Quotas enabled * LDAP Basically what's happening is that users are seeing large delays when navigating between different IMAP folders. So, for example, user "X" is sitting idle in their INBOX. If they then click on another folder there's a good 6-7 second delay before you can view its contents. If you immediately then navigate to other folders, you get a rapid response. But if the client then goes idle again for 10+ seconds, you will get this delay again. Some are reporting the OS saying "Outlook is not responding." (Everyone is running Windows 7.) Disclaimer: Yes, I know Outlook 2010 is a giant steaming pile of ____. However, everything worked dandy with Dovecot 2.1 and of course my customer is harping on this fact. Things I have tried/read about in the Dovecot list: * Check headers only -- doesn't seem to help. * Had a user completely remove their IMAP profile and re-add: no change. * Had a user set up an entirely new account on a new computer: same symptoms. * Someone mentioned to enable the "delay-newmail" workaround in 20-imap.conf, but this wasn't enabled in our 2.1 install, so that seems like it would not help our case. I was looking through the release notes of later Dovecots, and I noticed that in 2.2.14 there's mention of some issues with Outlook that were fixed, but it wasn't particularly specific about which versions. Should I be looking into that? And if so, is there a separate repository for newer dovecot? Currently the standard Debian 8.6 only has 2.2.13. Any help, suggestions, or pointers would be greatly appreciated. Thank you! - bryan

2 3

Problems with GSSAPI and LDAP
by Juha Koho 12 Oct '16

12 Oct '16

Hello, I have a Dovecot 2.2.25 set up with OpenLDAP back end. I was trying to set up a GSSAPI Kerberos authentication with the LDAP server but with little success. Seems no matter what I try I end up with the following error message: dovecot: auth: Error: LDAP: binding failed (dn (imap/host.example.com(a)EXAMPLE.COM)) Local error, SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (No Kerberos credentials available (default cache: FILE:/tmp/dovecot.krb5.ccache)) I have set the import_environment in dovecot.conf: import_environment = TZ CORE_OUTOFMEM CORE_ERROR LISTEN_PID LISTEN_FDS KRB5CCNAME=FILE:/tmp/dovecot.krb5.ccache And these in LDAP configuration: dn = imap/host.example.com(a)EXAMPLE.COM sasl_bind = yes sasl_mech = gssapi sasl_realm = EXAMPLE.COM sasl_authz_id = imap/host.example.com(a)EXAMPLE.COM I have tried with different values in dn and sasl_authz_id and also leaving them out completely but I always end up with the error message above. Using simple bind without GSSAPI works just fine. The credentials cache file exists and is valid for the principal imap/host.example.com(a)EXAMPLE.COM. The file is owned by dovecot user so it shouldn't be a permission problem either. GSSAPI in OpenLDAP works but I suppose it is irrelevant here since the connection attempt never reaches the LDAP server due to the error. I also have similar setup for Postfix and it works fine. Any ideas what to try next? Best regards, Juha

2 9