January 2012 - dovecot

[Dovecot] maildir vs mdbox
by Jean-Daniel Beaubien 29 Jan '12

29 Jan '12

Hi, I am planning on running on test between maildir and mdbox to see which is a better fit for my use case. And I'm just looking for general advice/recommendation. I will post any results I obtain here. Important question: I have multiple users hitting the same email account at the same time. Can be a problem with mdbox? (either via thunderbird or with custom webmail apps). I remember having huge issues with mbox a decade ago because of this. Maildir fixed this... will mdbox reintroduce this problem? This is a very important point for me. Here is my use case: - Ubuntu server (any specific recommandations on FS to use?) - Standard PC hardware (core i5 or i7, few gigs of ram, hdds at first, probably ssd afterwards, nothing very fancy) - Serving only a hand full of email accounts but some of the accounst have over 3 millions emails in them (with individual mail folders having 100k+ emails) - Will use latest dovecot (2.1 when it comes out) - fts-lucene or fts-solr? -jd

2 5

[Dovecot] sieve under lmtp using wrong homedir ?
by Frank Post 29 Jan '12

29 Jan '12

Hi, i have a problem with dovecot-2.0.15. All is working well except lmtp. Sieve scripts are correctly saved under /var/vmail/test.com/test/sieve, but under lmtp sieve will use /var/vmail//testuser/ Uid testuser has mail=test(a)test.com configured in ldap. As i could see in the debug logs, there is a difference between the auth "master out" lines, but why ? working if managesieve stores scripts: Jan 11 15:02:42 auth: Debug: master in: REQUEST 3533701121 23001 1 7ec31d3c65cb934785e8eb0f33a182ae Jan 11 15:02:42 auth: Debug: ldap(test(a)test.com,10.234.201.4): result: mail(user)=test(a)test.com Jan 11 15:02:42 auth: Debug: master out: USER 3533701121 test(a)test.com home=/var/vmail/test.com/test uid=5000 gid=5000 Jan 11 15:02:42 managesieve(test(a)test.com): Debug: Effective uid=5000, gid=5000, home=/var/vmail/test.com/test but under lmtp not: Jan 11 14:39:53 auth: Debug: master in: USER 1 testuser service=lmtp lip=10.234.201.9 rip=10.234.201.4 Jan 11 14:39:53 auth: Debug: auth(testuser,10.234.201.4): username changed testuser -> test(a)test.com Jan 11 14:39:53 auth: Debug: ldap(test(a)test.com,10.234.201.4): result: mail(user)=test(a)test.com Jan 11 14:39:53 auth: Debug: master out: USER 1 test(a)test.com home=/var/vmail//testuser uid=5000 gid=5000 Jan 11 14:39:53 lmtp(8499): Debug: auth input: test(a)test.com home=/var/vmail//testuser uid=5000 gid=5000 Jan 11 14:39:53 lmtp(8499): Debug: changed username to test(a)test.com Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: Effective uid=5000, gid=5000, home=/var/vmail//testuser Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: Quota root: name=User quota backend=maildir args= Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: Quota rule: root=User quota mailbox=* bytes=2147483648 messages=0 Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: Quota warning: bytes=1932735283 (90%) messages=0 reverse=no command=quota-warning 90 test(a)test.com Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: maildir++: root=/var/vmail/test.com/test/Maildir, index=/var/dovecot/indexes/test.com/test, control=, inbox=/var/vmail/test.com/test/Maildir, alt= Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: trash: No trash setting - plugin disabled Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: pla8CymRDU8zIQAAFrfQGQ: sieve: user's script path /var/vmail//testuser/.dovecot.sieve doesn't exist (using global script path in stead) Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: pla8CymRDU8zIQAAFrfQGQ: sieve: user has no valid personal script Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: pla8CymRDU8zIQAAFrfQGQ: sieve: no scripts to execute: reverting to default delivery. Jan 11 14:39:53 lmtp(8499, test(a)test.com) Debug: Namespace : Using permissions from /var/vmail/test.com/test/Maildir: mode=0700 gid=-1 Thanks, for your help. Frank

2 1

Re: [Dovecot] A small bug and a question about CONTEXT=SEARCH
by Anders 29 Jan '12

29 Jan '12

On Tue, 2012-01-10 at 15:05 +0100, Anders wrote: > > the socket connection being closed without warning: > > UID SEARCH RETURN (SAVE COUNT) CHARSET UTF-8 (UNDELETED TEXT "foo") > > You mean it closes with above also? It works fine with me. No, that also works fine here :-) > > UID SEARCH RETURN (COUNT MIN) CHARSET UTF-8 () $ > > This was fixed in v2.0.17. Great, thanks! > > Then I have question about RFC5267 and the announcement of > > CONTEXT=SEARCH > > in the capabilities. I think this RFC is supported by dovecot, or maybe > > just part of the RFC is supported? > > All of it is supported, as far as I know. > > At least when I include the CONTEXT ADDTO or REMOVEFROM keywords I get > > an error, > These are server notifications. Clients aren't supposed to send them. Sorry, apparently I was a bit too fast there. ADDTO and REMOVEFROM should not be sent by a client, but I think that a client can send CONTEXT as a hint to the server, see http://tools.ietf.org/html/rfc5267#section-4.2 Thanks! Regards Anders

2 1

[Dovecot] howto disable indexing on dovecot-lda ?
by Adrian Minta 28 Jan '12

28 Jan '12

Hello, is it possible to disable indexing on dovecot-lda ? Right now postfix delivers the mail directly to the nfs server without any problems. If I switch to dovecot-lda the system crashes do to the high I/O and locking. Indexing on lda is not very useful because the number of of imap logins is less than 5% that of incoming mails, so an user could wait for 3 sec to get his mail index, but a new mail can't. Dovecot version 1.2.15 mail_nfs_storage = yes mail_nfs_index = yes Than you !

3 8

[Dovecot] dovecot 2.0.15 - purge errors
by kjonca＠o2.pl 28 Jan '12

28 Jan '12

Dovecot 2.0.15, debian package, am I lost some mails? How can I check what is in *.broken file? --8<---------------cut here---------------start------------->8--- $doveadm -v purge doveadm(kjonca): Error: Corrupted dbox file /home/kjonca/Mail/0/storage/m.6469 (around offset=291530): purging found mismatched offsets (291500 vs 299692, 60/215) doveadm(kjonca): Warning: mdbox /home/kjonca/Mail/0/storage: rebuilding indexes doveadm(kjonca): Error: Corrupted dbox file /home/kjonca/Mail/0/storage/m.6469 (around offset=599914): metadata header has bad magic value doveadm(kjonca): Warning: dbox: Copy of the broken file saved to /home/kjonca/Mail/0/storage/m.6469.broken doveadm(kjonca): Warning: Transaction log file /home/kjonca/Mail/0/storage/dovecot.map.index.log was locked for 211 seconds doveadm(kjonca): Error: Purging namespace '' failed: Internal error occurred. Refer to server log for more information. [2012-01-13 06:45:07] --8<---------------cut here---------------end--------------->8--- doveconf -n --8<---------------cut here---------------start------------->8--- # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.38+3-64 x86_64 Debian wheezy/sid auth_debug = yes auth_mechanisms = digest-md5 cram-md5 login plain auth_verbose = yes listen = alfa log_path = /var/log/dovecot log_timestamp = "%Y-%m-%d %H:%M:%S " mail_debug = yes mail_location = mdbox:~/Mail/0 mail_log_prefix = "%Us(%u): " mail_plugins = zlib notify acl mail_privileged_group = mail namespace { hidden = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes type = private } namespace { hidden = no inbox = no list = yes location = mbox:~/Mail/Old:CONTROL=~/Mail/.dovecot/control/Old:INDEX=~/Mail/.dovecot/index/Old prefix = "#Old/" separator = / subscriptions = yes type = private } passdb { args = scheme=PLAIN /etc/security/dovecot.pwd driver = passwd-file } plugin { acl = vfile mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size zlib_save = bz2 zlib_save_level = 9 } protocols = imap service auth { user = root } service imap-login { process_limit = 2 process_min_avail = 1 } service imap { vsz_limit = 512 M } service pop3-login { process_limit = 2 process_min_avail = 1 } service pop3 { vsz_limit = 512 M } ssl = no userdb { driver = passwd } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 20 mail_plugins = zlib imap_zlib mail_log notify acl } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { deliver_log_format = msgid=%m: %$ log_path = ~/log/deliver.log postmaster_address = root@localhost } --8<---------------cut here---------------end--------------->8--- -- Gdyby ktoś miał zbędny Toshiba G450 - to chętnie przejmę ;) ---------------- Biologia poucza, ze jeśli cię coś ugryzło, to niemal pewne, ze była to samica.

2 2

[Dovecot] 2.1.rc3 (1a722c7676bb):dsync umlaut problems
by Pascal Volk 28 Jan '12

28 Jan '12

All umlauts in mailbox names are lost after converting mbox/Maildir mailboxes to mdbox. [location2 scp-ed from the old server] # ls -d /srv/import/Maildir/.Gel\&APY-schte\ Elemente/ /srv/import/Maildir/.Gel&APY-schte Elemente/ # dsync -u jane(a)example.com -v mirror maildir:/srv/import/Maildir/ … dsync(jane(a)example.com): Info: Gelöschte Elemente: only in dest … … # doveadm mailbox list -u jane(a)example.com Gel* Gel__schte_Elemente # ls -d mdbox/mailboxes/Gel* mdbox/mailboxes/Gel__schte_Elemente Regards, Pascal -- The trapper recommends today: cafefeed.1201303(a)localdomain.org

2 2

[Dovecot] where is subscribed list stored?
by Michael Makuch 28 Jan '12

28 Jan '12

I'm using $ /usr/sbin/dovecot --version 2.0.15 on $ cat /etc/fedora-release Fedora release 14 (Laughlin) and version 8 of Thunderbird. I use dovecot locally for internal only access to my email archives, of which I have many gigs of email archives. Over time I end up subscribing to a couple dozen different IMAP email folders. Problem is that periodically my list of subscribed folders get zapped to none, and I have to go and re-subscribe to a dozen or two folders again. Anyone seen this happen? It looks like the list of subscribed folders is here ~/Mail/.subscriptions and I can see in my daily backup that it reflects what appears in TBird. What might be zapping it? I use multiple email clients simultaneously on different hosts. (IOW I leave them open) Is this a problem? Does dovecot manage that in some way? Or is that my problem? I don't think this is the problem since this only occurs like a few times per year. If it were the problem I'd expect it to occur much more frequently. Thanks for any clues Mike

5 4

[Dovecot] Quota-warning and setresgid
by l.chelchowski 28 Jan '12

28 Jan '12

Hi! Please help me with this. The problem exists when quota-warning is executing: LOG: Jan 10 10:15:06 lmtp(85973): Debug: none: root=, index=, control=, inbox=, alt= Jan 10 10:15:06 lmtp(85973): Info: Connect from local Jan 10 10:15:06 lmtp(85973): Debug: Loading modules from directory: /usr/local/lib/dovecot Jan 10 10:15:06 lmtp(85973): Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so Jan 10 10:15:06 lmtp(85973): Debug: Module loaded: /usr/local/lib/dovecot/lib90_sieve_plugin.so Jan 10 10:15:06 lmtp(85973): Debug: auth input: tester(a)domain.eu home=/home/vmail/domain.eu/tester/ mail=maildir:/home/vmail/domain.eu/tester/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/public uid=101 gid=12 quota_rule=*:storage=2097 acl_groups= Jan 10 10:15:06 lmtp(85973): Debug: Added userdb setting: mail=maildir:/home/vmail/domain.eu/tester/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/public Jan 10 10:15:06 lmtp(85973): Debug: Added userdb setting: plugin/quota_rule=*:storage=2097 Jan 10 10:15:06 lmtp(85973): Debug: Added userdb setting: plugin/acl_groups= Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Effective uid=101, gid=12, home=/home/vmail/domain.eu/tester/ Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota root: name=user backend=dict args=:proxy::quotadict Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota rule: root=user mailbox=* bytes=2147328 messages=0 Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota rule: root=user mailbox=Trash bytes=+429465 (20%) messages=0 Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota rule: root=user mailbox=SPAM bytes=+429465 (20%) messages=0 Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota warning: bytes=1717862 (80%) messages=0 reverse=no command=quota-warning 80 tester(a)domain.eu Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota warning: bytes=1932595 (90%) messages=0 reverse=no command=quota-warning 90 tester(a)domain.eu Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Quota warning: bytes=2039961 (95%) messages=0 reverse=no command=quota-warning 95 tester(a)domain.eu Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: dict quota: user=tester(a)domain.eu, uri=proxy::quotadict, noenforcing=0 Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/vmail/domain.eu/tester/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/public Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: maildir++: root=/home/vmail/domain.eu/tester, index=/var/mail/vmail/domain.eu/tester(a)domain.eu/index/public, control=, inbox=/home/vmail/domain.eu/tester, alt= Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Namespace : type=public, prefix=Public/, sep=/, inbox=no, hidden=no, list=children, subscriptions=yes location=maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/domain.eu/tester/control/public:INDEX=/var/mail/vmail/domain.eu/tester/index/public:LAYOUT=fs Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: fs: root=/home/vmail/public, index=/var/mail/vmail/domain.eu/tester/index/public, control=/var/mail/vmail/domain.eu/tester/control/public, inbox=, alt= Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: Namespace : type=shared, prefix=Shared/%u/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=maildir:%h/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/shared/%u Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: shared: root=/var/run/dovecot, index=, control=, inbox=, alt= ... Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Debug: quota: Executing warning: quota-warning 95 tester(a)domain.eu Jan 10 10:15:06 lmtp(85973, tester(a)domain.eu) Info: bLUfAJoBDE/VTwEA9hAjDg: sieve: msgid=<4F0C0180.3040704(a)domain.eu>: stored mail into mailbox 'INBOX' Jan 10 10:15:06 lmtp(85973): Info: Disconnect from local: Client quit (in reset) Jan 10 10:15:06 lda: Debug: Loading modules from directory: /usr/local/lib/dovecot Jan 10 10:15:06 lda: Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so Jan 10 10:15:06 lda: Debug: Module loaded: /usr/local/lib/dovecot/lib10_quota_plugin.so Jan 10 10:15:06 lda: Debug: Module loaded: /usr/local/lib/dovecot/lib90_sieve_plugin.so Jan 10 10:15:06 lda: Debug: auth input: tester(a)domain.eu home=/home/vmail/domain.eu/tester/ mail=maildir:/home/vmail/domain.eu/tester/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/public uid=101 gid=12 quota_rule=*:storage=2097 acl_groups= Jan 10 10:15:06 lda: Debug: Added userdb setting: mail=maildir:/home/vmail/domain.eu/tester/:INDEX=/var/mail/vmail/domain.eu/tester@domain.eu/index/public Jan 10 10:15:06 lda: Debug: Added userdb setting: plugin/quota_rule=*:storage=2097 Jan 10 10:15:06 lda: Debug: Added userdb setting: plugin/acl_groups= Jan 10 10:15:06 lda(tester(a)domain.eu): Fatal: setresgid(12(mail),12(mail),101(vmail)) failed with euid=101(vmail): Operation not permitted Jan 10 10:15:06 master: Error: service(quota-warning): child 85974 returned error 75 dovecot -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-RELEASE-p3 amd64 auth_master_user_separator = * auth_mechanisms = plain login cram-md5 auth_username_format = %Lu dict { quotadict = mysql:/usr/local/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 101 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = vmail mail_plugins = " quota acl" mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/%d/%n/control/public:INDEX=/var/mail/vmail/%d/%n/index/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace { list = children location = maildir:%%h/:INDEX=/var/mail/vmail/%d/%u/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } plugin { acl = vfile:/usr/local/etc/dovecot/acls acl_shared_dict = file:/usr/local/etc/dovecot/shared/shared-mailboxes.db autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autocreate5 = Archives autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts autosubscribe5 = Public/Poczta autosubscribe6 = Archives fts = squat fts_squat = partial=4 full=10 quota = dict:user::proxy::quotadict quota_rule2 = Trash:storage=+20%% quota_rule3 = SPAM:storage=+20%% quota_warning = storage=80%% quota-warning 80 %u quota_warning2 = storage=90%% quota-warning 90 %u quota_warning3 = storage=95%% quota-warning 95 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_global_dir = /usr/local/etc/dovecot/sieve sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-userdb { group = mail mode = 0660 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap { executable = imap postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve { drop_priv_before_exec = yes } service pop3 { drop_priv_before_exec = yes } service postlogin { executable = script-login rawlog } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = " acl imap_acl autocreate fts fts_squat quota imap_quota" } protocol lmtp { mail_plugins = quota sieve } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve acl quota postmaster_address = postmaster(a)domain.eu sendmail_path = /usr/sbin/sendmail } -- Łukasz

2 2

[Dovecot] [PATCH] autoconf small fix
by Luca Di Vizio 28 Jan '12

28 Jan '12

Hi Timo, the attached patch seems to solve a warning from autoconf: libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree. Best regards, Luca

2 1

[Dovecot] Password auth scheme question with mysql
by Ed W 28 Jan '12

28 Jan '12

Hi, I have a current auth database using mysql with a "password" column in plain text. The config has "default_pass_scheme = PLAIN" specified In preparation for a more adaptable system I changed a password entry from "asdf" to "{PLAIN}asdf", but now auth fails. Works fine if I change it back to just "asdf". (I don't believe it's a caching problem) What might I be missing? I was under the impression that the password column can include a {scheme} prefix to indicate the password scheme (presumably this also means a password cannot start with a "{"?). Is this still true when using mysql and default_pass_scheme ? Thanks for any hints? Ed W

2 4