From kurt.armistead at eeb5.eu Wed Feb 1 01:16:48 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 00:16:48 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot Message-ID: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Hi guys. Recently we've moved our old (and I mean _really_ old) mailserver to a new hardware. Since I had quite some time on my hands, I decided to switch from courier to dovecot. Transition went pretty smoothly and the users experienced just a few hours' blackout. Only trouble now is that, after re-configuring some IOS handsets and tablets, not all messages are displayed correctly. For example, I had a long existing mailbox (roughly 1600 emails in INBOX and about a dozen other folders) that showed up perfectly on my iphone. Now it's been synced to the new dovecot server via imap and a whole month of messages seems to be missing. I checked on the server, via webmail and with outlook and everything is where it should be. The problem shows up on Apple devices (same thing happened on a couple of ipads). I haven't tried rebuilding the index or wiping out a phone to see if that resolves the problem, but before I do that I'd like to know if anyone experienced the same issue and how they fixed it Thanks! Kurt From tss at iki.fi Wed Feb 1 01:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 01:33:05 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 1.16, Armistead, Kurt wrote: > Recently we've moved our old (and I mean _really_ old) mailserver to a new > hardware. Since I had quite some time on my hands, I decided to switch > from courier to dovecot. Transition went pretty smoothly and the users > experienced just a few hours' blackout. > > Only trouble now is that, after re-configuring some IOS handsets and > tablets, not all messages are displayed correctly. Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely related to clients not realizing that they should drop their internal caches. From f.bonnet at esiee.fr Wed Feb 1 07:55:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Wed, 01 Feb 2012 06:55:48 +0100 Subject: [Dovecot] Temporary forbid some users login ? Message-ID: <4F28D3E4.8040408@esiee.fr> hello is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? I need to move their IMAP folders to another place with more disk space but I don't want to stop dovecot IMAP service for the other users as the moving process will be a bit long ( 1 Tb to move ) thanks From arne at drlinux.no Wed Feb 1 08:58:43 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Wed, 01 Feb 2012 07:58:43 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F28E2A3.3090200@drlinux.no> Den 01. feb. 2012 06:55, skrev Frank Bonnet: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > > > This has been covered before, but if you are using SQL for auth-lookups then you can have something like this ins your query; ... AND active='y' Arne -- Arne K. Haaje - Dr Linux http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From kurt.armistead at eeb5.eu Wed Feb 1 09:48:06 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 08:48:06 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: > On 1.2.2012, at 1.16, Armistead, Kurt wrote: > >> Recently we've moved our old (and I mean _really_ old) mailserver to a >> new >> hardware. Since I had quite some time on my hands, I decided to switch >> from courier to dovecot. Transition went pretty smoothly and the users >> experienced just a few hours' blackout. >> >> Only trouble now is that, after re-configuring some IOS handsets and >> tablets, not all messages are displayed correctly. > > Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new > IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely > related to clients not realizing that they should drop their internal > caches. > I sure did and it reported no issues. Kurt From joseba.torre at ehu.es Wed Feb 1 13:55:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 01 Feb 2012 12:55:44 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F292840.8080907@ehu.es> El 01/02/12 06:55, Frank Bonnet escribi?: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks Take a look to conf.d/auth-deny.conf.ext You can setup a new passdb (a passwd-file can do it) with deny = yes, and add/remove users to that passwd-file as needed. From weber at zackbummfertig.de Wed Feb 1 14:11:11 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:11:11 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: Message-ID: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Is this Howto http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL actual also for Dovecot 2.x ? Cause i read Dovecot 1.x in the Howto Thank you for any hints / tipps Marko (from cold Hamburg Harbour) From jsiegle at psu.edu Wed Feb 1 14:23:10 2012 From: jsiegle at psu.edu (Jonathan Siegle) Date: Wed, 01 Feb 2012 07:23:10 -0500 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Message-ID: <4F292EAE.2030601@psu.edu> Marko Weber said the following on 2/1/12 7:11 AM: > > Is this Howto > http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL > actual also for Dovecot 2.x ? > Cause i read Dovecot 1.x in the Howto > > Thank you for any hints / tipps > My gut feeling is version 1.2 because it was last modified on 2010-06-29 11:37:39(see bottom of page) which is in the 1.2 days(http://www.dovecot.org/oldnews.html). -Jonathan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4316 bytes Desc: S/MIME Cryptographic Signature URL: From janfrode at tanso.net Wed Feb 1 14:29:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 1 Feb 2012 13:29:50 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS Message-ID: <20120201122950.GA23395@dibs.tanso.net> I've been running continous dsync backups of our Maildirs for a few weeks now, with the destination dsync server using mdbox and SIS. The idea was that the destination server would act as a warm copy of all our active users data. The active servers are using Maildir, and has: $ df -h /usr/local/atmail/users/ Filesystem Size Used Avail Use% Mounted on /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users $ df -hi /usr/local/atmail/users/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users very little of this is compressed (zlib plugin enabled during christmas). I'm surprised that the destination server is so large, was expecting zlib and mdbox and SIS would compress it down to much less than what we're seeing (12TB -> 5TB): $ df -h /srv/mailbackup Filesystem Size Used Avail Use% Mounted on /dev/mapper/mailbackupvg-mailbackuplv 5.7T 4.8T 882G 85% /srv/mailbackup Lots and lots of the attachement storage is duplicated into identical files, instead of hard linked. When running "doveadm purge -u $user", we're seeing lots of Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is missing, but there are 205 other copies of this file named /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with identical sha1sum. Also we see corrupted indexes during the purge. This makes me quite uncertain if dsync is a workable backup solution.. or if we can trust mdboxes. Also on the source side, during dsync, we see too many problems. Some samples: Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX Error: command BOX-LIST failed Error: Worker server's mailbox iteration failed Error: read() from worker server failed: EOF Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Error: read() from proxy client failed: EOF Error: Unexpected finish reply: 1 596fec275888dbd89f6d1f5356c22db6 3720 0 \dsync-expunged 0 Error: Unexpected reply from server: 1 12200572a70726fca946da6f9378dc03 3721 0 \dsync-expunged 0 Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda Error: command BOX-LIST failed Error: read() failed: Broken pipe Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x367703c680] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x367703c765] -> /usr/lib64/dovecot/libdovecot.so.0 [0x367703bb93] -> /usr/bin/dsync [0x40f48d] -> /usr/bin/dsync [0x40f589] -> /usr/bin/dsync(dsync_worker_msg_save+0x8e) [0x40eb3e] -> /usr/bin/dsync [0x40d71a] -> /usr/bin/dsync [0x40cdbf] -> /usr/bin/dsync [0x40d105] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x3677047278] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xd5) [0x36770485c5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x367704720d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3677035a83] -> /usr/bin/dsync(main+0x71e) [0x406c4e] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e3941d994] -> /usr/bin/dsync [0x406369] Do you have any idea for what our problems might be? Should we: avoid SIS ? avoid doing Maildir on one side and mdbox on the other? try other dovecot version for dsync? anything else? -jf ------------- destination server, running dovecot v2.0.14 -------- mail_attachment_dir = /srv/mailbackup/attachments mail_location = mdbox:~/mdbox mail_plugins = zlib mdbox_rotate_size = 5 M namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = static } plugin { zlib_save = gz zlib_save_level = 9 } protocols = service auth-worker { user = $default_internal_user } service auth { unix_listener auth-userdb { mode = 0600 user = mailbackup } } ssl = no userdb { args = home=/srv/mailbackup/%256Hu/%d/%n driver = static } -------------/destination server -------- -jf From weber at zackbummfertig.de Wed Feb 1 14:32:47 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:32:47 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <4F292DA5.2000107@Media-Brokers.com> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> <4F292DA5.2000107@Media-Brokers.com> Message-ID: Thank you, i ask because in the first Part is written: At the time of writing this document we use these versions of the required software (probably newer versions also usable): CentOS 4.4 (or RHEL 4 update 4 or Fedora 6) Apache 2.0.52 PHP 5.1.6 MySQL 5.0.22 Dovecot 1.0rc15 And i think , config syntax and organisation of config changed alot from 1.x to 2.x , or ? Am 01.02.2012 13:18, schrieb Charles Marcus: > On 2012-02-01 7:11 AM, Marko Weber wrote: >> >> Is this Howto >> http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL >> actual also for Dovecot 2.x ? > > wiki2. stuff is for dovecot 2.x... > > wiki. stuff is for dovecot 1.x... From tompos at martos.bme.hu Wed Feb 1 14:40:58 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 13:40:58 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ Message-ID: <4F2932DA.4000600@martos.bme.hu> hi! What does patched mean in this repo? Does anybody know? Thank you, tamas From stephan at rename-it.nl Wed Feb 1 14:56:40 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 01 Feb 2012 13:56:40 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F2932DA.4000600@martos.bme.hu> References: <4F2932DA.4000600@martos.bme.hu> Message-ID: <4F293688.1030606@rename-it.nl> Op 2/1/2012 1:40 PM, Papp Tamas schreef: > hi! > > What does patched mean in this repo? > > Does anybody know? This is the v2.1 repository that includes unstable development patches: http://hg.rename-it.nl/dovecot-2.1-patches/ http://hg.rename-it.nl/pigeonhole-0.3-patches/ These are mostly meant for some side-projects I am working on. Previously, some of these patches were included in the normal automatic builds, making debugging difficult since `tainted' dovecot sources were used. The patches also caused a few rather stupid bugs. Therefore, as I announced in a mail a few months back, I started a separate builder for testing these patches. So far, there is no reason for anyone except me to install and try these. Just use the normal dovecot-2.1 repository as documented in the wiki. Regards, Stephan. From tompos at martos.bme.hu Wed Feb 1 15:01:42 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 14:01:42 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F293688.1030606@rename-it.nl> References: <4F2932DA.4000600@martos.bme.hu> <4F293688.1030606@rename-it.nl> Message-ID: <4F2937B6.2010409@martos.bme.hu> On 02/01/2012 01:56 PM, Stephan Bosch wrote: > Op 2/1/2012 1:40 PM, Papp Tamas schreef: >> hi! >> >> What does patched mean in this repo? >> >> Does anybody know? > > This is the v2.1 repository that includes unstable development patches: > > http://hg.rename-it.nl/dovecot-2.1-patches/ > http://hg.rename-it.nl/pigeonhole-0.3-patches/ > > These are mostly meant for some side-projects I am working on. > Previously, some of these patches were included in the normal > automatic builds, making debugging difficult since `tainted' dovecot > sources were used. The patches also caused a few rather stupid bugs. > Therefore, as I announced in a mail a few months back, I started a > separate builder for testing these patches. So far, there is no reason > for anyone except me to install and try these. Just use the normal > dovecot-2.1 repository as documented in the wiki. OK, thank you. tamas From kowalskm at cs.moravian.edu Wed Feb 1 16:09:11 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 09:09:11 -0500 Subject: [Dovecot] locking problem Message-ID: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log I tried killing the imap daemons and removing the lock file, but it keeps coming back. Any idea what else to do? This is the dovecot file: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-28-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem ssl_key_file: /etc/ssl/private/ssl-cert-snakeoil.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix --myron ================================= Myron Kowalski MoCoSIN Network/Systems Administrator Moravian College myron at moravian.edu (610) 861-1441 From gfinch at ldmltd.ca Wed Feb 1 17:41:04 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Wed, 01 Feb 2012 07:41:04 -0800 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F295D10.5010303@ldmltd.ca> On 2012-01-31 9:55 PM, Frank Bonnet wrote: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields see nologin -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From la at iki.fi Wed Feb 1 17:43:20 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 17:43:20 +0200 Subject: [Dovecot] Uncompressing dboxes Message-ID: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Hello. I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. I guess I could convert to mbox, and then back to mdbox, but for certain reasons I'm not very enthusiastic about doing anything mbox-related with dovecot any more. (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Thanks, Lauri From la at iki.fi Wed Feb 1 19:59:05 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 19:59:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? Message-ID: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Hello again. When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Thanks again, Lauri From rob0 at gmx.co.uk Wed Feb 1 21:56:53 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 1 Feb 2012 13:56:53 -0600 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F292840.8080907@ehu.es> References: <4F28D3E4.8040408@esiee.fr> <4F292840.8080907@ehu.es> Message-ID: <20120201195653.GS14195@harrier.slackbuilds.org> On Wed, Feb 01, 2012 at 12:55:44PM +0100, Joseba Torre wrote: > El 01/02/12 06:55, Frank Bonnet escribi?: > > > >is there a way to forbid SOME ( not all ) users's login with > >dovecot 2 ? I need to move their IMAP folders to another place > >with more disk space but I don't want to stop dovecot IMAP > >service for the other users as the moving process will be a > >bit long ( 1 Tb to move ) > > Take a look to conf.d/auth-deny.conf.ext > > You can setup a new passdb (a passwd-file can do it) with deny > = yes, and add/remove users to that passwd-file as needed. Heh, funny, three different answers in this thread and AFAICT they are all correct to some extent. I think the "passdb { deny=yes }" is the best answer. I implemented this in SQL using a tri-state "active" column. Standard "active=1" means the MTA accepts mail and the user can login. "active=0" will disable both. The third state, "active=-1" has the MTA continuing to accept mail, but triggers my deny=yes passdb. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Wed Feb 1 22:11:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:11:40 +0200 Subject: [Dovecot] Uncompressing dboxes In-Reply-To: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> References: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 17.43, Lauri Alanko wrote: > I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. Do something like: dsync backup dsync -o plugin/zlib_save= -o mail=mdbox:/tmp/uncompressed > (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Dovecot supports only read-only compressed mboxes. Also compressed mboxes are always pretty slow, since there's no way to quickly jump around in the file (to go to offset X, you need to uncompress the first X bytes of the mbox file, that's one reason why for mdbox I decided against using full file compression). From tss at iki.fi Wed Feb 1 22:24:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:24:54 +0200 Subject: [Dovecot] locking problem In-Reply-To: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> Message-ID: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> On 1.2.2012, at 16.09, myron wrote: > I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. > > dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log > > I tried killing the imap daemons and removing the lock file, > but it keeps coming back. Any idea what else to do? .. > mail_location: mbox:~/mail:INBOX=/var/mail/%u How large is the /var/mail/ mbox file? From tss at iki.fi Wed Feb 1 22:26:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:26:39 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 9.48, Armistead, Kurt wrote: > On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: >> On 1.2.2012, at 1.16, Armistead, Kurt wrote: >> >>> Recently we've moved our old (and I mean _really_ old) mailserver to a >>> new >>> hardware. Since I had quite some time on my hands, I decided to switch >>> from courier to dovecot. Transition went pretty smoothly and the users >>> experienced just a few hours' blackout. >>> >>> Only trouble now is that, after re-configuring some IOS handsets and >>> tablets, not all messages are displayed correctly. >> >> Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new >> IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely >> related to clients not realizing that they should drop their internal >> caches. > > I sure did and it reported no issues. Then I have no idea. Clients shouldn't have noticed any differences between old and new server. If you find a bug in something from Dovecot's side, let me know. From tss at iki.fi Wed Feb 1 22:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:29:51 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <4F27D981.7060304@noa.gr> References: <4F27D981.7060304@noa.gr> Message-ID: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> On 31.1.2012, at 14.07, Nikolaos Milas wrote: > I am running dovecot-2.0.13-1_128.el5 x86_64 RPM on CentOS 5.7. > > I would like to rename userx with email: userx at example.com to ux at example.com with a mailbox of ux (currently: userx) > > Of course the idea is that new mail will continue to be delivered to the same mailbox, although it has been renamed. > > How can I achieve it? Would it be enough (after changing the associated data in the associated LDAP entry) to simply rename the virtual user directory name, e.g. from /home/vmail/userx to /home/vmail/ux ? Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. If a mail is being delivered at the same time, it could tempfail but will be retried again. (A more complex method would be using dsync, but probably not worth the trouble.) From tss at iki.fi Wed Feb 1 22:36:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:36:11 +0200 Subject: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: References: <33BD52FA-1FE0-46D5-A1E8-9A54C406BE64@iki.fi> Message-ID: On 30.1.2012, at 11.24, Alexis Lelion wrote: >> Maybe you could work around it so that LMTP always proxies the mails, to >> localhost as well, but to a different port which doesn't do proxying at all. > Actually this was my first try, but I had proxying loops because > unlike for IMAP, the LMTP server doesn't seem to support 'proxy_maybe' > option yet, does it? Dovecot v2.1 supports it. From tss at iki.fi Wed Feb 1 22:39:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:39:58 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2656E7.8060501@esiee.fr> References: <4F2656E7.8060501@esiee.fr> Message-ID: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> On 30.1.2012, at 10.37, Frank Bonnet wrote: > We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > format to maildir format. I'd do it one user at a time with dsync, but unfortunately: > the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > another problem is disk space. The users's email data takes about > 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue. From tss at iki.fi Wed Feb 1 22:41:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:41:23 +0200 Subject: [Dovecot] question of different type of Mailbox In-Reply-To: <1327891063.2234.4.camel@localhost.localdomain> References: <1327891063.2234.4.camel@localhost.localdomain> Message-ID: <49D113F9-09BD-41E2-94C6-0647920EA54D@iki.fi> On 30.1.2012, at 4.37, ml wrote: > I just migrate my emails from one type of Maildir to Mailbox > I did as I was having problems reading speed with my webmail. > I did it in order to optimize when do you > > my current config work for me What's the question here? Anyway, mdbox should be faster than either maildir or mbox. From kowalskm at cs.moravian.edu Wed Feb 1 22:43:38 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 15:43:38 -0500 Subject: [Dovecot] locking problem In-Reply-To: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> Message-ID: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> On Feb 1, 2012, at 3:24 PM, Timo Sirainen wrote: > On 1.2.2012, at 16.09, myron wrote: > >> I have one user who can't read his e-mail. The /var/log/mail.log >> file has this statement. >> >> dovecot: IMAP(): Timeout while waiting for lock for >> transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> >> I tried killing the imap daemons and removing the lock file, >> but it keeps coming back. Any idea what else to do? > .. >> mail_location: mbox:~/mail:INBOX=/var/mail/%u > > How large is the /var/mail/ mbox file? -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ Can I just zero out the 3 files in that directory. 6240 Jan 31 13:26 dovecot.index 165888 Jan 31 14:39 dovecot.index.cache 25100 Jan 31 14:42 dovecot.index.log From tss at iki.fi Wed Feb 1 22:46:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:46:03 +0200 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: <4F260EAF.4090408@gedalya.net> References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: On 30.1.2012, at 5.29, Gedalya wrote: > The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > Also, what order does dovecot return the UIDLs in? The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: + pop3: Added support for showing messages in "POP3 order", which can be different from IMAP message order. This can be useful for migrations from other servers. Implemented it for Maildir as 'O' field in dovecot-uidlist. From gedalya at gedalya.net Wed Feb 1 22:50:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 15:50:39 -0500 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: <4F29A59F.8080501@gedalya.net> On 02/01/2012 03:46 PM, Timo Sirainen wrote: > On 30.1.2012, at 5.29, Gedalya wrote: > >> The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. > You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > >> Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. > Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > >> Also, what order does dovecot return the UIDLs in? > The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: > > + pop3: Added support for showing messages in "POP3 order", which can > be different from IMAP message order. This can be useful for > migrations from other servers. Implemented it for Maildir as 'O' > field in dovecot-uidlist. > > Thank you so much! This clears up a lot of fine details. From gedalya at gedalya.net Wed Feb 1 23:03:17 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 16:03:17 -0500 Subject: [Dovecot] Site-wide configurable Sieve Message-ID: <4F29A895.5030007@gedalya.net> Hi, I need some sieve actions to be specified per-user, but exposing actual sieve scripts to users is of the question. For example, I need to be able to either put spam messages into a spam folder, or discard them, based on user preference. Presumably this would be done using Sieve. As I understand, it's not possible to query a database from within a sieve script. If I can't use a single global script, I can imagine automatically generating per-user scripts when settings are saved in a GUI (e.g. webmail) and store them on the dovecot machine. This would work but I would love to hear if someone has a smarter, more robust idea. Thanks, Gedalya From tss at iki.fi Wed Feb 1 23:11:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 23:11:17 +0200 Subject: [Dovecot] locking problem In-Reply-To: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> Message-ID: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> On 1.2.2012, at 22.43, myron wrote: >>> dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> How large is the /var/mail/ mbox file? > > > -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ OK, so not very big. > Can I just zero out the 3 files in that directory. > > 6240 Jan 31 13:26 dovecot.index > 165888 Jan 31 14:39 dovecot.index.cache > 25100 Jan 31 14:42 dovecot.index.log You can delete them, but somehow I doubt that helps.. That error message anyway happens when a session has opened the mbox and is "syncing" it. This normally is a pretty fast operation, but the error message indicates that another session saw the first session doing it for over 3 minutes until it aborted. Do you have hanging imap processes eating 100% CPU? That could explain it if Dovecot doesn't like something in the mbox file. Also there may have been some other reasons for this that are already fixed in newer versions. From alessio at skye.it Thu Feb 2 09:46:55 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 02 Feb 2012 08:46:55 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <4F2A3F6F.6000604@skye.it> Il 01/02/2012 13:29, Jan-Frode Myklebust ha scritto: > I've been running continous dsync backups of our Maildirs for a few > weeks now, with the destination dsync server using mdbox and SIS. The > idea was that the destination server would act as a warm copy of > all our active users data. How many users there are in this installation? > The active servers are using Maildir, and has: > > $ df -h /usr/local/atmail/users/ > Filesystem Size Used Avail Use% Mounted on > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > $ df -hi /usr/local/atmail/users/ > Filesystem Inodes IUsed IFree IUse% Mounted on > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > very little of this is compressed (zlib plugin enabled during christmas). This is the old storage in Maildir format? > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): > > $ df -h /srv/mailbackup > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/mailbackupvg-mailbackuplv > 5.7T 4.8T 882G 85% /srv/mailbackup This is the new storage in mdbox format? What size you would expect? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From nmilas at noa.gr Thu Feb 2 10:17:24 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Thu, 02 Feb 2012 10:17:24 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> References: <4F27D981.7060304@noa.gr> <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> Message-ID: <4F2A4694.1070904@noa.gr> On 1/2/2012 10:29 ??, Timo Sirainen wrote: > Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. Thanks Timo, In fact, I could have the luxury to bring down the mail server (Postfix and Dovecot) for a couple of minutes and complete the changes. There is a mail gateway device queuing for delivery when the mail server is down. Nick From f.bonnet at esiee.fr Thu Feb 2 10:53:00 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 09:53:00 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> Message-ID: <4F2A4EEC.8020207@esiee.fr> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > On 30.1.2012, at 10.37, Frank Bonnet wrote: > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >> format to maildir format. > I'd do it one user at a time with dsync, but unfortunately: > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > >> another problem is disk space. The users's email data takes about >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >> the raid array of the server. > With one-user-at-a-time conversion this wouldn't be an issue. > so is there a massive conversion method ? thanks From janfrode at tanso.net Thu Feb 2 10:53:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 09:53:51 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <4F2A3F6F.6000604@skye.it> References: <20120201122950.GA23395@dibs.tanso.net> <4F2A3F6F.6000604@skye.it> Message-ID: <20120202085351.GA18559@dibs.tanso.net> On Thu, Feb 02, 2012 at 08:46:55AM +0100, Alessio Cecchi wrote: > > How many users there are in this installation? Quite a few :-) This is for an ISP. > >The active servers are using Maildir, and has: > > > > $ df -h /usr/local/atmail/users/ > > Filesystem Size Used Avail Use% Mounted on > > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > > $ df -hi /usr/local/atmail/users/ > > Filesystem Inodes IUsed IFree IUse% Mounted on > > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > > >very little of this is compressed (zlib plugin enabled during christmas). > > This is the old storage in Maildir format? Correct. > > >I'm surprised that the destination server is so large, was expecting zlib and > >mdbox and SIS would compress it down to much less than what we're seeing > >(12TB -> 5TB): > > > > $ df -h /srv/mailbackup > > Filesystem Size Used Avail Use% Mounted on > > /dev/mapper/mailbackupvg-mailbackuplv > > 5.7T 4.8T 882G 85% /srv/mailbackup > > This is the new storage in mdbox format? Correct. > What size you would expect? With Maildir I see message-files shrink to about 20%* of original size after turning on zlib with zlib_save_level=6. I was expecting better compression with mdbox (and zlib_save_level=9), and I would expect SIS to help even further. mdbox+SIS+zlib_save_level=9 variant taking up 40% the space of a mixed** compressed/non-compressed Maildir storage isn't very impressive to me -- and the mdbox backup isn't even complete (it's only the 25% most active users). Yes, I see there might be holes in my logic, expecting compressed messages to compress further after move to mdbox. But also I have expectation that most of the messages are not already compressed on the Maildir side. Sorry, expectations and guesses, not hard facts. [*] based on a couple of samples, not thourough research [**] Only messages saved after we enabled zlib on December 25. are compressed. -jf From tss at iki.fi Thu Feb 2 11:20:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 11:20:35 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2A4EEC.8020207@esiee.fr> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> Message-ID: <1328174435.11500.105.camel@innu> On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: > Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > > On 30.1.2012, at 10.37, Frank Bonnet wrote: > > > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > >> format to maildir format. > > I'd do it one user at a time with dsync, but unfortunately: > > > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > > > >> another problem is disk space. The users's email data takes about > >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > >> the raid array of the server. > > With one-user-at-a-time conversion this wouldn't be an issue. > > > > so is there a massive conversion method ? dsync works for that too: 1. dsync backup all users somewhere 2. wait close to the time you can shut down dovecot 3. dsync backup all users the second time 4. shutdown dovecot and MTA 5. dsync backup once more (hopefully this will be relatively quick) The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space.. From f.bonnet at esiee.fr Thu Feb 2 11:33:45 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 10:33:45 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328174435.11500.105.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> Message-ID: <4F2A5879.3090509@esiee.fr> Le 02/02/2012 10:20 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: >> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : >>> On 30.1.2012, at 10.37, Frank Bonnet wrote: >>> >>>> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >>>> format to maildir format. >>> I'd do it one user at a time with dsync, but unfortunately: >>> >>>> the machine use real users thru openldap ( pam_ldap + nss_ldap ) >>> nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. >>> >>>> another problem is disk space. The users's email data takes about >>>> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >>>> the raid array of the server. >>> With one-user-at-a-time conversion this wouldn't be an issue. >>> >> so is there a massive conversion method ? > dsync works for that too: > > 1. dsync backup all users somewhere > 2. wait close to the time you can shut down dovecot > 3. dsync backup all users the second time > 4. shutdown dovecot and MTA > 5. dsync backup once more (hopefully this will be relatively quick) > > The idea is that you can do incremental dsync backups. You'll of course > need to do this in a way that you don't run out of disk space.. > Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ? see below the output of doveconf -n thanks mail# doveconf -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.4-RELEASE amd64 auth_verbose_passwords = plain auth_worker_max_count = 100 debug_log_path = /var/log/dovecot/debug.log default_client_limit = 6144 default_process_limit = 1024 default_vsz_limit = 1 G disable_plaintext_auth = no info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/dovecot.log mail_location = mbox:%h/:INDEX=/raid/cache/dovecot/index/%u:INBOX=/raid/var/mail/%u:CONTROL=%h/ mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_lock_timeout = 3 mins mbox_write_locks = fcntl mmap_disable = yes passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at esiee.fr protocols = imap pop3 sendmail_path = /usr/local/sbin/sendmail service auth { client_limit = 7169 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 service_count = 1 vsz_limit = 3 G } service imap { process_limit = 2048 vsz_limit = 3 G } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/xxx.pem ssl_cert = References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> Message-ID: <1328177422.11500.108.camel@innu> On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: > > dsync works for that too: > > > > 1. dsync backup all users somewhere > > 2. wait close to the time you can shut down dovecot > > 3. dsync backup all users the second time > > 4. shutdown dovecot and MTA > > 5. dsync backup once more (hopefully this will be relatively quick) > > > > The idea is that you can do incremental dsync backups. You'll of course > > need to do this in a way that you don't run out of disk space.. > > > > Well I have NFS access to a Netapp filer with a LOT of disk space > I need some modifications in my dovecot configuration to use NFS > I think ? No need for any modifications. As long as you don't have multiple Dovecots in different servers accessing the same mails via NFS, there's no trouble. > mmap_disable = yes Looks like you also have this already enabled. This is recommended (not required) with NFS, but if you're not normally using NFS at all you shouldn't enable this. From tss at iki.fi Thu Feb 2 12:23:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 12:23:01 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <1328178181.11500.119.camel@innu> On Wed, 2012-02-01 at 13:29 +0100, Jan-Frode Myklebust wrote: > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): Note that with SIS the attachments aren't compressed. > Lots and lots of the attachement storage is duplicated into identical files, > instead of hard linked. Something's wrong then. > When running "doveadm purge -u $user", we're seeing lots of > > Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory Something's wrong. > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > missing, but there are 205 other copies of this file named > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > identical sha1sum. All of them have a link count of 2, with the other link being in hashes/ directory? > Also on the source side, during dsync, we see too many problems. That is most likely related to your troubles. If the dsync runs crash, the result could leave extra files lying around etc.. > Some samples: > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX This is a little bit strange. What is the doveconf -n output of the source server? > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Is this a namespace prefix? It shouldn't be trying to sync a mailbox named this (there's an extra "." suffix). > Error: read() from proxy client failed: EOF I guess the remote dsync crashes or otherwise aborted. > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda I guess some kind of mismatch related to namespace configuration. > Error: read() failed: Broken pipe > Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Probably can't handle properly when remote dsync dies. Of course it still shouldn't crash. There seems to be some bugs left when dsyncing to a remote host (instead of locally). It would help if I could reproduce the errors that you're seeing. Can you easily reproduce them with some accounts? If so, if you can give enough details for me to reproduce the problems I can fix them. (Except for the "file not found" issues, since that problems occurred earlier already. I should probably somehow make Dovecot fix those missing files though..) From janfrode at tanso.net Thu Feb 2 13:31:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:31:20 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <1328178181.11500.119.camel@innu> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> Message-ID: <20120202113120.GA25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:23:01PM +0200, Timo Sirainen wrote: > > Note that with SIS the attachments aren't compressed. Yes, I know. > > > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > > missing, but there are 205 other copies of this file named > > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > > identical sha1sum. > > All of them have a link count of 2, with the other link being in hashes/ > directory? No, these has link count=207. I don't know what you mean by link being in hashes directory. # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|head -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0069222e0c080f4f754a0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-00ffb9312a370e4f6b610000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0442c5163ad3114fb4780000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-04f288390052144f012d0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-053b6c0f185a0d4fc4210000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-06c98213c3b30e4fac3c0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-076573234fbd0b4fa8620000bf34f6ab This is just one example, I can provide tons of other examples.. Hmm, I see now that there are 206 files of that first example with the 207 links, and here's another other example with numlinks=7: # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|wc -l 206 and numlinks=4: # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l 3 is dovecot somehow creating numlinks+1 copies of every file it hardlinks?? Would explain my diskusage :-) > That is most likely related to your troubles. If the dsync runs crash, > the result could leave extra files lying around etc.. If dsync backup is supposed to be a viable backup solution, I think it should fail much better. If it see errors on the target side it should clear the target and do a full sync. Manually cleaning up after it's problems is too much work. > > > Some samples: > > > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > This is a little bit strange. What is the doveconf -n output of the > source server? ---------------------------------------------------------------------------- # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server # release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 mail_gid = 3000 mail_home = /srv/mailstore/%256RHu/%d/%n mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 mail_plugins = quota zlib mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve sieve_dir = /sieve/%1u/%1.1u/%u sieve_max_script_size = 1M zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib imap_quota } protocol pop3 { mail_plugins = quota zlib pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ---------------------------------------------------------------------------- > > > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > > Is this a namespace prefix? It shouldn't be trying to sync a mailbox > named this (there's an extra "." suffix). I believe it's a folder named "INBOX.ferie 2006.", with the user using the namespace separator in the folder name.. I believe dovecot allows this, so it should also handle backing it up. > > > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda > > I guess some kind of mismatch related to namespace configuration. They both have same namespace config I think, but Maildir vs. mdbox maybe changes something.. > > It would help if I could reproduce the errors that you're seeing. Can > you easily reproduce them with some accounts? If so, if you can give > enough details for me to reproduce the problems I can fix them. (Except > for the "file not found" issues, since that problems occurred earlier > already. I should probably somehow make Dovecot fix those missing files > though..) I'll look trough the logs to see if there are any errors that keep repeating for the same accounts. -jf From janfrode at tanso.net Thu Feb 2 13:51:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:51:03 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <20120202115102.GB25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:31:20PM +0100, Jan-Frode Myklebust wrote: > > and numlinks=4: > > # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l > 3 > > is dovecot somehow creating numlinks+1 copies of every file it > hardlinks?? Would explain my diskusage :-) > Sorry, brainfart.. Yes, these are hardlinks to the same inode.. # ls -i c31beb42ef78810f7fb81a7086144034fb0fd794* ../c31beb42ef78810f7fb81a7086144034fb0fd794* 2422693 c31beb42ef78810f7fb81a7086144034fb0fd794 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-13b405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-1cb405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-4eb405342e24284f61530000bf34f6ab -jf From tss at iki.fi Thu Feb 2 14:41:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 2 Feb 2012 14:41:11 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> On 2.2.2012, at 13.31, Jan-Frode Myklebust wrote: >>> "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is >>> missing, but there are 205 other copies of this file named >>> /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with >>> identical sha1sum. >> >> All of them have a link count of 2, with the other link being in hashes/ >> directory? > > No, these has link count=207. OK, so they aren't actual copies, they are links to the same file. > I don't know what you mean by link being in hashes directory. If you have e.g. aa/bb/aabbccdd-eeee file, there should be a matching aa/bb/hashes/aabbccdd file. >> That is most likely related to your troubles. If the dsync runs crash, >> the result could leave extra files lying around etc.. > > If dsync backup is supposed to be a viable backup solution, I think it > should fail much better. If it see errors on the target side it should > clear the target and do a full sync. Manually cleaning up after it's > problems is too much work. Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX What about: doveadm mailbox status -u user at domain guid '*' in source server? in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name >> >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox >> named this (there's an extra "." suffix). > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > the namespace separator in the folder name.. I believe dovecot allows > this, so it should also handle backing it up. It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: a select foo. a NO [CANNOT] Invalid mailbox name From kowalskm at cs.moravian.edu Thu Feb 2 15:45:46 2012 From: kowalskm at cs.moravian.edu (myron) Date: Thu, 2 Feb 2012 08:45:46 -0500 Subject: [Dovecot] locking problem In-Reply-To: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> Message-ID: <5C6469E0-FAC8-45EB-909B-1A58FC3DBE9E@cs.moravian.edu> On Feb 1, 2012, at 4:11 PM, Timo Sirainen wrote: > On 1.2.2012, at 22.43, myron wrote: > >>>> dovecot: IMAP(): Timeout while waiting for lock for >>>> transaction log file /users//mail/.imap/INBOX/ >>>> dovecot.index.log >>> How large is the /var/mail/ mbox file? >> >> >> -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ > > OK, so not very big. His isn't, but I have users with 500MB to a GB mailbox. Could they be the cause of the problem? > >> Can I just zero out the 3 files in that directory. >> >> 6240 Jan 31 13:26 dovecot.index >> 165888 Jan 31 14:39 dovecot.index.cache >> 25100 Jan 31 14:42 dovecot.index.log > > You can delete them, but somehow I doubt that helps.. That error > message anyway happens when a session has opened the mbox and is > "syncing" it. This normally is a pretty fast operation, but the > error message indicates that another session saw the first session > doing it for over 3 minutes until it aborted. > > Do you have hanging imap processes eating 100% CPU? Hardly anything: ps -eo pcpu,pid,user,args | sort -k 1 -r | head -20 %CPU PID USER COMMAND 0.0 9 root [events/0] 0.0 993 root /usr/sbin/sshd -D 0.0 984 root /usr/sbin/cupsd -C /etc/cups/cupsd.conf 0.0 9677 imap 0.0 9659 dovecot imap-login 0.0 959 root [ext4-dio-unwrit] 0.0 958 root [ext4-dio-unwrit] 0.0 957 root [jbd2/dm-1-8] 0.0 9442 root [lockd] 0.0 939 root [ext4-dio-unwrit] 0.0 938 root [ext4-dio-unwrit] 0.0 937 root [jbd2/dm-3-8] 0.0 930 root [ext4-dio-unwrit] 0.0 929 root [ext4-dio-unwrit] 0.0 928 root [jbd2/dm-2-8] 0.0 9192 root /usr/sbin/apache2 -k start 0.0 911 root [ext4-dio-unwrit] 0.0 910 root [ext4-dio-unwrit] 0.0 90 root [kconservative/1] > That could explain it if Dovecot doesn't like something in the mbox > file. Also there may have been some other reasons for this that are > already fixed in newer versions. I looked at the upgrade list on the server and dovecot isn't showing any upgrades via atp-get From tss at iki.fi Thu Feb 2 15:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 15:54:24 +0200 Subject: [Dovecot] LDAP auth improvements Message-ID: <1328190864.11500.130.camel@innu> v2.1 supports now multiple LDAP fields in a template e.g.: user_attrs = \ homeDirectory=home, \ uidNumber=uid, \ gidNumber=gid, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} which is the same as: user_attrs = \ =home=%{ldap:homeDirectory}, \ =uid=%{ldap:uidNumber}, \ =gid=%{ldap:gidNumber}, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} I was also thinking about adding an alternative simplified syntax for this: user_attrs { home = %{ldap:homeDirectory} uid = %{ldap:uidNumber} gid = %{ldap:gidNumber} mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} } Any ideas for further improvements before I do this change? One thing I'm still wondering about is what I should do when LDAP returns multiple values. Like perhaps: acl_groups = %{ldap:aclGroups:,} Which would mean that all the aclGroups values would be joined together separated by "," characters. And a bit more complex with multiple gidNumbers: gid = %{ldap:gidNumber[0]} mail_access_groups = %{ldap:gidNumber[1:]:,} From f.bonnet at esiee.fr Thu Feb 2 18:22:03 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 17:22:03 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328177422.11500.108.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> <1328177422.11500.108.camel@innu> Message-ID: <4F2AB82B.1000200@esiee.fr> Le 02/02/2012 11:10 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: >>> dsync works for that too: >>> >>> 1. dsync backup all users somewhere >>> 2. wait close to the time you can shut down dovecot >>> 3. dsync backup all users the second time >>> 4. shutdown dovecot and MTA >>> 5. dsync backup once more (hopefully this will be relatively quick) >>> >>> The idea is that you can do incremental dsync backups. You'll of course >>> need to do this in a way that you don't run out of disk space.. >>> >> Well I have NFS access to a Netapp filer with a LOT of disk space >> I need some modifications in my dovecot configuration to use NFS >> I think ? > No need for any modifications. As long as you don't have multiple > Dovecots in different servers accessing the same mails via NFS, there's > no trouble. > >> mmap_disable = yes > Looks like you also have this already enabled. This is recommended (not > required) with NFS, but if you're not normally using NFS at all you > shouldn't enable this. > OK thank you From dovecot at asheesh.org Thu Feb 2 21:59:58 2012 From: dovecot at asheesh.org (Asheesh Laroia) Date: Thu, 02 Feb 2012 14:59:58 -0500 Subject: [Dovecot] dsync deleting too many emails (sdbox) Message-ID: <1328212183-sup-560@rose.makesad.us> I'm using dsync to synchronize emails on a laptop where wifi connectivity sometimes fails in the middle of a sync. I have a shell script that runs dsync, and here is one line of it including the output of dsync: + dsync -f -m realmail mirror /home/paulproteus/projects/ssh-attach/run ssh rose.makesad.us dsync dsync-local(paulproteus): Error: dbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Unexpectedly lost uid=337460 dsync-local(paulproteus): Error: msg guid lookup failed: Internal error occurred. Refer to server log for more information. [2012-02-02 11:02:12] dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Rebuilding index dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 130608.broken dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 66159.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 336269.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 dsync-remote(paulproteus): Error: proxy server timed out dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 dsync-local(paulproteus): Warning: Transaction log file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/dovecot.index.log was locked for 1528 seconds That seemed problematic, but not dangerous. Then I ran a fresh sync, and found 3000 (of 60,000) messages had been deleted and expunged from the "realmail" box. I'm guessing this is some bad interaction with sdbox and partial file downloads? I haven't read the code for this, but I would guess the dsync process isn't being atomic about file transfers, so it is leaving half-completed transfers in place, which results in corrupt files when they're next examined. dovecot -n output: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-2-amd64 x86_64 Debian wheezy/sid mail_location = sdbox:~/Maildir/dbox passdb { driver = pam } protocols = " imap" ssl_cert = Hi This is my first post here so i will be as precise as i can I'm trying to setup a quota system using dovecot postfix and procmail dovecot version 1.2.9 postfix version 2.7.0 I have succesfully modified dovecot config to use fs quota system (so far imap and pop3) It reports correct quotas to imap and pop3 clients (both user and group) What im tryin to do now is to modify postfix->procmail->dovecot config in a way that if user is over quota mail delivery is delayed instead of bouncing. (is this possible?) Second thing i would like to achive is that when authenticated users close to quota/group quota for example 10MB to quota limit try sending email theyr mail is rejected and preferably an email is generated telling them to free some space for new mails first. (is such a thing possible?) MS Exchange servers have such a feature and I would like to get such a functionality on my linux mailserver I'm trying to prevent a situation when mails are lost due to quota problems. This setup was initially a typical virtualmin installation with a system on single partition. Thanks in advance for all Your suggestions From alessio at skye.it Fri Feb 3 09:20:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 03 Feb 2012 08:20:50 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <4F2B8AD2.5010905@skye.it> Il 02/02/2012 23:58, przemek.orzechowski at makolab.pl ha scritto: > > Hi > > This is my first post here so i will be as precise as i can > > I'm trying to setup a quota system using dovecot postfix and procmail > dovecot version 1.2.9 > postfix version 2.7.0 > > I have succesfully modified dovecot config to use fs quota system (so far > imap and pop3) > It reports correct quotas to imap and pop3 clients (both user and group) > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Yes, you can do it but with Dovecot LDA instead of procmail. See the options "quota_full_tempfail" for Dovecot LDA. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Fri Feb 3 09:36:04 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 08:36:04 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <20120203073604.GA6561@dibs.tanso.net> On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Check the quota_full_tempfail setting, http://wiki.dovecot.org/MainConfig > > Second thing i would like to achive is that when authenticated users close > to quota/group quota for example 10MB to quota limit > try sending email theyr mail is rejected and preferably an email is > generated telling them to free some space for new mails first. > (is such a thing possible?) Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 -jf From kirill at shutemov.name Fri Feb 3 13:20:49 2012 From: kirill at shutemov.name (Kirill A. Shutemov) Date: Fri, 3 Feb 2012 13:20:49 +0200 Subject: [Dovecot] Feature request - search and tag protocol - notmuch-alike approach Message-ID: <20120203112049.GA19701@shutemov.name> Hi all, notmuch[1] becomes more and more popular. At least in geek community. But notmuch is local-only (except ssh)... It looks like it's hard (if even possible) to use IMAP as transport for notmuch-alike work flow. I think it would be great to create a new protocol based global search and message tagging. All core notmuch operations (search, show, count, tag) can be mapped to the new protocol (more or less) directly. Search syntax can be reused as is, I think. The tricky part is initial tagging. It requires some kind of transaction to be able to tag new messages without racing with upcoming messages. Request (command) syntax can be similar to unix command. This way we can reuse notmuch syntax even deeper. Like: search --limit 10 --sort=oldest-first tag:to-me tag:unread Replies are often structured. I think it's reasonable to use JSON for all replies. Any ideas? I've started to write a proof-of-concept, but it moves too slow, since my knowledge of dovecot internals is poor and it's hard to find time for the project. I hope somebody else has similar idea in mind and just wait opportunity to discuss the idea and start to write code. ;) [1] http://notmuchmail.org/ -- Kirill A. Shutemov From janfrode at tanso.net Fri Feb 3 14:25:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:25:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> Message-ID: <20120203122552.GA12451@dibs.tanso.net> On Wed, Jan 18, 2012 at 09:03:18PM +0200, Timo Sirainen wrote: > > > > I think the way I originally planned LMTP proxying to work is simply too > > complex to work reliably, perhaps even if the code was bug-free. So > > instead of reading+writing DATA at the same time, this patch changes the > > DATA to be first read into memory or temp file, and then from there read > > and sent to the LMTP backends: > > > > http://hg.dovecot.org/dovecot-2.1/raw-rev/51d87deb5c26 > > > > ----8<----8<----8<-----8<-----8<-----8<----8<-----8<----8<----8<-- > > > > unfortunately I haven't tested that patch, so I have no idea if it > > fixed the issues or not... > > I'm not sure if that patch is useful or not. The important patch to fix it is http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c I now implemented this patch on our directors, and pointed postfix at them. No problem seen so far, but I'm still a bit uncertain about the LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite large delays when fsync'ing (slow IMAP APPEND). Do you think increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start seeing lmtp_proxy_output_timeout problems again ? -jf From janfrode at tanso.net Fri Feb 3 14:42:16 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:42:16 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> Message-ID: <20120203124215.GA13495@dibs.tanso.net> On Thu, Feb 02, 2012 at 02:41:11PM +0200, Timo Sirainen wrote: > > >> That is most likely related to your troubles. If the dsync runs crash, > >> the result could leave extra files lying around etc.. > > > > If dsync backup is supposed to be a viable backup solution, I think it > > should fail much better. If it see errors on the target side it should > > clear the target and do a full sync. Manually cleaning up after it's > > problems is too much work. > > Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. I'm reducing the complexity now, removing SIS and starting the backups from scratch again. I'll start posting the problems I see over the weekend.. > > >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > What about: > > doveadm mailbox status -u user at domain guid '*' > > in source server? INBOX guid=08b46439069d3d4db0490000e671bf84 INBOX.Sent guid=e8f6e431bf6e014f2d780000e671bf84 INBOX.Trash guid=c858f2234a1d5d4e1547000058d3d19f INBOX.Drafts guid=e9f6e431bf6e014f2d780000e671bf84 INBOX.Spam guid=eaf6e431bf6e014f2d780000e671bf84 INBOX.Sent Messages guid=d837512bed7d674e685c000058d3d19f INBOX.INBOX.Sent Messages guid=ebf6e431bf6e014f2d780000e671bf84 INBOX.Notes guid=c0d2250109645e4eed5c000058d3d19f > in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. Scratched dest server before I replied.. sorry. > > >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > >> > >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox > >> named this (there's an extra "." suffix). > > > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > > the namespace separator in the folder name.. I believe dovecot allows > > this, so it should also handle backing it up. > > It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: Oh, sorry.. then this is a problem created by @mail, which poked directly in the filesystem. Guess we'll have to clean these up manually. -jf From a23s4a2008 at yandex.ru Fri Feb 3 15:15:05 2012 From: a23s4a2008 at yandex.ru (User Max) Date: Fri, 03 Feb 2012 17:15:05 +0400 Subject: [Dovecot] openbsd and dovecot Message-ID: <725061328274905@web20.yandex.ru> I'm setting up dovecot in OpenBSD OS. The problem is that I can't set up dovecot to use kerberos authentication via bsdauth which is "similar to pam" I try to test dovecot via telnet: telnet localhost pop3 user test +OK pass password -ERR [IN-USE] Temporary authentication failure and some strings from logs Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted Feb 3 15:02:46 srv-mx-00 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Feb 3 15:29:13 srv-mx-00 dovecot: auth: Error: bsdauth(test,127.0.0.1): getpwnam() failed: Operation not permitted What am I doing wrong? Here is the output of dovecot -n auth_krb5_keytab = /etc/kerberosV/krb5.keytab auth_mechanisms = plain login default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 2000 first_valid_uid = 2000 last_valid_gid = 2000 last_valid_uid = 2000 mail_gid = vmail mail_location = maildir:/var/vmail/%u%Ud/Maildir mail_uid = vmail mbox_write_locks = fcntl mmap_disable = yes passdb { driver = bsdauth } service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0666 user = _postfix } } ssl_cert = Hi, I'm new to Dovecot. I'm trying to start Dovecot(2.0.9) but I'm getting an error. I think the error is because I'm doing a tutorial for an old version of Devcot(1.x.x). What I get is the following: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=imaps is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=pop3s is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:5: ssl_cert_file has been replaced by ssl_cert = References: <1328190864.11500.130.camel@innu> Message-ID: <4F2BE57D.1090508@in.tum.de> On 02.02.2012 14:54, Timo Sirainen wrote: > user_attrs { > home = %{ldap:homeDirectory} > uid = %{ldap:uidNumber} > gid = %{ldap:gidNumber} > mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} > } > > Any ideas for further improvements before I do this change? One thing I was recently missing was something like this: If the user is in LDAP group "group1", then use host = imap1 otherwise use host = imap2 However this would require a separate LDAP query since most LDAP servers do not list any group information in the user table. Another nice feature: Use some sort of script to compute the lookup result, e.g. an external shell script: #!/bin/bash if [[ $ldap_uid == fred ]]; then echo "quota_rule = storage=3G" else echo "quota_rule = storage=100M" fi Stuff like this is possible for SQL user databases, however LDAP does not have such flexible queries. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From mark.zealey at webfusion.com Fri Feb 3 16:16:27 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Fri, 03 Feb 2012 16:16:27 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? Message-ID: <4F2BEC3B.2050808@webfusion.com> Hi there, I was doing some testing on sdbox yesterday. Basically I did the following procedure: 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) 2) Create a copy of the index file (no cache file created yet) 3) deliver another message to the mailbox (u.3) 4) copy back index file from stage (2) 5) deliver new mail Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Thanks, Mark From lopes80andre at gmail.com Fri Feb 3 16:42:42 2012 From: lopes80andre at gmail.com (Andre Lopes) Date: Fri, 3 Feb 2012 14:42:42 +0000 Subject: [Dovecot] What does this /var/log/maillog message means? Message-ID: Hi, I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: [code] protocols = imap pop3 lmtp log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir ssl_cert = /etc/pki/dovecot/certs/dovecot.pem ssl_key = /etc/pki/dovecot/private/dovecot.pem namespace { type = private separator = . prefix = INBOX. inbox = yes } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at mailserver.com } service auth { user = root } passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } [/code] I can start the service correctly: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: [ OK ] [/code] But when I go to "/var/log/maillog" I get this: [code] Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 (by pid=16757 uid=0 code=kill) Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled) [/code] What does this mean? I'm doing something that is not correct? Best Regards, From list at airstreamcomm.net Fri Feb 3 17:22:50 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Fri, 03 Feb 2012 09:22:50 -0600 Subject: [Dovecot] Vacation via database Message-ID: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> We are moving our inbound mail to use dovecot LMTP in the near future and we are looking for suggestions on how to implement a mysql based vacation system. If anyone has experience with this, good or bad please let us know. From Ralf.Hildebrandt at charite.de Fri Feb 3 17:27:59 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 16:27:59 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> Message-ID: <20120203152758.GI29889@charite.de> * list at airstreamcomm.net : > We are moving our inbound mail to use dovecot LMTP in the near future and > we are looking for suggestions on how to implement a mysql based vacation > system. If anyone has experience with this, good or bad please let us > know. Why not use sieve? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 18:41:23 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 11:41:23 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <20120203152758.GI29889@charite.de> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> Message-ID: <4F2C0E33.5070708@gedalya.net> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: > * list at airstreamcomm.net: >> We are moving our inbound mail to use dovecot LMTP in the near future and >> we are looking for suggestions on how to implement a mysql based vacation >> system. If anyone has experience with this, good or bad please let us >> know. > Why not use sieve? > I am facing a similar issue. It looks like sieve can't fetch data from a database, and dovecot can't fetch sieve scripts from a database. It would be useful to have, perhaps, only a global sieve script be able to query a DB, and it would be very useful to allow dovecot to fetch sieve scripts (actual scripts, not a file path) from a database. Any way to do this? From robert at schetterer.org Fri Feb 3 20:32:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:32:12 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C282C.4040403@schetterer.org> Am 03.02.2012 17:41, schrieb Gedalya: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near future >>> and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from a > database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able to > query a DB, and it would be very useful to allow dovecot to fetch sieve > scripts (actual scripts, not a file path) from a database. Any way to do > this? > sieve with database is on roadmap,as it was discussed before ,as far i remember info http://wiki.dovecot.org/ManageSieve http://pigeonhole.dovecot.org/ anyway it seems there is a lot to do left http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO so sql support might have to wait but snip ---- * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## * Implement extlists extension as a plugin * Enotify extension: detect use of variable values extracted from the message that are used in the method argument. RFC reports this as a security issue. * Make the sieve storage a base class with (possibly) various implementations, just like mail-storage. This aims to provide support for alternate types of script storage like LDAP or SQL database. --- looks its on the way -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Fri Feb 3 20:34:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:34:18 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C282C.4040403@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> Message-ID: <4F2C28AA.90904@schetterer.org> Am 03.02.2012 19:32, schrieb Robert Schetterer: > Am 03.02.2012 17:41, schrieb Gedalya: >> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>> * list at airstreamcomm.net: >>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>> and >>>> we are looking for suggestions on how to implement a mysql based >>>> vacation >>>> system. If anyone has experience with this, good or bad please let us >>>> know. >>> Why not use sieve? >>> >> I am facing a similar issue. It looks like sieve can't fetch data from a >> database, and dovecot can't fetch sieve scripts from a database. It >> would be useful to have, perhaps, only a global sieve script be able to >> query a DB, and it would be very useful to allow dovecot to fetch sieve >> scripts (actual scripts, not a file path) from a database. Any way to do >> this? >> > > sieve with database is on roadmap,as it was discussed before ,as far i > remember > > info > > http://wiki.dovecot.org/ManageSieve > http://pigeonhole.dovecot.org/ > > anyway it seems there is a lot to do left > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO > so sql support might have to wait > > but snip > ---- > * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## > > * Implement extlists extension as a plugin > * Enotify extension: detect use of variable values extracted from the > message > that are used in the method argument. RFC reports this as a security > issue. > * Make the sieve storage a base class with (possibly) various > implementations, > just like mail-storage. This aims to provide support for alternate types > of script storage like LDAP or SQL database. > --- > > > looks its on the way by the way you can use some sql vacation script method like implemented in postfixadmin , and has support for horde webmail,squirrelmail,roundcube -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From Ralf.Hildebrandt at charite.de Fri Feb 3 20:35:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 19:35:06 +0100 Subject: [Dovecot] Managesieve Build failure Message-ID: <20120203183506.GU29889@charite.de> I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, got this error today: ... make[3]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' make[2]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' Making install in managesieve-login make[2]: Entering directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve-login' /bin/bash ../../libtool --tag=CC --mode=link gcc -I/usr/src/dovecot-2.0/dovecot-2.0/src/config -DPKG_LIBEXECDIR=\""/usr/local/libexec/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -module -avoid-version -g -Wl,--as-needed -o libmanagesieve_login_settings.la -rpath /usr/local/lib/dovecot/settings libmanagesieve_login_settings_la-managesieve-login-settings.lo libmanagesieve_login_settings_la-managesieve-login-settings-plugin.lo -lrt gcc -DHAVE_CONFIG_H -I. -I../.. -I/usr/src/dovecot-2.0/dovecot-2.0 -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dict -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dns -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-mail -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-imap -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-fs -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-charset -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index/raw -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-master -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-settings -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-auth -I/usr/src/dovecot-2.0/dovecot-2.0/src/login-common -I../.. -I../../src/lib-managesieve -g -Wl,--as-needed -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -MT managesieve-proxy.o -MD -MP -MF .deps/managesieve-proxy.Tpo -c -o managesieve-proxy.o managesieve-proxy.c managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? make[2]: *** [managesieve-proxy.o] Error 1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 20:49:21 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 13:49:21 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C28AA.90904@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> <4F2C28AA.90904@schetterer.org> Message-ID: <4F2C2C31.4040202@gedalya.net> On 02/03/2012 01:34 PM, Robert Schetterer wrote: > Am 03.02.2012 19:32, schrieb Robert Schetterer: >> Am 03.02.2012 17:41, schrieb Gedalya: >>> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>>> * list at airstreamcomm.net: >>>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>>> and >>>>> we are looking for suggestions on how to implement a mysql based >>>>> vacation >>>>> system. If anyone has experience with this, good or bad please let us >>>>> know. >>>> Why not use sieve? >>>> >>> I am facing a similar issue. It looks like sieve can't fetch data from a >>> database, and dovecot can't fetch sieve scripts from a database. It >>> would be useful to have, perhaps, only a global sieve script be able to >>> query a DB, and it would be very useful to allow dovecot to fetch sieve >>> scripts (actual scripts, not a file path) from a database. Any way to do >>> this? >>> >> sieve with database is on roadmap,as it was discussed before ,as far i >> remember >> >> info >> >> http://wiki.dovecot.org/ManageSieve >> http://pigeonhole.dovecot.org/ >> >> anyway it seems there is a lot to do left >> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO >> so sql support might have to wait >> >> but snip >> ---- >> * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## >> >> * Implement extlists extension as a plugin >> * Enotify extension: detect use of variable values extracted from the >> message >> that are used in the method argument. RFC reports this as a security >> issue. >> * Make the sieve storage a base class with (possibly) various >> implementations, >> just like mail-storage. This aims to provide support for alternate types >> of script storage like LDAP or SQL database. >> --- >> >> >> looks its on the way > by the way you can use some sql vacation script method > like implemented in postfixadmin , and has support for horde > webmail,squirrelmail,roundcube > Yeap, for now, in this case, it looks like I'll be hand-coding something into exim and adapt an existing roundcube plugin to my situation. In general, it's looking like I'll be using one, global, static sieve script which will follow instructions in the form of X-Something: Action headers, which I'll be conditionally inserting per user preference on the MX end (exim lmtp transport). Since this is clumsy, I'll try to do as much as possible within the MX, and leave to sieve only things that must happen there (e.g. not necessarily vacation). From stephan at rename-it.nl Sat Feb 4 00:34:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:34:14 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C60E6.7020700@rename-it.nl> On 2/3/2012 5:41 PM, Gedalya wrote: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near >>> future and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from > a database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able > to query a DB, and it would be very useful to allow dovecot to fetch > sieve scripts (actual scripts, not a file path) from a database. Any > way to do this? > Retrieving whole scripts from a DB is nice, but not strictly necessary for your application. Indeed, only fetching data from a DB is sufficient. Unfortunately, there is no standard Sieve feature that allows fetching string data from external sources. However, Pigeonhole has a few custom features: * Assuming you're using SQL, there is an experimental plugin for retrieval of external string data from a Dovecot dict lookup (there is no LDAP dict afaik): http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extdata You can give that a look. Should also compile against Dovecot v2.1 and Pigeonhole v0.3. * If you were using Dovecot v2.1 and Pigeonhole v0.3 (from hg) you could also tinker something with the new extprograms plugin (using the execute command): http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/ For more info read: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/README http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt And, as you suggested, Pigeonhole v0.3 for Dovecot v2.1 can also restrict the use of these plugins to global scripts. Regards, Stephan. From stephan at rename-it.nl Sat Feb 4 00:53:49 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:53:49 +0100 Subject: [Dovecot] Managesieve Build failure In-Reply-To: <20120203183506.GU29889@charite.de> References: <20120203183506.GU29889@charite.de> Message-ID: <4F2C657D.50703@rename-it.nl> On 2/3/2012 7:35 PM, Ralf Hildebrandt wrote: > I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, > got this error today: > > ... > > managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: > managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? > make[2]: *** [managesieve-proxy.o] Error 1 I didn't realize that this change was also performed for Dovecot v2.0 and for some reason I didn't get a failure message from the automatic builder. Fixed now: http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 Regards, Stephan. From Przemek.Orzechowski at makolab.net Fri Feb 3 16:38:05 2012 From: Przemek.Orzechowski at makolab.net (Przemek Orzechowski) Date: Fri, 3 Feb 2012 14:38:05 +0000 Subject: [Dovecot] Question about quota configuration In-Reply-To: <20120203073604.GA6561@dibs.tanso.net> References: <20120203073604.GA6561@dibs.tanso.net> Message-ID: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Thanks for quick reply >-----Original Message----- >From: Jan-Frode Myklebust >Reply-to: Jan-Frode Myklebust >To: przemek.orzechowski at makolab.pl >Cc: dovecot at dovecot.org >Subject: Re: [Dovecot] Question about quota configuration >Date: Fri, 3 Feb 2012 08:36:04 +0100 >Mailer: Mutt/1.5.20 (2009-12-10) > >On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: >> >> What im tryin to do now is to modify postfix->procmail->dovecot config >> in a way that if user is over quota mail delivery is delayed instead of >> bouncing. >> (is this possible?) > >Check the quota_full_tempfail setting, > > > http://wiki.dovecot.org/MainConfig > For some reason when i configure dovecot to act as the final LDA in procmail and set quota_full_tempfail = yes overquota mails get lost somewhere As Virtualmin is relying for some of its functons on procmail droping it compleetly is impossible or at least very difficult. >> >> Second thing i would like to achive is that when authenticated users close >> to quota/group quota for example 10MB to quota limit >> try sending email theyr mail is rejected and preferably an email is >> generated telling them to free some space for new mails first. >> (is such a thing possible?) > >Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 > I agree this is the solution for sending a warning email, but is there a method to actually prohibit users close to quota to send emails? Ie defining dovecot as authentication agent ? > > -jf to make things easier for everyone im posting snippets from my config of dovecot and procmail :0wi VIRTULAMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME EXITCODE=0 :0 * ?/usr/bin/test "$VIRTUALMIN" != "" { INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN } DEFAULT=$HOME/Maildir/ ORGMAIL=$HOME/Maildir/ DELIVER="/usr/lib/dovecot/deliver" DROPPRIVS=yes dovecot config snippets regarding LDA protocol lda { mail_plugins = quota quota_full_tempfail = yes } for some reason this results in mails lost without any trace when users are over quota (i would like to requeue those emails for later delivery) Most probably its not directly dovecot related but as most people here are more or less runing mail servers so maybe someone could help me further. From CMarcus at Media-Brokers.com Sat Feb 4 15:09:13 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sat, 04 Feb 2012 08:09:13 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Message-ID: <4F2D2DF9.1060608@Media-Brokers.com> On 2012-02-03 9:38 AM, Przemek Orzechowski wrote: > is there a method to actually prohibit users close to quota to send > emails? Ie defining dovecot as authentication agent? I wrote about a feature request on this list for something that would probably easily provide the ability for dovecot to do something like this a while back... I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. This would be a huge time saver for people who send a lot of messages with large attachments. Currently, the client has to 'send' (upload) the message to the server *twice* - once for 'sending' it, and again to 'save the sent copy'. Timo - is there a place for users to submit formal Feature Requests? And, is this idea of an LSA) something that you would even remotely consider at any point in time in the future? Maybe your new company could add a 'bounty' system, so users could submit Feature Requests and they (and others) could pledge money for the feature to be implemented? -- Best regards, Crles From gedalya at gedalya.net Sat Feb 4 15:09:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 04 Feb 2012 08:09:39 -0500 Subject: [Dovecot] What does this /var/log/maillog message means? In-Reply-To: References: Message-ID: <4F2D2E13.3080904@gedalya.net> On 02/03/2012 09:42 AM, Andre Lopes wrote: > Hi, > > I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: > > [code] > protocols = imap pop3 lmtp > > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > > ssl_cert = /etc/pki/dovecot/certs/dovecot.pem > ssl_key = /etc/pki/dovecot/private/dovecot.pem > > namespace { > type = private > separator = . > prefix = INBOX. > inbox = yes > } > > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at mailserver.com > } > > service auth { > user = root > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > userdb { > driver = static > args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes > } > [/code] > > I can start the service correctly: > > [code] > [root at mailserver dovecot]# service dovecot start > Starting Dovecot Imap: [ OK ] > [/code] > > But when I go to "/var/log/maillog" I get this: > > [code] > Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 > (by pid=16757 uid=0 code=kill) > Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core > dumps disabled) > [/code] > > What does this mean? I'm doing something that is not correct? > > Best Regards, There log entries just mean you have restarted dovecot. They are normal and don't indicate a problem. From amuehlem at gmail.com Sun Feb 5 17:00:34 2012 From: amuehlem at gmail.com (Andreas Muehlemann) Date: Sun, 05 Feb 2012 16:00:34 +0100 Subject: [Dovecot] Sieve notify messages Message-ID: <4F2E9992.9070701@gmail.com> Hi All I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. I'm trying to use the notifiy mechanism from sieve to send notifications when a mail arrives in the mailbox. The message is checked to be a 8bit message, otherwise it is replaced by the default message "Notification of new message." How can I create a 8bit message body within the sieve script that is accepted by the notification process? e.g. notify :message "Subject: ${subject}" "mailto:email at domain.com"; Do I have to put the message body into the :message string? Or how can I define a custom message body that is accepted by notify? Best regards Andreas From stephan at rename-it.nl Sun Feb 5 17:46:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 16:46:41 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2E9992.9070701@gmail.com> References: <4F2E9992.9070701@gmail.com> Message-ID: <4F2EA461.9000909@rename-it.nl> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: > Hi All > > I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. > > I'm trying to use the notifiy mechanism from sieve to send > notifications when a mail arrives in the mailbox. The message is > checked to be a 8bit message, otherwise it is replaced by the default > message "Notification of new message." How can I create a 8bit message > body within the sieve script that is accepted by the notification > process? > > e.g. > notify > :message "Subject: ${subject}" > "mailto:email at domain.com"; > > Do I have to put the message body into the :message string? Or how can > I define a custom message body that is accepted by notify? If I understand you correctly, the following should be the solution for your issue: This sends the following message: X-Sieve: Pigeonhole Sieve 0.3.0 Message-ID: Date: Sun, 05 Feb 2012 16:36:57 +0100 Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen From: Postmaster To: user at example.com Auto-Submitted: auto-notified; owner-email="234234234234 at ext2.domain.tld" Precedence: bulk X-Priority: 3 (Normal) Importance: Normal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem Durchschnitt schnell. Regards, Stephan From rog7993 at web.de Sun Feb 5 18:53:50 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 17:53:50 +0100 Subject: [Dovecot] user login on behalf of another user Message-ID: <4F2EB41E.6070100@web.de> Hello, we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Example: User "user1" and "user2" shall get access to the mailbox "info". We define the accounts "info~user1" and "info~user2" with the same home directory like "info". Until now, we use a passwd-file backend. With this setup we can simply copy the password hash from "user1" to "info~user1" and from "user2" to "info~user2". But we intend to change the passdb backend from a flat file to PAM for authentication against Active Directory. This seems to be simple with pam_krb5. But then we can't simply copy password hashes anymore. Is their another possibility for configuring this? Surely the preferable alternative would be the use of ACLs to give acccess to other users mailboxes. But we started this setup with Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. And now we have too much accounts and clients, which are configured this way and can't change this for the short term. The passdb/userdb file from the above example looks like this: info:!:501:501:Info:/home/mail01/info:: info~user1:PASSWORD_USER1:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info info~user2:PASSWORD_USER2:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info user1:PASSWORD_USER1:501:501:Info:/home/mail01/user1:: user2:PASSWORD_USER2:501:501:Info:/home/mail01/user2:: Ingo Rogalsky From sven at svenhartge.de Sun Feb 5 19:14:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 5 Feb 2012 18:14:51 +0100 Subject: [Dovecot] user login on behalf of another user References: <4F2EB41E.6070100@web.de> Message-ID: rog7993 at web.de wrote: > User "user1" and "user2" shall get access to the mailbox "info". We > define the accounts "info~user1" and "info~user2" with the same home > directory like "info". Isn't this the kind of scenario shared folders were made for? Gr??e, Sven. -- Sigmentation fault. Core dumped. From stephan at rename-it.nl Sun Feb 5 19:30:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 18:30:20 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2EA6E2.20700@gmail.com> References: <4F2E9992.9070701@gmail.com> <4F2EA461.9000909@rename-it.nl> <4F2EA6E2.20700@gmail.com> Message-ID: <4F2EBCAC.50403@rename-it.nl> On 2/5/2012 4:57 PM, Andreas Muehlemann wrote: > Hi Stephan > > Thank you very much, the encodeurl and ?body= are the trick! > For posterity, please reply to the mailing list as well, especially when it is a success message ;) Regards, Stephan. > > On 02/05/2012 04:46 PM, Stephan Bosch wrote: >> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: >>> Hi All >>> >>> I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. >>> >>> I'm trying to use the notifiy mechanism from sieve to send >>> notifications when a mail arrives in the mailbox. The message is >>> checked to be a 8bit message, otherwise it is replaced by the >>> default message "Notification of new message." How can I create a >>> 8bit message body within the sieve script that is accepted by the >>> notification process? >>> >>> e.g. >>> notify >>> :message "Subject: ${subject}" >>> "mailto:email at domain.com"; >>> >>> Do I have to put the message body into the :message string? Or how >>> can I define a custom message body that is accepted by notify? >> >> If I understand you correctly, the following should be the solution >> for your issue: >> >> >> >> This sends the following message: >> >> >> X-Sieve: Pigeonhole Sieve 0.3.0 >> Message-ID: >> Date: Sun, 05 Feb 2012 16:36:57 +0100 >> Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen >> From: Postmaster >> To: user at example.com >> Auto-Submitted: auto-notified; >> owner-email="234234234234 at ext2.domain.tld" >> Precedence: bulk >> X-Priority: 3 (Normal) >> Importance: Normal >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=UTF-8 >> Content-Transfer-Encoding: 8bit >> >> In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem >> Durchschnitt schnell. >> >> >> Regards, >> >> Stephan > From me at junc.org Sun Feb 5 22:01:32 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 05 Feb 2012 21:01:32 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: Den 2012-02-05 17:53, rog7993 at web.de skrev: > Surely the preferable alternative would be the use of ACLs to give > acccess to other users mailboxes. But we started this setup with > Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. > And now we have too much accounts and clients, which are configured > this way and can't change this for the short term. secureity wise i would also do this, had bots trying whole day here to get week passwords found to atleast find one login that works, if acl is used there is only one password found and the other user do not need to change his password, but only the acl or ask the other user to change his password, its still possible that both users is same user, imho it have no point if its pam users or not From rog7993 at web.de Sun Feb 5 23:49:54 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 22:49:54 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: References: <4F2EB41E.6070100@web.de> Message-ID: <4F2EF982.6080309@web.de> Am 05.02.2012 18:14, schrieb Sven Hartge: > Isn't this the kind of scenario shared folders were made for? Yes, of course. Although I didn't called it shared folders in my original post, I had this in mind, when I wrote that ACLs would be the better solution. But for historical reasons we are already in the situation having these kind of accounts. And I don't want to reconfigure about 100 clients now. This would need more time than I want spend now. Ingo From janfrode at tanso.net Mon Feb 6 15:59:17 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 14:59:17 +0100 Subject: [Dovecot] doveadm purge on shared storage Message-ID: <20120206135917.GA3399@dibs.tanso.net> We've finally (!) started to put some users on mdbox instead of maildir, and now I'm wondering about the purge step. As we're running GPFS for the mailboxes (and dovecot director in front of every dovecot service), is it important to run the "doveadm purge -u $user" on the same host as $user is logged into to avoid index corruption ? If so, will we need to run the "doveadm purge" trough the dovecot director as well? -jf From tss at iki.fi Mon Feb 6 17:05:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 17:05:51 +0200 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> On 5.2.2012, at 18.53, rog7993 at web.de wrote: > we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Master user doesn't necessarily have access to all users' mailboxes. In the passdb lookup you can decide if this master user is allowed to be this destination user. For example if you used passdb checkpassword, you could look at USER and MASTER_USER environment variables to figure out if this combination should be allowed or not. The checkpassword script can also do the actual authentication via PAM (I'd think there's a way to call it somehow). From cannewilson at googlemail.com Mon Feb 6 19:33:01 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Mon, 6 Feb 2012 17:33:01 +0000 Subject: [Dovecot] Configuration problem? Message-ID: I have a new Scientific Linux 6.1 mail server (dovecot -n below) and am seeing the following in the logs, with no idea what is happening: --------------------- Dovecot Begin ------------------------ Dovecot was killed, and not restarted afterwards. **Unmatched Entries** dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24296/89310: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24526/76772: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=26254/75778: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=27948/134120: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=29304/116087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=34802/914087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=4457/11719: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=5322/184434: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=61388/106618: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=6164/21347: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) dovecot: imap: Server shutting down. bytes=14215/97371: 1 Time(s) dovecot: imap: Server shutting down. bytes=1459/9382: 1 Time(s) dovecot: imap: Server shutting down. bytes=1688/26161: 1 Time(s) dovecot: imap: Server shutting down. bytes=1705/5038: 1 Time(s) dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) ---------------------- Dovecot End ------------------------- dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) mail_location = maildir:~/Maildir/ mbox_write_locks = fcntl passdb { driver = pam } protocols = imap ssl_cert = References: Message-ID: <20120206174932.GA5904@PC211.ikt.de> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > am seeing the following in the logs, with no idea what is happening: > > --------------------- Dovecot Begin ------------------------ > > > Dovecot was killed, and not restarted afterwards. You shut dovecot down and does not restart it. > **Unmatched Entries** > dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) [..] > dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) The user closed the connection. > dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) [..] > dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) The server closed the connection due to a shutdown command. > dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) Dovecot did what it just said: it started. Your logwatch is to old to knew about the messages dovecot emits to syslog. HTH Dennis [..] From tss at iki.fi Mon Feb 6 21:58:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:58:02 +0200 Subject: [Dovecot] Multiple instances Message-ID: v2.1 now saves the base_dir and instance_name to /var/lib/dovecot/instances when starting up. If you have for example two instances with names "backend" and "proxy", you can then use: doveadm -i proxy proxy list instead of something like: doveadm -c /etc/dovecot-proxy/dovecot.conf proxy list The same -i parameter works for all dovecot programs. I also added a couple of new doveadm commands: # doveadm instance list path name last used running /var/run/dovecot proxy 2012-02-06 21:41:39 yes # doveadm instance remove proxy It would be possible to add commands to start/shutdown some/all instances using doveadm, but is it all that useful? I'd guess people would have their own init.d scripts anyway doing that. Anything else that could be useful related to this? From tss at iki.fi Mon Feb 6 21:59:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:59:40 +0200 Subject: [Dovecot] doveadm purge on shared storage In-Reply-To: <20120206135917.GA3399@dibs.tanso.net> References: <20120206135917.GA3399@dibs.tanso.net> Message-ID: On 6.2.2012, at 15.59, Jan-Frode Myklebust wrote: > We've finally (!) started to put some users on mdbox instead of maildir, > and now I'm wondering about the purge step. As we're running GPFS for the > mailboxes (and dovecot director in front of every dovecot service), is > it important to run the "doveadm purge -u $user" on the same host as > $user is logged into to avoid index corruption ? > > If so, will we need to run the "doveadm purge" trough the dovecot director as > well? If you run doveadm purge on the director itself (or so that it uses director's config), then doveadm automatically goes through director to the correct server to run the purge. You need to have configured http://wiki2.dovecot.org/Director#Doveadm_server From tss at iki.fi Mon Feb 6 22:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:10:11 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F2D2DF9.1060608@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> Message-ID: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> On 4.2.2012, at 15.09, Charles Marcus wrote: > I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. > > But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. > > This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. Can't you already do this by configuring MTA to bcc mail to user+Sent at domain where there's a global Sieve script moving the +Sent mails to Sent folder (possibly checking that it's from localhost). > Timo - is there a place for users to submit formal Feature Requests? I wouldn't mind if they are simply sent to this list. Although would be a good idea to make it clear in the subject, I almost skipped this message. Most of the things that I want to implement end up in the TODO file in Dovecot sources, and the things that don't end up there are something I'd rather forget, at least until enough people start wanting them. From me at junc.org Mon Feb 6 22:24:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:24:56 +0100 Subject: [Dovecot] dovecot-antispam 1.3 and dovecot 1.2.17 fails with spool2dir Message-ID: <555d62880f4f8cab3bfb1f278f900d22@junc.org> or have i missed something in compile or configure in dovecot.conf ? running on gentoo so i can compile if need another option, i just like to have it working so each user can train bayes global db no matter what user agent users using From CMarcus at Media-Brokers.com Mon Feb 6 22:25:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 06 Feb 2012 15:25:19 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> Message-ID: <4F30372F.3080409@Media-Brokers.com> On 2012-02-06 3:10 PM, Timo Sirainen wrote: > On 4.2.2012, at 15.09, Charles Marcus wrote: >> This way, you could completely disable the 'Save a copy of Sent >> Messages to X folder' option in all mail clients like you can wit >> GMail accounts. > Can't you already do this by configuring MTA to bcc mail to > user+Sent at domain where there's a global Sieve script moving the > +Sent mails to Sent folder (possibly checking that it's from > localhost). You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... >> Timo - is there a place for users to submit formal Feature >> Requests? > I wouldn't mind if they are simply sent to this list. Although would > be a good idea to make it clear in the subject, I almost skipped > this message. Most of the things that I want to implement end up in > the TODO file in Dovecot sources, and the things that don't end up > there are something I'd rather forget, at least until enough people > start wanting them. Heh... understood, and no worries if my idea for a dovecot LSA doesn't make it to the TODO list... ;) -- Best regards, Charles From tss at iki.fi Mon Feb 6 22:27:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:27:05 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <725061328274905@web20.yandex.ru> References: <725061328274905@web20.yandex.ru> Message-ID: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> On 3.2.2012, at 15.15, User Max wrote: > Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted service auth { user = root } Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. From tss at iki.fi Mon Feb 6 22:29:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:29:03 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120203122552.GA12451@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> Message-ID: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > I now implemented this patch on our directors, and pointed postfix at them. > No problem seen so far, but I'm still a bit uncertain about the > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > seeing lmtp_proxy_output_timeout problems again ? Your fsyncs can run over 60 seconds? I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. From tss at iki.fi Mon Feb 6 22:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:33:18 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F30372F.3080409@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> Message-ID: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> On 6.2.2012, at 22.25, Charles Marcus wrote: > On 2012-02-06 3:10 PM, Timo Sirainen wrote: >> On 4.2.2012, at 15.09, Charles Marcus wrote: > >>> This way, you could completely disable the 'Save a copy of Sent >>> Messages to X folder' option in all mail clients like you can wit >>> GMail accounts. > >> Can't you already do this by configuring MTA to bcc mail to >> user+Sent at domain where there's a global Sieve script moving the >> +Sent mails to Sent folder (possibly checking that it's from >> localhost). > > You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... Dovecot LSA would pretty much have to implement full SMTP server functionality to be able to do this (although maybe it could get away with not doing queuing). I'm not really interested in writing an SMTP server. It would be much less trouble to change Postfix to work in a way that you want. From dinh at flic.net Mon Feb 6 22:37:05 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 12:37:05 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? Message-ID: Hi, I am running dovecot version 1.2.9 on a Ubuntu 10.04 server. To obtain the message's header info from this dovecot imap server, I run the following imap command on a client side's imap session: . fetch 88:90 (envelope) which returns (in this case) ------------------------------------------------------------- * 88 FETCH (ENVELOPE ("Tue, 28 Oct 2008 15:45:35 -0500" "Linear Actuators Bring Flexibility to Packaging" (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<490779CF.00000E7D at ak119.mta.exacttarget.com>")) * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<20081029154358.0C359AE169595F1B at insightmedia.info>")) * 90 FETCH (ENVELOPE ("Thu, 30 Oct 2008 04:24:31 +0800 (SGT)" "6 new products Now Online!" (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Dinh Le" NIL "dinh" "flic.net")) NIL NIL NIL "<352725277.1225311871753.JavaMail.web at fuchsia3.globalsources.com>")) . OK Fetch completed. -------------------------------------------------------------- Notice that message 89's Subject line Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available has quotes and this dovecot's ENVELOPE response failed to get parsed by a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). I have a hunch that this is dovecot's fault but I could be wrong. How should dovecot (or any imap server) deal with Subject lines with quotes inside ENVELOPE? ENVELOPE seems to encapsulate most of the important information and its format can be parsed cleanly and quickly. I prefer to fetch ENVELOP, at this point, rather than the other alternatives. Thanks for your help, Dinh From me at junc.org Mon Feb 6 22:39:33 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:39:33 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> Message-ID: <4e7f87f0748f1b70878567e9f5c07fda@junc.org> > Dovecot LSA would pretty much have to implement full SMTP server > functionality to be able to do this (although maybe it could get away > with not doing queuing). I'm not really interested in writing an SMTP > server. It would be much less trouble to change Postfix to work in a > way that you want. dovecot have lmtp, postfix have lmtp, so does quotas work in lmtp deliver ? else one need to use postfix sql to dovecot dict to get used quotas :/ From slusarz at curecanti.org Mon Feb 6 22:45:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 13:45:46 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: Message-ID: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Quoting Dinh Le : > * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} > 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL > NIL "dinh" "flic.net")) NIL NIL NIL > "<20081029154358.0C359AE169595F1B at insightmedia.info>")) [snip] > Notice that message 89's Subject line > > Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > > has quotes and this dovecot's ENVELOPE response failed to get parsed by > a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). > I have a hunch that this is dovecot's fault but I could be wrong. > How should dovecot (or any imap server) deal with Subject lines with > quotes inside ENVELOPE? The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. Dovecot is 100% correct in its return. michael From tss at iki.fi Mon Feb 6 22:47:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:47:41 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F2BEC3B.2050808@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> Message-ID: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> On 3.2.2012, at 16.16, Mark Zealey wrote: > I was doing some testing on sdbox yesterday. Basically I did the following procedure: > > 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) > 2) Create a copy of the index file (no cache file created yet) > 3) deliver another message to the mailbox (u.3) > 4) copy back index file from stage (2) > 5) deliver new mail > > Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. Hm. Yes, I guess there should be a check to avoid overwriting files. > I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. I'm not entirely sure what you mean by this. Does the above patch help with this problem also? > (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Well, ideally there shouldn't be so much corruption that this matters.. From tss at iki.fi Mon Feb 6 22:55:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:55:43 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6.2.2012, at 22.45, Michael M Slusarz wrote: > The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. > > Dovecot is 100% correct in its return. Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) From janfrode at tanso.net Mon Feb 6 23:01:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 22:01:03 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> Message-ID: <20120206210103.GA11888@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:29:03PM +0200, Timo Sirainen wrote: > On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > > > I now implemented this patch on our directors, and pointed postfix at them. > > No problem seen so far, but I'm still a bit uncertain about the > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > > seeing lmtp_proxy_output_timeout problems again ? > > Your fsyncs can run over 60 seconds? Hopefully not.. maybe just me being confused by the error message about "lmtp_proxy_output_timeout". After adding http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't seen any problems so it looks like this problem is solved. But it doesn't seem unthinkable that ext3 users might see more than 60s for fsyncs... "Some stalls on the order of minutes have been reported" ref: https://lwn.net/Articles/328363/ > I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. > My MTA's default is 10 minutes.. http://www.postfix.org/postconf.5.html#lmtp_data_done_timeout -jf From tss at iki.fi Mon Feb 6 23:16:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 23:16:24 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> On 6.2.2012, at 22.55, Timo Sirainen wrote: > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) BTW. If anyone wants to try, the attached patch makes Dovecot return (almost) all possible fields as literals. You'll need to delete dovecot.index.cache files to get the full effect. -------------- next part -------------- A non-text attachment was scrubbed... Name: break-imap-clients.diff Type: application/octet-stream Size: 425 bytes Desc: not available URL: From dinh at flic.net Mon Feb 6 23:18:33 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:18:33 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Thanks, I will try to come up From i.thierack at iwt-promotion.com Mon Feb 6 23:26:45 2012 From: i.thierack at iwt-promotion.com (Ingo Thierack) Date: Mon, 06 Feb 2012 22:26:45 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? Message-ID: <4F304595.9030007@iwt-promotion.com> Hello, try to use the new 2.1rc and don't get any data in the searchindex. Tried first lucene, and switched than back so solr. If I do an search in an mailfolder, i get in the dovecot-log 2012-02-06 22:17:11 | dovecot: | indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot Log from solr. Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} hits=0 status=0 QTime=1 Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository yesterday) With 2.0 i see, if i start an search, solr had to work on the mail, now happens nothing. Upgrade the schema.xml, delete the old index. Regard Ingo Thierack From dinh at flic.net Mon Feb 6 23:34:31 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:34:31 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Oops, my happy Alpine ^X finger accidently sent the previous reply. I meant to say thanks and will try to come up with a fix to this literal string problem for the Perl IMAPClient module. Don't want Timo's evil config setting to wreak havocs :-). From dovecot-20110531 at billmail.scconsult.com Mon Feb 6 23:47:07 2012 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Mon, 06 Feb 2012 16:47:07 -0500 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6 Feb 2012, at 16:34, Dinh Le wrote: > On Mon, 6 Feb 2012, Timo Sirainen wrote: > >> On 6.2.2012, at 22.45, Michael M Slusarz wrote: >> >>> The subject, because it contains quotes, can't be expressed in a >>> quoted string so it is instead sent in a literal string. >> >> Small correction: Quotes can be escaped in a quoted-string, but >> Dovecot just doesn't want to do it. Many other servers behave this >> way as well. >> >>> In a literal string, the data is exactly what appears in the >>> original message. No post-processing mechanism should be stripping >>> quotes or anything from this data - it is already in its canonical >>> state. >>> >>> Dovecot is 100% correct in its return. >> >> Yes. Looks like Perl's IMAP client hasn't been built properly to >> expect literals in places where they are allowed. My guess is that >> this is only one of the 100 other places where a literal is allowed >> by RFC but not by Perl.. >> >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) >> > > Oops, my happy Alpine ^X finger accidently sent the previous reply. I > meant to say thanks and will try to come up with a fix to this literal > string problem for the Perl IMAPClient module. > > Don't want Timo's evil config setting to wreak havocs :-). The best solution for you may be to to update the version of Mail::IMAPClient you are using. There were fixes for its handling of literals claimed in the CHANGES file of v3.26 (2011-01-31). I don't know if that will fix this particular problem, but if you are running an older version then it would be worth trying the current version. From slusarz at curecanti.org Tue Feb 7 00:14:27 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 15:14:27 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> Message-ID: <20120206151427.Horde.QTJpPoF5lbhPMFDDPHZkelA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 6.2.2012, at 22.55, Timo Sirainen wrote: > >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) > > BTW. If anyone wants to try, the attached patch makes Dovecot return > (almost) all possible fields as literals. You'll need to delete > dovecot.index.cache files to get the full effect. Very useful. Turns out the only thing I wasn't handling correctly was the MIME types/subtypes in BODYSTRUCTURE. This is a great tool for debugging client code. Not sure the difficulty to add as a config (or even a hidden config). Being able to easily turn on/off would increase my usage of the tool. michael From ybhu at hk1.ibm.com Tue Feb 7 08:26:43 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 7 Feb 2012 14:26:43 +0800 Subject: [Dovecot] Synchronization error in NFS Message-ID: Hi there, Not sure whether it's a bug or env setting issue. I am running some concurrent testings under NFS. The dovecot (version 2.0.13) are deployed in 2 servers (same setup), whose maildir are on the file server via NFS. Here are the output of dovecot -n: [root at MyMachine src]$dovecot -n # 2.0.13: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) nfs auth_anonymous_username = andy auth_debug = yes auth_debug_passwords = yes auth_mechanisms = anonymous plain auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = andy mail_location = maildir:/tmp/NFS mail_nfs_index = yes mail_nfs_storage = yes mail_uid = andy mmap_disable = yes passdb { driver = pam } ssl = no userdb { driver = passwd } Here are what I am doing: One session running loop of COPY commands (while(1) COPY...) connects to one dovecot server; The other session running loop of SELECT commands (while(1) SELECT...) connects to the other dovecot server. Both are accessing the same mail box (/tmp/NFS); After some while (not accurate duration, maybe 2 seconds, or 1 min), I found the number of EXISTS returned from SELECT command is not correct(less than the real number). Then I stop the both sessions. In the /tmp/log: Feb 07 03:44:59 imap(andy): Error: Corrupted transaction log file /tmp/NFS/dovecot.index.log seq 2: Unexpected garbage at EOF (sync_offset=2204) Feb 07 03:44:59 imap(andy): Error: Index /tmp/NFS/dovecot.index: Lost log for seq=2 offset=2204 Feb 07 03:44:59 imap(andy): Warning: fscking index file /tmp/NFS/dovecot.index Then I tried to dump the index, sometime it's failed because of the index corruption. If not, it says: [root at MyMachine src]$doveadm dump /tmp/NFS > /tmp/dump ; vi /tmp/dump doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor And sometime from the dump output, in the RECORD part: -- RECORDS: 5 RECORD: seq=1, uid=1, flags=0x00 RECORD: seq=2, uid=2, flags=0x00 RECORD: seq=3, uid=3, flags=0x00 RECORD: seq=4, uid=4, flags=0x00 RECORD: seq=5, uid=6, flags=0x00 The uid 5 is missed. but in uidlist file, it's there.. Here are all what I found. If you need additional information, pls let me know. The clock on the 3 machines are synchronized. You can also reproduce it if the 2 sessions are APPEND and SELECT. If both sessions are running towards the same dovecot server, even the maildir are on the NFS, it works very well without any error. From cannewilson at googlemail.com Tue Feb 7 10:08:24 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 7 Feb 2012 08:08:24 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120206174932.GA5904@PC211.ikt.de> References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: On 6 February 2012 17:49, Dennis Guhl wrote: > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and >> am seeing the following in the logs, with no idea what is happening: >> >> ?--------------------- Dovecot Begin ------------------------ >> >> >> ?Dovecot was killed, and not restarted afterwards. > > You shut dovecot down and does not restart it. > It appears to be doing things without my intervention. Despite the reports that it kept shutting down, Dovecot continued to serve messages throughout the day. >> ?**Unmatched Entries** >> ? ? dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) > > [..] > >> ? ? dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) > > The user closed the connection. > >> ? ? dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) > > [..] > >> ? ? dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) > > The server closed the connection due to a shutdown command. > >> ? ? dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) > > Dovecot did what it just said: it started. > > Your logwatch is to old to knew about the messages dovecot emits to > syslog. > > HTH > Dennis > > [..] You mean the version of logwatch is too old? I'm beginning to wonder whether running an Enterprise version is such a good idea after all. Anne From janfrode at tanso.net Tue Feb 7 10:25:10 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 7 Feb 2012 09:25:10 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120206210103.GA11888@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> Message-ID: <20120207082510.GA24432@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:01:03PM +0100, Jan-Frode Myklebust wrote: > > Your fsyncs can run over 60 seconds? > > Hopefully not.. maybe just me being confused by the error message about > "lmtp_proxy_output_timeout". After adding > http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't > seen any problems so it looks like this problem is solved. Crap, saw 6 "message might be sent more than once" messages from postfix yesterday, all at the time of this crash on the director postfix/lmtp was talking with: Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d680] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d6d6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193cb93] -> dovecot/lmtp [0x406d75] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xcd) [0x2ab6f194859d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x68) [0x2ab6f1949558] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x2ab6f194820d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x2ab6f1936a83] -> dovecot/lmtp(main+0x144) [0x403fa4] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x35f8a1d994] -> dovecot/lmtp [0x403da9] Feb 6 16:13:10 loadbalancer2 dovecot: master: Error: service(lmtp): child 6601 killed with signal 6 (core dumps disabled) Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any other ideas for what might be causing it ? -jf From Juergen.Obermann at hrz.uni-giessen.de Tue Feb 7 11:13:24 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 07 Feb 2012 10:13:24 +0100 Subject: [Dovecot] dsync error Mailbox has children, delete them first In-Reply-To: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> References: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> Message-ID: <16d56e5474247859b32601122db0b39f@imapproxy.hrz> Am 13.12.2011 11:47, schrieb J?rgen Obermann: > Hi, > > I use dsync to backup mailboxes from mbox format to mdbox on a remote > system. The first run for a user with dsync is OK, but during the > second there are lots of the following errors: > > dsync-remote(user): Error: Can't delete mailbox directory Example: > Mailbox has children, delete them first > > I see no way how I could influence the order dsync deletes mailboxes. > This happens with dovecot version 2.0.16 > > Thank you, > Juergen Obermann Hallo, after upgrade to dovecot 2.0.17 this problem went away. Greetings, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From nmilas at noa.gr Tue Feb 7 12:07:56 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 12:07:56 +0200 Subject: [Dovecot] Multiple userdb possible? Message-ID: <4F30F7FC.9050607@noa.gr> Hello, I am running v2.0.13. In my dovecot.conf I have: userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } Is it legitimate to include multiple ldap userdb's, like: userdb { args = /etc/dovecot/dovecot-usrdb-ldap1.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap1.conf driver = ldap } userdb { args = /etc/dovecot/dovecot-usrdb-ldap2.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap2.conf driver = ldap } If it is legitimate (in case configuration is different, please correct me), in which sequence userdb's are evaluated? Thanks, Nick From dg at dguhl.org Tue Feb 7 13:57:07 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 7 Feb 2012 12:57:07 +0100 Subject: [Dovecot] Configuration problem? In-Reply-To: References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: <20120207115707.GA30334@laptop-dg.leere.eu> On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: > On 6 February 2012 17:49, Dennis Guhl wrote: > > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > >> am seeing the following in the logs, with no idea what is happening: > >> > >> ?--------------------- Dovecot Begin ------------------------ > >> > >> > >> ?Dovecot was killed, and not restarted afterwards. > > > > You shut dovecot down and does not restart it. > > > It appears to be doing things without my intervention. Despite the > reports that it kept shutting down, Dovecot continued to serve > messages throughout the day. The messages logwatch shows appeared at some time within the analysed period and are not necessarily in a time sorted order. Btw do not rely on any summary of log files but look into the log yourself. [..] > You mean the version of logwatch is too old? I'm beginning to wonder Yes, the current version is 7.4.0 from march 2011 (http://www.logwatch.org). > whether running an Enterprise version is such a good idea after all. I don't know Scientific Linux but I use Debian stable on all my server and I'm very happy with it. Nonetheless do I manually upgrade some packages wich added needed features or are maintained by upstream. It is crucial to know and understand the philosophy behind a distribution and to decide if this works for you and if you can live with the caveats resulting. Dennis From mark.zealey at webfusion.com Tue Feb 7 14:08:09 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Tue, 07 Feb 2012 14:08:09 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> Message-ID: <4F311429.2000708@webfusion.com> 06-02-2012 22:47, Timo Sirainen yazm??: > On 3.2.2012, at 16.16, Mark Zealey wrote: > >> I was doing some testing on sdbox yesterday. Basically I did the following procedure: >> >> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >> 2) Create a copy of the index file (no cache file created yet) >> 3) deliver another message to the mailbox (u.3) >> 4) copy back index file from stage (2) >> 5) deliver new mail >> >> Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. > http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. >> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) > Well, ideally there shouldn't be so much corruption that this matters.. That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Mark From cannewilson at googlemail.com Tue Feb 7 17:20:18 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 07 Feb 2012 15:20:18 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120207115707.GA30334@laptop-dg.leere.eu> References: <20120206174932.GA5904@PC211.ikt.de> <20120207115707.GA30334@laptop-dg.leere.eu> Message-ID: <4F314132.7030604@googlemail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/02/12 11:57, Dennis Guhl wrote: > On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: >> On 6 February 2012 17:49, Dennis Guhl wrote: >>> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >>>> I have a new Scientific Linux 6.1 mail server (dovecot -n >>>> below) and am seeing the following in the logs, with no idea >>>> what is happening: >>>> >>>> --------------------- Dovecot Begin ------------------------ >>>> >>>> >>>> Dovecot was killed, and not restarted afterwards. >>> >>> You shut dovecot down and does not restart it. >>> >> It appears to be doing things without my intervention. Despite >> the reports that it kept shutting down, Dovecot continued to >> serve messages throughout the day. > > The messages logwatch shows appeared at some time within the > analysed period and are not necessarily in a time sorted order. > Btw do not rely on any summary of log files but look into the log > yourself. > > [..] > Actually, this morning there aren't the same messages, so perhaps I was restarting services while trying to get it right - in fact it seems very likely that that was so. Today there are a few like dovecot: imap(anne): Disconnected: Logged out bytes=11892/21219: 1 Time(s) I presume that refers to clients logging out of the imap connection? In which case, I can forget about that. I normally read the summary each morning and refer directly to the logs if I see something that looks unusual. Occasionally, as in this case, there are entries that I don't understand and I ask those who do :-) >> You mean the version of logwatch is too old? I'm beginning to >> wonder > > Yes, the current version is 7.4.0 from march 2011 > (http://www.logwatch.org). > >> whether running an Enterprise version is such a good idea after >> all. > > I don't know Scientific Linux but I use Debian stable on all my > server and I'm very happy with it. Nonetheless do I manually > upgrade some packages wich added needed features or are maintained > by upstream. > > It is crucial to know and understand the philosophy behind a > distribution and to decide if this works for you and if you can > live with the caveats resulting. > I've run CentOS for maybe 4 years, and it's similar to SL, both being RHEL clones, but maintained by different communities. On a server (even though this is a very "mild" server, being only file and print serving) the older packages are rarely a problem. I appreciate the time and trouble you are taking to educate me :-) Anne -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8xQSsACgkQj93fyh4cnBdynQCeO6tY/798/6YonBALxJ0baZcy qG4AoIEHcZWsjIqhz4v3+SMk76FLhjIk =/T9e -----END PGP SIGNATURE----- From rob0 at gmx.co.uk Tue Feb 7 18:00:57 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 7 Feb 2012 10:00:57 -0600 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <4F30F7FC.9050607@noa.gr> References: <4F30F7FC.9050607@noa.gr> Message-ID: <20120207160057.GX14195@harrier.slackbuilds.org> On Tue, Feb 07, 2012 at 12:07:56PM +0200, Nikolaos Milas wrote: > I am running v2.0.13. In my dovecot.conf I have: > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap.conf > driver = ldap > } > > Is it legitimate to include multiple ldap userdb's, like: Of course. The driver chosen is not significant; the query used is all that matters. Having two LDAP searches is conceptually no different than having system users and SQL users. > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap1.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap1.conf > driver = ldap > } > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap2.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap2.conf > driver = ldap > } > > If it is legitimate (in case configuration is different, please > correct me), in which sequence userdb's are evaluated? In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this in mind in setting up the queries. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From chayes at afo.net Tue Feb 7 22:44:20 2012 From: chayes at afo.net (Cliff Hayes) Date: Tue, 7 Feb 2012 14:44:20 -0600 Subject: [Dovecot] Fedora 16 configuration Message-ID: Hello, I am trying to get dovecot to work on a fedora 16 install with sendmail. I have been able to get it to work in the past with dovecot.conf but not with the new conf.d directory and associated config files. I keep seeing this in maillog: Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth attempts): rip=x.x.x.x, lip=x.x.x.x And the mail client comes back with username or password invalid. Is there instructions somewhere regarding fedora 16 installs? I found this one and tried it to no avail: http://www.server-world.info/en/note?os=Fedora_16&p=mail&f=2 Thanks in advance, Cliff From nmilas at noa.gr Tue Feb 7 22:56:02 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 22:56:02 +0200 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <20120207160057.GX14195@harrier.slackbuilds.org> References: <4F30F7FC.9050607@noa.gr> <20120207160057.GX14195@harrier.slackbuilds.org> Message-ID: <4F318FE2.9070604@noa.gr> On 7/2/2012 6:00 ??, /dev/rob0 wrote: > ... > Having two LDAP searches is conceptually no different than having > system users and SQL users. > ... > In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match > prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this > in mind in setting up the queries > ... Thank you for the clarifications! Regards, Nick From rog7993 at web.de Wed Feb 8 01:02:01 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Wed, 08 Feb 2012 00:02:01 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> References: <4F2EB41E.6070100@web.de> <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> Message-ID: <4F31AD69.2000109@web.de> Hello, Am 06.02.2012 16:05, schrieb Timo Sirainen: > Master user doesn't necessarily have access to all users' mailboxes. > In the passdb lookup you can decide if this master user is allowed to > be this destination user. For example if you used passdb > checkpassword, you could look at USER and MASTER_USER environment > variables to figure out if this combination should be allowed or not. > The checkpassword script can also do the actual authentication via > PAM (I'd think there's a way to call it somehow). > Thank you. I got an idea, how I could configure this. Ingo From manuel.bertrand at gmail.com Tue Feb 7 14:20:14 2012 From: manuel.bertrand at gmail.com (manuel.bertrand at gmail.com) Date: Tue, 07 Feb 2012 13:20:14 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <4F3116FE.4080704@gmail.com> Le 06/02/12 22:26, Ingo Thierack a ?crit : > Hello, > > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. > > If I do an search in an mailfolder, i get in the dovecot-log > > 2012-02-06 22:17:11 | dovecot: | > indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot > > > Log from solr. > Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/select > params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} > hits=0 status=0 QTime=1 > > > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. > > Regard > Ingo Thierack Same thing here. Tried with 2.1-rc1 and rc5. No results. From moseleymark at gmail.com Wed Feb 8 01:51:50 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 7 Feb 2012 15:51:50 -0800 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: On Tue, Feb 7, 2012 at 4:08 AM, Mark Zealey wrote: > 06-02-2012 22:47, Timo Sirainen yazm??: > >> On 3.2.2012, at 16.16, Mark Zealey wrote: >> >>> I was doing some testing on sdbox yesterday. Basically I did the >>> following procedure: >>> >>> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >>> 2) Create a copy of the index file (no cache file created yet) >>> 3) deliver another message to the mailbox (u.3) >>> 4) copy back index file from stage (2) >>> 5) deliver new mail >>> >>> Then the message delivered in stage 3 ie u.3 gets replaced with the >>> message delivered in (5) also called u.3. >> >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > > I've not actually tried this patch yet, but looking at it, it is perhaps > useful for the situation I described below when the index is corrupt. In > this case I am describing however, the not is NOT corrupt - it is simply an > older version (ie it only thinks there are the first 2 mails in the > directory, not the 3rd). This could happen for example when mails are being > stored on different storage than indexes; say for example you have 2 servers > with remote NFS stored mails but local indexes that rsync between the > servers every hour. You manually fail over one server to the other and you > then have a copy of the correct indexes but only from an hour ago. The mails > are all there on the shared storage but because the indexes are out of date, > when a new message comes in it will be automatically overwritten. > >>> (speaking of which, it would be great if force-resync also rebuilt the >>> cache files if there are valid cache files around, rather than just doing >>> away with them) >> >> Well, ideally there shouldn't be so much corruption that this matters.. > > > That's true, but in our experience we usually get corruption in batches > rather than a one-off occurrence. Our most common case is something like > this: Say for example there's an issue with the NFS server (assuming we are > storing indexes on there as well now) and so we have to killall -9 dovecot > processes or similar. In that case you get a number of corrupted indexes on > the server. Rebuilding the indexes generates an IO storm (say via lmtp or a > pop3 access); then the clients log in via imap and we have to re-read all > the messages to generate the cache files which is a second IO storm. If the > caches were rebuilt at least semi-intelligently (ie you could extract from > the cache files a list of things that had previously been cached) that would > reduce the effects of rare storage level issues such as this. > > Mark What about something like: a writer to an index/cache file checks for the existence of .1. If it doesn't exist or is over a day old, if the current index/cache file is not corrupt, take a snapshot of it as .1. Then if an index/cache file is corrupt, it can check for .1 and use that as the basis for a rebuild, so at least only a day's worth of email is reverted to its previous state (instead of all of it), assuming it's been modified in less than a day. Clearly it'd take up a bit more disk space, though the various dovecot.* files are pretty modest in size, even for big mailboxes. Or it might be a decent use case for some sort of journaling, so that the actual index/cache files don't ever get written to, except during a consolidation, to roll up journals once they've reached some threshold. There'd definitely be a performance price to pay though, not to mention breaking backwards compatibility. And I'm just throwing stuff out to see if any of it sticks, so don't mistake this for even remotely well thought-out suggestions :) From hmoreno at gmv.com Wed Feb 8 10:58:30 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Wed, 8 Feb 2012 09:58:30 +0100 Subject: [Dovecot] vsz_limit Message-ID: Hello Timo and dovecot teamwork, We have dovecot 2.0.10 installed with this configuration: # dovecot -n # 2.0.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.18.el5 x86_64 Scientific Linux SL release 5.2 (Boron) auth_cache_size = 500 k auth_cache_ttl = 1000 secs disable_plaintext_auth = no first_valid_uid = 501 lock_method = dotlock mail_fsync = always mail_gid = vmail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota autocreate mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin/autocreate = Spam plugin/autosubscribe = Spam plugin/quota = maildir:_backup:ns= plugin/quota_warning = storage=75%% quota-warning 75 %u plugin/quota_warning2 = storage=90%% quota-warning 90 %u plugin/sieve = %h/.dovecot.sieve plugin/sieve_dir = %h/sieve protocols = imap pop3 sieve service anvil { client_limit = 5000 } service auth { client_limit = 9000 unix_listener auth-master { group = vmail mode = 0700 user = vmail } user = root vsz_limit = 2 G } service imap-login { executable = /usr/local/dovecot/libexec/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2048 process_min_avail = 20 user = vmail } service imap { executable = /usr/local/dovecot/libexec/dovecot/imap process_limit = 2048 } service managesieve-login { executable = /usr/local/dovecot/libexec/dovecot/managesieve-login inet_listener sieve { address = * port = 4190 } inet_listener sieve_deprecated { address = * port = 12000 } user = vmail } service managesieve { executable = /usr/local/dovecot/libexec/dovecot/managesieve } service pop3-login { executable = /usr/local/dovecot/libexec/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 2048 process_min_avail = 20 user = vmail } service pop3 { executable = /usr/local/dovecot/libexec/dovecot/pop3 process_limit = 2048 } service quota-warning { executable = script /usr/local/dovecot/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0700 user = vmail } } ssl_cert = fue rechazado automaticamente por nuestro sistema :%n%r rejection_subject = Rechazado: %s sendmail_path = /usr/sbin/exim } protocol sieve { managesieve_implementation_string = dovecot Pigeonhole managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 } The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? Thank you very much in advanced. Kind regards. ________________________________ [cid:image001.gif at 01CCE647.65B8FB30] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [cid:image002.gif at 01CCE647.65B8FB30] [cid:image003.gif at 01CCE647.65B8FB30] [cid:image004.gif at 01CCE647.65B8FB30] [cid:image005.gif at 01CCE647.65B8FB30] ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From warden at geneseo.edu Wed Feb 8 18:20:31 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 8 Feb 2012 11:20:31 -0500 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <86E1DCD3-F1A8-43B5-AC08-E8CAF9094C38@geneseo.edu> On Feb 8, 2012, at 3:58 AM, H?ctor Moreno Blanco wrote: -snip- > service imap-login { > executable = /usr/local/dovecot/libexec/dovecot/imap-login > inet_listener imap { > address = * > port = 143 > } > inet_listener imaps { > address = * > port = 993 > } > process_limit = 2048 > process_min_avail = 20 > user = vmail > } > service imap { > executable = /usr/local/dovecot/libexec/dovecot/imap > process_limit = 2048 > } - snip, again - > > The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? I'm not 100% sure but if this is the case but it looks like you setting process_limit and process_min_avail in the imap-login service block is putting dovecot in "High Security" mode. http://wiki2.dovecot.org/LoginProcess has good details on that. In short, I suspect your settings are creating a new imap-login process for each new IMAP connection. As I understand it, vsz_limit is more important when running in "High Performance" mode where you have just 1 imap-login process that handles all new IMAP connections. That process can grow quite large if you have a lot of simultaneous IMAP clients. > > Thank you very much in advanced. > > Kind regards. > > ________________________________ > [cid:image001.gif at 01CCE647.65B8FB30] > > H?ctor Moreno Blanco > Divisi?n de Seguridad e Infraestructuras / > Security and Infrastructures Division > > GMV > Isaac Newton, 11 > P.T.M. Tres Cantos > E-28760 Madrid > Tel. +34 91 807 21 00 > Fax +34 91 807 21 99 > www.gmv.com > [cid:image002.gif at 01CCE647.65B8FB30] > > > > [cid:image003.gif at 01CCE647.65B8FB30] > > > > [cid:image004.gif at 01CCE647.65B8FB30] > > > > [cid:image005.gif at 01CCE647.65B8FB30] > > > > > > > ______________________ > This message including any attachments may contain confidential > information, according to our Information Security Management System, > and intended solely for a specific individual to whom they are addressed. > Any unauthorised copy, disclosure or distribution of this message > is strictly forbidden. If you have received this transmission in error, > please notify the sender immediately and delete it. > > ______________________ > Este mensaje, y en su caso, cualquier fichero anexo al mismo, > puede contener informacion clasificada por su emisor como confidencial > en el marco de su Sistema de Gestion de Seguridad de la > Informacion siendo para uso exclusivo del destinatario, quedando > prohibida su divulgacion copia o distribucion a terceros sin la > autorizacion expresa del remitente. Si Vd. ha recibido este mensaje > erroneamente, se ruega lo notifique al remitente y proceda a su borrado. > Gracias por su colaboracion. > > ______________________ > Hope this helps, David Warden From a23s4a2008 at yandex.ru Wed Feb 8 20:56:40 2012 From: a23s4a2008 at yandex.ru (Maxim) Date: Wed, 08 Feb 2012 22:56:40 +0400 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> Message-ID: <4F32C568.4050202@yandex.ru> 07.02.2012 0:27, Timo Sirainen ?????: > On 3.2.2012, at 15.15, User Max wrote: > >> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted > service auth { > user = root > } > > Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. From tss at iki.fi Thu Feb 9 01:26:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:26:04 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: <43ABD2D0-93D3-4E00-AC16-66EC75870EC2@iki.fi> On 7.2.2012, at 14.08, Mark Zealey wrote: >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. I don't recommend using local indexes with dbox, since there is actual data loss if they're not up to date (flags, and with mdbox the user may have copied/moved the mail elsewhere). Still, better to catch this situation than not: http://hg.dovecot.org/dovecot-2.1/rev/09db0f7aa6ce >>> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) >> Well, ideally there shouldn't be so much corruption that this matters.. > > That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Well, the decisions are now remembered: http://hg.dovecot.org/dovecot-2.1/rev/d8d214cc1936 That can't really be improved .. If nothing is deleted from cache, it might contain invalid data and doveadm force-resync wouldn't be doing its job right. If anything is added to cache, it would require reading and parsing the mail contents during rebuild, and that's not in any way better than letting the imap processes do it later when the mailbox isn't locked. From tss at iki.fi Thu Feb 9 01:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:29:20 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? From tss at iki.fi Thu Feb 9 01:36:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:36:50 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> On 6.2.2012, at 23.26, Ingo Thierack wrote: > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. Lucene should work.. > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. You mean you deleted Solr index, so that it's empty? That should work too. Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). From tss at iki.fi Thu Feb 9 01:40:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:40:19 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <4F32C568.4050202@yandex.ru> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> <4F32C568.4050202@yandex.ru> Message-ID: On 8.2.2012, at 20.56, Maxim wrote: >>> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted >> service auth { >> user = root >> } >> >> Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. >> > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. The problem is that auth worker process can't do getpwnam() call. Typically that would read from /etc/passwd, but I don't know if in OpenBSD if you have configured it to read from elsewhere (AD?). Anyway, I don't know why it would give EPERM error since the process is running as root. You could try ktraceing it to see what actually fails. From tss at iki.fi Thu Feb 9 01:42:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:42:11 +0200 Subject: [Dovecot] Fedora 16 configuration In-Reply-To: References: Message-ID: <1B36032D-0D9D-4CED-9D4C-6410C6C57D86@iki.fi> On 7.2.2012, at 22.44, Cliff Hayes wrote: > Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth > attempts): rip=x.x.x.x, lip=x.x.x.x From http://wiki2.dovecot.org/WhyDoesItNotWork: Aborted login (no auth attempts) means that the client isn't even attempting to log in. Most likely you have disable_plaintext_auth=yes (default) and the client isn't configured to use SSL/TLS (or you've also set ssl=no). From tss at iki.fi Thu Feb 9 01:48:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:48:09 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120207082510.GA24432@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) .. > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > other ideas for what might be causing it ? The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS (30 secs). It still shouldn't have crashed of course, and that crash is already fixed in v2.1 (in the LMTP simplification change). Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. From tss at iki.fi Thu Feb 9 01:49:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:49:38 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: Message-ID: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director From tss at iki.fi Thu Feb 9 03:04:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:04:04 +0200 Subject: [Dovecot] force-resync fails to recover all messages in mdbox In-Reply-To: <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> References: <20120131174839.13512v46jc7ur23b.lealanko@webmail.helsinki.fi> <38EB3A30-DFD5-484B-852B-327BDA5E936E@iki.fi> <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> Message-ID: <4BDF4036-EE45-42CC-9A34-D565D222107B@iki.fi> On 31.1.2012, at 18.34, Lauri Alanko wrote: > Well, well, well. This is interesting. Back with the indices > created by dsync: > > $ doveadm fetch guid all | grep guid: | sort | uniq -c | sort -n | tail > 17 guid: 1b28b22d4b2ee2885b5b81221c41201d > 17 guid: 730c692395661dd62f82088804b85652 > 17 guid: 865e1537fddba6698e010d0b9dbddd02 .. http://hg.dovecot.org/dovecot-2.0/rev/4a0b7dec3a22 avoids force-resync deleting these duplicates. It also logs a warning about the duplicates. http://hg.dovecot.org/dovecot-2.1/rev/2500de8f1f51 implements mbox_md5=all setting which avoids creation of these duplicates in the first place. I thought about adding some duplicate detection also to dsync (or anywhere in its path), but I couldn't do it without impacting performance in normal operation. > The complexity and opaqueness of the mdbox format is a worrisome. > It would ease my mind quite a bit if there were a simple tool > that would just dump out the plain message contents that are > stored inside the storage files, without involving any of > dovecot's index machinery. Then I would at least know that > whatever happens, as long as the storage files stay intact, I can > always migrate my mails into some other format. By using Dovecot indexes you could use e.g. "doveadm fetch" to dump them. Also "doveadm dump" can dump the dbox files' metadata, but not the message contents themselves. It probably wouldn't be difficult to implement that though. Also alternatively you could build something based on http://dovecot.org/tools/mdbox-obfuscate.pl From tss at iki.fi Thu Feb 9 03:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:45:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? In-Reply-To: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> References: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 19.59, Lauri Alanko wrote: > When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. > > This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Fixed: http://hg.dovecot.org/dovecot-2.0/rev/d83356f1fce6 From tss at iki.fi Thu Feb 9 03:59:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:59:30 +0200 Subject: [Dovecot] LSUB and subscribed folders In-Reply-To: <201112221334.06087.p.heinlein@heinlein-support.de> References: <201112221334.06087.p.heinlein@heinlein-support.de> Message-ID: <3477C497-8041-4F0C-8FB2-FA90394A231B@iki.fi> Hi, I've finally gotten to last year's mail backlog. :) On 22.12.2011, at 14.34, Peer Heinlein wrote: > User "bar" has some folders and one shared folder from "foo" if I use the > "LIST"-command, which should list AFAIK *all* folders: Yes. > Using the LSUB-command, which should only list subscribed folders I can see > some more shared folders. AFAIK the output of LSUB must be always less then > LIST: As Michael mentioned, LSUB can show some nonexistent mailboxes also that haven't been unsubscribed. > And why is /shared/foo/INBOX listed with "HasNoChildren"? It has no visible existing children (the LSUB entries didn't exist or no longer had necessary ACLs). > And, btw: Looks like's not possible to use the "doveadm"-command to get a > list of a user's complete folder list. Maybe "doveadm mailbox" should get > some extra parameters to geht all folders / only subscribe folders / only > shared folders: Yes, that would be nice .. but maybe this is enough for now: http://hg.dovecot.org/dovecot-2.1/rev/b600af017598 From tss at iki.fi Thu Feb 9 04:18:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:18:50 +0200 Subject: [Dovecot] crash on shared folders if remote user has empty $home In-Reply-To: <201112222201.41781.p.heinlein@heinlein-support.de> References: <201112222201.41781.p.heinlein@heinlein-support.de> Message-ID: On 22.12.2011, at 23.01, Peer Heinlein wrote: > d) If foo logs in, dovecot's exploring the shared folders given from the > non-existing user bar. But bar's entry in LDAP doesn't have $home any > more. imap crashes and closes the TCP-connection immediately. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/09f964850245 From tss at iki.fi Thu Feb 9 04:29:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:29:09 +0200 Subject: [Dovecot] doveadm index In-Reply-To: <4EF218F7.2050709@gmx.de> References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: On 21.12.2011, at 19.35, e-frog wrote: >> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' > > Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. Yes, looks like it does. I changed the doveadm help output to make it clearer. > At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. > > Is there a way to examine the search index? Doveadm dump doesn't seem to like it. > > $: doveadm dump dovecot.index.search > $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. From tss at iki.fi Thu Feb 9 04:31:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:31:44 +0200 Subject: [Dovecot] Syncing shared mailboxes In-Reply-To: <4EE62AEB.2020607@graband.de> References: <4EE62AEB.2020607@graband.de> Message-ID: Hi, On 12.12.2011, at 18.25, Knud Dombrowsky wrote: > while trying to sync the mailboxes of several users who use and share > their mailboxes dsync prints this message: > > dsync-local(): Warning: Subscriptions file > /home//Maildir/subscriptions: Removing > invalid entry: shared// > > The problem is: Every user has to subscribe the shared folder again > after each run of dsync... > Is dsync capable of syncing shared mailboxes? Are there any > preliminaries to do so? Sorry for late reply. This should be fixed in v2.1 nowadays. From tss at iki.fi Thu Feb 9 04:52:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:52:08 +0200 Subject: [Dovecot] 2.0.14 doveadm proxy list segfault In-Reply-To: <20110929172134.GA25237@hawkeye.rutgers.edu> References: <20110929172134.GA25237@hawkeye.rutgers.edu> Message-ID: <142FE442-A257-4BB5-A8D6-37884C7132D0@iki.fi> Hi, On 29.9.2011, at 20.21, Tom Pawlowski wrote: > Periodic segfaults with doveadm proxy list: > > root at director1:~) doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault (core dumped) Do you still get these? gdb backtrace would be helpful. It already says "core dumped", so all that should be needed is: gdb doveadm core bt full Also it shows connection to "ipc" server dying .. Does it also crash? Anything in Dovecot's error log? From tss at iki.fi Thu Feb 9 05:02:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 05:02:02 +0200 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <20110926165651.GA20769@informatik.tu-muenchen.de> References: <20110926165651.GA20769@informatik.tu-muenchen.de> Message-ID: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Hi, On 26.9.2011, at 19.56, Christoph Bussenius wrote: > /* make sure two hierarchy separators aren't next to each others */ > for (p = storage_name+1; *p != '\0'; p++) { > if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { > client_send_tagline(cmd, "NO Invalid mailbox name."); > return NULL; > } > } > > The loop iterates over the string storage_name starting from its second > byte. However, the string may be "". This is the case if you select > the root of a namespace, like in "SELECT shared" or > "SELECT shared/user1". In that case, the code will read past the end of > the buffer from random memory until it finds a zero byte or a duplicate > separator. I finally managed to get this far in my mail backlog. :) Yes, that's a bug. Fixed now slightly differently than you: http://hg.dovecot.org/dovecot-2.0/rev/d406e376f8ee From ybhu at hk1.ibm.com Thu Feb 9 10:36:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 9 Feb 2012 16:36:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: Thanks Timo, I just tried out the Director. One question is about the re-redirection. I know director will redirect all the simultaneous requests from the same user to only a single server at the same time. The question is how to manage the time period after last connection to re-decide to redirect which machine? director_user_expire? Look like not. I did one test, set director_user_expire = 1 min, then keep sending requests to the director in 2 min interval, the result is it keeps redirect to the same back end server. Actually what i want is the "secondary load balancer layer" can redirect requests to random back end. How to manage it? Only after the files on the previous back end is expired? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 07:49 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic17213.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From hmoreno at gmv.com Thu Feb 9 10:41:34 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 09:41:34 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: Hello Timo, I can see these errors, but I'm not sure if they have something to do with my problem: ... Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe ... Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) ... Do you see anything wrong? Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. I appreciate your answers. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 0:29 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From busseniu at in.tum.de Thu Feb 9 13:31:10 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Thu, 09 Feb 2012 12:31:10 +0100 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> References: <20110926165651.GA20769@informatik.tu-muenchen.de> <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Message-ID: <4F33AE7E.5070008@in.tum.de> Hi, I'm glad to see my report finally arrive, thank you :) On 09.02.2012 04:02, Timo Sirainen wrote: > Fixed now slightly differently than you: No problem ? I agree that my code was a bit kludgy. I noticed that my original mail might be a bit unclear: > > while trying to investigate the bug I reported last week, I found that > > there is a broken boundary check So I just want to make clear that this patch does not fix the other problem that I reported at http://www.dovecot.org/list/dovecot/2011-September/061316.html (?Strange behavior from shared namespaces and INBOX, probably a bug?). Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Thu Feb 9 14:52:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:52:42 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. From tss at iki.fi Thu Feb 9 14:55:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:55:45 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. From janfrode at tanso.net Thu Feb 9 14:56:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 9 Feb 2012 13:56:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: <20120209125652.GA18552@dibs.tanso.net> On Thu, Feb 09, 2012 at 01:48:09AM +0200, Timo Sirainen wrote: > On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > > > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) > .. > > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > > other ideas for what might be causing it ? > > > The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS > (30 secs). It's actually 60 sec in v2.0 http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 > It still shouldn't have crashed of course, and that crash is already fixed in v2.1 > (in the LMTP simplification change). Do you think we should rather run v2.1-rc* on our dovecot directors (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? > Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. Thanks, well consider that option if it crashes too often... Have only seen this problem once for the last week. -jf From tss at iki.fi Thu Feb 9 15:04:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 15:04:16 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120209125652.GA18552@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> <20120209125652.GA18552@dibs.tanso.net> Message-ID: On 9.2.2012, at 14.56, Jan-Frode Myklebust wrote: >>> Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any >>> other ideas for what might be causing it ? >> >> The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> (30 secs). > > It's actually 60 sec in v2.0 > > http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 30. LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is not LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> It still shouldn't have crashed of course, and that crash is already fixed in v2.1 >> (in the LMTP simplification change). > > Do you think we should rather run v2.1-rc* on our dovecot directors > (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? Yes, I've done a lot of improvements to proxying and error handling/logging in v2.1. Also I'm planning on finishing my email backlog soon and making the last v2.1-rc before renaming it to v2.1.0. From tss at iki.fi Thu Feb 9 16:34:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 16:34:22 +0200 Subject: [Dovecot] fts (lucene): indexing of virtual mailboxes? In-Reply-To: References: Message-ID: <1328798062.11500.131.camel@innu> Hi, On Fri, 2011-09-23 at 16:49 +0200, Lutz Pre?ler wrote: > Hello, > > (recent 2.1alpha2 variant - my test setup known to Timo). > No time to diagnose in depth at the moment, but I just noticed > that SEARCHing in virtual mailboxes seems not to create lucene > index content of its own but use those of referenced mailboxes? > The problem is that no new indexing takes place. > Example: with > INBOX > INBOX.in% > all > in dovecot-virtual, for a given query I only get matches from > those mailboxes searched in previously. This was a long time ago, but I just tested and looks like it works nowadays. From apm at one.com Thu Feb 9 16:35:21 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 09 Feb 2012 15:35:21 +0100 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox Message-ID: <4F33D9A9.2070605@one.com> Hi, Considering the scenario, where you have some old account with a different POP3 UIDL format and you migrate them to dovecot. So these old UIDLs would be saved to dovecot-uidlist. At some later time you want to introduce a virtual POP3 INBOX like described on: http://wiki.dovecot.org/Plugins/Virtual So you decide to make the new UIDL format "%f" - to make them unique across folders. So far so good. But then you decide to migrate to mdbox with all your old UIDLs. The docs says that saving old UIDLs is only supported in Maildir and that %f is only supported in Maildir. So is this at all possible? Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? /Peter From hmoreno at gmv.com Thu Feb 9 17:05:09 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 16:05:09 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> Message-ID: Hello Timo, There is no other problem as far as I know. That's why I think is has something to do with dovecot. Specifically with the imap an imap-login process, as I observe in the processes status. Anyway, the system load is not high enough to cause these problems. However, imap service doesn't work properly. So, in order to enable the login process in high performance mode I add the parameter service_count = 0. Right? I'll let you know if this helps. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 13:53 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: > Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: > Error: read(imap) failed: Remote closed connection (process_limit > reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: > fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth > request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From tss at iki.fi Thu Feb 9 17:19:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 17:19:40 +0200 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox In-Reply-To: <4F33D9A9.2070605@one.com> References: <4F33D9A9.2070605@one.com> Message-ID: <1328800780.11500.135.camel@innu> On Thu, 2012-02-09 at 15:35 +0100, Peter Mogensen wrote: > Hi, > > Considering the scenario, where you have some old account with a > different POP3 UIDL format and you migrate them to dovecot. > > So these old UIDLs would be saved to dovecot-uidlist. > > At some later time you want to introduce a virtual POP3 INBOX like > described on: > http://wiki.dovecot.org/Plugins/Virtual > > So you decide to make the new UIDL format "%f" - to make them unique > across folders. > > So far so good. Assuming the messages are in the same order, so far so good. > But then you decide to migrate to mdbox with all your old UIDLs. > The docs says that saving old UIDLs is only supported in Maildir and > that %f is only supported in Maildir. > > So is this at all possible? > > Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? %g and %f are equal with Maildir. And if you migrated with dsync from maildir to mdbox, then all GUIDs and POP3 UIDLs are preserved. But test it first! The main potential problem is that although UIDLs are preserved, their order isn't and POP3 clients don't like the order changing. With Maildir uidlist you can reorder POP3 mails to different than IMAP mails, but with mdbox you can't currently. From tss at iki.fi Thu Feb 9 19:34:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:34:06 +0200 Subject: [Dovecot] [dovecot] Getting duplicates when using snarf plugin with mbox backend In-Reply-To: <4DF609B1.8020309@psu.edu> References: <4DF609B1.8020309@psu.edu> Message-ID: <1328808846.11500.136.camel@innu> On Mon, 2011-06-13 at 08:59 -0400, Jonathan SIegle wrote: > Running dovecot version 2.0.11. To reproduce, open two imap sessions and > issue a check command from each at the same time with new mail in the queue. > > 0 login testuser testpw > 1 select inbox > -- Deliver mail -- > 2 check Finally fixed: http://hg.dovecot.org/dovecot-2.0/rev/76220f2b5966 From tss at iki.fi Thu Feb 9 19:52:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:52:43 +0200 Subject: [Dovecot] BUG(?): Incorrect responses for ACL prohibited actions In-Reply-To: <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> References: <20110510124458.Horde.LnlpeIF5lbhNyYeqlCXUIdA@bigworm.curecanti.org> <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> Message-ID: <1328809963.11500.146.camel@innu> Hi, Continuing this old thread: On Tue, 2011-05-10 at 22:46 -0600, Michael M Slusarz wrote: > But as far as the NO for a non-silent STORE, it seems that RFC 4314 > [4] disagrees with you: > > STORE operation SHOULD NOT fail if the user has rights to modify > at least one flag specified in the STORE, as the tagged NO > response to a STORE command is not handled very well by deployed > clients. > > To me, the negative inference from this statement would be: "STORE > operation SHOULD fail if the user has no rights to modify at least one > flag specified in the STORE." That's not the negative of it. :) > At a minimum, a NOPERM response should be thrown, or else there is no > feedback at all why the flag was not set (without parsing ACLs). Perhaps OK [NOPERM] or some other kind of informational message about it .. But there's no way to do it with Dovecot's current API. Also RFC 3501 recommends implementing "session flags" for flags that cannot be permanently stored. So even if user doesn't have access to set any flags, a "well behaving IMAP server" (so not Dovecot :( ) would set those flags for the duration of the current session. Anyway, you can look at PERMANENTFLAGS reply to see if it's possible to set the flag, no need to look at ACLs. > >> My reading of this is that NOPERM should be returned for ANY ACL > >> prohibited action, not just for selecting or creating a mailbox. > >> Dovecot 2.0.12 does not return NOPERM for DELETE/EXPUNGE actions > >> (at a minimum) that are prohibited. > > > > I'm not really sure. Maybe for EXPUNGE a NO would be okay. For flag > > changes it's just annoying to see clients popup pointless error > > messages when trying to set a \Seen flag (or \Answered flag when > > replying). Apparently I've tried this earlier, since there's a comment in code: ret = acl_mailbox_right_lookup(_mail->box, ACL_STORAGE_RIGHT_EXPUNGE); if (ret <= 0) { /* if we don't have permission, silently return success so users won't see annoying error messages in case their clients try automatic expunging. */ From tss at iki.fi Thu Feb 9 20:00:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:00:05 +0200 Subject: [Dovecot] Crash on mail folder delete In-Reply-To: References: <4F20922C.60206@amfes.com> <54B5D728-EC26-4633-A927-7EC040043BF5@iki.fi> <4F20939E.4010903@amfes.com> Message-ID: <1328810405.11500.151.camel@innu> On Wed, 2012-01-25 at 16:04 -0800, Daniel L. Miller wrote: > On 1/25/2012 3:43 PM, Daniel L. Miller wrote: > > On 1/25/2012 3:42 PM, Timo Sirainen wrote: > >> On 26.1.2012, at 1.37, Daniel L. Miller wrote: > >> > >>> Attempting to delete a folder from within the trash folder using > >>> Thunderbird. I see the following in the log: > >> Dovecot version? > >> > > 2.1.rc3. I'm compiling rc5 now... > > > Error still there on rc5. > > Jan 25 16:03:47 bubba dovecot: imap(dmiller at amfes.com): Panic: file > mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: > (mailbox_list_is_valid_pattern(_list, name)) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/95a9428fe68b From tss at iki.fi Thu Feb 9 20:24:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:24:58 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> Message-ID: <1328811898.11500.157.camel@innu> On Sat, 2010-12-25 at 10:08 +0100, Thomas Leuxner wrote: > plugin { > quota = dict:user::file:%h/mdbox/dovecot-quota > quota_rule = *:storage=1GB > quota_rule2 = Trash:storage=+10%% > } > > Kick off a manual backup: > > $ dsync -u user at domain.tld backup mdbox://mdbox > > This results in doubling the quota for the backed up user. This is problematic. With dict quota you'll have this problem, because both source and destination uses the same file. So it would kind of make sense to disable quota for the destination dsync.. Except with Maildir++ the quota is stored in the Maildir root directory. There are no problems with dsyncing it, and you most likely wouldn't want quota disabled there. So .. I'm not really sure what I can do about this. There are some workarounds you could do, like: dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= (works only with latest 2.0/2.1 hg, but with older versions you could do e.g. -o mail_plugins=) Still, it would be nice if there was some generic solution to this. Perhaps the destination username should be something different, like "backup". In dict-sql case then it would modify "backup" user's quota. For dict-file the %h could maybe expand to backup user's homedir.. The backup username probably should be a parameter to dsync I guess.. But an extra parameter wouldn't fix this automatically.. From tss at iki.fi Thu Feb 9 20:39:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:39:57 +0200 Subject: [Dovecot] 2.1.rc1 (8a63f621bd2e): SiS permission issue + crash In-Reply-To: <4EE2D39C.2030106@localhost.localdomain.org> References: <4EE2D39C.2030106@localhost.localdomain.org> Message-ID: <1328812797.11500.159.camel@innu> On Sat, 2011-12-10 at 04:35 +0100, Pascal Volk wrote: > dsync -u test-0 at example.com mirror maildir:/tmp/Maildir > rm -rf Maildir && cp -a Maildir_org Maildir && chown -R 70010:70002 Maildir > dsync -vu test-1 at example.com mirror maildir:/tmp/Maildir > dsync(test-1 at example.com): Error: stat(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.4f06409857c627e0) failed: Permission denied > dsync(test-1 at example.com): Error: safe_mkstemp(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.) failed: Permission denied > dsync(test-1 at example.com): Panic: file dsync-worker-local.c: line 1644 (local_worker_save_msg_continue): assertion failed: (ret == -1) I couldn't reproduce this crash, but I guess this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/e29bc3eb0ba6 Also fixed a related problem where if dbox failed to save a message it still added it to index: http://hg.dovecot.org/dovecot-2.1/rev/98a59ac1f3d0 From tss at iki.fi Thu Feb 9 20:54:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:54:02 +0200 Subject: [Dovecot] Segfaul probably during dsync In-Reply-To: <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> References: <1c3f5d96e58a77359a4b56dac645eeda@127.0.0.1> <277a7ac434966ec84de651577ae7ebbe@127.0.0.1> <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> Message-ID: <1328813642.11500.161.camel@innu> On Tue, 2011-04-12 at 14:52 +0200, Matthias Rieber wrote: > Hi, > > it's caused by a virtual folder: > > INBOX.IBX.Folder1 > INBOX.Ordner.Folder1 > INBOX.Ordner.Folder1.* > OR (OR (OR HEADER FROM bar.com HEADER FROM bar.de) HEADER FROM > foo.com) HEADER FROM barfoos.net NOT HEADER FROM root@ NOT HEADER FROM > www-data@ SINCE 1-Jan-2010 > > When I delete the dovecot.index.search* files it works for a while but > fails again. Are you still getting these crashes? I tried to reproduce but couldn't. From tss at iki.fi Thu Feb 9 21:02:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:02:03 +0200 Subject: [Dovecot] Strange behavior from shared namespaces and INBOX, probably a bug In-Reply-To: <20110923121329.GA13235@informatik.tu-muenchen.de> References: <20110923121329.GA13235@informatik.tu-muenchen.de> Message-ID: <1328814123.11500.163.camel@innu> On Fri, 2011-09-23 at 14:13 +0200, Christoph Bussenius wrote: > Some folders of user1, including the INBOX, have been shared using these IMAP commands: > . login user1 XXXXXX > . setacl INBOX user2 lrwstiekx > . setacl box-a user2 lrwstiekx > > Now if we use telnet to log in as user2 and select "shared/user1", it will contain the same > mails as "shared/user1/INBOX". > > The really strange thing is that "SELECT"-ing "shared/user1" succeeds only > if it is the first command afted logging in. If it not the first > command (if e. g. the "LIST" or "SELECT" command has already been used), > then dovecot will report that the mailbox does not exist. v2.1 always fails to select "shared/user1". I don't think I'll bother figuring out why v2.0 doesn't, might not be an easy fix. Much of the code related to this was rewritten in v2.1. From tss at iki.fi Thu Feb 9 21:47:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:47:47 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328212183-sup-560@rose.makesad.us> References: <1328212183-sup-560@rose.makesad.us> Message-ID: <1328816867.11500.167.camel@innu> On Thu, 2012-02-02 at 14:59 -0500, Asheesh Laroia wrote: > I'm guessing this is some bad interaction with sdbox and partial file downloads? > > I haven't read the code for this, but I would guess the dsync process isn't being atomic > about file transfers, so it is leaving half-completed transfers in place, which results > in corrupt files when they're next examined. There were some problems related to this in dbox, although in your case it seems to be worse than what it should.. I've anyway done several fixes in v2.1. Can you try if these problems happen with it too? And in any case cleanup the dbox from the *.broken files, so that "doveadm force-resync" won't give any errors. From tss at iki.fi Thu Feb 9 23:15:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 23:15:15 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328816867.11500.167.camel@innu> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> Message-ID: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> On 9.2.2012, at 21.47, Timo Sirainen wrote: > I've anyway done several fixes in v2.1. Can you try if these problems > happen with it too? And in any case cleanup the dbox from the *.broken > files, so that "doveadm force-resync" won't give any errors. A bit more specifically: The last such dbox bug was fixed only today, so you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this week. And in general: It would be helpful to have a clean fully working dbox, and then know the *first* error(s) that gets printed about dsync corrupting it. Otherwise it's difficult to guess what are some old problems and what are new ones and which problems happens only because of another problem. From asheesh at asheesh.org Thu Feb 9 23:16:32 2012 From: asheesh at asheesh.org (Asheesh Laroia) Date: Thu, 9 Feb 2012 16:16:32 -0500 (EST) Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> Message-ID: On Thu, 9 Feb 2012, Timo Sirainen wrote: > On 9.2.2012, at 21.47, Timo Sirainen wrote: > >> I've anyway done several fixes in v2.1. Can you try if these problems >> happen with it too? And in any case cleanup the dbox from the *.broken >> files, so that "doveadm force-resync" won't give any errors. > > A bit more specifically: The last such dbox bug was fixed only today, so > you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this > week. > > And in general: It would be helpful to have a clean fully working dbox, > and then know the *first* error(s) that gets printed about dsync > corrupting it. Otherwise it's difficult to guess what are some old > problems and what are new ones and which problems happens only because > of another problem. Good to know. This weekend I can try to set up something of a 'lab' for testing dsync + (s)dbox, both to see if I can reproduce the errors with the old versions, and to see if the new versions fix them. I'll keep in mind the consideration of knowing the first error that gets printed! -- Asheesh. From ybhu at hk1.ibm.com Fri Feb 10 05:12:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 11:12:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic08408.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ybhu at hk1.ibm.com Fri Feb 10 10:08:10 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 16:08:10 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: Searched the archive, and got the doveadm -a director-admin for local doveadm access, and -a host: port for remote doveadm access. And you said: http://dovecot.org/list/dovecot/2010-July/050731.html Now in my director, I have configed the userdb passwd, but the same error occur: doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but userdbs don't support it [root at c-dev1ws--01-01 conf_director.d]$dovecot -n -c /usr/local/etc/dovecot/dovecot_director.conf # 2.0.13: /usr/local/etc/dovecot/dovecot_director.conf # OS: Linux 2.6.18-274.17.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_debug = yes auth_debug_passwords = yes auth_mechanisms = xpreauth auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot_director director_mail_servers = 9.119.7.129 director_servers = 9.119.7.60:9090 disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = mdrop mail_nfs_index = yes mail_nfs_storage = yes mail_uid = mdrop mmap_disable = yes passdb { args = proxy=y port=144 nopassword=y driver = static } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { address = 9.119.7.60 port = 10143 } } ssl = no userdb { driver = passwd } By the way, I run the director and backend in the same server. Andy YB Hu/Hong Kong/IBM at IBMHK Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/10/2012 11:12 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic21544.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ms at mike2k.de Fri Feb 10 14:13:17 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 10 Feb 2012 13:13:17 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner Message-ID: Hi, I have shared one of my mailboxes to another user providing him full rights to that mailbox. $ doveadm acl get -u user1 doc ID Global Rights user=user2 admin create delete expunge insert lookup post read write write-deleted write-seen When that user creates a new mailbox within the shared mailbox, the mailbox is successfully created but with an empty ACL. $ doveadm acl get -u user1 doc/hallo ID Global Rights According to the wiki, it should inherit the ACL from its parent, i.e. have full permissions for user2. Dovecot logs the following errors when I access the parent folder the first time after the other user created the child mailbox: dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: Recent flags state corrupted for mailbox Sent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent This is on dovecot 2.1rc5. -Michael From giles at coochey.net Fri Feb 10 16:36:53 2012 From: giles at coochey.net (Giles Coochey) Date: Fri, 10 Feb 2012 14:36:53 +0000 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! Message-ID: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> So I went and migrated a mailbox from mdbox from maildir format. All seemed ok, but I've found that I now have some missing folders. The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What can be done? (I have a backup of the maildir box). Note though that mdbox is now a live mailbox and has sent and received mails since the migration... -- Message sent via my webmail account. From alexwbaule at gmail.com Fri Feb 10 19:39:39 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 10 Feb 2012 15:39:39 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? Message-ID: Hy Everyone...and Timo ! There is a way to know if a email was expunged (deleted from hard disk) inside a plugin ? I rewrite the expunge function, but the real expunge function is void, i can't know if was really expunged. There is a way to know this ? From mildred593+dovecot at gmail.com Fri Feb 10 20:54:36 2012 From: mildred593+dovecot at gmail.com (Mildred Ki'Lya) Date: Fri, 10 Feb 2012 19:54:36 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, I am trying to set up my own mail server (to get off Google) and I have a problem with mail delivery. I'm using Postfix as a SMTP front-end, and use the LMTP server included in dovecot to receive mail from postfix. I have a setup which works with the dovecot LDA, it involves calling dovecot-lda with the -d flag. Postfix computes the username by removing what's after the recipient_delimiter and the domain from the mail address, and gives this username to dovecot. But, I'd prefer using LMTP, as it is possible to reject unknown e-mails without having to duplicate the user database (it is explained here: http://wiki2.dovecot.org/LDA/Postfix ) I set up LMTP so that postfix uses the socket provided by dovecot, and it works. Only I get the following error message in the logs: Feb 10 18:05:53 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred at mildred.fr(in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> Feb 10 18:06:52 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred-test at mildred.fr (in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> I followed the explanations here: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP The problem is that my passwd-like file only contains a user named "mildred". I tried to add a user mildred at mildred.fr and mildred%mildred.frbut it doesn't work either. Do you know how I could tell dovecot to guess the username correctly ? Thanks, Mildred -- Mildred Ki'Lya http://mildred.fr From wrosenauer at gmail.com Fri Feb 10 22:27:51 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:27:51 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, On Fri, Feb 10, 2012 at 7:54 PM, Mildred Ki'Lya wrote: > I followed the explanations here: > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > The problem is that my passwd-like file only contains a user named > "mildred". I tried to add a user mildred at mildred.fr and > mildred%mildred.frbut it doesn't work either. > > Do you know how I could tell dovecot to guess the username correctly ? I think you need to set auth_username_format = %Ln which strips the domain part from the recipient. Wolfgang From e-frog at gmx.de Fri Feb 10 22:31:45 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 10 Feb 2012 21:31:45 +0100 Subject: [Dovecot] doveadm index In-Reply-To: References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: <4F357EB1.4000103@gmx.de> On 09.02.2012 03:29, wrote Timo Sirainen: > On 21.12.2011, at 19.35, e-frog wrote: > >>> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' >> >> Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. > > Yes, looks like it does. I changed the doveadm help output to make it clearer. Thanks. >> At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. >> >> Is there a way to examine the search index? Doveadm dump doesn't seem to like it. >> >> $: doveadm dump dovecot.index.search >> $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search > > No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. Ok, I'm meanwhile running v2.1 and switched to fts-lucene. Works like a charm! From wrosenauer at gmail.com Fri Feb 10 22:46:01 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:46:01 +0100 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 Message-ID: Hi, I finally migrated my IMAP server from cyrus to dovecot2. I have been using cyrus since many years and have not much experience with dovecot yet though. I've converted my imap server using cyrus2dovecot (several times until I thought it was correct) but there is one thing left which confuses me and I'm not sure what I can or need to do to change it. With Cyrus I used unixhierarchysep: yes and subfolders within Cyrus are always below INBOX. I'm used to that in my IMAP clients and wanted to preserve it with Dovecot. So during migration I finally made sure to create the subfolder again below INBOX wich looks fine in Thunderbird for example. But my mobile devices (Android based with K9) were displaying the folders with Cyrus just with their name w/o the INBOX prefix. (Mobile clients are bad in displaying folder hierarchies so it was convenient to have them looking like they were in parallel to the Inbox.) Now I get the folders displayed as INBOX.whatever and my Android mail clients fail to map them to the special (and localized) folders like Trash. While that seems consistent to my current hierarchy it's not what I wanted. So basically my two questions are: - Can I configure Dovecot in a way that it looks like Cyrus? Having subfolders below Inbox but let clients like those on Android display them besides Inbox? - If that's not possible, can I move the subfolders up one level so they are next to the Inbox? How would I do that? Sorry for the beginner questions. I played around with Dovecot before but using it in production environment disclosed some fine differences. Thanks, Wolfgang From glenmarton at gmail.com Fri Feb 10 23:54:42 2012 From: glenmarton at gmail.com (Glen Johnson) Date: Fri, 10 Feb 2012 16:54:42 -0500 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. Message-ID: I have an IMAP mail server running Postfix, and Dovecot. I have an old Notebook computer which I can view all e-mails. The problem I am having is when I try to use any other computer on the network to view the e-mail I can not see the contents of all the folders. I can see some of the folders and their content but not all. To answer a couple of preliminary questions: dovecot version 2.0.13, postfix version 2.8.3 dovecot.conf ------------------ # 2.0.13: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 4.0.1 i386 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir/ passdb { driver = passwd } passdb { args = /usr/pkg/etc/dovecot/passwd driver = passwd-file } protocols = imap service auth { user = root } service imap-login { executable = /usr/pkg/libexec/dovecot/imap-login } ssl_cert = References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 09.02.2012 um 19:24 schrieb Timo Sirainen: > > So .. I'm not really sure what I can do about this. There are some > workarounds you could do, like: > > dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= > Thanks tried this morning. I think it has some issues with options and crashes without options :/ $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] Aborted $ dovecot --version 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sat Feb 11 10:08:29 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 09:08:29 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 11.02.2012 um 08:58 schrieb Thomas Leuxner: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] > Aborted After having some coffee, above was inflicted by having insufficient disk space. Shouldn't crash anyway I guess. > $ dovecot --version > 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cannewilson at googlemail.com Sat Feb 11 12:09:29 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Sat, 11 Feb 2012 10:09:29 +0000 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. In-Reply-To: References: Message-ID: <201202111009.29259.cannewilson@googlemail.com> On Friday 10 February 2012 21:54:42 Anne Wilson wrote: > I have an IMAP mail server running Postfix, and Dovecot. I have an old > Notebook computer which I can view all e-mails. The problem I am having is > when I try to use any other computer on the network to view the e-mail I > can not see the contents of all the folders. I can see some of the folders > and their content but not all. Does your mail client support subscriptions? Check your menus for this - you may find that only some folders are subscribed to. Anne From tlx at leuxner.net Sat Feb 11 12:59:16 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 11:59:16 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox $ doveadm -f table quota get -u tlx at leuxner.net Please look into the crash though. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mark.zealey at webfusion.com Sat Feb 11 16:33:34 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 11 Feb 2012 16:33:34 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP Message-ID: <4F367C3E.2080806@webfusion.com> Hi there, I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: 451 4.3.0 Temporary user lookup failure and in the logs: 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly I'm sure this is an obvious issue for someone who's come across it before. The /var/run/dovecot/director-userdb socket is on the system; exactly the same permissions as the standard auth-userdb Here are the relevant bits of my config (I'm setting lmtp to listen on a different port than normal). Backend password database is mysql if that makes any difference. lmtp_proxy = yes service lmtp { inet_listener lmtp_test { port = 2450 } } protocol lmtp { auth_socket_path = director-userdb } Thanks, Mark From clube03 at gmail.com Sat Feb 11 20:46:32 2012 From: clube03 at gmail.com (Gustavo) Date: Sat, 11 Feb 2012 16:46:32 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: Ok, I study the wikki and the dovecot works. But when I try to send a mail using squirrel, I get this error on logs: Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from localhost.localdomain[127.0.0.1] Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication mechanisms Feb 11 18:41:56 jubileu postfix/master[18383]: warning: process /usr/lib/postfix/smtpd pid 19522 exit status 1 Feb 11 18:41:56 jubileu postfix/master[18383]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling How can I fix this? My conf files seems to be ok: ## Diretorio Base ## base_dir = /var/run/dovecot/ ## Protocolos que ele ira executar ## protocols = imap imaps pop3 pop3s log_timestamp = '%Y-%m-%d %H:%M:%S ' syslog_facility = mail ## Dados do SSL ## ssl_listen = localhost ssl_cert_file = /etc/postfix/mail-cert.pem ssl_key_file = /etc/postfix/mail-key.pem ssl_parameters_regenerate = 168 verbose_ssl = no mail_location = maildir:/var/vmail/%d/%u mail_access_groups = mail mail_debug = no first_valid_uid = 150 last_valid_uid = 150 maildir_copy_with_hardlinks = yes protocol imap { } protocol pop3 { # Login executable location. login_executable = /usr/lib/dovecot/pop3-login mail_executable = /usr/lib/dovecot/pop3 pop3_uidl_format = %08Xu%08Xv } protocol lda { sendmail_path = /usr/lib/sendmail auth_socket_path = /tmp/auth-master } auth_verbose = no auth_debug = no auth default { mechanisms = plain login passdb pam { } passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb passwd { } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /tmp/auth-master mode = 0600 user = vmail group = mail } client { path = /tmp/auth-client mode = 0660 user = vmail group = mail } } } dict { } plugin { } ######################################### And Postfix conf: # SASL Authentication smtpd_sasl_auth_enable = yes smtpd_sasl2_auth_enable = yes smtpd_sasl_exceptions_networks = $mynetworks smtpd_sasl_security_options = noanonymous broken_sasl_auth_clients = yes smtpd_sasl_path = private/auth smtpd_sasl_application_name = smtpd smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination permit_mynetworks = 127.0.0.0/8, 184.22.103.88/32, 184.22.103.87/32 Thanks!!! > -----Mensagem Original----- From: Timo Sirainen > Sent: Saturday, January 28, 2012 3:51 PM > To: Gustavo > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Problem with Postfix + Dovecot + MySQL + > Squirrelmail > > On 27.1.2012, at 14.32, Gustavo wrote: > > #service dovecot start >> Starting IMAP/POP3 mail server: dovecotLast died with error (see error >> log for more information): Auth process died too early - shutting down >> > > No need to keep guessing the problem. "See error log for more information" > like it says. http://wiki.dovecot.org/**Logging > > -- Gustavo - clube03 at gmail.com From tss at iki.fi Sun Feb 12 02:24:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 02:24:38 +0200 Subject: [Dovecot] [PATCH] Allow verification of user's hash in doveadm-pw In-Reply-To: References: Message-ID: <4F3706C6.10909@iki.fi> On 09/01/2011 01:17 AM, Spack wrote: > Hi, > > It may be useful for testing purpose to know if a generated password > hash correspond to a given clear password. This can be useful to check > if a hash generated by another program can be verified by Dovecot > without any errors. > > This patch adds the ability the verify a password hash using `doveadm > pw` via the `-V` option. Committed, with a few changes: http://hg.dovecot.org/dovecot-2.1/rev/fffa4d53e901 > + while ((ch = getopt(argc, argv, "lp:r:s:u:V::")) != -1) { "::" isn't standard, so I added a separate -t parameter to give the hash. From tss at iki.fi Sun Feb 12 05:39:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:39:20 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <4F367C3E.2080806@webfusion.com> References: <4F367C3E.2080806@webfusion.com> Message-ID: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> On 11.2.2012, at 16.33, Mark Zealey wrote: > I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: > > 451 4.3.0 Temporary user lookup failure > > and in the logs: > 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly Aren't there any other errors logged? I think there should be some error logged from either auth or director process. Although director is missing one error, but auth should have also logged an error in that case. Anyway, fixed by http://hg.dovecot.org/dovecot-2.1/rev/9b70d164e974 From tss at iki.fi Sun Feb 12 05:42:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:42:22 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Message-ID: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> On 11.2.2012, at 12.59, Thomas Leuxner wrote: > Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: > > $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. From tss at iki.fi Sun Feb 12 05:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:45:45 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> On 11.2.2012, at 9.58, Thomas Leuxner wrote: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 From tss at iki.fi Sun Feb 12 06:02:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:02:45 +0200 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 In-Reply-To: References: Message-ID: <5BD5F73F-E9D6-4803-BA51-59CC7B0D338E@iki.fi> On 10.2.2012, at 22.46, Wolfgang Rosenauer wrote: > With Cyrus I used > unixhierarchysep: yes > and subfolders within Cyrus are always below INBOX. .. > - Can I configure Dovecot in a way that it looks like Cyrus? Having > subfolders below Inbox but let clients like those on Android display > them besides Inbox? namespace { prefix = INBOX/ separator = / } From tss at iki.fi Sun Feb 12 06:06:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:06:58 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. From tss at iki.fi Sun Feb 12 06:09:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:09:55 +0200 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! In-Reply-To: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> References: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> Message-ID: <3F5BB562-6484-43B0-8914-434694F446DB@iki.fi> On 10.2.2012, at 16.36, Giles Coochey wrote: > So I went and migrated a mailbox from mdbox from maildir format. > All seemed ok, but I've found that I now have some missing folders. > > The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What Dovecot version? And give some specifics, like outputs of: ls -a ~/Maildir/ ls ~/mdbox/mailboxes/ or something similar. From tss at iki.fi Sun Feb 12 06:16:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:16:00 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: References: Message-ID: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> On 10.2.2012, at 14.13, Michael Stilkerich wrote: > I have shared one of my mailboxes to another user providing him full rights to > that mailbox. > > $ doveadm acl get -u user1 doc Is "doc" the namespace prefix? > When that user creates a new mailbox within the shared mailbox, the mailbox is successfully > created but with an empty ACL. Creates how? With IMAP client? > According to the wiki, it should inherit the ACL from its parent, i.e. have > full permissions for user2. Works with me. dovecof -n output? > Dovecot logs the following errors when I access the parent folder the first time > after the other user created the child mailbox: > > dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 > dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX > dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Are you trying to use per-user indexes? That's not going to work. From tss at iki.fi Sun Feb 12 06:19:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:19:38 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: Message-ID: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> On 10.2.2012, at 19.39, Alex Baule wrote: > Hy Everyone...and Timo ! > > There is a way to know if a email was expunged (deleted from hard > disk) inside a plugin ? I rewrite the expunge function, but the real > expunge function is void, i can't know if was really expunged. > > There is a way to know this ? It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. From tss at iki.fi Sun Feb 12 07:05:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:05:37 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSN for overquota. In-Reply-To: References: Message-ID: <4F3748A1.3030301@iki.fi> On 12/22/2010 08:08 PM, Mark Sapiro wrote: > It appears that there is some version of the Dovecot LDA that is > sending an RFC 3798 MDN rather than an RFC 3464 DSN to report > non-delivery to a user who is over quota. Fixed, better late than never :) http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 The MDN sending was simply copied from Cyrus. So Dovecot isn't the only server sending this. From tss at iki.fi Sun Feb 12 07:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:42:28 +0200 Subject: [Dovecot] v2.1.0 status Message-ID: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. I've gone through all of my old mails. Let me know if you think something is still missing / unfixed in v2.1 hg. Preferably today. Next up: Creating dovecot-2.2 tree with great new features. :) From tss at iki.fi Sun Feb 12 07:45:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:45:12 +0200 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> On 12.2.2012, at 7.42, Timo Sirainen wrote: > I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota and Sieve's reject mails now include DSN report instead of MDN report. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From tlx at leuxner.net Sun Feb 12 08:54:41 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:54:41 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: Am 12.02.2012 um 04:42 schrieb Timo Sirainen: > On 11.2.2012, at 12.59, Thomas Leuxner wrote: > >> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: >> >> $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox > > Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. > Indeed I did as it did not like the syntax: $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 08:56:24 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:56:24 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> Message-ID: <285AA29C-A430-44C5-9E9A-BD93A34291B8@leuxner.net> Am 12.02.2012 um 04:45 schrieb Timo Sirainen: > On 11.2.2012, at 9.58, Thomas Leuxner wrote: > >> $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox >> dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 Thanks. Crash gone. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 09:02:22 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 08:02:22 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: Am 01.01.2012 um 10:31 schrieb Thomas Leuxner: > $ doveadm acl get -u tlx at leuxner.net FOO > doveadm(tlx at leuxner.net): Error: Can't open mailbox FOO: Mailbox doesn't exist: FOO > ID Global Rights > $ echo $? > 0 Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Mark.Zealey at webfusion.com Sun Feb 12 10:23:21 2012 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Sun, 12 Feb 2012 08:23:21 +0000 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: Ah I should have checked this before - line before: 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Only other relevant piece of config: service auth { user = dovecot client_limit = 15000 } Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. Mark From listen at constabel.net Sun Feb 12 13:18:09 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 12:18:09 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash Message-ID: <20120212111808.GA27677@herakles.constabel-it.de> Hi, If I delete a folder "foo" in Thunderbird, it is moved to trash. [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/ insgesamt 16K drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 8 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:51 dbox-Mails drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 foo [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxx.de/test1/mdbox/mailboxes/Trash/foo/ insgesamt 12K drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 dbox-Mails [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/foo/dbox-Mails/ insgesamt 12K drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 . drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 .. -rw------- 1 vmail vmail 224 12. Feb 11:52 dovecot.index.log [root at dovecot1][tmp:exec](~) # If I delete foo from trash, thunderbird removed it from view, but if I restart thunderbird, the folder is still in trash. No error in dovecot log, no error in thunderbird. tcpdump: 156 45.101512 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=353 Ack=1617 Win=10144 Len=0 TSV=2414467591 TSER=251091441 157 45.349600 192.168.10.1 192.168.10.117 IMAP Request: 16 list "" "Trash/*" 158 45.349717 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\HasNoChildren) "/" "Trash/foo" 159 45.349756 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=375 Ack=1727 Win=10144 Len=0 TSV=2414467653 TSER=251091503 160 45.395546 192.168.10.1 192.168.10.117 IMAP Request: 17 list "" "Trash/foo/*" 161 45.395624 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\Noselect \HasNoChildren) "/" "Trash/foo/" 162 45.395662 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=401 Ack=1802 Win=10144 Len=0 TSV=2414467665 TSER=251091515 163 45.439998 192.168.10.1 192.168.10.117 IMAP Request: 18 delete "Trash/foo/" 164 45.440067 192.168.10.117 192.168.10.1 IMAP Response: 18 NO [CANNOT] Invalid mailbox name 165 45.440104 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=425 Ack=1839 Win=10144 Len=0 TSV=2414467676 TSER=251091526 166 45.493978 192.168.10.1 192.168.10.117 IMAP Request: 19 IDLE with dovecot 1.2 and Maildir I can delete folders from trash. Something wrong in my config? Thunderbird-Bug? # 2.1.rc5 (b9e74767cd39): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.4 auth_cache_size = 1 M auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot/dovecot-debug.log dict { expire = sqlite:/etc/dovecot/dovecot-dict-expire.conf.ext } disable_plaintext_auth = no hostname = mx4.xxxx.de info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_location = mdbox:~/mdbox mail_plugins = quota notify mail_log expire mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive/2011 { auto = subscribe } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = scheme=PLAIN username_format=%u /etc/imap.userdb driver = passwd-file } plugin { expire = Trash expire2 = Trash/* expire3 = Spam expire_dict = proxy::expire quota = dict:User quota::file:/mailspool1/%d/%n/dovecot-quota quota_rule = *:storage=9000M:messages=1000000 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at localhost protocols = " imap lmtp sieve pop3" ssl_ca = References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <4F37A343.1060702@localhost.localdomain.org> On 02/12/2012 12:18 PM Mike Constabel wrote: > Hi, > > If I delete a folder "foo" in Thunderbird, it is moved to trash. > ? > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. http://hg.dovecot.org/dovecot-2.1/file/b9e74767cd39/doc/example-config/conf.d/20-imap.conf#l47 Regards, Pascal -- The trapper recommends today: deadbeef.1204312 at localdomain.org From CMarcus at Media-Brokers.com Sun Feb 12 13:33:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:33:11 -0500 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: <4F37A377.80707@Media-Brokers.com> On 2012-02-11 1:46 PM, Gustavo wrote: > Ok, I study the wikki and the dovecot works. But when I try to send a mail > using squirrel, I get this error on logs: > > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from > localhost.localdomain[127.0.0.1] > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: > xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication > mechanisms Sorry, but this has nothing to do with dovecot, unless your intention is to use dovecot-sasl with your postfix... it appears you are using cyrus-sasl and postfix on your smtp server, that is where your problem lies. Easiest would be to configure postfix to use dovecot-sasl: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL -- Best regards, Charles From mstevens at imt-systems.com Sun Feb 12 13:37:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 12:37:18 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212111808.GA27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> On 12.02.2012 12:18, Mike Constabel wrote: > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. protocol imap { ... imap_client_workarounds = tb-extra-mailbox-sep } Best regards, Morten From CMarcus at Media-Brokers.com Sun Feb 12 13:38:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:38:11 -0500 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <4F37A4A3.4000204@Media-Brokers.com> On 2012-02-12 12:42 AM, Timo Sirainen wrote: > Next up: Creating dovecot-2.2 tree with great new features.:) Can't wait to read your follow-up email outlining these 'great new features' you expect to make it into 2.2... Thanks so much for dovecot Timo! I am amazed at how far it has come in the last few years, and apparently things will only get better. -- Best regards, Charles From ms at mike2k.de Sun Feb 12 14:14:03 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 13:14:03 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> Message-ID: <4F37AD0B.9040406@mike2k.de> Hi Timo, On 12-Feb-12 5:16, Timo Sirainen wrote: > On 10.2.2012, at 14.13, Michael Stilkerich wrote: > >> I have shared one of my mailboxes to another user providing him full rights to >> that mailbox. >> >> $ doveadm acl get -u user1 doc > > Is "doc" the namespace prefix? No, 'doc' is the name of the shared parent mailbox under that the 2nd user created the new mailbox 'hallo'. To clarify: user1 shares the mailbox 'doc' with max permissions to user2. user2 then creates a new mailbox under the parent 'doc'. >> When that user creates a new mailbox within the shared mailbox, the mailbox is successfully >> created but with an empty ACL. > > Creates how? With IMAP client? Yes, using Thunderbird / Roundcube Webmail. >> According to the wiki, it should inherit the ACL from its parent, i.e. have >> full permissions for user2. > > Works with me. dovecof -n output? The config is attached. The IMAP-Client is signaled an error by dovecot, but the log file does not contain an error. At the filesystem level - The mailbox was created as a sub folder of 'doc'. - It contains nothing but an empty sub folder 'dbox-Mails'. - It does specifically not contain an dovecot-acl file. - The sharing user's (user1) dovecot-acl-list file was not updated to list the newly created subfolder. - The acl dictionary 'shared-mailboxes' was also not touched, but this is probably correct since it already contains a record that user1 shares mailboxes to user2. - When user1 accesses the new mailbox that user2 created, the below error gets logged and a dovecot.index.log file is created in the until then empty 'dbox-Mails' folder of the new mailbox. >> Dovecot logs the following errors when I access the parent folder the first time >> after the other user created the child mailbox: >> >> dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 >> dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX >> dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent > > Are you trying to use per-user indexes? That's not going to work. I don't think so. I'm using dbox and thus have not added a custom :INDEX=... to the location. -Michael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot.conf URL: From listen at constabel.net Sun Feb 12 15:27:19 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 14:27:19 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> Message-ID: <20120212132719.GB27677@herakles.constabel-it.de> Hi, On Sun, 12 Feb 2012, Morten Stevens wrote: > On 12.02.2012 12:18, Mike Constabel wrote: > > >If I delete foo from trash, thunderbird removed it from view, > >but if I restart thunderbird, the folder is still in trash. > > > >No error in dovecot log, no error in thunderbird. > > protocol imap { > ... > imap_client_workarounds = tb-extra-mailbox-sep > } Ok, this seems to work, (I thought I tried it before.. but now it works. :) Another Question to this Option: # tb-extra-mailbox-sep: # With mbox storage a mailbox can contain either mails or submailboxes, # but not both. Thunderbird separates these two by forcing server to # accept '/' suffix in mailbox names in subscriptions list. I thought this is not needed because I use mdbox. But mbox here means also mdbox and sdbox? Also "With mbox storage a mailbox can contain either mails or submailboxes, but not both" is only correct for mbox? In mdbox this is possible. This description should contain a litte bit more info for mbox/sdbox/mdbox? Should I use all 3 workarounds if I don't know which clients are in use? LAYOUT=fs and mdbox Regards, Mike From CMarcus at Media-Brokers.com Sun Feb 12 15:43:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 08:43:51 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <4F37C217.7040707@Media-Brokers.com> Hi Timo, One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? Anyway here is the thread outlining the problem with dovecot's SIS and backups: http://www.dovecot.org/list/dovecot/2011-December/062340.html Have you given this any more thought? -- Best regards, Charles From forumer at smartmobili.com Sun Feb 12 15:52:54 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 14:52:54 +0100 Subject: [Dovecot] Lost in configuration Message-ID: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Hi, I am trying to configure dovecot/postfix with virtual users and sasl auth but there are so many tutorials with mistakes and subtle differences that at the end I am lost. So I am running Ubuntu server 11.04 (natty) and when I installed my server I had followed the following guide http://workaround.org/ispmail/etch Then I have migrated to dovecot 2.1 and now I am trying to allow authenticated user to use my smtp server because for the moment I get : Relay access denied Here is my configuration (first is file hierarchy followed by file contents) and I would like to know what I need to do to allow authenticated user send email. In this tutorial http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql it seems I have to add some configuration lines inside /etc/pam.d/smtp and add database configuration inside /etc/postfix/sasl/smtpd.conf. But what I find weird is the fact I have to configure some sql queries from smtpd.conf while normally dovecot has already all the information to do this kind of query. So before to keep on modifying my config I would like to be sure I am following the right way. Thanks ########################### /etc/postfix ########################### drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ -rw-r--r-- 1 root root 373 2011-07-24 11:50 dynamicmaps.cf -rw-r--r-- 1 root root 1928 2012-02-12 12:45 main.cf -rw-r--r-- 1 root root 5762 2011-08-03 22:06 master.cf -rw-rw---- 1 root postfix 140 2011-07-24 11:43 mysql-virtual-alias-maps.cf -rw-rw---- 1 root postfix 132 2011-07-24 11:47 mysql-virtual-mailbox-domains.cf -rw-rw---- 1 root postfix 128 2011-07-24 11:47 mysql-virtual-mailbox-maps.cf -rw-r--r-- 1 root root 19509 2011-04-05 06:07 postfix-files -rwxr-xr-x 1 root root 8729 2011-04-05 06:07 postfix-script* -rwxr-xr-x 1 root root 25752 2011-04-05 06:07 post-install* drwxr-xr-x 2 root root 4096 2012-02-12 12:41 sasl/ root at xa-12345:/etc/postfix# ll sasl/ drwxr-xr-x 2 root root 4096 2012-02-12 12:41 ./ drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ../ -rw-r--r-- 1 root root 26 2012-02-12 12:41 smtpd.conf ########################### /etc/dovecot ########################### drwxr-xr-x 3 root root 4096 2011-09-24 11:42 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ drwxr-xr-x 2 root dovecot 4096 2012-02-12 12:03 conf.d/ -rw-r--r-- 1 root dovecot 3693 2011-09-23 15:11 dovecot.conf -rw-r--r-- 1 root dovecot 410 2011-09-23 15:11 dovecot-db.conf.ext -rw-r--r-- 1 root dovecot 782 2011-09-23 15:11 dovecot-dict-sql.conf.ext -rw-r--r-- 1 root dovecot 5508 2011-09-23 15:38 dovecot-sql.conf -rw-r--r-- 1 root dovecot 5348 2011-09-23 15:11 dovecot-sql.conf.ext -rw-r--r-- 1 root dovecot 116 2011-09-23 15:11 README ########################### /etc/default/saslauthd ########################### -rw-r--r-- 1 root root 2043 2012-02-12 11:57 /etc/default/saslauthd ########################### /var/spool/postfix/var/run/ ########################### drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ./ drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ../ drwxr-xr-x 2 root sasl 4096 2012-02-12 12:35 saslauthd/ I have added postfix to the sasl group (don't know if it was necessary but I found this instruction on a website) /etc/group: sasl:x:45:postfix /etc/default/saslauthd: ---------------------- START=yes PWDIR="/var/spool/postfix/var/run/saslauthd" PARAMS="-m ${PWDIR}" PIDFILE="${PWDIR}/saslauthd.pid" DESC="SASL Authentication Daemon" NAME="saslauthd" MECHANISMS="pam" MECH_OPTIONS="" THREADS=5 OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" mkdir -p /var/spool/postfix/var/run/saslauthd chown -R root.sasl /var/spool/postfix/var/run/saslauthd Then I have checked saslauth is running after restart: root at xa-12345:/home/vmail# ps -ef | grep saslauthd root 6123 1 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6124 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6125 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6127 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6128 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 Then when I check after restarting if saslauthd socket is created I don't see anything (don't know if it's normal) root at xa-12345:/home/vmail# ls -lh /var/spool/postfix/var/run/saslauthd total 0 /etc/postfix/sasl/smtpd.conf: ----------------------------- pwcheck_method: saslauthd /etc/postfix/main.cf: --------------------- myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache myhostname = mail.foobar.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = foobar.com mydestination = xa-12345.dadibox.com, localhost.dadibox.com, localhost relayhost = mynetworks = 127.0.0.0/8 80.190.190.190 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all #virtual domains virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_mailbox_base = /home/vmail virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf # Dovecot LDA virtual_transport = dovecot dovecot_destination_recipient_limit = 1 debug_peer_level = 2 debug_peer_list = 127.0.0.1 smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination smtpd_sasl_application_name = smtpd broken_sasl_auth_clients = yes /etc/dovecot/conf.d/10-auth.conf: -------------------------------------- disable_plaintext_auth = no !include auth-sql.conf.ext !include auth-static.conf.ext /etc/dovecot/conf.d/auth-static.conf.ext: ---------------------------------------- userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } /etc/dovecot/conf.d/auth-sql.conf.ext: -------------------------------------- passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } /etc/dovecot/dovecot-sql.conf: ------------------------------ driver = mysql connect = host=127.0.0.1 dbname=mailserver user=myuser password=mypassword default_pass_scheme = CRAM-MD5 password_query = SELECT email as user, password FROM view_users WHERE email='%u'; /etc/dovecot/conf.d/10-master.conf: ---------------------------------- service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener lmtp { } } service imap { } service pop3 { } service auth { unix_listener auth-userdb { mode = 0600 user = vmail #group = } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { mode = 0777 } } service auth-worker { } service dict { unix_listener dict { mode = 0600 #user = group = vmail } } From forumer at smartmobili.com Sun Feb 12 16:37:26 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 15:37:26 +0100 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <321c6c9323618a11b96bccf49ac4d135@smartmobili.com> IN my previous email ignore the information about the missing socket because it's actually created -rw------- 1 root root 5 2012-02-12 15:28 saslauthd.pid From jom at grosjo.net Sun Feb 12 16:43:26 2012 From: jom at grosjo.net (Joan Moreau) Date: Sun, 12 Feb 2012 14:43:26 +0000 Subject: [Dovecot] v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <258ebed086b260ca88432119bd0bb042@grosjo.net> Thanks for 2.1.0 Timo Looking forward to beta-test 2.2 :) Le 12/02/2012 05:45, Timo Sirainen a ?crit : > On 12.2.2012, at 7.42, Timo Sirainen wrote: > >> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. > > Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota and Sieve's reject mails now include DSN report > instead of MDN report. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. From alexwbaule at gmail.com Sun Feb 12 16:52:05 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 12:52:05 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: I want to know because i have header and body splited. To erase the body, i must have shure, that header was expunged. Some tests that i made, the client call expunge, but i don't know why (there is notting in mail log) the header was not expunged, nut my body was, because i call it before expunge the header. something like this. if (found_body){ erase_body(); super.expunge(_mail); } Because that expunge that no happen, (the client call, but the email was not expunged) i think to do like this way: if (found_body){ super.expunge(_mail); if(_mail was expunged){ erase_body(); } } TKs Timo ! Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > On 10.2.2012, at 19.39, Alex Baule wrote: > >> Hy Everyone...and Timo ! >> >> There is a way to know if a email was expunged (deleted from hard >> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> expunge function is void, i can't know if was really expunged. >> >> There is a way to know this ? > > It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. > > If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). > > Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. > From tss at iki.fi Sun Feb 12 17:42:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:42:20 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: <52A730C6-230B-42B6-8808-A9F6AE3A44D5@iki.fi> On 12.2.2012, at 8.54, Thomas Leuxner wrote: > Indeed I did as it did not like the syntax: > > $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= > backup: invalid option -- 'o' > doveadm backup [-u |-A] [-S ] [-fR] [-m ] Oh. Hm. I'm not exactly sure why it's saying that, but adding some quotes fixes that: dsync -u tlx at leuxner.net backup 'dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota=' From rob0 at gmx.co.uk Sun Feb 12 17:42:39 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 12 Feb 2012 09:42:39 -0600 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <20120212154239.GG14195@harrier.slackbuilds.org> On Sun, Feb 12, 2012 at 02:52:54PM +0100, forumer at smartmobili.com wrote: > I am trying to configure dovecot/postfix with virtual users and > sasl auth but there are so many tutorials with mistakes and subtle > differences that at the end I am lost. All the more reason to stick to each project's official documentation. The sad reality is that most such tutorials you will find are written by people who should not be writing documentation. > So I am running Ubuntu server 11.04 (natty) and when I installed my > server I had followed the following guide > http://workaround.org/ispmail/etch One of the better ones; I know the author, and he is reasonably competent. But are you sure you need and "ISP-style" mail server? Perhaps something simpler makes sense to begin with. http://www.postfix.org/BASIC_CONFIGURATION_README.html > Then I have migrated to dovecot 2.1 and now I am trying to allow > authenticated user to use my smtp server because for the moment I > get : Relay access denied A Postfix issue, or possibly somewhat on topic here if you are using Dovecot SASL. You do not seem to know whether you are or not. This might be because you are mixing and confusing different tutorials without understanding what they are doing and why. > Here is my configuration (first is file hierarchy followed by > file contents) and I would like to know what I need to do to > allow authenticated user send email. http://www.postfix.org/SASL_README.html#server_dovecot http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > In this tutorial > http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql I have not reviewed this one, but the anchor title is patently absurd. If you are using Postfix and Dovecot, there is NEVER any reason to use Cyrus SASL's saslauthd. Consider that one useless. The rest of this is not worth answering. You went off on a Cyrus tangent which is pointless (and never on topic for this list!) Start over with a better understanding of what you need. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Sun Feb 12 17:46:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:46:12 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Hi, Yeah, you shouldn't do the erasing directly in expunge(), because it may still be aborted. Do it in sync_notify() like quota plugin does. On 12.2.2012, at 16.52, Alex Baule wrote: > I want to know because i have header and body splited. To erase the > body, i must have shure, that header was expunged. Some tests that i > made, the client call expunge, but i don't know why (there is notting > in mail log) the header was not expunged, nut my body was, because i > call it before expunge the header. something like this. > > if (found_body){ > erase_body(); > super.expunge(_mail); > } > > Because that expunge that no happen, (the client call, but the email > was not expunged) i think to do like this way: > > if (found_body){ > super.expunge(_mail); > if(_mail was expunged){ > erase_body(); > } > } > > TKs Timo ! > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >>> Hy Everyone...and Timo ! >>> >>> There is a way to know if a email was expunged (deleted from hard >>> disk) inside a plugin ? I rewrite the expunge function, but the real >>> expunge function is void, i can't know if was really expunged. >>> >>> There is a way to know this ? >> >> It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. >> >> If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). >> >> Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. >> > From tss at iki.fi Sun Feb 12 17:58:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:58:20 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <4F37C217.7040707@Media-Brokers.com> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: On 12.2.2012, at 15.43, Charles Marcus wrote: > One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I'm planning on creating a new "backup" mail storage backend, where you could basically run: doveadm backup -u user at domain backup: And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. Then for restoring you can use the same input for "doveadm import". So in the backups there would be no deduplication, unless implemented by the backup software. > I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? As long as the URL is random enough it shouldn't be possible to guess the links. But if they're HTML mails and such then hyperlinks would send the referrer-url to the next site. I guess you could also add HTTP authentication. From tss at iki.fi Sun Feb 12 18:02:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 18:02:39 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: On 12.2.2012, at 10.23, Mark Zealey wrote: > Ah I should have checked this before - line before: > > 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied > > This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Yeah, the auth-userdb socket default permissions aren't usable pretty much anywhere. In v2.1 it's better, but still not as good as they could be. I'm not sure what the best solution for this is. > Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. You mean if you restart one director some of the state is lost in all of them? I think v2.0 had some bugs related to this, but they should be fixed in v2.1. If you mean that if you stop all of the directors and state is lost when restarting then yes, that does happen. You can run "doveadm director dump" to get the current state and after a restart run its output to get back to the same state. What state specifically are you talking about anyway? From CMarcus at Media-Brokers.com Sun Feb 12 18:15:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 11:15:42 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <4F37E5AE.8060104@Media-Brokers.com> On 2012-02-12 10:58 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.43, Charles Marcus wrote: >> > I'm planning on creating a new "backup" mail storage backend, where > you could basically run: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). > So it would be similar to e.g. PostgreSQL's pg_dump. Then for > restoring you can use the same input for "doveadm import". So in the > backups there would be no deduplication, unless implemented by the > backup software. Hmmm, ok thanks... loss of dedupe on the backup server may or may not be an issue I'll be concerned about, but good to know that there is an option at least. >> It looks like using mimedefang would eliminate the problem with >> backups, and would provide the added benefit of eliminating large >> file attachments for external recipients as well, however I'm >> concerned about security - ie, how do you limit access to these >> attachments to only the recipients of the message with the >> hyperlink in it? > As long as the URL is random enough it shouldn't be possible to guess > the links. But if they're HTML mails and such then hyperlinks would > send the referrer-url to the next site. I guess you could also add > HTTP authentication. Thanks for that too... I'll have to a bit of research on this, because the more I think about it, the more I like the idea, *if* it can be done securely enough... -- Best regards, Charles From alexwbaule at gmail.com Sun Feb 12 18:48:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 14:48:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Tks timo ! I will see the plugin to do like it! Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > Hi, > > Yeah, you shouldn't do the erasing directly in expunge(), because it may > still be aborted. Do it in sync_notify() like quota plugin does. > > On 12.2.2012, at 16.52, Alex Baule wrote: > > > I want to know because i have header and body splited. To erase the > > body, i must have shure, that header was expunged. Some tests that i > > made, the client call expunge, but i don't know why (there is notting > > in mail log) the header was not expunged, nut my body was, because i > > call it before expunge the header. something like this. > > > > if (found_body){ > > erase_body(); > > super.expunge(_mail); > > } > > > > Because that expunge that no happen, (the client call, but the email > > was not expunged) i think to do like this way: > > > > if (found_body){ > > super.expunge(_mail); > > if(_mail was expunged){ > > erase_body(); > > } > > } > > > > TKs Timo ! > > > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > >> On 10.2.2012, at 19.39, Alex Baule wrote: > >> > >>> Hy Everyone...and Timo ! > >>> > >>> There is a way to know if a email was expunged (deleted from hard > >>> disk) inside a plugin ? I rewrite the expunge function, but the real > >>> expunge function is void, i can't know if was really expunged. > >>> > >>> There is a way to know this ? > >> > >> It's not known until transaction_commit() what messages are expunged. > What do you need to know this for? There are a few different ways. > >> > >> If you simply need one of the sessions to definitely know that it > expunged a message, you can do it like quota plugin does. (So even if there > are multiple clients doing EXPUNGE at the same time, quota is never > decreased more than once per mail.) See quota-storage.c and most > importantly quota_mailbox_sync_notify(). > >> > >> Quota code also shows the other method of doing it: keep track of what > mail_expunge()s have been called, and then in transaction commit check if > it succeeds and if it does do what you want to do. > >> > > > > From mark at msapiro.net Sun Feb 12 18:57:36 2012 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 12 Feb 2012 08:57:36 -0800 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: <4F3748A1.3030301@iki.fi> Message-ID: Timo Sirainen wrote: >On 12/22/2010 08:08 PM, Mark Sapiro wrote: >> It appears that there is some version of the Dovecot LDA that is >> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >> non-delivery to a user who is over quota. > >Fixed, better late than never :) >http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 >http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 Thanks. >The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >server sending this. :( -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tss at iki.fi Sun Feb 12 19:00:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:00:45 +0200 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212132719.GB27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> Message-ID: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> On 12.2.2012, at 15.27, Mike Constabel wrote: > Another Question to this Option: > > # tb-extra-mailbox-sep: > # With mbox storage a mailbox can contain either mails or submailboxes, > # but not both. Thunderbird separates these two by forcing server to > # accept '/' suffix in mailbox names in subscriptions list. > > > I thought this is not needed because I use mdbox. But mbox here > means also mdbox and sdbox? Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > Should I use all 3 workarounds if I don't know which clients are > in use? I don't know if delay-newmail is still relevant. From tss at iki.fi Sun Feb 12 19:03:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:03:10 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: References: Message-ID: <83BE4415-FA94-4F23-9DB4-7F59F71D245B@iki.fi> On 12.2.2012, at 18.57, Mark Sapiro wrote: >> On 12/22/2010 08:08 PM, Mark Sapiro wrote: >>> It appears that there is some version of the Dovecot LDA that is >>> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >>> non-delivery to a user who is over quota. >> >> Fixed, better late than never :) >> The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >> server sending this. > > :( It was actually a bit more complex than I thought. Sieve "reject" is supposed to send MDN (RFC 5228), it's only the out-of-quota error that was supposed to send DSN. So now Dovecot can send both: http://hg.dovecot.org/dovecot-2.1/rev/ad5298ba3229 From tss at iki.fi Sun Feb 12 19:17:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:17:30 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <4F37AD0B.9040406@mike2k.de> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> Message-ID: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f From ms at mike2k.de Sun Feb 12 20:34:31 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 19:34:31 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> Message-ID: <4F380637.30809@mike2k.de> On 12-Feb-12 18:17, Timo Sirainen wrote: > This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f Thanks Timo, it does :) -Michael From tss at iki.fi Sun Feb 12 21:11:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 21:11:44 +0200 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: On 12.2.2012, at 9.02, Thomas Leuxner wrote: > Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 From tss at iki.fi Sun Feb 12 23:01:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:01:10 +0200 Subject: [Dovecot] v2.1.rc6 released Message-ID: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota mails now include DSN report instead of MDN. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From javierdemiguel at us.es Sun Feb 12 23:38:58 2012 From: javierdemiguel at us.es (=?UTF-8?Q?Javier_Miguel_Rodr=C3=ADguez?=) Date: Sun, 12 Feb 2012 22:38:58 +0100 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Hello Reading 2.1rc6 changelog I see this: mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. Could you elaborate more on this? Has been ported to Dovecot 2.0? Regards Javier On Sun, 12 Feb 2012 23:01:10 +0200, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [1] > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [2] > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > > I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) > > Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota mails now include DSN report instead of MDN. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. > > _______________________________________________ > Dovecot-news mailing list > Dovecot-news at dovecot.org [3] > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news [4] Links: ------ [1] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [2] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [3] mailto:Dovecot-news at dovecot.org [4] http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news From tss at iki.fi Sun Feb 12 23:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:42:17 +0200 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Message-ID: <78639407-65C6-4D89-9F8C-2E27041422F2@iki.fi> On 12.2.2012, at 23.38, Javier Miguel Rodr?guez wrote: > Reading 2.1rc6 changelog I see this: > > mdbox: When saving > to alt storage, Dovecot didn't append as much > data to m.* files as it > could have. > > Could you elaborate more on this? It means that if you had e.g. mdbox_rotate_size = 2M, the files in alt storage probably were often much smaller than 2 MB. > Has been ported to Dovecot 2.0? No. You could try how well this applies to it: http://hg.dovecot.org/dovecot-2.1/rev/6881d8c59d9f From tss at iki.fi Sun Feb 12 23:57:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:57:29 +0200 Subject: [Dovecot] v2.0.18 released Message-ID: <881CA667-9F0C-4C32-9A61-87409F7DE773@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz.sig + DIGEST-MD5 authentication supports authorization id now. Patch by Yubao Liu + Added instance_name setting which is used to prefix Dovecot processes in ps output. - LDA/LMTP: Sending a large mail via submission_host or via LMTP proxy may have caused a hang. - Fixed dbox + mail_attachment_dir + zlib problems. - Login processes weren't logging all intended messages with auth_verbose=yes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - IMAP: CONTEXT search return option wasn't handled at all. - dbox: Various error handling fixes. - snarf plugin: Keep the mailbox locked during snarfing to avoid duplicates. From mstevens at imt-systems.com Mon Feb 13 00:32:36 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 23:32:36 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole Message-ID: Hi, It is probably a problem with pigeonhole 0.2.5. ... make[3]: *** [managesieve-proxy.o] Error 1 make[3]: *** Waiting for unfinished jobs.... Here is my build log: http://oss.imt-systems.com/mstevens/temp/build.log Someone an idea? Best regards, Morten From tss at iki.fi Mon Feb 13 00:43:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 00:43:26 +0200 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: Message-ID: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> On 13.2.2012, at 0.32, Morten Stevens wrote: > It is probably a problem with pigeonhole 0.2.5. Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. From interfasys at gmail.com Mon Feb 13 02:02:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Mon, 13 Feb 2012 00:02:29 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients Message-ID: <4F385315.5010600@gmail.com> Hello Pascal, I've just tested rc6 and get almost the same results. imap still crashes, but Horde still works. I forgot to mention that this is on FreeBSD 8 using gcc 4.6 and LTO. And contrary to what I said in my initial email, it doesn't seem to be ACL plugin related. imap crashes even if it's not in the plugins list. Here is the backtrace: # gdb /usr/libexec/dovecot/imap /path/to/core Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104b4bf6 in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x0000000000416b99 in cmd_list_continue.6853 () No symbol table info available. #3 0x000000000042166d in cmd_list_full () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x8080808080808080 in ?? () No symbol table info available. #6 0x0000000000000004 in ?? () No symbol table info available. #7 0x0000000010a3f1a8 in ?? () No symbol table info available. #8 0x0000000010aa4390 in ?? () No symbol table info available. #9 0x0000000010a3f174 in ?? () No symbol table info available. #10 0x00007fffffffe9a0 in ?? () No symbol table info available. #11 0x0000000010596e4f in p_strdup () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x0000000010a22200 in ?? () No symbol table info available. #13 0x0000000010aa4458 in ?? () No symbol table info available. #14 0x0000000010aa4340 in ?? () No symbol table info available. #15 0x0000000010aa4498 in ?? () No symbol table info available. #16 0x0000000110a3f040 in ?? () No symbol table info available. #17 0x0000000010a3f128 in ?? () No symbol table info available. #18 0x0000000010a3e840 in ?? () No symbol table info available. #19 0x0000000010a3f040 in ?? () No symbol table info available. #20 0x0000000010a22200 in ?? () No symbol table info available. #21 0x0000000000000001 in ?? () No symbol table info available. #22 0x0000000010a22200 in ?? () No symbol table info available. #23 0x0000000000000000 in ?? () No symbol table info available. #24 0x000000000041d320 in client_command_input.12433 () No symbol table info available. #25 0x00000000004203ca in client_handle_input () No symbol table info available. #26 0x0000000010a574e0 in ?? () No symbol table info available. #27 0x00000000105ae219 in i_stream_read () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #28 0x000000000041d13f in client_input () No symbol table info available. #29 0x00000000105ce276 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #30 0x00000000105c9ea3 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #31 0x00000000105c9f78 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #32 0x00000000105c9fb3 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #33 0x00000000004158bb in main () No symbol table info available. Regards, Olivier > On 01/18/2012 02:03 AM interfaSys s?rl wrote: > >/ Hello, > />/ > />/ I've just noticed that when Horde is connecting to Dovecot 2.1, it > />/ crashes the imap service if Dovecot is configured to use the ACL plugin. > />/ I'm not sure what's so special about the command Horde sends, but it > />/ shouldn't make Dovecot crash. Everything is fine when using Thunderbird. > />/ > />/ Here is the message in Dovecot's logs > />/ "Fatal: master: service(imap): child 89974 killed with signal 11 (core > />/ not dumped)" > />/ > />/ The message says that the core is not dumped, even though I did add > />/ drop_priv_before_exec=yes to my config file. > / > dovecot stop > ulimit -c unlimited > dovecot > > Now connect with Horde and let it crash. > > >/ I've tried connecting to the pid using gdb, but the process just hangs > />/ as soon as I'm connected. > />/ > / > continue > [wait for the crash] > bt full > detach > quit > > > Regards, > Pascal From tss at iki.fi Mon Feb 13 02:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 02:49:07 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F385315.5010600@gmail.com> References: <4F385315.5010600@gmail.com> Message-ID: On 13.2.2012, at 2.02, interfaSys s?rl wrote: > #0 0x00000000105c64e7 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104b4bf6 in mailbox_list_iter_next () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. > #2 0x0000000000416b99 in cmd_list_continue.6853 () > No symbol table info available. > #3 0x000000000042166d in cmd_list_full () This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. From bradley.giesbrecht at gmail.com Mon Feb 13 02:51:55 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Sun, 12 Feb 2012 16:51:55 -0800 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > On 13.2.2012, at 0.32, Morten Stevens wrote: > >> It is probably a problem with pigeonhole 0.2.5. > > > Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. Confirmed. Thanks. From janfrode at tanso.net Mon Feb 13 09:49:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 08:49:54 +0100 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <20120213074954.GA13267@dibs.tanso.net> On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. So only full backups, no incremental backups? Then what's the benefit over just copying the files (of a snapshot)? -jf From tlx at leuxner.net Mon Feb 13 09:57:46 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 13 Feb 2012 08:57:46 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: <20120213075746.GA28586@nihlus.leuxner.net> On Sun, Feb 12, 2012 at 09:11:44PM +0200, Timo Sirainen wrote: > Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. > > http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 Thanks. From fabian at eideo.de Mon Feb 13 10:00:25 2012 From: fabian at eideo.de (Kuran, Fabian) Date: Mon, 13 Feb 2012 08:00:25 +0000 Subject: [Dovecot] Problems sending email direct into publich folders Message-ID: <20120213080027.1CB5B214001@mail.eideo.de> Hi, >?Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent?> to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a?> sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder.? Is there a comparable way to do this with Dovecot? FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. -- Bye, Fabian Kuran From mstevens at imt-systems.com Mon Feb 13 12:11:30 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 13 Feb 2012 11:11:30 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: <493530c2afa3ced4228a0014ac495433@imt-systems.com> On 13.02.2012 01:51, Bradley Giesbrecht wrote: > On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > >> On 13.2.2012, at 0.32, Morten Stevens wrote: >> >>> It is probably a problem with pigeonhole 0.2.5. >> >> >> Yeah, unfortunately I had to break it to fix a bug.. Although I >> guess if I really wanted to I could have worked around it also. >> Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 >> fixes it. > > Confirmed. Thanks. Okay, also confirmed. Thank you. Best regards, Morten From stephan at rename-it.nl Mon Feb 13 12:17:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 11:17:23 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <493530c2afa3ced4228a0014ac495433@imt-systems.com> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> <493530c2afa3ced4228a0014ac495433@imt-systems.com> Message-ID: <4F38E333.7030705@rename-it.nl> Op 2/13/2012 11:11 AM, Morten Stevens schreef: > On 13.02.2012 01:51, Bradley Giesbrecht wrote: >> On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: >> >>> On 13.2.2012, at 0.32, Morten Stevens wrote: >>> >>>> It is probably a problem with pigeonhole 0.2.5. >>> >>> >>> Yeah, unfortunately I had to break it to fix a bug.. Although I >>> guess if I really wanted to I could have worked around it also. >>> Anyway, >>> http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes >>> it. >> >> Confirmed. Thanks. > > Okay, also confirmed. Release will follow later today. Regards, Stephan. From tss at iki.fi Mon Feb 13 12:36:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:36:47 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <20120213074954.GA13267@dibs.tanso.net> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> <20120213074954.GA13267@dibs.tanso.net> Message-ID: On 13.2.2012, at 9.49, Jan-Frode Myklebust wrote: > On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: >> >> doveadm backup -u user at domain backup: >> >> And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. > > So only full backups, no incremental backups? I know people are doing incremental backups with PostgreSQL, so it should be possible with Dovecot too. I'm not sure about the details yet tough. > Then what's the benefit over just copying the files (of a snapshot)? If SIS is enabled, it's a little difficult to restore specific user's mails from a filesystem based backup. From tss at iki.fi Mon Feb 13 12:39:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:39:03 +0200 Subject: [Dovecot] Problems sending email direct into publich folders In-Reply-To: <20120213080027.1CB5B214001@mail.eideo.de> References: <20120213080027.1CB5B214001@mail.eideo.de> Message-ID: <2F6B2448-017D-4056-9AA6-65BB03B64593@iki.fi> On 13.2.2012, at 10.00, Kuran, Fabian wrote: >> Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent > to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a > sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). > I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder. > > Is there a comparable way to do this with Dovecot? > FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. You can set: lmtp_save_to_detail_mailbox = yes And I think then you should be able to send the mail to any user who has write access to it. From tss at iki.fi Mon Feb 13 13:47:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 13:47:06 +0200 Subject: [Dovecot] Dovecot v2.2 plans Message-ID: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: - 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. - 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) - 3. priority: Things other people want to get implemented. There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. dsync-based replication ----------------------- I'll write a separate post about this later. Besides, it's coming for Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. Shared mailbox improvements --------------------------- Support for private flags for all mailbox formats: namespace { type = public prefix = Public/ mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public } - dsync needs to be able to replicate the private flags as well as shared flags. - might as well add a common way for all mailbox formats to specify which flags are shared and which aren't. $controldir/dovecot-flags would say which is the default (private or shared) and what flags/keywords are the opposite. - easy way to configure shared mailboxes to be accessed via imapc backend, which would allow easy shared mailbox accesses across servers or simply between two system users in same server. (this may be tricky to dsync.) - global ACLs read from a single file supporting wildcards, instead of multiple different files - default ACLs for each namespace/storage root (maybe implemented using the above..) Metadata / annotations ---------------------- Add support for server, mailbox and mail annotations. These need to be dsyncable, so their changes need to be stored in various .log files: 1. Per-server metadata. This is similar to subscriptions: Add changes to dovecot.mailbox.log file, with each entry name a hash of the metadata key that was changed. 2. Per-mailbox metadata. Changes to this belong inside mailbox_transaction_context, which write the changes to mailbox's dovecot.index.log files. Each log record contains a list of changed annotation keys. This gives each change a modseq, and also allows easily finding out what changes other clients have done, so if a client has done ENABLE METADATA Dovecot can easily push metadata changes to client by only reading the dovecot.index.log file. 3. Per-mail metadata. This is pretty much equivalent to per-mailbox metadata, except changes are associated to specific message UIDs. The permanent storage is in dict. The dict keys have components: - priv/ vs. shared/ for specifying private vs. shared metadata - server/ vs mailbox// vs. mail// - the metadata key name This would be a good time to improve the dict configuration to allow things like: - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a file, while the rest goes to sql) - allow sql dict to be used in more relational way, so that mail annotations could be stored with tables: mailbox (id, guid) and mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid everywhere. Things to think through: - How to handle quota? Probably needs to be different from regular mail quota. Probably some per-user "metadata quota bytes" counter/limit. - Dict lookups should be done asynchronously and prefetched as much as possible. For per-mail annotation lookups mail_alloc() needs to include a list of annotations that are wanted. Configuration ------------- Copy all mail settings to namespaces, so it'll be possible to use per-namespace mailbox settings. Especially important for imapc_* settings, but can be useful for others as well. Those settings that aren't explicitly defined in the namespace will use the global defaults. (Should doveconf -a show all of these values, or simply the explicitly set values?) Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration. There are mainly 3 config files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. The dict-sql is something I think needs a bigger redesign (mentioned above in "Metadata" section), but the sql/ldap auth configs could be merged. One way could be: sql_db sqlmails { # most settings from dovecot-sql.conf.ext, except for queries driver = mysql connect = ... } ldap_db ldapmails { # most settings from dovecot-ldap.conf.ext, except attributes/filters } passdb { driver = sql db = sqlmails sql_query = select password from users where username = '%u' } passdb { driver = ldap db = ldapmails ldap_attributes { password = %{ldap:userPassword} } ldap_filter = ... } The sql_db {} and ldap_db {} would be generic enough to be used everywhere (e.g. dict-sql), not just for passdb/userdb. Some problems: - Similar to the per-namespace mail settings, doveconf -a would output all sql_query, ldap_attributes, ldap_filter, etc. settings for all passdbs/userdbs. Perhaps a similar solution? - The database configs contain passwords, so they should be readable only by root. This makes running dovecot-lda and maybe doveadm difficult, since they fail at "permission denied" when trying to open the config. There are probably only two solutions: a) The db configs need to be !include_try'd or b) the configs can be world-readable, but only passwords are placed to only-root-readable files by using "password = backend protocol could be improved to support moving connection to another backend. Possibly using a separate control connection to avoid making the proxying less efficient in normal operation. 3. When restarting Dovecot, move all the connections to a process that keeps the connections open for a while. When Dovecot starts up, create imap processes back to the connections. This allows changing configuration for existing client connections (which sometimes may be bad! need to add checks against client-visible config conflicts), upgrading Dovecot, etc. without being visible to clients. The only problem is SSL connections: OpenSSL doens't provide a way to save/restore state, so either you need to set shutdown_clients=no (and possibly keep some imap-login processes doing SSL proxying for a long time), or SSL connections need to be killed. Of course the SSL handling could be outsourced to some other software/hardware outside Dovecot. The IMAP state saving isn't always easy. Initially it could be implemented only for the simple cases (which are a majority) and later extended to cover more. IMAP extensions --------------- - CATENATE is already implemented by Stephan - URLAUTH is also planned to be implemented, somewhat differently than in Apple's patch. The idea is to create a separate imap-urlauth service that provides extra security. - NOTIFY extension could be implemented efficiently using mailbox list indexes, which already exists in v2.1. - FILTERS extension can be easily implemented once METADATA is implemented - There are also other missing extensions, but they're probably less important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. Backups ------- Filesystem based backups have worked well enough with Dovecot in the past. But with new features like single instance storage it's becoming more difficult. There's no 100% consistent way to even get filesystem level backups with SIS enabled, because deleting both the message file and its attachment files can't be done atomically (although usually this isn't a real problem). Restoring SIS mails is more difficult though, first you need to restore the dbox mail files and then you need to figure out what attachment files from SIS need to be restored, and finally you'll need to do doveadm import to put them into their final destination. I don't have much experience with backup software, but other people in my company do. The initial idea is to implement a Dovecot backup agent to one (commercial) backup software, which allows doing online backups and restoring mails one user/mailbox/mail at a time. I don't know the details yet how exactly this is going to be implemented, but the basic plan is probably to implement a "backup" mail storage backend, which is a PostgreSQL pg_dump-like flat file containing mails from all mailboxes. doveadm backup/import can then export/import this format via stdout/stdin. Incremental backups could possibly be done by giving a timestamp of previous backup run (I'm not sure about this yet). Once I've managed to implement the first fully functional backup agent, it should become clearer how to implement it to other backup solutions. Random things ------------- - dovecot.index.cache file writing is too complex, should be simplified - Enable auth_debug[_passwords]=yes on-the-fly for some specific users/IPs via doveadm - Optimize virtual mailboxes using mailbox list indexes. It wouldn't anymore need to keep all the backend mailboxes' index files open. - Would be nice to go forward with supporting key-value databases as mail storage backends. From janfrode at tanso.net Mon Feb 13 15:54:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 14:54:50 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend Message-ID: <20120213135450.GA31004@dibs.tanso.net> We've been collecting some stats to see what kind of benefits UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is running in High-performance mode http://wiki2.dovecot.org/LoginProcess with authentication caching http://wiki2.dovecot.org/Authentication/Caching During the weekend two servers (webmail3 and webmail4) has been running with local imapproxy and two servers without (webmail1 and webmail2). Each server has served about 1 million http requests, over 3 days. server avg. response time # requests ---------------------------------------------------- webmail1.example.net 0.370411 1092386 webmail2.example.net 0.374227 1045141 webmail3.example.net 0.378097 1043919 imapproxy webmail4.example.net 0.378593 1028653 imapproxy ONLY requests that took more than 5 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 26.048 1125 webmail2.example.net 26.2997 1080 webmail3.example.net 28.5596 808 imapproxy webmail4.example.net 27.1004 964 imapproxy ONLY requests that took more than 10 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 49.1407 516 webmail2.example.net 53.0139 459 webmail3.example.net 59.7906 333 imapproxy webmail4.example.net 58.167 384 imapproxy The responstimes are not very fast, but they do seem to support the claim that an imapproxy isn't needed for dovecot. -jf From tss at iki.fi Mon Feb 13 16:14:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:14:22 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On 13.2.2012, at 15.54, Jan-Frode Myklebust wrote: > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Other things that would be interesting to try out (both from latency and disk IO usage point of view): - maildir_very_dirty_syncs - mail_prefetch_count (Linux+maildir only, v2.1+) From busseniu at in.tum.de Mon Feb 13 16:16:29 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 15:16:29 +0100 Subject: [Dovecot] dsync, hard-links and refcounts Message-ID: <4F391B3D.8020208@in.tum.de> Hi, when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. - Is there some way to make dsync notice the hard links? (I used dovecot 2.0.17) - Alternatively, is there some tool to deduplicate the mdbox after it has been converted from maildir? - NB: I am not talking about single-instance storage for attachments. Thank you very much, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Feb 13 16:40:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:40:05 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F391B3D.8020208@in.tum.de> References: <4F391B3D.8020208@in.tum.de> Message-ID: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. > > My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. > - Is there some way to make dsync notice the hard links? (I used > dovecot 2.0.17) It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. From forumer at smartmobili.com Mon Feb 13 17:35:57 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Mon, 13 Feb 2012 16:35:57 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: Hi, Is there any plan to port dovecot to windows ? From robert at schetterer.org Mon Feb 13 17:49:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:49:14 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F3930FA.2080802@schetterer.org> Am 13.02.2012 16:35, schrieb forumer at smartmobili.com: > Hi, > > Is there any plan to port dovecot to windows ? > cant wait Timos answer *g -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Feb 13 17:51:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:51:26 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F39317E.5090300@schetterer.org> Am 13.02.2012 12:47, schrieb Timo Sirainen: > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration might a question of taste, but i never liked the splitted up config style, so i like this idea -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Feb 13 17:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 17:57:03 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <4F39317E.5090300@schetterer.org> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <4F39317E.5090300@schetterer.org> Message-ID: On 13.2.2012, at 17.51, Robert Schetterer wrote: > Am 13.02.2012 12:47, schrieb Timo Sirainen: >> Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration > > might a question of taste, but i never liked the splitted up config > style, so i like this idea Note that I said *.conf.ext, not *.conf.. From tss at iki.fi Mon Feb 13 18:11:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:11:33 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On 13.2.2012, at 17.35, forumer at smartmobili.com wrote: > Is there any plan to port dovecot to windows ? It probably works via Cygwin (although I think Cygwin has to do some kind of POSIX filesystem emulation, which slows things down and might cause some trouble if server crashes). Actually we have discussed this a little within my company.. I'm personally not interested in spending much time on it, but that's why we're hiring more coders so I won't have to do everything. :) If there is enough commercial interest, we might build something better than using Cygwin. Anyway, one thing is certain: No Dovecot for Windows questions in this mailing list. Perhaps I'll create another dovecot-windows@ mailing list. Perhaps I'll even give it a different name (dovedows? wincot? glassdove?) From joshua at hybrid.pl Mon Feb 13 18:26:38 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 17:26:38 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 Message-ID: Hi, Finally I decided to give a try to dovecot 2.0, after spending few years with 1.2. Unfortunately, it doesn't go smooth - finally I managed to cope with LMTP transport and exim, but failed to run managesieve. I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. Here is my doveconf -n. Any ideas? # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.39.4-vs2.3.1-pre9.2-nfs41-nfsd x86_64 auth_debug = yes auth_debug_passwords = yes debug_log_path = /var/log/dovecot/debug.log dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_uid = 79 info_log_path = /var/log/dovecot/info.log last_valid_uid = 79 log_path = /var/log/dovecot/main.log mail_debug = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = /var/mail/sieve/%d/%u.sieve sieve_dir = /var/mail/sieve/%d/%u sieve_global_path = /var/mail/sieve/%d/global.sieve } postmaster_address = postmaster at hybrid.pl service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 } inet_listener imaps { address = 0.0.0.0 port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { address = 127.0.0.1 port = 110 } inet_listener pop3s { address = 0.0.0.0 port = 995 ssl = yes } } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " sieve quota" } protocol lda { mail_plugins = " sieve quota" } Best regards, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Mon Feb 13 18:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:42:21 +0200 Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: References: Message-ID: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> On 13.2.2012, at 18.26, Jacek Osiecki wrote: > I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. > > Here is my doveconf -n. Any ideas? > > # 2.0.16: /etc/dovecot/dovecot.conf Try adding: protocols = imap pop3 sieve From joshua at hybrid.pl Mon Feb 13 19:11:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 18:11:11 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> References: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> Message-ID: On Mon, 13 Feb 2012, Timo Sirainen wrote: > On 13.2.2012, at 18.26, Jacek Osiecki wrote: >> I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. >> Here is my doveconf -n. Any ideas? >> >> # 2.0.16: /etc/dovecot/dovecot.conf > Try adding: > protocols = imap pop3 sieve Great, it helped :) Thanks! Now time to see what surprises will show up while migrating there people from dovecot 1.0-stable :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From alexwbaule at gmail.com Mon Feb 13 19:28:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Mon, 13 Feb 2012 15:28:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Hi Again Timo ! I Trying to port quota plugin notify_sync to my plugin, but i found some issues. I have the hooks already, to replace the mailbx_allocated, like quota plugin does. static struct mail_storage_hooks emexis_mail_storage_hooks = { .mail_user_created = Mplugin_mail_user_created, .mailbox_allocated = Mplugin_mailbox_allocated, .mail_allocated = Mplugin_mail_allocated, .mailbox_list_created = Mplugin_mailbox_list_created, .mail_namespaces_created = Mplugin_antispam_mail_namespaces_created }; inside the Mplugin_mailbox_allocated, i have: static void emexis_mailbox_allocated(struct mailbox *box) { union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = box->v; box->v.open = Mplugin_mailbox_open; box->v.sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); } The v.open i rewrite because my plugin needs. I try to call the sync_notify like this: box->v.sync_notify = Mplugin_mailbox_sync_notify; and follow quota plugin, i try to to this too: struct mailbox_vfuncs *v = box->vlast; v->sync_notify = Mplugin_mailbox_sync_notify; But in this two cases, the Mplugin_mailbox_sync_notify is never called... i missing something ?? Tks ! Em 12 de fevereiro de 2012 14:48, Alex Baule escreveu: > Tks timo ! > > I will see the plugin to do like it! > > Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > >> Hi, >> >> Yeah, you shouldn't do the erasing directly in expunge(), because it may >> still be aborted. Do it in sync_notify() like quota plugin does. >> >> On 12.2.2012, at 16.52, Alex Baule wrote: >> >> > I want to know because i have header and body splited. To erase the >> > body, i must have shure, that header was expunged. Some tests that i >> > made, the client call expunge, but i don't ?know why (there is notting >> > in mail log) the header was not expunged, nut my body was, because i >> > call it before expunge the header. something like this. >> > >> > if (found_body){ >> > ? ?erase_body(); >> > ? ?super.expunge(_mail); >> > } >> > >> > Because that expunge that no happen, (the client call, but the email >> > was not expunged) i think to do like this way: >> > >> > if (found_body){ >> > ? super.expunge(_mail); >> > ? if(_mail was expunged){ >> > ? ? ?erase_body(); >> > ?} >> > } >> > >> > TKs Timo ! >> > >> > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >> >> >>> Hy Everyone...and Timo ! >> >>> >> >>> There is a way to know if a email was expunged (deleted from hard >> >>> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> >>> expunge function is void, i can't know if was really expunged. >> >>> >> >>> There is a way to know this ? >> >> >> >> It's not known until transaction_commit() what messages are expunged. >> >> What do you need to know this for? There are a few different ways. >> >> >> >> If you simply need one of the sessions to definitely know that it >> >> expunged a message, you can do it like quota plugin does. (So even if there >> >> are multiple clients doing EXPUNGE at the same time, quota is never >> >> decreased more than once per mail.) See quota-storage.c and most importantly >> >> quota_mailbox_sync_notify(). >> >> >> >> Quota code also shows the other method of doing it: keep track of what >> >> mail_expunge()s have been called, and then in transaction commit check if it >> >> succeeds and if it does do what you want to do. >> >> >> > >> > From busseniu at in.tum.de Mon Feb 13 20:31:51 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 19:31:51 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F395717.5010504@in.tum.de> On 02/13/2012 03:40 PM, Timo Sirainen wrote: > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. I?d very much appreciate that. We will have to migrate many large mailboxes, and it would be a pity to needlessly waste space. I got the same result with some different configurations, using "mirror", "backup", or "-R backup" (though I have not tried 2.1 yet); so it should be easy to reproduce. However let me know if you need my configuration or anything. Cheers, Christoph From moseleymark at gmail.com Mon Feb 13 21:08:48 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 13 Feb 2012 11:08:48 -0800 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On Mon, Feb 13, 2012 at 5:54 AM, Jan-Frode Myklebust wrote: > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 0.370411 ? ? ? ?1092386 > webmail2.example.net ? 0.374227 ? ? ? ?1045141 > webmail3.example.net ? 0.378097 ? ? ? ?1043919 ?imapproxy > webmail4.example.net ? 0.378593 ? ? ? ?1028653 ?imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 26.048 ? ? ? ? ?1125 > webmail2.example.net ? 26.2997 ? ? ? ? 1080 > webmail3.example.net ? 28.5596 ? ? ? ? 808 ? ? ?imapproxy > webmail4.example.net ? 27.1004 ? ? ? ? 964 ? ? ?imapproxy > > ONLY requests that took more than 10 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 49.1407 ? ? ? ? 516 > webmail2.example.net ? 53.0139 ? ? ? ? 459 > webmail3.example.net ? 59.7906 ? ? ? ? 333 ? ? ?imapproxy > webmail4.example.net ? 58.167 ? ? ? ? ?384 ? ? ?imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Out of curiosity, are you running dovecot locally on those webmail servers as well, or is it talking to remote dovecot servers? I ask because I'm looking at moving our webmail from an on-box setup to a remote pool to support director and was going to look into whether running imapproxyd would help there. We don't bother with it in the local setup, since dovecot is so fast, but remote (but still on a LAN) might be different. Though imapproxyd seems to make (wait for it...) squirrelmail unhappy (complains about IMAP errors, when sniffing shows none), though I've not bothered to debug it yet. From janfrode at tanso.net Mon Feb 13 21:36:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:36:30 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213193630.GA3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 04:14:22PM +0200, Timo Sirainen wrote: > > The responstimes are not very fast, but they do seem to support > > the claim that an imapproxy isn't needed for dovecot. > > That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Yes, this is maildirs (on GPFS). > > Other things that would be interesting to try out (both from latency and disk IO usage point of view): > > - maildir_very_dirty_syncs We already have $ doveconf maildir_very_dirty_syncs maildir_very_dirty_syncs = yes but I don't think this gave the advantage I was expecting.. Was expecting this to move most iops to the index-luns, but the maildir luns seems just as busy. > - mail_prefetch_count (Linux+maildir only, v2.1+) Will look into if this works with GPFS when we upgrade to v2.1. It has it's own page cache, so I have no idea if it will respect POSIX_FADV_WILLNEED or if one will need to use it's own API's for hinting: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html -jf From janfrode at tanso.net Mon Feb 13 21:43:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:43:57 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213194356.GB3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: > > Out of curiosity, are you running dovecot locally on those webmail > servers as well, or is it talking to remote dovecot servers? The webmail servers are talking with dovecot director servers which in turn are talking with the backend dovecot servers. Each service running on different servers. Webmail-servers -> director-servers -> backend-servers > I ask because I'm looking at moving our webmail from an on-box setup to a > remote pool to support director and was going to look into whether > running imapproxyd would help there. We don't bother with it in the > local setup, since dovecot is so fast, but remote (but still on a LAN) > might be different. Doesn't seem so to us... > Though imapproxyd seems to make (wait for it...) > squirrelmail unhappy (complains about IMAP errors, when sniffing shows > none), though I've not bothered to debug it yet. :-) -jf From tss at iki.fi Mon Feb 13 21:57:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 21:57:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213193630.GA3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: On 13.2.2012, at 21.36, Jan-Frode Myklebust wrote: >> Other things that would be interesting to try out (both from latency and disk IO usage point of view): >> >> - maildir_very_dirty_syncs > > We already have > > $ doveconf maildir_very_dirty_syncs > maildir_very_dirty_syncs = yes > > but I don't think this gave the advantage I was expecting.. Was > expecting this to move most iops to the index-luns, but the maildir > luns seems just as busy. This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. >> - mail_prefetch_count (Linux+maildir only, v2.1+) > > Will look into if this works with GPFS when we upgrade to v2.1. It has > it's own page cache, so I have no idea if it will respect > POSIX_FADV_WILLNEED or if one will need to use it's own API's for > hinting: > > http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html I guess if there's an easy way to lookup filename or fd -> blockNumber that wouldn't be difficult to implement with a plugin. From chayes at afo.net Mon Feb 13 22:03:02 2012 From: chayes at afo.net (Cliff Hayes) Date: Mon, 13 Feb 2012 14:03:02 -0600 Subject: [Dovecot] Logging to messages instead of secure Message-ID: <5CDFC2DC614E436F89C10AF7F19F7DAB@ciosys> I need to log to /var/log/messages instead of /var/log/secure on a fedora system. Is this the correct entry for dovecot.conf? syslog_facility = info From tss at iki.fi Mon Feb 13 22:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 22:18:42 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> On 13.2.2012, at 19.28, Alex Baule wrote: > static void emexis_mailbox_allocated(struct mailbox *box) > { > union mailbox_module_context *zbox; > > zbox = p_new(box->pool, union mailbox_module_context, 1); > zbox->super = box->v; > box->v.open = Mplugin_mailbox_open; > box->v.sync_notify = Mplugin_mailbox_sync_notify; > > MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); > } The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > struct mailbox_vfuncs *v = box->vlast; > v->sync_notify = Mplugin_mailbox_sync_notify; > > But in this two cases, the Mplugin_mailbox_sync_notify is never > called... i missing something ?? Never called at all? What storage backend are you using as the base? Maildir? Anyway, the way you should be calling it is the exact same way quota plugin does: void Mplugin_mailbox_allocated(struct mailbox *box) { struct mailbox_vfuncs *v = box->vlast; union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = *v; box->vlast = &zbox->super; v->open = Mplugin_mailbox_open; v->sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); } From slusarz at curecanti.org Mon Feb 13 22:24:25 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 13:24:25 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 0.370411 1092386 > webmail2.example.net 0.374227 1045141 > webmail3.example.net 0.378097 1043919 imapproxy > webmail4.example.net 0.378593 1028653 imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 26.048 1125 > webmail2.example.net 26.2997 1080 > webmail3.example.net 28.5596 808 imapproxy > webmail4.example.net 27.1004 964 imapproxy > > ONLY requests that took more than 10 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 49.1407 516 > webmail2.example.net 53.0139 459 > webmail3.example.net 59.7906 333 imapproxy > webmail4.example.net 58.167 384 imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Except you are most likely NOT leveraging the truly interesting part of imapproxy - the ability to restore the IMAP connection state via the XPROXYREUSE status response. This is a significant performance improvement since it also reduces processing load on the client side (everything before/including authentication needs to be done whether using imapproxy or not, so there is no client-side savings for these commands). For further information, see, e.g.: http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. michael From janfrode at tanso.net Mon Feb 13 22:26:23 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:26:23 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: <20120213202623.GA5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 09:57:31PM +0200, Timo Sirainen wrote: > > > > $ doveconf maildir_very_dirty_syncs > > maildir_very_dirty_syncs = yes > > > > but I don't think this gave the advantage I was expecting.. Was > > expecting this to move most iops to the index-luns, but the maildir > > luns seems just as busy. > > This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. With maildir_very_dirty_syncs = yes: ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 1.5 0.0 96 92 514 73 9 7 1.2 0.0 59 43 367 18 4 76 1.7 0.0 66 61 477 67 2 6 1.2 0.0 54 50 348 31 1 145 3.0 0.0 113 90 860 59 7 8 2.9 0.0 107 99 840 58 5 11 4.0 0.0 131 101 1117 77 2 65 With maildir_very_dirty_syncs = no (same node, a few seconds later): ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 4.6 0.9 125 91 1161 1096 41 6 2.3 0.7 200 170 697 127 86 16 1.1 0.6 124 99 406 61 48 109 2.7 0.1 212 144 755 114 74 15 2.7 0.0 159 133 818 70 78 194 0.8 1.2 86 73 225 60 16 9 1.9 0.0 124 116 573 53 30 6 So it seems to be working, good :-) -jf From janfrode at tanso.net Mon Feb 13 22:35:11 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:35:11 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <20120213203511.GB5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 01:24:25PM -0700, Michael M Slusarz wrote: > > Except you are most likely NOT leveraging the truly interesting part > of imapproxy - the ability to restore the IMAP connection state via > the XPROXYREUSE status response. This is a significant performance > improvement since it also reduces processing load on the client side > (everything before/including authentication needs to be done whether > using imapproxy or not, so there is no client-side savings for these > commands). Thanks for this info, good to know. I'll check with inverse/sogo if this is something they use/intend to use.. > > additional advantage. Note that the XPROXYREUSE-enabled MUA must be > the exclusive user of the imapproxy instance for this feature to > work correctly. Not a problem. Assuming it doesn't also need to be the only imap user of the account/folder. BTW: do you also have information on the state of select caching in the up-imapproxy? I got some very negative comments when googling it, and the changelog didn't suggest there had been any improvements since.. -jf From stephan at rename-it.nl Mon Feb 13 23:01:10 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 22:01:10 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.6 for Dovecot v2.0.18 Message-ID: <4F397A16.1010107@rename-it.nl> Hello Dovecot users, Pigeonhole v0.2.5 does not compile against the new Dovecot v2.0.18 release. That is why a release a new version now. Other changes are included as well, of which the one regarding the include extension requires special attention; Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. * Matched changes regarding auth_verbose setting in Dovecot. This means that this release will only compile against Dovecot v2.0.18. - Fixed problem in ManageSieve that caused it to omit a WARNINGS response code when the uploaded script compiled with warnings. - Made sure that locations of Sieve error never report `line 0'. - Fixed potential segfault occurring when interpreter initialization fails. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Mon Feb 13 23:32:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:32:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> On 13.2.2012, at 22.24, Michael M Slusarz wrote: > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html > > These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. > > Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. Perhaps a way for (trusted) clients to be able to do this? :) a logout save * OK [SAVEDSTATE 1234567890] * BYE logged out a OK ... b login (SAVEDSTATE 1234567890) user pass * OK [RESTOREDSTATE 1234567890] b ok vs. b login (SAVEDSTATE 1234567890) user pass * NO [UNKNOWNSTATE 1234567890] b ok From tss at iki.fi Mon Feb 13 23:44:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:44:47 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: On 13.2.2012, at 23.32, Timo Sirainen wrote: > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK Actually, this could probably be safely implemented by sending all of the state to the client as a string: * OK [SAVEDSTATE base64-encoded-state] There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). From ybhu at hk1.ibm.com Tue Feb 14 04:39:01 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 10:39:01 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: Another question is about the director failover. In http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html you said: "The main complexity here comes from how to handle proxy server failures in different situations. Those are less interesting to describe and I haven't yet implemented all of it, so let's just assume that in future it all works perfectly." So currently there is no good way to handle the director failover, or any 3rd party solution like poolmon by Brad Davidson for health monitoring of backend servers? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/12/2012 12:06 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09320.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Tue Feb 14 04:46:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 04:46:21 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: On 14.2.2012, at 4.39, Andy YB Hu wrote: > Another question is about the director failover. In > http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html > you said: "The main complexity here comes from how to handle proxy server > failures in different situations. Those are less interesting to describe > and I haven't yet implemented all of it, so let's just assume that in > future it all works perfectly." So currently there is no good way to > handle the director failover, or any 3rd party solution like poolmon by > Brad Davidson for health monitoring of backend servers? Your load balancer handles director downtimes so it connects only to directors that are up. Directors themselves figure out when one of them is down (by either explicit disconnection or timeout). That's the theory, and so far no one's told me it doesn't work that way. For the backend servers there's still no automation though. You'll need to explicitly tell director to stop trying to connect some specific backend. The poolmon is intended for that. It would be possible to implement this directly in the director itself, but so far it hasn't really been a priority since the companies who have paid for it have wanted to implement it internally themselves.. From dmiller at amfes.com Tue Feb 14 04:54:20 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 18:54:20 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 2/12/2012 1:01 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > With this configure line: ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene I get this error on make: make[4]: Entering directory `/opt/dovecot-2.1.rc6/src/plugins/fts-lucene' /bin/bash ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arh -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -o lib20_doveadm_s_lucene_plugin.la -rpath /usr/local/lib/dovecot/doveadm doveadm-fts-lucene.lo -lrt /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I..//../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF eps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/l-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wraer.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory compilation terminated. make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... -- Daniel From tss at iki.fi Tue Feb 14 05:00:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:00:46 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 14.2.2012, at 4.54, Daniel L. Miller wrote: > With this configure line: > ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene > > lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ From dmiller at amfes.com Tue Feb 14 05:01:59 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:01:59 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 2/13/2012 7:00 PM, Timo Sirainen wrote: > On 14.2.2012, at 4.54, Daniel L. Miller wrote: > >> With this configure line: >> ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene >> >> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory > Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ > Ah! I would've thought configure would catch that though - before make did. Ok. -- Daniel From dmiller at amfes.com Tue Feb 14 05:04:26 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:04:26 -0800 Subject: [Dovecot] Boehm Message-ID: As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? -- Daniel From tss at iki.fi Tue Feb 14 05:05:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:05:33 +0200 Subject: [Dovecot] Boehm In-Reply-To: References: Message-ID: <4E1CD0CF-7679-4D4D-9DF1-E5B101ECE797@iki.fi> On 14.2.2012, at 5.04, Daniel L. Miller wrote: > As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? Eh. Probably not. From slusarz at curecanti.org Tue Feb 14 05:12:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:12:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203511.GB5085@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <20120213203511.GB5085@dibs.tanso.net> Message-ID: <20120213201256.Horde.k3ITIIF5lbhPOdE4A_7B9KA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > BTW: do you also have information on the state of select caching in the > up-imapproxy? I got some very negative comments when googling it, and the > changelog didn't suggest there had been any improvements since.. I wouldn't trust it. IIRC, it was added years ago and given the syntax changes to the SELECT/EXAMINE call since then (e.g. QRESYNC extentsion), it is doubtful the code has been updated to handle these situations. And in a server-caching IMAP server like dovecot, all that SELECT information is not expensive anyway, so any gains would be miniscule. michael From slusarz at curecanti.org Tue Feb 14 05:19:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:19:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 22.24, Michael M Slusarz wrote: > >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html >> >> These posts neglect the fact that you don't need to issue a >> CAPABILITY command if the connection is reused either, so that's an >> additional advantage. Note that the XPROXYREUSE-enabled MUA must be >> the exclusive user of the imapproxy instance for this feature to >> work correctly. >> >> Somewhat topical, since Timo was just mentioning support for some >> sort of IMAP state save/restore feature possibly making it into 2.2. > > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK > > ... > > b login (SAVEDSTATE 1234567890) user pass > * OK [RESTOREDSTATE 1234567890] > b ok > > vs. > > b login (SAVEDSTATE 1234567890) user pass > * NO [UNKNOWNSTATE 1234567890] > b ok I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. And you don't allow reviving the state if using the AUTHENTICATE command. The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. michael From tss at iki.fi Tue Feb 14 05:23:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:23:16 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Message-ID: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> On 14.2.2012, at 5.19, Michael M Slusarz wrote: >> b login (SAVEDSTATE 1234567890) user pass > > I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. No breaking, extending :) > And you don't allow reviving the state if using the AUTHENTICATE command. Just as easy: b authenticate (savedstate ..) plain .. > The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. Yeah. Although not bad either, since you can pipeline that command + login/authenticate. From slusarz at curecanti.org Tue Feb 14 05:30:09 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:30:09 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 23.32, Timo Sirainen wrote: > >> Perhaps a way for (trusted) clients to be able to do this? :) >> >> a logout save >> * OK [SAVEDSTATE 1234567890] >> * BYE logged out >> a OK > > Actually, this could probably be safely implemented by sending all > of the state to the client as a string: > > * OK [SAVEDSTATE base64-encoded-state] > > There isn't a whole lot of state to be saved really. Mailbox GUID, > UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have > the language/etc. states. Clients could restore their earlier state > from days ago, as long as Dovecot still has the necessary .log > records available (similar to how QRESYNC works). Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. For a disconnected client (e.g. webmail), there's going to necessarily need to be startup costs of initializing the session anyway so re-creating the state can be buried in this other work. Re-creating becomes more important when users are doing actions they expect immediate (or at least very fast) responses for. Things like listing messages in a mailbox, viewing a message, or polling mailboxes. Saving 50ms per request becomes important in these situations since re-creating state now takes 10% of the total request time. So I don't think states don't need a terribly long lifetime. I almost think of something like an internal Dovecot IDLE queue - after a logout is received, state is stored for 30 minutes after which it is discarded. (Although not knowing anything about internal Dovecot state, don't know if this is overly resource intensive.) As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if you come up with a better way to represent state. michael From dmiller at amfes.com Tue Feb 14 05:34:44 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:34:44 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: On 2/12/2012 9:00 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.27, Mike Constabel wrote: > >> Another Question to this Option: >> >> # tb-extra-mailbox-sep: >> # With mbox storage a mailbox can contain either mails or submailboxes, >> # but not both. Thunderbird separates these two by forcing server to >> # accept '/' suffix in mailbox names in subscriptions list. >> >> >> I thought this is not needed because I use mdbox. But mbox here >> means also mdbox and sdbox? > Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > >> Should I use all 3 workarounds if I don't know which clients are >> in use? > I don't know if delay-newmail is still relevant. > I'm fighting the same issue. With the current release (2.1rc6) no errors are logged - but I'm having problems deleted folders from trash. I've enabled tb-extra-mailbox-sep. Two folders that were previously in Trash were successfully removed. I deleted four more - after several Thunderbird restarts, compacts, refreshes, I have killed three of them - there is still one folder that won't "die". Thunderbird reports the mailbox doesn't exist, yet it continues to appear. -- Daniel From slusarz at curecanti.org Tue Feb 14 05:35:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:35:46 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> Message-ID: <20120213203546.Horde.TaVqCoF5lbhPOdaS8PLB9SA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 14.2.2012, at 5.19, Michael M Slusarz wrote: > >>> b login (SAVEDSTATE 1234567890) user pass >> >> I guess the drawback for this approach is that you are explicitly >> breaking the LOGIN definition. > > No breaking, extending :) Heh. I should know better after reading about 1000 RFCs last year... Anyway, to play devil's advocate, another drawback of this approach is that in the (fairly remote) chance that the LOGIN command is extended in the future by an RFC, there is no guarantee that this syntax would be used. michael From ybhu at hk1.ibm.com Tue Feb 14 05:31:54 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 11:31:54 +0800 Subject: [Dovecot] index synchronization during local delivery Message-ID: Hi there, Now our system will take dovecot as the whole imap back end, and keep the LDA of ours, But our system has our own UID generation mechanism (in dovecot it's incremental from 1), which will be done while local delivery, So when local deliveries, it will first generate the UID, move the mail into maildir, then synchronize the mailbox index, which is part of dovecot LDA does. The question is what API or something else about index synchronization I can integrate into our own LDA program easily? Seems mailbox_copy() would be the possibility? Thanks. From busseniu at in.tum.de Tue Feb 14 17:04:25 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 14 Feb 2012 16:04:25 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit Message-ID: <4F3A77F9.8000002@in.tum.de> Hi, our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. The "doveadm proxy list" would fail with an error message: $ sudo doveadm proxy list doveadm(root): Error: LIST failed: Disconnected Segmentation fault The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. Anyway, I suggest changing the log message if the failure is not actually due to the client_limit. # 2.0.18: /etc/ports/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.2 LTS auth_username_format = %n default_client_limit = 60000 default_process_limit = 10000 disable_plaintext_auth = no mail_location = maildir:/nonexistent managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/ports/dovecot/passwd driver = passwd-file } protocols = imap pop3 lmtp sieve service anvil { client_limit = 100000 } service auth { client_limit = 60000 } service ipc { client_limit = 100000 } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } shutdown_clients = no ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From cinzia.delcoiro at libero.it Tue Feb 14 17:02:07 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:02:07 +0100 (CET) Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <25869845.713261329231727832.JavaMail.root@wmail51> Hi Timo, I'm writing to alert you that even in version 2.1. we have the error: "Can't handle mixed proxy/non-proxy destinations" when we write to two users: one existent and one not. The error occurs randomly. I configure LDAP query in the file : dovecot-ldap.conf in this way: pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, =proxy_timeout=600 specifying "=proxy=y" in order to proxy every mail. As workaround, can I configure a static "host" for non-existing users? This is my dovecot -n # 2.1.rc5: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_mechanisms = plain login base_dir = /usr/local/var/run/dovecot/ disable_plaintext_auth = no listen = 127.0.0.1 lmtp_proxy = yes mail_gid = mailusers mail_location = maildir:/mnt/mbox1/mailusers/%n/Maildir mail_uid = mailusers passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User Quota } protocols = lmtp service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no syslog_facility = local5 userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { mail_plugins = quota imap_quota } protocol lmtp { postmaster_address = info at amm2.miur.it } Regards Cinzia >----Messaggio originale---- >Da: tss at iki.fi >Data: 28-gen-2012 18.32 >A: "Alexis Lelion" >Cc: "Dovecot Mailing List" >Ogg: Re: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations > >On 27.1.2012, at 12.59, Alexis Lelion wrote: > >> Jan 25 09:05:12 mail01 postfix/lmtp[23934]: A92709300DB: to=< >> user_on_mail02 at domain.com>, relay=mail01.domain.com[private/dovecot-lmtp], >> delay=0.07, delays=0.01/0/0/0.06, dsn=4.3.0, status=deferred (host >> mail01.domain.com[private/dovecot-lmtp] said: 451 4.3.0 < >> user_on_mail02 at domain.com> Can't handle mixed proxy/non-proxy destinations >> (in reply to RCPT TO command)) >> >> I was wondering if there was another way of handling this, for example >> by triggering an immediate queue lookup from postfix or forwarding a >> copy of the mail to the other server. Note that the postfix >> "queue_run_delay" was increased to 15min on purpose, so I cannot change >> that. > >It would be possible to change the code to support mixed destinations, but it's probably not a simple change and I have other things to do.. > >Maybe you could work around it so that LMTP always proxies the mails, to localhost as well, but to a different port which doesn't do proxying at all. > > From CMarcus at Media-Brokers.com Tue Feb 14 17:42:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 14 Feb 2012 10:42:12 -0500 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <4F3A80D4.4000007@Media-Brokers.com> On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it wrote: > I'm writing to alert you that even in version 2.1. we have the error: "Can't > handle mixed proxy/non-proxy destinations" when we write to two users: one > existent and one not. why are you accepting mail for non existing users? Catch-alls are a very very bad idea in 99.999% of cases, and usually when someone thinks they need a catchall, they are mistaken. -- Best regards, Charles From cinzia.delcoiro at libero.it Tue Feb 14 17:54:38 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:54:38 +0100 (CET) Subject: [Dovecot] R: Re: R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <23144211.746481329234878394.JavaMail.root@wmail51> Charles I need to accept mail also for nonexistent users because I'm delivering a particular service (Certified Mail). Regards Cinzia >----Messaggio originale---- >Da: CMarcus at Media-Brokers.com >Data: 14-feb-2012 16.42 >A: "Dovecot Mailing List" >Ogg: Re: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations > >On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it > wrote: >> I'm writing to alert you that even in version 2.1. we have the error: "Can't >> handle mixed proxy/non-proxy destinations" when we write to two users: one >> existent and one not. > >why are you accepting mail for non existing users? > >Catch-alls are a very very bad idea in 99.999% of cases, and usually >when someone thinks they need a catchall, they are mistaken. > >-- > >Best regards, > >Charles > From janfrode at tanso.net Tue Feb 14 21:34:42 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 20:34:42 +0100 Subject: [Dovecot] pop3 not autocreating directory structure Message-ID: <20120214193442.GA5156@dibs.tanso.net> We use: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail: mdbox:~/mdbox and I just noticed one of our newly provisioned users initially failed to pop her mails. I saw several of these: dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX dovecot:: pop3(new.user at example.net): Couldn't open INBOX top=0/0, retr=0/0, del=0/0, size=0 before she switched to imap and then everything looked fine: dovecot:: imap(new.user at example.net): Disconnected: Logged out bytes=11/338 Looking at the timestamps in the filesystem I see that the users home directory wasn't created before switcing to imap. Is this a know problem? -jf From wgillespie+dovecot at es2eng.com Tue Feb 14 22:06:56 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Tue, 14 Feb 2012 13:06:56 -0700 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: <4F3ABEE0.6010702@es2eng.com> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: > I'm fighting the same issue. With the current release (2.1rc6) no errors > are logged - but I'm having problems deleted folders from trash. I've > enabled tb-extra-mailbox-sep. Two folders that were previously in Trash > were successfully removed. I deleted four more - after several > Thunderbird restarts, compacts, refreshes, I have killed three of them - > there is still one folder that won't "die". Thunderbird reports the > mailbox doesn't exist, yet it continues to appear. Out of curiosity -- does the mailbox actually exist if you look on the Dovecot server? Secondly, is it listed in your subscription file? From dmiller at amfes.com Tue Feb 14 22:40:02 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 14 Feb 2012 12:40:02 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3ABEE0.6010702@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> Message-ID: On 2/14/2012 12:06 PM, Willie Gillespie wrote: > On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >> I'm fighting the same issue. With the current release (2.1rc6) no errors >> are logged - but I'm having problems deleted folders from trash. I've >> enabled tb-extra-mailbox-sep. Two folders that were previously in Trash >> were successfully removed. I deleted four more - after several >> Thunderbird restarts, compacts, refreshes, I have killed three of them - >> there is still one folder that won't "die". Thunderbird reports the >> mailbox doesn't exist, yet it continues to appear. > > Out of curiosity -- does the mailbox actually exist if you look on the > Dovecot server? > Secondly, is it listed in your subscription file? At one time - yes they were. After further playing, they do not exist in the on-disk folder structure, nor in the subscription file. But still show in the client - with errors. -- Daniel From janfrode at tanso.net Tue Feb 14 22:47:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 21:47:51 +0100 Subject: [Dovecot] doveadm director proxy Message-ID: <20120214204751.GA6279@dibs.tanso.net> I'm trying to configure a doveadm service that will proxy trough our directors, following the recipie at: http://wiki2.dovecot.org/Director#Doveadm_server So on the backends I have: service doveadm { inet_listener { port = 24245 address = * } } doveadm_proxy_port = 24245 local 192.168.42.0/24 { doveadm_password = suPerSeecret } I assume the "local" line is supposed to point at my local network..? On the directors I have the same, plus: protocol doveadm { auth_socket_path = director-userdb } When testing "doveadm quota" on the directors, it complained quota plugin not being loaded, so I added: mail_plugins=quota Then it complained about "doveadm_password not set, can't authenticate", so I added: doveadm_password = suPerSeecret in the main section. Now I get trough to my backend servers, but they complain about: dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication Any ideas for what that might be? This is with dovecot v2.0.14. -jf From as at 3a.pl Tue Feb 14 23:29:19 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 14 Feb 2012 22:29:19 +0100 Subject: [Dovecot] domain quota in sql Message-ID: <4F3AD22F.5080904@3a.pl> Hi, I have a problem with setting up quota usage tracking for domains. Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. To recreate this situation, one has only to perform: doveadm quota recalc -u USER on a user, and the domain quota takes the value of this user quota. Have anyone a clue what could cause this problem? Quota tracking is set up both for users and domains. For users quota tracking is working fine, data is saved into dbase without problems. We are using dovecot 2.0.15. Quota enforcing (rules) is NOT a part of the problem. Important parts of config: dict { quota = mysql:/etc/dovecot/dovecot-dict-used-quota.conf.ext quota2 = mysql:/etc/dovecot/dovecot-dict-used-quota-domain.conf.ext } plugin { quota = dict:User quota::proxy::quota quota2 = dict:Domain quota:%d:proxy::quota2 } /etc/dovecot/dovecot-dict-used-quota.conf.ext map { pattern = priv/quota/storage table = used_quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = used_quota username_field = username value_field = messages } /etc/dovecot/dovecot-dict-used-quota-domain.conf.ext map { pattern = priv/quota/storage table = used_quota_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = used_quota_domain username_field = domain value_field = messages } -- Adam Szpakowski From tss at iki.fi Wed Feb 15 03:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:47:11 +0200 Subject: [Dovecot] doveadm director proxy In-Reply-To: <20120214204751.GA6279@dibs.tanso.net> References: <20120214204751.GA6279@dibs.tanso.net> Message-ID: On 14.2.2012, at 22.47, Jan-Frode Myklebust wrote: > I'm trying to configure a doveadm service that will proxy trough our > directors, following the recipie at: > > http://wiki2.dovecot.org/Director#Doveadm_server > > So on the backends I have: > > service doveadm { > inet_listener { > port = 24245 > address = * > } > } > doveadm_proxy_port = 24245 > local 192.168.42.0/24 { > doveadm_password = suPerSeecret > } > > I assume the "local" line is supposed to point at my local network..? Yes. Those IPs where you allow authentication from. > in the main section. Now I get trough to my backend servers, but they > complain about: > > dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication This means doveadm was talking incorrectly. > Any ideas for what that might be? This is with dovecot v2.0.14. Some v2.0 versions had bugs related to doveadm server. So this is probably a bug in v2.0.14. And I'm not entirely sure if doveadm server works correctly in v2.0 at all, since I did some bigger changes to v2.1 related to it. From tss at iki.fi Wed Feb 15 03:49:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:49:21 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120214193442.GA5156@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: On 14.2.2012, at 21.34, Jan-Frode Myklebust wrote: > dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX > > Looking at the timestamps in the filesystem I see that the users home > directory wasn't created before switcing to imap. > > Is this a know problem? Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). From tss at iki.fi Wed Feb 15 03:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:54:01 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3AD22F.5080904@3a.pl> References: <4F3AD22F.5080904@3a.pl> Message-ID: <147CCD39-537F-455A-9B55-212340747969@iki.fi> On 14.2.2012, at 23.29, Adam Szpakowski wrote: > Hi, > I have a problem with setting up quota usage tracking for domains. > Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. > To recreate this situation, one has only to perform: > doveadm quota recalc -u USER > on a user, and the domain quota takes the value of this user quota. > > Have anyone a clue what could cause this problem? Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: - set domain's quota to 0 - iterate *@domain users via userdb - for each user get the user's quota and add it to domain quota From tss at iki.fi Wed Feb 15 03:55:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:55:57 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <147CCD39-537F-455A-9B55-212340747969@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> Message-ID: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> On 15.2.2012, at 3.54, Timo Sirainen wrote: > Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: Oh, and you could actually already do this with a doveadm script: > - set domain's quota to 0 update .. | mysql > - iterate *@domain users via userdb doveadm user '*@domain' > - for each user get the user's quota and add it to domain quota doveadm quota get -u $user | some grep+sed magic | mysql Of course it would be nice if Dovecot internally supported this. From tss at iki.fi Wed Feb 15 04:04:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:04:24 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Message-ID: <6E30EEF1-1921-4E76-88AB-8743901E5E5C@iki.fi> On 14.2.2012, at 5.30, Michael M Slusarz wrote: >> Actually, this could probably be safely implemented by sending all of the state to the client as a string: >> >> * OK [SAVEDSTATE base64-encoded-state] >> >> There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). > > Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. There wouldn't be *any* additional state stored. All the necessary state is already stored to .log files for other purposes. The base64-encoded-state would be a self-contained description of the IMAP connection's entire state. > As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if > you come up with a better way to represent state. The client doesn't need to care about the data structure. The client simply reads a string and sends it later back to server if it wants to restore that state. Only the server needs to verify that the string looks reasonable. The same string wouldn't be sent to a different server implementation, so there wouldn't be any interoperability issues. Each server can implement it in whatever way they want to (although there should be some checks in case the same string is sent to different versions of the same server). From tss at iki.fi Wed Feb 15 04:09:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:09:57 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 14.2.2012, at 5.01, Daniel L. Miller wrote: >>> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory >> Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ >> > Ah! I would've thought configure would catch that though - before make did. Ok. Checking for C++ things in autoconf seems to be a little difficult.. From tss at iki.fi Wed Feb 15 04:19:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:19:00 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: Message-ID: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. From tss at iki.fi Wed Feb 15 04:24:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:24:19 +0200 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <2AF7DAA3-ABB0-47AC-9189-E499C68DA370@iki.fi> On 14.2.2012, at 17.02, cinzia.delcoiro at libero.it wrote: > I configure LDAP query in the file : dovecot-ldap.conf in this way: > > pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, > =proxy_timeout=600 > > specifying "=proxy=y" in order to proxy every mail. In LMTP case proxy=y and proxy_maybe=y are actually exactly the same. If you want the mail always proxied, you need to also use a different port. So I guess it'll work if you: 1) Add another LMTP service listener for port 10024 2) Add =proxy_port=10024 to pass_attrs. Although if you use this same passdb for imap/pop3 lookups as well, you need to make the port change lmtp-only: protocol lmtp { passdb { driver = sql args = dovecot-ldap-lmtp.conf } } protocol !lmtp { passdb { driver = sql args = dovecot-ldap.conf } } From tss at iki.fi Wed Feb 15 04:35:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:35:22 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3A77F9.8000002@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: > > Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? > I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. Do you see some error message also about "Too many open files", or why do you think the number of open files was the problem? If a process runs out of fds there should be such an error message. > Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. Connections to anvil and ipc processes were being dropped. That doesn't necessarily mean that everything else stops. > The "doveadm proxy list" would fail with an error message: > > $ sudo doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault > > The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. From tss at iki.fi Wed Feb 15 04:37:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:37:58 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 15.2.2012, at 4.35, Timo Sirainen wrote: >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Lets see if I can reproduce this then.. From list at airstreamcomm.net Wed Feb 15 05:08:05 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 14 Feb 2012 21:08:05 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> On Mon, 13 Feb 2012 13:47:06 +0200, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot > v2.2. Probably not all of them will make it, but I'm at least interested in > working on these if I have time. > > Previously I've mostly been working on things that different companies > were paying me to work on. This is the first time I have my own company, > but the prioritization still works pretty much the same way: > > - 1. priority: If your company is highly interested in getting something > implemented, we can do it as a project via my company. This guarantees > that you'll get the feature implemented in a way that integrates well into > your system. > - 2. priority: Companies who have bought Dovecot support contract can let > me know what they're interested in getting implemented. It's not a > guarantee that it gets implemented, but it does affect my priorities. :) > - 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are > before the 2. priority above. I guess we'll see how things work out. > > dsync-based replication > ----------------------- > > I'll write a separate post about this later. Besides, it's coming for > Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. > > Shared mailbox improvements > --------------------------- > > Support for private flags for all mailbox formats: > > namespace { > type = public > prefix = Public/ > mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public > } > > - dsync needs to be able to replicate the private flags as well as shared > flags. > - might as well add a common way for all mailbox formats to specify which > flags are shared and which aren't. $controldir/dovecot-flags would say > which is the default (private or shared) and what flags/keywords are the > opposite. > - easy way to configure shared mailboxes to be accessed via imapc > backend, which would allow easy shared mailbox accesses across servers or > simply between two system users in same server. (this may be tricky to > dsync.) > - global ACLs read from a single file supporting wildcards, instead of > multiple different files > - default ACLs for each namespace/storage root (maybe implemented using > the above..) > > Metadata / annotations > ---------------------- > > Add support for server, mailbox and mail annotations. These need to be > dsyncable, so their changes need to be stored in various .log files: > > 1. Per-server metadata. This is similar to subscriptions: Add changes to > dovecot.mailbox.log file, with each entry name a hash of the metadata key > that was changed. > > 2. Per-mailbox metadata. Changes to this belong inside > mailbox_transaction_context, which write the changes to mailbox's > dovecot.index.log files. Each log record contains a list of changed > annotation keys. This gives each change a modseq, and also allows easily > finding out what changes other clients have done, so if a client has done > ENABLE METADATA Dovecot can easily push metadata changes to client by only > reading the dovecot.index.log file. > > 3. Per-mail metadata. This is pretty much equivalent to per-mailbox > metadata, except changes are associated to specific message UIDs. > > The permanent storage is in dict. The dict keys have components: > - priv/ vs. shared/ for specifying private vs. shared metadata > - server/ vs mailbox// vs. mail// > - the metadata key name > > This would be a good time to improve the dict configuration to allow > things like: > - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a > file, while the rest goes to sql) > - allow sql dict to be used in more relational way, so that mail > annotations could be stored with tables: mailbox (id, guid) and > mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid > everywhere. > > Things to think through: > - How to handle quota? Probably needs to be different from regular mail > quota. Probably some per-user "metadata quota bytes" counter/limit. > - Dict lookups should be done asynchronously and prefetched as much as > possible. For per-mail annotation lookups mail_alloc() needs to include a > list of annotations that are wanted. > > Configuration > ------------- > > Copy all mail settings to namespaces, so it'll be possible to use > per-namespace mailbox settings. Especially important for imapc_* settings, > but can be useful for others as well. Those settings that aren't explicitly > defined in the namespace will use the global defaults. (Should doveconf -a > show all of these values, or simply the explicitly set values?) > > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so > doveconf -n outputs ALL of the configuration. There are mainly 3 config > files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. > The dict-sql is something I think needs a bigger redesign (mentioned above > in "Metadata" section), but the sql/ldap auth configs could be merged. One > way could be: > > sql_db sqlmails { > # most settings from dovecot-sql.conf.ext, except for queries > driver = mysql > connect = ... > } > > ldap_db ldapmails { > # most settings from dovecot-ldap.conf.ext, except attributes/filters > } > > passdb { > driver = sql > db = sqlmails > sql_query = select password from users where username = '%u' > } > passdb { > driver = ldap > db = ldapmails > ldap_attributes { > password = %{ldap:userPassword} > } > ldap_filter = ... > } > > The sql_db {} and ldap_db {} would be generic enough to be used everywhere > (e.g. dict-sql), not just for passdb/userdb. > > Some problems: > - Similar to the per-namespace mail settings, doveconf -a would output > all sql_query, ldap_attributes, ldap_filter, etc. settings for all > passdbs/userdbs. Perhaps a similar solution? > - The database configs contain passwords, so they should be readable only > by root. This makes running dovecot-lda and maybe doveadm difficult, since > they fail at "permission denied" when trying to open the config. There are > probably only two solutions: a) The db configs need to be !include_try'd > or b) the configs can be world-readable, but only passwords are placed to > only-root-readable files by using "password = > IMAP state saving/restoring > --------------------------- > > IMAP connections are often long running. Problems with this: > > 1. Currently each connection requires a separate process (at least to work > reliably), which means each connection also uses quite a lot of memory even > when they aren't doing anything for a long time. > 2. Some clients don't handle lost connections very nicely. So Dovecot > can't be upgraded without causing some user annoyance. Also in a cluster if > you want to bring down one server, the connections have to be disconnected > before they can be moved to another server. > > If IMAP session state could be reliably saved and later restored to > another process, both of the above problems could be avoided entirely. > Typically when a connection is IDLEing there are really just 4 things that > need to be remembered: username, selected mailbox name, its UIDVALIDITY and > HIGHESTMODSEQ. With this information the IMAP session can be fully restored > in another process without losing any state. So, what we could do is: > > 1. When an IMAP connection has bee IDLEing for a while (configurable > initial time, could be dynamically adjusted): > - move the IMAP state and the connection fd to imap-idle process > - the old imap process is destroyed > - imap-idle process can handle lots of IMAP connections > - imap-idle process also uses inotify/etc. to watch for changes in the > specified mailbox > - if any mailbox changes happen or IMAP client sends a command, start up > a new imap process, restore the state and continue from where we left off > - This could save quite a lot of memory at the expense of some CPU usage > > 2. Dovecot proxy <-> backend protocol could be improved to support moving > connection to another backend. Possibly using a separate control connection > to avoid making the proxying less efficient in normal operation. > > 3. When restarting Dovecot, move all the connections to a process that > keeps the connections open for a while. When Dovecot starts up, create imap > processes back to the connections. This allows changing configuration for > existing client connections (which sometimes may be bad! need to add checks > against client-visible config conflicts), upgrading Dovecot, etc. without > being visible to clients. The only problem is SSL connections: OpenSSL > doens't provide a way to save/restore state, so either you need to set > shutdown_clients=no (and possibly keep some imap-login processes doing SSL > proxying for a long time), or SSL connections need to be killed. Of course > the SSL handling could be outsourced to some other software/hardware > outside Dovecot. > > The IMAP state saving isn't always easy. Initially it could be implemented > only for the simple cases (which are a majority) and later extended to > cover more. > > IMAP extensions > --------------- > > - CATENATE is already implemented by Stephan > - URLAUTH is also planned to be implemented, somewhat differently than in > Apple's patch. The idea is to create a separate imap-urlauth service that > provides extra security. > - NOTIFY extension could be implemented efficiently using mailbox list > indexes, which already exists in v2.1. > - FILTERS extension can be easily implemented once METADATA is implemented > - There are also other missing extensions, but they're probably less > important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, > CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. > > Backups > ------- > > Filesystem based backups have worked well enough with Dovecot in the past. > But with new features like single instance storage it's becoming more > difficult. There's no 100% consistent way to even get filesystem level > backups with SIS enabled, because deleting both the message file and its > attachment files can't be done atomically (although usually this isn't a > real problem). Restoring SIS mails is more difficult though, first you need > to restore the dbox mail files and then you need to figure out what > attachment files from SIS need to be restored, and finally you'll need to > do doveadm import to put them into their final destination. > > I don't have much experience with backup software, but other people in my > company do. The initial idea is to implement a Dovecot backup agent to one > (commercial) backup software, which allows doing online backups and > restoring mails one user/mailbox/mail at a time. I don't know the details > yet how exactly this is going to be implemented, but the basic plan is > probably to implement a "backup" mail storage backend, which is a > PostgreSQL pg_dump-like flat file containing mails from all mailboxes. > doveadm backup/import can then export/import this format via stdout/stdin. > Incremental backups could possibly be done by giving a timestamp of > previous backup run (I'm not sure about this yet). > > Once I've managed to implement the first fully functional backup agent, it > should become clearer how to implement it to other backup solutions. > > Random things > ------------- > > - dovecot.index.cache file writing is too complex, should be simplified > - Enable auth_debug[_passwords]=yes on-the-fly for some specific > users/IPs via doveadm > - Optimize virtual mailboxes using mailbox list indexes. It wouldn't > anymore need to keep all the backend mailboxes' index files open. > - Would be nice to go forward with supporting key-value databases as mail > storage backends. Timo, I know you mentioned you would cover this in a coming post, but we were curious what the new dsync replication will be capable of. Would it monitor changes to mailboxes and push automatic replication to the remote mail store, and if this is the case could it be an N-way replication setup in which any host in a cluster can participate in the replication? Do you consider this to be a high availability solution? Thanks, Michael From tss at iki.fi Wed Feb 15 05:46:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 05:46:35 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: On 13.2.2012, at 16.40, Timo Sirainen wrote: > On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > >> when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. >> >> My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. >> - Is there some way to make dsync notice the hard links? (I used >> dovecot 2.0.17) > > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2890 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:07:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:07:55 +0200 Subject: [Dovecot] v2.1.rc7 released Message-ID: http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz.sig Okay, one more RC, because I needed a new feature implemented and forgot I wasn't supposed to do any changes anymore before v2.1.0. After that I figured I'll just do a few more changes and release RC7. So, v2.1.0 will probably be released within 1-2 days. So, the one new feature is: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. Also (not really worth mentioning in NEWS) if you now connect to imap/pop3 port when auth process is misconfigured and dies, instead of hanging the client for 5 mins it gets disconnected pretty much immediately. From ybhu at hk1.ibm.com Wed Feb 15 06:13:18 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Wed, 15 Feb 2012 12:13:18 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: I can generate the UID once I create the uidlist lock, and the UID is incremental globally. Some questions: 1, Whether I can write the mail to tmp/ after creating dovecot-uidlist.lock? 2, I did the 5 steps manually, found that new mail is set to OLD flag, so that when SELECT command, it told there was no NEW mail. Looks like need to change code in maildir_uidlist_sync_next_uid()? 3, About the dovecot-uidlist.lock file creation and deletion, I checked the code, there are functions in dovecot like maildir_uidlist_lock_timeout and maildir_uidlist_unlock. If I did that via calling these functions, there are lots of extra codes need to call(like auth, mailbox open...). So the question is if I do that in my codes, how to handle the race condition correctly? (At least there would be other sessions want to lock the files also / already locked it while I want to create the dovecot-uidlist.lock) Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 10:19 Subject AM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09089.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:25:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:25:29 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h From as at 3a.pl Wed Feb 15 09:26:28 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 08:26:28 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> Message-ID: <4F3B5E24.2080407@3a.pl> On 15.02.2012 02:55, Timo Sirainen wrote: > On 15.2.2012, at 3.54, Timo Sirainen wrote: > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > Oh, and you could actually already do this with a doveadm script: [cut] I'm using something like this, but directly on the dbase level. Unfortunately such operation is quite db intensive, so I'm looking for a simpler solution. Is there a way to add domainname as another collumn in dbase? This will simplify the SQL query and the GROUP BY could be used. > Of course it would be nice if Dovecot internally supported this. I was under impression, that dovecot does support this. In examples one can see: /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: # Multiple quota roots are also possible, for example this gives each user # their own 100MB quota and one shared 1GB quota within the domain: plugin { #quota = dict:user::proxy::quota #quota2 = dict:domain:%d:proxy::quota_domain } Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): "If username is left empty, the logged in username is used (this is typically what you want). Another useful username is '%d' for supporting domain-wide quotas." -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 09:47:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 08:47:22 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <4F3B630A.4070305@schetterer.org> Am 15.02.2012 08:26, schrieb Adam Szpakowski: > On 15.02.2012 02:55, Timo Sirainen wrote: >> On 15.2.2012, at 3.54, Timo Sirainen wrote: >> >>> Domain-quota support in Dovecot is only partial. Recalculating domain >>> quota would require more code. Last time when someone asked about >>> this I wasn't even sure how it could be implemented, but nowadays it >>> actually would be possible to do with: >> Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. > > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. > >> Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: > > # Multiple quota roots are also possible, for example this gives each user > # their own 100MB quota and one shared 1GB quota within the domain: > plugin { > #quota = dict:user::proxy::quota > #quota2 = dict:domain:%d:proxy::quota_domain > } > > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): > > "If username is left empty, the logged in username is used (this is > typically what you want). Another useful username is '%d' for supporting > domain-wide quotas." > > Hi i am not sure what are trying to goal with domain quota, if you setup your accounts i.e with postfixadmin etc then its up to the gui logic , stopping postmasters to setup more accounts ( with quotas ) as you ( the superadmin ) want. For everything else user quotas usally are enough thinkable maybe calculation of all users quotas in dict from one domain in a sql postfix query table to stop deliver in more mail, but i dont think this makes real sense sorry if i missunderstand your question.. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:07:04 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:07:04 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B630A.4070305@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> Message-ID: <4F3B67A8.2030109@3a.pl> On 15.02.2012 08:47, Robert Schetterer wrote: > Hi i am not sure what are trying to goal > with domain quota, if you setup your accounts i.e with postfixadmin etc > then its up to the gui logic , stopping postmasters to setup more > accounts ( with quotas ) as you ( the superadmin ) want. My fault, I should add some "big picture" info. We are using admin panel based on iRedMail Panel, and want to show quota usage on the domain level. Quota enforcing is only on user level and this part works nicely. What I need is to have in dbase live data which allows me to get quota usage on the domain level to show it in the admin panel. The problem is, that this operation must not by db intensive. Getting entire db and the processing it with some script is fine for daily reports but not for "live" view. My example with doveadm for quota recalc was only a way to show how the problem can be triggered. We are not using doveadm for this purpose. > For everything else user quotas usally are enough > > thinkable maybe calculation of all users quotas in dict from one domain > in a sql postfix query table to stop deliver in more mail, but i dont > think this makes real sense In future the quota enforcement on domain level could be a nice add-on, but for now it is not so important for us. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 10:10:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:10:27 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F246339.708@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> Message-ID: <4F3B6873.2010601@schetterer.org> Am 28.01.2012 22:06, schrieb Robert Schetterer: > Am 28.01.2012 21:07, schrieb Timo Sirainen: >> On 13.1.2012, at 20.29, Mark Moseley wrote: >> >>> If there are multiple hosts, it seems like the most robust thing to do >>> would be to exhaust the existing connections and if none of those >>> succeed, then start a new connection to one of them. It will probably >>> result in much more convoluted logic but it'd probably match better >>> what people expect from a retry. >> >> Done: http://hg.dovecot.org/dovecot-2.0/rev/4e7676b890f1 >> > > Hi Timo > > doc/example-config/dovecot-sql.conf.ext > from hg > has something like > > # Database connection string. This is driver-specific setting. > # HA / round-robin load-balancing is supported by giving multiple host > # settings, like: host=sql1.host.org host=sql2.host.org > > but i dont find it in > http://wiki2.dovecot.org/AuthDatabase/SQL > > Hi Timo,just to make sure i have an extra dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) connect = host=192.168.123.150 dbname=.. user=... password=... map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } is it possible to have i.e connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... password=... there too? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 10:19:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:19:57 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <4F3B6AAD.7060901@schetterer.org> Am 15.02.2012 09:07, schrieb Adam Szpakowski: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. as workaround it should be possible to calculate filllevel by domain with some bash script and cron sending an info mail to you might be not exact, but enough to see whats going on ( depends on your real domain/mailbox setup...maildir etc) i.e like du -sch schetterer.org/ 1,7G schetterer.org/ or du -sch schetterer.org/* 24K schetterer.org/.... at schetterer.org 431M schetterer.org/... at schetterer.org 1,3G schetterer.org/.... at schetterer.org 1,7G sum perhaps the same works with getting it out from sql > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:37:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:37:02 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B6AAD.7060901@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> <4F3B6AAD.7060901@schetterer.org> Message-ID: <4F3B6EAE.2010201@3a.pl> On 15.02.2012 09:19, Robert Schetterer wrote: > as workaround it should be possible to calculate filllevel > by domain with some bash script and cron sending an info mail to you > might be not exact, but enough to see whats going on ( depends on your > real domain/mailbox setup...maildir etc) [cut] I'm affraid that this is not a viable option. It should be done quite often to achieve "live data" feel and the amount of I/O to do so on storage/filesystem level is staggering. Good enough solution will be to have an additional column in db table with domain for each quota record. This will allow us to do something like this: SELECT domain, sum(bytes) as sum_bytes, sum(messages) as sum_messages FROM used_quota GROUP BY domain The missing domain info can be updated regularly via some cron script. This could be a workable solution. -- Adam Szpakowski From busseniu at in.tum.de Wed Feb 15 10:51:26 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 09:51:26 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B720E.6010705@in.tum.de> Hi, thank you for the reply. On 02/15/2012 03:35 AM, Timo Sirainen wrote: > On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > >> our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: >> >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? At the moment, there are - 1118 imap-login processes, - 1136 fds in anvil according to lsof - 1123 connections to anvil according to its proctitle - 1123 fds in ipc - 1111 connections to ipc > >> I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. > > The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. > Do you see some error message also about "Too many open files", Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files - Twice: config: Error: net_accept() failed: Too many open files - Twice: ipc: Error: net_accept() failed: Too many open files - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped (Dovecot has been restarted once during this hour.) > or why do you think the number of open files was the problem? The default ulimit for open files is 1024 on this Ubuntu system (I should have noted that). The problems (log messages and "doveadm proxy list" error, as described in my last mail) appeared when there were about 1000 connections. I restarted dovecot a couple of times, and each time the problems re-appeared at ~1000 connections. Once I put "ulimit -n 6000" in the init script, the problems disappeared. I am pretty sure that this is what fixed everything for us. Since there have been no log messages about client_limit after we increased the ulimit, I think we can assume that the client_limit has never been the problem. This is why I reported the log message. As the ulimit was 1024 in the beginning, I guess there is no way the connections to anvil or ipc could possibly have reached 100.000. In fact, I never saw such a high number in the proctitle. From my very naive understanding of the source code (master/service-monitor.c): if (service->last_drop_warning + SERVICE_DROP_WARN_INTERVAL_SECS < ioloop_time) { service->last_drop_warning = ioloop_time; limit = service->process_limit > 1 ? service->process_limit : service->client_limit; i_warning("service(%s): %s (%u) reached, " "client connections are being dropped", service->set->name, service->process_limit > 1 ? "process_limit" : "client_limit", limit); } The actual number of open connections is not explicitly compared against client_limit before the warning. Maybe this leads to wrong numbers in the warning? (This part of the code looks similar in hg 2.0 and 2.1., though I have only used 2.0.18.) >> The "doveadm proxy list" would fail with an error message: >> >> $ sudo doveadm proxy list >> doveadm(root): Error: LIST failed: Disconnected >> Segmentation fault >> >> The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. > > http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. Thanks? however I am afraid I have not been able to reproduce the segfault since yesterday. Cheers, Christoph From przemek.orzechowski at makolab.pl Wed Feb 15 11:09:47 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Wed, 15 Feb 2012 10:09:47 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <9b9cf0ba2ada5b598eae8b4fc42e75e8@localhost> Hi Im not an expert in Dovecot but what prohibits you adding a field to the table Dovecot will not use it thats all Im not sure what version of database You are using but postgres,oracle and never versions of mysql have something called views (could be used to hide the additional column from dovecot if necessary) and as far as i remember triggers that could be launched on insert, update to populate that additional column Other solution (not sure if applicable for Your needs) is to enable file system quota without actually setting up any quotas/group quotas (all users from single domain must belong to same group tho) this allows quick disk usage check without excess disk io On Wed, 15 Feb 2012 09:07:04 +0100, Adam Szpakowski wrote: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. -- Pozdrawiam, Przemys?aw Orzechowski Administrator Sieci/Network Administrator e: przemek.orzechowski at makolab.net t: +48 42 683 74 96 MakoLab S.A. ul. Demokratyczna 46, 93-430 ??d? www.makolab.pl Sp??ka zarejestrowana w Krajowym Rejestrze S?dowym przez S?d Rejonowy dla ?odzi - ?r?dmie?cia w ?odzi XX Wydzia? Krajowego Rejestru S?dowego pod numerem KRS 0000289179. Wysoko?? kapita?u zak?adowego wynosi 707 473 PLN. Kapita? zak?adowy zosta? wp?acony w ca?o?ci. NIP 7250015526, REGON 471343117 Wiadomo?? ta jest przeznaczona jedynie dla osoby lub podmiotu b?d?cego jej adresatem i mo?e zawiera? poufne lub uprzywilejowane informacje. Zakazane jest przegl?danie, przesy?anie, rozpowszechnianie lub inne wykorzystywanie tych informacji, jak r?wnie? podejmowanie dzia?a? na ich podstawie, przez osoby lub podmioty inne ni? zamierzony adresat. Je?li otrzymali Pa?stwo t? wiadomo?? przez pomy?k?, prosimy o poinformowanie nadawcy i usuni?cie jej z komputera. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please notify the sender and delete the material from your computer. From busseniu at in.tum.de Wed Feb 15 11:50:15 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 10:50:15 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B7FD7.1000109@in.tum.de> On 15.02.2012 03:37, Timo Sirainen wrote: > On 15.2.2012, at 4.35, Timo Sirainen wrote: > Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Since yesterday all our users (2000 active) are being routed over this proxy, and we have not experienced any performance problems. The system load is below 0.1 most of the time. So afaict there is no reason to use high-performance mode. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 15 12:12:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 11:12:35 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F3B8513.2040806@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. Thanks. I guess it would be wise to upgrade our new Dovecot mail store to 2.1 before we migrate all our Courier maildir users to it... Cheers, Chnristoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Wed Feb 15 12:30:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 12:30:14 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3B720E.6010705@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 10.51, Christoph Bu?enius wrote: >> Do you see some error message also about "Too many open files", > > Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: > > - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files > - Twice: config: Error: net_accept() failed: Too many open files > - Twice: ipc: Error: net_accept() failed: Too many open files > - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. From janfrode at tanso.net Wed Feb 15 13:52:09 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 15 Feb 2012 12:52:09 +0100 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: <20120215115209.GA21047@dibs.tanso.net> On Wed, Feb 15, 2012 at 03:49:21AM +0200, Timo Sirainen wrote: > > > > Looking at the timestamps in the filesystem I see that the users home > > directory wasn't created before switcing to imap. > > > > Is this a know problem? > > Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). > Is it maybe changeset 11683:148fccbe9f32 you remeber: - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX This was just the updated to the todo-list, but I can't see what the fix was. Also, if it was just occationally failing, it might not be that critical.. So far it's only happended for one user for the last 36 hours, so either it's only occationally failing, or the other new users are visiting webmail/imap before pop. -jf From tss at iki.fi Wed Feb 15 15:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:01:05 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120215115209.GA21047@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> <20120215115209.GA21047@dibs.tanso.net> Message-ID: <1329310865.11500.170.camel@innu> On Wed, 2012-02-15 at 12:52 +0100, Jan-Frode Myklebust wrote: > Is it maybe changeset 11683:148fccbe9f32 you remeber: > > - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives > - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX That's probably maildir-specific. But dboxes have had same problems. > This was just the updated to the todo-list, but I can't see what the fix > was. Also, if it was just occationally failing, it might not be that > critical.. So far it's only happended for one user for the last 36 > hours, so either it's only occationally failing, or the other new users > are visiting webmail/imap before pop. Yes, it's most likely only temporary and the second login works. From tss at iki.fi Wed Feb 15 15:05:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:05:38 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <1329311138.11500.175.camel@innu> On Wed, 2012-02-15 at 08:26 +0100, Adam Szpakowski wrote: > On 15.02.2012 02:55, Timo Sirainen wrote: > > On 15.2.2012, at 3.54, Timo Sirainen wrote: > > > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > > Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. Isn't it basically one table scan? To do UPDATE .. WHERE username = '% @domain' > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. Not currently. Perhaps for v2.2 dict-sql redesign. > > Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf .. > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan a problem anyway? Does your domain quota value keep breaking? In normal operation it should work fine. From tss at iki.fi Wed Feb 15 15:07:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:07:11 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3B6873.2010601@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> Message-ID: <1329311231.11500.177.camel@innu> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: > Hi Timo,just to make sure > i have an extra > dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) > > connect = host=192.168.123.150 dbname=.. user=... password=... .. > is it possible to have i.e > > connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... > password=... > > there too? Assuming your SQL database is configured to do synchronous multi-master replication, sure. From r.zimmermann at siegnetz.de Wed Feb 15 16:55:28 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 15:55:28 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 Message-ID: <4F3BC760.7080202@siegnetz.de> Hi all, today I have compiled Dovecot-2.0.18 with Dovecot-Pigeonwhole-0.2.6. Now I have following Problem with sieve: ------------------------%<--------------------------------------------- main_script: line 249: error: included personal script 'vacation.sieve' does not exist. main_script: error: validation failed. ------------------------%<--------------------------------------------- I have a definition for 'sieve_dir' and the sieve script 'vacation.sieve' exits in the folder. # Dovecot Sieve Parameters plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same configuration I can define include :personal "vacation.sieve"; in my main sieve script. All works fine. Greets Ralf From tss at iki.fi Wed Feb 15 17:13:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 17:13:33 +0200 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <96E451B3-17B0-4C57-AAF7-D4D7D0BEF653@iki.fi> On 15.2.2012, at 16.55, Ralf Zimmermann wrote: > main_script: line 249: error: included personal script 'vacation.sieve' > does not exist. .. > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; include :personal "vacation"; From dmiller at amfes.com Wed Feb 15 17:14:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Wed, 15 Feb 2012 07:14:34 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3AFF83.4020707@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> <4F3AC6A2.5010605@amfes.com> <4F3AFF83.4020707@es2eng.com> Message-ID: On 2/14/2012 4:42 PM, Willie Gillespie wrote: > On 02/14/2012 01:40 PM, Daniel L. Miller wrote: >> On 2/14/2012 12:06 PM, Willie Gillespie wrote: >>> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >>>> I'm fighting the same issue. With the current release (2.1rc6) no >>>> errors >>>> are logged - but I'm having problems deleted folders from trash. I've >>>> enabled tb-extra-mailbox-sep. Two folders that were previously in >>>> Trash >>>> were successfully removed. I deleted four more - after several >>>> Thunderbird restarts, compacts, refreshes, I have killed three of >>>> them - >>>> there is still one folder that won't "die". Thunderbird reports the >>>> mailbox doesn't exist, yet it continues to appear. >>> >>> Out of curiosity -- does the mailbox actually exist if you look on the >>> Dovecot server? >>> Secondly, is it listed in your subscription file? >> >> At one time - yes they were. After further playing, they do not exist in >> the on-disk folder structure, nor in the subscription file. But still >> show in the client - with errors. > > Sounds like a Thunderbird bug then. With Thunderbird shut down, you > could browse into the profile. > ~\.thunderbird or > C:\Users\name\AppData\Roaming\Thunderbird\whatever\ImapMail\server.name > > Find the mailbox that is not supposed to be there and move/delete it. > The filename will usually just be the mailbox name. A .msf file is > Thunderbird's index file. It will create .sbd folders for any > sub-mailboxes. I tried going in via squirrelmail - saw no subfolders under trash. Went back in with Thunderbird - all gone. -- Daniel From tlx at leuxner.net Wed Feb 15 17:14:55 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 15 Feb 2012 16:14:55 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <20120215151455.GA1896@nihlus.leuxner.net> On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; > > in my main sieve script. All works fine. Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. Try: include :personal "vacation"; Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.zimmermann at siegnetz.de Wed Feb 15 18:06:07 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 17:06:07 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <20120215151455.GA1896@nihlus.leuxner.net> References: <4F3BC760.7080202@siegnetz.de> <20120215151455.GA1896@nihlus.leuxner.net> Message-ID: <4F3BD7EF.2070204@siegnetz.de> Thanks Thomas and Timo for the fast answer. Next time I'll read the News file ;-) It' works now. On 02/15/2012 04:14 PM, Thomas Leuxner wrote: > On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: >> If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same >> configuration I can define >> >> include :personal "vacation.sieve"; >> >> in my main sieve script. All works fine. > > Changelog v0.2.6: > > * This release fixes unintentional behavior of the include extension. > Included script names with a name like "name.sieve" would implicitly > map to a script file called "name.sieve" and not "name.sieve.sieve". > Keep in mind that the .sieve file extension has no meaning from > within the Sieve language. A Sieve script is always stored with an > appended .sieve file extension, also when the name already ends with > a .sieve suffix. > > Try: > > include :personal "vacation"; > > Regards > Thomas Regards Ralf From robert at schetterer.org Wed Feb 15 18:10:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 17:10:22 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <1329311231.11500.177.camel@innu> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> Message-ID: <4F3BD8EE.5010007@schetterer.org> Am 15.02.2012 14:07, schrieb Timo Sirainen: > On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >> Hi Timo,just to make sure >> i have an extra >> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >> >> connect = host=192.168.123.150 dbname=.. user=... password=... > .. >> is it possible to have i.e >> >> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >> password=... >> >> there too? > > Assuming your SQL database is configured to do synchronous multi-master > replication, sure. > > jep ,thx Timo -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 20:22:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 19:22:21 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BD8EE.5010007@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> Message-ID: <4F3BF7DD.8080302@schetterer.org> Am 15.02.2012 17:10, schrieb Robert Schetterer: > Am 15.02.2012 14:07, schrieb Timo Sirainen: >> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >>> Hi Timo,just to make sure >>> i have an extra >>> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >>> >>> connect = host=192.168.123.150 dbname=.. user=... password=... >> .. >>> is it possible to have i.e >>> >>> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >>> password=... >>> >>> there too? >> >> Assuming your SQL database is configured to do synchronous multi-master >> replication, sure. >> >> > jep ,thx Timo > Hi Timo, sorry for some more question what is the default behave if dict quota over sql cant be reached i.e with lmtp something like log warning and deliver anyway ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Feb 15 20:43:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:43:10 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BF7DD.8080302@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> Message-ID: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> On 15.2.2012, at 20.22, Robert Schetterer wrote: > sorry for some more question > what is the default behave if dict quota over sql cant be reached > i.e with lmtp > > something like log warning and deliver anyway ? I think it tempfails. Try. From tss at iki.fi Wed Feb 15 20:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:51:59 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On 15.2.2012, at 5.08, wrote: > I know you mentioned you would cover this in a coming post, but we were > curious what the new dsync replication will be capable of. Would it > monitor changes to mailboxes and push automatic replication to the remote > mail store, Yes. > and if this is the case could it be an N-way replication setup > in which any host in a cluster can participate in the replication? Initially 2-way, but I don't think anything prevents it being N-way. > Do you consider this to be a high availability solution? The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. The other possibility is to run Dovecot in two completely separate data centers and replicate through ssh. Here are more possibilities for how to do HA, but some of them also have downsides.. dovecot.fi mails are actually done this way, and can be accessed from either server at any time. I've been thinking about soon making half of my clients use one server and half the other one to see if I can find any dsync bugs (I've always 3-4 IMAP clients connected). From tss at iki.fi Wed Feb 15 20:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:54:01 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: <5260D0A9-A72F-48CE-8D42-50A2E818E7BA@iki.fi> On 15.2.2012, at 20.51, Timo Sirainen wrote: > The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages two replicated storages.. ugh. > are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. From moseleymark at gmail.com Wed Feb 15 21:02:42 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 15 Feb 2012 11:02:42 -0800 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On Mon, Feb 13, 2012 at 3:47 AM, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. > > Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: > > ?- 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. > ?- 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) > ?- 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. > Not to beat a dead horse, but the ability to use remote directors might be interesting. It'd make moving into a director setup probably a bit more easy. Then any server could proxy to the backend servers, but without losing the advantage of director-based locality. If a box sees one of its own IPs in the director_servers list, then it knows it's part of the ring. If it doesn't, then it could contact a randomly selected director IP. From tss at iki.fi Wed Feb 15 21:09:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 21:09:18 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <3C05A36D-F6D3-49C6-AEF1-6013E6AA01AF@iki.fi> On 15.2.2012, at 21.02, Mark Moseley wrote: > Not to beat a dead horse, but the ability to use remote directors > might be interesting. It'd make moving into a director setup probably > a bit more easy. Then any server could proxy to the backend servers, > but without losing the advantage of director-based locality. If a box > sees one of its own IPs in the director_servers list, then it knows > it's part of the ring. If it doesn't, then it could contact a randomly > selected director IP. It should already be possible to do that, although not automatically based on looking at your own IP.. Anyway, non-director servers could simply have the passdb return proxy=y host=director-servers, where director-servers expands to a round-robin list of director IPs (Dovecot uses the first one). I guess it would be possible to do this automatically if passdb lookup returns proxy=y but no host (means director isn't enabled), but if director_servers is non-empty one of the IPs would be randomly chosen. A little kludgy though.. From as at 3a.pl Wed Feb 15 21:09:18 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 20:09:18 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <1329311138.11500.175.camel@innu> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <1329311138.11500.175.camel@innu> Message-ID: <4F3C02DE.7080605@3a.pl> On 15.02.2012 14:05, Timo Sirainen wrote: >> Is there a way to add domainname as another collumn in dbase? This will >> simplify the SQL query and the GROUP BY could be used. > Not currently. Perhaps for v2.2 dict-sql redesign. It will be great feature. > Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan > a problem anyway? Does your domain quota value keep breaking? In normal > operation it should work fine. Domain quota gathering based on config examples works "funny". Its the best description I can get (yes, very descriptive ;) ). There are both entries for users (user at domain) and domains. It is not making sense for me, so I'm assuming that its the missing percentage of the working part ;) Tu sum-up the thread. I went for updating periodically the basic quota table with missing domain names. Working great. Thanks all for suggestions. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 21:10:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:10:53 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> Message-ID: <4F3C033D.9000503@schetterer.org> Am 15.02.2012 19:43, schrieb Timo Sirainen: > On 15.2.2012, at 20.22, Robert Schetterer wrote: > >> sorry for some more question >> what is the default behave if dict quota over sql cant be reached >> i.e with lmtp >> >> something like log warning and deliver anyway ? > > I think it tempfails. Try. hm, test servers arent up recently cant do it on production but i will test before i go try dove 2.1 and report, @list perhaps anyone other having it tested before? > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 21:30:46 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 14:30:46 -0500 Subject: [Dovecot] 2.1b6 imapc sync error Message-ID: <1329334246.30459.140661037015245@webmail.messagingengine.com> I received this error at the end of a migration from imapc to mdbox. Is it because I wanted doveadm 'sync' instead of 'backup'? I didn't get time to do any testing, but since you're pushing for the release, here it is: kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o imapc_password=****** -o imapc_host=*.*.*.* sync -R -u jsample_dc at mydomain.com imapc:/tmp/imapc dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2011: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2012: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox INBOX: Not supported ...etc, for all mailboxes in the account. This is my userdb entry: jsample_dc at mydomain.com:{***}password:502:6:IMAPC RECIPIENT::/bin/false: kerio2:~ adminuser$ /opt/dovecot/bin/doveconf -n # 2.1.rc6: /opt/dovecot-2.1.rc6/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_debug = yes auth_mechanisms = plain login auth_username_format = %n at mydomain.com auth_verbose = yes disable_plaintext_auth = no instance_name = dovecot-2.1.rc6 listen = 192.168.20.40 mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox mail_plugin_dir = /opt/dovecot/lib/dovecot mail_plugins = fts fts_squat # I know, this is deprecated in 2.1 mail_privileged_group = mail maildir_very_dirty_syncs = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat } postmaster_address = postmaster at mydomain.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = adminuser } } service imap { executable = imap postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = _postfix mode = 0666 user = _postfix } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_ca = References: <1329334246.30459.140661037015245@webmail.messagingengine.com> Message-ID: <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> On 15.2.2012, at 21.30, Metro Domain Admin wrote: > I received this error at the end of a migration from imapc to mdbox. Is > it because I wanted doveadm 'sync' instead of 'backup'? I didn't get > time to do any testing, but since you're pushing for the release, here > it is: > > kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o > imapc_password=****** -o imapc_host=*.*.*.* sync -R -u > jsample_dc at mydomain.com imapc:/tmp/imapc > dsync(jsample_dc at mydomain.com): Error: Can't update mailbox > Archives/2011: Not supported Yes, it's because of "sync". The "Not supported" comes from imapc backend, because Dovecot tried to do some changes to it that weren't possible with IMAP protocol (modseq changes probably). I think the migration still was successful though. From robert at schetterer.org Wed Feb 15 21:38:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:38:24 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? tests done In-Reply-To: <4F3C033D.9000503@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> <4F3C033D.9000503@schetterer.org> Message-ID: <4F3C09B0.8040500@schetterer.org> Am 15.02.2012 20:10, schrieb Robert Schetterer: > Am 15.02.2012 19:43, schrieb Timo Sirainen: >> On 15.2.2012, at 20.22, Robert Schetterer wrote: >> >>> sorry for some more question >>> what is the default behave if dict quota over sql cant be reached >>> i.e with lmtp >>> >>> something like log warning and deliver anyway ? >> >> I think it tempfails. Try. > > hm, test servers arent up recently > cant do it on production > > but i will test before i go try dove 2.1 > and report, @list perhaps anyone other having it tested before? >> > ok the test server was good enough for small tests with 2.0.13 dove private/dovecot-lmtp] said: 451 4.2.0 <.... at ....com> Internal error occurred. Refer to server log for more information. dict: Error: mysql(localhost2): Connect failed to database (postfix): Unknown MySQL server host '...' (1) - waiting for 125 seconds before retry youre right looks like default is tempfail 451 so it stays in postfix queue and is permanent retry after sql comming back , its getting delivered i.e force with Flush Queue so thats good enough for me i think -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 23:05:52 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 16:05:52 -0500 Subject: [Dovecot] 2.1b6 imapc sync error In-Reply-To: References: <1329334246.30459.140661037015245@webmail.messagingengine.com> <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> <1329338885.26216.140661037021897@webmail.messagingengine.com> Message-ID: <1329339952.472.140661037052961@webmail.messagingengine.com> On Wed, Feb 15, 2012, at 10:50 PM, Timo Sirainen wrote: > On 15.2.2012, at 22.48, Metro Domain Admin wrote: > > > On Wed, Feb 15, 2012, at 09:36 PM, Timo Sirainen wrote: > >> Yes, it's because of "sync". The "Not supported" comes from imapc > >> backend, because Dovecot tried to do some changes to it that weren't > >> possible with IMAP protocol (modseq changes probably). I think the > >> migration still was successful though. > > > > The migration went beautifully otherwise. But are the errors minor > > enough that I can continue with sync? > > Probably better to continue with "backup" just in case. :) > > > I had the idea that I could sync back to the imapc account, as a > > fallback option (and, presumably I'd want to preserve the /tmp/imapc > > directory for this?). > > That's not something that I've tested, and I don't know how well it works > (if at all). > OK, "backup" it is then. Thanks. From interfasys at gmail.com Thu Feb 16 03:12:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 01:12:07 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> Message-ID: <4F3C57E7.8070105@gmail.com> Hello, Here is a more meanigful backtrace. It happens every time I log in, using any imap client. ********************* Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00000000104e6eda in autocreate_iter_next.20443 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x00000000104dd72e in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #4 0x0000000000416351 in list_namespace_mailboxes.7882 (ctx=0x10a3f158) at cmd-list.c:408 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (struct buffer *) 0x10a04290 mutf7_name = (struct buffer *) 0x10a043e0 name = (const __unknown__ *) 0x10a10040 "INBOX" ret = 1 #5 0x000000000041c2ac in cmd_list_continue (cmd=0x10a3f040) at cmd-list.c:823 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000041c1d0 in cmd_list_full (cmd=0x10a3f040, lsub=false) at cmd-list.c:999 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = (const __unknown__ *) 0x10a04268 "INBOX" patterns_strarr = (const __unknown__ * const *) 0x10a3f1e0 str = (struct buffer *) 0x10a041f0 #7 0x000000000041bc80 in cmd_list (cmd=0x10a3f040) at cmd-list.c:1014 No locals. #8 0x0000000000411264 in command_exec (cmd=0x10a3f040) at imap-commands.c:147 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x000000000040df3c in client_command_input (cmd=0x10a3f040) at imap-client.c:673 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" __FUNCTION__ = "client_command_input" #10 0x000000000040de42 in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:740 size = 279029312 #11 0x000000000040d7b5 in client_handle_input (client=0x10a3e840) at imap-client.c:777 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" __FUNCTION__ = "client_handle_input" #12 0x000000000040d35c in client_input (client=0x10a3e840) at imap-client.c:816 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" __FUNCTION__ = "client_input" #13 0x00000000105cdffa in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #14 0x00000000105c1e16 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #15 0x00000000105c1cc4 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #16 0x00000000105c19ff in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x000000000041eda9 in main (argc=1, argv=0x7fffffffec60) at main.c:384 set_roots = {0x4230c0, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ede9 , failure_callback = 0x4190af } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x4230c0, 0x0} ****************** Cheers, Olivier On 13/02/2012 00:49, Timo Sirainen wrote: > On 13.2.2012, at 2.02, interfaSys s?rl wrote: > >> #0 0x00000000105c64e7 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104b4bf6 in mailbox_list_iter_next () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. >> #2 0x0000000000416b99 in cmd_list_continue.6853 () >> No symbol table info available. >> #3 0x000000000042166d in cmd_list_full () > > This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? > > Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. > > From ybhu at hk1.ibm.com Thu Feb 16 05:59:27 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 11:59:27 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: Tried it, found it could trigger NFS issue. The test is on one machine runs the script to lock the dovecot-uidlist, moving mail into cur/new, append one line to dovecot-uidlist, and unlock it. On another different machine, run one script to SELECT, another script to COPY. Found the similar issue in http://dovecot.org/pipermail/dovecot/2012-February/063539.html Whether it means the dovecot LDA also needs to run on the same server with the backend server? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 12:25 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09186.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From finn at kirstineslund.dk Thu Feb 16 03:04:52 2012 From: finn at kirstineslund.dk (Finn B) Date: Thu, 16 Feb 2012 02:04:52 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? Message-ID: <4F3C5634.2010906@kirstineslund.dk> Moi Timo / all. I've been using Dovecot on my Qmailtoaster production mailserver for some years - since 1.0.7 and current Version 2.0.10 - works perfectly. Now I am installing a backupsystem and a VM testsystem (2 boxes) - and no matter how and what Version I am installing on these 2 boxes I get below messages whenever I try to test connection with Telnet (localhost 110 / 995). I have tried to install all Dovecot versions from 2.010 to 2.018 as rpms, yums and as source and compiled it. It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I have done many times before) I have tried to copy all my settings from my current production system (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot 2.0.10) - nothing helps. Now I come to think of it I realize that my production system is intel based and the 2 new ones are AMD based and wonder if this may be the case ? Dovecot.log master: Error: Service(auth): command startup failed, throttling log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) pop3-login: Error: Timeout waiting for handshake from auth server. my pid=1858, input bytes=0 Regards, Finn B From tss at iki.fi Thu Feb 16 06:54:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 06:54:10 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. From ybhu at hk1.ibm.com Thu Feb 16 08:25:09 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 14:25:09 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: Sounds a little tricky. For backend server, director makes sense. But for LDA, it's also suggested to deploy the LDA on the same machine with backend server? Topologically, it's very possible the machine where LDA is running is separated from one of backend server. I will run the similar tests with pure dovecot LDA, instead of ours to see the result. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/16/2012 12:54 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic02130.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Thu Feb 16 08:29:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 08:29:32 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: <4CDB78DC-AF5D-4A70-919D-CE0BAA7D4A41@iki.fi> On 16.2.2012, at 8.25, Andy YB Hu wrote: > Sounds a little tricky. For backend server, director makes sense. But for > LDA, it's also suggested to deploy the LDA on the same machine with backend > server? Topologically, it's very possible the machine where LDA is running > is separated from one of backend server. MTA/Antivirus/Antispam is often in a separate backend server, but it can do the actual mail writing by sending the mail to Dovecot via LMTP via director. From pw at wk-serv.de Thu Feb 16 08:54:12 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 16 Feb 2012 07:54:12 +0100 Subject: [Dovecot] =?utf-8?q?Something_is_strange_-_perhaps_my_AMD_process?= =?utf-8?q?ors_=3F?= In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <79a90b1bae68f2ac4c9f57ed8ca84a99@wk-serv.de> On Thu, 16 Feb 2012 02:04:52 +0100, Finn B wrote: > Now I come to think of it I realize that my production system is > intel > based and the 2 new ones are AMD based and wonder if this may be the > case ? Dovecot 2.0.3 to 2.0.16 works fine on my DualCore Opterons (VMs). Patrick From joshua at hybrid.pl Thu Feb 16 10:41:52 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 09:41:52 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? Message-ID: Hi, I'm using finally dovecot 2.0 after quite a smooth upgrade from 1.0 :) with virtual mailboxes stored in sql database. Sometimes I need to check if a mailbox has been used recently or not. Checking for logins in dovecot logs sometimes means searching through gigabytes of info... I was just wondering if there is any possibility of running another query after successful login - just to fill some extra field like "last_login"? So far my only idea is to replace user/password query with procedure like login(user,password) that would also return a line as required by user/password query. However, some mysql implementations (for example, standard php-mysql) can't handle procedures which return values - so will it work in dovecot? Or maybe some other ideas how to do that? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 11:20:48 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 10:20:48 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: Message-ID: <20120216092048.GA10124@dibs.tanso.net> On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: > > I was just wondering if there is any possibility of running another > query after successful login - just to fill some extra field like > "last_login"? We touch a file in /var/log/activemailaccounts/$username on every successful login trough postlogin scripting: http://wiki2.dovecot.org/PostLoginScripting -jf From joshua at hybrid.pl Thu Feb 16 11:25:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 10:25:11 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting Great, that's exactly what I needed :) Thanks! Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Thu Feb 16 13:18:15 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 12:18:15 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting By the way, is such thing possible for other processess? For example, I'd like to set in mysql table information that mail has been delivered using lmtp. Would something like this work? protocol lmtp { mail_plugins = $mail_plugins sieve executable = lmtp postlmtp } Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 14:18:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 13:18:54 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: <20120216121854.GA13072@dibs.tanso.net> On Thu, Feb 16, 2012 at 12:18:15PM +0100, Jacek Osiecki wrote: > > By the way, is such thing possible for other processess? For > example, I'd like to set in mysql table information that mail has > been delivered using lmtp. Would something like this work? > > protocol lmtp { > mail_plugins = $mail_plugins sieve > executable = lmtp postlmtp > } I've been inquiring the same lately, and unfortunately that's not possible. There's no "login" involved with lmtp, and each lmtp-session can have multiple recipients.. Maybe it can be solved trough a global sieve script? -jf From joshua at hybrid.pl Thu Feb 16 17:35:14 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 16:35:14 +0100 (CET) Subject: [Dovecot] Recalculate quota when quota=dict ? Message-ID: Hi, My current setting for quota is: plugin { #quota = dirsize:User quota #quota = maildir:User quota quota = dict:User quota::proxy::quota #quota = fs:User quota } I have used mb2md.pl and copied result files manually to a desired mail folder - but quota was not recalculated. And it seems, that no matter what I do - the quota usage remains the same. I tried uncommenting line with: quota = maildir:User quota but it did not help. Any other idea how to recalculate quota? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Thu Feb 16 18:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:18:22 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3C57E7.8070105@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> Message-ID: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> On 16.2.2012, at 3.12, interfaSys s?rl wrote: > Here is a more meanigful backtrace. It happens every time I log in, > using any imap client. .. > #0 0x00000000105e0d88 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. From tss at iki.fi Thu Feb 16 18:20:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:20:08 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 16.2.2012, at 17.35, Jacek Osiecki wrote: > quota = dict:User quota::proxy::quota > > Any other idea how to recalculate quota? With v2.x you can simply do: doveadm quota recalc -u username With v1.x you can delete the user's quota row from sql. From tss at iki.fi Thu Feb 16 18:21:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:21:55 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: On 16.2.2012, at 3.04, Finn B wrote: > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. From interfasys at gmail.com Thu Feb 16 19:04:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 17:04:33 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> Message-ID: <4F3D3721.7080203@gmail.com> That patch didn't fix it, but I've now compiled Dovecot without LTO and got even more info :) Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 189 array.h: No such file or directory. in array.h (gdb) bt full #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 ctx = {inboxcase_end = 0x7fffffffe7d0 "\237\004", sep = 64 '@', inboxcase = 248} i = 0 ret = IMAP_MATCH_NO match = IMAP_MATCH_NO #1 0x000000001049a119 in autocreate_iter_autobox (ctx=0x10a0f840, autobox=0x10a0f9a8) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 match = 32767 __FUNCTION__ = "autocreate_iter_autobox" #2 0x000000001049a484 in autocreate_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 info = (const struct mailbox_info *) 0x0 autoboxes = (const struct autocreate_box *) 0x10a0f9a8 autobox = (const struct autocreate_box *) 0x10a0f9a8 count = 1 __FUNCTION__ = "autocreate_iter_next" #3 0x000000001049a561 in mailbox_list_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 info = (const struct mailbox_info *) 0x7fffffffe86c #4 0x000000000040d06a in list_namespace_mailboxes (ctx=0x10a3f158) at array.h:189 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (string_t *) 0x10a04290 mutf7_name = (string_t *) 0x10a043e0 name = 0x10a10040 "INBOX" ret = 1 #5 0x000000000040dc25 in cmd_list_continue (cmd=0x10a3f040) at array.h:189 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000040e3b7 in cmd_list_full (cmd=0x10a3f040, lsub=false) at array.h:189 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = 0x10a04268 "INBOX" patterns_strarr = (const char * const *) 0x10a3f1e0 str = (string_t *) 0x10a041f0 #7 0x000000000040e40c in cmd_list (cmd=0x10a3f040) at array.h:189 No locals. #8 0x00000000004140a0 in command_exec (cmd=0x10a3f040) at imap-commands.c:90 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x00000000004130b9 in client_command_input (cmd=0x10a3f040) at imap-client.c:94 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" #10 0x000000000041337d in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:94 size = 279029312 #11 0x00000000004134c3 in client_handle_input (client=0x10a3e840) at imap-client.c:94 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #12 0x0000000000413639 in client_input (client=0x10a3e840) at imap-client.c:94 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" #13 0x00000000105cf961 in io_loop_call_io (io=0x10a22ec0) at ioloop.c:26 ioloop = (struct ioloop *) 0x10a20040 t_id = 2 #14 0x00000000105d1382 in io_loop_handler_run (ioloop=0x10a20040) at array.h:189 ctx = (struct ioloop_handler_context *) 0x10a21080 events = (struct kevent *) 0x10a27000 event = (const struct kevent *) 0x10a27000 tv = {tv_sec = 1799, tv_usec = 999243} ts = {tv_sec = 1799, tv_nsec = 999243000} io = (struct io_file *) 0x10a22ec0 events_count = 6 ret = 1 i = 0 __FUNCTION__ = "io_loop_handler_run" #15 0x00000000105cfa04 in io_loop_run (ioloop=0x10a20040) at ioloop.c:26 No locals. #16 0x00000000105b3f1c in master_service_run (service=0x10a1a140, callback=0x41f71b ) at master-service.c:74 No locals. #17 0x000000000041fa23 in main (argc=1, argv=0x7fffffffec60) at main.c:68 set_roots = {0x422d80, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41f514 , failure_callback = 0x41f692 } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = -1 On 16/02/2012 16:18, Timo Sirainen wrote: > On 16.2.2012, at 3.12, interfaSys s?rl wrote: > >> Here is a more meanigful backtrace. It happens every time I log in, >> using any imap client. > .. >> #0 0x00000000105e0d88 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. > > Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 > > I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. > > From tss at iki.fi Thu Feb 16 19:06:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:06:59 +0200 Subject: [Dovecot] v2.1.0 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Changes since rc7: FS layout related crashfix, two man pages added. The biggest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * Temporary authentication failures sent to IMAP/POP3 clients now includes the server's hostname and timestamp. This makes it easier to find the error message from logs. * dsync was merged into doveadm. There is still "dsync" symlink pointing to "doveadm", which you can use the old way for now. The preferred ways to run dsync are "doveadm sync" (for old "dsync mirror") and "doveadm backup". + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + IMAP SPECIAL-USE extension to describe mailboxes + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. + SCRAM-SHA-1 authentication mechanism by Florian Zeitz + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + auth: Implemented support for Postfix's "TCP map" sockets for user existence lookups. - listescape plugin works perfectly now From alexwbaule at gmail.com Thu Feb 16 19:12:25 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 16 Feb 2012 15:12:25 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. You say it "old way", when it's change ? in dovecot 2.0.18 is in the old or new way ? I asking because if i update dovecot, i need to update the plugin. Tks ! Em 13 de fevereiro de 2012 18:18, Timo Sirainen escreveu: > On 13.2.2012, at 19.28, Alex Baule wrote: > >> static void emexis_mailbox_allocated(struct mailbox *box) >> { >> ? ?union mailbox_module_context *zbox; >> >> ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); >> ? ?zbox->super = box->v; >> ? ?box->v.open = Mplugin_mailbox_open; >> ? ?box->v.sync_notify = Mplugin_mailbox_sync_notify; >> >> ? ?MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); >> } > > The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > >> struct mailbox_vfuncs *v = box->vlast; >> v->sync_notify = Mplugin_mailbox_sync_notify; >> >> But in this two cases, the Mplugin_mailbox_sync_notify is never >> called... i missing something ?? > > Never called at all? What storage backend are you using as the base? Maildir? > > Anyway, the way you should be calling it is the exact same way quota plugin does: > > void Mplugin_mailbox_allocated(struct mailbox *box) > { > ? ? ? ?struct mailbox_vfuncs *v = box->vlast; > ? ? ? ?union mailbox_module_context *zbox; > > ? ? ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); > ? ? ? ?zbox->super = *v; > ? ? ? ?box->vlast = &zbox->super; > > ? ? ? ?v->open = Mplugin_mailbox_open; > ? ? ? ?v->sync_notify = Mplugin_mailbox_sync_notify; > ? ? ? ?MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); > } > From tss at iki.fi Thu Feb 16 19:15:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:15:45 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <1933A805-19A9-4476-B0C2-B331FC1033F6@iki.fi> On 16.2.2012, at 19.04, interfaSys s?rl wrote: > That patch didn't fix it, but I've now compiled Dovecot without LTO and > got even more info :) What's your doveconf -n output? From tss at iki.fi Thu Feb 16 19:21:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:21:14 +0200 Subject: [Dovecot] v2.1.0 released In-Reply-To: References: Message-ID: On 16.2.2012, at 19.06, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig > > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > Changes since rc7: FS layout related crashfix, two man pages added. Correction: I messed up by not doing a "hg pull -u" before building the v2.1.0 release, so this release is identical to rc7. Still, it's good enough that I won't bother with v2.1.1 yet (the FS layout crashfix doesn't affect any real clients). From tss at iki.fi Thu Feb 16 19:27:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:27:01 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: On 16.2.2012, at 19.12, Alex Baule wrote: > hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. > > You say it "old way", when it's change ? in dovecot 2.0.18 is in the > old or new way ? It changed before v2.0.0 release, so you should use the new way. > I asking because if i update dovecot, i need to update the plugin. It works the same in v2.1. From dluke at geeklair.net Thu Feb 16 19:42:18 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:42:18 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: References: Message-ID: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Looks like there's a problem on Mac OS X 10.5 (ppc): Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I took a cursory look at the source, but didn't notice anything obvious... -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:47:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:47:53 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> Message-ID: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> On 16.2.2012, at 19.42, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > > Looks like there's a problem on Mac OS X 10.5 (ppc): > > Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) > Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > I took a cursory look at the source, but didn't notice anything obvious... Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. Anyway, try if it works with: configure --with-ioloop=select From dluke at geeklair.net Thu Feb 16 19:49:48 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:49:48 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> Message-ID: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> On Feb 16, 2012, at 12:47 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.42, Daniel J. Luke wrote: > >> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >>> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. >> >> >> Looks like there's a problem on Mac OS X 10.5 (ppc): >> >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> I took a cursory look at the source, but didn't notice anything obvious... > > Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. > > Anyway, try if it works with: configure --with-ioloop=select 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). I'll let you know if --with-ioloop=select works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:51:59 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> Message-ID: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). OK, I bet it's the FIFOs then. See if this helps: service stats { fifo_listener stats-mail { mode = 0 } } From tss at iki.fi Thu Feb 16 20:04:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 20:04:30 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 From dluke at geeklair.net Thu Feb 16 20:20:27 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 13:20:27 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> Message-ID: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.49, Daniel J. Luke wrote: > >>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). > > OK, I bet it's the FIFOs then. See if this helps: > > service stats { > fifo_listener stats-mail { > mode = 0 > } > } That works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From bradley.giesbrecht at gmail.com Thu Feb 16 20:31:31 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Thu, 16 Feb 2012 10:31:31 -0800 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> Message-ID: <2F3C7FE5-7D31-4000-A0E4-75B1E1169332@gmail.com> On Feb 16, 2012, at 10:20 AM, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: >> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >> >>>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >>> >>> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). >> >> OK, I bet it's the FIFOs then. See if this helps: >> >> service stats { >> fifo_listener stats-mail { >> mode = 0 >> } >> } > > That works. Cool. I am surprised I missed this thread: http://dovecot.org/pipermail/dovecot/2011-November/062122.html Regards, Bradley Giesbrecht (pixilla) From interfasys at gmail.com Thu Feb 16 20:31:42 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 18:31:42 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <4F3D4B8E.2070103@gmail.com> Yep, works perfectly now. Thank you for fixing it :) On 16 February 2012 18:04:30, Timo Sirainen wrote: > OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 > > From stephan at rename-it.nl Thu Feb 16 23:29:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:29:56 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 Message-ID: <4F3D7554.1010205@rename-it.nl> Hello Dovecot users, Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most significant new features are support for the Sieve editheader extension and the sieve-filter tool. I haven't received much feedback on these new features, which can either mean that they are mostly bug-free or that not many people have tried these, so these are still considered experimental. Changelog v0.3.0: * Renamed sieve_global_path setting to sieve_default for clarity. Old name is still recognized for backwards compatibility. Support for the ancient (pre v1.1) name for this setting "global_script_path" is now dropped. * Added means to prohibit use of redirect action. Setting sieve_max_redirects to 0 now means that redirect is disallowed in stead of unlimited. Default value remains four. * Fixed interaction of Sieve include extension with ManageSieve. It is updated to match new requirements in the draft include specification. Missing included scripts are no longer an error at upload time. * Updated RFC2822 header field body verification to exclude non-printing characters (RFC5322). Only Sieve actions that can create unstructured header values (currently enotify/mailto and editheader) are affected by this change. + Completed sieve-filter tool to a useful state. The sieve-filter tool provides a means to (re)filter messages in a mailbox through a Sieve script. + Implemented the Sieve editheader extension. It is now possible to add and remove message headers from within Sieve. + ManageSieve: added support for reading quoted and literal strings as a stream. Fixes support for handing large SASL responses (analogous to similar changes in Dovecot). It is now also allowed to use a quoted string for the PUTSCRIPT script argument. + Added code to cleanup tmp directory in Sieve storage directory (sieve_dir) every once in a while. + Added support for substituting the entire message during Sieve processing. This is used for the filter action provided by the new sieve_extprograms plugin (provided separately for now). The filter action allows passing the message through an external program. + Added support for restricting certain Sieve language extensions to (admin-controled) global scripts. Restricted extensions can be configured using the new sieve_global_extensions setting. This is particularly useful for some of the Dovecot-specific (plugin-based) Sieve extensions, that can be somewhat hazardous when under direct control of users (e.g. sieve_extprograms). The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From stephan at rename-it.nl Thu Feb 16 23:35:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:35:43 +0100 Subject: [Dovecot] Sieve_extprograms plugin for Pigeonhole v0.3 Message-ID: <4F3D76AF.9060600@rename-it.nl> As indicated in the release announcement of Pigeonhole v0.3.0, there is also a new experimental plugin available for Pigeonhole v0.3: the sieve_extprograms plugin. This can be used to pipe messages to and filter messages through external programs (shell scripts). Also, string data can be retrieved from such external programs. The plugin can currently only be obtained from its Mercurial repository: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms Refer to the INSTALL and README files for more information: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/INSTALL http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/README The vendor-specific Sieve language extension provided by this plugin are documented here: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt Please test this plugin. The plan is to include it into the main Pigeonhole release eventually. Regards, Stephan. From matrixworkstation at gmail.com Thu Feb 16 23:40:32 2012 From: matrixworkstation at gmail.com (aledr) Date: Thu, 16 Feb 2012 19:40:32 -0200 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: <4F3D7554.1010205@rename-it.nl> References: <4F3D7554.1010205@rename-it.nl> Message-ID: Correct links are: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig On Thu, Feb 16, 2012 at 7:29 PM, Stephan Bosch wrote: > Hello Dovecot users, > > Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most > significant new features are support for the Sieve editheader extension and > the sieve-filter tool. I haven't received much feedback on these new > features, which can either mean that they are mostly bug-free or that not > many people have tried these, so these are still considered experimental. > > Changelog v0.3.0: > > ?* Renamed sieve_global_path setting to sieve_default for clarity. Old > ? name is still recognized for backwards compatibility. Support for > ? the ancient (pre v1.1) name for this setting "global_script_path" is > ? now dropped. > ?* Added means to prohibit use of redirect action. Setting > ? sieve_max_redirects to 0 now means that redirect is disallowed in > ? stead of unlimited. Default value remains four. > ?* Fixed interaction of Sieve include extension with ManageSieve. It is > ? updated to match new requirements in the draft include > ? specification. Missing included scripts are no longer an error at > ? upload time. > ?* Updated RFC2822 header field body verification to exclude > ? non-printing characters (RFC5322). Only Sieve actions that can > ? create unstructured header values (currently enotify/mailto and > ? editheader) are affected by this change. > ?+ Completed sieve-filter tool to a useful state. The sieve-filter tool > ? provides a means to (re)filter messages in a mailbox through a Sieve > ? script. > ?+ Implemented the Sieve editheader extension. It is now possible to > ? add and remove message headers from within Sieve. > ?+ ManageSieve: added support for reading quoted and literal strings as > ? a stream. Fixes support for handing large SASL responses (analogous > ? to similar changes in Dovecot). It is now also allowed to use a > ? quoted string for the ?PUTSCRIPT script argument. > ?+ Added code to cleanup tmp directory in Sieve storage directory > ? (sieve_dir) every once in a while. > ?+ Added support for substituting the entire message during Sieve > ? processing. This is used for the filter action provided by the new > ? sieve_extprograms plugin (provided separately for now). The filter > ? action allows passing the message through an external program. > ?+ Added support for restricting certain Sieve language extensions to > ? (admin-controled) global scripts. Restricted extensions can be > ? configured using the new sieve_global_extensions setting. This is > ? particularly useful for some of the Dovecot-specific (plugin-based) > ? Sieve extensions, that can be somewhat hazardous when under direct > ? control of users (e.g. sieve_extprograms). > > The release is available as follows: > > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig > > Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more > information. Have fun testing this new release and don't hesitate to notify > me when there are any problems. > > Regards, > > -- > Stephan Bosch > stephan at rename-it.nl > > > > > > > > > > > -- [ ]'s Aledr - Alexandre www.exatati.com.br "OpenSource Solutions for SmallBusiness Problems" From list at airstreamcomm.net Thu Feb 16 23:48:40 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 16 Feb 2012 15:48:40 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On Wed, 15 Feb 2012 20:51:59 +0200, Timo Sirainen wrote: > On 15.2.2012, at 5.08, > wrote: > >> I know you mentioned you would cover this in a coming post, but we were >> curious what the new dsync replication will be capable of. Would it >> monitor changes to mailboxes and push automatic replication to the remote >> mail store, > > Yes. > >> and if this is the case could it be an N-way replication setup >> in which any host in a cluster can participate in the replication? > > Initially 2-way, but I don't think anything prevents it being N-way. > >> Do you consider this to be a high availability solution? > > > The initial version is really about doing all of this with NFS. In NFS > setup if two replaced storages are both mounted and the primary storage > dies, Dovecot will start using the replica. So that's HA. > > The other possibility is to run Dovecot in two completely separate data > centers and replicate through ssh. Here are more possibilities for how to > do HA, but some of them also have downsides.. dovecot.fi mails are actually > done this way, and can be accessed from either server at any time. I've > been thinking about soon making half of my clients use one server and half > the other one to see if I can find any dsync bugs (I've always 3-4 IMAP > clients connected). Just to throw our thoughts into the mix, finding an open source multi-site active/active mail solution that does not require building super expensive multi-site storage systems would be a really refreshing way to purse this level of availability. Maybe the only way to accurately get this level of availability is to cluster the storage between sites? From stephan at rename-it.nl Fri Feb 17 00:03:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 23:03:07 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: References: <4F3D7554.1010205@rename-it.nl> Message-ID: <4F3D7D1B.6030404@rename-it.nl> On 2/16/2012 10:40 PM, aledr wrote: > Correct links are: > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Oops :) Regards, Stephan. From mstevens at imt-systems.com Fri Feb 17 00:27:46 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Thu, 16 Feb 2012 23:27:46 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 Message-ID: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Hi, First, Timo, Thank you for Dovecot 2.1.0. I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for RHEL6, CentOS 6 and Scientific Linux 6. If anyone is interested in testing these RPMs: http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ Feedback would be appreciated. After some testing, we will use these RPMs on our IMAP servers. Best regards, Morten From przemek.orzechowski at makolab.pl Fri Feb 17 00:39:17 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Thu, 16 Feb 2012 23:39:17 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x Message-ID: <220331515776550936318f562808607f@localhost> Hi I have already read this page but there is no info on how to delete old mails in dovecot 1.2 http://wiki1.dovecot.org/QuestionsAndAnswers#How_can_I_create_a_cron_or_something_to_delete_old_messages.3F I need to delete old mails from over 100 mailboxes with average of 10k mails / mailbox Is there a way to delete old mails (by message date not file creation date) as with doveadm in dovecot 2 ? Thanks in advance bellow my dovecot -n (not sure if it is neccessary or not so just in case) # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/makolab/star_makolab_pl.crt ssl_key_file: /etc/ssl/makolab/star_makolab_pl.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: passdb: driver: pam userdb: driver: passwd plugin: quota: fs:Limit uzytkownika:user quota2: fs:Limit domeny:group From tss at iki.fi Fri Feb 17 01:25:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:25:18 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 12.30, Timo Sirainen wrote: >>> Do you see some error message also about "Too many open files", >> >> Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: >> >> - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files >> - Twice: config: Error: net_accept() failed: Too many open files >> - Twice: ipc: Error: net_accept() failed: Too many open files >> - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped >> - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > > Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. Actually this solves it nicely: http://hg.dovecot.org/dovecot-2.1/rev/701709f472e9 It simply keeps giving net_accept() failures. From tss at iki.fi Fri Feb 17 01:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:28:28 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <220331515776550936318f562808607f@localhost> References: <220331515776550936318f562808607f@localhost> Message-ID: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > I need to delete old mails from over 100 mailboxes with average of 10k > mails / mailbox > Is there a way to delete old mails (by message date not file creation > date) as with doveadm in dovecot 2 ? You can delete them by mtime or by ctime: http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent If by "message date" you mean the Date: header, then there's no easy way. From przemek.orzechowski at makolab.pl Fri Feb 17 01:42:22 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Fri, 17 Feb 2012 00:42:22 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <6f78990c40e3c083ec43dca148edfe0d@localhost> On Fri, 17 Feb 2012 01:28:28 +0200, Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > >> I need to delete old mails from over 100 mailboxes with average of 10k >> mails / mailbox >> Is there a way to delete old mails (by message date not file creation >> date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy way. Yes Date header unfortunatly (seems i will have to write some kind of script for that) Just to be sure just deleting mail files is safe? Or do i have to run some additional commands to recreate dovecots index files? I'm using Maildir. Asking just to be sure that I will not screw up something. From slitt at troubleshooters.com Fri Feb 17 01:52:31 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 18:52:31 -0500 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <20120216185231.1eed027c@mydesk> On Fri, 17 Feb 2012 01:28:28 +0200 Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > > > I need to delete old mails from over 100 mailboxes with average of > > 10k mails / mailbox > > Is there a way to delete old mails (by message date not file > > creation date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy > way. > Would something like the following work for the date header? #!/bin/bash for msg in *; do if b4.py $msg 20080216; then rm $msg fi done #!/usr/bin/python datestring = get_dateline() ## FIND DATE HDR AND RETURN THE STRING yyyymmdd = format2yyyymmdd(datestring) ## CONVERT TO YYYYMMDD if yyyymmdd < sys.argv[1]: sys.exit(0) else: sys.exit(1) In the preceding, would the deletions mess up maildir indices? Thanks SteveT From tss at iki.fi Fri Feb 17 01:53:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:53:39 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <20120216185231.1eed027c@mydesk> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> <20120216185231.1eed027c@mydesk> Message-ID: <66CFB9DE-1A67-4EDF-B777-E482AF1226AF@iki.fi> On 17.2.2012, at 1.52, Steve Litt wrote: > In the preceding, would the deletions mess up maildir indices? no From ml at smtp.fakessh.eu Fri Feb 17 02:41:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 17 Feb 2012 01:41:06 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: Le 2012-02-16 23:27, Morten Stevens a ?crit?: > Hi, > > First, Timo, Thank you for Dovecot 2.1.0. > > I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for > RHEL6, CentOS 6 and Scientific Linux 6. > > If anyone is interested in testing these RPMs: > http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ > > Feedback would be appreciated. > > After some testing, we will use these RPMs on our IMAP servers. > > Best regards, > > Morten is there any source rpm packages for centos 5 ? best regards -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From finn at kirstineslund.dk Fri Feb 17 03:07:45 2012 From: finn at kirstineslund.dk (Finn B) Date: Fri, 17 Feb 2012 02:07:45 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3DA861.7060902@kirstineslund.dk> Moi Timo /all. Well I have looked everywhere to find any other clues - I cann't find any traces from this incident. But I have spend a lot of time figuring out what is going on so I may becom e'blind' for all the obvious so let me know if more can be found- though I have inserted more lines from Dovecot.log. Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:08 pop3-login: Error: Timeout waiting for handshake from auth serve r. my pid=29388, input bytes=0 Feb 15 01:41:38 pop3-login: Info: Disconnected: Inactivity (no auth attempts): r ip=127.0.0.1, lip=127.0.0.1, secured Feb 15 01:41:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 log: Error: service(auth): child 29450 returned error 89 (Fatal failure) Feb 15 01:42:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:43:08 imap-login: Error: Timeout waiting for handshake from auth serve r. my pid=29448, input bytes=0 This is my dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir raw SQL drivers: Passdb: checkpassword pam passwd passwd-file shadow vpopmail Userdb: checkpassword nss passwd prefetch passwd-file vpopmail and here is my Dovecot.conf's (this one is from QMT install )- further below is my Dovecot.conf taken from my production system and also used without more luck # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.18.1.el5 i686 CentOS release 5.7 (Final) auth_cache_size = 32 M auth_username_format = %Lu first_valid_uid = 89 listen = *,:: log_path = /var/log/dovecot.log login_greeting = Dovecot toaster ready. namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = cache_key=%u webmail=127.0.0.1 driver = vpopmail } plugin { quota = maildir } protocols = imap pop3 ssl_cert = method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = 89 mail_home = mail_location = maildir:~/Maildir mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = 89 mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = webmail=127.0.0.1 deny = no driver = vpopmail master = no pass = no } plugin/quota = maildir:ignore=Trash plugin/quota_rule = ?:storage=0 pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = root vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 0 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 0 } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 ssl = no } inet_listener pop3s { address = port = 995 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = On 16.2.2012, at 3.04, Finn B wrote: > >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. > > From tss at iki.fi Fri Feb 17 03:11:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 03:11:31 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3DA861.7060902@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> <4F3DA861.7060902@kirstineslund.dk> Message-ID: On 17.2.2012, at 3.07, Finn B wrote: > Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed This is your problem. Your vpopmail is somehow broken. I can't really help any further than that, since I haven't used vpopmail myself. From dovecot at r.paypc.com Fri Feb 17 04:46:23 2012 From: dovecot at r.paypc.com (Robin) Date: Thu, 16 Feb 2012 18:46:23 -0800 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> Message-ID: <4F3DBF7F.4030504@r.paypc.com> > You mean you deleted Solr index, so that it's empty? That should work too. > > Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: plugin { fts = solr fts_solr = break-imap-search url=http://solrhost:8983/solr/ } Should I be adding fts/fts_solr to the global mail_plugins setting? I have Solr up and running, without any firewalling between the hosts, and it never seems to even try to use it. The logs show, even after importing fresh mail and issuing a "search text "your" command to the server, which takes it about 5m or so to return results. I see the following in the log: Feb 16 17:51:54 indexer-worker(testuser): Info: Indexed 0 messages in INBOX2010 GET /solr/ issued to http://solrhost:8983/ via TELNET reports A-OK, and Solr Admin shows ready status when the admin console is loaded into a web browser. I can see there is ZERO traffic between the hosts during the SEARCH text command's execution, though I can see an open connection to the solr host in netstat: tcp 0 0 linuxcode:56393 solrhost:8983 ESTABLISHED =R= From slitt at troubleshooters.com Fri Feb 17 05:20:42 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 22:20:42 -0500 Subject: [Dovecot] Favorable mention in Linux Productivity Magazine Message-ID: <20120216222042.68a5a58c@mydesk> Hi all, The latest Linux Productivity Magazine, http://www.troubleshooters.com/lpm/201202/201202.htm, discusses the positive role Dovecot played in my migration away from Kmail. This is currently topical as Kmail is in the midst to transforming to Kmail2, which requires an Akonadi database and is not popular. I hope you enjoy the magazine. SteveT Steve Litt Troubleshooters.Com From klimenko.n at theitidea.ru Fri Feb 17 07:21:06 2012 From: klimenko.n at theitidea.ru (=?UTF-8?B?0JrQu9C40LzQtdC90LrQviDQndC40LrQvtC70LDQuQ==?=) Date: Fri, 17 Feb 2012 09:21:06 +0400 Subject: [Dovecot] Dovecot lda parameters Message-ID: <4F3DE3C2.1060308@theitidea.ru> hi all I have the following scheme. Postfix with virtual users with dovecot LDA. AD as database. Dovecot and Postfix get the following Idap attributes from AD: sAMAccountName, mail and mail_alias. The problem is that I can?t make dovecot lda move the user?s mail to the folder with the name from sAMAccountName attribute, as mail always moves to folder with e-mail address name. Is it posible to delivery e-mail to folder with name from ldap attributes sAMAccountName? -- From joshua at hybrid.pl Fri Feb 17 11:51:20 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 10:51:20 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: W dniu 2012-02-16 17:20, Timo Sirainen napisa?(a): > On 16.2.2012, at 17.35, Jacek Osiecki wrote: >> Any other idea how to recalculate quota? > With v2.x you can simply do: doveadm quota recalc -u username Great, That's exactly what I needed :) Sorry, it was lame - I should have found it in wiki2... By the way: what might have caused such a warning? root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Greetings, -- Jacek Osiecki From mstevens at imt-systems.com Fri Feb 17 14:40:17 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Fri, 17 Feb 2012 13:40:17 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: On 17.02.2012 01:41, ml wrote: > is there any source rpm packages for centos 5 ? No, only for CentOS6/RHEL6. Otherwise, look here: http://packages.atrpms.net/dist/el5/dovecot-2.1.x/ Best regards, Morten From joshua at hybrid.pl Fri Feb 17 15:40:32 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 14:40:32 +0100 Subject: [Dovecot] Quota warnings and LMTP Message-ID: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Hello, I have dovecot 2.0 configured with virtual users in MySQL, SMTP is handled by exim which uses dovecot LMTP transport. I'd like to test quota warning messages, but in wiki2 there is only an example for delivering it with dovecot-lda. I tried several ways of using lmtp binary, but without success. How to replace this: cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" From: postmaster at domain.com Subject: quota warning Your mailbox is now $PERCENT% full. EOF with lmtp? Greetings, -- Jacek Osiecki From as at 3a.pl Fri Feb 17 16:17:31 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 17 Feb 2012 15:17:31 +0100 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <4F3E617B.6050801@3a.pl> On 17.02.2012 14:40, joshua at hybrid.pl wrote: > Hello, Hi, > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o > "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? As far as I known, there is no specific config for LMTP transport when quota is considered. When testing quota messages, the most problems for me was with the fact, that the warning is triggered only when the limit is reached. Only once for each limit. The script we are using: #!/bin/bash PERCENT=$1 USER=$2 DOMAIN=$3 cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::noenforcing:proxy::quota" From: postmaster@$DOMAIN Subject: ostrze?enie o przekroczeniu wolnego miejsca / quota warning Twa skrzynka jest zape?niona w $PERCENT%. Your mailbox is now $PERCENT% full. EOF -- Adam Szpakowski From sanito at progresas.lt Fri Feb 17 17:39:53 2012 From: sanito at progresas.lt (sanito at progresas.lt) Date: Fri, 17 Feb 2012 17:39:53 +0200 (EET) Subject: [Dovecot] user_filter problem Message-ID: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Hello, I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 with user information stored in the LDAP database. The main problem is that email address and pop3 username are different for every mailbox (I cannot change this behaviour, because thousands of people already configured their email client for such a setup). So in my situation, user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it should be: user_filter=(cn=%u). Please, see: E-mail address: john at example.com pop3 username: 0005a pop3 password: very_secure_password And ldap ldiff for this mailbox: dn: cn=0005a,dc=example.com,ou=localmail,ou=mail uid: 0005a mail: john at example.com mailbox: /data/mail/example.com/0005a/Maildir/ sn: 0005a description: testing homeDirectory: /data/mail/example.com/0005a cn: 0005a userPassword:: e0NSWVBUfWxpOXBVaEw5SWJkRWc= objectClass: inetOrgPerson objectClass: CourierMailAccount objectClass: top I think I need to use different user_filter for lmtp and pop3, but it's not possible or I cannot find how to do it. Please, help! -- Best regards, dammit From marc at perkel.com Fri Feb 17 19:09:16 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 09:09:16 -0800 Subject: [Dovecot] 2.1 upgrade docs? Message-ID: <4F3E89BC.3070107@perkel.com> Just wondering if there's any upgrade docs for 2.1 online? From e-frog at gmx.de Fri Feb 17 19:17:03 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 17 Feb 2012 18:17:03 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E89BC.3070107@perkel.com> References: <4F3E89BC.3070107@perkel.com> Message-ID: <4F3E8B8F.1000208@gmx.de> On 17.02.2012 18:09, wrote Marc Perkel: > Just wondering if there's any upgrade docs for 2.1 online? http://wiki2.dovecot.org/Upgrading/2.1 From micah at riseup.net Fri Feb 17 19:17:53 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 12:17:53 -0500 Subject: [Dovecot] 2.1 upgrade docs? References: <4F3E89BC.3070107@perkel.com> Message-ID: <877gzlxttq.fsf@algae.riseup.net> Marc Perkel writes: > Just wondering if there's any upgrade docs for 2.1 online? yes there are. (hint: search terms "dovecot upgrade") From micah at riseup.net Fri Feb 17 20:25:44 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 13:25:44 -0500 Subject: [Dovecot] expunging Message-ID: <87r4xtwc47.fsf@algae.riseup.net> I am having trouble scaling some regular dovecot cleanup operations on our servers. On a daily basis, I'm wanting to do this on each server, which contains its own isolated set of user storage: /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d but these are a very expensive operations. For example, just doing the Spam expunge takes 30 minutes (or more, depending on the load) of heavy disk operations, on each machine it is run on. There are approximately 20k users on each machine. It seems like it does not use the iterate query, but rather it looks into the database at the expires table and rather it iterates over every user mentioned there. This is a problem because I've got multiple dovecot machines with different sets of users on them, using the same table, so that means that its doing stat() calls on each system for every user mentioned there and failing to find the user on the filesystem (because the user is on another system). In these cases it spits out an error: doveadm(user): Info: User no longer exists, skipping For users that do exist on this system, it seems to do something on the order of 15 stat() calls, at minimum. What are some things I can do to make this less of an expensive process? If I had a shared storage system that each machine used, this would cut down on the resource waste because I'd only need to run the query once, but unfortuntely, that isn't how these systems were designed. thanks for any ideas, tips etc. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From frank at moltke28.B.Shuttle.DE Fri Feb 17 20:48:17 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 17 Feb 2012 19:48:17 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? Message-ID: Hi all, are there any pitfalls known for the migration from 2.0.18 to 2.1? Can I simply stop dovecot, make install and start again? --Frank Elsner From marc at perkel.com Fri Feb 17 20:51:45 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 10:51:45 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error Message-ID: <4F3EA1C1.3080903@perkel.com> Not sure what this means: Warning: fd limit (ulimit -n) is lower than required under max. load (4096 < 20000), because of service auth { client_limit } What do I need to do? From marc at perkel.com Fri Feb 17 21:14:04 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 11:14:04 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error In-Reply-To: <4F3EA1C1.3080903@perkel.com> References: <4F3EA1C1.3080903@perkel.com> Message-ID: <4F3EA6FC.1000902@perkel.com> Never mind - I figured it out. ulimit -n 20000 On 2/17/2012 10:51 AM, Marc Perkel wrote: > Not sure what this means: > > Warning: fd limit (ulimit -n) is lower than required under max. load > (4096 < 20000), because of service auth { client_limit } > > What do I need to do? > > From eric at yorba.org Sat Feb 18 03:54:51 2012 From: eric at yorba.org (Eric Gregory) Date: Fri, 17 Feb 2012 17:54:51 -0800 Subject: [Dovecot] Order of operations in Dovecot Message-ID: Hi, I'm working on an IMAP client and had a question about the order in which commands are responded to in Dovecot. In other IMAP servers I've looked at, when you issue two commands simultaneously (i.e. pipelining) it looks something like this: a fetch something b fetch something else * 1 FETCH (result) a OK FETCH completed * 1 FETCH (result) b OK FETCH completed This works well because it's clear which results are associated with which tag; the 'a' fetch results immediately precede the a's OK response, and likewise the 'b' fetch results immediately precede the b's OK response. But in Dovecot (version 2.0.13) the results of the two simultaneous commands are ordered like this: a fetch something b fetch something else * 1 FETCH (result) * 1 FETCH (result) a OK FETCH completed b OK FETCH completed This makes associating the results of the fetch commands quite a bit more difficult, since it's not entirely clear which results are paired with which tag. My question is this: is this Dovecot's intended behavior, or did I find a bug? I realize the IMAP spec is really vague about this. If this is the intended behavior, is there any advice for clients on how to determine which results are associated with which command? Thanks! - Eric From jk at jkart.de Sat Feb 18 15:21:23 2012 From: jk at jkart.de (Jim Knuth) Date: Sat, 18 Feb 2012 14:21:23 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: References: Message-ID: <4F3FA5D3.8050101@jkart.de> am 17.02.12 19:48 schrieb Frank Elsner : > > Hi all, > > are there any pitfalls known for the migration from 2.0.18 to 2.1? > > Can I simply stop dovecot, make install and start again? THIS also interests me. > > --Frank Elsner > > > > > > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Alle M?nner hatten schon immer Angst davor, da? die Frauen ohne sie zurechtkommen k?nnten. [Margaret Mead] From tlx at leuxner.net Sat Feb 18 15:37:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 18 Feb 2012 14:37:20 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F3FA5D3.8050101@jkart.de> References: <4F3FA5D3.8050101@jkart.de> Message-ID: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> >> are there any pitfalls known for the migration from 2.0.18 to 2.1? http://wiki2.dovecot.org/Upgrading/2.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roliverio at vaslibre.org.ve Sat Feb 18 18:30:20 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:00:20 -0430 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3FD21C.6030305@vaslibre.org.ve> > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > Could you check how it's vpopmail connecting? Looks like a SSL related thing with it. HTH, -- ---------------------------------- Julio C. Ortega - VaSLibre http://www.vaslibre.org.ve Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Sat Feb 18 18:48:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:48:21 +0200 Subject: [Dovecot] Order of operations in Dovecot In-Reply-To: References: Message-ID: <4359C586-3C71-4B98-AE72-CF7BE97D2519@iki.fi> On 18.2.2012, at 3.54, Eric Gregory wrote: > But in Dovecot (version 2.0.13) the results of the two simultaneous > commands are ordered like this: > > a fetch something > b fetch something else > * 1 FETCH (result) > * 1 FETCH (result) > a OK FETCH completed > b OK FETCH completed > > This makes associating the results of the fetch commands quite a bit more > difficult, since it's not entirely clear which results are paired with > which tag. > > My question is this: is this Dovecot's intended behavior, or did I find a > bug? I realize the IMAP spec is really vague about this. If this is the > intended behavior, is there any advice for clients on how to determine > which results are associated with which command? This is intended behavior. And if you issue two long FETCH commands, their replies may even be intertwined. IMAP protocol allows all kinds of variations for this actually (e.g. it would be valid for a server to return each FETCHed field in its own untagged FETCH reply). You're not supposed to care about matching FETCH replies to FETCH commands, you just need to know that you've received all the data you asked for (after all tagged commands have replied). http://imapwiki.org/ClientImplementation From tss at iki.fi Sat Feb 18 18:51:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:51:33 +0200 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <02578EB7-C380-40E8-9D4F-79EBCBD79ECE@iki.fi> On 17.2.2012, at 15.40, joshua at hybrid.pl wrote: > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? This command is only about adding the warning mail to user's mailbox, bypassing any quota. Just because you deliver mails normally via LMTP doesn't mean that you have to do it for warning mails. This same dovecot-lda command should work just fine. From tss at iki.fi Sat Feb 18 18:52:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:52:57 +0200 Subject: [Dovecot] expunging In-Reply-To: <87r4xtwc47.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> Message-ID: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> On 17.2.2012, at 20.25, Micah Anderson wrote: > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > It seems like it does not use the iterate query, but rather it looks > into the database at the expires table and rather it iterates over every > user mentioned there. This is a problem because I've got multiple > dovecot machines with different sets of users on them, using the same > table, so that means that its doing stat() calls on each system for > every user mentioned there and failing to find the user on the > filesystem (because the user is on another system). In these cases it > spits out an error: doveadm(user): Info: User no longer exists, skipping Upgrade to v2.1: * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. From campbell at cnpapers.com Sat Feb 18 19:04:49 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 12:04:49 -0500 Subject: [Dovecot] I'm confused about my namespace parms Message-ID: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Hi, I'm new to the list and to dovecot. I'm having a strange situation that provides no errors, so I'm having a little problem diagnosing what's going on. I've been running a Centos 3 mail server with sendmail for our company for quite a few years. I finally got approval for new servers this year and put Centos 6.2 on it. I'm sticking with sendmail, but that's not important. Up to this point, I've got all working fairly well. I've found that if I configure dovecot to use the mixed format (pop and imap INBOX in /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the examples for "namespace", most works fine. Anyone who has used imap previously on our old system and work as if nothing has changed. Anyone who has never used Imap before can't seem to access the server using our webmail app (Horde/Imp). Dovecot is providing both pop and imap. An account that is set up from a normal mail client as imap has no problems as well. Sendmail is set up to create a new user in the /var/spool/mail folder. When I moved from the old server to the new server, I copied /home and /var/spool/mail to the new server - in other words, I kept the same scheme. There are no errors in the maillog, and it indicates that the user is getting logged in and out, but the return to Horde is that the user entered the wrong name or password. I'm thinking that this is probably due to how dovecot is trying to look for the imap folders, but the lack or any errors makes it a little difficult to diagnose. I'm fairly certain that what was installed on the old server was UW-Imap and was provided by the imap-2002 rpm. I tried using the examples for UW-Imap compatability but that didn't work at all. I haven't been able to figure out what's different about the old imap folder contents and what's missing in the newly created imap folders to allow Horde to access this account. I'll look at the Horde config to see if anything is there, but I'm cautious about changing anything that might break the current users' ability. Most of the accounts are set up as POP with the ability to see what's left on the server using Horde, webmail, and IMAP. Any help in tracing this down would really help. I think my biggest problem is I don't understand the "namespace" parameter as well as I should. So there it is. It seems I've got at least 3 areas to concentrate on. Steve Campbell ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From Ralf.Hildebrandt at charite.de Sat Feb 18 19:18:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:18:24 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? Message-ID: <20120218171824.GG8161@charite.de> Today I encountered this after upgrading: Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4104a) [0xb75cc04a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x410bb) [0xb75cc0bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb75cc9b7] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8052bac] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_list_full+0x4af) [0x80530df] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_lsub+0x17) [0x8053537] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](command_exec+0x32) [0x8057c92] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056c7d] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056d95] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_handle_input+0x135) [0x8056fa5] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_input+0x60) [0x80578a0] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x44) [0xb75dafb4] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xce) [0xb75dbe5e] -> /usr/local/lib/dovecot/libdovecot.so.0 a) It should not PANIC (I think) b) How can I globally "fix" the folder names (preferably in such a way that I am able to switch back and forth between 2.0 and 2.1) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From roliverio at vaslibre.org.ve Sat Feb 18 19:17:43 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:47:43 -0430 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <1329584689.4f3fda31e5952@perdition.cnpapers.net> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Message-ID: <4F3FDD37.2000903@vaslibre.org.ve> El 18/02/12 12:34, Steve Campbell escribi?: > Hi, > > I'm new to the list and to dovecot. I'm having a strange situation that provides > no errors, so I'm having a little problem diagnosing what's going on. > > I've been running a Centos 3 mail server with sendmail for our company for quite > a few years. I finally got approval for new servers this year and put Centos 6.2 > on it. I'm sticking with sendmail, but that's not important. > > Up to this point, I've got all working fairly well. I've found that if I > configure dovecot to use the mixed format (pop and imap INBOX in > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the > examples for "namespace", most works fine. Anyone who has used imap previously > on our old system and work as if nothing has changed. Anyone who has never used > Imap before can't seem to access the server using our webmail app (Horde/Imp). > Dovecot is providing both pop and imap. An account that is set up from a normal > mail client as imap has no problems as well. > > Sendmail is set up to create a new user in the /var/spool/mail folder. When I > moved from the old server to the new server, I copied /home and /var/spool/mail > to the new server - in other words, I kept the same scheme. > > There are no errors in the maillog, and it indicates that the user is getting > logged in and out, but the return to Horde is that the user entered the wrong > name or password. I'm thinking that this is probably due to how dovecot is > trying to look for the imap folders, but the lack or any errors makes it a > little difficult to diagnose. > Looks like a Mail Directory seek problem. what's your mail_location set up to?, also, does this location exist for that NEW user on it's first login? -- ---------------------------------- Julio C. Ortega - VaSLibre Usuario GNU/Linux Valencia - Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Sat Feb 18 19:25:14 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:25:14 +0100 Subject: [Dovecot] Move templates args to override_fields setting? Message-ID: <20120218172514.GH8161@charite.de> After upgrading from 2.0.x to 2.1.x I get a lot of: Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting OK, it's just a warning, but how do I fix that in a 2.0.x compatible way? The only place userdb is used is in this contaxt: # for user*masteruser logins passdb { args = /usr/local/etc/dovecot/dovecot.masteruser driver = passwd-file master = yes } # User via passwd userdb { driver = passwd args = cache_key=%u } -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Sat Feb 18 19:34:14 2012 From: micah at riseup.net (micah anderson) Date: Sat, 18 Feb 2012 12:34:14 -0500 Subject: [Dovecot] expunging In-Reply-To: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> Message-ID: <87obswvyeh.fsf@algae.riseup.net> On Sat, 18 Feb 2012 18:52:57 +0200, Timo Sirainen wrote: > On 17.2.2012, at 20.25, Micah Anderson wrote: > > > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > > > It seems like it does not use the iterate query, but rather it looks > > into the database at the expires table and rather it iterates over every > > user mentioned there. This is a problem because I've got multiple > > dovecot machines with different sets of users on them, using the same > > table, so that means that its doing stat() calls on each system for > > every user mentioned there and failing to find the user on the > > filesystem (because the user is on another system). In these cases it > > spits out an error: doveadm(user): Info: User no longer exists, skipping > > Upgrade to v2.1: > > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. Great, i'll give that version a spin, thanks! Is there a similar dict row cleanup for quota? micah From tss at iki.fi Sat Feb 18 19:48:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:48:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Today I encountered this after upgrading: > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe .. > a) It should not PANIC (I think) Right. > b) How can I globally "fix" the folder names (preferably in such a way > that I am able to switch back and forth between 2.0 and 2.1) What does this look in filesystem? Is it not in mUTF-7? If the problem is in filesystem, rename it to mUTF7 (you can use doveadm mailbox mutf7) From tss at iki.fi Sat Feb 18 19:53:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:53:22 +0200 Subject: [Dovecot] expunging In-Reply-To: <87obswvyeh.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> <87obswvyeh.fsf@algae.riseup.net> Message-ID: On 18.2.2012, at 19.34, micah anderson wrote: >> * Expire plugin: Only go through users listed by userdb iteration. >> Delete dict rows for nonexistent users, unless >> expire_keep_nonexistent_users=yes. > > Great, i'll give that version a spin, thanks! > > Is there a similar dict row cleanup for quota? No. If they're in SQL you should be able to build an SQL query for that.. From tss at iki.fi Sat Feb 18 19:56:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:56:15 +0200 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20120218172514.GH8161@charite.de> References: <20120218172514.GH8161@charite.de> Message-ID: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> On 18.2.2012, at 19.25, Ralf Hildebrandt wrote: > After upgrading from 2.0.x to 2.1.x I get a lot of: > Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting > > OK, it's just a warning, but how do I fix that in a 2.0.x compatible > way? .. > # User via passwd > userdb { > driver = passwd > args = cache_key=%u > } Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. From Ralf.Hildebrandt at charite.de Sat Feb 18 20:15:15 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:15:15 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> Message-ID: <20120218181515.GI8161@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Today I encountered this after upgrading: > > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > .. > > a) It should not PANIC (I think) > > Right. > > > b) How can I globally "fix" the folder names (preferably in such a way > > that I am able to switch back and forth between 2.0 and 2.1) > > What does this look in filesystem? Is it not in mUTF-7? If the problem > is in filesystem, rename it to mUTF7 (you can use doveadm mailbox > mutf7) # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe /home/h/e/hermel/Maildir/.Entw&APw-rfe /home/h/l/hlammert/Maildir/.Entw&APw-rfe ... /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Sat Feb 18 20:19:01 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:19:01 +0100 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> References: <20120218172514.GH8161@charite.de> <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> Message-ID: <20120218181901.GJ8161@charite.de> * Timo Sirainen : > > # User via passwd > > userdb { > > driver = passwd > > args = cache_key=%u > > } > > Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. Removed. Thanks! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From trashcan at odo.in-berlin.de Sat Feb 18 20:26:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 18 Feb 2012 19:26:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror Message-ID: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Hi -- I'm used to dsync my two mailservers running 2.0.16. Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Same error with old running command: vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here's an excerpt of my doveconf -n: mail> doveconf -n mail_home = /var/mail/homedirs/%n mail_location = mdbox:/var/mail/maildirs/%n passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } Reverted back for the time being (production servers) until I do understand the issue. Searching Google with "Fatal: All your namespaces have a location setting." only shows closed bugs back in 2010. Do I have to upgrade both servers to get syncing running? Any hint, help is highly appreciated. Thanks, Michael From Ralf.Hildebrandt at charite.de Sat Feb 18 20:49:33 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:49:33 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218181515.GI8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> Message-ID: <20120218184933.GM8161@charite.de> * Ralf Hildebrandt : > # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* > /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe > /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe > /home/h/e/hermel/Maildir/.Entw&APw-rfe > /home/h/l/hlammert/Maildir/.Entw&APw-rfe > ... > /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe > /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe > /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe Some stats: # Name =================== 2 .Entw&-APw-rfe 9 .Entw&,,0-rfe 91 .Entw&APw-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 20:59:39 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 19:59:39 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218184933.GM8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: Am 18. Februar 2012 19:49 schrieb Ralf Hildebrandt : > * Ralf Hildebrandt : > >> # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* >> /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe >> /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe >> /home/h/e/hermel/Maildir/.Entw&APw-rfe >> /home/h/l/hlammert/Maildir/.Entw&APw-rfe >> ... >> /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe >> /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe >> /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe > > Some stats: > > ? ? ?# ?Name > ? ?=================== > ? ? ?2 .Entw&-APw-rfe > ? ? ?9 .Entw&,,0-rfe > ? ? 91 .Entw&APw-rfe > Use something like "Entwuerfe" and not "Entw?rfe". -- Chris From Ralf.Hildebrandt at charite.de Sat Feb 18 21:04:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:04:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <20120218190443.GN8161@charite.de> * Chris : > > Some stats: > > > > ? ? ?# ?Name > > ? ?=================== > > ? ? ?2 .Entw&-APw-rfe > > ? ? ?9 .Entw&,,0-rfe > > ? ? 91 .Entw&APw-rfe > > > > Use something like "Entwuerfe" and not "Entw?rfe". Haha. Tell that to my users. By default, everybody gets a "Drafts" folder anyway. I'm not sure which program insists on creating "Entw?rfe" -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at luigirosa.com Sat Feb 18 21:13:13 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:13:13 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218190443.GN8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> Message-ID: <4F3FF849.8060605@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:04: > I'm not sure which program insists on creating "Entw?rfe" Outlook? ;) Ciao, luigi - -- / +--[Luigi Rosa]-- \ It is apparent to me that the possibilities of the aeroplane, which two or three years ago were thought to hold the solution to the [flying machine] problem, have been exhausted, and that we must turn elsewhere. --Thomas Edison, 1895 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8/+EUACgkQ3kWu7Tfl6ZSbogCgvIReVrSzSLg8LhtyW6d4usqh vYUAn3Br/ACxznjJ9lH0yQ5lXwLPkWvJ =OMT5 -----END PGP SIGNATURE----- From Ralf.Hildebrandt at charite.de Sat Feb 18 21:15:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:15:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: <20120218191544.GO8161@charite.de> > > I'm not sure which program insists on creating "Entw?rfe" > > Outlook? ;) Isn't it always? I hate this shit. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 21:16:03 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 20:16:03 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: 2012/2/18 Luigi Rosa : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ralf Hildebrandt said the following on 18/02/12 20:04: > >> I'm not sure which program insists on creating "Entw?rfe" > > > Outlook? ? ?;) Yes... M$ Outlook crap do this :) -- Chris From lists at luigirosa.com Sat Feb 18 21:42:45 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:42:45 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218191544.GO8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> <20120218191544.GO8161@charite.de> Message-ID: <4F3FFF35.4000600@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:15: >>> I'm not sure which program insists on creating "Entw?rfe" >> Outlook? ;) > Isn't it always? I hate this shit. Giulty by suspicion. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Airplanes are interesting toys but of no military value. --Marechal Ferdinand Foch, Professor of Strategy, Ecole Superieure de Guerre, 1904 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8//zUACgkQ3kWu7Tfl6ZRZJgCgpZJr3p0D39ssa6glFlFnH4/X WjMAnRapaPUmf9szE1iC5YajT9/oDpXq =bmTJ -----END PGP SIGNATURE----- From lists at necoro.eu Sat Feb 18 22:39:50 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Sat, 18 Feb 2012 21:39:50 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <4F400C96.4000505@necoro.eu> Am 18.02.2012 19:59, schrieb Chris: > Use something like "Entwuerfe" and not "Entw?rfe". Come on ... software having problems with non-ASCII-stuff in 2012 is just broken by design. Hence there should be no reason to use 'ue' instead of ? anywhere. - Ren? (P.S.: The intention of this mail was not to blame dovecot at all -- because at least until now, umlauts work exceptionally good here.) > -- > Chris > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From xchris89x at googlemail.com Sat Feb 18 23:31:55 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 22:31:55 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F400C96.4000505@necoro.eu> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <4F400C96.4000505@necoro.eu> Message-ID: 2012/2/18 Ren? Neumann : > Am 18.02.2012 19:59, schrieb Chris: >> Use something like "Entwuerfe" and not "Entw?rfe". > > Come on ... software having problems with non-ASCII-stuff in 2012 is > just broken by design. Hence there should be no reason to use 'ue' > instead of ? anywhere. Dovecot has no problems with non-ASCII stuff. The german language is broken by design ... ? ? ? ?... :-) -- Chris From campbell at cnpapers.com Sun Feb 19 02:35:18 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 19:35:18 -0500 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <4F3FDD37.2000903@vaslibre.org.ve> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> <4F3FDD37.2000903@vaslibre.org.ve> Message-ID: <1329611718.4f4043c6a9c6f@perdition.cnpapers.net> Quoting "Julio C. Ortega" : > El 18/02/12 12:34, Steve Campbell escribi?: > > Hi, > > > > I'm new to the list and to dovecot. I'm having a strange situation that > provides > > no errors, so I'm having a little problem diagnosing what's going on. > > > > I've been running a Centos 3 mail server with sendmail for our company for > quite > > a few years. I finally got approval for new servers this year and put > Centos 6.2 > > on it. I'm sticking with sendmail, but that's not important. > > > > Up to this point, I've got all working fairly well. I've found that if I > > configure dovecot to use the mixed format (pop and imap INBOX in > > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in > the > > examples for "namespace", most works fine. Anyone who has used imap > previously > > on our old system and work as if nothing has changed. Anyone who has never > used > > Imap before can't seem to access the server using our webmail app > (Horde/Imp). > > Dovecot is providing both pop and imap. An account that is set up from a > normal > > mail client as imap has no problems as well. > > > > Sendmail is set up to create a new user in the /var/spool/mail folder. When > I > > moved from the old server to the new server, I copied /home and > /var/spool/mail > > to the new server - in other words, I kept the same scheme. > > > > There are no errors in the maillog, and it indicates that the user is > getting > > logged in and out, but the return to Horde is that the user entered the > wrong > > name or password. I'm thinking that this is probably due to how dovecot is > > trying to look for the imap folders, but the lack or any errors makes it a > > little difficult to diagnose. > > > > Looks like a Mail Directory seek problem. what's your mail_location set > up to?, also, does this location exist for that NEW user on it's first > login? > > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > Usuario GNU/Linux > Valencia - Venezuela > > > My 10-mail.conf has the following: mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u and my namespaces are as follows: namespace { type = private separator = / prefix = "#mbox/" location = mbox:~/mail:INBOX=/var/mail/%u inbox = yes hidden = yes list = no } namespace { type = private separator = / prefix = location = maildir:~/mail } As I stated, I need to brush up on what all this really means. So far, reading the documentation has cleared up much for me. I'm sure it'll make sense later, but for now, I'm a little hazy. This machine was tested last weekend, and I had problems with reading any email. I'm using the examples found in the wiki and the above worked. Our old server is so anemic that we're getting delays of multiple hours for delivery after the server accepts it, so this weekend, I left the new server up and running. Thanks for any help and the replies. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From alessio at skye.it Sun Feb 19 12:39:02 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 19 Feb 2012 11:39:02 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL Message-ID: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Hi, I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" and SASL for postfix's SMTP-Auth, like explained ih this howto: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL Authentications over IMAP (local dovecot -> remote dovecot) works fine, and it's great. Also smtp-auth on postfix (over SASL) works fine. But in the dovecot log I found this error: Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): child 30506 killed with signal 11 (core dumps disabled) even though all works fine. This is my dovecot configuration: # 2.1.0: dovecot.conf-imapc # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_mechanisms = plain login default_login_user = nobody imapc_host = 10.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=10.0.0.1 default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl = no userdb { driver = prefetch } But if instead of "unix_listener /var/spool/postfix/private/auth" I use: service auth { unix_listener auth-userdb { } inet_listener { port = 12345 } } no errors appears in the log. In both cases all works fine. Is there any errors in my dovecot's configuration? -- Alessio Cecchi is: on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi Linux -> http://www.cecchi.biz/ Hosting Linux -> http://www.hosting-linux.it @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From mstevens at imt-systems.com Sun Feb 19 14:11:08 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 19 Feb 2012 13:11:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: On 19.02.2012 11:39, Alessio Cecchi wrote: > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: > service(auth): child 30506 killed with signal 11 (core dumps > disabled) signal 11 = segmentation fault That's probably a dovecot bug. Is it possible to reproduce this signal 11 crash? See: http://dovecot.org/bugreport.html Best regards, Morten From stephan at rename-it.nl Sun Feb 19 14:21:44 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 19 Feb 2012 13:21:44 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. Message-ID: <4F40E958.1090102@rename-it.nl> Hi, For our testing enthusiasts I've created new automatic Debian repositories for Dovecot v2.2. Repositories are available for Debian oldstable, stable and testing, on both i386 and am64. Read the following wiki item for more information: http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages The repository for Dovecot v1.2 has been defunct for a few months now and is expected to be removed soon. For the coming month or so, the builders at Xi will not be active during Dutch daytime. There are some issues with the virtual host's I/O performance that need to be resolved first. Regards, Stephan. From postmaster at mathiesen.info Sun Feb 19 14:53:25 2012 From: postmaster at mathiesen.info (postmaster) Date: Sun, 19 Feb 2012 13:53:25 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F40F0C5.2010100@mathiesen.info> I'm completely stumped by this problem and I've tried everything at present - including installing dovecot from scratch. The problem is that when trying to get email from the server -AND- there's email in the account, the pop3 or imap process freezes and no email is fetched. The crazy thing is, that it has been working perfectly, but suddenly I'm getting these freezes both when using telnet from localhost, roundcubemail on the same server and Seamonkey from another computer. Logging seems to work perfectly. For an account without email, dovecot finds the maildir++ and acts correctly. For an account -with- email, dovecot seems to find the maildir++, but also seems to try Namespace of which I haven't defined any. I've tried to get rawlog to work also, but that didn't work - the telnet process froze on login even though I'm quite sure I did the right thing and ps showed an active rawlog process Postfix is delivering the email straight into the mailboxes. Here is the information 4604ds1-ynoe:~ root# port installed | grep dovecot dovecot2 @2.0.18_0+mysql5+postgresql91 (active) dovecot2-sieve @0.2.5_1 (active) postfix @2.9.0_0+dovecot_sasl+mysql5+pcre+postgresql91 (active) 4604ds1-ynoe:~ root# 4604ds1-ynoe:log root# cat dovecot.log.txt Feb 19 11:46:52 4604ds1-ynoe dovecot[441]: master: Dovecot v2.0.18 starting up (core dumps disabled) Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=495) Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49180 resp=AGFidXNlQG1hdGhpZXNlbi5pbmZvAHRlc3R0ZXN0 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: mysql(/macports/var/run/mysql5/mysqld.sock): Connected to database infomathiesenpostfixadmin Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=abuse at mathiesen.info Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 738852865 495 1 687658e957c0e4f473891548eecc49be Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 738852865 abuse at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=503, secured Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/storage) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, alt= Feb 19 11:53:01 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Disconnected: Logged out bytes=45/750 Feb 19 11:53:06 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=512) Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49182 resp=AG1hY2ludG9zaEBtYXRoaWVzZW4uaW5mbwB0ZXN0dGVzdA== Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=macintosh at mathiesen.info Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 1866334209 512 1 332714ca2a070d7adb342d4baba4a19a Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 1866334209 macintosh at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=522, secured Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/storage) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, alt= Feb 19 11:54:13 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info: mode=0700 gid=-1 Feb 19 12:12:16 4604ds1-ynoe dovecot[441]: master: Error: service(imap): child 522 killed with signal 3 Feb 19 12:15:48 4604ds1-ynoe dovecot[441]: master: Warning: Killed with signal 15 (by pid=1086 uid=0 code=unknown 0) 4604ds1-ynoe:log root# 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 441 1 441 56804e0 0 Ss ?? 0:00.01 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 442 441 441 56804e0 0 S ?? 0:00.01 dovecot/anvil root 443 441 441 56804e0 0 S ?? 0:00.01 dovecot/log 5000 522 441 441 56804e0 0 S ?? 0:00.01 dovecot/imap root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 543 380 542 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# kill -QUIT 522 4604ds1-ynoe:mail root# dovecot stop 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 1091 380 1090 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "abuse at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1326334461] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a OK [READ-WRITE] Select completed. a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "macintosh at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox Connection closed by foreign host. 4604ds1-ynoe:~ root# 4604ds1-ynoe:~ root# dovecot -n # 2.0.18: /macports/etc/dovecot/dovecot.conf # OS: Darwin 10.8.0 i386 hfs auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /macports/var/run/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 mail_debug = yes mail_gid = 5000 mail_location = maildir:/Volumes/BrugerData/PostFix/mail/%d/%u mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 service postlogin { executable = script-login -d rawlog } ssl = no syslog_facility = local2 userdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol pop3 { pop3_client_workarounds = oe-ns-eoh outlook-no-nuls } 4604ds1-ynoe:~ root# From robert at schetterer.org Sun Feb 19 17:30:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 19 Feb 2012 16:30:37 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. In-Reply-To: <4F40E958.1090102@rename-it.nl> References: <4F40E958.1090102@rename-it.nl> Message-ID: <4F41159D.6000803@schetterer.org> Am 19.02.2012 13:21, schrieb Stephan Bosch: > Hi, > > For our testing enthusiasts I've created new automatic Debian > repositories for Dovecot v2.2. Repositories are available for Debian > oldstable, stable and testing, on both i386 and am64. Read the following > wiki item for more information: > > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > > The repository for Dovecot v1.2 has been defunct for a few months now > and is expected to be removed soon. > > For the coming month or so, the builders at Xi will not be active during > Dutch daytime. There are some issues with the virtual host's I/O > performance that need to be resolved first. > > Regards, > > Stephan. Hi Stephan, anyway, thx for keeping care of the rep ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From rob0 at gmx.co.uk Sun Feb 19 20:09:27 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 19 Feb 2012 12:09:27 -0600 Subject: [Dovecot] A Postfix/Dovecot example with SQLite backend [crossposted] Message-ID: <20120219180926.GB14195@harrier.slackbuilds.org> There are many mail howtos on the web ... can one more hurt? http://rob0.nodns4.us/howto/README http://rob0.nodns4.us/howto/ http://rob0.nodns4.us/howto/latest.tar.gz (all files) (Sorry, not HTML yet. That is on the agenda.) This is a multiple address class sample implementation of a Postfix MTA and Dovecot IMAP server using a SQLite3 data backend. Domain lookups, user maps, access and transport maps: all using a single, shared SQLite database file. What, other than the SQLite backend, distinguishes this from other mail system howtos? The Postfix high points include a complete implementation of all address classes and per-address transport(5) maps, virtual(8) UID/GID maps, and smtpd(8) recipient access(5) maps. (The latter is using smtpd_restriction_classes, which are not discussed in detail, but are implemented in an interesting way.) On the Dovecot side, it's mostly standard stuff. The SQL deny userdb implementation, and the seamless integration of system and SQL users, might be interesting. I think the database itself is the best part of this example. It's as close to "normalized" as I think it can reasonably be. A significant fact is that each revision of the system has tended to simplify the schema. That's a good sign, I think. One central Domain table lists all domains and hostnames to which the server makes reference. Likewise, a central Address table lists all addresses (with a pointer to the Domain table for each record.) The Alias table defines relationships between Address entries. (Both local(8) and virtual(5) alias maps exist in that table.) Comments and suggestions are welcome, on-list if it's topical to whichever list (please don't crosspost unless comments are relevant to both lists), or offlist to the address in the README file (or as detailed below.) Thanks for your interest. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From trashcan at odo.in-berlin.de Sun Feb 19 23:14:21 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 19 Feb 2012 22:14:21 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <622C2EAD-4DEA-4603-9C56-94CA87A2FC06@odo.in-berlin.de> Hi -- On 18.02.2012, at 19:26, Michael Grimm wrote: > Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: > > mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. [...] > Do I have to upgrade both servers to get syncing running? JFTR: 1) Upgrading both servers ends up in identical error messages. 2) After some investigation I came across a hidden upgrade info in [1], but 'doveadm dsync-server' doesn't run either :-( I'm stuck, thus I did upgrade to 2.0.18 instead ... Regards, Michael [1] http://dovecot.org/list/dovecot/2012-January/062933.html From finn_buhelt at privat.dk Mon Feb 20 01:19:17 2012 From: finn_buhelt at privat.dk (Finn_Buhelt privat) Date: Mon, 20 Feb 2012 00:19:17 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3FD21C.6030305@vaslibre.org.ve> References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <4F418375.7080609@privat.dk> Hi Julio. I'm not sure what You're asking. Can You specify and I will provide the information. I'm so puzzled since this is easy to install and I have done so before without many issues. I think I'd better get the source and go through it . Cheers, Finn B On 02/18/2012 05:30 PM, Julio C. Ortega wrote: >> Dovecot.log >> >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) >> pop3-login: Error: Timeout waiting for handshake from auth server. my >> pid=1858, input bytes=0 >> >> >> Regards, >> Finn B >> > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > From achekalin at lazurit.com Mon Feb 20 08:57:15 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 20 Feb 2012 09:57:15 +0300 Subject: [Dovecot] Homedir vs locations vs mail_location? Message-ID: <4F41EECB.2070405@lazurit.com> Hello, I use Dovecot for rather long time, but I'm still in doubt for some small things. Here they are: 1. The homedir value points to the place where everything for the user stored at, while mail_location is something (some place) where mail stored at. if I deal with pure virtual users (all users are in sql tables and no system homes for them at all), should I ever care for returning meaningful value for 'homedir' (via password_query's userdb_home), or I can simple return empty or constant ('' or '123') for it and it won't mess anything? 2. If I use single (default) namespace, should I set namespace's location (to the same value as global mail_location), and should I expect anything strange if I skip it to set? Reversely, is it possible not to set global mail_location and set only namespace's location (which would be more logical as namespace definition is compact and easy to find in config)? I've reread my questions and see it is dummy ones, but I'd like to know that for sure. Thank you in advance, Alexander From alessio at skye.it Mon Feb 20 09:26:25 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 20 Feb 2012 08:26:25 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F41F5A1.30401@skye.it> Il 16/02/2012 02:04, Finn B ha scritto: > Moi Timo / all. > > I've been using Dovecot on my Qmailtoaster production mailserver for > some years - since 1.0.7 and current Version 2.0.10 - works perfectly. > > Now I am installing a backupsystem and a VM testsystem (2 boxes) - and > no matter how and what Version I am installing on these 2 boxes I get > below messages whenever I try to test connection with Telnet (localhost > 110 / 995). > I have tried to install all Dovecot versions from 2.010 to 2.018 as > rpms, yums and as source and compiled it. > > It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I > have done many times before) > I have tried to copy all my settings from my current production system > (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot > 2.0.10) - nothing helps. > > Now I come to think of it I realize that my production system is intel > based and the 2 new ones are AMD based and wonder if this may be the case ? > > > > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > > > Hi, can you check if your vpopmail is build with "--disable-auth-logging" option? If yes first rebuild vpopmail without this options, rebuild dovecot and check again. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Mon Feb 20 11:24:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 20 Feb 2012 10:24:53 +0100 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <4F41EECB.2070405@lazurit.com> References: <4F41EECB.2070405@lazurit.com> Message-ID: <20120220092453.GA6316@dibs.tanso.net> On Mon, Feb 20, 2012 at 09:57:15AM +0300, Alexander Chekalin wrote: > > 1. The homedir value points to the place where everything for the > user stored at, while mail_location is something (some place) where > mail stored at. if I deal with pure virtual users (all users are in > sql tables and no system homes for them at all), should I ever care > for returning meaningful value for 'homedir' (via password_query's > userdb_home), or I can simple return empty or constant ('' or '123') > for it and it won't mess anything? Dovecot will store non-mailfiles in the homedir. F.ex. quota-files, sieve scripts, subscription file, .dovecot-lda.dupes, and probably more. So do yourself a favour and create a real homedir for each user :-) http://wiki2.dovecot.org/VirtualUsers/Home > > 2. If I use single (default) namespace, should I set namespace's > location (to the same value as global mail_location), and should I > expect anything strange if I skip it to set? Reversely, is it > possible not to set global mail_location and set only namespace's > location (which would be more logical as namespace definition is > compact and easy to find in config)? > We have a single namespace, with blank location: namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } But I don't really know the purpose of this location field vs. mail_location. -jf From aydin.demirel at endersys.com Mon Feb 20 11:29:43 2012 From: aydin.demirel at endersys.com (=?UTF-8?B?QXlkxLFuIERlbWlyZWw=?=) Date: Mon, 20 Feb 2012 11:29:43 +0200 Subject: [Dovecot] Feature request for maillog plugin Message-ID: <4F421287.3070101@endersys.com> Hi, Is it possible to add the modification time and file name of the mail file to maillog plugin? Thanks -- *Ayd?n Demirel Endersys Ltd. Sistem Destek M?hendisi/ System Support Engineer* * *<> Phone : +90 216 470 9423 | GSM : +90 530 401 8203 Fax : +90 216 470 9508 | Web : http://www.endersys.com Blog : http://blog.endersys.com Twitter : http://www.twitter.com/endersys LPI : The #1 Linux Certification for IT Professionals LPI (Linux Professional Institute) Turkey http://www.lpi-turkey.com From l.chelchowski at eurocar.pl Mon Feb 20 13:59:52 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 20 Feb 2012 12:59:52 +0100 Subject: [Dovecot] virtual plugin and shared namespace Message-ID: <4F4235B8.6090107@eurocar.pl> Hi, I have a problem with virtual plugin and geting mails form shared namespace. dovecot-virtual: ---- Shared/* Public/* * unseen ---- dovecot.conf: ---- namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/%d/%u/control/public:INDEX=/var/mail/vmail/%d/%u/index/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace { list = children location = maildir:%%h/:INDEX=/var/mail/vmail/%d/%u/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children prefix = Virtual/ separator = / location = virtual:~/virtual } ---- When I enter the virtual mailbox Unread I get this debug info: virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ Is it a bug or misconfig? -- ?ukasz Che?chowski From mhlavink at redhat.com Mon Feb 20 18:11:14 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 20 Feb 2012 17:11:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E8B8F.1000208@gmx.de> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> Message-ID: <4F4270A2.2040005@redhat.com> On 02/17/2012 06:17 PM, e-frog wrote: > On 17.02.2012 18:09, wrote Marc Perkel: >> Just wondering if there's any upgrade docs for 2.1 online? > > http://wiki2.dovecot.org/Upgrading/2.1 > I know about one Fedora user who had to also change: namespace { .... inbox=yes ... } to namespace inbox { .... and also in sieve (for pigeonhole): change include :personal "00-script1.sieve"; to include :personal "00-script1"; (I did not test dovecot 2.1, I still have 2.0.x on my server) From busseniu at in.tum.de Mon Feb 20 18:17:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 20 Feb 2012 17:17:28 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F427218.80709@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > On 13.2.2012, at 16.40, Timo Sirainen wrote: >> It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. > > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. After replacing "doveadm/dsync" with "dsync", the patch applied in 2.0.18 and works fine. (Is there any chance this will be in a 2.0 bugfix release?) Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From jcblanco at fi.upm.es Mon Feb 20 18:43:18 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Mon, 20 Feb 2012 17:43:18 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: <4F427826.3050506@fi.upm.es> On 20/02/2012 17:11, Michal Hlavinka wrote: > On 02/17/2012 06:17 PM, e-frog wrote: >> On 17.02.2012 18:09, wrote Marc Perkel: >>> Just wondering if there's any upgrade docs for 2.1 online? >> >> http://wiki2.dovecot.org/Upgrading/2.1 >> > > I know about one Fedora user who had to also change: > namespace { > .... > inbox=yes > ... > } > > to namespace inbox { .... > > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; > > (I did not test dovecot 2.1, I still have 2.0.x on my server) I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like user_attrs=....,ldap-attribute=quota_rule=*:storage that worked fine on version 2.0, will cause an unlimited quota for the user in version 2.1, you must adjust your configuration to look like user_attrs=....,ldap-attribute=quota_rule=*:storage=%$ as advised in the dovecot wiki http://wiki2.dovecot.org/Quota/Configuration#LDAP Regards Juan C. Blanco From metro_domain_admin at fastmail.fm Mon Feb 20 19:30:11 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Mon, 20 Feb 2012 12:30:11 -0500 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> Message-ID: <4F428323.8050404@fastmail.fm> On 2/18/12 8:37 AM, Thomas Leuxner wrote: >> are there any pitfalls known for the migration from 2.0.18 to 2.1? > http://wiki2.dovecot.org/Upgrading/2.1 Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html From campbell at cnpapers.com Mon Feb 20 20:10:10 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:10:10 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion Message-ID: <4F428C82.4010700@cnpapers.com> The more I read about all of this, the more I'm thinking about moving to maildir format. My switchover this weekend is full of holes due to the way user's imap folders were laid out. Some had folders in their home directory and others might have folders in their /home/mail directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is not doing so well. I've been reading about the conversion technique to convert to maildir, but I'm wondering if I can do this based on the way our users are set up. Most are pop accounts. Our webmail is configured to read the inbox from /var/spool/mail and their imap folders from /home or /home/mail. Some have imap accounts on their desktop defined through their mail client. RH/Centos seems to want me to switch to postfix since most of the documentation I find doesn't mention Sendmail. My filesystem is laid out to handle Inboxes in /var/spool/mail and imap folders in /home(/mail) and these aren't logical volumes but true partitions, so I've got to consider this as I make any changes. I also run pop using dovecot, so I'll need to figure all this out. I've read the technique linked to on the dovecot site that points to "Replacing UW-IMAP with Dovecot on RHEL 3" written back in 2004, and that's about the only real pointers I'm finding. I don't want to have to go around to each machine and change their clients to IMAP.. Anyone care to comment on anything similar they've done and how many pitfalls they ran into and what they were? steve campbell From CMarcus at Media-Brokers.com Mon Feb 20 20:28:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:28:49 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F4290E1.1020000@Media-Brokers.com> On 2012-02-20 1:10 PM, Steve Campbell wrote: > Our webmail is configured to read the inbox from /var/spool/mail and > their imap folders from /home or /home/mail. I can't help with your specific problem, but I do know that having mail stored directly in /home will cause problems - you will need to fix that. -- Best regards, Charles From campbell at cnpapers.com Mon Feb 20 20:34:11 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:34:11 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F4290E1.1020000@Media-Brokers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> Message-ID: <4F429223.1040202@cnpapers.com> On 2/20/2012 1:28 PM, Charles Marcus wrote: > On 2012-02-20 1:10 PM, Steve Campbell wrote: >> Our webmail is configured to read the inbox from /var/spool/mail and >> their imap folders from /home or /home/mail. > > I can't help with your specific problem, but I do know that having > mail stored directly in /home will cause problems - you will need to > fix that. That was a type. It was supposed to be ~ (user's home) or ~/mail. Thanks anyway for pointing this out. steve From CMarcus at Media-Brokers.com Mon Feb 20 20:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:45:01 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F429223.1040202@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> <4F429223.1040202@cnpapers.com> Message-ID: <4F4294AD.4020505@Media-Brokers.com> On 2012-02-20 1:34 PM, Steve Campbell wrote: > > > On 2/20/2012 1:28 PM, Charles Marcus wrote: >> On 2012-02-20 1:10 PM, Steve Campbell wrote: >>> Our webmail is configured to read the inbox from /var/spool/mail and >>> their imap folders from /home or /home/mail. >> >> I can't help with your specific problem, but I do know that having >> mail stored directly in /home will cause problems - you will need to >> fix that. > > That was a type. It was supposed to be ~ (user's home) or ~/mail. Right - that's what I meant... you do not want to ever store mail directly in the users home directory (~)... -- Best regards, Charles From mstevens at imt-systems.com Mon Feb 20 21:38:20 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 20 Feb 2012 20:38:20 +0100 Subject: [Dovecot] =?utf-8?q?2=2E1_upgrade_docs=3F?= In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: On 20.02.2012 17:11, Michal Hlavinka wrote: > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; Hi Michal, See pigeonhole changelog: v0.2.6 13-02-2012 Stephan Bosch * This release fixes unintentional behavior of the include extension. Included scriptnames with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. Best regards, Morten From jtl+dovecot at uvm.edu Mon Feb 20 21:40:29 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Mon, 20 Feb 2012 14:40:29 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F42A1AD.2000304@uvm.edu> On 2/20/12 1:10 PM, Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving > to maildir format. My switchover this weekend is full of holes due to > the way user's imap folders were laid out. Some had folders in their > home directory and others might have folders in their /home/mail > directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is > not doing so well. Not sure what problem maildir will solve for you. Large mbox mailboxes under UW-IMAP are an awful bottleneck and will bring the most powerful storage subsystems I've used to their knees, but add Dovecot indexing on top and you will be surprised how much faster everything is. We're still using mbox today and if we were to change it would be to mdbox, not maildir. > I've been reading about the conversion technique to convert to > maildir, but I'm wondering if I can do this based on the way our users > are set up. Most are pop accounts. Our webmail is configured to read > the inbox from /var/spool/mail and their imap folders from /home or > /home/mail. Some have imap accounts on their desktop defined through > their mail client. When we did our conversion from uw-imap to dovecot (about 30k accounts) we had to do a lot of clean-up work. Mail folders in the homedir needed to be moved to ~/mail. This can be scripted: if you're using mbox format, look for the first line of the file starting with "From ". (note the space) We use the uw-imap backwards compatibility option documented at http://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_UW-IMAP which allows users who have specified "mail/" as their IMAP namespace prefix to see the same message folders as people who don't. This works most of the time, but we have seen some clients (old versions of Horde/IMP) which still try to create a mail/ folder, which ends up creating ~/mail/mail, and that becomes inaccessible. Fortunately, since moving to Horde 4/IMP 5 it hasn't been a problem. You might want to keep an eye out for this if you go the backwards compatibility route. We also kept (and still keep) inboxes in /var/spool/mail (which is actually a link tree, pointing to other filesystems.) We had enough users that we needed to do "shard" directories to prevent directory lock contention from bringing everything to a crawl, e.g. jsmith's mail is in /var/spool/mail/j/s/jsmith. Dovecot nicely supports this as a config option. > > RH/Centos seems to want me to switch to postfix since most of the > documentation I find doesn't mention Sendmail. My filesystem is laid > out to handle Inboxes in /var/spool/mail and imap folders in > /home(/mail) and these aren't logical volumes but true partitions, so > I've got to consider this as I make any changes. We still use sendmail here, which calls procmail for delivery, mostly for historical reasons - many users have complex .procmailrc files. If I had to start over I'd certainly give postfix a shot. > > I also run pop using dovecot, so I'll need to figure all this out. Make sure you've read http://wiki.dovecot.org/Migration/UW , particularly for the pop3_uidl_format option, so clients do not re-download everything when you switch from UW -> Dovecot. Cheers, Jim Lawson From solo at privat.dk Tue Feb 21 01:44:30 2012 From: solo at privat.dk (solo at privat.dk) Date: Tue, 21 Feb 2012 00:44:30 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <70.4B.28132.FDAD24F4@fep46.mail.dk> Hi Julio / All. I've found the issue : the password was wrong/missing in /home/vpopmail/etc/vpopmail.mysql Now all is well. Thanks,for Your help Finn B ----- Original meddelelse ----- > Fra: Julio C. Ortega > Til: dovecot at dovecot.org > Dato: L?r, 18. feb 2012 16:30 > Emne: Re: [Dovecot] Something is strange - perhaps my AMD processors > ? > > > > Dovecot.log > > > > master: Error: Service(auth): command startup failed, throttling > > log: Error: Service(auth): child 1859 returned error 89 (Fatal > failure) > > pop3-login: Error: Timeout waiting for handshake from auth server. > my > > pid=1858, input bytes=0 > > > > > > Regards, > > Finn B > > > > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > http://www.vaslibre.org.ve > Venezuela From tss at iki.fi Tue Feb 21 02:17:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:17:20 +0200 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F427826.3050506@fi.upm.es> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: On 20.2.2012, at 18.43, Juan C. Blanco wrote: > I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. > > You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like > > user_attrs=....,ldap-attribute=quota_rule=*:storage This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 From tss at iki.fi Tue Feb 21 02:18:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:18:34 +0200 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F428323.8050404@fastmail.fm> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> Message-ID: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> On 20.2.2012, at 19.30, Metro Domain Admin wrote: > On 2/18/12 8:37 AM, Thomas Leuxner wrote: >>> are there any pitfalls known for the migration from 2.0.18 to 2.1? >> http://wiki2.dovecot.org/Upgrading/2.1 > > Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Yes, but it should still work.. From jtam.home at gmail.com Tue Feb 21 02:25:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 20 Feb 2012 16:25:46 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving to > maildir format. My switchover this weekend is full of holes due to the > way user's imap folders were laid out. Some had folders in their home > directory and others might have folders in their /home/mail directory. I had the same problem, and handled it more or less the same way as the later poster (Jim Lawson ) handled his site. I retained use of mbox format as well. I planned the migration like this - scan user home directories for mailboxes (especially "Trash", "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. This was piped into a script that Email'd users about the changes that was going to happen, what they would expect to see, and a FAQ on how to set up a mail client correctly. - during the cutover, - mailboxes left on the home directory were moved to ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid name collision for users that had a mixture of correct and blank prefixes. (I think I deleted "Trash", "Junk", etc. anyways). - Namespace aliasing was used so that prefixes "", "mail", "~/mail", and whatever darn fool settings my users used, would map to the same directory. - .subscription files were moved into the mail folder (don't have to edit prefixes since the aliasing will take care of that). - depending on what POP3 client you used (I used qpopper), you may need to configure pop3_reuse_xuidl = yes to avoid a massive re-downloading from POP3 clients after cutover. - the mail clients I control centrally (e.g. webmail, public server mail clients, etc). shouldn't need updating since they ought to have been set up properly in the first place. - after cutover, a second notification was sent for users that didn't move their mailboxes the first time around, and was it done for them during migration. And of course, test like crazy and watch the logs like a hawk. Other gotcha's: - your setup is fairly close to mine, so you may also run into the problem of user having mailboxes with group ownership that users are not part of (for example, group "mail" for INBOX set by your LDA or personal mailboxes with groups the user is no longer a member of) *and* with group permission not mode=0. You'll have to treat these (set mode=0, or change the group to something the user is part of), or the dovecot index creation will fail and they won't be able to access their mailboxes. I think my migration went pretty smoothly. Less than a handful of wazzup' Email problem reports. Joseph Tam From tss at iki.fi Tue Feb 21 02:27:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:27:28 +0200 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F40F0C5.2010100@mathiesen.info> References: <4F40F0C5.2010100@mathiesen.info> Message-ID: <3AE863BC-AEFF-478B-BBFA-99E90513F8D8@iki.fi> On 19.2.2012, at 14.53, postmaster wrote: > Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Probably won't help with your problem, but it's better if you don't fetch "mail" field. Just fetch the home and set: mail_location = maildir:~/ This avoids maildir autodetection. > a list "" * > * LIST (\HasNoChildren) "." "INBOX" > a OK List completed. > a select inbox > Connection closed by foreign host. > 4604ds1-ynoe:~ root# You mean the session hangs here until it's killed? Try attaching gdb into it: gdb -p bt full or alternatively try: gdb --args /usr/local/libexec/dovecot/imap -u user at domain run a select inbox bt full > # OS: Darwin 10.8.0 i386 hfs With HFS+ it's a good idea to set maildir_copy_with_hardlinks=no From tss at iki.fi Tue Feb 21 02:29:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:29:24 +0200 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <20120220092453.GA6316@dibs.tanso.net> References: <4F41EECB.2070405@lazurit.com> <20120220092453.GA6316@dibs.tanso.net> Message-ID: <27AB0CCC-F41B-4F99-A6BA-9EB8F1972DBE@iki.fi> On 20.2.2012, at 11.24, Jan-Frode Myklebust wrote: >> 2. If I use single (default) namespace, should I set namespace's >> location (to the same value as global mail_location), and should I >> expect anything strange if I skip it to set? Reversely, is it >> possible not to set global mail_location and set only namespace's >> location (which would be more logical as namespace definition is >> compact and easy to find in config)? >> > > We have a single namespace, with blank location: .. > But I don't really know the purpose of this location field vs. > mail_location. If namespace location is empty, it defaults to mail_location. Typically you specify namespace location only when you have more than one namespace. From tss at iki.fi Tue Feb 21 02:35:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:35:26 +0200 Subject: [Dovecot] Feature request for maillog plugin In-Reply-To: <4F421287.3070101@endersys.com> References: <4F421287.3070101@endersys.com> Message-ID: On 20.2.2012, at 11.29, Ayd?n Demirel wrote: > Is it possible to add the modification time That would be the same as IMAP INTERNALDATE / received-date. It would be possible to add this pretty easily. > and file name of the mail file to maillog plugin? This would be much more difficult, because the filename isn't known at the time when the logging is normally done. UIDs have the same problem, but they were specially implemented, and adding such support for filenames would be much more extra work. From tss at iki.fi Tue Feb 21 02:44:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:44:28 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: <4F4235B8.6090107@eurocar.pl> References: <4F4235B8.6090107@eurocar.pl> Message-ID: On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > I have a problem with virtual plugin and geting mails form shared namespace. > dovecot-virtual: > ---- > Shared/* > Public/* > * .. > When I enter the virtual mailbox Unread I get this debug info: > virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > Is it a bug or misconfig? Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 From tss at iki.fi Tue Feb 21 03:02:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:02:00 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> On 18.2.2012, at 20.26, Michael Grimm wrote: > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb From tss at iki.fi Tue Feb 21 03:09:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:09:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). From tss at iki.fi Tue Feb 21 03:11:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:11:10 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <4F3DBF7F.4030504@r.paypc.com> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> <4F3DBF7F.4030504@r.paypc.com> Message-ID: On 17.2.2012, at 4.46, Robin wrote: >> You mean you deleted Solr index, so that it's empty? That should work too. >> >> Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). > > # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts > > I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: > > plugin { > fts = solr > fts_solr = break-imap-search url=http://solrhost:8983/solr/ > } Plugin settings don't do anything if plugin isn't enabled. And if you enabled it only for imap, then it works only in imap and not in pop3/lda/doveadm/etc.. > Should I be adding fts/fts_solr to the global mail_plugins setting? Yes. Probably fixes all your troubles. From tss at iki.fi Tue Feb 21 03:13:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:13:05 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > By the way: what might have caused such a warning? > > root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. From tss at iki.fi Tue Feb 21 03:15:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:15:04 +0200 Subject: [Dovecot] user_filter problem In-Reply-To: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> References: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Message-ID: On 17.2.2012, at 17.39, sanito at progresas.lt wrote: > I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 > with user information stored in the LDAP database. The main problem is > that email address and pop3 username are different for every mailbox (I > cannot change this behaviour, because thousands of people already > configured their email client for such a setup). So in my situation, > user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it > should be: user_filter=(cn=%u). Can't it be either? user_filter = (|(mail=%u)(cn=%u)) > I think I need to use different user_filter for lmtp and pop3, but it's > not possible or I cannot find how to do it. Please, help! With v2.x you can also do something like: protocol lmtp { userdb { driver = ldap args = dovecot-ldap-lmtp.conf.ext } } protocol !lmtp { userdb { driver = ldap args = dovecot-ldap.conf.ext } } From nti at w4w.net Tue Feb 21 03:20:03 2012 From: nti at w4w.net (nti at w4w.net) Date: Tue, 21 Feb 2012 02:20:03 +0100 Subject: [Dovecot] dovecot 2 doesn't start with managesieve Message-ID: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Hi I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). Error: socket() failed: Protocol not supported Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported Fatal: Failed to start listeners I controlled the config, build pigeonhole a second time - nothing helps. Any idea ? Is there a possibility to debug the start? Nicola ... plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve ... service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service managesieve { process_limit = 1024 } ... protocol sieve { managesieve_max_compile_errors = 10 } From tss at iki.fi Tue Feb 21 03:25:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:25:26 +0200 Subject: [Dovecot] dovecot 2 doesn't start with managesieve In-Reply-To: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> References: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Message-ID: <48C8BE3A-32D9-41F7-819F-FEEB78A911E3@iki.fi> On 21.2.2012, at 3.20, nti at w4w.net wrote: > I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). > > Error: socket() failed: Protocol not supported > Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported > Fatal: Failed to start listeners > > I controlled the config, build pigeonhole a second time - nothing helps. > > Any idea ? Is there a possibility to debug the start? You don't have IPv6 supported, so listening in "::" fails. Disable it with: listen = * From javierdemiguel at us.es Tue Feb 21 09:01:11 2012 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_Miguel_Rodr=EDguez?=) Date: Tue, 21 Feb 2012 08:01:11 +0100 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: <4F434137.6070705@us.es> I have seen this behaviour with a local ext4 iSCSI filesystem. When the system is hammered by I/O (example, perfoming a full backup), I also see those messages in the log. Regards Javier > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > >> By the way: what might have caused such a warning? >> >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > From alessio at skye.it Tue Feb 21 09:46:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 08:46:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <4F434BC0.2030300@skye.it> Il 19/02/2012 13:11, Morten Stevens ha scritto: > On 19.02.2012 11:39, Alessio Cecchi wrote: >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: >> service(auth): child 30506 killed with signal 11 (core dumps disabled) > > signal 11 = segmentation fault > > That's probably a dovecot bug. > > Is it possible to reproduce this signal 11 crash? See: > http://dovecot.org/bugreport.html > > Best regards, > > Morten > Hi, I will try to provide as much as possibile information to reproduce the bug. Give me only some days to provide all informations. Thank you -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From tss at iki.fi Tue Feb 21 11:09:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:09:01 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: <4F434137.6070705@us.es> References: <4F434137.6070705@us.es> Message-ID: <1329815341.11500.223.camel@innu> Maybe http://hg.dovecot.org/dovecot-2.1/rev/2ea29ab6f10f fixes it? On Tue, 2012-02-21 at 08:01 +0100, Javier Miguel Rodr?guez wrote: > I have seen this behaviour with a local ext4 iSCSI filesystem. When the > system is hammered by I/O (example, perfoming a full backup), I also see > those messages in the log. > > Regards > > Javier > > > > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > > > >> By the way: what might have caused such a warning? > >> > >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > > > From jcblanco at fi.upm.es Tue Feb 21 11:16:14 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 21 Feb 2012 10:16:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: <4F4360DE.7070708@fi.upm.es> On 21/02/2012 1:17, Timo Sirainen wrote: > On 20.2.2012, at 18.43, Juan C. Blanco wrote: > >> I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. >> >> You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like >> >> user_attrs=....,ldap-attribute=quota_rule=*:storage > > This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 OK, thanks Timo even if the workaround was not difficult at all > -- +----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco at fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+ From tss at iki.fi Tue Feb 21 11:16:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:16:25 +0200 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <1329815785.11500.224.camel@innu> On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" > and SASL for postfix's SMTP-Auth, like explained ih this howto: > http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > > Authentications over IMAP (local dovecot -> remote dovecot) works fine, > and it's great. Also smtp-auth on postfix (over SASL) works fine. But in > the dovecot log I found this error: > > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): > child 30506 killed with signal 11 (core dumps disabled) http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. From ngoonee.talk at gmail.com Tue Feb 21 11:33:34 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 17:33:34 +0800 Subject: [Dovecot] passwd-file - lower limit for username? Message-ID: Hi, previously I could use the following in my passwd-file:- AB:{plain}password I'm not sure when it stopped working, its not an account I access with any regularity, but today it would not work (2.1.0, 64-bit Arch Linux), with an 'unknown user' error in the logs. Is there some new lower limit for username length? From Ralf.Hildebrandt at charite.de Tue Feb 21 11:43:16 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 21 Feb 2012 10:43:16 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> References: <20120218171824.GG8161@charite.de> <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> Message-ID: <20120221094316.GD11284@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > > http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). I think that's very likely. I'll update later on. Thanks again. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 21 11:54:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:54:57 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818097.11500.233.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Dovecot doesn't think the user exists. auth_debug=yes may be helpful, and doveconf -n output. > Is there some new lower limit for username length? No. From postmaster at mathiesen.info Tue Feb 21 11:58:27 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 10:58:27 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436AC3.1040800@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 On a note regarding my hardware : the computer has three ethernet-interfaces: but that doesn't really seem to be an issue as telnet from localhost also hangs indefinetely I'm seeing the issue no matter which interface I connect to, whether the two external WAN or the internal LAN IP-number . root# system_profiler SPEthernetDataType 2012-01-22 01:34:36.524 system_profiler[47377:903] Found bsdName (0x2603) for AppleUSBEthernet 2012-01-22 01:34:36.626 system_profiler[47377:903] Found bsdName (0x2c03) for AppleUSBEthernet Ethernet Cards: Apple USB Ethernet Adaptor: Type: USB BSD name: en4 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8a:f3:5a Product ID: 5122 Vendor ID: 1452 Apple USB Ethernet Adaptor: Type: USB BSD name: en3 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8b:26:c1 Product ID: 5122 Vendor ID: 1452 Marvell Yukon Gigabit Adapter 88E8053 Singleport Copper SA: Name: ethernet Type: Ethernet Controller Bus: PCI Vendor ID: 0x11ab Device ID: 0x4362 Subsystem Vendor ID: 0x11ab Subsystem ID: 0x5321 Revision ID: 0x0022 Link Width: x1 BSD name: en0 Kext name: AppleYukon2.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleYukon2.kext Version: 3.2.1b1 ethernet: Type: Ethernet Controller Bus: PCI Slot: AirPort Vendor ID: 0x168c Device ID: 0x001c Subsystem Vendor ID: 0x106b Subsystem ID: 0x0086 Revision ID: 0x0001 Link Width: x1 BSD name: en1 Kext name: AirPortAtheros21.kext Location: /System/Library/Extensions/IO80211Family.kext/Contents/PlugIns/AirPortAtheros21.kext Version: 425.14.7 root# ifconfig lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 gif0: flags=8010 mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863 mtu 1500 ether 00:16:cb:a8:7d:01 inet6 fe80::216:cbff:fea8:7d01%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.37 netmask 0xffffff00 broadcast 192.168.1.255 media: autoselect (1000baseT ) status: active en4: flags=8863 mtu 1500 ether 00:1f:f3:8a:f3:5a inet6 fe80::21f:f3ff:fe8a:f35a%en4 prefixlen 64 scopeid 0x5 inet 90.184.13.xxx netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active en3: flags=8863 mtu 1500 ether 00:1f:f3:8b:26:c1 inet6 fe80::21f:f3ff:fe8b:26c1%en3 prefixlen 64 scopeid 0x6 inet 90.184.12.yyy netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active fw0: flags=8863 mtu 2030 lladdr 00:17:f2:ff:fe:8e:e6:d2 media: autoselect status: inactive en1: flags=8863 mtu 1500 ether 00:17:f2:52:62:9e media: autoselect () status: inactive root# system_profiler SPHardwareDataType Hardware: Hardware Overview: Model Name: Mac mini Model Identifier: Macmini1,1 Processor Name: Intel Core Duo Processor Speed: 1,83 GHz Number Of Processors: 1 Total Number Of Cores: 2 L2 Cache: 2 MB Memory: 2 GB Bus Speed: 667 MHz Boot ROM Version: MM11.0055.B08 SMC Version (system): 1.3f4 root# system_profiler SPSoftwareDataType Software: System Software Overview: System Version: Mac OS X 10.6.8 (10K549) Kernel Version: Darwin 10.8.0 Boot Volume: X Boot Mode: Normal Computer Name: MiniWeb User Name: System Administrator (root) Secure Virtual Memory: Enabled 64-bit Kernel and Extensions: No Time since boot: 5 days 1:31 >On 19.2.2012, at 14.53, postmaster wrote: > >> Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: >sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM >mailbox WHERE username =3D 'abuse at mathiesen.info' AND domain =3D >'mathiesen.info' AND active =3D 1 > >Probably won't help with your problem, but it's better if you don't >fetch "mail" field. Just fetch the home and set: > >mail_location = maildir:~/ > >This avoids maildir autodetection. That seemed to fix the autodetection :-) > >> # OS: Darwin 10.8.0 i386 hfs > >With HFS+ it's a good idea to set maildir_copy_with_hardlinks = no OK - set to no as advised > >> a list "" * >> * LIST (\HasNoChildren) "." "INBOX" >> a OK List completed. >> a select inbox >> Connection closed by foreign host. >> 4604ds1-ynoe:~ root# > >You mean the session hangs here until it's killed? Yes ... It does some of what it's supposed to do : the dovecot-uidlist.lock is set email is copied from new to cur, but then nothing more seems to happen :-( 4604ds1-ynoe:mail root# ls -l mathiesen.info/macintosh\@mathiesen.info/ total 40 drwx------ 2 5000 5000 4352 Feb 21 09:58 cur -rw------- 1 5000 5000 8119 Feb 21 10:28 dovecot-uidlist -rw------- 1 5000 5000 32 Feb 21 10:28 dovecot-uidlist.lock -rw------- 1 5000 5000 8 Feb 21 10:28 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 Feb 21 10:28 dovecot-uidvalidity.4f4363ac -rw------- 1 5000 5000 40 Feb 21 10:28 dovecot.index.log drwxr-xr-x 2 5000 5000 68 Feb 19 11:18 dovecot.rawlog drwx------ 2 5000 5000 68 Feb 21 09:58 new drwx------ 2 5000 5000 68 Feb 21 09:11 tmp 4604ds1-ynoe:mail root# >Try attaching gdb into it: > >gdb -p >bt full 4604ds1-ynoe:~ root# ps -axj | grep dovecot root 40655 1 40655 4d84750 0 Ss ?? 0:00.02 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 40657 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/anvil root 40658 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/log root 40660 40655 40655 4d84750 0 S ?? 0:00.06 dovecot/config _dovecot 40662 40655 40655 4d84750 0 S ?? 0:00.02 dovecot/auth root 40663 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/auth -w 5000 40677 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40689 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40803 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 root 40618 362 40618 5680af8 1 S+ s001 0:00.01 tail -f dovecot.log root 40952 396 40951 4d85d40 2 R+ s003 0:00.00 grep dovecot 4604ds1-ynoe:~ root# gdb -p 40803 GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin". /private/var/root/40803: No such file or directory Attaching to process 40803. Reading symbols for shared libraries . done Reading symbols for shared libraries ...... done 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001f35 in login_client_connected () No symbol table info available. #22 0x000eb1a8 in master_login_auth_finish () No symbol table info available. #23 0x000eae80 in master_login_auth_callback () No symbol table info available. #24 0x000ebb3a in master_login_auth_input () No symbol table info available. #25 0x001001a7 in io_loop_call_io () No symbol table info available. #26 0x00101492 in io_loop_handler_run () No symbol table info available. #27 0x00100148 in io_loop_run () No symbol table info available. #28 0x000ed25a in master_service_run () No symbol table info available. #29 0x00001df8 in main () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from process 40803. 4604ds1-ynoe:~ root# > >or alternatively try: > >gdb --args /usr/local/libexec/dovecot/imap -u user at domain >run >a select inbox > >bt full 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/: mode=0700 gid=-1 a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001d71 in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From tss at iki.fi Tue Feb 21 11:58:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:58:41 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818321.11500.236.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Is there some new > lower limit for username length? Oh, I think it's because AB is uppercased. Either lowercase it "ab" or set this to empty: auth_username_format = I hadn't thought of this being a problem before.. Hm. Perhaps it's not a common problem, so I'll just change the Upgrading text to mention this possibility also.. From tss at iki.fi Tue Feb 21 12:08:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 12:08:06 +0200 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F436AC3.1040800@mathiesen.info> References: <4F436AC3.1040800@mathiesen.info> Message-ID: <1329818886.11500.239.camel@innu> On Tue, 2012-02-21 at 10:58 +0100, postmaster wrote: > > #0 0x90003afa in mach_msg_trap () > No symbol table info available. > #1 0x90004267 in mach_msg () > No symbol table info available. > #2 0x9001b431 in vproc_mig_look_up2 () > No symbol table info available. > #3 0x9001b26a in bootstrap_look_up2 () > No symbol table info available. > #4 0x9001b0e4 in bootstrap_look_up () > No symbol table info available. > #5 0x9001af30 in notify_register_check () > No symbol table info available. > #6 0x9001ada7 in notify_register_tz () > No symbol table info available. > #7 0x9001a621 in tzsetwall_basic () > No symbol table info available. > #8 0x90041903 in localtime () > No symbol table info available. All of the above functions are OSX's internal functions in libc. > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. From postmaster at mathiesen.info Tue Feb 21 12:11:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:11:26 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436DCE.5050001@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 arrrrgh -- used pop3 instead of imap :-( 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/imap -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/imap -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as postmaster at mathiesen.info a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x00092a2f in mail_index_update_day_headers () No symbol table info available. #10 0x0008ec95 in mail_index_transaction_finish () No symbol table info available. #11 0x0008d80a in mail_index_transaction_commit_v () No symbol table info available. #12 0x0008dacf in mail_index_transaction_commit_full () No symbol table info available. #13 0x0008db18 in mail_index_transaction_commit () No symbol table info available. #14 0x00096434 in mail_index_sync_commit () No symbol table info available. #15 0x00066f02 in maildir_sync_index_finish () No symbol table info available. #16 0x00068376 in maildir_sync_context () No symbol table info available. #17 0x000686f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000295fd in mailbox_sync () No symbol table info available. #19 0x00007833 in cmd_select_full () No symbol table info available. #20 0x00007d49 in cmd_select () No symbol table info available. #21 0x00009f63 in client_command_input () No symbol table info available. #22 0x0000a0c2 in client_command_input () No symbol table info available. #23 0x0000a1d3 in client_handle_input () No symbol table info available. #24 0x000095bd in client_input () No symbol table info available. #25 0x001171a7 in io_loop_call_io () No symbol table info available. #26 0x00118492 in io_loop_handler_run () No symbol table info available. #27 0x00117148 in io_loop_run () No symbol table info available. #28 0x0010425a in master_service_run () No symbol table info available. #29 0x00013c5a in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From postmaster at mathiesen.info Tue Feb 21 12:20:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:20:26 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436FEA.8000606@mathiesen.info> From: Timo Sirainen > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. OK - Thanks for your help :-) I'll take it up on the macports mailing lists then ! -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From ngoonee.talk at gmail.com Tue Feb 21 12:22:39 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 18:22:39 +0800 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: <1329818321.11500.236.camel@innu> References: <1329818321.11500.236.camel@innu> Message-ID: On Tue, Feb 21, 2012 at 5:58 PM, Timo Sirainen wrote: > On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: >> Hi, previously I could use the following in my passwd-file:- >> >> AB:{plain}password >> >> I'm not sure when it stopped working, its not an account I access with >> any regularity, but today it would not work (2.1.0, 64-bit Arch >> Linux), with an 'unknown user' error in the logs. Is there some new >> lower limit for username length? > > Oh, I think it's because AB is uppercased. Either lowercase it "ab" or > set this to empty: > > auth_username_format = > > I hadn't thought of this being a problem before.. Hm. Perhaps it's not a > common problem, so I'll just change the Upgrading text to mention this > possibility also.. > Yes, its probably the upper-casing. No problem, its just that it used to work and now doesn't, I've already changed the username (to something longer which isn't all uppercase) anyway. Probably SHOULD be mentioned briefly somewhere. Thanks much. From busseniu at in.tum.de Tue Feb 21 12:23:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 11:23:03 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F437087.3000407@in.tum.de> Hi, On 15.02.2012 04:46, Timo Sirainen wrote: > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. sorry to bother you again, but I think there is a problem with this patch: If a maildir contains several copies of the same message all in the same folder, dsync will not deduplicate them. While IMAP cannot directly create copies of a message in the same folder, it does still happen if you copy (or move) a message back and forth between two folders. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 13:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 13:04:25 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F437087.3000407@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> Message-ID: <1329822265.11500.254.camel@innu> On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: > Hi, > > On 15.02.2012 04:46, Timo Sirainen wrote: > > > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. > > sorry to bother you again, but I think there is a problem with this patch: > > If a maildir contains several copies of the same message all in the same > folder, dsync will not deduplicate them. Correct. I nearly finished implementing this also, but then I thought it just makes the code unnecessarily complex for no good reason. > While IMAP cannot directly create copies of a message in the same > folder, It can: SELECT INBOX, COPY 1 INBOX > it does still happen if you copy (or move) a message back and > forth between two folders. Is it common enough to be an actual problem? From joshua at hybrid.pl Tue Feb 21 13:25:51 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 21 Feb 2012 12:25:51 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: <7304de1adf6749ec476ef30957bb931c@hybrid.pl> W dniu 2012-02-21 02:13, Timo Sirainen napisa?(a): > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: >> By the way: what might have caused such a warning? >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u >> joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp >> is different than current time (1329464622 vs 1329464672): >> /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't > happen unless remote storage server's clock and local server's clock > aren't synced. OK, that's the case - /var/mail is stored on remote NFS. I'll check if I have time properly set, but if so - I'll probably have to live with it (I have no influence on NFS server). Greetings, -- Jacek Osiecki From busseniu at in.tum.de Tue Feb 21 13:55:05 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Tue, 21 Feb 2012 12:55:05 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <1329822265.11500.254.camel@innu> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> Message-ID: <4F438619.70405@in.tum.de> On 21.02.2012 12:04, Timo Sirainen wrote: > On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: >> Hi, >> >> On 15.02.2012 04:46, Timo Sirainen wrote: >>> >>> It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. >> >> sorry to bother you again, but I think there is a problem with this patch: >> >> If a maildir contains several copies of the same message all in the same >> folder, dsync will not deduplicate them. > > Correct. I nearly finished implementing this also, but then I thought it > just makes the code unnecessarily complex for no good reason. > >> While IMAP cannot directly create copies of a message in the same >> folder, > > It can: SELECT INBOX, COPY 1 INBOX Oh, mea culpa :) > >> it does still happen if you copy (or move) a message back and >> forth between two folders. > > Is it common enough to be an actual problem? Actually we have some mailboxes with massively duplicated messages in the same folder. Of course I cannot tell how common it is in general. I could imagine that some people routinely copy all INBOX messages into archive folders and do not check whether the archive already contains these messages. Apart from the waste of space, I was wondering: Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 14:15:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:15:33 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F438619.70405@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: On 21.2.2012, at 13.55, Christoph Bu?enius wrote: > Apart from the waste of space, I was wondering: > Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? No corruption. And they might even become deduplicated if you do doveadm force-resync + purge. From campbell at cnpapers.com Tue Feb 21 14:19:33 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:19:33 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F42C212.5090509@uvm.edu> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> Message-ID: <4F438BD5.4020107@cnpapers.com> On 2/20/2012 4:58 PM, Jim Lawson wrote: > On 2/20/12 3:36 PM, Steve Campbell wrote: >> >> Thanks for that input. I still think I'm missing something since I >> too used the compatibility link that you pointed to. Only thing is >> that proceeding those namespaces, I used the first example of: >> >> namespace { >> type = private >> separator = / >> prefix = "#mbox/" >> location = mbox:~/mail:INBOX=/var/mail/%u >> inbox = yes >> hidden = yes >> list = no >> } >> namespace { >> type = private >> separator = / >> prefix = >> location = maildir:~/mail >> } >> >> This seemed to get the ball rolling so that users could access >> anything at all. I still have the problem of client imap folders >> being different from webmail imap folders. That's pretty much why I'm >> thinking of using mbox as the INBOX and all other imap folders in the >> ~/mail directory in maildir format. > You ought to be able to get webmail to see the same folders as the > imap clients. > >> >> Should I remove the first two namespaces, in your opinion? Right now, >> the whole thing is kind of fragile. >> > > From your added namespaces, it looks like you are trying to support > mbox and maildir and ~/mail at the same time. Do you already have > maildir folders to support? If not, I would try to get things working > well with mbox first before I started a conversion to maildir. I > would also ask why you're thinking of moving to maildir. Yes, there > are caching benefits, but when you add the Dovecot indexing on top of > mbox, it's pretty much a wash. If you are using file-level backup, > rather than some sort of snapshot technology, maildir will be much, > much slower to back up. Your system will spend all its time walking > directories, opening and closing files. If you don't have many users > to worry about it might be OK, but make sure it's worth it. A lot of > sites went to maildir in the 1998-2004 era and have regretted the > decision as their systems get overloaded with files and they can't > back them up. No, I've got all mbox formats. Previous posts probably suggested that I wanted to move to maildir, but all of the replies I've received have convinced me that I do not want to do that. The folders in ~ and ~/mail are mboxes, so I need to see what damage I've done with a maildir namespace. The maildir reference could be part of the problems I'm seeing. I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. > >> Horde/Imp updates are probably out of the question until I can get a >> server to install the upgrade on. > For Dovecot and IMP both, you should set up an alternate server to > test out your config changes on before you put them into production. > If you are running on a bare metal single server, set up another > Dovecot instance on a different set of ports (I commonly use 20143 > (imap), 20993 (imaps), 20110 (pop3)) which you can fiddle with > freely. Once you are satisfied with the result in your various > clients, put it into production. You can do the same with Horde/IMP > by putting an installation in a different location on your webserver. > > Jim I've got a second server that is totally independent of the one I'm messing with. I've done the horde/imp alternate port/config. It all went well. The downside is that I didn't realize those secondary folders in ~/mail were being hidden. Kinda late to switch back, but the secondary server supports another domain, so I have it to test with. Again, the damage I've caused to those secondary imap folders needs to be determined to get this fixed properly. Most users are seing the secondary folders and can use them. It's just those users who have folders in ~ that are seeing problems as far as I can tell. Thanks steve > > From campbell at cnpapers.com Tue Feb 21 14:27:35 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:27:35 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <4F438DB7.5020001@cnpapers.com> On 2/20/2012 7:25 PM, Joseph Tam wrote: > > Steve Campbell wrote: > >> The more I read about all of this, the more I'm thinking about moving >> to maildir format. My switchover this weekend is full of holes due to >> the way user's imap folders were laid out. Some had folders in their >> home directory and others might have folders in their /home/mail >> directory. > > I had the same problem, and handled it more or less the same way as > the later poster (Jim Lawson ) handled his site. > I retained use of mbox format as well. > > I planned the migration like this > > - scan user home directories for mailboxes (especially "Trash", > "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. > > This was piped into a script that Email'd users about the changes > that was going to happen, what they would expect to see, and a > FAQ on how to set up a mail client correctly. Here, I'm not sure what should be done. The users with the secondary folders that are not in ~/mail can't seem to get the client configured. > > - during the cutover, > > - mailboxes left on the home directory were moved to > ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid > name collision for users that had a mixture of correct > and blank prefixes. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot is ignoring this and I'm not sure what it's looking at to determine the imap folders. > (I think I deleted "Trash", "Junk", etc. anyways). > > - Namespace aliasing was used so that prefixes "", "mail", > "~/mail", and whatever darn fool settings my users used, > would map to the same directory. > > - .subscription files were moved into the mail folder > (don't have to edit prefixes since the aliasing > will take care of that). And when would these .subscription files be created? The first time the folders would be accessed, or when? This seems to be part of my fix that I'm getting lost on. > > - depending on what POP3 client you used (I used > qpopper), you may need to configure > > pop3_reuse_xuidl = yes > > to avoid a massive re-downloading from POP3 clients > after cutover. > > - the mail clients I control centrally (e.g. webmail, > public server mail clients, etc). shouldn't need updating > since they ought to have been set up properly in the first > place. > > - after cutover, a second notification was sent for users that > didn't move their mailboxes the first time around, and was it > done for them during migration. > > And of course, test like crazy and watch the logs like a hawk. > > Other gotcha's: > > - your setup is fairly close to mine, so you may also run into > the problem of user having mailboxes with group ownership that > users are not part of (for example, group "mail" for INBOX set > by your LDA or personal mailboxes with groups the user is no > longer a member of) *and* with group permission not mode=0. Sendmail sort of requires the "mail" group, does it not? I'll take a look and see if all users are part of this or not. A crazy solution would be in order here? > > You'll have to treat these (set mode=0, or change the group to > something the user is part of), or the dovecot index creation > will fail and they won't be able to access their mailboxes. > > I think my migration went pretty smoothly. Less than a handful of > wazzup' > Email problem reports. Except for those users with the different folder locations, it seems that all is going pretty well. Maybe they're just not notifying me yet, though. > > Joseph Tam > > Thanks steve From tss at iki.fi Tue Feb 21 14:28:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:28:30 +0200 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F438BD5.4020107@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> <4F438BD5.4020107@cnpapers.com> Message-ID: <56BC47FD-F1EB-4294-A25C-5205E30AAA36@iki.fi> On 21.2.2012, at 14.19, Steve Campbell wrote: > I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. With new enough Dovecot version (2.0, maybe 1.2) you can add :SUBSCRIPTIONS=.mailboxlist to mail_location. From dag at newtech.fi Tue Feb 21 14:56:01 2012 From: dag at newtech.fi (Dag Nygren) Date: Tue, 21 Feb 2012 14:56:01 +0200 Subject: [Dovecot] Where did the message end up? Message-ID: <12808509.iPEzrCISOc@dag> Hi all! Just migrated out UW-IMAP installation to Dovecot and so far so good. Had small teething problems though which left me with one question. The scenario was that I have a set of public folders that are only needed by the main company group (gid = company). Then there are some external users as well, not directly in the company group. I set up the public mailbox with permissions 660, owner me and group "company". That is the externals didn't have access to the mail folder. At this point all looked fine, but later there was an email to an external person and this was passed on to deliver by procmail. Procmail logs says all was OK, but there is an entry about the public folder not accessible in the Dovecot log. The email is nowhere to be found. My question: Where is the delivered email? Best Dag From alessio at skye.it Tue Feb 21 15:29:53 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 14:29:53 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <1329815785.11500.224.camel@innu> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> <1329815785.11500.224.camel@innu> Message-ID: <4F439C51.3090907@skye.it> Il 21/02/2012 10:16, Timo Sirainen ha scritto: > On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > >> I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" >> and SASL for postfix's SMTP-Auth, like explained ih this howto: >> http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL >> >> Authentications over IMAP (local dovecot -> remote dovecot) works fine, >> and it's great. Also smtp-auth on postfix (over SASL) works fine. But in >> the dovecot log I found this error: >> >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): >> child 30506 killed with signal 11 (core dumps disabled) > > http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. > > Patch applied, after rebuilding all works fine. Thanks Timo -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From lists at wildgooses.com Tue Feb 21 16:33:24 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 21 Feb 2012 14:33:24 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213194356.GB3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> Message-ID: <4F43AB34.9040907@wildgooses.com> On 13/02/2012 19:43, Jan-Frode Myklebust wrote: > On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: >> Out of curiosity, are you running dovecot locally on those webmail >> servers as well, or is it talking to remote dovecot servers? > The webmail servers are talking with dovecot director servers which in > turn are talking with the backend dovecot servers. Each service running > on different servers. > > Webmail-servers -> director-servers -> backend-servers > I think the original question was still sensible. In your case it seems like the ping times are identical between: webmail -> imap-proxy webmail -> imap server I think your results show that a proxy has little (or negative) benefit in this situation, but it seems feasible that a proxy could eliminate several RTT trips in the event that the proxy is "closer" than the imap server? This might happen if say the imap server is in a different datacenter (webmail on an office server machine?) I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). However, using imap-proxy has the benefit of "clustering" logins a little and this makes log files a little easier to understand in the face of users with desktop mail clients plus webmail users. Possibly this idea useful to someone else... Thanks for measuring this! Ed W From crohmann at netcologne.de Tue Feb 21 17:05:39 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 21 Feb 2012 16:05:39 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? Message-ID: <4F43B2C3.3080006@netcologne.de> Hey dovecot-users, clients that speak telnet based protocols like http, smtp usually encode IDN (International Domain Names) containing UTF-8 characters into ACE (ASCII Compatible Encoding) punycode. I am wondering what the case was with managesieve (RFC 5804). For example if I was using a username like "user@???.example.com" with the AUTHENTICATE command. Would that need to be encoded into ACE or be left in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal Syntax" suggests that quoted or literal UTF-8 is correct. Thanks Christian From florob at babelmonkeys.de Tue Feb 21 18:11:28 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 21 Feb 2012 17:11:28 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? In-Reply-To: <4F43B2C3.3080006@netcologne.de> References: <4F43B2C3.3080006@netcologne.de> Message-ID: <4F43C230.6000408@babelmonkeys.de> Am 21.02.2012 16:05, schrieb Christian Rohmann: > Hey dovecot-users, > > clients that speak telnet based protocols like http, smtp usually encode > IDN (International Domain Names) containing UTF-8 characters into ACE > (ASCII Compatible Encoding) punycode. > > I am wondering what the case was with managesieve (RFC 5804). For > example if I was using a username like "user@???.example.com" with the > AUTHENTICATE command. Would that need to be encoded into ACE or be left > in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal > Syntax" suggests that quoted or literal UTF-8 is correct. > Hy, what you're asking for is the correct format of the simple username. This is in fact deployment specific. There is in general no requirement to include the domain at all. The only requirement for to fulfil is that it must not contain any codepoints prohibited by the SASLPrep stringprep profile. As such both the ACE and the IDN variant are valid. Which one to choose, or whether to use only "user" depends on local policies. There might be some implementation specifics in dovecot/pigeonhole that I'm not aware of though. Regards, Florian From busseniu at in.tum.de Tue Feb 21 18:36:18 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 17:36:18 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: <4F43C802.7010702@in.tum.de> On 21.02.2012 13:15, Timo Sirainen wrote: > > And they might even become deduplicated if you do doveadm force-resync + purge. > I hadn't tried that yet. Thanks for the hint, this is probably all we need. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From trashcan at odo.in-berlin.de Tue Feb 21 19:29:12 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 18:29:12 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> Message-ID: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Hi -- On 21.02.2012, at 02:02, Timo Sirainen wrote: > On 18.2.2012, at 20.26, Michael Grimm wrote: >> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb Hmm, now I cannot clients any longer: dovecot: master: Dovecot v20120221 (7d0d76df719f+) starting up dovecot: imap-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, mpid=65444, TLS dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test dovecot: imap(test): Error: Invalid user settings. Refer to server log for more information. [... repeated for every user clients are trying to connect ...] I switched back immediately (production server), thus I didn't test if dsync might run now. Here's the relevant part of my config: mail_home = /var/mail/.homedirs/%n mail_location = mdbox:/var/mail/.maildirs/%n User's individual mail storage formats are fetched from userdb (sqlite3), which normally are mdbox, except for test which is sdbox. Any idea what to test next? Thanks and regards, Michael From tss at iki.fi Tue Feb 21 22:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:36:32 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> On 21.2.2012, at 16.33, Ed W wrote: > I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: void postlogout_init(struct module *module) { } void postlogout_deinit(void) { system("/usr/local/bin/dovecot-postlogout.sh"); } Add a few missing #includes and compile and enable for imap/pop3 and that should be it. From tss at iki.fi Tue Feb 21 22:52:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:52:14 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Message-ID: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Hi, On 21.2.2012, at 19.29, Michael Grimm wrote: >>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb > > Hmm, now I cannot clients any longer: > > dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. From trashcan at odo.in-berlin.de Tue Feb 21 23:48:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 22:48:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: Hi -- On 21.02.2012, at 21:52, Timo Sirainen wrote: > On 21.2.2012, at 19.29, Michael Grimm wrote: >>>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >>> >>> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb >> >> Hmm, now I cannot connect clients any longer: >> >> dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test > > Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. Thanks, that omits the first and second error, and I believe that you solved this issue. Now, I can run my old command as used for 2.0.x and ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test ... a "newer" call like ... vmail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test ... without complaining about namespaces as listed above. But, now I do end with a message like ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test usage: dsync [-C ] [-m ] [-u ] [-frRv] mirror | [@] | dsync-local(test): Error: read() from worker server failed: EOF ... irrespectively if I use the old "dsync" or the new "doveadm sync". Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Weird. I must have missed something. I'll back to 2.0.18 for the time being. Any hints are highly welcome ... Thanks for your efforts, and thanks for dovecot! Regards, Michael From tss at iki.fi Tue Feb 21 23:57:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 23:57:21 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> On 21.2.2012, at 23.48, Michael Grimm wrote: > But, now I do end with a message like ... > > vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test > usage: dsync [-C ] [-m ] [-u ] [-frRv] > mirror | [@] | > dsync-local(test): Error: read() from worker server failed: EOF > > ... irrespectively if I use the old "dsync" or the new "doveadm sync". > > Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Added it there. From trashcan at odo.in-berlin.de Wed Feb 22 00:03:52 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 23:03:52 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 22:57, Timo Sirainen wrote: > On 21.2.2012, at 23.48, Michael Grimm wrote: >> But, now I do end with a message like ... >> >> vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test >> usage: dsync [-C ] [-m ] [-u ] [-frRv] >> mirror | [@] | >> dsync-local(test): Error: read() from worker server failed: EOF >> >> ... irrespectively if I use the old "dsync" or the new "doveadm sync". >> >> Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, > > Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). Ok. Then I will try both servers upgraded to 2.1 within the next days. Thanks for that valuable info that I really missed, sorry. >> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. > > Added it there. Great. And thanks again, Michael From as at 3a.pl Wed Feb 22 00:46:48 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 21 Feb 2012 23:46:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy Message-ID: <4F441ED8.20908@3a.pl> Hi, In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Each of this emails has the same Message-ID. Is there a way to detect those duplicates and discard them when saving copies into boss account? There was a discussion on the list a while ago (Feb 10/11, 2011) mentioning this problem, but with no clear solution. We are using Dovecot 2.0.15. Mail is delivered via LMTP (MDA/postfix is on separate machine then MAA/dovecot). -- Adam Szpakowski From tss at iki.fi Wed Feb 22 00:57:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 00:57:17 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F441ED8.20908@3a.pl> References: <4F441ED8.20908@3a.pl> Message-ID: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> On 22.2.2012, at 0.46, Adam Szpakowski wrote: > In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). > There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? From as at 3a.pl Wed Feb 22 01:02:23 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:02:23 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> Message-ID: <4F44227F.9030502@3a.pl> On 21.02.2012 23:57, Timo Sirainen wrote: > On 22.2.2012, at 0.46, Adam Szpakowski wrote: >> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. > Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. We are using dedicated sieve_before script for each user based on %u variable. -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:09:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:09:35 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F44227F.9030502@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> Message-ID: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> On 22.2.2012, at 1.02, Adam Szpakowski wrote: > On 21.02.2012 23:57, Timo Sirainen wrote: >> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? > Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. > We are using dedicated sieve_before script for each user based on %u variable. Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: fileinto :copy :x-deduplicate "boss"; Anyway, probably not going to be implemented anytime soon. From as at 3a.pl Wed Feb 22 01:15:30 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:15:30 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> Message-ID: <4F442592.608@3a.pl> On 22.02.2012 00:09, Timo Sirainen wrote: > On 22.2.2012, at 1.02, Adam Szpakowski wrote: >> On 21.02.2012 23:57, Timo Sirainen wrote: >>> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >>> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? >> Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. >> We are using dedicated sieve_before script for each user based on %u variable. > Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. > > I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: > > fileinto :copy :x-deduplicate "boss"; > > Anyway, probably not going to be implemented anytime soon. Maybe there is a way to use a procmail with something like this: :0 Wh: msgid.lock | formail -D 8192 .msgid.cache But is there a safe way to use it together with sieve? Using Pigeonhole Sieve Pipe Plugin? -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:32:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:32:56 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <12808509.iPEzrCISOc@dag> References: <12808509.iPEzrCISOc@dag> Message-ID: On 21.2.2012, at 14.56, Dag Nygren wrote: > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > Had small teething problems though which left me with one question. > > The scenario was that I have a set of public folders that are only needed by > the main company group (gid = company). Then there are some external users as > well, not directly in the company group. So this is in a public namespace? > I set up the public mailbox with permissions 660, owner me and group > "company". That is the externals didn't have access to the mail folder. > > At this point all looked fine, but later there was an email to an external > person and this was passed on to deliver by procmail. Procmail logs says all > was OK, but there is an entry about the public folder not accessible in the > Dovecot log. The email is nowhere to be found. Exact error message would be helpful. Was the error from deliver, or imap/pop3? If it was from deliver, it probably returned temporary failure. > Where is the delivered email? Have you checked what happens if deliver tempfails (e.g. make procmail call tempfail.sh script that just exits with 75), does procmail also correctly tempfail and tell MTA to retry the delivery later? From jtam.home at gmail.com Wed Feb 22 02:04:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 21 Feb 2012 16:04:30 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell writes: > > This was piped into a script that Email'd users about the changes > > that was going to happen, what they would expect to see, and a > > FAQ on how to set up a mail client correctly. > > Here, I'm not sure what should be done. The users with the secondary > folders that are not in ~/mail can't seem to get the client configured. "Great artists steal". Google around for a good set of instructions on how to set up IMAP prefix's for various mail clients, and refer your users to that. I also referred them to our webmail (with the correct mail prefix) as a reference to a list of mailboxes they ought to see. If they don't match, then that's a symptom of a bad prefix. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > is ignoring this and I'm not sure what it's looking at to determine the > imap folders. > > > - .subscription files were moved into the mail folder > > (don't have to edit prefixes since the aliasing > > will take care of that). > > And when would these .subscription files be created? The first time the > folders would be accessed, or when? This seems to be part of my fix that > I'm getting lost on. Sorry, as Timo intimated, dovecot uses .subscription, which can be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't have to edit it if you set up namespace aliasing. Just move it from the top-level into your user's mail directory during cutover: (if you use dovecot's default subscription filename) mv $USER/.mailboxlist $USER/mail/.subscription If your IMAP client is still ignoring subscriptions, it usually means the client was configured that way. > > - your setup is fairly close to mine, so you may also run into > > the problem of user having mailboxes with group ownership that > > users are not part of (for example, group "mail" for INBOX set > > by your LDA or personal mailboxes with groups the user is no > > longer a member of) *and* with group permission not mode=0. > > Sendmail sort of requires the "mail" group, does it not? I'll take a > look and see if all users are part of this or not. A crazy solution > would be in order here? I think sendmail works fine if you set your mail spool mode=1777 (like /tmp), but have each inbox mode=0600. This setting will persist. It's sendmail LDA mail.local that's the problem: it autocreates new inboxes with permissions user:mail/660 and unless the user is in group "mail", chaos will ensue. You can maybe do a cron script or some other hack, or better yet, use dovecot LDA, which sets up the perms correctly (and keeps indexes updated). A one time "chmod go-rwx" during cutover is all that is required then. The log entry that points this problem out looks like dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) failed: Operation not permitted (egid=5678(goodgroup), group based on /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not owner Joseph Tam From campbell at cnpapers.com Wed Feb 22 04:29:31 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 21:29:31 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <1329877771.4f44530b83411@perdition.cnpapers.net> Quoting Joseph Tam : > > Steve Campbell writes: > > > > This was piped into a script that Email'd users about the changes > > > that was going to happen, what they would expect to see, and a > > > FAQ on how to set up a mail client correctly. > > > > Here, I'm not sure what should be done. The users with the secondary > > folders that are not in ~/mail can't seem to get the client configured. > > "Great artists steal". Google around for a good set of instructions > on how to set up IMAP prefix's for various mail clients, and refer > your users to that. This conversion to dovecot has been a real learning experience for me. The old stuff that came with Centos 3 just worked out of the box. I never realized how much could be modified. It was always there, I just never needed it. > > I also referred them to our webmail (with the correct mail prefix) as a > reference to a list of mailboxes they ought to see. If they don't match, > then that's a symptom of a bad prefix. Our web mail, Imp, seems to be doing fine now that all of the mboxes have been moved to ~/mail. > > > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > > is ignoring this and I'm not sure what it's looking at to determine the > > imap folders. > > > > > - .subscription files were moved into the mail folder > > > (don't have to edit prefixes since the aliasing > > > will take care of that). > > > > And when would these .subscription files be created? The first time the > > folders would be accessed, or when? This seems to be part of my fix that > > I'm getting lost on. > > Sorry, as Timo intimated, dovecot uses .subscription, which can > be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't > have to edit it if you set up namespace aliasing. Just move it from > the top-level into your user's mail directory during cutover: > > (if you use dovecot's default subscription filename) > mv $USER/.mailboxlist $USER/mail/.subscription > > If your IMAP client is still ignoring subscriptions, it usually > means the client was configured that way. > > > > - your setup is fairly close to mine, so you may also run into > > > the problem of user having mailboxes with group ownership that > > > users are not part of (for example, group "mail" for INBOX set > > > by your LDA or personal mailboxes with groups the user is no > > > longer a member of) *and* with group permission not mode=0. > > > > Sendmail sort of requires the "mail" group, does it not? I'll take a > > look and see if all users are part of this or not. A crazy solution > > would be in order here? > > I think sendmail works fine if you set your mail spool mode=1777 (like > /tmp), but have each inbox mode=0600. This setting will persist. > > It's sendmail LDA mail.local that's the problem: it autocreates new > inboxes with permissions user:mail/660 and unless the user is in group > "mail", chaos will ensue. > So far, all things seem OK in this area. No complaints are being forwarded. > You can maybe do a cron script or some other hack, or better yet, > use dovecot LDA, which sets up the perms correctly (and keeps indexes > updated). A one time "chmod go-rwx" during cutover is all that is > required then. > > The log entry that points this problem out looks like > > dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) > failed: Operation not permitted (egid=5678(goodgroup), group based on > /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) > dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not > owner > > Joseph Tam For now, it looks like the only problem I'm seeing is that one user who uses Outlook Express that has a pop account for one email account and an imap account for another email account is having problems with the "Sent" folder. When she sends an email using the imap account, the "Sent" copy goes into the pop account's sent item folder. When using webmail, the sent copy goes into the sent-mail file as it's supposed to. I'm guessing for now that that google you suggested might point me to a fix for that. Thanks all for all the great help. Shared accounts are next, so wish me luck. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From dag at newtech.fi Wed Feb 22 09:34:27 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 09:34:27 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> Message-ID: <1651793.SOPdjt8lSW@eseries> onsdag 22 februari 2012 01:32:56 skrev Timo Sirainen: > On 21.2.2012, at 14.56, Dag Nygren wrote: > > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > > > Had small teething problems though which left me with one question. > > > > The scenario was that I have a set of public folders that are only > > needed by the main company group (gid = company). Then there are some > > external users as well, not directly in the company group. > > So this is in a public namespace? Yep, it was. The definition: namespace { type = public separator = / prefix = MailingLists/ location = mdbox:/usr/local/var/mail/MailingLists inbox = no subscriptions = no # v1.1+ } > > I set up the public mailbox with permissions 660, owner me and group > > "company". That is the externals didn't have access to the mail folder. > > > > At this point all looked fine, but later there was an email to an > > external person and this was passed on to deliver by procmail. Procmail > > logs says all was OK, but there is an entry about the public folder not > > accessible in the Dovecot log. The email is nowhere to be found. > > Exact error message would be helpful. Was the error from deliver, or > imap/pop3? If it was from deliver, it probably returned temporary failure. The log parts with final names and groups changed.. Deliver error: Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization failed: Namespace 'MailingLists/': stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: /usr/local/var/mail/MailingLists, euid is not dir owner) Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer to server log for more information. Procmail log: >From mailing-1341698764359 at mailpv.net Tue Feb 21 10:02:24 2012 Subject: =?CP1250?Q?P=E4=E4si=E4issuunnitelmat?= Folder: deliver -d extuser -m inbox 56979 > > Where is the delivered email? > > Have you checked what happens if deliver tempfails (e.g. make procmail call > tempfail.sh script that just exits with 75), does procmail also correctly > tempfail and tell MTA to retry the delivery later? No, haven't done that. yet. And I don't think it will pass on the failure code. Procmail settings are all from the HOWTO:s though. The .forward file looks like this: "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" And the applicable rule in .procmailrc: :0 w { :0 | $DELIVERYAGENT -d $LOGNAME -m inbox } Have to point out that this case wasn't important, but I wouldn't like the system to loose an important mail later. From tss at iki.fi Wed Feb 22 09:43:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 09:43:58 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1651793.SOPdjt8lSW@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: On 22.2.2012, at 9.34, Dag Nygren wrote: > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization > failed: Namespace 'MailingLists/': > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied > (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > /usr/local/var/mail/MailingLists, euid is not dir owner) > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer > to server log for more information. Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually v2.1.0 is the first release where you can easily configure Dovecot to simply skip the namespace when it doesn't have permission to access it: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. >>> Where is the delivered email? >> >> Have you checked what happens if deliver tempfails (e.g. make procmail call >> tempfail.sh script that just exits with 75), does procmail also correctly >> tempfail and tell MTA to retry the delivery later? > > No, haven't done that. yet. And I don't think it will pass on the failure > code. Procmail settings are all from the HOWTO:s though. > > The .forward file looks like this: > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > And the applicable rule in .procmailrc: > :0 w > { > :0 > | $DELIVERYAGENT -d $LOGNAME -m inbox > } > > Have to point out that this case wasn't important, but I wouldn't like the > system to loose an important mail later. You should definitely make sure that procmail won't lose any mails if Dovecot LDA returns tempfailure, since they do sometimes happen. You could also get rid of procmail completely by moving to sieve. From dag at newtech.fi Wed Feb 22 10:03:36 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 10:03:36 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: <1958645.10kxPAq8Bz@eseries> > On 22.2.2012, at 9.34, Dag Nygren wrote: > > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: > > Initialization failed: Namespace 'MailingLists/': > > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission > > denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > > /usr/local/var/mail/MailingLists, euid is not dir owner) > > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. > > Refer to server log for more information. > > Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually > v2.1.0 is the first release where you can easily configure Dovecot to > simply skip the namespace when it doesn't have permission to access it: > > + Added ignore_on_failure setting for namespaces. If namespace > initialization fails with this enabled (e.g. permission denied), > the namespace is silently skipped for the user. Ok. Many thanks for answering this. I am using FC14 and dovecot 2.0.4 at the moment so I don't have that option yet. It would be nice BTW if dovecot added the exit status it used in the log if it fails? > >>> Where is the delivered email? > >> > >> Have you checked what happens if deliver tempfails (e.g. make procmail > >> call tempfail.sh script that just exits with 75), does procmail also > >> correctly tempfail and tell MTA to retry the delivery later? > > > > No, haven't done that. yet. And I don't think it will pass on the > > failure > > code. Procmail settings are all from the HOWTO:s though. > > > > The .forward file looks like this: > > > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > > > And the applicable rule in .procmailrc: > > :0 w > > > > { > > > > :0 > > : > > | $DELIVERYAGENT -d $LOGNAME -m inbox > > > > } > > > > Have to point out that this case wasn't important, but I wouldn't like > > the system to loose an important mail later. > > You should definitely make sure that procmail won't lose any mails if > Dovecot LDA returns tempfailure, since they do sometimes happen. You could > also get rid of procmail completely by moving to sieve. Will check the path out. Any way of forcing dovecot to return tempfail for the tests? Preferrably only for one user :-) Not familiar with sieve yet, but will for sure look into it. Thanks again Dag From tss at iki.fi Wed Feb 22 10:18:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 10:18:00 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1958645.10kxPAq8Bz@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> <1958645.10kxPAq8Bz@eseries> Message-ID: On 22.2.2012, at 10.03, Dag Nygren wrote: > It would be nice BTW if dovecot added > the exit status it used in the log if it fails? This wouldn't be very useful normally. If you see "Fatal" prefixed error from dovecot-lda, it's nearly always tempfail (75). If it crashes, the error is different, but then it wouldn't be able to log the error anyway. > Will check the path out. Any way of forcing dovecot to return tempfail for the > tests? Preferrably only for one user :-) Run a script instead, something like: #!/bin/sh if [ $USER = "foo" ]; then exit 75 else exec .../dovecot-lda $@ fi From janfrode at tanso.net Wed Feb 22 10:25:36 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 09:25:36 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <20120222082536.GA8678@dibs.tanso.net> On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: > > I think the original question was still sensible. In your case it > seems like the ping times are identical between: > webmail -> imap-proxy > webmail -> imap server > > I think your results show that a proxy has little (or negative) > benefit in this situation, but it seems feasible that a proxy could > eliminate several RTT trips in the event that the proxy is "closer" > than the imap server? This might happen if say the imap server is > in a different datacenter (webmail on an office server machine?) The webmail/imapproxy were actually running in a different datacenter to the dovecot director/backend servers, but only about 20KM away. Ping tests: webmail->director: rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms director->backend: rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms webmail->localhost: rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms -jf From lists at wildgooses.com Wed Feb 22 11:31:55 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:31:55 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120222082536.GA8678@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> Message-ID: <4F44B60B.4000105@wildgooses.com> On 22/02/2012 08:25, Jan-Frode Myklebust wrote: > On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: >> I think the original question was still sensible. In your case it >> seems like the ping times are identical between: >> webmail -> imap-proxy >> webmail -> imap server >> >> I think your results show that a proxy has little (or negative) >> benefit in this situation, but it seems feasible that a proxy could >> eliminate several RTT trips in the event that the proxy is "closer" >> than the imap server? This might happen if say the imap server is >> in a different datacenter (webmail on an office server machine?) > The webmail/imapproxy were actually running in a different datacenter to > the dovecot director/backend servers, but only about 20KM away. > > Ping tests: > > webmail->director: > > rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms > > director->backend: > > rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms > > webmail->localhost: > > rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms > > > -jf Hmm, not sure I understand the original numbers then? It seems intuitive that the proxy installed locally could save you 2x RTT increment, which is about 0.8ms in your case. So I might expect the proxy to reduce rendering times by around 1.6ms simply because it reduces the number of round trips to login? Kind of curious why that's not achieved..? Cheers Ed W From lists at wildgooses.com Wed Feb 22 11:38:47 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:38:47 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> Message-ID: <4F44B7A7.6030303@wildgooses.com> On 21/02/2012 20:36, Timo Sirainen wrote: > On 21.2.2012, at 16.33, Ed W wrote: > >> I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). > You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: > > void postlogout_init(struct module *module) { } > void postlogout_deinit(void) { > system("/usr/local/bin/dovecot-postlogout.sh"); > } > > Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? Cheers Ed W From janfrode at tanso.net Wed Feb 22 11:48:46 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 10:48:46 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B60B.4000105@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> <4F44B60B.4000105@wildgooses.com> Message-ID: <20120222094846.GA10314@dibs.tanso.net> On Wed, Feb 22, 2012 at 09:31:55AM +0000, Ed W wrote: > > It seems intuitive that the proxy installed locally could save you > 2x RTT increment, which is about 0.8ms in your case. So I might > expect the proxy to reduce rendering times by around 1.6ms simply > because it reduces the number of round trips to login? Kind of > curious why that's not achieved..? Each http-request can probably trigger several IMAP requests. Maybe these work better in parallel directly to dovecot, than serialized (?) trough the imapproxy ? No idea if that's what's happening... or maybe the imapproxy just adds more overhead than the 2xRTT + imap logins it's supposed to save us ? -jf From tss at iki.fi Wed Feb 22 21:49:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 21:49:54 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B7A7.6030303@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> Message-ID: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> On 22.2.2012, at 11.38, Ed W wrote: >> void postlogout_init(struct module *module) { } >> void postlogout_deinit(void) { >> system("/usr/local/bin/dovecot-postlogout.sh"); >> } >> >> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. > > Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. From clube03 at gmail.com Wed Feb 22 22:49:01 2012 From: clube03 at gmail.com (Gustavo) Date: Wed, 22 Feb 2012 18:49:01 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <4F37A377.80707@Media-Brokers.com> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> <4F37A377.80707@Media-Brokers.com> Message-ID: Ok, I do some changes and now I can send mails but not recieve mail. On log I get this message: Feb 22 20:02:04 localhost dovecot: deliver(teste at clubedolimite.com.br): Fatal: setgid(12(man)) failed with euid=150(vmail), gid=8(mail), egid=8(mail): Operation not permitted (This binary should probably be called with process group set to 12(man) instead of 8(mail)) Where is the problem now? Thanks! On Sun, Feb 12, 2012 at 9:33 AM, Charles Marcus wrote: > On 2012-02-11 1:46 PM, Gustavo wrote: > >> Ok, I study the wikki and the dovecot works. But when I try to send a mail >> using squirrel, I get this error on logs: >> >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from >> localhost.localdomain[127.0.0.**1] >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: >> xsasl_cyrus_server_get_**mechanism_list: no applicable SASL mechanisms >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL >> authentication >> mechanisms >> > > Sorry, but this has nothing to do with dovecot, unless your intention is > to use dovecot-sasl with your postfix... it appears you are using > cyrus-sasl and postfix on your smtp server, that is where your problem lies. > > Easiest would be to configure postfix to use dovecot-sasl: > > http://wiki2.dovecot.org/**HowTo/PostfixAndDovecotSASL > > -- > > Best regards, > > Charles > -- Gustavo - clube03 at gmail.com From lists at wildgooses.com Thu Feb 23 00:54:53 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 22:54:53 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> Message-ID: <4F45723D.2090003@wildgooses.com> On 22/02/2012 19:49, Timo Sirainen wrote: > On 22.2.2012, at 11.38, Ed W wrote: > >>> void postlogout_init(struct module *module) { } >>> void postlogout_deinit(void) { >>> system("/usr/local/bin/dovecot-postlogout.sh"); >>> } >>> >>> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. >> Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. >> >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. Do you think you could keep something similar on your low priority backlog? Clearly parsing log files or hacking the code is possible, but I think the interest in the login scripting shows there is general interest, and having a full log of logon/logoff/bytes is clearly interesting to more minority users? >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. I think SWIG is for wrapping dovecot's api into the scripting language? (ie you could call dovecot methods from say perl/python/etc). What I had in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot normally calls a plugin method it will also run any [lua] scripts. I'm sure you know how to use google, but just so we are on the same page, top hit (below) from google shows how straight forward this is (lua has been built to be extremely fast and easy to embed, ie it's not an arbitrary choice) http://heavycoder.com/tutorials/lua_embed.php Cheers Ed W From tss at iki.fi Thu Feb 23 01:41:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 01:41:18 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F45723D.2090003@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> Message-ID: <1329954078.11500.266.camel@innu> On Wed, 2012-02-22 at 22:54 +0000, Ed W wrote: > >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > > Do you think you could keep something similar on your low priority > backlog? Clearly parsing log files or hacking the code is possible, but > I think the interest in the login scripting shows there is general > interest, and having a full log of logon/logoff/bytes is clearly > interesting to more minority users? What do you need the statistics for? I could make imap_client and pop3_client support some virtual methods, like user.destroy() initially, which would be enough for your use. I guess I could add that for v2.2. > >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. > > I think SWIG is for wrapping dovecot's api into the scripting language? > (ie you could call dovecot methods from say perl/python/etc). What I had > in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot > normally calls a plugin method it will also run any [lua] scripts. Yes, but I think SWIG can do that too. You'll need Dovecot's API implemented for scripting language if you want to do anything useful in the embedded script. That's the biggest job actually. Actually embeddeding some scripting language to Dovecot processes would be simple, they just couldn't really do anything useful. Also the nice thing about generically implementing Dovecot's APIs means that they could be used to build independent Dovecot applications, not just plugins. > I'm sure you know how to use google, but just so we are on the same > page, top hit (below) from google shows how straight forward this is > (lua has been built to be extremely fast and easy to embed, ie it's not > an arbitrary choice) > http://heavycoder.com/tutorials/lua_embed.php I've heard LUA being a commonly used embedded language, but I'd prefer to instead support several very widely used languages, such as Perl/Python. From lists at wildgooses.com Thu Feb 23 01:56:28 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 23:56:28 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <4F4580AC.5030600@wildgooses.com> On 22/02/2012 23:41, Timo Sirainen wrote: > I've heard LUA being a commonly used embedded language, but I'd prefer > to instead support several very widely used languages, such as > Perl/Python. I'm a perl/ruby fan myself, but I would still recommend a good look at lua (or python) simply because they seem to be performant, easy to use, and on the surface seem to have had some thought about making them "embeddable". My new favourite editor "Sublime Text 2" has python as it's scripting language. Lua has been used for some big name games amongst other things. Perl has some memory management issues if you leave it long running, also writing XS code looks ok on the surface, but is an exercise in hair pulling in practice.... Ruby is a beautiful language, but unsure of how easy to embed and speed + memory management is an unknown (for high performance applications) I think it has potential though. I think a lot of the current plugins on the website could easily be rewritten, likely without performance concerns, using a scripting based plugin system. I could see that some other big picture pieces could potentially benefit also Thanks for considering it Ed W From tss at iki.fi Thu Feb 23 02:56:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 02:56:29 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <1329958589.11500.273.camel@innu> On Wed, 2012-02-22 at 23:56 +0000, Ed W wrote: > On 22/02/2012 23:41, Timo Sirainen wrote: > > I've heard LUA being a commonly used embedded language, but I'd prefer > > to instead support several very widely used languages, such as > > Perl/Python. > > I'm a perl/ruby fan myself, but I would still recommend a good look at > lua (or python) simply because they seem to be performant, easy to use, > and on the surface seem to have had some thought about making them > "embeddable". SWIG appears to generate Lua bindings just as well. But yes, I noticed Lua 5.1 has a feature called FFI that makes calling C functions quite easy (but then again, SWIG is supposed to do the same thing). > Perl has some memory management issues if you leave it long running, > also writing XS code looks ok on the surface, but is an exercise in hair > pulling in practice.... I've written a Perl plugin to irssi before directly with XS, and that's my main motivation for using some kind of an easy generator this time. It's way too much trouble to write any glue functions, especially when it works for only a single scripting language. From stephan at rename-it.nl Thu Feb 23 03:15:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 02:15:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F442592.608@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> Message-ID: <4F459344.5020407@rename-it.nl> On 2/22/2012 12:15 AM, Adam Szpakowski wrote: > On 22.02.2012 00:09, Timo Sirainen wrote: >> Well, it would be possible to build a doveadm script that deletes the >> duplicates after delivery, but currently there's no implementation to >> avoid delivering duplicate Message-IDs in the first place. >> >> I don't really like such a Message-ID-based deduplication feature >> enabled by default, but something like this could be nice: >> >> fileinto :copy :x-deduplicate "boss"; >> >> Anyway, probably not going to be implemented anytime soon. > Maybe there is a way to use a procmail with something like this: > > :0 Wh: msgid.lock > | formail -D 8192 .msgid.cache > > But is there a safe way to use it together with sieve? Using > Pigeonhole Sieve Pipe Plugin? > There are a few options: * You can use Procmail as primary delivery agent and invoke dovecot-lda/sieve from within Procmail once Procmail has determined that it is not a duplicate. * Invoke procmail from Sieve using the pipe extension (i.e. the other way around). This has the disadvantage that Procmail will have to take care of final delivery, meaning the Dovecot indexes are not updated. * For Pigeonhole v0.3 there is the possibility to "filter" the message through Procmail using the sieve_extprograms plugin, but I haven't actually tested something like that. * I've just created an alternative that implements something similar to the Procmail code you posted above, but from within Sieve itself. It is a custom language extension called vnd.dovecot.duplicate and it adds the "duplicate" test. This test keeps track of which Message-IDs it has seen before in earlier deliveries and yields a true result if the message was seen before, e.g.: require "vnd.dovecot.duplicate"; if duplicate { discard; } Read the specification for details ("name" argument is not yet implemented): http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate/raw-file/4b1dbda4d3fc/doc/rfc/spec-bosch-sieve-duplicate.txt The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate This plugin is only a few hours old, experimental, and largely untested, so test it thoroughly before considering to use this. Read the INSTALL file for compile and installation instructions. Comments are welcome. Regards, Stephan. From atoth at atoth.sote.hu Thu Feb 23 07:20:26 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 06:20:26 +0100 Subject: [Dovecot] How to achieve proper privilege separation? Message-ID: I'm using a simple mbox config with regular Unix users and pam authentication. I'm also using grsecurity. That's why I see what dovecot does in which users' name. As times goes by and new versions are coming I can frustratedly see, that more and more tasks are performed as root. Why? When I used 1.x series of Dovecot, imap process started in the name of the user whose mbox was accessed. Now I can see, that nearly every task is performed by root. Why? It even tampers with the mail directories of each user as root instead of the user as it was usual long before. Please let me know how should I change the config to make dovecot stop using root privileges and revert it back to the old behavior. Thanks for your help: Dw. Here's my current config: dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = host.name last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at host.name protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = References: Message-ID: On 23.2.2012, at 7.20, T?th Attila wrote: > I'm using a simple mbox config with regular Unix users and pam > authentication. > > I'm also using grsecurity. That's why I see what dovecot does in which > users' name. As times goes by and new versions are coming I can > frustratedly see, that more and more tasks are performed as root. Why? Less tasks should be running as root now. The master process code is a lot smaller. > When I used 1.x series of Dovecot, imap process started in the name of the > user whose mbox was accessed. > Now I can see, that nearly every task is performed by root. Why? It even > tampers with the mail directories of each user as root instead of the user > as it was usual long before. The imap process starts as root, does a userdb lookup and then drops privileges to that user. It worked this way before too, only the userdb lookup code was done by master process. From atoth at atoth.sote.hu Thu Feb 23 10:03:37 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 09:03:37 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: Message-ID: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap accesses both the inbox and the mail directories of the user as root. Moreover, it creates the lock file as root. I can see no process running as the user. How could I teach dovecot to start the imap process as the user. What configuration options I should blame? Thx: Dw. dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster@ protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = On 23.2.2012, at 7.20, T?th Attila wrote: > >> I'm using a simple mbox config with regular Unix users and pam >> authentication. >> >> I'm also using grsecurity. That's why I see what dovecot does in which >> users' name. As times goes by and new versions are coming I can >> frustratedly see, that more and more tasks are performed as root. Why? > > Less tasks should be running as root now. The master process code is a lot > smaller. > >> When I used 1.x series of Dovecot, imap process started in the name of >> the >> user whose mbox was accessed. >> Now I can see, that nearly every task is performed by root. Why? It even >> tampers with the mail directories of each user as root instead of the >> user >> as it was usual long before. > > The imap process starts as root, does a userdb lookup and then drops > privileges to that user. It worked this way before too, only the userdb > lookup code was done by master process. > > From tss at iki.fi Thu Feb 23 10:15:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:15:12 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Message-ID: <1329984912.11500.277.camel@innu> On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: > Unfortunately I can see, that in my case /usr/libexec/dovecot/imap > accesses both the inbox and the mail directories of the user as root. > Moreover, it creates the lock file as root. I can see no process running > as the user. > > How could I teach dovecot to start the imap process as the user. What > configuration options I should blame? Well, that's strange. There shouldn't be any way for you to make imap access mails as root, even if you wanted to do that. If you log in as root, it'll fail with: Error: user root: Invalid settings in userdb: userdb returned 0 as uid Fatal: Invalid user settings. Refer to server log for more information. If there's a bug and it just somehow manages to get through that check, it fails with: Fatal: We couldn't drop root privileges So.. I'm not really sure what could be wrong. It makes me think maybe Gentoo's hardening features somehow mess this up, but I can't really think of how that could either. Set auth_debug=yes and mail_debug=yes. What does it log when logging in? From tss at iki.fi Thu Feb 23 10:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:54:24 +0200 Subject: [Dovecot] v2.1.1 released Message-ID: <1329987264.11500.282.camel@innu> http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz.sig v2.1.0 had a couple of bugs, but overall people have been pretty quiet about it. I'm hoping v2.1.2 won't need to be released for many more weeks. :) BTW. v2.2 branch has also been opened in hg. Not much stuff in there yet though. + dsync: If message with same GUID is saved multiple times in session, copy it instead of re-saving. - acl plugin + autocreated mailboxes crashed when listing mailboxes - doveadm force-resync: Don't skip autocreated mailboxes (especially INBOX). - If process runs out of fds, stop listening for new connections only temporarily, not permanently (avoids hangs with process_limit=1 services) - auth: passdb imap crashed for non-login authentication (e.g. smtp). From campbell at cnpapers.com Thu Feb 23 17:16:34 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:16:34 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process Message-ID: <4F465852.6030506@cnpapers.com> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. Now I see where messages are in my maillog of the type: auth: pam_unix(dovecot:auth) : authentication failure ........ Why is dovecot involved in my smtp processes and how do I fix this. I've got some very mad users. The 10-auth.conf file is pretty much stock except for allowing plain text logins. steve campbell From as at 3a.pl Thu Feb 23 17:30:12 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:30:12 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465B84.6090300@3a.pl> On 23.02.2012 16:16, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail and > sasl2 stuff seems to be proper, but the user can't use the system on > port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix this. > I've got some very mad users. The 10-auth.conf file is pretty much > stock except for allowing plain text logins. Dovecot can be used as authentication source. It has its own implementation of SASL library which can be used instead of Cyrus SASL implementation. In Dovecot2 the config for auth is in 10-master.conf, service "auth". Please see: for dovecot 1.x: http://wiki.dovecot.org/Sasl for dovecot 2.x: http://wiki2.dovecot.org/Sasl on the postfix side: http://www.postfix.org/SASL_README.html#server_dovecot -- Adam Szpakowski From as at 3a.pl Thu Feb 23 17:37:46 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:37:46 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F465D4A.1070005@3a.pl> On 23.02.2012 02:15, Stephan Bosch wrote: > On 2/22/2012 12:15 AM, Adam Szpakowski wrote: >> Maybe there is a way to use a procmail with something like this: >> >> :0 Wh: msgid.lock >> | formail -D 8192 .msgid.cache >> >> But is there a safe way to use it together with sieve? Using >> Pigeonhole Sieve Pipe Plugin? >> > There are a few options: > > * You can use Procmail as primary delivery agent and invoke > dovecot-lda/sieve from within Procmail once Procmail has determined > that it is not a duplicate. This is great option but if I understand correctly not when Postfix is using LMTP to talk to Dovecot. In this client of mine, the Postfix (MDA) and Dovecot (MAA) are in different physical locations and LMTP is tunnelled via SSH. > * I've just created an alternative that implements something similar > to the Procmail code you posted above, but from within Sieve itself. [cut] Great extension. I'll try and use it on one of the servers as soon as possible. Thanks for the work. BTW... There is another solution to my problem, which ends up to be implemented in this case. I'm using Postfix to BCC all mails on the domain level to the "archive" account. In this account I've set up all the logic which ends calling "redirect :copy". This solution eliminates copies from multiple To/CC. -- Adam Szpakowski From campbell at cnpapers.com Thu Feb 23 17:47:40 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:47:40 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465B84.6090300@3a.pl> References: <4F465852.6030506@cnpapers.com> <4F465B84.6090300@3a.pl> Message-ID: <4F465F9C.4080600@cnpapers.com> On 2/23/2012 10:30 AM, Adam Szpakowski wrote: > On 23.02.2012 16:16, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the system >> on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix this. >> I've got some very mad users. The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > Dovecot can be used as authentication source. It has its own > implementation of SASL library which can be used instead of Cyrus SASL > implementation. > > In Dovecot2 the config for auth is in 10-master.conf, service "auth". > > Please see: > for dovecot 1.x: http://wiki.dovecot.org/Sasl > for dovecot 2.x: http://wiki2.dovecot.org/Sasl > > on the postfix side: > http://www.postfix.org/SASL_README.html#server_dovecot > That's all great, but what about sendmail. What should my pam.d/dovecot file look like, what should my dovecot.conf file look like, all this just runs me around in circles. Thanks for the pointers, but I've been over that a million times and nothing seems to work. steve From CMarcus at Media-Brokers.com Thu Feb 23 17:47:25 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 10:47:25 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465F8D.2020402@Media-Brokers.com> On 2012-02-23 10:16 AM, Steve Campbell wrote: > Why is dovecot involved in my smtp processes Because you told it to? > and how do I fix this. Depends on what your intent is - what MTA you use - how it is configured - etc... > I've got some very mad users. Don't make changes to an existing and/or go live with a new system without properly testing? > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. This file is irrelevant unless you have your MTA configured to use dovecot-sasl... dovecot only *assists* an MTA in authenticating, it isn't an MTA. -- Best regards, Charles From stephan at rename-it.nl Thu Feb 23 18:14:57 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 17:14:57 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F465D4A.1070005@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> <4F465D4A.1070005@3a.pl> Message-ID: <4F466601.6000303@rename-it.nl> Op 2/23/2012 4:37 PM, Adam Szpakowski schreef: > On 23.02.2012 02:15, Stephan Bosch wrote: >> * I've just created an alternative that implements something similar >> to the Procmail code you posted above, but from within Sieve itself. > [cut] > > Great extension. I'll try and use it on one of the servers as soon as > possible. Thanks for the work. > > BTW... > There is another solution to my problem, which ends up to be > implemented in this case. > I'm using Postfix to BCC all mails on the domain level to the > "archive" account. In this account I've set up all the logic which > ends calling "redirect :copy". > This solution eliminates copies from multiple To/CC. Ok, please keep me uprised. Once it is tested a bit more I'll add it to the main Pigeonhole package. This plugin does not add any dependencies, so this is no problem whatsoever. Regards, Stephan. From campbell at cnpapers.com Thu Feb 23 18:15:21 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:15:21 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465F8D.2020402@Media-Brokers.com> References: <4F465852.6030506@cnpapers.com> <4F465F8D.2020402@Media-Brokers.com> Message-ID: <4F466619.1090103@cnpapers.com> On 2/23/2012 10:47 AM, Charles Marcus wrote: > On 2012-02-23 10:16 AM, Steve Campbell wrote: >> Why is dovecot involved in my smtp processes > > Because you told it to? Where did I tell it to do this. > >> and how do I fix this. > > Depends on what your intent is - what MTA you use - how it is > configured - etc... > Sendmail - pretty much default except I'm trying to use port submission (587). >> I've got some very mad users. > > Don't make changes to an existing and/or go live with a new system > without properly testing? > Unfortunately, switching over to a new server didn't show me most of the problems until after it had run a while. Dovecot is probably a great application, but the wiki sucks when you run sendmail, the help files are a little vague and run me around in circles. I postponed (actually move the server back to the old server) for a week to fix the problems I ran into upon first putting it into server. After fixing those, it was only later that the last batch of problems showed up. You right that I should have tested more, but that's not what I need to be hearing right now. Most of my conversions go without problems and I've been doing this for quite a few years (decades). >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > > This file is irrelevant unless you have your MTA configured to use > dovecot-sasl... dovecot only *assists* an MTA in authenticating, it > isn't an MTA. > My sendmail configs don't even list dovecot. steve From alexwbaule at gmail.com Thu Feb 23 18:19:30 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 23 Feb 2012 14:19:30 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin Message-ID: Hi Timo.... I Follow your instruction to look inside quota plugin to get the notification from dovecot that expunged a message, that's Ok. Follow the quota, it's use a array to keep all uids from a email expunged, to compare with uid that is got in notify_sync, that's OK too, but to me i need to keep uid and a char* with my body name, to erase it too. I made this: struct emexis_ids_x_uis{ uint32_t ids; char *uis_file; }; struct emexis_mailbox { union mailbox_module_context module_ctx; ARRAY_DEFINE(expunge_uids, struct emexis_ids_x_uis); }; ok, inside my expunged rewrite function i create the array like quota does, and append it to my array. struct emexis_ids_x_uis append_uis; append_uis.ids = _mail->uid; append_uis.uis_file = bodyFile; array_append(&ebox->expunge_uids, &append_uis, 1); But when i do a foreach in this Array, my ids is OK, but my uis_file has tha same value every time (the value is the lasted value inserted). Append to UID array (156) --> (/storage/gss/emexis/common/t/33/88/t33889eb1bccd28fd6643ebc3ecf548b12a1ad095) Append to UID array (157) --> (/storage/gss/emexis/common/t/e3/35/te335c20e978e12d37ad0a35ae009245cd34080c0) Append to UID array (158) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Calling Sync Expunged --> (156) Sync Expunged (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged Match (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (157 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (158 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Finish Calling Sync Expunged --> (156) Can i use a struct as a array Item ?? something is wrong with my array creation, append and etc ? Tks Timo !! From pgiessel at mac.com Thu Feb 23 18:23:28 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 07:23:28 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> > > I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. From kevingaspard at lavabit.com Thu Feb 23 18:26:45 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:26:45 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F4668C5.1020902@lavabit.com> Hi, I've a problem with Dovecot and Thunderbird. Actually I'm trying to setup a virtual host server mail with postfix + dovecot (2.0.9) on CentOS 6.2. This is my dovecot -n : http://pastebin.com/wZWAE5Cj (I guess the errors are not important, that's just saying there is some obsoletes values, I will fix that later until it's not inconvenient to setup correctly dovecot). I've a file to store password (/etc/dovecot/passwd) and when I'm trying to connect with telnet to my host on port 143 it works, after I can log myself with the command ?a login user at domain.tld passwd? and it works, this is the entire output of the telnet connection : http://pastebin.com/h10cQVqf but actually when I'm trying to connect with thunderbird I've an error about the password and the login, this is a screenshoot : http://images.koshie.fr/TB_problem.png and to finish, there is nothing into the log... The password and the user are corrects ! Of course, dovecot and postfix are started, this is the netstat -a | grep "dovecot" : http://pastebin.com/bwfsS4Tf If you need more informations tell me. PS : Excuse me for my bad english, also I'm new with dovecot. Cordially, GASPARD K?vin From rob0 at gmx.co.uk Thu Feb 23 18:33:40 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 10:33:40 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <20120223163340.GM24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail > and sasl2 stuff seems to be proper, but the user can't use the > system on port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix > this. I would question that these failures are in fact related to what Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it does not, therefore there is no way that Dovecot could possibly interfere with SMTP AUTH in Sendmail. > I've got some very mad users. And you are jumping to conclusions. I suggest that you take this matter to a Sendmail forum. When you do, provide all relevant configuration as well as complete logging to show the problem. No useful help is possible with what you posted here. > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From campbell at cnpapers.com Thu Feb 23 18:36:07 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:36:07 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> References: <4F465852.6030506@cnpapers.com> <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> Message-ID: <4F466AF7.5030009@cnpapers.com> On 2/23/2012 11:23 AM, Peter A. Giessel wrote: >> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. > My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. You're correct. This server is our smtp, imap, and pop server. All authentication is done there, supposedly through PAM, but dovecot seems to interfere with sendmail's smtp auth processes and overrides what typically was done in a non-interferred way with sasl. > From kevingaspard at lavabit.com Thu Feb 23 18:36:33 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:36:33 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F466B11.6020906@lavabit.com> I'm sorry, some one tell me my netstat (on IRC) is useless. Also my pastes will be destroyed in something like one day, that's stupid (that's originally for IRC), so this is the output of my : dovecot -n : http://pastebin.com/Tewcb7bm telnet : http://pastebin.com/6gY2puxF netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf Sorry for the error. From sm at resistor.net Thu Feb 23 18:34:46 2012 From: sm at resistor.net (SM) Date: Thu, 23 Feb 2012 08:34:46 -0800 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <6.2.5.6.2.20120223083039.09229478@resistor.net> Hi Steve, At 07:16 23-02-2012, Steve Campbell wrote: >I've been trying to get smtp auth set up for days. All my sendmail >and sasl2 stuff seems to be proper, but the user can't use the >system on port 587, which is where I require authorization. > >Now I see where messages are in my maillog of the type: > >auth: pam_unix(dovecot:auth) : authentication failure ........ > >Why is dovecot involved in my smtp processes and how do I fix this. >I've got some very If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Regards, -sm From campbell at cnpapers.com Thu Feb 23 19:10:20 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 12:10:20 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <20120223163340.GM24983@harrier.slackbuilds.org> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> Message-ID: <4F4672FC.1080802@cnpapers.com> On 2/23/2012 11:33 AM, /dev/rob0 wrote: > On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the >> system on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix >> this. > I would question that these failures are in fact related to what > Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it > does not, therefore there is no way that Dovecot could possibly > interfere with SMTP AUTH in Sendmail. Why is sendmail using Dovecot sasl when I have the regular sasl set up. If I turn off dovecot, telnet to 587, I get no failures in my logs. If I turn on dovecot and do the same, I get auth failures in my secure file. > >> I've got some very mad users. > And you are jumping to conclusions. I suggest that you take this > matter to a Sendmail forum. When you do, provide all relevant > configuration as well as complete logging to show the problem. No > useful help is possible with what you posted here. In other words, don't use sendmail if I use dovecot? > >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. I'm really having problems following the logic here. Seems that postfix and dovecot are the only way to go if I use alternate ports with smtp auth. Is that what everyone is implying? I'll try to see what sendmail guys are saying, but I don't think they'll provide much as long as it involves dovecot. Thanks for the help steve From pgiessel at mac.com Thu Feb 23 19:12:53 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 08:12:53 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <6.2.5.6.2.20120223083039.09229478@resistor.net> References: <4F465852.6030506@cnpapers.com> <6.2.5.6.2.20120223083039.09229478@resistor.net> Message-ID: <115F4A79-2861-4B1F-AB82-35B21F2C9091@mac.com> > > If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Not totally true. Typical cyrus sasl2 implementation used with sendmail uses an imap server for smtp authentication. I recall having to tweak the sasl2 configuration on my own sendmail server to talk correctly with dovecot. I don't recall the exact tweaks. I suggest reviewing the sasl2 implementation's documentation (the OP has not to my knowledge stated which sasl implementation he is using). It sounds like an sasl2 problem, not necessarily a dovecot problem. From keith1christian at gmail.com Thu Feb 23 19:31:49 2012 From: keith1christian at gmail.com (Keith Christian) Date: Thu, 23 Feb 2012 10:31:49 -0700 Subject: [Dovecot] Settings for authentication without an extra password file Message-ID: Hello, I'm new to dovecot, and, due to having an older version of an OS installed on a test server, I'll be using an older version of dovecot (dovecot-1.0-1.2) to provide IMAP services. On this wiki page (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) There are instructions for creating a separate dovecot password file: echo "$USER:{PLAIN}password" > passwd.dovecot sudo mv passwd.dovecot /etc So, it appears that two password files will be needed, one for dovecot and one for regular system logins. Is the statement above correct, or am I misreading something? Is there a setting in dovecot.conf which will authenticate IMAP using /etc/password and /etc/shadow, without a separate dovecot password file? ========Keith From rob0 at gmx.co.uk Thu Feb 23 19:41:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 11:41:55 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <20120223174155.GO24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 12:10:20PM -0500, Steve Campbell wrote: > On 2/23/2012 11:33 AM, /dev/rob0 wrote: > >On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > >>Why is dovecot involved in my smtp processes and how do I fix > >>this. > >I would question that these failures are in fact related to what > >Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK > >it does not, therefore there is no way that Dovecot could possibly > >interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set > up. Fortunately it seems that Peter has identified the issue: Cyrus SASL being configured to use IMAP for authentication. snip > In other words, don't use sendmail if I use dovecot? I didn't say that at all, and did not mean to imply it. > I'm really having problems following the logic here. Seems that > postfix and dovecot are the only way to go if I use alternate ports > with smtp auth. Is that what everyone is implying? One thing I *did* say is that what you posted was inadequate to be able to provide real help. And it seems that your issue is only tangentially related to Dovecot. > I'll try to see what sendmail guys are saying, but I don't think > they'll provide much as long as it involves dovecot. As Peter said, consult the Cyrus SASL documentation. If your SASL will be using IMAP for authentication, you need to ensure that it does so correctly for your Dovecot IMAP. As an alternative, change how Cyrus SASL is configured. The usual suggestion for Sendmail users is to use the same data backend for Cyrus SASL and Dovecot. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Thu Feb 23 20:01:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 13:01:21 -0500 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird In-Reply-To: <4F466B11.6020906@lavabit.com> References: <4F466B11.6020906@lavabit.com> Message-ID: <4F467EF1.2030803@Media-Brokers.com> On 2012-02-23 11:36 AM, GASPARD K?vin wrote: > Also my pastes will be destroyed in something like one day, that's > stupid (that's originally for IRC), so this is the output of my : > > dovecot -n : http://pastebin.com/Tewcb7bm > telnet : http://pastebin.com/6gY2puxF > netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf > > Sorry for the error. Please don't do that... just post the output in the body of the email. For one it complicates things (people reading email from terminals will not be able to help you), and as well many people simply will not click URL shortened links like those in an email. There is no reason to not simply paste the contents into the email body. -- Best regards, Charles From atoth at atoth.sote.hu Thu Feb 23 20:53:28 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 19:53:28 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <767de6d20cc46bcbccef60bf1d21ab13.squirrel@atoth.sote.hu> http://dovecot.org/list/dovecot/2009-March/038435.html -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 18:10 id?pontban Steve Campbell ezt ?rta: > > > On 2/23/2012 11:33 AM, /dev/rob0 wrote: >> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >>> I've been trying to get smtp auth set up for days. All my sendmail >>> and sasl2 stuff seems to be proper, but the user can't use the >>> system on port 587, which is where I require authorization. >>> >>> Now I see where messages are in my maillog of the type: >>> >>> auth: pam_unix(dovecot:auth) : authentication failure ........ >>> >>> Why is dovecot involved in my smtp processes and how do I fix >>> this. >> I would question that these failures are in fact related to what >> Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it >> does not, therefore there is no way that Dovecot could possibly >> interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set up. > > If I turn off dovecot, telnet to 587, I get no failures in my logs. If I > turn on dovecot and do the same, I get auth failures in my secure file. >> >>> I've got some very mad users. >> And you are jumping to conclusions. I suggest that you take this >> matter to a Sendmail forum. When you do, provide all relevant >> configuration as well as complete logging to show the problem. No >> useful help is possible with what you posted here. > > In other words, don't use sendmail if I use dovecot? >> >>> The 10-auth.conf file is pretty much >>> stock except for allowing plain text logins. > > I'm really having problems following the logic here. Seems that postfix > and dovecot are the only way to go if I use alternate ports with smtp > auth. Is that what everyone is implying? > > I'll try to see what sendmail guys are saying, but I don't think they'll > provide much as long as it involves dovecot. > > Thanks for the help > > steve > From trashcan at odo.in-berlin.de Thu Feb 23 21:55:55 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Thu, 23 Feb 2012 20:55:55 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 23:03, Michael Grimm wrote: > On 21.02.2012, at 22:57, Timo Sirainen wrote: >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > Ok. Then I will try both servers upgraded to 2.1 within the next days. JFTR: It took me some time to find a working syntax ;-) My working 2.0.18 syntax threw the following error: vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(vmail): Fatal: Unknown print formatter: -u dsync-local(test): Error: read() from worker server failed: EOF Omitting '-f' or '-v -f' results in: vmail> dsync -v -u test ssh vmail at remote-host.tld dsync -v -u test dsync: illegal option -- u dsync-local(test): Error: read() from worker server failed: EOF Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test doveadm: illegal option -- v doveadm sync [-u |-A] [-S ] [-fR] [-m ] ok, following those hints: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm sync -u test ssh: Could not resolve hostname dsync-server: hostname nor servname provided, or not known dsync-local(test): Error: read() from worker server failed: EOF dsync-local(test): Error: read() from worker server failed: EOF Hmm, 'man doveadm' doesn't show 'sync', and 'man sync' shows old syntax. Luckily, I remembered finding an mail of yours in http://www.dovecot.org/list/dovecot/2012-January/062933.html : |>>> dsync mirror -> doveadm mirror |>>> dsync backup -> doveadm backup |>>> dsync server -> doveadm dsync-server (could be hidden from the doveadm commands list) | I did this now, with mirror -> sync. After some trial by error I finally found a working syntax: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test >>> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. >> >> Added it there. Therefore you might add a warning about different syntax as well? Please, don't get me wrong, I'm not complaining. This mail is JFTR, only. And, this mail might help others to save some time after an upgrade to 2.1 and those who mirror mailboxes sitting at different servers. Thanks and regards, Michael From atoth at atoth.sote.hu Thu Feb 23 21:56:39 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 20:56:39 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <1329984912.11500.277.camel@innu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> Message-ID: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> In the mean time I've upgraded to 2.1. I've enabled debug logging and logged in. I suspect that hardening features can be blamed for my problem. After booting a previous kernel the behavior was reverted. Here is what I got. As I can make it out it uses the proper user for the imap process according to the logs. Thx: Dw. Feb 23 20:49:39 atoth dovecot: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 23 20:50:12 atoth dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16584) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50264 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 3337879553 16584 1 0a36f4227122eb3d59466523e937b25b Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 3337879553 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16587, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:12 atoth dovecot: imap(atoth): Disconnected: Logged out in=44 out=747 Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16588) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50265 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 401211393 16588 1 59b6d569049f955f31991ac3cfb1f54c Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 401211393 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16589, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=42671 out=174898 Feb 23 20:50:14 atoth dovecot: auth: Debug: auth client connected (pid=16600) Feb 23 20:50:14 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50276 resp= Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:14 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:14 atoth dovecot: auth: Debug: master in: REQUEST 3933732865 16600 1 8382f23ff412178311e55bf74162e4cd Feb 23 20:50:14 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:14 atoth dovecot: auth: Debug: master out: USER 3933732865 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16601, secured Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=405 out=9240 -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 09:15 id?pontban Timo Sirainen ezt ?rta: > On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: >> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap >> accesses both the inbox and the mail directories of the user as root. >> Moreover, it creates the lock file as root. I can see no process running >> as the user. >> >> How could I teach dovecot to start the imap process as the user. What >> configuration options I should blame? > > Well, that's strange. There shouldn't be any way for you to make imap > access mails as root, even if you wanted to do that. If you log in as > root, it'll fail with: > > Error: user root: Invalid settings in userdb: userdb returned 0 as uid > Fatal: Invalid user settings. Refer to server log for more information. > > If there's a bug and it just somehow manages to get through that check, > it fails with: > > Fatal: We couldn't drop root privileges > > So.. I'm not really sure what could be wrong. It makes me think maybe > Gentoo's hardening features somehow mess this up, but I can't really > think of how that could either. > > Set auth_debug=yes and mail_debug=yes. What does it log when logging in? > > From marc at perkel.com Thu Feb 23 22:41:51 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 23 Feb 2012 12:41:51 -0800 Subject: [Dovecot] From Maildir back to smtp Message-ID: <4F46A48F.2040304@perkel.com> Not sure how this can be done. Right now I'm running a free backup MX record service. People point their high MX records to my servers and if they go down we store the email. When they come back up it's delivered. What I want to do is have a premium service that would allow them to view through IMAP the stored email. In order to do that I would have to deliver the email locally to a dovecot server - to a single account perhaps - and they can view/forward/delete etc. But - when their server comes back online I'd like to gather up all the delivered email stored in Maildir format and run it back into smtp for delivery. Has anyone even done something like this? Or is there an easier way that I'm missing? Thanks in advance From mailing at securitylabs.it Fri Feb 24 00:12:14 2012 From: mailing at securitylabs.it (mailing at securitylabs.it) Date: Thu, 23 Feb 2012 23:12:14 +0100 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46A48F.2040304@perkel.com> References: <4F46A48F.2040304@perkel.com> Message-ID: <4F46B9BE.3000801@securitylabs.it> Il 23/02/2012 21:41, Marc Perkel ha scritto: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > Hi, store how? In a queue with smtp server? > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. > If you store the messages in a queue you can configure your smtp (your MX) server to send a copy of all message it receive also to another mailbox, see always_bcc in posfix. From wilberth.perez at uady.mx Fri Feb 24 00:33:46 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Thu, 23 Feb 2012 16:33:46 -0600 Subject: [Dovecot] How to limit max number of connections for ip address Message-ID: <4F46BECA.3020208@uady.mx> Hi List: Does any one , knows if is possible configure dovecot for limit max number of connections for IP address? I would like to prevent future fork-bombing attacks for pop3 and imap login process in my email server. Our dovecot version is : 1.2.10 Thanks, Wilberth. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From ar-dovecotlist at acrconsulting.co.uk Fri Feb 24 01:03:34 2012 From: ar-dovecotlist at acrconsulting.co.uk (Andrew Richards) Date: 23 Feb 2012 23:03:34 +0000 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <201202232303.34310.ar-dovecotlist@acrconsulting.co.uk> On Thursday 23 February 2012 22:12:14 mailing at securitylabs.it wrote: > Il 23/02/2012 21:41, Marc Perkel ha scritto: > > Not sure how this can be done. > > > > Right now I'm running a free backup MX record service. People point > > their high MX records to my servers and if they go down we store the > > email. When they come back up it's delivered. > > Hi, store how? In a queue with smtp server? > > > What I want to do is have a premium service that would allow them to > > view through IMAP the stored email. In order to do that I would have > > to deliver the email locally to a dovecot server - to a single account > > perhaps - and they can view/forward/delete etc. > > > > But - when their server comes back online I'd like to gather up all > > the delivered email stored in Maildir format and run it back into smtp > > for delivery. Sounds like a job for serialmail, a sister package to qmail: Use serialmail to forward mail from a mailbox (in Maildir format which you have) to the primary MX when it's back up. I've provided a mailbox/forwarding secondary MX service for several clients using this method (esp. 'maildirsmtp' program from serialmail with cron). More info/help on the qmail or serialmail mailing lists; URL for serialmail is, http://cr.yp.to/serialmail.html (it includes man pages within the package) A couple of particular advantages of having secondary MX email in a mailbox as you suggest for forwarding: - Unlike mail in a queue, the messages don't expire, so if the primary MX takes forever to come back up that's fine (except for your disk storage) - Since the mail is in a mailbox you can access this with IMAP or POP3 as you suggest [until it's emptied when the primary MX returns] I've not considered running serialmail in an non-qmail environment, but it would probably still work for the purpose you have in mind if you're using a different MTA. > If you store the messages in a queue you can configure your smtp (your > MX) server to send a copy of all message it receive also to another > mailbox, see always_bcc in posfix. cheers, Andrew. From tss at iki.fi Fri Feb 24 01:21:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:21:44 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F46BECA.3020208@uady.mx> References: <4F46BECA.3020208@uady.mx> Message-ID: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> On 24.2.2012, at 0.33, Wilberth Perez wrote: > Does any one , knows if is possible configure dovecot for limit max > number of connections for IP address? > > I would like to prevent future fork-bombing attacks for pop3 and imap > login process in my email server. > > Our dovecot version is : 1.2.10 There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. From tss at iki.fi Fri Feb 24 01:26:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:26:08 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> Message-ID: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> On 23.2.2012, at 21.56, T?th Attila wrote: > In the mean time I've upgraded to 2.1. > I've enabled debug logging and logged in. > > I suspect that hardening features can be blamed for my problem. After > booting a previous kernel the behavior was reverted. OK. > Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, > gid=100, home=/home/atoth This says that the IMAP is running as UID 1000. The code that produces this is: i_debug("Effective uid=%s, gid=%s, home=%s", dec2str(geteuid()), dec2str(getegid()), home); So if the process is still creating files as root, the kernel is lying.. From tss at iki.fi Fri Feb 24 01:28:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:28:47 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: On 24.2.2012, at 1.26, Timo Sirainen wrote: > On 23.2.2012, at 21.56, T?th Attila wrote: > >> In the mean time I've upgraded to 2.1. >> I've enabled debug logging and logged in. >> >> I suspect that hardening features can be blamed for my problem. After >> booting a previous kernel the behavior was reverted. > > OK. BTW. It would be interesting to know if the attached patch causes it to fail with the hardening patches. -------------- next part -------------- A non-text attachment was scrubbed... Name: euid.diff Type: application/octet-stream Size: 532 bytes Desc: not available URL: From tss at iki.fi Fri Feb 24 01:32:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:32:06 +0200 Subject: [Dovecot] Settings for authentication without an extra password file In-Reply-To: References: Message-ID: <19807B92-11DB-4901-8909-CCFA6003FEB9@iki.fi> On 23.2.2012, at 19.31, Keith Christian wrote: > Hello, I'm new to dovecot, and, due to having an older version of an > OS installed on a test server, I'll be using an older version of > dovecot (dovecot-1.0-1.2) to provide IMAP services. I think Dovecot v2.x works mostly on the same OSes as v1.x. > On this wiki page > (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) > > There are instructions for creating a separate dovecot password file: > > echo "$USER:{PLAIN}password" > passwd.dovecot > sudo mv passwd.dovecot /etc > > So, it appears that two password files will be needed, one for dovecot > and one for regular system logins. Only if you want to. > Is there a setting in dovecot.conf which will authenticate IMAP using > /etc/password and /etc/shadow, without a separate dovecot password > file? Use passdb pam + userdb passwd (not passwd-file). See password + user databases under http://wiki.dovecot.org/Authentication From jtam.home at gmail.com Fri Feb 24 01:32:50 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 23 Feb 2012 15:32:50 -0800 (PST) Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: References: Message-ID: From: "Peter A. Giessel" : > Not totally true. Typical cyrus sasl2 implementation used with > sendmail uses an imap server for smtp authentication. I recall having > to tweak the sasl2 configuration on my own sendmail server to talk > correctly with dovecot. I don't recall the exact tweaks. Just to make it clear to the OP, I have a Dovecot/Sendmail/SASL working just fine (which is independent of dovecot). I suspect Peter hit the nail on the head here. Look at the contents of $SASL_INSTALL/lib/Sendmail.conf Mine is (only) pwcheck_method:saslauthd which I think looks up my name service backend via the login and plain plugins. I'm a little hazy on how this was set up, but if you are able to successfully login without dovecot, then it appears you have some sort of authentication stack that will try IMAP first, then fallback to your the other method. Joseph Tam From tss at iki.fi Fri Feb 24 01:34:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:34:48 +0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: References: Message-ID: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> On 23.2.2012, at 18.19, Alex Baule wrote: > ok, inside my expunged rewrite function i create the array like quota > does, and append it to my array. > > struct emexis_ids_x_uis append_uis; > append_uis.ids = _mail->uid; > append_uis.uis_file = bodyFile; .. > But when i do a foreach in this Array, my ids is OK, but my uis_file > has tha same value every time (the value is the lasted value > inserted). I guess the memory isn't permanently allocated for it, so you need to do: append_uis.uis_file = i_strdup(bodyFile); Note that you'll also need to later i_free() it to avoid leaking memory. From lists at wildgooses.com Fri Feb 24 01:41:37 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 23 Feb 2012 23:41:37 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <4F46CEB1.1040604@wildgooses.com> On 22/02/2012 23:56, Ed W wrote: > I think it has potential though. I think a lot of the current plugins > on the website could easily be rewritten, likely without performance > concerns, using a scripting based plugin system. I could see that > some other big picture pieces could potentially benefit also One interesting test case for such a scripting hooks solution might be login restrictions. There seem to be regular requests for the ability to setup arbitrarily complicated restrictions on users per IP, attempts per second, etc (and my logging interest is kind of related also). Not trying to bump the item up any todo lists, just trying to chuck in some concrete ideas for actually testing a specific implementation... I guess a substantially more performance orientated area that seems to get some interest would be various spam, expunge, delete ideas and the hooks needed for those. These seem much more tricky to implement a scripting hook and still stay performant. Again just ideas for real things people might want to do? Cheers Ed W From amk at krell.zikzak.de Fri Feb 24 03:22:47 2012 From: amk at krell.zikzak.de (Andreas M. Kirchwitz) Date: Fri, 24 Feb 2012 02:22:47 +0100 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build Message-ID: <20120224012247.GA6512@krell.zikzak.de> Hello Dovecot users! There seems to be a new dependency in some modules (eg, lib-storage, libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those modules didn't require OpenSSL, but 2.1 does. For the linking process the path to the OpenSSL library isn't specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't properly used as well.) Example for lib-storage: env SSL_CFLAGS="-I/usr/local/ssl/include" SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto -lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl --with-ssldir=/usr/local/${DOVECOTVER}/etc/dovecot/certs make gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-master -I../../src/lib-settings -I../../src/lib-mail -I../../src/lib-storage -DPKG_RUNDIR=\""/usr/local/Dovecot-2.1.1/var/run/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -MT master-connection.o -MD -MP -MF .deps/master-connection.Tpo -c -o master-connection.o master-connection.c mv -f .deps/master-connection.Tpo .deps/master-connection.Po /bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -no-undefined -Wl,--as-needed -o indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o ../../src/lib-storage/libdovecot-storage.la ../../src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -Wl,--as-needed -o .libs/indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o -Wl,--export-dynamic ../../src/lib-storage/.libs/libdovecot-storage.so ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/Dovecot-2.1.1/lib/dovecot /usr/bin/ld: warning: libcrypto.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libssl.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `OBJ_txt2nid' ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `SSL_set_ex_data' [...] # ldd src/lib*/.libs/*.so src/lib-dovecot/.libs/libdovecot.so: linux-gate.so.1 => (0x00979000) libdl.so.2 => /lib/libdl.so.2 (0x00c31000) librt.so.1 => /lib/librt.so.1 (0x009ed000) libc.so.6 => /lib/libc.so.6 (0x0054d000) /lib/ld-linux.so.2 (0x00c12000) libpthread.so.0 => /lib/libpthread.so.0 (0x008f4000) src/lib-lda/.libs/libdovecot-lda.so: linux-gate.so.1 => (0x0096d000) libdovecot-storage.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-storage/.libs/libdovecot-storage.so.0 (0x00ae4000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00110000) librt.so.1 => /lib/librt.so.1 (0x00189000) libc.so.6 => /lib/libc.so.6 (0x00192000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x0031c000) libpthread.so.0 => /lib/libpthread.so.0 (0x00ebc000) /lib/ld-linux.so.2 (0x003f8000) src/lib-sql/.libs/libdovecot-sql.so: linux-gate.so.1 => (0x00a95000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x002b6000) libdl.so.2 => /lib/libdl.so.2 (0x00ab8000) librt.so.1 => /lib/librt.so.1 (0x0072a000) libc.so.6 => /lib/libc.so.6 (0x008f7000) /lib/ld-linux.so.2 (0x00ebd000) libpthread.so.0 => /lib/libpthread.so.0 (0x00619000) src/lib-ssl-iostream/.libs/libdovecot-ssl.so: linux-gate.so.1 => (0x00da6000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x00df0000) librt.so.1 => /lib/librt.so.1 (0x00112000) libc.so.6 => /lib/libc.so.6 (0x00b5e000) /lib/ld-linux.so.2 (0x00e76000) libpthread.so.0 => /lib/libpthread.so.0 (0x005a6000) src/lib-storage/.libs/libdovecot-storage.so: linux-gate.so.1 => (0x006ed000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00c66000) libdl.so.2 => /lib/libdl.so.2 (0x00491000) librt.so.1 => /lib/librt.so.1 (0x00be2000) libc.so.6 => /lib/libc.so.6 (0x00110000) /lib/ld-linux.so.2 (0x0036b000) libpthread.so.0 => /lib/libpthread.so.0 (0x0029a000) Without SSL_CFLAGS and SSL_LIBS but just PKG_CONFIG_PATH, it also won't work. Help appreciated ... Andreas From alexwbaule at gmail.com Fri Feb 24 04:32:47 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 24 Feb 2012 00:32:47 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> References: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> Message-ID: Tks Timo... Em 23 de fevereiro de 2012 21:34, Timo Sirainen escreveu: > On 23.2.2012, at 18.19, Alex Baule wrote: > >> ok, inside my expunged rewrite function i create the array like quota >> does, and append it to my array. >> >> struct emexis_ids_x_uis append_uis; >> append_uis.ids = _mail->uid; >> append_uis.uis_file = bodyFile; > .. >> But when i do a foreach in this Array, my ids is OK, but my uis_file >> has tha same value every time (the value is the lasted value >> inserted). > > I guess the memory isn't permanently allocated for it, so you need to do: > > append_uis.uis_file = i_strdup(bodyFile); > > Note that you'll also need to later i_free() it to avoid leaking memory. > From CMarcus at Media-Brokers.com Fri Feb 24 14:18:43 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 07:18:43 -0500 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <4F478023.9080706@Media-Brokers.com> On 23/02/2012 21:41, Marc Perkel wrote: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. I'm thinking it would be possible to use dsync for this? Ideally you would have to have a way to maintain a duplicate of the AUTH database, so that users could use their current passwords for accessing it (this would obviously require the approval and cooperation of whoever you are providing this service to), but this way you could store each users mail individually and provide relatively painless access (since all each user would need to know is the backup host name and their username/password). You'd have to configure a script on your end to: 1. determine a reliable way to differentiate between a temp-fail situation on the primary (in which case you just act as a normal backup MX), or an extended outage (15 minutes? 1 hour? I'd probably let the customer decide this one), which would determine when you start actually taking delivery of their mail to the 'standby' server 2. monitor for when the primary comes back online 3. wait xx interval to make sure the primary stays up (again, I'd let the customer decide this, but with a sane minimum that you define) 4. once the primary stays up for the required interval, dsync a one way 'dump' of the stored mails, being sure to test for success - and this process would obviously have to take into account the possibility of the primary going down again in the middle of the dsync 5. mv successfully restored mails on the backup mx to a temp location for xx 'safety' interval 'just in case' 6. delete the backed-up/stored mail that has been successfully dsync'd after the safety interval has elapsed -- Best regards, Charles From troy at surety-networks.com Fri Feb 24 15:55:57 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 08:55:57 -0500 Subject: [Dovecot] Apple od/ad authentication problem Message-ID: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. OS X 10.7.3, behind a Cisco asa NAT. When trying to authenticate (ssl or not) via the webmail interface I get this error... NAT IP is the public ip I assigned. It doesn't work for an AD account or an account local to the Mac server. However it DOES work on the internal LAN. Feb 22 18:15:36 mail dovecot[3501]: auth: Error: od (username,natIP): verify plain: lookup failed for user: username Feb 22 18:15:51 mail dovecot[3501]: auth: Error: od[getpwnam_ext](username,natIP): No record for user I figured I'm missing something in the dovecot config. I have followed apple's kb articles for AD webmail authentication and kerberos, and that works internally. From przemek.orzechowski at makolab.pl Fri Feb 24 16:22:13 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q? Przemys=C5=82aw.Orzechowski ?=) Date: Fri, 24 Feb 2012 15:22:13 +0100 Subject: [Dovecot] iPhone and custom imap folders problem Message-ID: Hi Im using Dovecot 1.0.10 and everything seems work ok except one client He is accessing his mailbox using iPhone There are several custom folders created and mail is directed to them using procmail The problem is iPhone mail app only sees changes in INBOX and when new mail arrives to other folders it will not see it unlers user accesses this particular folder Is there some kind of workaround for this? Im quite sure its iPhones fault as all other email clients are workin perfectly fine. From jacob at alltechservices-ia.com Fri Feb 24 16:27:14 2012 From: jacob at alltechservices-ia.com (jacob at alltechservices-ia.com) Date: Fri, 24 Feb 2012 08:27:14 -0600 Subject: [Dovecot] Virtual Folders Message-ID: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Hello, I am trying to get virtual folders to work properly, and I can't seem to. I will be as detailed as possible to remove confusion. Thank you in advance. Goal: Essentially, a gmail clone, but one that I can make work on cPanel hosts with roundcube and squirrel mail, etc. By default, cpanel creates user mail dir structure of (maildir++) inbox, drafts, sent, junk, and trash (with no dovecot specific files until first login) I would like to replace all those folders with virtual representations with more capabilities. I would like to achieve the conversation view that gmail has pulled off so well. cPanel, stock, generates the following dovecot -n: (i am still not sure how it negotiates mail_location yet) # 1.2.17: /etc/dovecot.conf # OS: Linux 2.6.18-238.5.1.el5xen x86_64 CentOS release 5.7 (Final) ssl_cert_file: /etc/dovecot/ssl/dovecot.crt ssl_key_file: /etc/dovecot/ssl/dovecot.key ssl_cipher_list: ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login login_process_per_connection: no login_processes_count: 10 login_max_processes_count: 100 login_max_connections: 500 mail_max_userip_connections: 200 maildir_copy_preserve_filename: yes mail_plugins: quota imap_quota namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at example.com auth default: mechanisms: plain login cache_size: 2048 username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+% passdb: driver: checkpassword args: /usr/local/cpanel/bin/dovecot-auth userdb: driver: prefetch socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 plugin: quota: maildir quota_rule: Trash:ignore I started a test box with Debian running the same version of dovecot (1.2.x) and on that system I run into problems every time I make a virtual folder a root folder. I assume it is trying to look up a normal folder instead of the virtual ones and I am confusing it. So I came here because it would be really really neat if I could pull this off. Again, thank you in advance. -cornfeed From CMarcus at Media-Brokers.com Fri Feb 24 16:47:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 09:47:44 -0500 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47A310.9000308@Media-Brokers.com> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski wrote: > Im using Dovecot 1.0.10 Very old - an upgrade would likeley fix your problems... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From dg at dguhl.org Fri Feb 24 17:11:58 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 24 Feb 2012 16:11:58 +0100 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Message-ID: <20120224151158.GB16307@PC211.ikt.de> On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: > Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. > > OS X 10.7.3, behind a Cisco asa NAT. Deactivate the SMTP improvements your ASA helpfully provide. Dennis [..] From radecki.rafal at gmail.com Fri Feb 24 17:20:36 2012 From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=) Date: Fri, 24 Feb 2012 16:20:36 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. Message-ID: Hi all. I am currently gathering information about migration of Maildir/ structures from Courier to Dovecot. What do you think about http://wiki.dovecot.org/Migration/Courier ? For each courier Maildir account I want to use http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce dovecot-uidlist file mv courierimapsubscribed subscriptions sed sed -i 's/INBOX\.//' subscriptions rm -r courierimaphieracl rm courierimapacl rm courierimapkeywords I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf Should I look into something else? Have I missed something? Best regards, Rafal. From troy at surety-networks.com Fri Feb 24 17:43:03 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 10:43:03 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <45684F00-E4C4-4FD3-AD71-4E21B74C471A@surety-networks.com> Looking over the running config I don't see anything relating to smtp. I'm not using the esmtp inspection either. Am I looking in the wrong place? Would this cause the error I'm getting in mailaccess log? - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From micha at krausam.de Fri Feb 24 18:26:24 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 24 Feb 2012 17:26:24 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method Message-ID: <4F47BA30.2080704@krausam.de> Hi, I want to deliver mails directly to subfolders, based on address extensions. Example: Emails sent to: micha+test at example.com Should be stored into user micha, folder INBOX.test I was able to reproduce this with exim and dovecot-lda: command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" But I was wondering if I could do this within dovecot, which would enable me to use LMTP instead of dovecot-lda. Which brings me to my second question: Which method is "better" for dovecot, lda or LMTP? Micha Krause From gfinch at ldmltd.ca Fri Feb 24 18:36:00 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Fri, 24 Feb 2012 08:36:00 -0800 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47BC70.40308@ldmltd.ca> On 2012-02-24 6:22 AM, Przemys?aw.Orzechowski wrote: > Hi > > Im using Dovecot 1.0.10 and everything seems work ok except one client > He is accessing his mailbox using iPhone > There are several custom folders created and mail is directed to them > using procmail > The problem is iPhone mail app only sees changes in INBOX and when new > mail arrives to other folders it will not see it unlers user accesses this > particular folder > Is there some kind of workaround for this? > > Im quite sure its iPhones fault as all other email clients are workin > perfectly fine. As far as I know, that is a limitation of the iPhone mail client: it only IDLEs on the currently selected folder, and it only checks the currently selected folder for new messages. You will not see new mail for the other folders unless you select them. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From as at 3a.pl Fri Feb 24 18:46:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 24 Feb 2012 17:46:02 +0100 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <4F47A310.9000308@Media-Brokers.com> References: <4F47A310.9000308@Media-Brokers.com> Message-ID: <4F47BECA.4020104@3a.pl> On 24.02.2012 15:47, Charles Marcus wrote: > On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski > wrote: >> Im using Dovecot 1.0.10 > Very old - an upgrade would likeley fix your problems... A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. Only inbox is synced automatically. -- Adam Szpakowski From wilberth.perez at uady.mx Fri Feb 24 17:07:20 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Fri, 24 Feb 2012 09:07:20 -0600 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> Message-ID: <4F47A7A8.4090009@uady.mx> Hi Timo: My question was because constantly we received brute force attack from some of ip address which uses pop3 service to affect dovecot's login proccess. For example: Error: Temporary failure in creating login processes, slowing down for now pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A auth(default): Info: shadow(best,A.B.C.D): unknown user dovecot: Error: pipe() failed: Too many open files dovecot: Error: Temporary failure in creating login processes, slowing down for now In the log above from dovecto.log file, we observed a lot of conections from IP address A.B.C.D to our email server with ip address X.Y.Z.A using pop3 login process. Is possible prevent this type of attacks with any dovecot option (maybe limit the number of max connections from one ip address or maybe upgrading my dovecot version)? Thanks for you help and time. Wilberth. El 23/02/2012 05:21 p.m., Timo Sirainen escribi?: > On 24.2.2012, at 0.33, Wilberth Perez wrote: > >> Does any one , knows if is possible configure dovecot for limit max >> number of connections for IP address? >> >> I would like to prevent future fork-bombing attacks for pop3 and imap >> login process in my email server. >> >> Our dovecot version is : 1.2.10 > > There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. > > Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From marc at perkel.com Fri Feb 24 19:09:12 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 24 Feb 2012 09:09:12 -0800 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F478023.9080706@Media-Brokers.com> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> <4F478023.9080706@Media-Brokers.com> Message-ID: <4F47C438.2090300@perkel.com> On 2/24/2012 4:18 AM, Charles Marcus wrote: > On 23/02/2012 21:41, Marc Perkel wrote: >> Not sure how this can be done. >> >> Right now I'm running a free backup MX record service. People point >> their high MX records to my servers and if they go down we store the >> email. When they come back up it's delivered. >> >> What I want to do is have a premium service that would allow them to >> view through IMAP the stored email. In order to do that I would have >> to deliver the email locally to a dovecot server - to a single account >> perhaps - and they can view/forward/delete etc. >> >> But - when their server comes back online I'd like to gather up all >> the delivered email stored in Maildir format and run it back into smtp >> for delivery. > > I'm thinking it would be possible to use dsync for this? Ideally you > would have to have a way to maintain a duplicate of the AUTH database, > so that users could use their current passwords for accessing it (this > would obviously require the approval and cooperation of whoever you > are providing this service to), but this way you could store each > users mail individually and provide relatively painless access (since > all each user would need to know is the backup host name and their > username/password). > > You'd have to configure a script on your end to: > > 1. determine a reliable way to differentiate between a temp-fail > situation on the primary (in which case you just act as a normal > backup MX), or an extended outage (15 minutes? 1 hour? I'd probably > let the customer decide this one), which would determine when you > start actually taking delivery of their mail to the 'standby' server > > 2. monitor for when the primary comes back online > > 3. wait xx interval to make sure the primary stays up (again, I'd let > the customer decide this, but with a sane minimum that you define) > > 4. once the primary stays up for the required interval, dsync a one > way 'dump' of the stored mails, being sure to test for success - and > this process would obviously have to take into account the possibility > of the primary going down again in the middle of the dsync > > 5. mv successfully restored mails on the backup mx to a temp location > for xx 'safety' interval 'just in case' > > 6. delete the backed-up/stored mail that has been successfully dsync'd > after the safety interval has elapsed I was wondering if I could just gather up the maildir files, reprocess them into batch SMTP format and run them back into Exim? From miha.vrhovnik at cordia.si Fri Feb 24 19:10:58 2012 From: miha.vrhovnik at cordia.si (Miha Vrhovnik) Date: Fri, 24 Feb 2012 18:10:58 +0100 Subject: [Dovecot] =?iso-8859-1?q?iPhone_and_custom_imap_folders_problem?= In-Reply-To: <4F47BECA.4020104@3a.pl> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> Message-ID: <20120224181059.66377BF108F@miha> "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >On 24.02.2012 15:47, Charles Marcus wrote: >> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >> wrote: >>> Im using Dovecot 1.0.10 >> Very old - an upgrade would likeley fix your problems... >A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >Only inbox is synced automatically. > >-- >Adam Szpakowski So the solution is simple go bug Apple. Oh yeah. they say if something isn't implemented on Apple platform or it doesn't work, then you don't need it, because their devs know what's best for their users. Regards, M From julioarr at fisica.uh.cu Fri Feb 24 19:44:37 2012 From: julioarr at fisica.uh.cu (julioarr at fisica.uh.cu) Date: Fri, 24 Feb 2012 12:44:37 -0500 Subject: [Dovecot] remove messages once downloaded Message-ID: I need some help with the dovecot configuration. I want to remove downloaded messages from Mail Server once the messages have been successfully downloaded by pop3-clients, even when the clients have been configured to save copy of messages in the Server. thanks, Julio From trashcan at odo.in-berlin.de Fri Feb 24 17:27:56 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 24 Feb 2012 16:27:56 +0100 Subject: [Dovecot] dsync 2.0.x versus 2.1.x field report Message-ID: <47ce619d5887f172ef96c56b24b19b0e@mx2.enfer-du-nord.net> Hi -- I used to run two mailservers with identical MX priorities (poor man's fail-over approach) with Maildir mail folders accessed by Dovecot. Synchronization has been achieved using unison, mirroring all files except dovecot index files. Once in a while some oddities could be observed, like reappearing of deleted mails, duplicates and such alike. But the all-over performance was quite good. Then I wanted to migrate to mdbox, and I became interested in dsync. I did start with 2.0.16 a couple of month ago, and recently upgraded to 2.0.18. My experience with the 2.0.x dsync version is as follows (synchronization time: every minute): -- As long as mail is not stored in corresponding mailboxes at both servers at the same time, all mailboxes become synchronized perfectly well, and no oddites like reappearing of deleted mails or generation of duplicates can be observed. -- If mail is stored into corresponding mailboxes at both servers simultaneously in between two synching runs, it gets worrying. -- If I do run dsync without '-f' those two mailboxes end up in becoming asynchronous. -- If I do run dsync with '-f' both mailboxes will become synchronized let's say in 95% of all synchronization runs. BUT in 5%, I do experience asynchronicity. Here a 'doveadm force-resync' sometimes works, but not always, what worries me a lot. -- I can see the missing mail in the mdbox storage file using UNIX' little helpers but not with the help of dovecot tools. (Or I was unable to use them properly.) -- Modifying MX priorities in a way, that I do now have a primary and backup mailserver scenario, very much reduced the risk of losing legitimate mail. -- But sometimes I do lose some syslog mail being produced at the same time. Therefore I modified runtimes in way to omit synchronicity. Ok, I could have lived with that, but I was anxious to learn, if 2.1.x dsync would perform better ;-) Here my observations sofar (synchronization time: every minute): -- No losses sofar using 'doveadm sync' with '-f' -- A *lot* of reappearing deleted mails or mails moved from one mailbox to the other. -- Quite often I am unable to delete those mails using client functionality (Mail.app, Roundcube). -- In those cases one needs to stop synchronization, run 'doveadm force-resync', delete all reappearing mail (quite a lot sometimes), empty 'Deleted' folders, and finally run 'doveadm purge'. -- Running of 'doveadm sync' without '-f' doesn't improve the situation. Still I do have to struggle with reappearing mails, and sometimes with appearing duplicates. -- I do have a test setup running for 24 hours now, where I do generate test mails in a 6 minutes interval at both servers. Mails become synchronized perfectly well, irrespectively if '-f' is set or not. This is a major improvement to 2.0x. dsync (sofar). -- But running the synchronization without '-f' still shows a *lot* of reappering mail, though. I do have the feeling that dsync became very much improved with regard to missing mail in synchronization, good. But those reappearing mails make me think about switching back to 2.0.x dsync and live with the drawback of preventing mail to arrive synchronously at both servers. Is someone out there experiencing a similar behaviour with dsync 2.1.x regarding reappearing, thus indelible mail? Regards, Michael From CMarcus at Media-Brokers.com Fri Feb 24 21:29:26 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 14:29:26 -0500 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <4F47E516.3090101@Media-Brokers.com> On 2012-02-24 10:07 AM, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. This is quickly becoming an FAQ... If this is linux, and these attacks are from the *same* IP address(es) (sounds like that is the case), then your best option is something like fail2ban. If, however, they are from bots that change IP address each time, fail2ban won't be effective... -- Best regards, Charles From atoth at atoth.sote.hu Fri Feb 24 21:35:46 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Fri, 24 Feb 2012 20:35:46 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: It was probably caused by a bug already corrected. And not by dovecot. commit 4fd554e3a097b22c5049fcdc423897477deff5ef Author: Brad Spengler Date: Mon Feb 20 09:17:57 2012 -0500 Fix wrong logic on capability checks for switching roles, broke policies Thanks to Richard Kojedzinszky for reporting I'll test it with an updated kernel containing the fix. Thx: Dw. -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 24.(P) 00:28 id?pontban Timo Sirainen ezt ?rta: > On 24.2.2012, at 1.26, Timo Sirainen wrote: > >> On 23.2.2012, at 21.56, T?th Attila wrote: >> >>> In the mean time I've upgraded to 2.1. >>> I've enabled debug logging and logged in. >>> >>> I suspect that hardening features can be blamed for my problem. After >>> booting a previous kernel the behavior was reverted. >> >> OK. > > BTW. It would be interesting to know if the attached patch causes it to > fail with the hardening patches. > From dhenderson at mediatemple.net Sat Feb 25 00:49:50 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 14:49:50 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 Message-ID: Greetings, This email is both a request for assistance/help and a heads-up. [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. dovecot: imap-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: pop3-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/pop3-login(main+0x49) [0x401b99] dovecot: master: Error: service(pop3-login): child 27603 killed with signal 6 (core not dumped - add -D parameter to service pop3-login { executable } dovecot: master: Error: service(pop3-login): command startup failed, throttling dovecot: imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: imap-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/imap-login(main+0x39) [0x402069] dovecot: master: Error: service(imap-login): child 27604 killed with signal 6 (core not dumped - add -D parameter to service imap-login { executable } Once this patch was removed, everything started working again. Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? We haven't dug deeper yet, but the error is being thrown from the method io_loop_handle_add in ioloop-epoll.c http://hg.dovecot.org/dovecot-2.0/file/aa8dfa085a99/src/lib/ioloop-epoll.c Thanks Doug From tss at iki.fi Sat Feb 25 02:27:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:27:22 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <3A91726D-8D74-4AB2-ACAA-101376543133@iki.fi> On 24.2.2012, at 17.07, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. > > For example: > > dovecot: Error: pipe() failed: Too many open files > dovecot: Error: Temporary failure in creating login processes, slowing > down for now This is your main problem. Increase ulimit -n so Dovecot won't stop creating login processes (or alternatively decrease the max number of login processes). Newer Dovecot versions warn at startup if the fd limit is too low. From tss at iki.fi Sat Feb 25 02:39:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:15 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On 25.2.2012, at 0.49, Doug Henderson wrote: > [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. > > After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument .. > Once this patch was removed, everything started working again. > > Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). From tss at iki.fi Sat Feb 25 02:39:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:41 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Not possible. From tss at iki.fi Sat Feb 25 02:47:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:47:05 +0200 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <4F47BA30.2080704@krausam.de> References: <4F47BA30.2080704@krausam.de> Message-ID: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> On 24.2.2012, at 18.26, Micha Krause wrote: > I want to deliver mails directly to subfolders, based on address extensions. .. > I was able to reproduce this with exim and dovecot-lda: > > command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" > > > But I was wondering if I could do this within dovecot, which would enable me to use > LMTP instead of dovecot-lda. lmtp_save_to_detail_mailbox = yes > Which brings me to my second question: > > Which method is "better" for dovecot, lda or LMTP? LMTP is often easier to setup and faster. From tss at iki.fi Sat Feb 25 02:50:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:50:34 +0200 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: On 24.2.2012, at 17.20, Rafa? Radecki wrote: > I am currently gathering information about migration of Maildir/ > structures from Courier to Dovecot. > What do you think about http://wiki.dovecot.org/Migration/Courier ? > For each courier Maildir account I want to use > http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce > dovecot-uidlist file > mv courierimapsubscribed subscriptions > sed sed -i 's/INBOX\.//' subscriptions The migration script migrates subscriptions, so those shouldn't be necessary. > rm -r courierimaphieracl > rm courierimapacl You don't mind losing any ACLs? > I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). From msidoric at mac.com Sat Feb 25 03:07:27 2012 From: msidoric at mac.com (Michael Sidoric) Date: Fri, 24 Feb 2012 19:07:27 -0600 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: > > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. timo is quite correct -- consider imap instead, that's why the protocol exists. From ml at smtp.fakessh.eu Sat Feb 25 04:42:27 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 03:42:27 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question Message-ID: <1330137747.13977.3.camel@localhost> hi guys hi master of "Fu" hi Timo I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I encounter fatal errors. I created it myself rpms available here http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm I can not run dovecot with this config file like this http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm i run this config ks37777 ~]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1 vsz_limit = 64 M } service imap { client_limit = 1024 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From troy at surety-networks.com Sat Feb 25 04:47:59 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 21:47:59 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <02C084C7-5EE0-4CE8-A60E-C40D880EF3E3@surety-networks.com> The correct answer was in the log file. Behind a NAT you have to set login_trusted_networks = internal/24 externalIP - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From tss at iki.fi Sat Feb 25 04:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 04:49:07 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330137747.13977.3.camel@localhost> References: <1330137747.13977.3.camel@localhost> Message-ID: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> On 25.2.2012, at 4.42, ml wrote: > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > encounter fatal errors. And what are the error messages? From ml at smtp.fakessh.eu Sat Feb 25 05:32:50 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 04:32:50 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> Message-ID: <1330140770.15576.1.camel@localhost> Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 4.42, ml wrote: > > > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > > encounter fatal errors. > > And what are the error messages? > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child 15407 killed with signal 11 (core dumps disabled) my current config ks37777 i386]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 128 vsz_limit = 64 M } service imap { client_limit = 1 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 128 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 128 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From tss at iki.fi Sat Feb 25 05:45:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:45:39 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330140770.15576.1.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> Message-ID: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> On 25.2.2012, at 5.32, ml wrote: > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : >> On 25.2.2012, at 4.42, ml wrote: >> >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I >>> encounter fatal errors. >> >> And what are the error messages? >> > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > 15407 killed with signal 11 (core dumps disabled) Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Sat Feb 25 05:54:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:54:22 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 Message-ID: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). From ml at smtp.fakessh.eu Sat Feb 25 06:51:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 05:51:06 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> Message-ID: <1330145466.15576.4.camel@localhost> Le samedi 25 f?vrier 2012 ? 05:45 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 5.32, ml wrote: > > > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > >> On 25.2.2012, at 4.42, ml wrote: > >> > >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > >>> encounter fatal errors. > >> > >> And what are the error messages? > >> > > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > > 15407 killed with signal 11 (core dumps disabled) > > Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > I have not succeeded well I got a son of fakessh I attached with gdb-p but bt full no me back stack my poor english -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From tss at iki.fi Sat Feb 25 07:17:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:17:46 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 In-Reply-To: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> References: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> Message-ID: <8AA21A89-D80B-4B82-A9A8-9E14A3A1F385@iki.fi> On 25.2.2012, at 5.54, Timo Sirainen wrote: > I just committed a couple of features that will make life easier for some types of proxying setups: Oh, and: 4. Added proxy_always=y passdb field. When used with proxy_maybe=y you can create a setup where the connections are proxied to either * when user's host matches locally, director assigns a local backend's host for the connection * when user's host doesn't match locally, Dovecot connects to that host So basically you can have a bunch of Dovecot cluster installations behind different load balancer IPs, and have all of the Dovecot proxies redirect the connection to the proper cluster's load balancer, or if it's for a local cluster then redirect to one of the backends. 5. Currently in v2.2 only, could be backported if needed: Added TTL for the proxy connections, so in case two clusters both think the other one should be handling the connection they won't end up creating thousands of connections to each others until finally failing when login process limits are reached. The hard coded TTL is 5, should be more than enough. From tss at iki.fi Sat Feb 25 07:18:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:18:54 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330145466.15576.4.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> Message-ID: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> On 25.2.2012, at 6.51, ml wrote: >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> > > I have not succeeded well I got a son of fakessh I attached with gdb-p > but bt full no me back stack Did you try without antispam plugin? My guess is that is the reason it crashes. Or if it still crashes, try disabling all plugins. From dmiller at amfes.com Sat Feb 25 08:09:09 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 24 Feb 2012 22:09:09 -0800 Subject: [Dovecot] Solr not indexing Message-ID: What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" doveadm -D fts rescan -u dmiller at amfes.com doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(dmiller at amfes.com): Debug: Added userdb setting: mail=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: Added userdb setting: plugin/acl_groups='allshared' doveadm(dmiller at amfes.com): Debug: Effective uid=5000, gid=8, home=/var/mail/amfes.com/dmiller doveadm(dmiller at amfes.com): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: fs: root=/var/mail/amfes.com/dmiller/mdbox, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: Namespace : Using permissions from /var/mail/amfes.com/dmiller/mdbox: mode=0700 gid=-1 doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 1 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls doveadm(dmiller at amfes.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=mdbox:/var/mail/%d/%n/mdbox doveadm(dmiller at amfes.com): Debug: shared: root=/usr/local/var/run/dovecot, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/' doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 0 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls -- Daniel From tss at iki.fi Sat Feb 25 08:24:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:24:20 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: Message-ID: On 25.2.2012, at 8.09, Daniel L. Miller wrote: > What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" That's intentional. "shared/%n/" namespace has no mails. Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. From dhenderson at mediatemple.net Sat Feb 25 08:32:12 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 22:32:12 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > On 25.2.2012, at 0.49, Doug Henderson wrote: > >> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >> >> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. > > I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 see: https://rhn.redhat.com/errata/RHSA-2012-0150.html "A flaw was found in the way the Linux kernel's Event Poll (epoll) subsystem handled large, nested epoll structures. A local, unprivileged user could use this flaw to cause a denial of service. (CVE-2011-1083, Moderate)" Our automated patching (ksplice) installed it at around 10am PST today. Other distributions may vary. > >> dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument > .. >> Once this patch was removed, everything started working again. >> >> Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? > > It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). > We haven't spent enough time investigating to be sure, but epoll_ctl was certainly "in the thick of it". The only outward evidence (in logs, even with debug turned on) that there was anything wrong with Dovecot at all was the Panic shown for that method. Dovecot may have been an innocent bystander in this case - but something was causing it to fail on inbound IMAP/POP3 connections, and when the patch was removed everything started working again. From tss at iki.fi Sat Feb 25 08:49:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:49:10 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <876F3440-67CC-47B2-AA33-96A067A6A22D@iki.fi> On 25.2.2012, at 8.32, Doug Henderson wrote: >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html Yes, but CVE-2011-1083 shows it was reported almost a year ago, so I'd think it was fixed in upstream kernel a long time ago. I'm running in my desktop about 2 months old kernel (from git) and I don't see any problems with it. But yeah, maybe Redhat's patches did it differently than upstream kernel and it broke because of that.. From mstevens at imt-systems.com Sat Feb 25 13:15:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sat, 25 Feb 2012 12:15:49 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> On 25.02.2012 07:32, Doug Henderson wrote: > On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > >> On 25.2.2012, at 0.49, Doug Henderson wrote: >> >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail >>> servers, most/all IMAP/POP3 connections began experiencing time-outs >>> trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so >> other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat > for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html > > "A flaw was found in the way the Linux kernel's Event Poll (epoll) > subsystem handled large, nested epoll structures. A local, > unprivileged > user could use this flaw to cause a denial of service. > (CVE-2011-1083, > Moderate)" > > Our automated patching (ksplice) installed it at around 10am PST > today. > > Other distributions may vary. Try it without ksplice. (yum update and reboot) Which kernel is running exactly? Best regards, Morten From busseniu at in.tum.de Sat Feb 25 21:16:20 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Sat, 25 Feb 2012 20:16:20 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: <4F493384.9010703@in.tum.de> On 02/25/2012 01:50 AM, Timo Sirainen wrote: > On 24.2.2012, at 17.20, Rafa? Radecki wrote: >> I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf > > That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). > The migration script takes the UIDLs from "courierpop3dsizelist" and writes them into "dovecot-uidlist". Thus Dovecot will use the same UIDL as Courier for the converted mails. %08Xu%08Xv will only be used for new mails. Cheers, Christoph From dmiller at amfes.com Sun Feb 26 03:42:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 25 Feb 2012 17:42:22 -0800 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 2/24/2012 10:24 PM, Timo Sirainen wrote: > On 25.2.2012, at 8.09, Daniel L. Miller wrote: > >> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" > That's intentional. "shared/%n/" namespace has no mails. > > Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? -- Daniel From tss at iki.fi Sun Feb 26 04:26:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 04:26:07 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 26.2.2012, at 3.42, Daniel L. Miller wrote: > On 2/24/2012 10:24 PM, Timo Sirainen wrote: >> On 25.2.2012, at 8.09, Daniel L. Miller wrote: >> >>> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" >> That's intentional. "shared/%n/" namespace has no mails. >> >> Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. >> > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? The cached "last fts indexed message" gets dropped with: doveadm force-resync INBOX From dhenderson at mediatemple.net Sun Feb 26 04:55:59 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sat, 25 Feb 2012 18:55:59 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> Message-ID: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: > > Try it without ksplice. (yum update and reboot) I don't know if I'll be permitted to do that in a production environment - possibly a test one. I'll need to get some opinions from our Ops people as to if/how they might want to go about it. > Which kernel is running exactly? 2.6.18-274.3.1.el5 > Best regards, > > Morten From f.bonnet at esiee.fr Sun Feb 26 09:22:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Sun, 26 Feb 2012 08:22:48 +0100 Subject: [Dovecot] FreeBSD maintainer ? Message-ID: <4F49DDC8.2060506@esiee.fr> Hello Does the FreeBSD Dovecot's port maintainer read this mailing-list ? Thanks From duihi77 at gmail.com Sun Feb 26 10:54:56 2012 From: duihi77 at gmail.com (Duane Hill) Date: Sun, 26 Feb 2012 08:54:56 +0000 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <1527928288.20120226085456@gmail.com> On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr confabulated: > Hello > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > Thanks Have you tried sending message to the maintainer? The email address can be found by: cd /usr/ports/mail/dovecot && make maintainer or cd /usr/ports/mail/dovecot2 && make maintainer There appears to be two different email addresses between maintainers for version 1.2.x and 2.0.x. -- If at first you don't succeed... ...so much for skydiving. From tss at iki.fi Sun Feb 26 12:30:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 12:30:43 +0200 Subject: [Dovecot] IMAP NOTIFY extension Message-ID: <7075A103-8131-491F-9496-C632418BAF67@iki.fi> http://hg.dovecot.org/dovecot-2.2-notify/ contains the initial implementation of IMAP NOTIFY extension. It's less than half-working, but the difficult part of it is done I think. If I give it a command like: x notify set (personal (messagenew messageexpunge flagchange)) and then in another session I copy a message to another mailbox, it sends: * STATUS "foobar" (MESSAGES 77 UIDNEXT 78 UIDVALIDITY 1329431281 UNSEEN 75) This requires mailbox_list_index=yes to work. It works by following the dovecot.list.index.log changes, so it's going to be pretty efficient. Some TODO that I've so far noticed: - NOTIFY is handled currently only for non-selected mailboxes. (handling the new features for selected mailbox should be simple) - initial STATUS sending isn't done with STATUS option - only MessageNew, MessageExpunge and FlagChange notifications are sent currently - MailboxName handling probably needs to be able to get "old" and "new" views, figure out what changed in them and send those changes (dsync has similar code) - mailbox list indexes seem to be adding too much stuff, which results in: - STATUS duplicates, but there should already be code to avoid these so I'm not sure what's going on - ACLs and shared mailboxes don't work right - add 0.5s delay before notifying IMAP code, so multiple changes can be bundled into one From mstevens at imt-systems.com Sun Feb 26 12:44:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 26 Feb 2012 11:44:18 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: On 26.02.2012 03:55, Doug Henderson wrote: > On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >> >> Try it without ksplice. (yum update and reboot) > > I don't know if I'll be permitted to do that in a production > environment - possibly a test one. > I'll need to get some opinions from our Ops people as to if/how they > might want to go about it. > >> Which kernel is running exactly? > > 2.6.18-274.3.1.el5 That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. Best regards, Morten From lists at wildgooses.com Sun Feb 26 13:52:37 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 11:52:37 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4A1D05.2070402@wildgooses.com> On 25/02/2012 00:39, Timo Sirainen wrote: > On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > Not possible. > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. Is it feasible to implement both of these solutions using the current plugin architecture? I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Cheers Ed W From jerry at seibercom.net Sun Feb 26 13:56:42 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 06:56:42 -0500 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <1527928288.20120226085456@gmail.com> References: <4F49DDC8.2060506@esiee.fr> <1527928288.20120226085456@gmail.com> Message-ID: <20120226065642.24deacd8@scorpio> On Sun, 26 Feb 2012 08:54:56 +0000 Duane Hill articulated: > On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr > confabulated: > > > Hello > > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > > > Thanks > > Have you tried sending message to the maintainer? The email address > can be found by: > > cd /usr/ports/mail/dovecot && make maintainer > > or > > cd /usr/ports/mail/dovecot2 && make maintainer > > There appears to be two different email addresses between maintainers > for version 1.2.x and 2.0.x. That is because there are two different maintainers: dovecot-1.2.17 == yds at CoolRat.org dovecot-2.0.16_1 == bra at fsn.hu If you are having a problem with the port, I would suggest that you post on the FreeBSD Ports forum: freebsd-ports at freebsd.org and CC the maintainer of the port in question. It works quite effectively. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Sun Feb 26 14:31:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 14:31:40 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4A1D05.2070402@wildgooses.com> References: <4F4A1D05.2070402@wildgooses.com> Message-ID: On 26.2.2012, at 13.52, Ed W wrote: > On 25/02/2012 00:39, Timo Sirainen wrote: >> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >> >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have been >>> configured to save copy of messages in the Server. >> Not possible. >> > > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) > > In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. What does the notification do? > Is it feasible to implement both of these solutions using the current plugin architecture? Yes. > I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? > > All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. From lists at wildgooses.com Sun Feb 26 15:23:59 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 13:23:59 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: <4F4A1D05.2070402@wildgooses.com> Message-ID: <4F4A326F.8000803@wildgooses.com> On 26/02/2012 12:31, Timo Sirainen wrote: > On 26.2.2012, at 13.52, Ed W wrote: > >> On 25/02/2012 00:39, Timo Sirainen wrote: >>> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >>> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. >>> Not possible. >>> >> If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) >> >> In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. > What does the notification do? Sends another email... (you know like the annoying "message read" indicators that lots of mail readers support)... (Several of our competitors have implemented these solutions very badly and we get mail loops and other nasties...) >> I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? >> >> All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? > Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. Thanks - it's off my radar for a while due to other pressures, but the hint is appreciated and I will look into it in the future - many thanks! Ed W From fxcode at me.com Sun Feb 26 15:44:05 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 13:44:05 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted Message-ID: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Hi, I am relatively new to linux, and have set up my first lamp server around a month ago. The server is housed in a data centre a few miles from my house, and is working as a web server (with apache, tomcat, mysql) just fine. I have now been trying for a week to get Postfix and Dovecot working with postfixadmin and mysql. (below I have replaced my test domain name with [***domain_name***]) The story so far; I can send 'welcome' emails from within postfix admin to an external email address... and they are received by the mail client on my computer at home no problem. I can set up alias email addresses within postfix admin, such as webmaster@[***domain_name***], and emails sent to this address are redirected to the proper address as expected. But - if I set up a standard mailbox within postfix admin - test@[***domain_name***] , and email it from my home system, nothing appear in postfixadmin (after a fetch request) or the mysql database. Here is the fail point in my mail.log file ; Feb 26 12:50:27 quantum dovecot: deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with euid=8(mail), gid=8(mail), egid=8(mail): Operation not permitted I don't know what the setguid(5000) is, or where it is being set, or what it should be! I have been changing permissions with the 'socket listen' section of my /etc/dovecot/dovecot.conf file, and when I restart dovecot it creates the auth-master file with the permissions as expected; socket listen { master { path = /var/run/dovecot/auth-master mode = 0777 user = vmail group = mail } client { path = /var/spool/postfix/private/auth mode = 432 #user = postfix #group = postfix } } my mail folders should be located at /var/vmail, but the folder is empty. I have chmod the permissions to be as follows; me at myserver: ls -ld /var/vmail drwxrwxrwx 2 vmail mail 4096 2012-02-11 08:40 /var/vmail Can anyone spot where this is going wrong? Thanks in advance for any help, it is very very much appreciated!!! My hosts file is as follows; 127.0.0.1 localhost [**server_ip***] quantum.[***domain_name***] quantum # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ...and 'dovecot -n' replies as follows; me at myserver: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:/var/vmail/%d/%n mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: log_path: /var/log/dovecot-deliver.log auth_socket_path: /var/run/dovecot/auth-master postmaster_address: fxcode at me.com mail_plugins: sieve auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 master: path: /var/run/dovecot/auth-master mode: 511 user: vmail group: mail From sk71 at gmx.de Sun Feb 26 17:54:22 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 16:54:22 +0100 Subject: [Dovecot] Deny create folders Message-ID: <4F4A55AE.1030402@gmx.de> Hi all, is it possible to configure dovecot in that way that special folders do not create automatically by some imap clients? i.e: outlook or thunderbird creates a folder junk-mail i would like that such folders are not created automatically or by users Best regards Stefan From tss at iki.fi Sun Feb 26 18:02:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 18:02:47 +0200 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: On 26.2.2012, at 17.54, S. Kremer wrote: > is it possible to configure dovecot in that way that special folders do > not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by users You could maybe create a global ACL that denies the access to create them. The clients might not be happy about it though. It's not a very good idea to go doing things like this I think. From jerry at seibercom.net Sun Feb 26 18:41:25 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 11:41:25 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: <20120226114125.59e0d407@scorpio> On Sun, 26 Feb 2012 16:54:22 +0100 S. Kremer articulated: > Hi all, > > is it possible to configure dovecot in that way that special folders > do not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by > users Exactly what problem are you attempting to alleviate? In the short term you might seriously upset some of your clients and in the long term cause a stability problem. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From fxcode at me.com Sun Feb 26 22:26:20 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 20:26:20 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted In-Reply-To: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> References: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Message-ID: <11EBA09F-36C6-4319-874A-44207A6ED435@me.com> Hi, I have fixed this fault - the following line in my /etc/dovecot/dovecot.conf file was setting the setgid to 5000, and the user id to 5000 also auth default { ... userdb static { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } ... } I changed these two values to 8 ( the id of mail on my system) and emails are now being delivered to the proper folders thanks Christy From sk71 at gmx.de Sun Feb 26 23:31:35 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 22:31:35 +0100 Subject: [Dovecot] Deny create folders In-Reply-To: <20120226114125.59e0d407@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> Message-ID: <4F4AA4B7.4030909@gmx.de> Ok, thanks for answer. Every user's mailbox has a directory called Spam and Ham. I scan this two folders by a script to learn my spamassassin database. This script does not scan folders like spam, ham, junk-email etc. But some users have such folders in their mailbox so spamassassin does not learn from these folders. My aim is to force users to use the folders Spam and Ham and not other folders for spam or ham messages. Am 26.02.2012 17:41, schrieb Jerry: > On Sun, 26 Feb 2012 16:54:22 +0100 > S. Kremer articulated: > >> Hi all, >> >> is it possible to configure dovecot in that way that special folders >> do not create automatically by some imap clients? >> >> i.e: >> outlook or thunderbird creates a folder junk-mail >> >> i would like that such folders are not created automatically or by >> users > > Exactly what problem are you attempting to alleviate? In the short term > you might seriously upset some of your clients and in the long term > cause a stability problem. > Greetings Stefan From jerry at seibercom.net Mon Feb 27 00:28:27 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 17:28:27 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4AA4B7.4030909@gmx.de> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> Message-ID: <20120226172827.112de657@scorpio> On Sun, 26 Feb 2012 22:31:35 +0100 S. Kremer articulated: > Am 26.02.2012 17:41, schrieb Jerry: > > On Sun, 26 Feb 2012 16:54:22 +0100 > > S. Kremer articulated: > > > >> Hi all, > >> > >> is it possible to configure dovecot in that way that special > >> folders do not create automatically by some imap clients? > >> > >> i.e: > >> outlook or thunderbird creates a folder junk-mail > >> > >> i would like that such folders are not created automatically or by > >> users > > > > Exactly what problem are you attempting to alleviate? In the short > > term you might seriously upset some of your clients and in the long > > term cause a stability problem. > Ok, thanks for answer. > > Every user's mailbox has a directory called Spam and Ham. I scan this > two folders by a script to learn my spamassassin database. > This script does not scan folders like spam, ham, junk-email etc. > But some users have such folders in their mailbox so spamassassin does > not learn from these folders. > > My aim is to force users to use the folders Spam and Ham and not other > folders for spam or ham messages. Please don't top post. It makes following a thread a lot harder than it needs to be. Couldn't you just make a soft link between the two folders and save yourself and your clientele a lot of trouble? I have not messed with "spamassassin" in years; however, I thought it was configurable on what it scans. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ml at smtp.fakessh.eu Mon Feb 27 00:37:35 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 26 Feb 2012 23:37:35 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330295855.14524.12.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I'm in front of a problem. despite having to disable the antispam plugin as it appears in dovecot.conf always load so I did not succeed to disable I do not know how can you tell me that I continue my tests without the antispam plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From jd.beaubien at gmail.com Mon Feb 27 01:26:02 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Sun, 26 Feb 2012 18:26:02 -0500 Subject: [Dovecot] testing fts-solr? Message-ID: hi everyone, However, how can I verify that solr is working properly? Background: - I'm trying to setup 2.1 with fts_solr and eventually test maildir/mdbox. - I used the following webpage as reference to setup solr: http://www.roessner-network-solutions.com/2012/02/19/full-text-search-with-solr-and-dovecot-on-ubuntu-10-04/ I'm looking at /var/log/tomcat6/catalina.out file, but see nothing telling me that it's being used (same for /var/log/mail.log, etc.) Any tips? Thanks From ml at smtp.fakessh.eu Mon Feb 27 01:39:58 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 00:39:58 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330299598.14524.13.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I managed dovecot 2.1. 1 works well in removing completely antispam plugin work well and fine -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From maria_arrea at gmx.com Mon Feb 27 02:07:00 2012 From: maria_arrea at gmx.com (Maria Arrea) Date: Mon, 27 Feb 2012 01:07:00 +0100 Subject: [Dovecot] Dovecot altmove questions Message-ID: <20120227000700.134750@gmx.com> Hello We are using Dovecot 2.0.17 with mdbox+zlib and we are evaluating alternative storage. These are the relevant settings mdbox_rotate_interval = 1d mdbox_rotate_size=60m [...] zlib_save_level = 9 # 1..9 zlib_save = gz # or bz2 [...] zlib plugin enabled in dovecot-lda These are my questions: 1? General mdbox question. We are happy usings zlib_save, but we would like to try bzip2 compression. Can we change zlib to bzlib without problems? In other words, can we mix & match gzipped and bzipped messages in the same mdbox file ? 2? With doveadm altmove we can move messages matching a pattern to alternate storage. I have read an older post in this mailing list about passing specific mdbox options to doveadm altmove. Should be a good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d and mdox_rotate_size=60m and an "alternate area" with bzip9, mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area should only be "old" mail (1 year older). 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is taking almost 16 hours to make a full backup. Our main problem is that indexes are backupd hours before mdboxes, and that bring us some inconsistencies. What is the "right" way of backing up for separate indexes and mdboxes? Thank you for your support. Maria From ml at smtp.fakessh.eu Mon Feb 27 03:11:22 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 02:11:22 +0100 Subject: [Dovecot] value for client_limit Message-ID: <1330305082.20912.5.camel@localhost> hello guys hello master Timo i build the last release of dovecot on my centos 5 work well and fine but what the correct value for client limit in service auth { client_limit=6000 } and service anvil { client_limit=6000 } my current dovecot -n ks37777 i386]# dovecot -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service anvil { client_limit = 6000 } service auth { client_limit = 6000 process_limit = 1 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { client_limit = 0 inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1024 service_count = 1 vsz_limit = 64 M } service imap { process_limit = 1024 process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From sven at svenhartge.de Mon Feb 27 04:26:11 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 03:26:11 +0100 Subject: [Dovecot] Dovecot altmove questions References: <20120227000700.134750@gmx.com> Message-ID: <78jhe1341pv8@mids.svenhartge.de> Maria Arrea wrote: > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). > We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? Bacula? Excellent! If you put your indexes and mdboxes on a LVM (seperate volume group obviously, because of the different storages both reside on), you can snapshot both volumes at the same time, using an FD-based script, mount them somewhere else (like /backup/data and /backup/index) and then you have (nearly) all the time you want to backup them. And because of the snapshots taken at the same time, no inconsistencies can happen. (OK, not entirely true, since both snapshots are taken with a minimal gap between them, but if you start the backup during off-hours the inconsistencies should be minimal.) If you don't use a LVM but have a storage which is able to do snapshots on its own, you can also use this. If you don't have a LVM and no storage with such functionality, then you are kind of lost, as far as I can see. My additional advise on the long time of full backups: don't do them! Switch to accurate backups and only do incremental and differential backups, replacing the full backups with virtual full ones. This will greatly reduce the load on your mail systems and speed up the backup operation at the same time. But you have to use "accurate" type backups, or your virtual full backup will grow and grow and grow, since Bacula without "accurate" is unable to know about deleted files and will include them in every virtual full backup. Gr??e, Sven. -- Sigmentation fault. Core dumped. From dhenderson at mediatemple.net Mon Feb 27 05:20:36 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sun, 26 Feb 2012 19:20:36 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: <775A9BD8-884C-47D5-BF82-2BBDF414BF56@mediatemple.net> On Feb 26, 2012, at 2:44 AM, Morten Stevens wrote: > On 26.02.2012 03:55, Doug Henderson wrote: >> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >>> >>> Try it without ksplice. (yum update and reboot) >> >> I don't know if I'll be permitted to do that in a production >> environment - possibly a test one. >> I'll need to get some opinions from our Ops people as to if/how they >> might want to go about it. >> >>> Which kernel is running exactly? >> >> 2.6.18-274.3.1.el5 > > That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. > > Best regards, > > Morten Thanks Morten, We'll install the latest kernel on a test machine tomorrow and see how things go - we'll probably also attempt to reinstall the patch (if appropriate) and see if it still breaks things. Doug From mark at ecs.vuw.ac.nz Mon Feb 27 03:17:32 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 14:17:32 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail Message-ID: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Hi, have dovecot 2.0.13 running fine on a NetBSD box, with lots of clients happily authenticating to it with GSSAPI. I've just updated an ArchLinux box to latest new version which has meant its switched from using heimdal to using mit krb5 and its kmail is now also slightly newer. Thunderbird successfully authenticates to dovecot from this machine but kmail now fails. Kmail reports: Unable to authenticate via GSSAPI. The server ... replied: SASL(-1): generic failure: GSSAPI Error: Invalid token was supplied (Token header is malformed or corrupt) The dovecot log reports: Feb 27 13:42:18 bats dovecot: auth: GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in continued response Any suggestions on how to track down what specifically has changed and how to fix it? cheers mark From bra at fsn.hu Mon Feb 27 09:07:24 2012 From: bra at fsn.hu (Attila Nagy) Date: Mon, 27 Feb 2012 08:07:24 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <4F4B2BAC.2030400@fsn.hu> On 02/26/12 08:22, Frank Bonnet wrote: > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > If you read it, you may know the answer (depending on which port do you use). From f.bonnet at esiee.fr Mon Feb 27 09:13:44 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 27 Feb 2012 08:13:44 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F4B2BAC.2030400@fsn.hu> References: <4F49DDC8.2060506@esiee.fr> <4F4B2BAC.2030400@fsn.hu> Message-ID: <4F4B2D28.5010600@esiee.fr> hello I'm wondering when you will port 2.1.1 version ? ;-) Thanks Le 27/02/2012 08:07, Attila Nagy a ?crit : > On 02/26/12 08:22, Frank Bonnet wrote: >> >> Does the FreeBSD Dovecot's port maintainer read this mailing-list ? >> > If you read it, you may know the answer (depending on which port do > you use). From busseniu at in.tum.de Mon Feb 27 09:23:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Mon, 27 Feb 2012 08:23:14 +0100 Subject: [Dovecot] [PATCH] Pop3 order in courier migration script Message-ID: <4F4B2F62.1020204@in.tum.de> Hi, I found a problem in the courier conversion script (courier-dovecot-migrate.pl). In some cases, it does not correctly preserve the order of POP3 UIDLs. Consider the following case, when courierpop3dsizelist contains: /2 9000 1327481629 AAA1:2,S 10000 21:1327481629 AAA2:2,S 10000 22:1327481629 AAA3:2,S 10000 23:1327481629 AAA4:2,S 10000 24:1327481629 AAA5:2,S 10000 25:1327481629 BBB11:2,S 10000 11:1327481629 BBB12:2,S 10000 12:1327481629 BBB13:2,S 10000 13:1327481629 BBB14:2,S 10000 14:1327481629 BBB15:2,S 10000 15:1327481629 and courierimapuiddb contains: 1 1327480919 9000 11 BBB11 12 BBB12 13 BBB13 14 BBB14 15 BBB15 16 CCC16 17 CCC17 18 CCC18 19 CCC19 20 CCC20 21 AAA1 22 AAA2 23 AAA3 24 AAA4 25 AAA5 then call courier-dovecot-migrate.pl --to-dovecot --convert and it will create dovecot-uidlist: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 :CCC17 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 :CCC16 9003 PUID15-1327481629 W10000 :BBB15:2,S 9004 :CCC20 9005 PUID14-1327481629 W10000 :BBB14:2,S 9006 PUID13-1327481629 W10000 :BBB13:2,S 9007 :CCC18 9008 PUID11-1327481629 W10000 :BBB11:2,S 9009 :CCC19 The problem here is that the BBB files are not in the same order as in courierpop3dsizelist, and will therefore have a different order in the UIDL response in POP3. Also, the CCC files are interleaved while they should appear at the end. The reason is how the script assigns new UIDs: It does not do this in the order of courierpop3dsizelist. In fact the order is random because it depends on how perl traverses hashes. The attached patch fixes this by sorting the messages: First the POP3 messages get their new UIDs in order of their appearance in courierpop3dsizelist (so that the UIDL order will really be the same as in Courier), after that, the IMAP-only messages will be checked for UID clashes. The patch also changes how the $prev_uid variable is handled because that was also needed to always get the order right. The algorithm could probably be optimized to reduce the number of UID re-assignments to a minimum, however this would complicate the code and would only be noticed in a few cases. The patched courier-dovecot-migrate.pl will create dovecot-uidlist like this: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 PUID11-1327481629 W10000 :BBB11:2,S 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 PUID13-1327481629 W10000 :BBB13:2,S 9003 PUID14-1327481629 W10000 :BBB14:2,S 9004 PUID15-1327481629 W10000 :BBB15:2,S 9005 :CCC16 9006 :CCC17 9007 :CCC18 9008 :CCC19 9009 :CCC20 Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching -------------- next part -------------- A non-text attachment was scrubbed... Name: courier-dovecot-migrate.patch Type: text/x-patch Size: 1386 bytes Desc: not available URL: From tss at iki.fi Mon Feb 27 10:34:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 10:34:42 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <1330331682.11500.291.camel@innu> On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > What do you need the statistics for? I could make imap_client and > pop3_client support some virtual methods, like user.destroy() initially, > which would be enough for your use. I guess I could add that for v2.2. http://dovecot.org/patches/2.2/imap-logout-plugin.c From Ralf.Hildebrandt at charite.de Mon Feb 27 10:39:18 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 09:39:18 +0100 Subject: [Dovecot] Error with 2.0.18 Message-ID: <20120227083917.GD16162@charite.de> Today I found this in my log: Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3d44a) [0xb771644a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3d4bb) [0xb77164bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7716d87] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x3bfa) [0xb74e1bfa] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(squat_trie_build_deinit+0x9fc) [0xb74e4d3c] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x2a95) [0xb74e0a95] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_build_deinit+0x1b) [0xb752f6bb] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x3726) [0xb7530726] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x51d3) [0xb75321d3] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x5c34) [0xb7532c34] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x3b) [0xb778b0cb] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARCH]() [0x805ce6e] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARC Feb 26 14:27:55 postamt dovecot: master: Error: service(imap): child 18468 killed with signal 6 (core dumped) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at wildgooses.com Mon Feb 27 10:46:31 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 27 Feb 2012 08:46:31 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1330331682.11500.291.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> Message-ID: <4F4B42E7.4090101@wildgooses.com> On 27/02/2012 08:34, Timo Sirainen wrote: > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > >> What do you need the statistics for? I could make imap_client and >> pop3_client support some virtual methods, like user.destroy() initially, >> which would be enough for your use. I guess I could add that for v2.2. > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > Thanks - can I assume that a pop-logout would be basically the same? Also, how might I access the bytes in/out statistics from that context? Thanks Ed W From tss at iki.fi Mon Feb 27 11:05:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:05:44 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4B42E7.4090101@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> <4F4B42E7.4090101@wildgooses.com> Message-ID: <1330333544.11500.293.camel@innu> On Mon, 2012-02-27 at 08:46 +0000, Ed W wrote: > On 27/02/2012 08:34, Timo Sirainen wrote: > > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > > > >> What do you need the statistics for? I could make imap_client and > >> pop3_client support some virtual methods, like user.destroy() initially, > >> which would be enough for your use. I guess I could add that for v2.2. > > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > > > > > Thanks - can I assume that a pop-logout would be basically the same? Yes, basically s/imap/pop3. > Also, how might I access the bytes in/out statistics from that context? input: i_stream_get_absolute_offset(client->input) output: client->output->offset From tss at iki.fi Mon Feb 27 11:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:10:03 +0200 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <20120227083917.GD16162@charite.de> References: <20120227083917.GD16162@charite.de> Message-ID: <1330333803.11500.295.camel@innu> On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > Today I found this in my log: > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Large index, process VSZ limit reached. You might need to increase service imap { vsz_limit }. > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Bugs caused by the above. I don't think I'm going to spend time on Squat anymore, with v2.1 Lucene backend works better and faster. From tss at iki.fi Mon Feb 27 12:24:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:24:27 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: Message-ID: <1330338267.11500.299.camel@innu> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > hi everyone, > > However, how can I verify that solr is working properly? You should see in logs things like: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX It should be in Dovecot's info log (see "doveadm log find"). If you can't find it, see if "doveadm index -u user at domain INBOX" does any work. Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't give an error. From tss at iki.fi Mon Feb 27 12:28:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:28:16 +0200 Subject: [Dovecot] Dovecot altmove questions In-Reply-To: <20120227000700.134750@gmx.com> References: <20120227000700.134750@gmx.com> Message-ID: <1330338496.11500.303.camel@innu> On Mon, 2012-02-27 at 01:07 +0100, Maria Arrea wrote: > 1? General mdbox question. We are happy usings zlib_save, but we > would like to try bzip2 compression. Can we change zlib to bzlib > without problems? In other words, can we mix & match gzipped and > bzipped messages in the same mdbox file ? Yes. > 2? With doveadm altmove we can move messages matching a pattern to > alternate storage. I have read an older post in this mailing list > about passing specific mdbox options to doveadm altmove. Should be a > good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d > and mdox_rotate_size=60m and an "alternate area" with bzip9, > mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area > should only be "old" mail (1 year older). I guess it makes sense. > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid > 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? The eventual solution is a Dovecot backup agent, but for now I guess there isn't anything better than taking filesystem snapshots. From tss at iki.fi Mon Feb 27 12:30:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:30:04 +0200 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <1330338604.11500.304.camel@innu> On Mon, 2012-02-27 at 02:11 +0100, ml wrote: > i build the last release of dovecot on my centos 5 > work well and fine > > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } They just need to be "high enough". Dovecot logs a warning at startup if they're not high enough. 6000 is fine. From tss at iki.fi Mon Feb 27 12:32:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:32:10 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4AD9AC.5000300@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Message-ID: <1330338730.11500.306.camel@innu> On Mon, 2012-02-27 at 14:17 +1300, Mark Davies wrote: > Feb 27 13:42:18 bats dovecot: auth: > GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in > continued response Looks like kmail is sending some kind of garbage to Dovecot. Set auth_debug_passwords=yes to make Dovecot log the auth traffic. From Ralf.Hildebrandt at charite.de Mon Feb 27 12:46:40 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 11:46:40 +0100 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <1330333803.11500.295.camel@innu> References: <20120227083917.GD16162@charite.de> <1330333803.11500.295.camel@innu> Message-ID: <20120227104640.GI16162@charite.de> * Timo Sirainen : > On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > > Today I found this in my log: > > > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory > > Large index, process VSZ limit reached. You might need to increase > service imap { vsz_limit }. Indeed. That user has quite a bit of mail (3GB) > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) > > Bugs caused by the above. I don't think I'm going to spend time on Squat > anymore, with v2.1 Lucene backend works better and faster. Too lazy to set that up (yet). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Mon Feb 27 12:50:39 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 23:50:39 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <1330338730.11500.306.camel@innu> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> Message-ID: <4F4B5FFF.9090201@ecs.vuw.ac.nz> On 02/27/12 23:32, Timo Sirainen wrote: > Looks like kmail is sending some kind of garbage to Dovecot. Set > auth_debug_passwords=yes to make Dovecot log the auth traffic. Yeah, I did a network trace and it seems kmail is not sending the full authentication request before trying to carry on. > 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" > 11 0.045486 130.195.5.13 130.195.5.88 IMAP 122 Response: 1 NO [ALERT] Invalid base64 data in continued response now to figure out why its started truncating it. cheers mark From nick at mobilia.it Mon Feb 27 13:06:04 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 12:06:04 +0100 Subject: [Dovecot] LDA alias resolution problems.. Message-ID: <4F4B639C.9040207@mobilia.it> I use postfix + dovecot LDA + mysql virtual users and up til friday, everything was working just fine. Dovecot is 2.0.beta6 (3156315704ef) Like I said, up til friday, when I added a vacation entry into postfix (which I have now removed to see if it was the reason) mails to the alias virusalert (which is aliased to me) went through just fine. Now I get these sort of errors from the LDA: Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: to=, relay=dovecot, delay=357414, delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary failure. Command output: lda: Fatal: Unknown user ) Other aliases work perfectly, the only difference is that these emails are of local origin, (they're messages from Amavisd) I'm not sure why this has started to be a problem. Can anyone give me an idea where I should start looking? Nick From tss at iki.fi Mon Feb 27 13:11:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:11:47 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4B5FFF.9090201@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: On 27.2.2012, at 12.50, Mark Davies wrote: > On 02/27/12 23:32, Timo Sirainen wrote: >> Looks like kmail is sending some kind of garbage to Dovecot. Set >> auth_debug_passwords=yes to make Dovecot log the auth traffic. > > Yeah, I did a network trace and it seems kmail is not sending the > full authentication request before trying to carry on. >> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe It uses SASL-IR to send the first seponse. >> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + Dovecot says "OK, give me more". >> 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" kmail thinks the authentication is done and sends LIST command, even though it should finish the authentication. Clearly a kmail bug. You could try what happens if you remove SASL-IR from Dovecot's imap_capability list. Maybe kmail is buggy only with SASL-IR. From tss at iki.fi Mon Feb 27 13:36:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:36:00 +0200 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: <20120224012247.GA6512@krell.zikzak.de> References: <20120224012247.GA6512@krell.zikzak.de> Message-ID: <1330342560.11500.308.camel@innu> On Fri, 2012-02-24 at 02:22 +0100, Andreas M. Kirchwitz wrote: > There seems to be a new dependency in some modules (eg, lib-storage, > libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those > modules didn't require OpenSSL, but 2.1 does. > > For the linking process the path to the OpenSSL library isn't > specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL > is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't > properly used as well.) Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes everything? You can try a new nightly snapshot from http://dovecot.org/nightly/ once it gets there in a few minutes. From tss at iki.fi Mon Feb 27 13:43:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:43:35 +0200 Subject: [Dovecot] Virtual Folders In-Reply-To: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> References: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Message-ID: <1330343015.11500.313.camel@innu> On Fri, 2012-02-24 at 08:27 -0600, jacob at alltechservices-ia.com wrote: > Essentially, a gmail clone, but one that > I can make work on cPanel hosts with roundcube and squirrel mail, etc. > > By default, cpanel creates user mail dir structure of (maildir++) > inbox, drafts, sent, junk, and trash (with no dovecot specific files > until first login) > > I would like to replace all those folders with > virtual representations with more capabilities. I would like to achieve > the conversation view that gmail has pulled off so well. It's kind of possible to do that, but Dovecot has some problems with updating the view I think. Or maybe it was only when email was deleted that it didn't properly update the view. > namespace: > type: private > prefix: INBOX. > inbox: yes > list: > yes > subscriptions: yes This is your real mail's namespace. > box with Debian running the same version of dovecot (1.2.x) and on that > system I run into problems every time I make a virtual folder a root > folder. I assume it is trying to look up a normal folder instead of the > virtual ones and I am confusing it. namespace { prefix = separator = / location = virtual:/etc/dovecot/virtual:INDEX=~/Maildir/virtual inbox = yes } namespace { prefix = RealMails/ separator = / list = no hidden = yes } This has the problem that user now can't create any mailboxes other than what you have created globally.. There's really no automatic way to do this. From micha at krausam.de Mon Feb 27 13:47:27 2012 From: micha at krausam.de (Micha Krause) Date: Mon, 27 Feb 2012 12:47:27 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> References: <4F47BA30.2080704@krausam.de> <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> Message-ID: <4F4B6D4F.2070708@krausam.de> Hi Timo, > lmtp_save_to_detail_mailbox = yes Works like a charm, thank you. Micha Krause From CMarcus at Media-Brokers.com Mon Feb 27 13:51:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:51:11 -0500 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <4F4B6E2F.7010105@Media-Brokers.com> On 2012-02-26 8:11 PM, ml wrote: > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } Timo already replied that yours should be fine, but I'd just add... You can now use doveconf -d to see what the default values for any setting should be (thanks for adding that Timo!)... and the default values for both of these are 0. Two lessons I learned a long time ago when I first installed and started using postfix: 1. never change a default value/setting without a very good reason 2. never just randomly start changing values of settings in an attempt to fix a problem unless you understand precisely what these changes do So, either you changed them, or the CentOS package that you used changed them (most likely), which is one reason (among many others) that I would never use such systems for mission critical applications. I think Timo, like Wietse (postfix's author) does his best to set sane default values which should not be changed without a good reason. -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 13:58:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:58:29 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B6FE5.3040703@Media-Brokers.com> On 2012-02-27 6:06 AM, Nick Warr wrote: > Dovecot is 2.0.beta6 (3156315704ef) Seriously? You're having problems with an ancient pre-release/beta? Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read all of the upgrading docs on the wiki), then try again... -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 14:25:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 07:25:36 -0500 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F4B7640.5070700@Media-Brokers.com> On 2012-02-22 8:15 PM, Stephan Bosch wrote: > * I've just created an alternative that implements something similar to > the Procmail code you posted above, but from within Sieve itself. It is > a custom language extension called vnd.dovecot.duplicate and it adds the > "duplicate" test. This test keeps track of which Message-IDs it has seen > before in earlier deliveries and yields a true result if the message was > seen before, e.g.: > > require "vnd.dovecot.duplicate"; > > if duplicate { > discard; > } Interesting... I'll have to look into this later to see how it might be leveraged only for deduplicating mailing list messages (suppress duplicate messages when someone is a member of multiple lists that are members of other lists - I use mailman currently). -- Best regards, Charles From tss at iki.fi Mon Feb 27 14:45:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 14:45:09 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: <1330346709.11500.324.camel@innu> On Thu, 2012-02-23 at 20:55 +0100, Michael Grimm wrote: > Hi -- > > On 21.02.2012, at 23:03, Michael Grimm wrote: > > On 21.02.2012, at 22:57, Timo Sirainen wrote: > > >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > > > Ok. Then I will try both servers upgraded to 2.1 within the next days. > > JFTR: It took me some time to find a working syntax ;-) > > My working 2.0.18 syntax threw the following error: > > vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(vmail): Fatal: Unknown print formatter: -u > dsync-local(test): Error: read() from worker server failed: EOF You left out "mirror" from that command, but after adding it the latest hg version works. > Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: > > vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test > doveadm: illegal option -- v > doveadm sync [-u |-A] [-S ] [-fR] [-m ] The -v parameter is in wrong place now, needs to be "doveadm -v sync". > After some trial by error I finally found a working syntax: > > vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test Oh, hmm. I hadn't thought about this problem, it shouldn't have been necessary to give the dsync-server parameter. But I guess there's not a whole lot of other possibilities to do this. Hmm. BTW. I think you can do this simply: doveadm sync -f -u test test at remote-host.tld From nick at mobilia.it Mon Feb 27 16:24:26 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 15:24:26 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B6FE5.3040703@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> Message-ID: <4F4B921A.3080705@mobilia.it> Il 27/02/2012 12.58, Charles Marcus ha scritto: > On 2012-02-27 6:06 AM, Nick Warr wrote: >> Dovecot is 2.0.beta6 (3156315704ef) > > Seriously? You're having problems with an ancient pre-release/beta? > > Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read > all of the upgrading docs on the wiki), then try again... > dovecot --version 2.0.9 Same exact problem. any suggestions? From tom at whyscream.net Mon Feb 27 16:26:13 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 27 Feb 2012 15:26:13 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B9285.1080004@whyscream.net> On 27/02/12 12:06, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. Usually, aliases are processed by postfix, and dovecot only gets e-mail for actual users, not aliases. Depending on how your mail processing flow works, mail generated by amavis might not visit postfix before it hits dovecot... -- Regards, Tom From CMarcus at Media-Brokers.com Mon Feb 27 16:58:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 09:58:34 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B921A.3080705@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> Message-ID: <4F4B9A1A.9070702@Media-Brokers.com> On 2012-02-27 9:24 AM, Nick Warr wrote: > Il 27/02/2012 12.58, Charles Marcus ha scritto: >> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >> all of the upgrading docs on the wiki), then try again... > dovecot --version > 2.0.9 > > Same exact problem. > > any suggestions? Update to the *latest* version - for 2.0, that would be 2.0.18, for 2.1, it is 2.1.1 I'm not necessarily saying this will solve your problems (in fact, in your case it probably won't), but upgrading to the latest version should *always* be step #1 before asking for help (at least in my book)... -- Best regards, Charles From nick at mobilia.it Mon Feb 27 17:10:09 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:10:09 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9A1A.9070702@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> Message-ID: <4F4B9CD1.8000008@mobilia.it> Il 27/02/2012 15.58, Charles Marcus ha scritto: > On 2012-02-27 9:24 AM, Nick Warr wrote: >> Il 27/02/2012 12.58, Charles Marcus ha scritto: >>> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >>> all of the upgrading docs on the wiki), then try again... > >> dovecot --version >> 2.0.9 >> >> Same exact problem. >> >> any suggestions? > > Update to the *latest* version - for 2.0, that would be 2.0.18, for > 2.1, it is 2.1.1 > > I'm not necessarily saying this will solve your problems (in fact, in > your case it probably won't), but upgrading to the latest version > should *always* be step #1 before asking for help (at least in my > book)... > Talk to the CentOS guys for me, and tell em to get on the ball :) From nick at mobilia.it Mon Feb 27 17:13:17 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:13:17 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9285.1080004@whyscream.net> References: <4F4B639C.9040207@mobilia.it> <4F4B9285.1080004@whyscream.net> Message-ID: <4F4B9D8D.7070909@mobilia.it> Il 27/02/2012 15.26, Tom Hendrikx ha scritto: > On 27/02/12 12:06, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. > > Usually, aliases are processed by postfix, and dovecot only gets > e-mail for actual users, not aliases. Depending on how your mail > processing flow works, mail generated by amavis might not visit > postfix before it hits dovecot... > > -- > Regards, > Tom That does seem to be logical, since amavis is generating the email. Normal emails go postfix -> amavis -> postfix -> dovecot These go amavis -> postfix -> dovecot if this is so, I may need to talk to the postfix list, thanks! From CMarcus at Media-Brokers.com Mon Feb 27 17:22:45 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 10:22:45 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9CD1.8000008@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> <4F4B9CD1.8000008@mobilia.it> Message-ID: <4F4B9FC5.6000702@Media-Brokers.com> On 2012-02-27 10:10 AM, Nick Warr wrote: > Il 27/02/2012 15.58, Charles Marcus ha scritto: >> Update to the *latest* version - for 2.0, that would be 2.0.18, for >> 2.1, it is 2.1.1 > Talk to the CentOS guys for me, and tell em to get on the ball :) Don't point fingers - this is why I would never use a distro that locks you into ancient/unsupported versions of mission critical software. That said, there is usually a viable solution - I'm sure there are alternative CentOS repositories that offer up to date versions of things like postfix, dovecot, apache and other mission critical software. Even Debian stable (worst offender for locking you into ancient versions of software) offers such repositories. -- Best regards, Charles From tomislav.mihalicek at gmail.com Mon Feb 27 17:30:01 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Mon, 27 Feb 2012 07:30:01 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue Message-ID: <33400491.post@talk.nabble.com> I have a proxy setup for pop/imap. The proxies are defined in login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 connections displays the ip address of proxy IP... Dovecots are both 1.2 from the debian repo deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-1.2 main thanks -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33400491.html Sent from the Dovecot mailing list archive at Nabble.com. From bind at enas.net Mon Feb 27 17:52:05 2012 From: bind at enas.net (Urban Loesch) Date: Mon, 27 Feb 2012 16:52:05 +0100 Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <33400491.post@talk.nabble.com> References: <33400491.post@talk.nabble.com> Message-ID: <4F4BA6A5.90806@enas.net> Same here on 2.0.x. But I think this is because it's only implemented for IMAP. See e-mail from Timo 2 days ago: ... Subject: Proxying improvements in v2.1.2 I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). .... regards Urban On 27.02.2012 16:30, Tomislav Mihalicek wrote: > > I have a proxy setup for pop/imap. The proxies are defined in > login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 > connections displays the ip address of proxy IP... Dovecots are both 1.2 > from the debian repo deb http://xi.rename-it.nl/debian/ > stable-auto/dovecot-1.2 main > > thanks From joshua at hybrid.pl Mon Feb 27 18:44:06 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 27 Feb 2012 17:44:06 +0100 (CET) Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <20120224181059.66377BF108F@miha> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> <20120224181059.66377BF108F@miha> Message-ID: On Fri, 24 Feb 2012, Miha Vrhovnik wrote: > "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >> On 24.02.2012 15:47, Charles Marcus wrote: >>> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >>> wrote: >>>> Im using Dovecot 1.0.10 >>> Very old - an upgrade would likeley fix your problems... >> A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >> Only inbox is synced automatically. > So the solution is simple go bug Apple. > Oh yeah. they say if something isn't implemented on Apple platform or it > doesn't work, then you don't need it, because their devs know what's > best for their users. Exactly, no hope here :( What's even more annoying, apple doesn't put those idiotic limitations on Exchange accounts - seems they like Microsoft more than free solutions :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Mon Feb 27 18:54:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 11:54:49 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... Message-ID: <4F4BB559.6050405@Media-Brokers.com> Hello all/Timo, Up until now, my main Clients office has consisted of a single location, and I have never had to deal with the situation of multiple locations for a single company. They have just told me that they are acquiring an additional floor at a building that is about 4 minutes away - but obviously far enough away that I now have to deal with supporting users in the same domain but at two disparate physical locations. These two locations will be connected via a private Gb ethernet connection, and each location will have its own internet connection (I think - still waiting on some numbers to present to the owner to see what he wants to do in that regard, but that will be my recommendation), so bandwidth for replication won't be an issue. I have a couple of months to get this done, and I am already planning on hiring Timo's new commercial support company to help with the final and actual design and implementation, but obviously first I need to know what my actual options are. Just a rough idea of what I'd like to do is: 1. Set up one dovecot server at each location (these will be VMs), so users at each location are accessing the local server for that office 2. Full replication between the two for the mail/indexes, and configure them such that each can act as a failover for the other in case one goes down for whatever reason This is my first/main question... I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? Or, if it is going to take more work than my client is willing to pay for (I'm hoping not, since you said it was on the map for 2.1, not 2.2+), maybe the notify plugin could be leveraged in some way to provide something 'close enough' until it is fully implemented in dsync? On that note (something 'close enough' until dsync fully supports this natively), would setting up a dsync cron job, say, every 5 or 10 minutes, be asking for trouble? Our mail server is not all that busy, really, so in 5 or 10 minutes, there wouldn't be many changes at all. If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? and 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Thanks to any/all for reading this far and for any thoughts, suggestions and/or ideas... -- Best regards, Charles From as at 3a.pl Mon Feb 27 19:59:14 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 18:59:14 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: <4F4BC472.4000805@3a.pl> On 27.02.2012 17:54, Charles Marcus wrote: > These two locations will be connected via a private Gb ethernet > connection, and each location will have its own internet connection (I > think - still waiting on some numbers to present to the owner to see > what he wants to do in that regard, but that will be my > recommendation), so bandwidth for replication won't be an issue. [cut] I do have a basic question... How many users will be in this new, remote location? Will the traffic be so vast, that 1GbE link will not be enough, or are you using two servers for reliability? The simpler the configuration, it is almost always the better. Maybe you can stay with one server in yours primary location? -- Adam Szpakowski From rob0 at gmx.co.uk Mon Feb 27 20:12:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Mon, 27 Feb 2012 12:12:55 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <20120227181255.GV24983@harrier.slackbuilds.org> On Mon, Feb 27, 2012 at 06:59:14PM +0100, Adam Szpakowski wrote: > On 27.02.2012 17:54, Charles Marcus wrote: > >These two locations will be connected via a private Gb ethernet > >connection, and each location will have its own internet > >connection (I think - still waiting on some numbers to present to > >the owner to see what he wants to do in that regard, but that will > >be my recommendation), so bandwidth for replication won't be an > >issue. > [cut] > > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will > not be enough, or are you using two servers for reliability? > > The simpler the configuration, it is almost always the better. > Maybe you can stay with one server in yours primary location? This was exactly my thought as reading it. If you have some control over client configuration, use "offline IMAP," where clients maintain a local copy of what's on the server. (That's a good idea anyway, distributed backups of mail which possibly is important.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Mon Feb 27 20:21:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:21:23 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <4F4BC9A3.1010404@Media-Brokers.com> On 2012-02-27 12:59 PM, Adam Szpakowski wrote: > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will not > be enough, or are you using two servers for reliability? Yeah, I guess I should have mentioned this... Each location is an entire floor of a 6 story building. The remote location has the capacity for about 60 users, the new location about 100. We only allow IMAP access to email, so if everyone is using email at the same time, that would be a lot of traffic over a single Gb link I think... > The simpler the configuration, it is almost always the better. Maybe > you can stay with one server in yours primary location? I had considered this, but like I said, this is not purely for performance - I'd also like to get redundancy in the deal (up until now, we haven't had any - admittedly - knock on wood - we haven't needed it, but I'd still like to implement it). -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 20:22:53 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:22:53 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <20120227181255.GV24983@harrier.slackbuilds.org> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <20120227181255.GV24983@harrier.slackbuilds.org> Message-ID: <4F4BC9FD.5030806@Media-Brokers.com> On 2012-02-27 1:12 PM, /dev/rob0 wrote: > If you have some control over client configuration, use "offline > IMAP," where clients maintain a local copy of what's on the server. > (That's a good idea anyway, distributed backups of mail which > possibly is important.) Most of these users have many multiple Gb of email, so keeping local cached copies of all it if is silly, in my opinion... and again, the *main* purpose for the two separate servers is for high availability (redundancy/failover)... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From sven at svenhartge.de Mon Feb 27 20:34:23 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 19:34:23 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <88jj6q741pv8@mids.svenhartge.de> Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > Yeah, I guess I should have mentioned this... > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link > I think... Naa, most clients download mails only once and then keep them cached locally (at least Thunderbird and Outlook do). Looking at the used bandwidth of the mailserver of my small university (10.000 users, about 1000 concurrently active during the daytime) shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and out. Remember: your outgoing bandwidth will be roughly the bandwidth of mail going into the server. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Mon Feb 27 20:38:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:38:39 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <88jj6q741pv8@mids.svenhartge.de> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> Message-ID: <4F4BCDAF.50003@Media-Brokers.com> On 2012-02-27 1:34 PM, Sven Hartge wrote: > Charles Marcus wrote: >> Each location is an entire floor of a 6 story building. The remote >> location has the capacity for about 60 users, the new location about >> 100. We only allow IMAP access to email, so if everyone is using email >> at the same time, that would be a lot of traffic over a single Gb link >> I think... > Naa, most clients download mails only once and then keep them cached > locally (at least Thunderbird and Outlook do). > > Looking at the used bandwidth of the mailserver of my small university > (10.000 users, about 1000 concurrently active during the daytime) > shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and > out. Interesting - thanks for the numbers... But, again, my main reason for 2 servers is not performance, it is for redundancy... -- Best regards, Charles From as at 3a.pl Mon Feb 27 20:48:55 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 19:48:55 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC9A3.1010404@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <4F4BD017.5010806@3a.pl> On 27.02.2012 19:21, Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > > Yeah, I guess I should have mentioned this... > > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link I > think... I'm not sure that the bandwidth will be a problem. One of our clients is a civic design office. Around 60 people and lots of multi megabyte files in multiple copies. Autocad 3D files are flying all around ;). All accounts are IMAP ones, there is also local SAMBA service. The server has 1GbE connection to almost all workstations and the bandwidth utilization is low, very low, on average much less then 1%. We are talking about over 50 heavy duty users. >> The simpler the configuration, it is almost always the better. Maybe >> you can stay with one server in yours primary location? > > I had considered this, but like I said, this is not purely for > performance - I'd also like to get redundancy in the deal (up until > now, we haven't had any - admittedly - knock on wood - we haven't > needed it, but I'd still like to implement it). IMHO use something simple for redundancy such us DRBD in active/passive mode in single location. Manual migration to do not have to deal with split brain problems. As a additional layer of security against local cataclysm (fire in the building) use nightly backup to the second office. You will not have automatic, 99.999% reliability, but for most clients it is ok. They do not need this. The market for highly available, redundant services is quite small. -- Adam Szpakowski From list at airstreamcomm.net Mon Feb 27 22:51:54 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 27 Feb 2012 14:51:54 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BCDAF.50003@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> Message-ID: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> On Mon, 27 Feb 2012 13:38:39 -0500, Charles Marcus wrote: > On 2012-02-27 1:34 PM, Sven Hartge wrote: >> Charles Marcus wrote: >>> Each location is an entire floor of a 6 story building. The remote >>> location has the capacity for about 60 users, the new location about >>> 100. We only allow IMAP access to email, so if everyone is using email >>> at the same time, that would be a lot of traffic over a single Gb link >>> I think... > >> Naa, most clients download mails only once and then keep them cached >> locally (at least Thunderbird and Outlook do). >> >> Looking at the used bandwidth of the mailserver of my small university >> (10.000 users, about 1000 concurrently active during the daytime) >> shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and >> out. > > Interesting - thanks for the numbers... > > But, again, my main reason for 2 servers is not performance, it is for > redundancy... I too have been tasked with multisite redundancy, and have been experimenting with GlusterFS (http://www.gluster.org/community/documentation/index.php/Main_Page), which is a distributed file system. In our network we have a dedicated 10GB link between two datacenters 100 miles apart, and I have a GlusterFS node at each site setup in Distriubted Replicated mode with 2 replicas which means the servers are mirrored. The file writes are done to all the replica servers (2 servers in this case), so depending on network latency the writes could potentially slow down. GlusterFS has it's own file serving protocol that allows automatic and immediate failover in the case that a storage node disappears, but there are some caveats to restoring a failed storage node (takes forever to resync the data). I have not put this experiment into production, but I can say that it's extremely simple to manage, and performance testing has shown that it could handle mail traffic just fine. You could also look at GPFS (http://www-03.ibm.com/systems/software/gpfs/), which is not open source but it's apparently rock solid and I believe supports multisite clustering. From pc8888 at gmail.com Mon Feb 27 23:52:26 2012 From: pc8888 at gmail.com (Pierre Chiu) Date: Mon, 27 Feb 2012 16:52:26 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4BFB1A.1010401@gmail.com> Give this a try. Find out the location of the verify cache file and delete it. This is my setting in main.cf address_verify_map = btree:$data_directory/verify_cache On 2/27/2012 6:06 AM, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. > > Can anyone give me an idea where I should start looking? > > Nick From janfrode at tanso.net Tue Feb 28 01:58:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 00:58:29 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> Message-ID: <20120227235829.GA29841@dibs.tanso.net> On Mon, Feb 27, 2012 at 02:51:54PM -0600, list at airstreamcomm.net wrote: > > You could also look at GPFS > (http://www-03.ibm.com/systems/software/gpfs/), which is not open source > but it's apparently rock solid and I believe supports multisite clustering. GPFS supports different modes of clustering. I think the appropriate solution here would be to deploy a single cluster spanning 3 sites (3. site is needed for quorum node, two sites can't work because you can't protect it from split brain). The simplest config would then be 3 nodes (but you could have any number of nodes at each site): quorum node1 on site1 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node2 on site2 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node3 on site3 The filesystem would be replicated (over IP) between the disk on site1 and site2. Should one site go down, the other site would survive as long as it could still see the quorum node on site3. After a site has been down, one would need to sync up the NSDs (mmrestripefs) to re-establish the replication of any blocks that has been changed while it was down. -jf From jd.beaubien at gmail.com Tue Feb 28 02:14:57 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 19:14:57 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <1330338267.11500.299.camel@innu> References: <1330338267.11500.299.camel@innu> Message-ID: Ok, I had to fix a few things. First of all, I had to declare mail_plugins = fts fts_solr in global file (dovecot.conf). After doing that, running doveadm fts optimize stopped giving me errors. I ran doveadm index -user INBOX (and other folders). I'm assuming this actually did something because java/tomcat6 were really busy for a few mins and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). However, whenever I run a search thru thunderbird (searching the from/to/subject/body fields), I don't see anything related to solr in the dovecot logs (I never see the line you mentioned or anything close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). After playing with the logging level, I manage to see this in the logs: Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from directory: /var/opt/dovecot/lib/dovecot Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so But nothing indicating that solr_plugin is being used. Any idea? i'd really like to be able to clearly see that I'm using solr. Regards, -JD On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > > hi everyone, > > > > However, how can I verify that solr is working properly? > > You should see in logs things like: > > indexer-worker(user at domain): Info: Indexed 1 messages in INBOX > > It should be in Dovecot's info log (see "doveadm log find"). If you > can't find it, see if "doveadm index -u user at domain INBOX" does any > work. > > Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't > give an error. > > > From jd.beaubien at gmail.com Tue Feb 28 03:37:56 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 20:37:56 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: Btw, since 2.1 is out-performing our old solution so badly, I thought I'd drop some numbers. - The old setup takes over ~4.5 seconds to perform our most common operation. - The new setup takes ~0.2 second to perform the same operation on the same dataset. - approx 22 faster, with less than half the storage; zlib is awesome, for the test account 15GB transforms into 6.5GB. Old setup: dovecot 1.0.9 (Athlon X2 processor, standard hdd) New solution: dovecot 2.1, mdbox, zlib, fts_solr (i think solr works, but not sure) (core i7-2600, standard hdd). Thank you very much for the awesome software. On Mon, Feb 27, 2012 at 7:14 PM, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I don't see anything related to solr in the > dovecot logs (I never see the line you mentioned or anything > close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). > > After playing with the logging level, I manage to see this in the logs: > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from > directory: /var/opt/dovecot/lib/dovecot > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so > > But nothing indicating that solr_plugin is being used. Any idea? i'd > really like to be able to clearly see that I'm using solr. > > Regards, > > -JD > > > > > On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > >> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: >> > hi everyone, >> > >> > However, how can I verify that solr is working properly? >> >> You should see in logs things like: >> >> indexer-worker(user at domain): Info: Indexed 1 messages in INBOX >> >> It should be in Dovecot's info log (see "doveadm log find"). If you >> can't find it, see if "doveadm index -u user at domain INBOX" does any >> work. >> >> Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't >> give an error. >> >> >> > From tss at iki.fi Tue Feb 28 04:04:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 04:04:17 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). OK, so it's indexing. > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: telnet loclahost 143 a login user pass b select inbox c search text hello From nick at mobilia.it Tue Feb 28 09:49:10 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 08:49:10 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4BFB1A.1010401@gmail.com> References: <4F4B639C.9040207@mobilia.it> <4F4BFB1A.1010401@gmail.com> Message-ID: <4F4C86F6.9010702@mobilia.it> Il 27/02/2012 22.52, Pierre Chiu ha scritto: > Give this a try. > Find out the location of the verify cache file and delete it. > > This is my setting in main.cf > address_verify_map = btree:$data_directory/verify_cache > > > > On 2/27/2012 6:06 AM, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. >> >> Can anyone give me an idea where I should start looking? >> >> Nick Thanks, though I actually understood where the problem originated after explaining where the email was injected, it was a postfix configuration issue (the vacation script has you configure the amavis line in master.cf to not expand aliases, so it doesn't send multiple vacation replies.. I hadn't considered mail not originating from postfix) , it was just that dovecot was the one who had to deal with the error. Thanks to Tom Hendrikx for the clue. From joshua at hybrid.pl Tue Feb 28 09:50:10 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 08:50:10 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Maan, that's evil :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Tue Feb 28 10:09:01 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 09:09:01 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, Michael Sidoric wrote: >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > timo is quite correct -- consider imap instead, that's why the protocol exists. This rather seems like anti-imap - kind of desperate search for free disk space... IMAP does quite the opposite thing, since users acutally never remove emails :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From nick at mobilia.it Tue Feb 28 10:17:54 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 09:17:54 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4C8DB2.9060302@mobilia.it> Il 28/02/2012 9.09, Jacek Osiecki ha scritto: > On Fri, 24 Feb 2012, Michael Sidoric wrote: > >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have >>> been >>> configured to save copy of messages in the Server. >> timo is quite correct -- consider imap instead, that's why the >> protocol exists. > > This rather seems like anti-imap - kind of desperate search for free > disk space... IMAP does quite the opposite thing, since users acutally > never remove emails :) > > Greetings, I'd suggest this: http://archivemail.sourceforge.net/ You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. From tlx at leuxner.net Tue Feb 28 10:46:00 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:46:00 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4C8DB2.9060302@mobilia.it> References: <4F4C8DB2.9060302@mobilia.it> Message-ID: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Am 28.02.2012 um 09:17 schrieb Nick Warr: > Il 28/02/2012 9.09, Jacek Osiecki ha scritto: >> On Fri, 24 Feb 2012, Michael Sidoric wrote: >> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. > I'd suggest this: > > http://archivemail.sourceforge.net/ > > You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. Assuming this is the task you could also use: $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen Play around with the scope a bit before you actually run it: $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Tue Feb 28 10:53:40 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:53:40 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> References: <4F4C8DB2.9060302@mobilia.it> <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Message-ID: Am 28.02.2012 um 09:46 schrieb Thomas Leuxner: > Assuming this is the task you could also use: > > $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen > > Play around with the scope a bit before you actually run it: > > $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l Of course this was supposed to read 'seen' :/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tomislav.mihalicek at gmail.com Tue Feb 28 11:23:18 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 01:23:18 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405413.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405413.html Sent from the Dovecot mailing list archive at Nabble.com. From dovecot at r.paypc.com Tue Feb 28 12:15:27 2012 From: dovecot at r.paypc.com (Robin) Date: Tue, 28 Feb 2012 02:15:27 -0800 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CA93F.3080100@r.paypc.com> > I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: > > telnet localhost 143 > a login user pass > b select inbox > c search text hello Yes, you definitely want to check things are being accelerated by issuing direct IMAP commands via telnet. Many clients try to "help" by performing local searches, which will only obfuscate things for you. Even with 150K+ messages, it shouldn't take fts_solr more than 20ms or so to give you results. I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. You can check your solr index data directory too. A freshly installed solr index occupies almost no space, but that grows QUICKLY once it's indexed anything. =R= From tomislav.mihalicek at gmail.com Tue Feb 28 12:18:29 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 02:18:29 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405654.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405654.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Tue Feb 28 12:36:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 12:36:18 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CA93F.3080100@r.paypc.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CA93F.3080100@r.paypc.com> Message-ID: <9DBA17BC-FA96-4DC1-A3A3-30DEA66A98B5@iki.fi> On 28.2.2012, at 12.15, Robin wrote: > I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. That reads only in wiki1 docs, in wiki2 docs it should only talk about globals.. I wonder if it would be a bad idea to show some kind of a popup in wiki1 asking if you really want to be reading it or not, and use a cookie to remember it?.. From p at state-of-mind.de Tue Feb 28 13:10:58 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 28 Feb 2012 12:10:58 +0100 Subject: [Dovecot] ANN: automx - automated mail account provisioning Message-ID: <20120228111057.GC27530@state-of-mind.de> automx I am pleased to announce automx a tool that makes setting up a mail account easy. All your users need to provide is real name, mail address and password. Their mail client and automx will safely handle the rest. Say goodbye to mistyped server settings! Put an end to endless phone calls trying to coach users to configure settings, whose dialogs they can't find. Raise user satisfaction, relieve support and regain time to work on other issues. automx unifies Microsofts and Mozillas mail account provisioning standards in one powerful Open Source tool. Choose from many backends, including LDAP and SQL, and let automx create standard and individualized profiles for multiple domains on the fly! See the details on http://automx.org. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3603 bytes Desc: not available URL: From joshua at hybrid.pl Tue Feb 28 13:14:04 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 12:14:04 +0100 (CET) Subject: [Dovecot] Deny create folders In-Reply-To: <20120226172827.112de657@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: On Sun, 26 Feb 2012, Jerry wrote: > On Sun, 26 Feb 2012 22:31:35 +0100 > S. Kremer articulated: >> Every user's mailbox has a directory called Spam and Ham. I scan this >> two folders by a script to learn my spamassassin database. >> This script does not scan folders like spam, ham, junk-email etc. >> But some users have such folders in their mailbox so spamassassin does >> not learn from these folders. [...] > Couldn't you just make a soft link between the two folders and save > yourself and your clientele a lot of trouble? I have not messed with > "spamassassin" in years; however, I thought it was configurable on what > it scans. Oh, it reminds me of one thing I would like to have - maybe it is possible? Some IMAP clients (especially outlooks) create junk/sent/etc. folders with various names, depending on national settings. Is it possible to provide a - let's say - dictionary, which would always return the same folder (like Sent) no matter if client asks for "Sent", "Sent Mail" or "Elementy wys&AUI-ane"? Hmm... Now, when I formed the question it seems that this MIGHT be a problem :) What should dovecot provide when client asks for list of subscribed folders? If I know that there is a polish version of MS Outlook, I should provide "Elementy wys&AUI-ane", but "Sent" if it is roundcube... Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Tue Feb 28 14:54:38 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 07:54:38 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CCE8E.2000103@Media-Brokers.com> On 2012-02-27 9:04 PM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: >> However, whenever I run a search thru thunderbird (searching the >> from/to/subject/body fields), > I think Thunderbird does this search internally, not via IMAP. You could test by opening the Advanced Search window (CTRL-SHIFT-F or Edit > Find > Search Messages), check the box to 'Run search on server', make sure the right account is selected, then perform a search... There are 2 bugs open for Thunderbird dealing with server side IMAP searches, please feel free to go vote for one or both of them: Main one: https://bugzilla.mozilla.org/show_bug.cgi?id=564168 and one that is tangentially related: https://bugzilla.mozilla.org/show_bug.cgi?id=543416 -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 15:28:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 08:28:06 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: <4F4CD666.3040809@Media-Brokers.com> On 2012-02-28 6:14 AM, Jacek Osiecki wrote: > Some IMAP clients (especially outlooks) create junk/sent/etc. > folders with various names, depending on national settings. > Is it possible to provide a - let's say - dictionary, which would always > return the same folder (like Sent) no matter if client asks for "Sent", > "Sent Mail" or "Elementy wys&AUI-ane"? This has been discussed before, this is probably the most recent thread about it, but it is quite long: http://www.dovecot.org/list/dovecot/2011-December/062327.html -- Best regards, Charles From kadafax at gmail.com Tue Feb 28 15:41:01 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 14:41:01 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CD96D.5010706@gmail.com> On 02/28/2012 03:04 AM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > >> Ok, I had to fix a few things. First of all, I had to declare mail_plugins >> = fts fts_solr in global file (dovecot.conf). After doing that, running >> doveadm fts optimize stopped giving me errors. >> >> I ran doveadm index -user INBOX (and other folders). I'm assuming this >> actually did something because java/tomcat6 were really busy for a few mins >> and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > OK, so it's indexing. Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the solr's data directory is growing and log shows things like this: Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages in SubFolder but thunderbird returns nothing (it says not match found for "hello" for example). I dont know what more can I do to debug this... Solr is working as the nobody user, is it OK for dovecot ? Does dovecot need to be have write access to the solr's data directory ? Thanks From janfrode at tanso.net Tue Feb 28 16:02:33 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 15:02:33 +0100 Subject: [Dovecot] fts size Message-ID: <20120228140233.GA18412@dibs.tanso.net> Does anybody have any numbers for how large storage one will need for the fts indexing server? I see the wiki says 30% of mailbox size for Squat (partial=4 full=4). Is it similar for lucene/solr? Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene will create an index for each user in his home directory? Will this be accounted for in the users' quota? -jf From tss at iki.fi Tue Feb 28 16:03:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:03:54 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication Message-ID: <1330437834.2081.2.camel@innu> This document describes a design for a dsync-replicated Dovecot cluster. This design can be used to build at least two different types of dsync clusters, which are both described here. Ville has also drawn overview pictures of these two setups, see http://www.dovecot.org/img/dsync-director-replication.png and http://www.dovecot.org/img/dsync-director-replication-ssh.png First of all, why dsync replication instead of block level filesystem replication? - dsync won't replicate filesystem corruption. - A cold restart of replication won't go through all of the data in the disks, but instead quickly finds out what has changed. - Split brain won't result in downtime or losing any data. If both sides did changes, the changes are merged without data loss. - If using more than 2 storages, the users' replicas can be divided among the other storages. So if one storage goes down, the extra load is shared by all the other storages, not just one. Replication mail plugin ----------------------- This is a simple plugin based on notify plugin. It listens for all changes that happen to mailboxes (new mails, flag changes, etc.) Once it sees a change, it sends an asynchronous (username, priority) notification to replication-notify-fifo. The priority can be either high (new mails) or low (everything else). Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP. So, what is listening in those replication-notify* sockets? It depends on if Dovecot is running on director-based setup or not. Aggregator ---------- When running in Dovecot director-based setup, all of the Dovecot backends (where replication plugin runs) also run "aggregator" process. Its job is very simple: It proxies the notifications from mail plugin and sends them via a single TCP connection to the replicator process running in Dovecot proxies. This is simply an optimization to avoid tons of short lived TCP connections directly from replication plugin to director server. When not running in Dovecot director setup (i.e. there is only a single Dovecot instance that handles all of the users), there is no point in having an aggregator proxy, because the replicator process is running on the same server. In this kind of setup the replicator process directly listens on the replication-notify* sockets. Replicator ---------- The initial design for replicator isn't very complex either: It keeps a priority queue of all users, and replicates those users at the top of the queue. Notifications about changes to user's mailboxes (may) move the user up in the priority queue. If the user at the top of the queue already has been replicated "recently enough", the replicator stops its work until new changes arrive or the "recently enough" is no longer that. dsync can do two types of syncs: quick syncs and full syncs. A quick sync trusts indexes and does the replication with the least amount of work and network traffic. A quick sync is normally enough to replicate all changes, but just in case something has gone wrong there's also the full sync option, which guarantees that the mailboxes end up being fully synced. A full sync is slower though, and uses more network traffic. The priority queue is sorted by: 1. Priority (updated by a notification from replication plugin) 2. If priority!=none: Last fast sync (those users are replicated first whose last replication time is oldest) 2. If priority=none: Last full sync (these users should already be fully synced, but do a full sync for them once in a while anyway) All users get added to the replication queue at replicator startup with "none" priority. The list of users is looked up via userdb iteration. If the previous replication state is found from a disk dump, it's used to update the priorities, last_*_sync timestamps and other replication state. Replicator process creates such dumps periodically [todo: every few mins? maybe a setting?]. Replicator starts replicating users at the top of the queue, setting their priorities to "none" before starting. This means that if another change notification arrives during replication, the priority is bumped up and no changes get lost. replication_max_conns setting specifies how many users are replicated simultaneously. If the user's last_full_sync is older than replication_full_sync_interval setting, a full sync is done instead of a fast sync. If the user at the top of the queue has "none" priority and the last_full_sync is newer than replication_full_sync_interval, the replication stops. [todo: it would be nice to prefer doing all the full syncs at night when there's hopefully less disk I/O] (A global replication_max_conns setting isn't optimal in proxy-based setup, where different backend servers are doing the replication. There it should maybe be a per-backend setting. Then again, it doesn't account for the replica servers that also need to do replication work. Also to properly handle this each backend should have its own replication queue, but this requires doing a userdb lookup for each user to find out their replication server, and this would need to be done periodically in case the backend changes, which can easily happen often with director-based setup. So all in all, none of this is being done in the initial implementation. Ideally the users are distributed in a way that a global replication queue would work well enough.) In director-based setup each director runs a replicator server, but only one of them (master) actually asks the backends to do the replication. The rest of them just keep track of what's happening, and if the master dies or hangs, one of the others becomes the new master. The server with lowest IP address is always the master. The replicators are connected to a ring like the directors, using the same director_servers setting. The communication between them is simply about notifications of what's happening to users' priorities. Preferably the aggregators would always connect to the master server, but this isn't required. In general there's not much that can go wrong, since it's not a problem if two replicators request a backend to start replication for the same user or if the replication queue states aren't identical. If the replication is running too slowly [todo: means what exactly?], log a warning and send an email to admin. So, how does the actual replication happen? Replicator connects to doveadm server and sends a "sync -u user at domain" command. In director-based setup the doveadm server redirects this command to the proper backend. doveadm sync ------------ This is an independent feature from all of the above. Even with none of it implemented, you could run this to replicate a user. Most of this is already implemented. The only problem is that currently you need to explicitly tell it where to sync. So, when the destination isn't specified, it could do a userdb lookup and use the returned "mail_replica" field as the destination. Multiple (sequentially replicated) destinations could be supported by returning "mail_replica2", "mail_replica3" etc. field. In NFS-based (or shared filesystem-based in general) setup the mail_replica setting is identical to mail_location setting. So your primary mail_location would be in /storage1/user/Maildir, while the secondary mail_replica would be in /storage2/user/Maildir. Simple. In non-NFS-based setup two Dovecot servers talk dsync protocol to each others. Currently dsync already supports SSH-based connections. It would also be easy to implement direct TCP-based connections between two doveadm servers. In future these connections could be SSL-encrypted. Initially I'm only supporting SSH-based connections, as they're already implemented. So what does the mail_replica setting look like in this kind of a setup? I'm not entirely sure. I'm thinking that it could be either "ssh:host" or "ssh:user at host", where user is the SSH login user (this is opposite of the current doveadm sync command line usage). In future then it could support also tcp:host[:port]. Both of these ssh: and tcp: prefixes would also be supported by doveadm sync command line usage (and perhaps the prefixless user at domain be deprecated). dsync can run without any long lived locking and it typically works fine. In case mailbox was modified during dsync, the replicas may not end up being identical, but nothing breaks. dsync currently usually notices this and logs a warning. When these conflicting changes was caused by imap/pop3/lda/etc. this isn't a problem, they've already notified replicator already to perform another sync that will fix it. Running two dsyncs at the same time is more problematic though, mainly related to new emails. Both dsyncs notice that mail X needs to be replicated, so both save it and it results in having a duplicate. To avoid this, there should be a dsync-lock. If this lock exists, dsync should wait until the previous dsync is done and then do it again, just in case there were more changes since the previous sync started. This should conclude everything needed for replication itself. High-availability NFS setup --------------------------- Once you have replication, it's of course nice if the system automatically recovers from a broken storage. In NFS-based setups the idea is to do soft mounts, so if the NFS server goes away things start failing with EIO errors, which Dovecot notices and switches to using the secondary storage(s). In v2.1.0 Dovecot already keeps track of mounted filesystems. Initially they're all marked as "online". When multiple I/O errors occur in a filesystem [todo: how many exactly? where are these errors checked, all around in the code or checking the log?] the mountpoint is marked as "offline" and the connections accessing that storage are killed [todo: again how exactly?]. Another job for replication plugin is to hook into namespace creation. If mail_location points to a mountpoint marked as "offline", it's replaced with mail_replica. This way the user can access mails from the secondary storage without downtime. If the replica isn't fully up to date, this means that some of the mails (or other changes) may temporarily be lost. These will come back again after the original storage has come back up and replication has finished its job. So as long as mails aren't lost in the original storage, there won't be any permanent mail loss. When an offline storage comes back online, its mountpoint's status is initially changed to "failover" (as opposed to "online"). During this state the replication plugin works a bit differently when the user's primary mail_location is in this storage: It first checks if the user is fully replicated, and if so uses the primary storage, otherwise it uses the replica storage. Long running IMAP protocesses check the replication state periodically and kill themselves once the user is replicated, to move back to primary storage. Once replicator notices that all users have been replicated, it tells the backends' to change the "failover" state to "online" (via doveadm server). High-availability non-NFS setup ------------------------------- One possibility is to use Dovecot proxies, which know which servers are down. Instead of directing users to those servers, it would direct them to replica servers. The server states could be handled similar to NFS setup's online vs. failover vs. offline states. Another possibility would be to do the same as above, except without separate proxy servers. Just make "mail.example.com" DNS point to two IP addresses, and if one Dovecot notices that it's not the user's primary server, it proxies to the secondary server, unless it's down. If one IP is down, clients hopefully connect to the other. From tss at iki.fi Tue Feb 28 16:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:16:47 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <1330438607.2081.3.camel@innu> On Tue, 2012-02-28 at 14:41 +0100, kfx wrote: > but thunderbird returns nothing (it says not match found for "hello" > for example). Test by talking IMAP protocol, not by using Thunderbird. From CMarcus at Media-Brokers.com Tue Feb 28 16:23:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:23:29 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <4F4CE361.3010705@Media-Brokers.com> On 2012-02-28 8:41 AM, kfx wrote: > Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the > solr's data directory is growing and log shows things like this: > > Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages > in SubFolder > > > but thunderbird returns nothing (it says not match found for "hello" > for example). *How* are you testing with Thunderbird? Did you enable the 'Run search on server' option in the Advanced Search window? Doing this *should* result in Thunderbird using dovecots indexes server side. -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:25:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:25:30 +0200 Subject: [Dovecot] fts size In-Reply-To: <20120228140233.GA18412@dibs.tanso.net> References: <20120228140233.GA18412@dibs.tanso.net> Message-ID: <1330439130.2081.8.camel@innu> On Tue, 2012-02-28 at 15:02 +0100, Jan-Frode Myklebust wrote: > Does anybody have any numbers for how large storage one will need for > the fts indexing server? I see the wiki says 30% of mailbox size for > Squat (partial=4 full=4). Is it similar for lucene/solr? My initial tests for CLucene were that it would take 30% of mailbox size (compared to 50% for Xapian). But this was before I actually implemented it to Dovecot.. I haven't really looked at how large the indexes actually are. Looking at Lucene indexes now for all of Dovecot and Dovecot-cvs mailing list archives, the indexes are 29.1% of the original mbox sizes. > Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene > will create an index for each user in his home directory? Yes. > Will this be accounted for in the users' quota? Only if you're using filesystem quota. From CMarcus at Media-Brokers.com Tue Feb 28 16:27:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:27:56 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330437834.2081.2.camel@innu> References: <1330437834.2081.2.camel@innu> Message-ID: <4F4CE46C.3000507@Media-Brokers.com> On 2012-02-28 9:03 AM, Timo Sirainen wrote: > This document describes a design for a dsync-replicated Dovecot cluster. Wow, talk about timing... this looks like a perfect answer to my previous question about setting up two servers in two different locations to serve two different sets of users locally... Looks like it wouldn't really matter which server they connected to externally, as any changes would simply be replicated. Timo, state above that this 'describes a design'... does this mean that this is doable right now? Which of the following terms would you use to describe it at this point in time? Potentially problematic Reliable Rock-solid Thanks!! -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:43:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:43:38 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4CE46C.3000507@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> Message-ID: <1330440218.2081.14.camel@innu> On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: > On 2012-02-28 9:03 AM, Timo Sirainen wrote: > > This document describes a design for a dsync-replicated Dovecot cluster. > > > > Wow, talk about timing... this looks like a perfect answer to my > previous question about setting up two servers in two different > locations to serve two different sets of users locally... Yeah, I was delaying answering it until I posted this. > Looks like it wouldn't really matter which server they connected to > externally, as any changes would simply be replicated. Well, preferably users would connect to the same server. If both servers have done changes at the same time, there can be some temporary (possibly user-visible) confusion. > Timo, state above that this 'describes a design'... does this mean that > this is doable right now? Which of the following terms would you use to > describe it at this point in time? > > Potentially problematic Not finished yet, so closest to this. :) I'm planning on making our own mails use this within a few weeks (currently it's a dsync in crontab every 5 minutes). From kadafax at gmail.com Tue Feb 28 16:47:53 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 15:47:53 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE361.3010705@Media-Brokers.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> Message-ID: <4F4CE919.8070505@gmail.com> On 02/28/2012 03:23 PM, Charles Marcus wrote: > On 2012-02-28 8:41 AM, kfx wrote: >> Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the >> solr's data directory is growing and log shows things like this: >> >> Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages >> in SubFolder >> >> >> but thunderbird returns nothing (it says not match found for "hello" >> for example). > > *How* are you testing with Thunderbird? > > Did you enable the 'Run search on server' option in the Advanced Search > window? Doing this *should* result in Thunderbird using dovecots indexes > server side. > Yes I did. Some more info: by telnet'ing directly and issuing: c search text pattern * SEARCH 1208 c OK Search completed (0.003 secs). But the same search in thunderbird return "No matches found" :( Here are the solr logs, first search is the "c search text pattern" second is from thunderbird: Feb 28, 2012 3:26:21 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=(hdr:"pattern"+OR+body:"pattern")&rows=108227} hits=1 status=0 QTime=1 Feb 28, 2012 3:27:49 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=body:"pattern"&rows=108227} hits=0 status=0 QTime=1 It seems thunderbird is only searching in the body and not the header unlike the "c search text" method and that the pattern was found in the header of some random email. So none of the two method is working when searching in the body of the emails but work for the headers (???). fts_solr module seems to be correctly launched: Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so [root at server log]# grep -R mail_plugins /usr/local/etc/dovecot/ | grep -v \# /usr/local/etc/dovecot/conf.d/10-mail.conf:mail_plugins = autocreate fts fts_solr quota /usr/local/etc/dovecot/conf.d/15-lda.conf: mail_plugins = $mail_plugins sieve [root at server log]# grep fts /usr/local/etc/dovecot/conf.d/90-plugin.conf fts = solr fts_solr = url=http://127.0.0.1:8983/solr/ break-imap-search Don't know where to search :( From tss at iki.fi Tue Feb 28 16:57:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:57:22 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <1330441042.2081.24.camel@innu> On Tue, 2012-02-28 at 15:47 +0100, kfx wrote: > > Did you enable the 'Run search on server' option in the Advanced Search > > window? Doing this *should* result in Thunderbird using dovecots indexes > > server side. > > > > Yes I did. > > Some more info: > > by telnet'ing directly and issuing: > c search text pattern > * SEARCH 1208 > c OK Search completed (0.003 secs). So, Solr in Dovecot works perfectly. > But the same search in thunderbird return "No matches found" :( Thunderbird problem, nothing you can do about it from Dovecot's side. From rago at lal.in2p3.fr Tue Feb 28 18:11:55 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 16:11:55 +0000 Subject: [Dovecot] Master user In-Reply-To: <4F4BB687.3020701@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> Message-ID: <4F4CFCCB.8060903@lal.in2p3.fr> Hi, I'd like to have a master user that can read the other users mailboxes. I have only one user who read and write the mail (i.e. I use the static userdb) and I use namespace for public and shared mailboxes. I can authenticate to the server but I am not able to list the mailboxes: > emiliano at dovecot:~$ telnet localhost imap > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. > 1 login user*master password > 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in > 5 LIST "" * > * LIST (\Unmarked) "." "INBOX" > * LIST (\Noselect \HasNoChildren) "." "shared" > * LIST (\Noselect \HasChildren) "." "public" > 5 OK List completed. > * BYE Server shutting down. > Connection closed by foreign host. This is my conf: > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) ext4 > auth_master_user_separator = * > disable_plaintext_auth = no > mail_location = maildir:/data/MAIL/MAILDIR/%u:INBOX=/data/MAIL/INBOX/%u:INDEX=/data/MAIL/METADATA/%u > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > list = yes > location = maildir:/data/MAIL/MAILDIR/%%u:INDEX=/data/MAIL/MAILDIR/shared/%u/%%u > prefix = shared.%%u. > separator = . > subscriptions = no > type = shared > } > namespace { > list = yes > location = maildir:/data/MAIL/SHARED > prefix = public. > separator = . > subscriptions = no > type = public > } > passdb { > driver = pam > } > passdb { > args = /etc/dovecot/master-shared > driver = passwd-file > master = yes > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_anyone = allow > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db > } > protocols = imap > service auth { > unix_listener auth-userdb { > group = mailreader > mode = 0600 > user = mailreader > } > } > ssl = no > ssl_cert = ssl_key = userdb { > args = uid=mailreader gid=mailreader home=/ allow_all_users=yes > driver = static > } > protocol lda { > mail_plugins = acl > } > protocol imap { > mail_plugins = acl imap_acl > } And in /etc/dovecot/master-shared: > master:{PLAIN}password What I did wrong? Thanks, Emiliano From tss at iki.fi Tue Feb 28 17:37:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 17:37:22 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4CFCCB.8060903@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> Message-ID: <1330443442.2081.29.camel@innu> On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > I'd like to have a master user that can read the other users mailboxes. > I have only one user who read and write the mail (i.e. I use the static > userdb) and I use namespace for public and shared mailboxes. > I can authenticate to the server but I am not able to list the mailboxes: With ACL plugin enabled, the master user's ACLs are used. And your master user has no ACLs, so nothing is shown. The proper fix for this would be to set a default ACL for the master user, but default ACLs don't exist yet. So you'll have to kludge this, by changing the master user back to the login username. Try if this works: userdb { args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u driver = static } Also don't set your home to /. Better not to set it at all. http://wiki2.dovecot.org/VirtualUsers/Home From kadafax at gmail.com Tue Feb 28 18:05:11 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 17:05:11 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <4F4CFB37.2080505@gmail.com> Ok I feel ashame... it was a third party init scrip who was the problem :( Sorry for the noise and thank you for dovecot From CMarcus at Media-Brokers.com Tue Feb 28 18:28:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:28:15 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CFB37.2080505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> <4F4CFB37.2080505@gmail.com> Message-ID: <4F4D009F.7000107@Media-Brokers.com> On 2012-02-28 11:05 AM, kfx wrote: > Ok I feel ashame... it was a third party init scrip who was the problem :( > > Sorry for the noise and thank you for dovecot So... you're saying that Thunderbird now correctly uses server side search? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 18:33:28 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:33:28 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? Message-ID: <4F4D01D8.7000704@Media-Brokers.com> Hey Timo, Subject says it all... This question is a result of an ongoing discussion on the mozilla enterprise list... Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search (with indexes)? Thanks, -- Best regards, Charles From tss at iki.fi Tue Feb 28 18:38:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:38:32 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D01D8.7000704@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.33, Charles Marcus wrote: > This question is a result of an ongoing discussion on the mozilla enterprise list... > > Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search All IMAP servers are required to support SEARCH command. Some crappy ones don't, but I think all widely used ones do. > (with indexes)? No way to know that. Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. (I don't know if there are any other servers besides Dovecot implementing FUZZY.) From tss at iki.fi Tue Feb 28 18:51:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:51:57 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.38, Timo Sirainen wrote: > Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. I meant SEARCH=FUZZY > And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. v2.1.2 will no longer advertise it unless fts=solr or fts=lucene: http://hg.dovecot.org/dovecot-2.1/rev/bdc881838b00 From steve.platt at mrc-bsu.cam.ac.uk Tue Feb 28 19:45:49 2012 From: steve.platt at mrc-bsu.cam.ac.uk (Steve Platt) Date: Tue, 28 Feb 2012 17:45:49 +0000 Subject: [Dovecot] migrating/converting from system users -> virtual users Message-ID: Hello all, We currently have a traditional mail server where all users have system accounts (ie entries in the NIS passwd map) and mbox-format mail folders in their (system) home directories. I'm trying to setup a dovecot server in which we want all users to have "virtual" accounts (in dovecot) and no entry in the passwd file at all (ie no access to the mail server). This is for security reasons, we want to imagine that users connecting to the mail server in future will not be able to access any resources on the server other than access their email messages. To this end we've set up a "vmail" user that owns all the messages in the Maildir directories. Most of this is working but I'm stuck on how to convert users' mail folders from the existing setup to the new one. I'm using the convert plugin but of course the problem is that the plugin executes as the "vmail" user and cannot access the existing mail folders that belong to the users: and I'd be worried if it could, of course! So I've converted a few users already by making copies of their old-world mailboxes, changing the ownership of the copies and pointing the convert plugin to these copies, instead of the originals. This works (modulo the dreaded "timeouted" error messages) but is very ugly. So I tried looking at the convert-tool but can't find a man page for it so am not sure what I'm doing with it! I have the idea that I should be able to run some command (as a privileged user) on the mail server and have it do the conversion for me, changing the ownership/permissions on the way. Can convert-tool do this? I'd prefer to go with the automatic (plugin) conversion if I can bodge the ownership issues somehow. Failing that, some tool or script may be the next best answer. This is Dovecot 1.2 by the way on Solaris 10. Any suggestions would be welcome! Steve Platt From list at airstreamcomm.net Tue Feb 28 20:11:29 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 12:11:29 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup Message-ID: We are considering using the post-login scripting to clear trash older than 90 days from user accounts. has anyone done this, and if so did this cause logins to slow down too much waiting for the trash to purge? One idea was to execute the trash purge script once a day by tracking their logins and seeing that it has already ran that day. Another idea was to call the trash purge script in the background and continue without acknowledging that it has finished to keep logins speedy. From robert at schetterer.org Tue Feb 28 20:26:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 28 Feb 2012 19:26:11 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D1C43.5000209@schetterer.org> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. look here if this match/solve your problem http://wiki2.dovecot.org/Plugins/Expire -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From CMarcus at Media-Brokers.com Tue Feb 28 20:29:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 13:29:32 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: <4F4D1D0C.7040708@Media-Brokers.com> On 2012-02-28 11:51 AM, Timo Sirainen wrote: > On 28.2.2012, at 18.38, Timo Sirainen wrote: > >> Well, okay, actually if server advertises FUZZY extension you can >> be quite certain that it supports indexed server side searches. > > I meant SEARCH=FUZZY Thanks for the replies Timo... Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Thanks again... hopefully something will comes of this for Thunderbird. -- Best regards, Charles From tss at iki.fi Tue Feb 28 20:31:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:31:46 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D1D0C.7040708@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> <4F4D1D0C.7040708@Media-Brokers.com> Message-ID: <70448DE8-056A-4453-BC45-79C82C781C78@iki.fi> On 28.2.2012, at 20.29, Charles Marcus wrote: > Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Neither. Probably no servers besides Dovecot. But it is less than a year old RFC.. From rago at lal.in2p3.fr Tue Feb 28 20:45:14 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 19:45:14 +0100 Subject: [Dovecot] Master user In-Reply-To: <1330443442.2081.29.camel@innu> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> Message-ID: <4F4D20BA.3060501@lal.in2p3.fr> On 02/28/2012 04:37 PM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > >> I'd like to have a master user that can read the other users mailboxes. >> I have only one user who read and write the mail (i.e. I use the static >> userdb) and I use namespace for public and shared mailboxes. >> I can authenticate to the server but I am not able to list the mailboxes: > > With ACL plugin enabled, the master user's ACLs are used. And your > master user has no ACLs, so nothing is shown. The proper fix for this > would be to set a default ACL for the master user, but default ACLs > don't exist yet. So you'll have to kludge this, by changing the master > user back to the login username. Try if this works: > > userdb { > args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u > driver = static > } Thanks, but nothing has changed. Emiliano From Ralf.Hildebrandt at charite.de Tue Feb 28 20:46:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 19:46:20 +0100 Subject: [Dovecot] 2.1.1: subscriptions: Removing invalid entry Message-ID: <20120228184619.GA3250@charite.de> Admittedly, it's not a valid folder name and dovecot is right in removing it: Feb 28 17:50:05 postamt dovecot: imap(kbaessle): Warning: Subscriptions file /home/k/b/kbaessle/Maildir/subscriptions: Removing invalid entry: Entw?rfe ...but when looking at the subscription file AFTER the removal, that file contained an empty line. I don't think that's correct. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 28 20:47:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:47:37 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D20BA.3060501@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> Message-ID: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> On 28.2.2012, at 20.45, Emiliano Rago wrote: >> userdb { >> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >> driver = static >> } > > Thanks, > but nothing has changed. Set auth_debug=yes. What does it log when you login? From Ralf.Hildebrandt at charite.de Tue Feb 28 21:02:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 20:02:24 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 Message-ID: <20120228190224.GB3250@charite.de> With 2.0.18 I was using a quota warning script like this: plugin { ... # Quota quota = maildir quota_rule = INBOX.Trash:storage=+2048M quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u } # der schickt die Quota warnmails service quota-warning { executable = script /usr/local/scripts/quota-warning2.1 user = root unix_listener quota-warning { mode = 0666 user = vmail group = users } } -rwxr-xr-x 1 root staff 3081 Feb 28 16:56 /usr/local/scripts/quota-warning2.1 But now I'm gettting this in my log: Feb 28 19:19:03 postamt quota-warning2(phxxxpsu): writing message failed Feb 28 19:19:03 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 15065 returned error 1 Feb 28 19:40:13 postamt quota-warning2(lxxxve): writing message failed Feb 28 19:40:13 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 3318 returned error 1 What is failing here? #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit some warning in german EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de -------------- next part -------------- #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Was ist los? ============ Wir haben festgestellt, dass Ihre Mailbox auf dem Server postamt.charite.de das erlaubte Gr?ssenlimit zu mindestens ${MAIL_QUOTA_PERCENT}% aussch?pft. D.h. die maximal erlaubte Gesamtgr??e der Mailbox wird bald erreicht sein oder ist es sogar schon! Was kann ich nun tun? ===================== Bitte schaffen Sie Platz in Ihrer Mailbox, indem Sie die Mails z.B. auf dem Server l?schen oder in Ihre lokale Mailbox auf Ihrem eigenen Rechner verschieben und so wieder Platz auf dem Server schaffen! Wie kann ich die F?llung meines Postfaches auf dem Server ?berpr?fen? ===================================================================== Sie k?nnen die F?llung Ihres Postfaches ?ber Webmail https://webmail.charite.de/ leicht ?berpr?fen: Nach dem Anmelden erscheint im linken Rahmen unter Ihrer Ordnerliste ein Link namens "Ordnergr??en". Sie k?nnen in Webmail ggf. auch gleich die Mails l?schen (vergessen Sie hinterher nicht, den Papierkorb / Trash Ordner zu leeren!). Warum ist mein Postfach ?berhaupt so voll? ========================================== Dies kann z.B. passieren, wenn Sie mehrere Rechner nutzen um Ihre Mail zu lesen und statt des IMAP Protokolls das POP3-Protokoll mit der aktivierten Option "Leave Mail on Server" / "Mail auf Server belassen" in Ihrem Mailprogramm verwenden. Oft wird auch vergessen, da? der Ordner mit den "Gesendeten Objekten" (Sent Folder/Sent Items) zum Gr??enlimit der Mailbox z?hlen. Besondere Platzfresser sind Mails mit gro?en Anh?ngen, wie z.B. PowerPoint-, PDF- oder Bilddateien -- insbesondere im "Sent" Ordner! Wer sind meine Ansprechpartner? =============================== Sie k?nnen uns unter postmaster at charite.de oder telefonisch unter 450-570600 erreichen. Mit freundlichen Gr??en EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log From rago at lal.in2p3.fr Tue Feb 28 21:04:23 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 20:04:23 +0100 Subject: [Dovecot] Master user In-Reply-To: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> Message-ID: <4F4D2537.7070507@lal.in2p3.fr> On 02/28/2012 07:47 PM, Timo Sirainen wrote: > On 28.2.2012, at 20.45, Emiliano Rago wrote: > >>> userdb { >>> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >>> driver = static >>> } >> >> Thanks, >> but nothing has changed. > > Set auth_debug=yes. What does it log when you login? That's the log: Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60873#011resp= Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: auth(admin,127.0.0.1,master): Master user lookup for login: rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: passwd-file(admin,127.0.0.1,master): lookup: user=admin file=/etc/dovecot/master-shared Feb 28 19:57:48 oneprivvm-239 dovecot: auth: passdb(admin,127.0.0.1,master): Master user logging in as rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client out: OK#0111#011user=rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master in: REQUEST#0113607363585#0112591#0111#0114d6e591e980edb0fabccfa9c1cd4e198 Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master out: USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Feb 28 19:57:48 oneprivvm-239 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2596, secured Thanks for answering, Emiliano From tss at iki.fi Tue Feb 28 21:09:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 21:09:02 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D2537.7070507@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> Message-ID: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> On 28.2.2012, at 21.04, Emiliano Rago wrote: > USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting From list at airstreamcomm.net Tue Feb 28 21:26:35 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 13:26:35 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <4F4D1C43.5000209@schetterer.org> References: <4F4D1C43.5000209@schetterer.org> Message-ID: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer wrote: > Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >> We are considering using the post-login scripting to clear trash older >> than 90 days from user accounts. has anyone done this, and if so did >> this >> cause logins to slow down too much waiting for the trash to purge? One >> idea was to execute the trash purge script once a day by tracking their >> logins and seeing that it has already ran that day. Another idea was to >> call the trash purge script in the background and continue without >> acknowledging that it has finished to keep logins speedy. > > look here if this match/solve your problem > > http://wiki2.dovecot.org/Plugins/Expire Expire looks to be useful, but it appears to be something that enhances expunging of messages rather than automates the process if I am reading correctly. We would like to make the process for expunging old Trash messages as automated and inline as possible. From lists at wildgooses.com Tue Feb 28 21:56:22 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 28 Feb 2012 19:56:22 +0000 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D3166.90809@wildgooses.com> On 28/02/2012 18:11, list at airstreamcomm.net wrote: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. I think you can also use doveadm to achieve this? So you could schedule something for all accounts at some out of hours period - should speed up backups also? Ed W From CMarcus at Media-Brokers.com Tue Feb 28 22:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 15:10:12 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330440218.2081.14.camel@innu> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> Message-ID: <4F4D34A4.6090106@Media-Brokers.com> On 2012-02-28 9:43 AM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: >> On 2012-02-28 9:03 AM, Timo Sirainen wrote: >>> This document describes a design for a dsync-replicated Dovecot cluster. >> >> >> >> Wow, talk about timing... this looks like a perfect answer to my >> previous question about setting up two servers in two different >> locations to serve two different sets of users locally... > Yeah, I was delaying answering it until I posted this. Cool, thanks - I look forward to any comments you might have on my current plan. >> Looks like it wouldn't really matter which server they connected to >> externally, as any changes would simply be replicated. > Well, preferably users would connect to the same server. If both servers > have done changes at the same time, there can be some temporary > (possibly user-visible) confusion. Point taken - and thinking about it, the amount of traffic would be negligible, since users accessing emails remotely is much less than internally. But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. >> Timo, state above that this 'describes a design'... does this mean that >> this is doable right now? Which of the following terms would you use to >> describe it at this point in time? >> >> Potentially problematic > Not finished yet, so closest to this. :) I'm planning on making our own > mails use this within a few weeks (currently it's a dsync in crontab > every 5 minutes). Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Thanks! -- Best regards, Charles From Ralf.Hildebrandt at charite.de Tue Feb 28 23:30:21 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 22:30:21 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 In-Reply-To: <20120228190224.GB3250@charite.de> References: <20120228190224.GB3250@charite.de> Message-ID: <20120228213021.GG3250@charite.de> * Ralf Hildebrandt : > With 2.0.18 I was using a quota warning script like this: Must have been a PEBKAC - I fixed my installation by harmonising $PREFIX (seperate $PREFIX for 2.0 and 2.1, removig an old installation along the way) as well as cleaning up some old scripts and alas: Feb 28 22:13:15 postamt quota-warning2(lscxxxk) 90%: delivered! Feb 28 22:15:40 postamt quota-warning2(sxxxwart) 85%: delivered! Feb 28 22:17:39 postamt quota-warning2.1(sxxxwart) 90%: delivered! Feb 28 22:27:55 postamt quota-warning2.1(rxxxid) 85%: delivered! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From rago at lal.in2p3.fr Wed Feb 29 03:06:37 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Wed, 29 Feb 2012 02:06:37 +0100 Subject: [Dovecot] Master user In-Reply-To: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> Message-ID: <4F4D7A1D.9040002@lal.in2p3.fr> On 02/28/2012 08:09 PM, Timo Sirainen wrote: > On 28.2.2012, at 21.04, Emiliano Rago wrote: > >> USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin > > Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting > Great, with postlogin script it works! Thanks a lot, Emiliano From amk at spamfence.net Wed Feb 29 03:03:58 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Wed, 29 Feb 2012 01:03:58 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <20120224012247.GA6512@krell.zikzak.de> <1330342560.11500.308.camel@innu.invalid> Message-ID: Timo Sirainen wrote: >> There seems to be a new dependency in some modules (eg, lib-storage, >> libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those >> modules didn't require OpenSSL, but 2.1 does. >> >> For the linking process the path to the OpenSSL library isn't >> specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL >> is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't >> properly used as well.) > > Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes > everything? That's the way to go. Makes things better, but I've found three more dependencies. This patch is against the daily snapshot 20120228. (Sorry for the changes to Makefile.in which you won't need. But this way I don't have to rebuild Makefile.in from Makefile.am when compiling. ;-) diff -ur dovecot-20120228.original/src/auth/Makefile.am dovecot-20120228/src/auth/Makefile.am --- dovecot-20120228.original/src/auth/Makefile.am 2012-02-26 04:00:02.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.am 2012-02-29 01:38:41.000000000 +0100 @@ -167,7 +167,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ -I$(top_srcdir)/src/lib-imap \ diff -ur dovecot-20120228.original/src/auth/Makefile.in dovecot-20120228/src/auth/Makefile.in --- dovecot-20120228.original/src/auth/Makefile.in 2012-02-28 04:00:12.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.in 2012-02-29 01:38:44.000000000 +0100 @@ -499,7 +499,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ diff -ur dovecot-20120228.original/src/lib-storage/Makefile.am dovecot-20120228/src/lib-storage/Makefile.am --- dovecot-20120228.original/src/lib-storage/Makefile.am 2012-02-10 04:00:02.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -85,7 +85,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/lib-storage/Makefile.in dovecot-20120228/src/lib-storage/Makefile.in --- dovecot-20120228.original/src/lib-storage/Makefile.in 2012-02-28 04:00:15.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -397,7 +397,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic test_programs = \ diff -ur dovecot-20120228.original/src/login-common/Makefile.am dovecot-20120228/src/login-common/Makefile.am --- dovecot-20120228.original/src/login-common/Makefile.am 2011-11-09 04:00:01.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -40,6 +40,6 @@ pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/login-common/Makefile.in dovecot-20120228/src/login-common/Makefile.in --- dovecot-20120228.original/src/login-common/Makefile.in 2012-02-28 04:00:18.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -320,7 +320,7 @@ pkginc_lib_HEADERS = $(headers) pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic all: all-am From robert at schetterer.org Wed Feb 29 08:44:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 29 Feb 2012 07:44:54 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> References: <4F4D1C43.5000209@schetterer.org> <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> Message-ID: <4F4DC966.4000609@schetterer.org> Am 28.02.2012 20:26, schrieb list at airstreamcomm.net: > On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer > wrote: >> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >>> We are considering using the post-login scripting to clear trash older >>> than 90 days from user accounts. has anyone done this, and if so did >>> this >>> cause logins to slow down too much waiting for the trash to purge? One >>> idea was to execute the trash purge script once a day by tracking their >>> logins and seeing that it has already ran that day. Another idea was > to >>> call the trash purge script in the background and continue without >>> acknowledging that it has finished to keep logins speedy. >> >> look here if this match/solve your problem >> >> http://wiki2.dovecot.org/Plugins/Expire > > Expire looks to be useful, but it appears to be something that enhances > expunging of messages rather than automates the process if I am reading > correctly. We would like to make the process for expunging old Trash > messages as automated and inline as possible. wouldnt doveadm expunge -A mailbox Trash savedbefore 90d as cron job do what you want ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From busseniu at in.tum.de Wed Feb 29 11:31:38 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 10:31:38 +0100 Subject: [Dovecot] doveadm fetch prints duplicate results in 2.1 Message-ID: <4F4DF07A.7020408@in.tum.de> Hi, when the private namespace has "prefix = INBOX." and you use doveadm fetch to search for "mailbox INBOX", then it prints every message twice: # doveadm fetch -u testuser 'mailbox uid guid' mailbox INBOX mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 Without "prefix = INBOX." in the configuration, the messages are only printed once. In 2.0, they were never printed twice. doveconf -n # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mdbox mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From michel at linuxint.com Wed Feb 29 09:53:05 2012 From: michel at linuxint.com (Michel Bisson) Date: Wed, 29 Feb 2012 07:53:05 +0000 (UTC) Subject: [Dovecot] per-user limit? References: <20120116091521.GA10944@gir.theapt.org> <1326814211.11500.11.camel@innu> Message-ID: Timo Sirainen iki.fi> writes: > On Mon, 2012-01-16 at 10:15 +0100, Peter Hessler wrote: > > I am seeing a problem where users are limited to 6 imap logins total. > > One of my users has a bunch of phones and computers, and wants them all > > on at the same time. > > I'm looking through my configuration, and I cannot see a limit on how > > many times a single user can connect. He is connecting from different > > IPs. > > Any ideas? My logs show the following error when they attempt to auth > > for a 7th time: Hi, I also get the same problem of the same user getting a password mismatch error when may devices are tryibg to login to the same IMAP account at the same time. I didn't check how manylogins triggers this nasty behavior but noticed that this unwanted feature started right after I upgraded the the server from Debian Lenny to Squeeze. If anybody can point me to a hint about how I coukd change this behavior it woul be very much appreciated. From busseniu at in.tum.de Wed Feb 29 12:03:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:03:35 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> Message-ID: <4F4DF7F7.8020405@in.tum.de> On 21.02.2012 01:18, Timo Sirainen wrote: > On 20.2.2012, at 19.30, Metro Domain Admin wrote: > >> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Yes, but it should still work.. As far as I can tell from my tests, squat has stopped working in 2.1. Let's take a user who does not have any mails yet, and deliver a first mail to him: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now create an IMAP session, select INBOX, and do a search: * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1330509552] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. >>> . search text test * SEARCH . OK Search completed (0.006 secs). (Should have returned 1 message.) Deliver the same message again: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now in the existing session, run the search command three more times: >>> . search text test * SEARCH * 2 EXISTS * 2 RECENT . OK Search completed (0.000 secs). >>> . search text test * SEARCH 2 . OK Search completed (0.002 secs). >>> . search text test * SEARCH . OK Search completed (0.000 secs). It found the message once, but the next time it didn't. So the squat search does not actually seem to work any more. I know it's deprecated, I just wanted to note this. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 29 12:05:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:05:28 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <4F4DF868.3090107@in.tum.de> On 29.02.2012 11:03, Christoph Bu?enius wrote: > It found the message once, but the next time it didn't. So the squat > search does not actually seem to work any more. Sorry, I forgot to include the configuration: # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = fts fts_squat mail_uid = vmail namespace { inbox = yes location = prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat fts_squat = partial=4 full=4 } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From Ralf.Hildebrandt at charite.de Wed Feb 29 12:09:57 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:09:57 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <20120229100957.GX13045@charite.de> * Christoph Bu?enius : > On 21.02.2012 01:18, Timo Sirainen wrote: > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. What else do I need for fts_lucene to work? Any 3rd party packages? Libs? Compile Time Options? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Wed Feb 29 12:15:52 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Wed, 29 Feb 2012 23:15:52 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: <4F4DFAD8.8040002@ecs.vuw.ac.nz> On 02/28/12 00:11, Timo Sirainen wrote: >>> Looks like kmail is sending some kind of garbage to Dovecot. Set >>> auth_debug_passwords=yes to make Dovecot log the auth traffic. >> >> Yeah, I did a network trace and it seems kmail is not sending the >> full authentication request before trying to carry on. > >>> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > > It uses SASL-IR to send the first seponse. > >>> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > > Dovecot says "OK, give me more". I poked some more at the kmail end of this but I cant see what its doing differently from what it used to (but clearly there is something). The new kmail sends 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n and receives + \r\n from dovecot, which it doesn't like and reports clientAuthenticate: sasl_client_step failed with: -1 an older (working kmail) sends 1 AUTHENTICATE GSSAPI YIICiAYJKoZIhvcSAQICAQBugg[...]MpPurY7cZfRSEw==\r\n and receives + YIGaBgkqhkiG9xIBAgI[...]iYoSGi9/uKVGyE64TAvkf25rCbFkNqk1D12g==\r\n and carries on. So what is it that differs in the two cases to cause dovecot to respond differently? cheers mark From mstevens at imt-systems.com Wed Feb 29 12:22:06 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 11:22:06 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: On 29.02.2012 11:09, Ralf Hildebrandt wrote: > * Christoph Bu?enius : >> On 21.02.2012 01:18, Timo Sirainen wrote: >> >On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> > >> >>Squat is apparently deprecated: >> http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Build it with lucene support. --with-lucene Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 12:22:50 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:22:50 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: <20120229102250.GY13045@charite.de> * Ralf Hildebrandt : > * Christoph Bu?enius : > > On 21.02.2012 01:18, Timo Sirainen wrote: > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Rebuilding with --with-lucene --with-stemmer, let's see what happens. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 12:34:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:34:44 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4DFAD8.8040002@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.15, Mark Davies wrote: > 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n > > and receives > > + \r\n > > from dovecot, which it doesn't like and reports > clientAuthenticate: sasl_client_step failed with: -1 Oh, yes, this is a bug in Dovecot.. From tss at iki.fi Wed Feb 29 12:41:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:41:45 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.34, Timo Sirainen wrote: > On 29.2.2012, at 12.15, Mark Davies wrote: > >> 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n >> >> and receives >> >> + \r\n >> >> from dovecot, which it doesn't like and reports >> clientAuthenticate: sasl_client_step failed with: -1 > > Oh, yes, this is a bug in Dovecot.. Hmm. Or it looked like a bug, since it replied only with "+", so I thought all auth mechanisms would have such a bug, but no.. So I'm not really sure why it's not sending more data. I don't have a Kerberos setup to test this with. v2.1's GSSAPI code is anyway identical to v2.0's. From tss at iki.fi Wed Feb 29 12:53:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:53:55 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <8F350712-55B7-457D-BB66-DD1E2CC536C8@iki.fi> On 29.2.2012, at 12.03, Christoph Bu?enius wrote: > On 21.02.2012 01:18, Timo Sirainen wrote: >> On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> >>> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html >> >> Yes, but it should still work.. > > As far as I can tell from my tests, squat has stopped working in 2.1. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ee6f3f375dd3 From piotr.tarnowski at us.edu.pl Wed Feb 29 14:57:35 2012 From: piotr.tarnowski at us.edu.pl (Piotr Tarnowski) Date: Wed, 29 Feb 2012 13:57:35 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) Message-ID: <4F4E20BF.8040707@us.edu.pl> Hi, I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but when I want to compile Sieve for this version (dovecot-2.1-pigeonhole-0.3.0) I get an error: "sieve.c", line 578: identifier redeclared: sieve_multiscript_run and I see that declaration in sieve.h is different from definition in sieve.c, so compilation process fails with SunStudio cc compiler. I have done a quick hack to correct this, and then compiled successfully. Patch is included as attachment. Full output, OS and compiler versions below: libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I/home/soft/dovecot/dovecot-2.1.1 -I/home/soft/dovecot/dovecot-2.1.1/src/lib -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dict -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dns -I/home/soft/dovecot/dovecot-2.1.1/src/lib-mail -I/home/soft/dovecot/dovecot-2.1.1/src/lib-imap -I/home/soft/dovecot/dovecot-2.1.1/src/lib-fs -I/home/soft/dovecot/dovecot-2.1.1/src/lib-charset -I/home/soft/dovecot/dovecot-2.1.1/src/lib-index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index/raw -I/home/soft/dovecot/dovecot-2.1.1/src/lib-lda -I/home/soft/dovecot/dovecot-2.1.1/src/lda -I/home/soft/dovecot/dovecot-2.1.1/src/lib-master -I/home/soft/dovecot/dovecot-2.1.1/src/lib-settings -DMODULEDIR=\"/usr/local/dovecot-2.1.1/lib/dovecot\" -I/usr/local/openldap/include -I/usr/local/ssl/include -I/usr/local/include -xO3 -xautopar -c sieve.c -KPIC -DPIC -o .libs/sieve.o "sieve-common.h", line 33: warning: useless declaration "/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "sieve.c", line 578: identifier redeclared: sieve_multiscript_run current : function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, unsigned int, _Bool) returning _Bool previous: function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, enum sieve_runtime_flags {SIEVE_RUNTIME_FLAG_NOGLOBAL(1)}, _Bool) returning _Bool : "sieve.h", line 168 cc: acomp failed for sieve.c *** Error code 1 dmake: Fatal error: Command failed for target `sieve.lo' Current working directory /home/soft/dovecot/dovecot-2.1-pigeonhole-0.3.0/src/lib-sieve *** Error code 1 The following command caused the error: fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='plugins'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && dmake $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ dmake "$target-am" || exit 1; \ fi; test -z "$fail" dmake: Fatal error: Command failed for target `all-recursive' % uname -a SunOS soft 5.10 Generic_142909-17 sun4u sparc SUNW,Sun-Fire-V890 Solaris % version Machine hardware: sun4u OS version: 5.10 Processor type: sparc Hardware: SUNW,Sun-Fire-V890 The following components are installed on your system: Sun Studio 12 update 1 Sun Studio 12 update 1 C Compiler Sun Studio 12 update 1 C++ Compiler Sun Studio 12 update 1 Tools.h++ 7.1 Sun Studio 12 update 1 C++ Standard 64-bit Class Library Sun Studio 12 update 1 Garbage Collector Sun Studio 12 update 1 Fortran 95 Sun Studio 12 update 1 Debugging Tools (including dbx) Sun Studio 12 update 1 IDE Sun Studio 12 update 1 Performance Analyzer (including collect, ...) Sun Studio 12 update 1 Performance Library Sun Studio 12 update 1 Scalapack Sun Studio 12 update 1 LockLint Sun Studio 12 update 1 Building Software (including dmake) Sun Studio 12 update 1 Documentation Set Sun Studio 12 update 1 /usr symbolic links and GNOME menu item version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/cc": Sun C 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/CC": Sun C++ 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/f90": Sun Fortran 95 8.4 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dbx": Sun DBX Debugger 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/analyzer": Sun Analyzer 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dmake": Sun Distributed Make 7.9 SunOS_sparc 2009/06/03 Regards -- [ Piotr Tarnowski piotr.tarnowski at us.edu.pl ] [ Unix Administrator at University of Silesia, Katowice, Poland ] -------------- next part -------------- A non-text attachment was scrubbed... Name: sieve.h-redeclared.patch Type: text/x-patch Size: 486 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5810 bytes Desc: S/MIME Cryptographic Signature URL: From Ralf.Hildebrandt at charite.de Wed Feb 29 15:27:19 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:27:19 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229102250.GY13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> Message-ID: <20120229132718.GN13045@charite.de> * Ralf Hildebrandt : > * Ralf Hildebrandt : > > * Christoph Bu?enius : > > > On 21.02.2012 01:18, Timo Sirainen wrote: > > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > > What else do I need for fts_lucene to work? Any 3rd party packages? > > Libs? Compile Time Options? > > Rebuilding with --with-lucene --with-stemmer, let's see what happens. The plugin doesn't build: postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/config/compiler.h:40:0, from /usr/include/CLucene/StdHeader.h:17, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/config/CompilerGcc.h:12:68: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. make: *** [lucene-wrapper.lo] Error 1 # dpkg -L libclucene-dev |grep -i clucene-config /usr/lib/CLucene/clucene-config.h That's retarded! Who's putting includes into /usr/lib/CLucene/ ??? ARGH! After symlinking, I'm getting: # make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:24:36: fatal error: CLucene/util/CLStreams.h: No such file or directory compilation terminated. Where do I get this include from (on Debian)? # dpkg -l |grep lucene ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 15:29:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 15:29:44 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229132718.GN13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> Message-ID: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > The plugin doesn't build: > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make .. > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) It requires CLucene 2.3.2. From Ralf.Hildebrandt at charite.de Wed Feb 29 15:36:55 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:36:55 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <20120229133655.GO13045@charite.de> * Timo Sirainen : > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > The plugin doesn't build: > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > .. > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > It requires CLucene 2.3.2. ARGH. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 15:54:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 14:54:49 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> On 29.02.2012 14:29, Timo Sirainen wrote: > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > >> The plugin doesn't build: >> >> postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# >> make > .. >> ii libclucene-dev 0.9.21b-2+b1 library for >> full-featured text search engine (development) >> ii libclucene0ldbl 0.9.21b-2+b1 library for >> full-featured text search engine (runtime) > > It requires CLucene 2.3.2. Hi Timo, I have the same problem. It does not build with 2.3.3. [mockbuild at x86-002 result]$ cat root.log | grep clucene DEBUG util.py:257: --> clucene-core-devel-2.3.3.4-3.fc16.x86_64 DEBUG util.py:257: clucene-core-devel x86_64 2.3.3.4-3.fc16 updates-released 258 k DEBUG util.py:257: clucene-core x86_64 2.3.3.4-3.fc16 updates-released 507 k DEBUG util.py:257: clucene-core-devel.x86_64 0:2.3.3.4-3.fc16 DEBUG util.py:257: clucene-core.x86_64 0:2.3.3.4-3.fc16 build.log: Making all in fts-lucene make[4]: Entering directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o doveadm-fts-lucene.lo doveadm-fts-lucene.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-lucene-plugin.lo fts-lucene-plugin.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-backend-lucene.lo fts-backend-lucene.c /bin/sh ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-backend-lucene.c -fPIC -DPIC -o .libs/fts-backend-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c doveadm-fts-lucene.c -fPIC -DPIC -o .libs/doveadm-fts-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-lucene-plugin.c -fPIC -DPIC -o .libs/fts-lucene-plugin.o libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/StdHeader.h:20:0, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/SharedHeader.h:18:36: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. /bin/sh ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -Wl,-z,relro -o lib20_doveadm_fts_lucene_plugin.la -rpath /usr/lib64/dovecot/doveadm doveadm-fts-lucene.lo -lrt make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... libtool: link: gcc -shared -fPIC -DPIC .libs/doveadm-fts-lucene.o -lrt -O2 -m64 -mtune=generic -Wl,-z -Wl,relro -Wl,-soname -Wl,lib20_doveadm_fts_lucene_plugin.so -o .libs/lib20_doveadm_fts_lucene_plugin.so libtool: link: ( cd ".libs" && rm -f "lib20_doveadm_fts_lucene_plugin.la" && ln -s "../lib20_doveadm_fts_lucene_plugin.la" "lib20_doveadm_fts_lucene_plugin.la" ) make[4]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' make[3]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins' make[3]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src' make[2]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1' make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 RPM build errors: error: Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Child returncode was: 1 EXCEPTION: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/mockbuild/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 352, in do raise mockbuild.exception.Error, ("Command failed. See logs for output.\n # %s" % (command,), child.returncode) Error: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] LEAVE do --> EXCEPTION RAISED Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 15:58:51 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:58:51 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> Message-ID: <20120229135851.GU13045@charite.de> * Morten Stevens : > >It requires CLucene 2.3.2. > > Hi Timo, > > I have the same problem. It does not build with 2.3.3. I'm right now trying exactly this :) > In file included from /usr/include/CLucene/StdHeader.h:20:0, > from /usr/include/CLucene.h:11, > from lucene-wrapper.cc:23: > /usr/include/CLucene/SharedHeader.h:18:36: fatal error: > CLucene/clucene-config.h: No such file or directory Did you try symlinking the file? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:09:41 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:09:41 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229135851.GU13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> Message-ID: <1be342370509d17ae81682aede00f016@imt-systems.com> On 29.02.2012 14:58, Ralf Hildebrandt wrote: > * Morten Stevens : > >> >It requires CLucene 2.3.2. >> >> Hi Timo, >> >> I have the same problem. It does not build with 2.3.3. > > I'm right now trying exactly this :) > >> In file included from /usr/include/CLucene/StdHeader.h:20:0, >> from /usr/include/CLucene.h:11, >> from lucene-wrapper.cc:23: >> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >> CLucene/clucene-config.h: No such file or directory > > Did you try symlinking the file? I'm not sure if this is possible because we use a chroot build environment to build RPM packages for Fedora and RHEL. See: http://fedoraproject.org/wiki/Projects/Mock Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 16:27:39 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:27:39 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229133655.GO13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> Message-ID: <20120229142738.GW13045@charite.de> * Ralf Hildebrandt : > * Timo Sirainen : > > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > > > The plugin doesn't build: > > > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > > .. > > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > > > It requires CLucene 2.3.2. > > ARGH. I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the info. Maybe the configure should check for a version >=2.3.2! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:29:29 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:29:29 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <1be342370509d17ae81682aede00f016@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> Message-ID: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> On 29.02.2012 15:09, Morten Stevens wrote: > On 29.02.2012 14:58, Ralf Hildebrandt wrote: >> * Morten Stevens : >> >>> >It requires CLucene 2.3.2. >>> >>> Hi Timo, >>> >>> I have the same problem. It does not build with 2.3.3. >> >> I'm right now trying exactly this :) >> >>> In file included from /usr/include/CLucene/StdHeader.h:20:0, >>> from /usr/include/CLucene.h:11, >>> from lucene-wrapper.cc:23: >>> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >>> CLucene/clucene-config.h: No such file or directory >> >> Did you try symlinking the file? > > I'm not sure if this is possible because we use a chroot build > environment to build RPM packages for Fedora and RHEL. > See: http://fedoraproject.org/wiki/Projects/Mock Update: This is a Fedora-specific problem, because clucene (build requirement) is not correctly packaged. Best regards, Morten From tss at iki.fi Wed Feb 29 16:29:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 16:29:57 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229142738.GW13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> <20120229142738.GW13045@charite.de> Message-ID: <8DB2FDBF-02E3-4E55-A817-85D62668C756@iki.fi> On 29.2.2012, at 16.27, Ralf Hildebrandt wrote: > I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the > info. Maybe the configure should check for a version >=2.3.2! I've thought about adding such checks, but autoconf seems to make it difficult to do C++ checks so I haven't bothered.. But I added that info to wiki. From Ralf.Hildebrandt at charite.de Wed Feb 29 16:30:38 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:30:38 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> References: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> Message-ID: <20120229143038.GX13045@charite.de> * Morten Stevens : > This is a Fedora-specific problem, because clucene (build > requirement) is not correctly packaged. Well, debian showed the same packaging (wrong place). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 17:51:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 17:51:47 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4D34A4.6090106@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> <4F4D34A4.6090106@Media-Brokers.com> Message-ID: <99E3DC46-0709-4DFD-A751-DCA9190E4874@iki.fi> On 28.2.2012, at 22.10, Charles Marcus wrote: > But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. Yes, there are some ways to do this. Although you would most likely want Dovecot proxy without director feature enabled. >>> Timo, state above that this 'describes a design'... does this mean that >>> this is doable right now? Which of the following terms would you use to >>> describe it at this point in time? >>> >>> Potentially problematic > >> Not finished yet, so closest to this. :) I'm planning on making our own >> mails use this within a few weeks (currently it's a dsync in crontab >> every 5 minutes). > > Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Possibly. :) From tss at iki.fi Wed Feb 29 18:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 18:10:11 +0200 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: On 27.2.2012, at 18.54, Charles Marcus wrote: > I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? I'll initially build it for only 2 locations, but I think it will be pretty simple to scale to more than 2. > If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? Synchronous drbd replication for a master/slave setup should also work, since the latency between your servers is probably quite low. I wouldn't use asynchronous replication since it can lose some of the last changes when failure happens. Then there are of course all the cluster filesystems, but I don't have much experience with them other than what I've read in this list. I think GPFS is the only one I haven't read any complains of (but it could be also that so few people have actually used it). > 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? With a clusterfs setup you could do this. With dsync-replicated setup you could assign a primary location for the user, and proxy the connection there if user got connected to wrong server, except when the primary server is down. > I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Like other mentioned, I don't think the cross-office traffic will be that much of a problem, especially for external connections from outside office. For internal connections you should be able to mostly avoid it. From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:27:34 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:27:34 +0100 Subject: [Dovecot] What's behind the message ... Message-ID: (auto-added) ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount Hello, the above message appeared when I restarted dovecot. What has dovecot to deal with my USB-Backup-Disk? Dovecot cares about filesystem without any relation to the mail system? Why this? What's bedind? Thanks in advance, Frank Elsner From tss at iki.fi Wed Feb 29 19:31:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 19:31:12 +0200 Subject: [Dovecot] What's behind the message ... In-Reply-To: (auto-added) References: (auto-added) Message-ID: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> On 29.2.2012, at 19.27, Frank Elsner wrote: > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > Hello, > the above message appeared when I restarted dovecot. > > What has dovecot to deal with my USB-Backup-Disk? > Dovecot cares about filesystem without any relation to the mail system? No, but Dovecot doesn't really know what the mail filesystems are. You can add: doveadm mount add '/media/*' ignore Perhaps this should be a default. > Why this? What's bedind? http://markmail.org/message/ad6f4fg3js6ljykg From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:59:06 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:59:06 +0100 Subject: [Dovecot] What's behind the message ... In-Reply-To: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> References: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> Message-ID: (auto-added) On Wed, 29 Feb 2012 19:31:12 +0200 Timo Sirainen wrote: > On 29.2.2012, at 19.27, Frank Elsner wrote: > > > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > > > Hello, > > the above message appeared when I restarted dovecot. > > > > What has dovecot to deal with my USB-Backup-Disk? > > Dovecot cares about filesystem without any relation to the mail system? > > No, but Dovecot doesn't really know what the mail filesystems are. You can add: > > doveadm mount add '/media/*' ignore > > Perhaps this should be a default. > > > Why this? What's bedind? > > http://markmail.org/message/ad6f4fg3js6ljykg Thanks for the pointer! --Frank Elsner From stephan at rename-it.nl Wed Feb 29 23:54:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 29 Feb 2012 22:54:15 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) In-Reply-To: <4F4E20BF.8040707@us.edu.pl> References: <4F4E20BF.8040707@us.edu.pl> Message-ID: <4F4E9E87.7020608@rename-it.nl> Hi Piotr, On 2/29/2012 1:57 PM, Piotr Tarnowski wrote: > I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but > when I want to compile Sieve for this version > (dovecot-2.1-pigeonhole-0.3.0) I get an error: > > "sieve.c", line 578: identifier redeclared: sieve_multiscript_run > > and I see that declaration in sieve.h is different from definition in > sieve.c, so compilation process fails with SunStudio cc compiler. > > I have done a quick hack to correct this, and then compiled successfully. Thanks for the report. Apparently GCC doesn't see this. Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/9da3a8398ea4 Regards, Stephan. From kurt.armistead at eeb5.eu Wed Feb 1 01:16:48 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 00:16:48 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot Message-ID: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Hi guys. Recently we've moved our old (and I mean _really_ old) mailserver to a new hardware. Since I had quite some time on my hands, I decided to switch from courier to dovecot. Transition went pretty smoothly and the users experienced just a few hours' blackout. Only trouble now is that, after re-configuring some IOS handsets and tablets, not all messages are displayed correctly. For example, I had a long existing mailbox (roughly 1600 emails in INBOX and about a dozen other folders) that showed up perfectly on my iphone. Now it's been synced to the new dovecot server via imap and a whole month of messages seems to be missing. I checked on the server, via webmail and with outlook and everything is where it should be. The problem shows up on Apple devices (same thing happened on a couple of ipads). I haven't tried rebuilding the index or wiping out a phone to see if that resolves the problem, but before I do that I'd like to know if anyone experienced the same issue and how they fixed it Thanks! Kurt From tss at iki.fi Wed Feb 1 01:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 01:33:05 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 1.16, Armistead, Kurt wrote: > Recently we've moved our old (and I mean _really_ old) mailserver to a new > hardware. Since I had quite some time on my hands, I decided to switch > from courier to dovecot. Transition went pretty smoothly and the users > experienced just a few hours' blackout. > > Only trouble now is that, after re-configuring some IOS handsets and > tablets, not all messages are displayed correctly. Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely related to clients not realizing that they should drop their internal caches. From f.bonnet at esiee.fr Wed Feb 1 07:55:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Wed, 01 Feb 2012 06:55:48 +0100 Subject: [Dovecot] Temporary forbid some users login ? Message-ID: <4F28D3E4.8040408@esiee.fr> hello is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? I need to move their IMAP folders to another place with more disk space but I don't want to stop dovecot IMAP service for the other users as the moving process will be a bit long ( 1 Tb to move ) thanks From arne at drlinux.no Wed Feb 1 08:58:43 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Wed, 01 Feb 2012 07:58:43 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F28E2A3.3090200@drlinux.no> Den 01. feb. 2012 06:55, skrev Frank Bonnet: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > > > This has been covered before, but if you are using SQL for auth-lookups then you can have something like this ins your query; ... AND active='y' Arne -- Arne K. Haaje - Dr Linux http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From kurt.armistead at eeb5.eu Wed Feb 1 09:48:06 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 08:48:06 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: > On 1.2.2012, at 1.16, Armistead, Kurt wrote: > >> Recently we've moved our old (and I mean _really_ old) mailserver to a >> new >> hardware. Since I had quite some time on my hands, I decided to switch >> from courier to dovecot. Transition went pretty smoothly and the users >> experienced just a few hours' blackout. >> >> Only trouble now is that, after re-configuring some IOS handsets and >> tablets, not all messages are displayed correctly. > > Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new > IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely > related to clients not realizing that they should drop their internal > caches. > I sure did and it reported no issues. Kurt From joseba.torre at ehu.es Wed Feb 1 13:55:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 01 Feb 2012 12:55:44 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F292840.8080907@ehu.es> El 01/02/12 06:55, Frank Bonnet escribi?: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks Take a look to conf.d/auth-deny.conf.ext You can setup a new passdb (a passwd-file can do it) with deny = yes, and add/remove users to that passwd-file as needed. From weber at zackbummfertig.de Wed Feb 1 14:11:11 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:11:11 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: Message-ID: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Is this Howto http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL actual also for Dovecot 2.x ? Cause i read Dovecot 1.x in the Howto Thank you for any hints / tipps Marko (from cold Hamburg Harbour) From jsiegle at psu.edu Wed Feb 1 14:23:10 2012 From: jsiegle at psu.edu (Jonathan Siegle) Date: Wed, 01 Feb 2012 07:23:10 -0500 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Message-ID: <4F292EAE.2030601@psu.edu> Marko Weber said the following on 2/1/12 7:11 AM: > > Is this Howto > http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL > actual also for Dovecot 2.x ? > Cause i read Dovecot 1.x in the Howto > > Thank you for any hints / tipps > My gut feeling is version 1.2 because it was last modified on 2010-06-29 11:37:39(see bottom of page) which is in the 1.2 days(http://www.dovecot.org/oldnews.html). -Jonathan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4316 bytes Desc: S/MIME Cryptographic Signature URL: From janfrode at tanso.net Wed Feb 1 14:29:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 1 Feb 2012 13:29:50 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS Message-ID: <20120201122950.GA23395@dibs.tanso.net> I've been running continous dsync backups of our Maildirs for a few weeks now, with the destination dsync server using mdbox and SIS. The idea was that the destination server would act as a warm copy of all our active users data. The active servers are using Maildir, and has: $ df -h /usr/local/atmail/users/ Filesystem Size Used Avail Use% Mounted on /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users $ df -hi /usr/local/atmail/users/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users very little of this is compressed (zlib plugin enabled during christmas). I'm surprised that the destination server is so large, was expecting zlib and mdbox and SIS would compress it down to much less than what we're seeing (12TB -> 5TB): $ df -h /srv/mailbackup Filesystem Size Used Avail Use% Mounted on /dev/mapper/mailbackupvg-mailbackuplv 5.7T 4.8T 882G 85% /srv/mailbackup Lots and lots of the attachement storage is duplicated into identical files, instead of hard linked. When running "doveadm purge -u $user", we're seeing lots of Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is missing, but there are 205 other copies of this file named /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with identical sha1sum. Also we see corrupted indexes during the purge. This makes me quite uncertain if dsync is a workable backup solution.. or if we can trust mdboxes. Also on the source side, during dsync, we see too many problems. Some samples: Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX Error: command BOX-LIST failed Error: Worker server's mailbox iteration failed Error: read() from worker server failed: EOF Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Error: read() from proxy client failed: EOF Error: Unexpected finish reply: 1 596fec275888dbd89f6d1f5356c22db6 3720 0 \dsync-expunged 0 Error: Unexpected reply from server: 1 12200572a70726fca946da6f9378dc03 3721 0 \dsync-expunged 0 Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda Error: command BOX-LIST failed Error: read() failed: Broken pipe Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x367703c680] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x367703c765] -> /usr/lib64/dovecot/libdovecot.so.0 [0x367703bb93] -> /usr/bin/dsync [0x40f48d] -> /usr/bin/dsync [0x40f589] -> /usr/bin/dsync(dsync_worker_msg_save+0x8e) [0x40eb3e] -> /usr/bin/dsync [0x40d71a] -> /usr/bin/dsync [0x40cdbf] -> /usr/bin/dsync [0x40d105] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x3677047278] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xd5) [0x36770485c5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x367704720d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3677035a83] -> /usr/bin/dsync(main+0x71e) [0x406c4e] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e3941d994] -> /usr/bin/dsync [0x406369] Do you have any idea for what our problems might be? Should we: avoid SIS ? avoid doing Maildir on one side and mdbox on the other? try other dovecot version for dsync? anything else? -jf ------------- destination server, running dovecot v2.0.14 -------- mail_attachment_dir = /srv/mailbackup/attachments mail_location = mdbox:~/mdbox mail_plugins = zlib mdbox_rotate_size = 5 M namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = static } plugin { zlib_save = gz zlib_save_level = 9 } protocols = service auth-worker { user = $default_internal_user } service auth { unix_listener auth-userdb { mode = 0600 user = mailbackup } } ssl = no userdb { args = home=/srv/mailbackup/%256Hu/%d/%n driver = static } -------------/destination server -------- -jf From weber at zackbummfertig.de Wed Feb 1 14:32:47 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:32:47 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <4F292DA5.2000107@Media-Brokers.com> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> <4F292DA5.2000107@Media-Brokers.com> Message-ID: Thank you, i ask because in the first Part is written: At the time of writing this document we use these versions of the required software (probably newer versions also usable): CentOS 4.4 (or RHEL 4 update 4 or Fedora 6) Apache 2.0.52 PHP 5.1.6 MySQL 5.0.22 Dovecot 1.0rc15 And i think , config syntax and organisation of config changed alot from 1.x to 2.x , or ? Am 01.02.2012 13:18, schrieb Charles Marcus: > On 2012-02-01 7:11 AM, Marko Weber wrote: >> >> Is this Howto >> http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL >> actual also for Dovecot 2.x ? > > wiki2. stuff is for dovecot 2.x... > > wiki. stuff is for dovecot 1.x... From tompos at martos.bme.hu Wed Feb 1 14:40:58 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 13:40:58 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ Message-ID: <4F2932DA.4000600@martos.bme.hu> hi! What does patched mean in this repo? Does anybody know? Thank you, tamas From stephan at rename-it.nl Wed Feb 1 14:56:40 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 01 Feb 2012 13:56:40 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F2932DA.4000600@martos.bme.hu> References: <4F2932DA.4000600@martos.bme.hu> Message-ID: <4F293688.1030606@rename-it.nl> Op 2/1/2012 1:40 PM, Papp Tamas schreef: > hi! > > What does patched mean in this repo? > > Does anybody know? This is the v2.1 repository that includes unstable development patches: http://hg.rename-it.nl/dovecot-2.1-patches/ http://hg.rename-it.nl/pigeonhole-0.3-patches/ These are mostly meant for some side-projects I am working on. Previously, some of these patches were included in the normal automatic builds, making debugging difficult since `tainted' dovecot sources were used. The patches also caused a few rather stupid bugs. Therefore, as I announced in a mail a few months back, I started a separate builder for testing these patches. So far, there is no reason for anyone except me to install and try these. Just use the normal dovecot-2.1 repository as documented in the wiki. Regards, Stephan. From tompos at martos.bme.hu Wed Feb 1 15:01:42 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 14:01:42 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F293688.1030606@rename-it.nl> References: <4F2932DA.4000600@martos.bme.hu> <4F293688.1030606@rename-it.nl> Message-ID: <4F2937B6.2010409@martos.bme.hu> On 02/01/2012 01:56 PM, Stephan Bosch wrote: > Op 2/1/2012 1:40 PM, Papp Tamas schreef: >> hi! >> >> What does patched mean in this repo? >> >> Does anybody know? > > This is the v2.1 repository that includes unstable development patches: > > http://hg.rename-it.nl/dovecot-2.1-patches/ > http://hg.rename-it.nl/pigeonhole-0.3-patches/ > > These are mostly meant for some side-projects I am working on. > Previously, some of these patches were included in the normal > automatic builds, making debugging difficult since `tainted' dovecot > sources were used. The patches also caused a few rather stupid bugs. > Therefore, as I announced in a mail a few months back, I started a > separate builder for testing these patches. So far, there is no reason > for anyone except me to install and try these. Just use the normal > dovecot-2.1 repository as documented in the wiki. OK, thank you. tamas From kowalskm at cs.moravian.edu Wed Feb 1 16:09:11 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 09:09:11 -0500 Subject: [Dovecot] locking problem Message-ID: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log I tried killing the imap daemons and removing the lock file, but it keeps coming back. Any idea what else to do? This is the dovecot file: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-28-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem ssl_key_file: /etc/ssl/private/ssl-cert-snakeoil.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix --myron ================================= Myron Kowalski MoCoSIN Network/Systems Administrator Moravian College myron at moravian.edu (610) 861-1441 From gfinch at ldmltd.ca Wed Feb 1 17:41:04 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Wed, 01 Feb 2012 07:41:04 -0800 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F295D10.5010303@ldmltd.ca> On 2012-01-31 9:55 PM, Frank Bonnet wrote: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields see nologin -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From la at iki.fi Wed Feb 1 17:43:20 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 17:43:20 +0200 Subject: [Dovecot] Uncompressing dboxes Message-ID: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Hello. I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. I guess I could convert to mbox, and then back to mdbox, but for certain reasons I'm not very enthusiastic about doing anything mbox-related with dovecot any more. (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Thanks, Lauri From la at iki.fi Wed Feb 1 19:59:05 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 19:59:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? Message-ID: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Hello again. When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Thanks again, Lauri From rob0 at gmx.co.uk Wed Feb 1 21:56:53 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 1 Feb 2012 13:56:53 -0600 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F292840.8080907@ehu.es> References: <4F28D3E4.8040408@esiee.fr> <4F292840.8080907@ehu.es> Message-ID: <20120201195653.GS14195@harrier.slackbuilds.org> On Wed, Feb 01, 2012 at 12:55:44PM +0100, Joseba Torre wrote: > El 01/02/12 06:55, Frank Bonnet escribi?: > > > >is there a way to forbid SOME ( not all ) users's login with > >dovecot 2 ? I need to move their IMAP folders to another place > >with more disk space but I don't want to stop dovecot IMAP > >service for the other users as the moving process will be a > >bit long ( 1 Tb to move ) > > Take a look to conf.d/auth-deny.conf.ext > > You can setup a new passdb (a passwd-file can do it) with deny > = yes, and add/remove users to that passwd-file as needed. Heh, funny, three different answers in this thread and AFAICT they are all correct to some extent. I think the "passdb { deny=yes }" is the best answer. I implemented this in SQL using a tri-state "active" column. Standard "active=1" means the MTA accepts mail and the user can login. "active=0" will disable both. The third state, "active=-1" has the MTA continuing to accept mail, but triggers my deny=yes passdb. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Wed Feb 1 22:11:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:11:40 +0200 Subject: [Dovecot] Uncompressing dboxes In-Reply-To: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> References: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 17.43, Lauri Alanko wrote: > I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. Do something like: dsync backup dsync -o plugin/zlib_save= -o mail=mdbox:/tmp/uncompressed > (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Dovecot supports only read-only compressed mboxes. Also compressed mboxes are always pretty slow, since there's no way to quickly jump around in the file (to go to offset X, you need to uncompress the first X bytes of the mbox file, that's one reason why for mdbox I decided against using full file compression). From tss at iki.fi Wed Feb 1 22:24:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:24:54 +0200 Subject: [Dovecot] locking problem In-Reply-To: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> Message-ID: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> On 1.2.2012, at 16.09, myron wrote: > I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. > > dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log > > I tried killing the imap daemons and removing the lock file, > but it keeps coming back. Any idea what else to do? .. > mail_location: mbox:~/mail:INBOX=/var/mail/%u How large is the /var/mail/ mbox file? From tss at iki.fi Wed Feb 1 22:26:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:26:39 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 9.48, Armistead, Kurt wrote: > On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: >> On 1.2.2012, at 1.16, Armistead, Kurt wrote: >> >>> Recently we've moved our old (and I mean _really_ old) mailserver to a >>> new >>> hardware. Since I had quite some time on my hands, I decided to switch >>> from courier to dovecot. Transition went pretty smoothly and the users >>> experienced just a few hours' blackout. >>> >>> Only trouble now is that, after re-configuring some IOS handsets and >>> tablets, not all messages are displayed correctly. >> >> Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new >> IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely >> related to clients not realizing that they should drop their internal >> caches. > > I sure did and it reported no issues. Then I have no idea. Clients shouldn't have noticed any differences between old and new server. If you find a bug in something from Dovecot's side, let me know. From tss at iki.fi Wed Feb 1 22:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:29:51 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <4F27D981.7060304@noa.gr> References: <4F27D981.7060304@noa.gr> Message-ID: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> On 31.1.2012, at 14.07, Nikolaos Milas wrote: > I am running dovecot-2.0.13-1_128.el5 x86_64 RPM on CentOS 5.7. > > I would like to rename userx with email: userx at example.com to ux at example.com with a mailbox of ux (currently: userx) > > Of course the idea is that new mail will continue to be delivered to the same mailbox, although it has been renamed. > > How can I achieve it? Would it be enough (after changing the associated data in the associated LDAP entry) to simply rename the virtual user directory name, e.g. from /home/vmail/userx to /home/vmail/ux ? Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. If a mail is being delivered at the same time, it could tempfail but will be retried again. (A more complex method would be using dsync, but probably not worth the trouble.) From tss at iki.fi Wed Feb 1 22:36:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:36:11 +0200 Subject: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: References: <33BD52FA-1FE0-46D5-A1E8-9A54C406BE64@iki.fi> Message-ID: On 30.1.2012, at 11.24, Alexis Lelion wrote: >> Maybe you could work around it so that LMTP always proxies the mails, to >> localhost as well, but to a different port which doesn't do proxying at all. > Actually this was my first try, but I had proxying loops because > unlike for IMAP, the LMTP server doesn't seem to support 'proxy_maybe' > option yet, does it? Dovecot v2.1 supports it. From tss at iki.fi Wed Feb 1 22:39:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:39:58 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2656E7.8060501@esiee.fr> References: <4F2656E7.8060501@esiee.fr> Message-ID: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> On 30.1.2012, at 10.37, Frank Bonnet wrote: > We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > format to maildir format. I'd do it one user at a time with dsync, but unfortunately: > the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > another problem is disk space. The users's email data takes about > 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue. From tss at iki.fi Wed Feb 1 22:41:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:41:23 +0200 Subject: [Dovecot] question of different type of Mailbox In-Reply-To: <1327891063.2234.4.camel@localhost.localdomain> References: <1327891063.2234.4.camel@localhost.localdomain> Message-ID: <49D113F9-09BD-41E2-94C6-0647920EA54D@iki.fi> On 30.1.2012, at 4.37, ml wrote: > I just migrate my emails from one type of Maildir to Mailbox > I did as I was having problems reading speed with my webmail. > I did it in order to optimize when do you > > my current config work for me What's the question here? Anyway, mdbox should be faster than either maildir or mbox. From kowalskm at cs.moravian.edu Wed Feb 1 22:43:38 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 15:43:38 -0500 Subject: [Dovecot] locking problem In-Reply-To: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> Message-ID: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> On Feb 1, 2012, at 3:24 PM, Timo Sirainen wrote: > On 1.2.2012, at 16.09, myron wrote: > >> I have one user who can't read his e-mail. The /var/log/mail.log >> file has this statement. >> >> dovecot: IMAP(): Timeout while waiting for lock for >> transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> >> I tried killing the imap daemons and removing the lock file, >> but it keeps coming back. Any idea what else to do? > .. >> mail_location: mbox:~/mail:INBOX=/var/mail/%u > > How large is the /var/mail/ mbox file? -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ Can I just zero out the 3 files in that directory. 6240 Jan 31 13:26 dovecot.index 165888 Jan 31 14:39 dovecot.index.cache 25100 Jan 31 14:42 dovecot.index.log From tss at iki.fi Wed Feb 1 22:46:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:46:03 +0200 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: <4F260EAF.4090408@gedalya.net> References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: On 30.1.2012, at 5.29, Gedalya wrote: > The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > Also, what order does dovecot return the UIDLs in? The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: + pop3: Added support for showing messages in "POP3 order", which can be different from IMAP message order. This can be useful for migrations from other servers. Implemented it for Maildir as 'O' field in dovecot-uidlist. From gedalya at gedalya.net Wed Feb 1 22:50:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 15:50:39 -0500 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: <4F29A59F.8080501@gedalya.net> On 02/01/2012 03:46 PM, Timo Sirainen wrote: > On 30.1.2012, at 5.29, Gedalya wrote: > >> The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. > You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > >> Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. > Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > >> Also, what order does dovecot return the UIDLs in? > The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: > > + pop3: Added support for showing messages in "POP3 order", which can > be different from IMAP message order. This can be useful for > migrations from other servers. Implemented it for Maildir as 'O' > field in dovecot-uidlist. > > Thank you so much! This clears up a lot of fine details. From gedalya at gedalya.net Wed Feb 1 23:03:17 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 16:03:17 -0500 Subject: [Dovecot] Site-wide configurable Sieve Message-ID: <4F29A895.5030007@gedalya.net> Hi, I need some sieve actions to be specified per-user, but exposing actual sieve scripts to users is of the question. For example, I need to be able to either put spam messages into a spam folder, or discard them, based on user preference. Presumably this would be done using Sieve. As I understand, it's not possible to query a database from within a sieve script. If I can't use a single global script, I can imagine automatically generating per-user scripts when settings are saved in a GUI (e.g. webmail) and store them on the dovecot machine. This would work but I would love to hear if someone has a smarter, more robust idea. Thanks, Gedalya From tss at iki.fi Wed Feb 1 23:11:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 23:11:17 +0200 Subject: [Dovecot] locking problem In-Reply-To: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> Message-ID: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> On 1.2.2012, at 22.43, myron wrote: >>> dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> How large is the /var/mail/ mbox file? > > > -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ OK, so not very big. > Can I just zero out the 3 files in that directory. > > 6240 Jan 31 13:26 dovecot.index > 165888 Jan 31 14:39 dovecot.index.cache > 25100 Jan 31 14:42 dovecot.index.log You can delete them, but somehow I doubt that helps.. That error message anyway happens when a session has opened the mbox and is "syncing" it. This normally is a pretty fast operation, but the error message indicates that another session saw the first session doing it for over 3 minutes until it aborted. Do you have hanging imap processes eating 100% CPU? That could explain it if Dovecot doesn't like something in the mbox file. Also there may have been some other reasons for this that are already fixed in newer versions. From alessio at skye.it Thu Feb 2 09:46:55 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 02 Feb 2012 08:46:55 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <4F2A3F6F.6000604@skye.it> Il 01/02/2012 13:29, Jan-Frode Myklebust ha scritto: > I've been running continous dsync backups of our Maildirs for a few > weeks now, with the destination dsync server using mdbox and SIS. The > idea was that the destination server would act as a warm copy of > all our active users data. How many users there are in this installation? > The active servers are using Maildir, and has: > > $ df -h /usr/local/atmail/users/ > Filesystem Size Used Avail Use% Mounted on > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > $ df -hi /usr/local/atmail/users/ > Filesystem Inodes IUsed IFree IUse% Mounted on > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > very little of this is compressed (zlib plugin enabled during christmas). This is the old storage in Maildir format? > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): > > $ df -h /srv/mailbackup > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/mailbackupvg-mailbackuplv > 5.7T 4.8T 882G 85% /srv/mailbackup This is the new storage in mdbox format? What size you would expect? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From nmilas at noa.gr Thu Feb 2 10:17:24 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Thu, 02 Feb 2012 10:17:24 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> References: <4F27D981.7060304@noa.gr> <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> Message-ID: <4F2A4694.1070904@noa.gr> On 1/2/2012 10:29 ??, Timo Sirainen wrote: > Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. Thanks Timo, In fact, I could have the luxury to bring down the mail server (Postfix and Dovecot) for a couple of minutes and complete the changes. There is a mail gateway device queuing for delivery when the mail server is down. Nick From f.bonnet at esiee.fr Thu Feb 2 10:53:00 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 09:53:00 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> Message-ID: <4F2A4EEC.8020207@esiee.fr> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > On 30.1.2012, at 10.37, Frank Bonnet wrote: > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >> format to maildir format. > I'd do it one user at a time with dsync, but unfortunately: > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > >> another problem is disk space. The users's email data takes about >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >> the raid array of the server. > With one-user-at-a-time conversion this wouldn't be an issue. > so is there a massive conversion method ? thanks From janfrode at tanso.net Thu Feb 2 10:53:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 09:53:51 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <4F2A3F6F.6000604@skye.it> References: <20120201122950.GA23395@dibs.tanso.net> <4F2A3F6F.6000604@skye.it> Message-ID: <20120202085351.GA18559@dibs.tanso.net> On Thu, Feb 02, 2012 at 08:46:55AM +0100, Alessio Cecchi wrote: > > How many users there are in this installation? Quite a few :-) This is for an ISP. > >The active servers are using Maildir, and has: > > > > $ df -h /usr/local/atmail/users/ > > Filesystem Size Used Avail Use% Mounted on > > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > > $ df -hi /usr/local/atmail/users/ > > Filesystem Inodes IUsed IFree IUse% Mounted on > > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > > >very little of this is compressed (zlib plugin enabled during christmas). > > This is the old storage in Maildir format? Correct. > > >I'm surprised that the destination server is so large, was expecting zlib and > >mdbox and SIS would compress it down to much less than what we're seeing > >(12TB -> 5TB): > > > > $ df -h /srv/mailbackup > > Filesystem Size Used Avail Use% Mounted on > > /dev/mapper/mailbackupvg-mailbackuplv > > 5.7T 4.8T 882G 85% /srv/mailbackup > > This is the new storage in mdbox format? Correct. > What size you would expect? With Maildir I see message-files shrink to about 20%* of original size after turning on zlib with zlib_save_level=6. I was expecting better compression with mdbox (and zlib_save_level=9), and I would expect SIS to help even further. mdbox+SIS+zlib_save_level=9 variant taking up 40% the space of a mixed** compressed/non-compressed Maildir storage isn't very impressive to me -- and the mdbox backup isn't even complete (it's only the 25% most active users). Yes, I see there might be holes in my logic, expecting compressed messages to compress further after move to mdbox. But also I have expectation that most of the messages are not already compressed on the Maildir side. Sorry, expectations and guesses, not hard facts. [*] based on a couple of samples, not thourough research [**] Only messages saved after we enabled zlib on December 25. are compressed. -jf From tss at iki.fi Thu Feb 2 11:20:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 11:20:35 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2A4EEC.8020207@esiee.fr> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> Message-ID: <1328174435.11500.105.camel@innu> On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: > Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > > On 30.1.2012, at 10.37, Frank Bonnet wrote: > > > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > >> format to maildir format. > > I'd do it one user at a time with dsync, but unfortunately: > > > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > > > >> another problem is disk space. The users's email data takes about > >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > >> the raid array of the server. > > With one-user-at-a-time conversion this wouldn't be an issue. > > > > so is there a massive conversion method ? dsync works for that too: 1. dsync backup all users somewhere 2. wait close to the time you can shut down dovecot 3. dsync backup all users the second time 4. shutdown dovecot and MTA 5. dsync backup once more (hopefully this will be relatively quick) The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space.. From f.bonnet at esiee.fr Thu Feb 2 11:33:45 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 10:33:45 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328174435.11500.105.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> Message-ID: <4F2A5879.3090509@esiee.fr> Le 02/02/2012 10:20 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: >> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : >>> On 30.1.2012, at 10.37, Frank Bonnet wrote: >>> >>>> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >>>> format to maildir format. >>> I'd do it one user at a time with dsync, but unfortunately: >>> >>>> the machine use real users thru openldap ( pam_ldap + nss_ldap ) >>> nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. >>> >>>> another problem is disk space. The users's email data takes about >>>> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >>>> the raid array of the server. >>> With one-user-at-a-time conversion this wouldn't be an issue. >>> >> so is there a massive conversion method ? > dsync works for that too: > > 1. dsync backup all users somewhere > 2. wait close to the time you can shut down dovecot > 3. dsync backup all users the second time > 4. shutdown dovecot and MTA > 5. dsync backup once more (hopefully this will be relatively quick) > > The idea is that you can do incremental dsync backups. You'll of course > need to do this in a way that you don't run out of disk space.. > Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ? see below the output of doveconf -n thanks mail# doveconf -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.4-RELEASE amd64 auth_verbose_passwords = plain auth_worker_max_count = 100 debug_log_path = /var/log/dovecot/debug.log default_client_limit = 6144 default_process_limit = 1024 default_vsz_limit = 1 G disable_plaintext_auth = no info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/dovecot.log mail_location = mbox:%h/:INDEX=/raid/cache/dovecot/index/%u:INBOX=/raid/var/mail/%u:CONTROL=%h/ mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_lock_timeout = 3 mins mbox_write_locks = fcntl mmap_disable = yes passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at esiee.fr protocols = imap pop3 sendmail_path = /usr/local/sbin/sendmail service auth { client_limit = 7169 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 service_count = 1 vsz_limit = 3 G } service imap { process_limit = 2048 vsz_limit = 3 G } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/xxx.pem ssl_cert = References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> Message-ID: <1328177422.11500.108.camel@innu> On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: > > dsync works for that too: > > > > 1. dsync backup all users somewhere > > 2. wait close to the time you can shut down dovecot > > 3. dsync backup all users the second time > > 4. shutdown dovecot and MTA > > 5. dsync backup once more (hopefully this will be relatively quick) > > > > The idea is that you can do incremental dsync backups. You'll of course > > need to do this in a way that you don't run out of disk space.. > > > > Well I have NFS access to a Netapp filer with a LOT of disk space > I need some modifications in my dovecot configuration to use NFS > I think ? No need for any modifications. As long as you don't have multiple Dovecots in different servers accessing the same mails via NFS, there's no trouble. > mmap_disable = yes Looks like you also have this already enabled. This is recommended (not required) with NFS, but if you're not normally using NFS at all you shouldn't enable this. From tss at iki.fi Thu Feb 2 12:23:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 12:23:01 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <1328178181.11500.119.camel@innu> On Wed, 2012-02-01 at 13:29 +0100, Jan-Frode Myklebust wrote: > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): Note that with SIS the attachments aren't compressed. > Lots and lots of the attachement storage is duplicated into identical files, > instead of hard linked. Something's wrong then. > When running "doveadm purge -u $user", we're seeing lots of > > Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory Something's wrong. > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > missing, but there are 205 other copies of this file named > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > identical sha1sum. All of them have a link count of 2, with the other link being in hashes/ directory? > Also on the source side, during dsync, we see too many problems. That is most likely related to your troubles. If the dsync runs crash, the result could leave extra files lying around etc.. > Some samples: > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX This is a little bit strange. What is the doveconf -n output of the source server? > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Is this a namespace prefix? It shouldn't be trying to sync a mailbox named this (there's an extra "." suffix). > Error: read() from proxy client failed: EOF I guess the remote dsync crashes or otherwise aborted. > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda I guess some kind of mismatch related to namespace configuration. > Error: read() failed: Broken pipe > Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Probably can't handle properly when remote dsync dies. Of course it still shouldn't crash. There seems to be some bugs left when dsyncing to a remote host (instead of locally). It would help if I could reproduce the errors that you're seeing. Can you easily reproduce them with some accounts? If so, if you can give enough details for me to reproduce the problems I can fix them. (Except for the "file not found" issues, since that problems occurred earlier already. I should probably somehow make Dovecot fix those missing files though..) From janfrode at tanso.net Thu Feb 2 13:31:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:31:20 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <1328178181.11500.119.camel@innu> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> Message-ID: <20120202113120.GA25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:23:01PM +0200, Timo Sirainen wrote: > > Note that with SIS the attachments aren't compressed. Yes, I know. > > > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > > missing, but there are 205 other copies of this file named > > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > > identical sha1sum. > > All of them have a link count of 2, with the other link being in hashes/ > directory? No, these has link count=207. I don't know what you mean by link being in hashes directory. # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|head -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0069222e0c080f4f754a0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-00ffb9312a370e4f6b610000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0442c5163ad3114fb4780000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-04f288390052144f012d0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-053b6c0f185a0d4fc4210000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-06c98213c3b30e4fac3c0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-076573234fbd0b4fa8620000bf34f6ab This is just one example, I can provide tons of other examples.. Hmm, I see now that there are 206 files of that first example with the 207 links, and here's another other example with numlinks=7: # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|wc -l 206 and numlinks=4: # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l 3 is dovecot somehow creating numlinks+1 copies of every file it hardlinks?? Would explain my diskusage :-) > That is most likely related to your troubles. If the dsync runs crash, > the result could leave extra files lying around etc.. If dsync backup is supposed to be a viable backup solution, I think it should fail much better. If it see errors on the target side it should clear the target and do a full sync. Manually cleaning up after it's problems is too much work. > > > Some samples: > > > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > This is a little bit strange. What is the doveconf -n output of the > source server? ---------------------------------------------------------------------------- # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server # release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 mail_gid = 3000 mail_home = /srv/mailstore/%256RHu/%d/%n mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 mail_plugins = quota zlib mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve sieve_dir = /sieve/%1u/%1.1u/%u sieve_max_script_size = 1M zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib imap_quota } protocol pop3 { mail_plugins = quota zlib pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ---------------------------------------------------------------------------- > > > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > > Is this a namespace prefix? It shouldn't be trying to sync a mailbox > named this (there's an extra "." suffix). I believe it's a folder named "INBOX.ferie 2006.", with the user using the namespace separator in the folder name.. I believe dovecot allows this, so it should also handle backing it up. > > > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda > > I guess some kind of mismatch related to namespace configuration. They both have same namespace config I think, but Maildir vs. mdbox maybe changes something.. > > It would help if I could reproduce the errors that you're seeing. Can > you easily reproduce them with some accounts? If so, if you can give > enough details for me to reproduce the problems I can fix them. (Except > for the "file not found" issues, since that problems occurred earlier > already. I should probably somehow make Dovecot fix those missing files > though..) I'll look trough the logs to see if there are any errors that keep repeating for the same accounts. -jf From janfrode at tanso.net Thu Feb 2 13:51:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:51:03 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <20120202115102.GB25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:31:20PM +0100, Jan-Frode Myklebust wrote: > > and numlinks=4: > > # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l > 3 > > is dovecot somehow creating numlinks+1 copies of every file it > hardlinks?? Would explain my diskusage :-) > Sorry, brainfart.. Yes, these are hardlinks to the same inode.. # ls -i c31beb42ef78810f7fb81a7086144034fb0fd794* ../c31beb42ef78810f7fb81a7086144034fb0fd794* 2422693 c31beb42ef78810f7fb81a7086144034fb0fd794 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-13b405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-1cb405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-4eb405342e24284f61530000bf34f6ab -jf From tss at iki.fi Thu Feb 2 14:41:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 2 Feb 2012 14:41:11 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> On 2.2.2012, at 13.31, Jan-Frode Myklebust wrote: >>> "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is >>> missing, but there are 205 other copies of this file named >>> /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with >>> identical sha1sum. >> >> All of them have a link count of 2, with the other link being in hashes/ >> directory? > > No, these has link count=207. OK, so they aren't actual copies, they are links to the same file. > I don't know what you mean by link being in hashes directory. If you have e.g. aa/bb/aabbccdd-eeee file, there should be a matching aa/bb/hashes/aabbccdd file. >> That is most likely related to your troubles. If the dsync runs crash, >> the result could leave extra files lying around etc.. > > If dsync backup is supposed to be a viable backup solution, I think it > should fail much better. If it see errors on the target side it should > clear the target and do a full sync. Manually cleaning up after it's > problems is too much work. Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX What about: doveadm mailbox status -u user at domain guid '*' in source server? in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name >> >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox >> named this (there's an extra "." suffix). > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > the namespace separator in the folder name.. I believe dovecot allows > this, so it should also handle backing it up. It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: a select foo. a NO [CANNOT] Invalid mailbox name From kowalskm at cs.moravian.edu Thu Feb 2 15:45:46 2012 From: kowalskm at cs.moravian.edu (myron) Date: Thu, 2 Feb 2012 08:45:46 -0500 Subject: [Dovecot] locking problem In-Reply-To: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> Message-ID: <5C6469E0-FAC8-45EB-909B-1A58FC3DBE9E@cs.moravian.edu> On Feb 1, 2012, at 4:11 PM, Timo Sirainen wrote: > On 1.2.2012, at 22.43, myron wrote: > >>>> dovecot: IMAP(): Timeout while waiting for lock for >>>> transaction log file /users//mail/.imap/INBOX/ >>>> dovecot.index.log >>> How large is the /var/mail/ mbox file? >> >> >> -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ > > OK, so not very big. His isn't, but I have users with 500MB to a GB mailbox. Could they be the cause of the problem? > >> Can I just zero out the 3 files in that directory. >> >> 6240 Jan 31 13:26 dovecot.index >> 165888 Jan 31 14:39 dovecot.index.cache >> 25100 Jan 31 14:42 dovecot.index.log > > You can delete them, but somehow I doubt that helps.. That error > message anyway happens when a session has opened the mbox and is > "syncing" it. This normally is a pretty fast operation, but the > error message indicates that another session saw the first session > doing it for over 3 minutes until it aborted. > > Do you have hanging imap processes eating 100% CPU? Hardly anything: ps -eo pcpu,pid,user,args | sort -k 1 -r | head -20 %CPU PID USER COMMAND 0.0 9 root [events/0] 0.0 993 root /usr/sbin/sshd -D 0.0 984 root /usr/sbin/cupsd -C /etc/cups/cupsd.conf 0.0 9677 imap 0.0 9659 dovecot imap-login 0.0 959 root [ext4-dio-unwrit] 0.0 958 root [ext4-dio-unwrit] 0.0 957 root [jbd2/dm-1-8] 0.0 9442 root [lockd] 0.0 939 root [ext4-dio-unwrit] 0.0 938 root [ext4-dio-unwrit] 0.0 937 root [jbd2/dm-3-8] 0.0 930 root [ext4-dio-unwrit] 0.0 929 root [ext4-dio-unwrit] 0.0 928 root [jbd2/dm-2-8] 0.0 9192 root /usr/sbin/apache2 -k start 0.0 911 root [ext4-dio-unwrit] 0.0 910 root [ext4-dio-unwrit] 0.0 90 root [kconservative/1] > That could explain it if Dovecot doesn't like something in the mbox > file. Also there may have been some other reasons for this that are > already fixed in newer versions. I looked at the upgrade list on the server and dovecot isn't showing any upgrades via atp-get From tss at iki.fi Thu Feb 2 15:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 15:54:24 +0200 Subject: [Dovecot] LDAP auth improvements Message-ID: <1328190864.11500.130.camel@innu> v2.1 supports now multiple LDAP fields in a template e.g.: user_attrs = \ homeDirectory=home, \ uidNumber=uid, \ gidNumber=gid, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} which is the same as: user_attrs = \ =home=%{ldap:homeDirectory}, \ =uid=%{ldap:uidNumber}, \ =gid=%{ldap:gidNumber}, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} I was also thinking about adding an alternative simplified syntax for this: user_attrs { home = %{ldap:homeDirectory} uid = %{ldap:uidNumber} gid = %{ldap:gidNumber} mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} } Any ideas for further improvements before I do this change? One thing I'm still wondering about is what I should do when LDAP returns multiple values. Like perhaps: acl_groups = %{ldap:aclGroups:,} Which would mean that all the aclGroups values would be joined together separated by "," characters. And a bit more complex with multiple gidNumbers: gid = %{ldap:gidNumber[0]} mail_access_groups = %{ldap:gidNumber[1:]:,} From f.bonnet at esiee.fr Thu Feb 2 18:22:03 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 17:22:03 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328177422.11500.108.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> <1328177422.11500.108.camel@innu> Message-ID: <4F2AB82B.1000200@esiee.fr> Le 02/02/2012 11:10 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: >>> dsync works for that too: >>> >>> 1. dsync backup all users somewhere >>> 2. wait close to the time you can shut down dovecot >>> 3. dsync backup all users the second time >>> 4. shutdown dovecot and MTA >>> 5. dsync backup once more (hopefully this will be relatively quick) >>> >>> The idea is that you can do incremental dsync backups. You'll of course >>> need to do this in a way that you don't run out of disk space.. >>> >> Well I have NFS access to a Netapp filer with a LOT of disk space >> I need some modifications in my dovecot configuration to use NFS >> I think ? > No need for any modifications. As long as you don't have multiple > Dovecots in different servers accessing the same mails via NFS, there's > no trouble. > >> mmap_disable = yes > Looks like you also have this already enabled. This is recommended (not > required) with NFS, but if you're not normally using NFS at all you > shouldn't enable this. > OK thank you From dovecot at asheesh.org Thu Feb 2 21:59:58 2012 From: dovecot at asheesh.org (Asheesh Laroia) Date: Thu, 02 Feb 2012 14:59:58 -0500 Subject: [Dovecot] dsync deleting too many emails (sdbox) Message-ID: <1328212183-sup-560@rose.makesad.us> I'm using dsync to synchronize emails on a laptop where wifi connectivity sometimes fails in the middle of a sync. I have a shell script that runs dsync, and here is one line of it including the output of dsync: + dsync -f -m realmail mirror /home/paulproteus/projects/ssh-attach/run ssh rose.makesad.us dsync dsync-local(paulproteus): Error: dbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Unexpectedly lost uid=337460 dsync-local(paulproteus): Error: msg guid lookup failed: Internal error occurred. Refer to server log for more information. [2012-02-02 11:02:12] dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Rebuilding index dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 130608.broken dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 66159.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 336269.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 dsync-remote(paulproteus): Error: proxy server timed out dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 dsync-local(paulproteus): Warning: Transaction log file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/dovecot.index.log was locked for 1528 seconds That seemed problematic, but not dangerous. Then I ran a fresh sync, and found 3000 (of 60,000) messages had been deleted and expunged from the "realmail" box. I'm guessing this is some bad interaction with sdbox and partial file downloads? I haven't read the code for this, but I would guess the dsync process isn't being atomic about file transfers, so it is leaving half-completed transfers in place, which results in corrupt files when they're next examined. dovecot -n output: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-2-amd64 x86_64 Debian wheezy/sid mail_location = sdbox:~/Maildir/dbox passdb { driver = pam } protocols = " imap" ssl_cert = Hi This is my first post here so i will be as precise as i can I'm trying to setup a quota system using dovecot postfix and procmail dovecot version 1.2.9 postfix version 2.7.0 I have succesfully modified dovecot config to use fs quota system (so far imap and pop3) It reports correct quotas to imap and pop3 clients (both user and group) What im tryin to do now is to modify postfix->procmail->dovecot config in a way that if user is over quota mail delivery is delayed instead of bouncing. (is this possible?) Second thing i would like to achive is that when authenticated users close to quota/group quota for example 10MB to quota limit try sending email theyr mail is rejected and preferably an email is generated telling them to free some space for new mails first. (is such a thing possible?) MS Exchange servers have such a feature and I would like to get such a functionality on my linux mailserver I'm trying to prevent a situation when mails are lost due to quota problems. This setup was initially a typical virtualmin installation with a system on single partition. Thanks in advance for all Your suggestions From alessio at skye.it Fri Feb 3 09:20:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 03 Feb 2012 08:20:50 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <4F2B8AD2.5010905@skye.it> Il 02/02/2012 23:58, przemek.orzechowski at makolab.pl ha scritto: > > Hi > > This is my first post here so i will be as precise as i can > > I'm trying to setup a quota system using dovecot postfix and procmail > dovecot version 1.2.9 > postfix version 2.7.0 > > I have succesfully modified dovecot config to use fs quota system (so far > imap and pop3) > It reports correct quotas to imap and pop3 clients (both user and group) > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Yes, you can do it but with Dovecot LDA instead of procmail. See the options "quota_full_tempfail" for Dovecot LDA. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Fri Feb 3 09:36:04 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 08:36:04 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <20120203073604.GA6561@dibs.tanso.net> On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Check the quota_full_tempfail setting, http://wiki.dovecot.org/MainConfig > > Second thing i would like to achive is that when authenticated users close > to quota/group quota for example 10MB to quota limit > try sending email theyr mail is rejected and preferably an email is > generated telling them to free some space for new mails first. > (is such a thing possible?) Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 -jf From kirill at shutemov.name Fri Feb 3 13:20:49 2012 From: kirill at shutemov.name (Kirill A. Shutemov) Date: Fri, 3 Feb 2012 13:20:49 +0200 Subject: [Dovecot] Feature request - search and tag protocol - notmuch-alike approach Message-ID: <20120203112049.GA19701@shutemov.name> Hi all, notmuch[1] becomes more and more popular. At least in geek community. But notmuch is local-only (except ssh)... It looks like it's hard (if even possible) to use IMAP as transport for notmuch-alike work flow. I think it would be great to create a new protocol based global search and message tagging. All core notmuch operations (search, show, count, tag) can be mapped to the new protocol (more or less) directly. Search syntax can be reused as is, I think. The tricky part is initial tagging. It requires some kind of transaction to be able to tag new messages without racing with upcoming messages. Request (command) syntax can be similar to unix command. This way we can reuse notmuch syntax even deeper. Like: search --limit 10 --sort=oldest-first tag:to-me tag:unread Replies are often structured. I think it's reasonable to use JSON for all replies. Any ideas? I've started to write a proof-of-concept, but it moves too slow, since my knowledge of dovecot internals is poor and it's hard to find time for the project. I hope somebody else has similar idea in mind and just wait opportunity to discuss the idea and start to write code. ;) [1] http://notmuchmail.org/ -- Kirill A. Shutemov From janfrode at tanso.net Fri Feb 3 14:25:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:25:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> Message-ID: <20120203122552.GA12451@dibs.tanso.net> On Wed, Jan 18, 2012 at 09:03:18PM +0200, Timo Sirainen wrote: > > > > I think the way I originally planned LMTP proxying to work is simply too > > complex to work reliably, perhaps even if the code was bug-free. So > > instead of reading+writing DATA at the same time, this patch changes the > > DATA to be first read into memory or temp file, and then from there read > > and sent to the LMTP backends: > > > > http://hg.dovecot.org/dovecot-2.1/raw-rev/51d87deb5c26 > > > > ----8<----8<----8<-----8<-----8<-----8<----8<-----8<----8<----8<-- > > > > unfortunately I haven't tested that patch, so I have no idea if it > > fixed the issues or not... > > I'm not sure if that patch is useful or not. The important patch to fix it is http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c I now implemented this patch on our directors, and pointed postfix at them. No problem seen so far, but I'm still a bit uncertain about the LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite large delays when fsync'ing (slow IMAP APPEND). Do you think increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start seeing lmtp_proxy_output_timeout problems again ? -jf From janfrode at tanso.net Fri Feb 3 14:42:16 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:42:16 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> Message-ID: <20120203124215.GA13495@dibs.tanso.net> On Thu, Feb 02, 2012 at 02:41:11PM +0200, Timo Sirainen wrote: > > >> That is most likely related to your troubles. If the dsync runs crash, > >> the result could leave extra files lying around etc.. > > > > If dsync backup is supposed to be a viable backup solution, I think it > > should fail much better. If it see errors on the target side it should > > clear the target and do a full sync. Manually cleaning up after it's > > problems is too much work. > > Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. I'm reducing the complexity now, removing SIS and starting the backups from scratch again. I'll start posting the problems I see over the weekend.. > > >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > What about: > > doveadm mailbox status -u user at domain guid '*' > > in source server? INBOX guid=08b46439069d3d4db0490000e671bf84 INBOX.Sent guid=e8f6e431bf6e014f2d780000e671bf84 INBOX.Trash guid=c858f2234a1d5d4e1547000058d3d19f INBOX.Drafts guid=e9f6e431bf6e014f2d780000e671bf84 INBOX.Spam guid=eaf6e431bf6e014f2d780000e671bf84 INBOX.Sent Messages guid=d837512bed7d674e685c000058d3d19f INBOX.INBOX.Sent Messages guid=ebf6e431bf6e014f2d780000e671bf84 INBOX.Notes guid=c0d2250109645e4eed5c000058d3d19f > in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. Scratched dest server before I replied.. sorry. > > >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > >> > >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox > >> named this (there's an extra "." suffix). > > > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > > the namespace separator in the folder name.. I believe dovecot allows > > this, so it should also handle backing it up. > > It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: Oh, sorry.. then this is a problem created by @mail, which poked directly in the filesystem. Guess we'll have to clean these up manually. -jf From a23s4a2008 at yandex.ru Fri Feb 3 15:15:05 2012 From: a23s4a2008 at yandex.ru (User Max) Date: Fri, 03 Feb 2012 17:15:05 +0400 Subject: [Dovecot] openbsd and dovecot Message-ID: <725061328274905@web20.yandex.ru> I'm setting up dovecot in OpenBSD OS. The problem is that I can't set up dovecot to use kerberos authentication via bsdauth which is "similar to pam" I try to test dovecot via telnet: telnet localhost pop3 user test +OK pass password -ERR [IN-USE] Temporary authentication failure and some strings from logs Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted Feb 3 15:02:46 srv-mx-00 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Feb 3 15:29:13 srv-mx-00 dovecot: auth: Error: bsdauth(test,127.0.0.1): getpwnam() failed: Operation not permitted What am I doing wrong? Here is the output of dovecot -n auth_krb5_keytab = /etc/kerberosV/krb5.keytab auth_mechanisms = plain login default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 2000 first_valid_uid = 2000 last_valid_gid = 2000 last_valid_uid = 2000 mail_gid = vmail mail_location = maildir:/var/vmail/%u%Ud/Maildir mail_uid = vmail mbox_write_locks = fcntl mmap_disable = yes passdb { driver = bsdauth } service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0666 user = _postfix } } ssl_cert = Hi, I'm new to Dovecot. I'm trying to start Dovecot(2.0.9) but I'm getting an error. I think the error is because I'm doing a tutorial for an old version of Devcot(1.x.x). What I get is the following: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=imaps is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=pop3s is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:5: ssl_cert_file has been replaced by ssl_cert = References: <1328190864.11500.130.camel@innu> Message-ID: <4F2BE57D.1090508@in.tum.de> On 02.02.2012 14:54, Timo Sirainen wrote: > user_attrs { > home = %{ldap:homeDirectory} > uid = %{ldap:uidNumber} > gid = %{ldap:gidNumber} > mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} > } > > Any ideas for further improvements before I do this change? One thing I was recently missing was something like this: If the user is in LDAP group "group1", then use host = imap1 otherwise use host = imap2 However this would require a separate LDAP query since most LDAP servers do not list any group information in the user table. Another nice feature: Use some sort of script to compute the lookup result, e.g. an external shell script: #!/bin/bash if [[ $ldap_uid == fred ]]; then echo "quota_rule = storage=3G" else echo "quota_rule = storage=100M" fi Stuff like this is possible for SQL user databases, however LDAP does not have such flexible queries. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From mark.zealey at webfusion.com Fri Feb 3 16:16:27 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Fri, 03 Feb 2012 16:16:27 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? Message-ID: <4F2BEC3B.2050808@webfusion.com> Hi there, I was doing some testing on sdbox yesterday. Basically I did the following procedure: 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) 2) Create a copy of the index file (no cache file created yet) 3) deliver another message to the mailbox (u.3) 4) copy back index file from stage (2) 5) deliver new mail Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Thanks, Mark From lopes80andre at gmail.com Fri Feb 3 16:42:42 2012 From: lopes80andre at gmail.com (Andre Lopes) Date: Fri, 3 Feb 2012 14:42:42 +0000 Subject: [Dovecot] What does this /var/log/maillog message means? Message-ID: Hi, I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: [code] protocols = imap pop3 lmtp log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir ssl_cert = /etc/pki/dovecot/certs/dovecot.pem ssl_key = /etc/pki/dovecot/private/dovecot.pem namespace { type = private separator = . prefix = INBOX. inbox = yes } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at mailserver.com } service auth { user = root } passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } [/code] I can start the service correctly: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: [ OK ] [/code] But when I go to "/var/log/maillog" I get this: [code] Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 (by pid=16757 uid=0 code=kill) Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled) [/code] What does this mean? I'm doing something that is not correct? Best Regards, From list at airstreamcomm.net Fri Feb 3 17:22:50 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Fri, 03 Feb 2012 09:22:50 -0600 Subject: [Dovecot] Vacation via database Message-ID: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> We are moving our inbound mail to use dovecot LMTP in the near future and we are looking for suggestions on how to implement a mysql based vacation system. If anyone has experience with this, good or bad please let us know. From Ralf.Hildebrandt at charite.de Fri Feb 3 17:27:59 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 16:27:59 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> Message-ID: <20120203152758.GI29889@charite.de> * list at airstreamcomm.net : > We are moving our inbound mail to use dovecot LMTP in the near future and > we are looking for suggestions on how to implement a mysql based vacation > system. If anyone has experience with this, good or bad please let us > know. Why not use sieve? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 18:41:23 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 11:41:23 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <20120203152758.GI29889@charite.de> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> Message-ID: <4F2C0E33.5070708@gedalya.net> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: > * list at airstreamcomm.net: >> We are moving our inbound mail to use dovecot LMTP in the near future and >> we are looking for suggestions on how to implement a mysql based vacation >> system. If anyone has experience with this, good or bad please let us >> know. > Why not use sieve? > I am facing a similar issue. It looks like sieve can't fetch data from a database, and dovecot can't fetch sieve scripts from a database. It would be useful to have, perhaps, only a global sieve script be able to query a DB, and it would be very useful to allow dovecot to fetch sieve scripts (actual scripts, not a file path) from a database. Any way to do this? From robert at schetterer.org Fri Feb 3 20:32:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:32:12 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C282C.4040403@schetterer.org> Am 03.02.2012 17:41, schrieb Gedalya: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near future >>> and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from a > database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able to > query a DB, and it would be very useful to allow dovecot to fetch sieve > scripts (actual scripts, not a file path) from a database. Any way to do > this? > sieve with database is on roadmap,as it was discussed before ,as far i remember info http://wiki.dovecot.org/ManageSieve http://pigeonhole.dovecot.org/ anyway it seems there is a lot to do left http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO so sql support might have to wait but snip ---- * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## * Implement extlists extension as a plugin * Enotify extension: detect use of variable values extracted from the message that are used in the method argument. RFC reports this as a security issue. * Make the sieve storage a base class with (possibly) various implementations, just like mail-storage. This aims to provide support for alternate types of script storage like LDAP or SQL database. --- looks its on the way -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Fri Feb 3 20:34:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:34:18 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C282C.4040403@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> Message-ID: <4F2C28AA.90904@schetterer.org> Am 03.02.2012 19:32, schrieb Robert Schetterer: > Am 03.02.2012 17:41, schrieb Gedalya: >> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>> * list at airstreamcomm.net: >>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>> and >>>> we are looking for suggestions on how to implement a mysql based >>>> vacation >>>> system. If anyone has experience with this, good or bad please let us >>>> know. >>> Why not use sieve? >>> >> I am facing a similar issue. It looks like sieve can't fetch data from a >> database, and dovecot can't fetch sieve scripts from a database. It >> would be useful to have, perhaps, only a global sieve script be able to >> query a DB, and it would be very useful to allow dovecot to fetch sieve >> scripts (actual scripts, not a file path) from a database. Any way to do >> this? >> > > sieve with database is on roadmap,as it was discussed before ,as far i > remember > > info > > http://wiki.dovecot.org/ManageSieve > http://pigeonhole.dovecot.org/ > > anyway it seems there is a lot to do left > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO > so sql support might have to wait > > but snip > ---- > * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## > > * Implement extlists extension as a plugin > * Enotify extension: detect use of variable values extracted from the > message > that are used in the method argument. RFC reports this as a security > issue. > * Make the sieve storage a base class with (possibly) various > implementations, > just like mail-storage. This aims to provide support for alternate types > of script storage like LDAP or SQL database. > --- > > > looks its on the way by the way you can use some sql vacation script method like implemented in postfixadmin , and has support for horde webmail,squirrelmail,roundcube -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From Ralf.Hildebrandt at charite.de Fri Feb 3 20:35:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 19:35:06 +0100 Subject: [Dovecot] Managesieve Build failure Message-ID: <20120203183506.GU29889@charite.de> I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, got this error today: ... make[3]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' make[2]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' Making install in managesieve-login make[2]: Entering directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve-login' /bin/bash ../../libtool --tag=CC --mode=link gcc -I/usr/src/dovecot-2.0/dovecot-2.0/src/config -DPKG_LIBEXECDIR=\""/usr/local/libexec/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -module -avoid-version -g -Wl,--as-needed -o libmanagesieve_login_settings.la -rpath /usr/local/lib/dovecot/settings libmanagesieve_login_settings_la-managesieve-login-settings.lo libmanagesieve_login_settings_la-managesieve-login-settings-plugin.lo -lrt gcc -DHAVE_CONFIG_H -I. -I../.. -I/usr/src/dovecot-2.0/dovecot-2.0 -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dict -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dns -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-mail -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-imap -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-fs -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-charset -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index/raw -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-master -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-settings -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-auth -I/usr/src/dovecot-2.0/dovecot-2.0/src/login-common -I../.. -I../../src/lib-managesieve -g -Wl,--as-needed -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -MT managesieve-proxy.o -MD -MP -MF .deps/managesieve-proxy.Tpo -c -o managesieve-proxy.o managesieve-proxy.c managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? make[2]: *** [managesieve-proxy.o] Error 1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 20:49:21 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 13:49:21 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C28AA.90904@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> <4F2C28AA.90904@schetterer.org> Message-ID: <4F2C2C31.4040202@gedalya.net> On 02/03/2012 01:34 PM, Robert Schetterer wrote: > Am 03.02.2012 19:32, schrieb Robert Schetterer: >> Am 03.02.2012 17:41, schrieb Gedalya: >>> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>>> * list at airstreamcomm.net: >>>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>>> and >>>>> we are looking for suggestions on how to implement a mysql based >>>>> vacation >>>>> system. If anyone has experience with this, good or bad please let us >>>>> know. >>>> Why not use sieve? >>>> >>> I am facing a similar issue. It looks like sieve can't fetch data from a >>> database, and dovecot can't fetch sieve scripts from a database. It >>> would be useful to have, perhaps, only a global sieve script be able to >>> query a DB, and it would be very useful to allow dovecot to fetch sieve >>> scripts (actual scripts, not a file path) from a database. Any way to do >>> this? >>> >> sieve with database is on roadmap,as it was discussed before ,as far i >> remember >> >> info >> >> http://wiki.dovecot.org/ManageSieve >> http://pigeonhole.dovecot.org/ >> >> anyway it seems there is a lot to do left >> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO >> so sql support might have to wait >> >> but snip >> ---- >> * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## >> >> * Implement extlists extension as a plugin >> * Enotify extension: detect use of variable values extracted from the >> message >> that are used in the method argument. RFC reports this as a security >> issue. >> * Make the sieve storage a base class with (possibly) various >> implementations, >> just like mail-storage. This aims to provide support for alternate types >> of script storage like LDAP or SQL database. >> --- >> >> >> looks its on the way > by the way you can use some sql vacation script method > like implemented in postfixadmin , and has support for horde > webmail,squirrelmail,roundcube > Yeap, for now, in this case, it looks like I'll be hand-coding something into exim and adapt an existing roundcube plugin to my situation. In general, it's looking like I'll be using one, global, static sieve script which will follow instructions in the form of X-Something: Action headers, which I'll be conditionally inserting per user preference on the MX end (exim lmtp transport). Since this is clumsy, I'll try to do as much as possible within the MX, and leave to sieve only things that must happen there (e.g. not necessarily vacation). From stephan at rename-it.nl Sat Feb 4 00:34:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:34:14 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C60E6.7020700@rename-it.nl> On 2/3/2012 5:41 PM, Gedalya wrote: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near >>> future and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from > a database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able > to query a DB, and it would be very useful to allow dovecot to fetch > sieve scripts (actual scripts, not a file path) from a database. Any > way to do this? > Retrieving whole scripts from a DB is nice, but not strictly necessary for your application. Indeed, only fetching data from a DB is sufficient. Unfortunately, there is no standard Sieve feature that allows fetching string data from external sources. However, Pigeonhole has a few custom features: * Assuming you're using SQL, there is an experimental plugin for retrieval of external string data from a Dovecot dict lookup (there is no LDAP dict afaik): http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extdata You can give that a look. Should also compile against Dovecot v2.1 and Pigeonhole v0.3. * If you were using Dovecot v2.1 and Pigeonhole v0.3 (from hg) you could also tinker something with the new extprograms plugin (using the execute command): http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/ For more info read: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/README http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt And, as you suggested, Pigeonhole v0.3 for Dovecot v2.1 can also restrict the use of these plugins to global scripts. Regards, Stephan. From stephan at rename-it.nl Sat Feb 4 00:53:49 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:53:49 +0100 Subject: [Dovecot] Managesieve Build failure In-Reply-To: <20120203183506.GU29889@charite.de> References: <20120203183506.GU29889@charite.de> Message-ID: <4F2C657D.50703@rename-it.nl> On 2/3/2012 7:35 PM, Ralf Hildebrandt wrote: > I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, > got this error today: > > ... > > managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: > managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? > make[2]: *** [managesieve-proxy.o] Error 1 I didn't realize that this change was also performed for Dovecot v2.0 and for some reason I didn't get a failure message from the automatic builder. Fixed now: http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 Regards, Stephan. From Przemek.Orzechowski at makolab.net Fri Feb 3 16:38:05 2012 From: Przemek.Orzechowski at makolab.net (Przemek Orzechowski) Date: Fri, 3 Feb 2012 14:38:05 +0000 Subject: [Dovecot] Question about quota configuration In-Reply-To: <20120203073604.GA6561@dibs.tanso.net> References: <20120203073604.GA6561@dibs.tanso.net> Message-ID: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Thanks for quick reply >-----Original Message----- >From: Jan-Frode Myklebust >Reply-to: Jan-Frode Myklebust >To: przemek.orzechowski at makolab.pl >Cc: dovecot at dovecot.org >Subject: Re: [Dovecot] Question about quota configuration >Date: Fri, 3 Feb 2012 08:36:04 +0100 >Mailer: Mutt/1.5.20 (2009-12-10) > >On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: >> >> What im tryin to do now is to modify postfix->procmail->dovecot config >> in a way that if user is over quota mail delivery is delayed instead of >> bouncing. >> (is this possible?) > >Check the quota_full_tempfail setting, > > > http://wiki.dovecot.org/MainConfig > For some reason when i configure dovecot to act as the final LDA in procmail and set quota_full_tempfail = yes overquota mails get lost somewhere As Virtualmin is relying for some of its functons on procmail droping it compleetly is impossible or at least very difficult. >> >> Second thing i would like to achive is that when authenticated users close >> to quota/group quota for example 10MB to quota limit >> try sending email theyr mail is rejected and preferably an email is >> generated telling them to free some space for new mails first. >> (is such a thing possible?) > >Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 > I agree this is the solution for sending a warning email, but is there a method to actually prohibit users close to quota to send emails? Ie defining dovecot as authentication agent ? > > -jf to make things easier for everyone im posting snippets from my config of dovecot and procmail :0wi VIRTULAMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME EXITCODE=0 :0 * ?/usr/bin/test "$VIRTUALMIN" != "" { INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN } DEFAULT=$HOME/Maildir/ ORGMAIL=$HOME/Maildir/ DELIVER="/usr/lib/dovecot/deliver" DROPPRIVS=yes dovecot config snippets regarding LDA protocol lda { mail_plugins = quota quota_full_tempfail = yes } for some reason this results in mails lost without any trace when users are over quota (i would like to requeue those emails for later delivery) Most probably its not directly dovecot related but as most people here are more or less runing mail servers so maybe someone could help me further. From CMarcus at Media-Brokers.com Sat Feb 4 15:09:13 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sat, 04 Feb 2012 08:09:13 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Message-ID: <4F2D2DF9.1060608@Media-Brokers.com> On 2012-02-03 9:38 AM, Przemek Orzechowski wrote: > is there a method to actually prohibit users close to quota to send > emails? Ie defining dovecot as authentication agent? I wrote about a feature request on this list for something that would probably easily provide the ability for dovecot to do something like this a while back... I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. This would be a huge time saver for people who send a lot of messages with large attachments. Currently, the client has to 'send' (upload) the message to the server *twice* - once for 'sending' it, and again to 'save the sent copy'. Timo - is there a place for users to submit formal Feature Requests? And, is this idea of an LSA) something that you would even remotely consider at any point in time in the future? Maybe your new company could add a 'bounty' system, so users could submit Feature Requests and they (and others) could pledge money for the feature to be implemented? -- Best regards, Crles From gedalya at gedalya.net Sat Feb 4 15:09:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 04 Feb 2012 08:09:39 -0500 Subject: [Dovecot] What does this /var/log/maillog message means? In-Reply-To: References: Message-ID: <4F2D2E13.3080904@gedalya.net> On 02/03/2012 09:42 AM, Andre Lopes wrote: > Hi, > > I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: > > [code] > protocols = imap pop3 lmtp > > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > > ssl_cert = /etc/pki/dovecot/certs/dovecot.pem > ssl_key = /etc/pki/dovecot/private/dovecot.pem > > namespace { > type = private > separator = . > prefix = INBOX. > inbox = yes > } > > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at mailserver.com > } > > service auth { > user = root > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > userdb { > driver = static > args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes > } > [/code] > > I can start the service correctly: > > [code] > [root at mailserver dovecot]# service dovecot start > Starting Dovecot Imap: [ OK ] > [/code] > > But when I go to "/var/log/maillog" I get this: > > [code] > Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 > (by pid=16757 uid=0 code=kill) > Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core > dumps disabled) > [/code] > > What does this mean? I'm doing something that is not correct? > > Best Regards, There log entries just mean you have restarted dovecot. They are normal and don't indicate a problem. From amuehlem at gmail.com Sun Feb 5 17:00:34 2012 From: amuehlem at gmail.com (Andreas Muehlemann) Date: Sun, 05 Feb 2012 16:00:34 +0100 Subject: [Dovecot] Sieve notify messages Message-ID: <4F2E9992.9070701@gmail.com> Hi All I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. I'm trying to use the notifiy mechanism from sieve to send notifications when a mail arrives in the mailbox. The message is checked to be a 8bit message, otherwise it is replaced by the default message "Notification of new message." How can I create a 8bit message body within the sieve script that is accepted by the notification process? e.g. notify :message "Subject: ${subject}" "mailto:email at domain.com"; Do I have to put the message body into the :message string? Or how can I define a custom message body that is accepted by notify? Best regards Andreas From stephan at rename-it.nl Sun Feb 5 17:46:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 16:46:41 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2E9992.9070701@gmail.com> References: <4F2E9992.9070701@gmail.com> Message-ID: <4F2EA461.9000909@rename-it.nl> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: > Hi All > > I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. > > I'm trying to use the notifiy mechanism from sieve to send > notifications when a mail arrives in the mailbox. The message is > checked to be a 8bit message, otherwise it is replaced by the default > message "Notification of new message." How can I create a 8bit message > body within the sieve script that is accepted by the notification > process? > > e.g. > notify > :message "Subject: ${subject}" > "mailto:email at domain.com"; > > Do I have to put the message body into the :message string? Or how can > I define a custom message body that is accepted by notify? If I understand you correctly, the following should be the solution for your issue: This sends the following message: X-Sieve: Pigeonhole Sieve 0.3.0 Message-ID: Date: Sun, 05 Feb 2012 16:36:57 +0100 Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen From: Postmaster To: user at example.com Auto-Submitted: auto-notified; owner-email="234234234234 at ext2.domain.tld" Precedence: bulk X-Priority: 3 (Normal) Importance: Normal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem Durchschnitt schnell. Regards, Stephan From rog7993 at web.de Sun Feb 5 18:53:50 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 17:53:50 +0100 Subject: [Dovecot] user login on behalf of another user Message-ID: <4F2EB41E.6070100@web.de> Hello, we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Example: User "user1" and "user2" shall get access to the mailbox "info". We define the accounts "info~user1" and "info~user2" with the same home directory like "info". Until now, we use a passwd-file backend. With this setup we can simply copy the password hash from "user1" to "info~user1" and from "user2" to "info~user2". But we intend to change the passdb backend from a flat file to PAM for authentication against Active Directory. This seems to be simple with pam_krb5. But then we can't simply copy password hashes anymore. Is their another possibility for configuring this? Surely the preferable alternative would be the use of ACLs to give acccess to other users mailboxes. But we started this setup with Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. And now we have too much accounts and clients, which are configured this way and can't change this for the short term. The passdb/userdb file from the above example looks like this: info:!:501:501:Info:/home/mail01/info:: info~user1:PASSWORD_USER1:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info info~user2:PASSWORD_USER2:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info user1:PASSWORD_USER1:501:501:Info:/home/mail01/user1:: user2:PASSWORD_USER2:501:501:Info:/home/mail01/user2:: Ingo Rogalsky From sven at svenhartge.de Sun Feb 5 19:14:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 5 Feb 2012 18:14:51 +0100 Subject: [Dovecot] user login on behalf of another user References: <4F2EB41E.6070100@web.de> Message-ID: rog7993 at web.de wrote: > User "user1" and "user2" shall get access to the mailbox "info". We > define the accounts "info~user1" and "info~user2" with the same home > directory like "info". Isn't this the kind of scenario shared folders were made for? Gr??e, Sven. -- Sigmentation fault. Core dumped. From stephan at rename-it.nl Sun Feb 5 19:30:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 18:30:20 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2EA6E2.20700@gmail.com> References: <4F2E9992.9070701@gmail.com> <4F2EA461.9000909@rename-it.nl> <4F2EA6E2.20700@gmail.com> Message-ID: <4F2EBCAC.50403@rename-it.nl> On 2/5/2012 4:57 PM, Andreas Muehlemann wrote: > Hi Stephan > > Thank you very much, the encodeurl and ?body= are the trick! > For posterity, please reply to the mailing list as well, especially when it is a success message ;) Regards, Stephan. > > On 02/05/2012 04:46 PM, Stephan Bosch wrote: >> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: >>> Hi All >>> >>> I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. >>> >>> I'm trying to use the notifiy mechanism from sieve to send >>> notifications when a mail arrives in the mailbox. The message is >>> checked to be a 8bit message, otherwise it is replaced by the >>> default message "Notification of new message." How can I create a >>> 8bit message body within the sieve script that is accepted by the >>> notification process? >>> >>> e.g. >>> notify >>> :message "Subject: ${subject}" >>> "mailto:email at domain.com"; >>> >>> Do I have to put the message body into the :message string? Or how >>> can I define a custom message body that is accepted by notify? >> >> If I understand you correctly, the following should be the solution >> for your issue: >> >> >> >> This sends the following message: >> >> >> X-Sieve: Pigeonhole Sieve 0.3.0 >> Message-ID: >> Date: Sun, 05 Feb 2012 16:36:57 +0100 >> Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen >> From: Postmaster >> To: user at example.com >> Auto-Submitted: auto-notified; >> owner-email="234234234234 at ext2.domain.tld" >> Precedence: bulk >> X-Priority: 3 (Normal) >> Importance: Normal >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=UTF-8 >> Content-Transfer-Encoding: 8bit >> >> In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem >> Durchschnitt schnell. >> >> >> Regards, >> >> Stephan > From me at junc.org Sun Feb 5 22:01:32 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 05 Feb 2012 21:01:32 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: Den 2012-02-05 17:53, rog7993 at web.de skrev: > Surely the preferable alternative would be the use of ACLs to give > acccess to other users mailboxes. But we started this setup with > Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. > And now we have too much accounts and clients, which are configured > this way and can't change this for the short term. secureity wise i would also do this, had bots trying whole day here to get week passwords found to atleast find one login that works, if acl is used there is only one password found and the other user do not need to change his password, but only the acl or ask the other user to change his password, its still possible that both users is same user, imho it have no point if its pam users or not From rog7993 at web.de Sun Feb 5 23:49:54 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 22:49:54 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: References: <4F2EB41E.6070100@web.de> Message-ID: <4F2EF982.6080309@web.de> Am 05.02.2012 18:14, schrieb Sven Hartge: > Isn't this the kind of scenario shared folders were made for? Yes, of course. Although I didn't called it shared folders in my original post, I had this in mind, when I wrote that ACLs would be the better solution. But for historical reasons we are already in the situation having these kind of accounts. And I don't want to reconfigure about 100 clients now. This would need more time than I want spend now. Ingo From janfrode at tanso.net Mon Feb 6 15:59:17 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 14:59:17 +0100 Subject: [Dovecot] doveadm purge on shared storage Message-ID: <20120206135917.GA3399@dibs.tanso.net> We've finally (!) started to put some users on mdbox instead of maildir, and now I'm wondering about the purge step. As we're running GPFS for the mailboxes (and dovecot director in front of every dovecot service), is it important to run the "doveadm purge -u $user" on the same host as $user is logged into to avoid index corruption ? If so, will we need to run the "doveadm purge" trough the dovecot director as well? -jf From tss at iki.fi Mon Feb 6 17:05:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 17:05:51 +0200 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> On 5.2.2012, at 18.53, rog7993 at web.de wrote: > we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Master user doesn't necessarily have access to all users' mailboxes. In the passdb lookup you can decide if this master user is allowed to be this destination user. For example if you used passdb checkpassword, you could look at USER and MASTER_USER environment variables to figure out if this combination should be allowed or not. The checkpassword script can also do the actual authentication via PAM (I'd think there's a way to call it somehow). From cannewilson at googlemail.com Mon Feb 6 19:33:01 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Mon, 6 Feb 2012 17:33:01 +0000 Subject: [Dovecot] Configuration problem? Message-ID: I have a new Scientific Linux 6.1 mail server (dovecot -n below) and am seeing the following in the logs, with no idea what is happening: --------------------- Dovecot Begin ------------------------ Dovecot was killed, and not restarted afterwards. **Unmatched Entries** dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24296/89310: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24526/76772: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=26254/75778: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=27948/134120: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=29304/116087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=34802/914087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=4457/11719: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=5322/184434: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=61388/106618: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=6164/21347: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) dovecot: imap: Server shutting down. bytes=14215/97371: 1 Time(s) dovecot: imap: Server shutting down. bytes=1459/9382: 1 Time(s) dovecot: imap: Server shutting down. bytes=1688/26161: 1 Time(s) dovecot: imap: Server shutting down. bytes=1705/5038: 1 Time(s) dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) ---------------------- Dovecot End ------------------------- dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) mail_location = maildir:~/Maildir/ mbox_write_locks = fcntl passdb { driver = pam } protocols = imap ssl_cert = References: Message-ID: <20120206174932.GA5904@PC211.ikt.de> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > am seeing the following in the logs, with no idea what is happening: > > --------------------- Dovecot Begin ------------------------ > > > Dovecot was killed, and not restarted afterwards. You shut dovecot down and does not restart it. > **Unmatched Entries** > dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) [..] > dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) The user closed the connection. > dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) [..] > dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) The server closed the connection due to a shutdown command. > dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) Dovecot did what it just said: it started. Your logwatch is to old to knew about the messages dovecot emits to syslog. HTH Dennis [..] From tss at iki.fi Mon Feb 6 21:58:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:58:02 +0200 Subject: [Dovecot] Multiple instances Message-ID: v2.1 now saves the base_dir and instance_name to /var/lib/dovecot/instances when starting up. If you have for example two instances with names "backend" and "proxy", you can then use: doveadm -i proxy proxy list instead of something like: doveadm -c /etc/dovecot-proxy/dovecot.conf proxy list The same -i parameter works for all dovecot programs. I also added a couple of new doveadm commands: # doveadm instance list path name last used running /var/run/dovecot proxy 2012-02-06 21:41:39 yes # doveadm instance remove proxy It would be possible to add commands to start/shutdown some/all instances using doveadm, but is it all that useful? I'd guess people would have their own init.d scripts anyway doing that. Anything else that could be useful related to this? From tss at iki.fi Mon Feb 6 21:59:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:59:40 +0200 Subject: [Dovecot] doveadm purge on shared storage In-Reply-To: <20120206135917.GA3399@dibs.tanso.net> References: <20120206135917.GA3399@dibs.tanso.net> Message-ID: On 6.2.2012, at 15.59, Jan-Frode Myklebust wrote: > We've finally (!) started to put some users on mdbox instead of maildir, > and now I'm wondering about the purge step. As we're running GPFS for the > mailboxes (and dovecot director in front of every dovecot service), is > it important to run the "doveadm purge -u $user" on the same host as > $user is logged into to avoid index corruption ? > > If so, will we need to run the "doveadm purge" trough the dovecot director as > well? If you run doveadm purge on the director itself (or so that it uses director's config), then doveadm automatically goes through director to the correct server to run the purge. You need to have configured http://wiki2.dovecot.org/Director#Doveadm_server From tss at iki.fi Mon Feb 6 22:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:10:11 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F2D2DF9.1060608@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> Message-ID: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> On 4.2.2012, at 15.09, Charles Marcus wrote: > I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. > > But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. > > This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. Can't you already do this by configuring MTA to bcc mail to user+Sent at domain where there's a global Sieve script moving the +Sent mails to Sent folder (possibly checking that it's from localhost). > Timo - is there a place for users to submit formal Feature Requests? I wouldn't mind if they are simply sent to this list. Although would be a good idea to make it clear in the subject, I almost skipped this message. Most of the things that I want to implement end up in the TODO file in Dovecot sources, and the things that don't end up there are something I'd rather forget, at least until enough people start wanting them. From me at junc.org Mon Feb 6 22:24:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:24:56 +0100 Subject: [Dovecot] dovecot-antispam 1.3 and dovecot 1.2.17 fails with spool2dir Message-ID: <555d62880f4f8cab3bfb1f278f900d22@junc.org> or have i missed something in compile or configure in dovecot.conf ? running on gentoo so i can compile if need another option, i just like to have it working so each user can train bayes global db no matter what user agent users using From CMarcus at Media-Brokers.com Mon Feb 6 22:25:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 06 Feb 2012 15:25:19 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> Message-ID: <4F30372F.3080409@Media-Brokers.com> On 2012-02-06 3:10 PM, Timo Sirainen wrote: > On 4.2.2012, at 15.09, Charles Marcus wrote: >> This way, you could completely disable the 'Save a copy of Sent >> Messages to X folder' option in all mail clients like you can wit >> GMail accounts. > Can't you already do this by configuring MTA to bcc mail to > user+Sent at domain where there's a global Sieve script moving the > +Sent mails to Sent folder (possibly checking that it's from > localhost). You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... >> Timo - is there a place for users to submit formal Feature >> Requests? > I wouldn't mind if they are simply sent to this list. Although would > be a good idea to make it clear in the subject, I almost skipped > this message. Most of the things that I want to implement end up in > the TODO file in Dovecot sources, and the things that don't end up > there are something I'd rather forget, at least until enough people > start wanting them. Heh... understood, and no worries if my idea for a dovecot LSA doesn't make it to the TODO list... ;) -- Best regards, Charles From tss at iki.fi Mon Feb 6 22:27:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:27:05 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <725061328274905@web20.yandex.ru> References: <725061328274905@web20.yandex.ru> Message-ID: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> On 3.2.2012, at 15.15, User Max wrote: > Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted service auth { user = root } Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. From tss at iki.fi Mon Feb 6 22:29:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:29:03 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120203122552.GA12451@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> Message-ID: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > I now implemented this patch on our directors, and pointed postfix at them. > No problem seen so far, but I'm still a bit uncertain about the > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > seeing lmtp_proxy_output_timeout problems again ? Your fsyncs can run over 60 seconds? I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. From tss at iki.fi Mon Feb 6 22:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:33:18 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F30372F.3080409@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> Message-ID: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> On 6.2.2012, at 22.25, Charles Marcus wrote: > On 2012-02-06 3:10 PM, Timo Sirainen wrote: >> On 4.2.2012, at 15.09, Charles Marcus wrote: > >>> This way, you could completely disable the 'Save a copy of Sent >>> Messages to X folder' option in all mail clients like you can wit >>> GMail accounts. > >> Can't you already do this by configuring MTA to bcc mail to >> user+Sent at domain where there's a global Sieve script moving the >> +Sent mails to Sent folder (possibly checking that it's from >> localhost). > > You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... Dovecot LSA would pretty much have to implement full SMTP server functionality to be able to do this (although maybe it could get away with not doing queuing). I'm not really interested in writing an SMTP server. It would be much less trouble to change Postfix to work in a way that you want. From dinh at flic.net Mon Feb 6 22:37:05 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 12:37:05 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? Message-ID: Hi, I am running dovecot version 1.2.9 on a Ubuntu 10.04 server. To obtain the message's header info from this dovecot imap server, I run the following imap command on a client side's imap session: . fetch 88:90 (envelope) which returns (in this case) ------------------------------------------------------------- * 88 FETCH (ENVELOPE ("Tue, 28 Oct 2008 15:45:35 -0500" "Linear Actuators Bring Flexibility to Packaging" (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<490779CF.00000E7D at ak119.mta.exacttarget.com>")) * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<20081029154358.0C359AE169595F1B at insightmedia.info>")) * 90 FETCH (ENVELOPE ("Thu, 30 Oct 2008 04:24:31 +0800 (SGT)" "6 new products Now Online!" (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Dinh Le" NIL "dinh" "flic.net")) NIL NIL NIL "<352725277.1225311871753.JavaMail.web at fuchsia3.globalsources.com>")) . OK Fetch completed. -------------------------------------------------------------- Notice that message 89's Subject line Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available has quotes and this dovecot's ENVELOPE response failed to get parsed by a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). I have a hunch that this is dovecot's fault but I could be wrong. How should dovecot (or any imap server) deal with Subject lines with quotes inside ENVELOPE? ENVELOPE seems to encapsulate most of the important information and its format can be parsed cleanly and quickly. I prefer to fetch ENVELOP, at this point, rather than the other alternatives. Thanks for your help, Dinh From me at junc.org Mon Feb 6 22:39:33 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:39:33 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> Message-ID: <4e7f87f0748f1b70878567e9f5c07fda@junc.org> > Dovecot LSA would pretty much have to implement full SMTP server > functionality to be able to do this (although maybe it could get away > with not doing queuing). I'm not really interested in writing an SMTP > server. It would be much less trouble to change Postfix to work in a > way that you want. dovecot have lmtp, postfix have lmtp, so does quotas work in lmtp deliver ? else one need to use postfix sql to dovecot dict to get used quotas :/ From slusarz at curecanti.org Mon Feb 6 22:45:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 13:45:46 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: Message-ID: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Quoting Dinh Le : > * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} > 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL > NIL "dinh" "flic.net")) NIL NIL NIL > "<20081029154358.0C359AE169595F1B at insightmedia.info>")) [snip] > Notice that message 89's Subject line > > Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > > has quotes and this dovecot's ENVELOPE response failed to get parsed by > a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). > I have a hunch that this is dovecot's fault but I could be wrong. > How should dovecot (or any imap server) deal with Subject lines with > quotes inside ENVELOPE? The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. Dovecot is 100% correct in its return. michael From tss at iki.fi Mon Feb 6 22:47:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:47:41 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F2BEC3B.2050808@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> Message-ID: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> On 3.2.2012, at 16.16, Mark Zealey wrote: > I was doing some testing on sdbox yesterday. Basically I did the following procedure: > > 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) > 2) Create a copy of the index file (no cache file created yet) > 3) deliver another message to the mailbox (u.3) > 4) copy back index file from stage (2) > 5) deliver new mail > > Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. Hm. Yes, I guess there should be a check to avoid overwriting files. > I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. I'm not entirely sure what you mean by this. Does the above patch help with this problem also? > (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Well, ideally there shouldn't be so much corruption that this matters.. From tss at iki.fi Mon Feb 6 22:55:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:55:43 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6.2.2012, at 22.45, Michael M Slusarz wrote: > The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. > > Dovecot is 100% correct in its return. Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) From janfrode at tanso.net Mon Feb 6 23:01:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 22:01:03 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> Message-ID: <20120206210103.GA11888@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:29:03PM +0200, Timo Sirainen wrote: > On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > > > I now implemented this patch on our directors, and pointed postfix at them. > > No problem seen so far, but I'm still a bit uncertain about the > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > > seeing lmtp_proxy_output_timeout problems again ? > > Your fsyncs can run over 60 seconds? Hopefully not.. maybe just me being confused by the error message about "lmtp_proxy_output_timeout". After adding http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't seen any problems so it looks like this problem is solved. But it doesn't seem unthinkable that ext3 users might see more than 60s for fsyncs... "Some stalls on the order of minutes have been reported" ref: https://lwn.net/Articles/328363/ > I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. > My MTA's default is 10 minutes.. http://www.postfix.org/postconf.5.html#lmtp_data_done_timeout -jf From tss at iki.fi Mon Feb 6 23:16:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 23:16:24 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> On 6.2.2012, at 22.55, Timo Sirainen wrote: > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) BTW. If anyone wants to try, the attached patch makes Dovecot return (almost) all possible fields as literals. You'll need to delete dovecot.index.cache files to get the full effect. -------------- next part -------------- A non-text attachment was scrubbed... Name: break-imap-clients.diff Type: application/octet-stream Size: 425 bytes Desc: not available URL: From dinh at flic.net Mon Feb 6 23:18:33 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:18:33 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Thanks, I will try to come up From i.thierack at iwt-promotion.com Mon Feb 6 23:26:45 2012 From: i.thierack at iwt-promotion.com (Ingo Thierack) Date: Mon, 06 Feb 2012 22:26:45 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? Message-ID: <4F304595.9030007@iwt-promotion.com> Hello, try to use the new 2.1rc and don't get any data in the searchindex. Tried first lucene, and switched than back so solr. If I do an search in an mailfolder, i get in the dovecot-log 2012-02-06 22:17:11 | dovecot: | indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot Log from solr. Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} hits=0 status=0 QTime=1 Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository yesterday) With 2.0 i see, if i start an search, solr had to work on the mail, now happens nothing. Upgrade the schema.xml, delete the old index. Regard Ingo Thierack From dinh at flic.net Mon Feb 6 23:34:31 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:34:31 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Oops, my happy Alpine ^X finger accidently sent the previous reply. I meant to say thanks and will try to come up with a fix to this literal string problem for the Perl IMAPClient module. Don't want Timo's evil config setting to wreak havocs :-). From dovecot-20110531 at billmail.scconsult.com Mon Feb 6 23:47:07 2012 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Mon, 06 Feb 2012 16:47:07 -0500 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6 Feb 2012, at 16:34, Dinh Le wrote: > On Mon, 6 Feb 2012, Timo Sirainen wrote: > >> On 6.2.2012, at 22.45, Michael M Slusarz wrote: >> >>> The subject, because it contains quotes, can't be expressed in a >>> quoted string so it is instead sent in a literal string. >> >> Small correction: Quotes can be escaped in a quoted-string, but >> Dovecot just doesn't want to do it. Many other servers behave this >> way as well. >> >>> In a literal string, the data is exactly what appears in the >>> original message. No post-processing mechanism should be stripping >>> quotes or anything from this data - it is already in its canonical >>> state. >>> >>> Dovecot is 100% correct in its return. >> >> Yes. Looks like Perl's IMAP client hasn't been built properly to >> expect literals in places where they are allowed. My guess is that >> this is only one of the 100 other places where a literal is allowed >> by RFC but not by Perl.. >> >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) >> > > Oops, my happy Alpine ^X finger accidently sent the previous reply. I > meant to say thanks and will try to come up with a fix to this literal > string problem for the Perl IMAPClient module. > > Don't want Timo's evil config setting to wreak havocs :-). The best solution for you may be to to update the version of Mail::IMAPClient you are using. There were fixes for its handling of literals claimed in the CHANGES file of v3.26 (2011-01-31). I don't know if that will fix this particular problem, but if you are running an older version then it would be worth trying the current version. From slusarz at curecanti.org Tue Feb 7 00:14:27 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 15:14:27 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> Message-ID: <20120206151427.Horde.QTJpPoF5lbhPMFDDPHZkelA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 6.2.2012, at 22.55, Timo Sirainen wrote: > >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) > > BTW. If anyone wants to try, the attached patch makes Dovecot return > (almost) all possible fields as literals. You'll need to delete > dovecot.index.cache files to get the full effect. Very useful. Turns out the only thing I wasn't handling correctly was the MIME types/subtypes in BODYSTRUCTURE. This is a great tool for debugging client code. Not sure the difficulty to add as a config (or even a hidden config). Being able to easily turn on/off would increase my usage of the tool. michael From ybhu at hk1.ibm.com Tue Feb 7 08:26:43 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 7 Feb 2012 14:26:43 +0800 Subject: [Dovecot] Synchronization error in NFS Message-ID: Hi there, Not sure whether it's a bug or env setting issue. I am running some concurrent testings under NFS. The dovecot (version 2.0.13) are deployed in 2 servers (same setup), whose maildir are on the file server via NFS. Here are the output of dovecot -n: [root at MyMachine src]$dovecot -n # 2.0.13: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) nfs auth_anonymous_username = andy auth_debug = yes auth_debug_passwords = yes auth_mechanisms = anonymous plain auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = andy mail_location = maildir:/tmp/NFS mail_nfs_index = yes mail_nfs_storage = yes mail_uid = andy mmap_disable = yes passdb { driver = pam } ssl = no userdb { driver = passwd } Here are what I am doing: One session running loop of COPY commands (while(1) COPY...) connects to one dovecot server; The other session running loop of SELECT commands (while(1) SELECT...) connects to the other dovecot server. Both are accessing the same mail box (/tmp/NFS); After some while (not accurate duration, maybe 2 seconds, or 1 min), I found the number of EXISTS returned from SELECT command is not correct(less than the real number). Then I stop the both sessions. In the /tmp/log: Feb 07 03:44:59 imap(andy): Error: Corrupted transaction log file /tmp/NFS/dovecot.index.log seq 2: Unexpected garbage at EOF (sync_offset=2204) Feb 07 03:44:59 imap(andy): Error: Index /tmp/NFS/dovecot.index: Lost log for seq=2 offset=2204 Feb 07 03:44:59 imap(andy): Warning: fscking index file /tmp/NFS/dovecot.index Then I tried to dump the index, sometime it's failed because of the index corruption. If not, it says: [root at MyMachine src]$doveadm dump /tmp/NFS > /tmp/dump ; vi /tmp/dump doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor And sometime from the dump output, in the RECORD part: -- RECORDS: 5 RECORD: seq=1, uid=1, flags=0x00 RECORD: seq=2, uid=2, flags=0x00 RECORD: seq=3, uid=3, flags=0x00 RECORD: seq=4, uid=4, flags=0x00 RECORD: seq=5, uid=6, flags=0x00 The uid 5 is missed. but in uidlist file, it's there.. Here are all what I found. If you need additional information, pls let me know. The clock on the 3 machines are synchronized. You can also reproduce it if the 2 sessions are APPEND and SELECT. If both sessions are running towards the same dovecot server, even the maildir are on the NFS, it works very well without any error. From cannewilson at googlemail.com Tue Feb 7 10:08:24 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 7 Feb 2012 08:08:24 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120206174932.GA5904@PC211.ikt.de> References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: On 6 February 2012 17:49, Dennis Guhl wrote: > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and >> am seeing the following in the logs, with no idea what is happening: >> >> ?--------------------- Dovecot Begin ------------------------ >> >> >> ?Dovecot was killed, and not restarted afterwards. > > You shut dovecot down and does not restart it. > It appears to be doing things without my intervention. Despite the reports that it kept shutting down, Dovecot continued to serve messages throughout the day. >> ?**Unmatched Entries** >> ? ? dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) > > [..] > >> ? ? dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) > > The user closed the connection. > >> ? ? dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) > > [..] > >> ? ? dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) > > The server closed the connection due to a shutdown command. > >> ? ? dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) > > Dovecot did what it just said: it started. > > Your logwatch is to old to knew about the messages dovecot emits to > syslog. > > HTH > Dennis > > [..] You mean the version of logwatch is too old? I'm beginning to wonder whether running an Enterprise version is such a good idea after all. Anne From janfrode at tanso.net Tue Feb 7 10:25:10 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 7 Feb 2012 09:25:10 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120206210103.GA11888@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> Message-ID: <20120207082510.GA24432@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:01:03PM +0100, Jan-Frode Myklebust wrote: > > Your fsyncs can run over 60 seconds? > > Hopefully not.. maybe just me being confused by the error message about > "lmtp_proxy_output_timeout". After adding > http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't > seen any problems so it looks like this problem is solved. Crap, saw 6 "message might be sent more than once" messages from postfix yesterday, all at the time of this crash on the director postfix/lmtp was talking with: Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d680] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d6d6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193cb93] -> dovecot/lmtp [0x406d75] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xcd) [0x2ab6f194859d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x68) [0x2ab6f1949558] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x2ab6f194820d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x2ab6f1936a83] -> dovecot/lmtp(main+0x144) [0x403fa4] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x35f8a1d994] -> dovecot/lmtp [0x403da9] Feb 6 16:13:10 loadbalancer2 dovecot: master: Error: service(lmtp): child 6601 killed with signal 6 (core dumps disabled) Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any other ideas for what might be causing it ? -jf From Juergen.Obermann at hrz.uni-giessen.de Tue Feb 7 11:13:24 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 07 Feb 2012 10:13:24 +0100 Subject: [Dovecot] dsync error Mailbox has children, delete them first In-Reply-To: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> References: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> Message-ID: <16d56e5474247859b32601122db0b39f@imapproxy.hrz> Am 13.12.2011 11:47, schrieb J?rgen Obermann: > Hi, > > I use dsync to backup mailboxes from mbox format to mdbox on a remote > system. The first run for a user with dsync is OK, but during the > second there are lots of the following errors: > > dsync-remote(user): Error: Can't delete mailbox directory Example: > Mailbox has children, delete them first > > I see no way how I could influence the order dsync deletes mailboxes. > This happens with dovecot version 2.0.16 > > Thank you, > Juergen Obermann Hallo, after upgrade to dovecot 2.0.17 this problem went away. Greetings, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From nmilas at noa.gr Tue Feb 7 12:07:56 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 12:07:56 +0200 Subject: [Dovecot] Multiple userdb possible? Message-ID: <4F30F7FC.9050607@noa.gr> Hello, I am running v2.0.13. In my dovecot.conf I have: userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } Is it legitimate to include multiple ldap userdb's, like: userdb { args = /etc/dovecot/dovecot-usrdb-ldap1.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap1.conf driver = ldap } userdb { args = /etc/dovecot/dovecot-usrdb-ldap2.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap2.conf driver = ldap } If it is legitimate (in case configuration is different, please correct me), in which sequence userdb's are evaluated? Thanks, Nick From dg at dguhl.org Tue Feb 7 13:57:07 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 7 Feb 2012 12:57:07 +0100 Subject: [Dovecot] Configuration problem? In-Reply-To: References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: <20120207115707.GA30334@laptop-dg.leere.eu> On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: > On 6 February 2012 17:49, Dennis Guhl wrote: > > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > >> am seeing the following in the logs, with no idea what is happening: > >> > >> ?--------------------- Dovecot Begin ------------------------ > >> > >> > >> ?Dovecot was killed, and not restarted afterwards. > > > > You shut dovecot down and does not restart it. > > > It appears to be doing things without my intervention. Despite the > reports that it kept shutting down, Dovecot continued to serve > messages throughout the day. The messages logwatch shows appeared at some time within the analysed period and are not necessarily in a time sorted order. Btw do not rely on any summary of log files but look into the log yourself. [..] > You mean the version of logwatch is too old? I'm beginning to wonder Yes, the current version is 7.4.0 from march 2011 (http://www.logwatch.org). > whether running an Enterprise version is such a good idea after all. I don't know Scientific Linux but I use Debian stable on all my server and I'm very happy with it. Nonetheless do I manually upgrade some packages wich added needed features or are maintained by upstream. It is crucial to know and understand the philosophy behind a distribution and to decide if this works for you and if you can live with the caveats resulting. Dennis From mark.zealey at webfusion.com Tue Feb 7 14:08:09 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Tue, 07 Feb 2012 14:08:09 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> Message-ID: <4F311429.2000708@webfusion.com> 06-02-2012 22:47, Timo Sirainen yazm??: > On 3.2.2012, at 16.16, Mark Zealey wrote: > >> I was doing some testing on sdbox yesterday. Basically I did the following procedure: >> >> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >> 2) Create a copy of the index file (no cache file created yet) >> 3) deliver another message to the mailbox (u.3) >> 4) copy back index file from stage (2) >> 5) deliver new mail >> >> Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. > http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. >> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) > Well, ideally there shouldn't be so much corruption that this matters.. That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Mark From cannewilson at googlemail.com Tue Feb 7 17:20:18 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 07 Feb 2012 15:20:18 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120207115707.GA30334@laptop-dg.leere.eu> References: <20120206174932.GA5904@PC211.ikt.de> <20120207115707.GA30334@laptop-dg.leere.eu> Message-ID: <4F314132.7030604@googlemail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/02/12 11:57, Dennis Guhl wrote: > On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: >> On 6 February 2012 17:49, Dennis Guhl wrote: >>> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >>>> I have a new Scientific Linux 6.1 mail server (dovecot -n >>>> below) and am seeing the following in the logs, with no idea >>>> what is happening: >>>> >>>> --------------------- Dovecot Begin ------------------------ >>>> >>>> >>>> Dovecot was killed, and not restarted afterwards. >>> >>> You shut dovecot down and does not restart it. >>> >> It appears to be doing things without my intervention. Despite >> the reports that it kept shutting down, Dovecot continued to >> serve messages throughout the day. > > The messages logwatch shows appeared at some time within the > analysed period and are not necessarily in a time sorted order. > Btw do not rely on any summary of log files but look into the log > yourself. > > [..] > Actually, this morning there aren't the same messages, so perhaps I was restarting services while trying to get it right - in fact it seems very likely that that was so. Today there are a few like dovecot: imap(anne): Disconnected: Logged out bytes=11892/21219: 1 Time(s) I presume that refers to clients logging out of the imap connection? In which case, I can forget about that. I normally read the summary each morning and refer directly to the logs if I see something that looks unusual. Occasionally, as in this case, there are entries that I don't understand and I ask those who do :-) >> You mean the version of logwatch is too old? I'm beginning to >> wonder > > Yes, the current version is 7.4.0 from march 2011 > (http://www.logwatch.org). > >> whether running an Enterprise version is such a good idea after >> all. > > I don't know Scientific Linux but I use Debian stable on all my > server and I'm very happy with it. Nonetheless do I manually > upgrade some packages wich added needed features or are maintained > by upstream. > > It is crucial to know and understand the philosophy behind a > distribution and to decide if this works for you and if you can > live with the caveats resulting. > I've run CentOS for maybe 4 years, and it's similar to SL, both being RHEL clones, but maintained by different communities. On a server (even though this is a very "mild" server, being only file and print serving) the older packages are rarely a problem. I appreciate the time and trouble you are taking to educate me :-) Anne -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8xQSsACgkQj93fyh4cnBdynQCeO6tY/798/6YonBALxJ0baZcy qG4AoIEHcZWsjIqhz4v3+SMk76FLhjIk =/T9e -----END PGP SIGNATURE----- From rob0 at gmx.co.uk Tue Feb 7 18:00:57 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 7 Feb 2012 10:00:57 -0600 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <4F30F7FC.9050607@noa.gr> References: <4F30F7FC.9050607@noa.gr> Message-ID: <20120207160057.GX14195@harrier.slackbuilds.org> On Tue, Feb 07, 2012 at 12:07:56PM +0200, Nikolaos Milas wrote: > I am running v2.0.13. In my dovecot.conf I have: > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap.conf > driver = ldap > } > > Is it legitimate to include multiple ldap userdb's, like: Of course. The driver chosen is not significant; the query used is all that matters. Having two LDAP searches is conceptually no different than having system users and SQL users. > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap1.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap1.conf > driver = ldap > } > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap2.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap2.conf > driver = ldap > } > > If it is legitimate (in case configuration is different, please > correct me), in which sequence userdb's are evaluated? In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this in mind in setting up the queries. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From chayes at afo.net Tue Feb 7 22:44:20 2012 From: chayes at afo.net (Cliff Hayes) Date: Tue, 7 Feb 2012 14:44:20 -0600 Subject: [Dovecot] Fedora 16 configuration Message-ID: Hello, I am trying to get dovecot to work on a fedora 16 install with sendmail. I have been able to get it to work in the past with dovecot.conf but not with the new conf.d directory and associated config files. I keep seeing this in maillog: Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth attempts): rip=x.x.x.x, lip=x.x.x.x And the mail client comes back with username or password invalid. Is there instructions somewhere regarding fedora 16 installs? I found this one and tried it to no avail: http://www.server-world.info/en/note?os=Fedora_16&p=mail&f=2 Thanks in advance, Cliff From nmilas at noa.gr Tue Feb 7 22:56:02 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 22:56:02 +0200 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <20120207160057.GX14195@harrier.slackbuilds.org> References: <4F30F7FC.9050607@noa.gr> <20120207160057.GX14195@harrier.slackbuilds.org> Message-ID: <4F318FE2.9070604@noa.gr> On 7/2/2012 6:00 ??, /dev/rob0 wrote: > ... > Having two LDAP searches is conceptually no different than having > system users and SQL users. > ... > In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match > prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this > in mind in setting up the queries > ... Thank you for the clarifications! Regards, Nick From rog7993 at web.de Wed Feb 8 01:02:01 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Wed, 08 Feb 2012 00:02:01 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> References: <4F2EB41E.6070100@web.de> <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> Message-ID: <4F31AD69.2000109@web.de> Hello, Am 06.02.2012 16:05, schrieb Timo Sirainen: > Master user doesn't necessarily have access to all users' mailboxes. > In the passdb lookup you can decide if this master user is allowed to > be this destination user. For example if you used passdb > checkpassword, you could look at USER and MASTER_USER environment > variables to figure out if this combination should be allowed or not. > The checkpassword script can also do the actual authentication via > PAM (I'd think there's a way to call it somehow). > Thank you. I got an idea, how I could configure this. Ingo From manuel.bertrand at gmail.com Tue Feb 7 14:20:14 2012 From: manuel.bertrand at gmail.com (manuel.bertrand at gmail.com) Date: Tue, 07 Feb 2012 13:20:14 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <4F3116FE.4080704@gmail.com> Le 06/02/12 22:26, Ingo Thierack a ?crit : > Hello, > > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. > > If I do an search in an mailfolder, i get in the dovecot-log > > 2012-02-06 22:17:11 | dovecot: | > indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot > > > Log from solr. > Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/select > params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} > hits=0 status=0 QTime=1 > > > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. > > Regard > Ingo Thierack Same thing here. Tried with 2.1-rc1 and rc5. No results. From moseleymark at gmail.com Wed Feb 8 01:51:50 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 7 Feb 2012 15:51:50 -0800 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: On Tue, Feb 7, 2012 at 4:08 AM, Mark Zealey wrote: > 06-02-2012 22:47, Timo Sirainen yazm??: > >> On 3.2.2012, at 16.16, Mark Zealey wrote: >> >>> I was doing some testing on sdbox yesterday. Basically I did the >>> following procedure: >>> >>> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >>> 2) Create a copy of the index file (no cache file created yet) >>> 3) deliver another message to the mailbox (u.3) >>> 4) copy back index file from stage (2) >>> 5) deliver new mail >>> >>> Then the message delivered in stage 3 ie u.3 gets replaced with the >>> message delivered in (5) also called u.3. >> >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > > I've not actually tried this patch yet, but looking at it, it is perhaps > useful for the situation I described below when the index is corrupt. In > this case I am describing however, the not is NOT corrupt - it is simply an > older version (ie it only thinks there are the first 2 mails in the > directory, not the 3rd). This could happen for example when mails are being > stored on different storage than indexes; say for example you have 2 servers > with remote NFS stored mails but local indexes that rsync between the > servers every hour. You manually fail over one server to the other and you > then have a copy of the correct indexes but only from an hour ago. The mails > are all there on the shared storage but because the indexes are out of date, > when a new message comes in it will be automatically overwritten. > >>> (speaking of which, it would be great if force-resync also rebuilt the >>> cache files if there are valid cache files around, rather than just doing >>> away with them) >> >> Well, ideally there shouldn't be so much corruption that this matters.. > > > That's true, but in our experience we usually get corruption in batches > rather than a one-off occurrence. Our most common case is something like > this: Say for example there's an issue with the NFS server (assuming we are > storing indexes on there as well now) and so we have to killall -9 dovecot > processes or similar. In that case you get a number of corrupted indexes on > the server. Rebuilding the indexes generates an IO storm (say via lmtp or a > pop3 access); then the clients log in via imap and we have to re-read all > the messages to generate the cache files which is a second IO storm. If the > caches were rebuilt at least semi-intelligently (ie you could extract from > the cache files a list of things that had previously been cached) that would > reduce the effects of rare storage level issues such as this. > > Mark What about something like: a writer to an index/cache file checks for the existence of .1. If it doesn't exist or is over a day old, if the current index/cache file is not corrupt, take a snapshot of it as .1. Then if an index/cache file is corrupt, it can check for .1 and use that as the basis for a rebuild, so at least only a day's worth of email is reverted to its previous state (instead of all of it), assuming it's been modified in less than a day. Clearly it'd take up a bit more disk space, though the various dovecot.* files are pretty modest in size, even for big mailboxes. Or it might be a decent use case for some sort of journaling, so that the actual index/cache files don't ever get written to, except during a consolidation, to roll up journals once they've reached some threshold. There'd definitely be a performance price to pay though, not to mention breaking backwards compatibility. And I'm just throwing stuff out to see if any of it sticks, so don't mistake this for even remotely well thought-out suggestions :) From hmoreno at gmv.com Wed Feb 8 10:58:30 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Wed, 8 Feb 2012 09:58:30 +0100 Subject: [Dovecot] vsz_limit Message-ID: Hello Timo and dovecot teamwork, We have dovecot 2.0.10 installed with this configuration: # dovecot -n # 2.0.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.18.el5 x86_64 Scientific Linux SL release 5.2 (Boron) auth_cache_size = 500 k auth_cache_ttl = 1000 secs disable_plaintext_auth = no first_valid_uid = 501 lock_method = dotlock mail_fsync = always mail_gid = vmail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota autocreate mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin/autocreate = Spam plugin/autosubscribe = Spam plugin/quota = maildir:_backup:ns= plugin/quota_warning = storage=75%% quota-warning 75 %u plugin/quota_warning2 = storage=90%% quota-warning 90 %u plugin/sieve = %h/.dovecot.sieve plugin/sieve_dir = %h/sieve protocols = imap pop3 sieve service anvil { client_limit = 5000 } service auth { client_limit = 9000 unix_listener auth-master { group = vmail mode = 0700 user = vmail } user = root vsz_limit = 2 G } service imap-login { executable = /usr/local/dovecot/libexec/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2048 process_min_avail = 20 user = vmail } service imap { executable = /usr/local/dovecot/libexec/dovecot/imap process_limit = 2048 } service managesieve-login { executable = /usr/local/dovecot/libexec/dovecot/managesieve-login inet_listener sieve { address = * port = 4190 } inet_listener sieve_deprecated { address = * port = 12000 } user = vmail } service managesieve { executable = /usr/local/dovecot/libexec/dovecot/managesieve } service pop3-login { executable = /usr/local/dovecot/libexec/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 2048 process_min_avail = 20 user = vmail } service pop3 { executable = /usr/local/dovecot/libexec/dovecot/pop3 process_limit = 2048 } service quota-warning { executable = script /usr/local/dovecot/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0700 user = vmail } } ssl_cert = fue rechazado automaticamente por nuestro sistema :%n%r rejection_subject = Rechazado: %s sendmail_path = /usr/sbin/exim } protocol sieve { managesieve_implementation_string = dovecot Pigeonhole managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 } The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? Thank you very much in advanced. Kind regards. ________________________________ [cid:image001.gif at 01CCE647.65B8FB30] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [cid:image002.gif at 01CCE647.65B8FB30] [cid:image003.gif at 01CCE647.65B8FB30] [cid:image004.gif at 01CCE647.65B8FB30] [cid:image005.gif at 01CCE647.65B8FB30] ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From warden at geneseo.edu Wed Feb 8 18:20:31 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 8 Feb 2012 11:20:31 -0500 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <86E1DCD3-F1A8-43B5-AC08-E8CAF9094C38@geneseo.edu> On Feb 8, 2012, at 3:58 AM, H?ctor Moreno Blanco wrote: -snip- > service imap-login { > executable = /usr/local/dovecot/libexec/dovecot/imap-login > inet_listener imap { > address = * > port = 143 > } > inet_listener imaps { > address = * > port = 993 > } > process_limit = 2048 > process_min_avail = 20 > user = vmail > } > service imap { > executable = /usr/local/dovecot/libexec/dovecot/imap > process_limit = 2048 > } - snip, again - > > The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? I'm not 100% sure but if this is the case but it looks like you setting process_limit and process_min_avail in the imap-login service block is putting dovecot in "High Security" mode. http://wiki2.dovecot.org/LoginProcess has good details on that. In short, I suspect your settings are creating a new imap-login process for each new IMAP connection. As I understand it, vsz_limit is more important when running in "High Performance" mode where you have just 1 imap-login process that handles all new IMAP connections. That process can grow quite large if you have a lot of simultaneous IMAP clients. > > Thank you very much in advanced. > > Kind regards. > > ________________________________ > [cid:image001.gif at 01CCE647.65B8FB30] > > H?ctor Moreno Blanco > Divisi?n de Seguridad e Infraestructuras / > Security and Infrastructures Division > > GMV > Isaac Newton, 11 > P.T.M. Tres Cantos > E-28760 Madrid > Tel. +34 91 807 21 00 > Fax +34 91 807 21 99 > www.gmv.com > [cid:image002.gif at 01CCE647.65B8FB30] > > > > [cid:image003.gif at 01CCE647.65B8FB30] > > > > [cid:image004.gif at 01CCE647.65B8FB30] > > > > [cid:image005.gif at 01CCE647.65B8FB30] > > > > > > > ______________________ > This message including any attachments may contain confidential > information, according to our Information Security Management System, > and intended solely for a specific individual to whom they are addressed. > Any unauthorised copy, disclosure or distribution of this message > is strictly forbidden. If you have received this transmission in error, > please notify the sender immediately and delete it. > > ______________________ > Este mensaje, y en su caso, cualquier fichero anexo al mismo, > puede contener informacion clasificada por su emisor como confidencial > en el marco de su Sistema de Gestion de Seguridad de la > Informacion siendo para uso exclusivo del destinatario, quedando > prohibida su divulgacion copia o distribucion a terceros sin la > autorizacion expresa del remitente. Si Vd. ha recibido este mensaje > erroneamente, se ruega lo notifique al remitente y proceda a su borrado. > Gracias por su colaboracion. > > ______________________ > Hope this helps, David Warden From a23s4a2008 at yandex.ru Wed Feb 8 20:56:40 2012 From: a23s4a2008 at yandex.ru (Maxim) Date: Wed, 08 Feb 2012 22:56:40 +0400 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> Message-ID: <4F32C568.4050202@yandex.ru> 07.02.2012 0:27, Timo Sirainen ?????: > On 3.2.2012, at 15.15, User Max wrote: > >> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted > service auth { > user = root > } > > Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. From tss at iki.fi Thu Feb 9 01:26:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:26:04 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: <43ABD2D0-93D3-4E00-AC16-66EC75870EC2@iki.fi> On 7.2.2012, at 14.08, Mark Zealey wrote: >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. I don't recommend using local indexes with dbox, since there is actual data loss if they're not up to date (flags, and with mdbox the user may have copied/moved the mail elsewhere). Still, better to catch this situation than not: http://hg.dovecot.org/dovecot-2.1/rev/09db0f7aa6ce >>> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) >> Well, ideally there shouldn't be so much corruption that this matters.. > > That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Well, the decisions are now remembered: http://hg.dovecot.org/dovecot-2.1/rev/d8d214cc1936 That can't really be improved .. If nothing is deleted from cache, it might contain invalid data and doveadm force-resync wouldn't be doing its job right. If anything is added to cache, it would require reading and parsing the mail contents during rebuild, and that's not in any way better than letting the imap processes do it later when the mailbox isn't locked. From tss at iki.fi Thu Feb 9 01:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:29:20 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? From tss at iki.fi Thu Feb 9 01:36:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:36:50 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> On 6.2.2012, at 23.26, Ingo Thierack wrote: > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. Lucene should work.. > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. You mean you deleted Solr index, so that it's empty? That should work too. Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). From tss at iki.fi Thu Feb 9 01:40:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:40:19 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <4F32C568.4050202@yandex.ru> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> <4F32C568.4050202@yandex.ru> Message-ID: On 8.2.2012, at 20.56, Maxim wrote: >>> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted >> service auth { >> user = root >> } >> >> Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. >> > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. The problem is that auth worker process can't do getpwnam() call. Typically that would read from /etc/passwd, but I don't know if in OpenBSD if you have configured it to read from elsewhere (AD?). Anyway, I don't know why it would give EPERM error since the process is running as root. You could try ktraceing it to see what actually fails. From tss at iki.fi Thu Feb 9 01:42:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:42:11 +0200 Subject: [Dovecot] Fedora 16 configuration In-Reply-To: References: Message-ID: <1B36032D-0D9D-4CED-9D4C-6410C6C57D86@iki.fi> On 7.2.2012, at 22.44, Cliff Hayes wrote: > Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth > attempts): rip=x.x.x.x, lip=x.x.x.x From http://wiki2.dovecot.org/WhyDoesItNotWork: Aborted login (no auth attempts) means that the client isn't even attempting to log in. Most likely you have disable_plaintext_auth=yes (default) and the client isn't configured to use SSL/TLS (or you've also set ssl=no). From tss at iki.fi Thu Feb 9 01:48:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:48:09 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120207082510.GA24432@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) .. > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > other ideas for what might be causing it ? The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS (30 secs). It still shouldn't have crashed of course, and that crash is already fixed in v2.1 (in the LMTP simplification change). Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. From tss at iki.fi Thu Feb 9 01:49:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:49:38 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: Message-ID: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director From tss at iki.fi Thu Feb 9 03:04:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:04:04 +0200 Subject: [Dovecot] force-resync fails to recover all messages in mdbox In-Reply-To: <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> References: <20120131174839.13512v46jc7ur23b.lealanko@webmail.helsinki.fi> <38EB3A30-DFD5-484B-852B-327BDA5E936E@iki.fi> <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> Message-ID: <4BDF4036-EE45-42CC-9A34-D565D222107B@iki.fi> On 31.1.2012, at 18.34, Lauri Alanko wrote: > Well, well, well. This is interesting. Back with the indices > created by dsync: > > $ doveadm fetch guid all | grep guid: | sort | uniq -c | sort -n | tail > 17 guid: 1b28b22d4b2ee2885b5b81221c41201d > 17 guid: 730c692395661dd62f82088804b85652 > 17 guid: 865e1537fddba6698e010d0b9dbddd02 .. http://hg.dovecot.org/dovecot-2.0/rev/4a0b7dec3a22 avoids force-resync deleting these duplicates. It also logs a warning about the duplicates. http://hg.dovecot.org/dovecot-2.1/rev/2500de8f1f51 implements mbox_md5=all setting which avoids creation of these duplicates in the first place. I thought about adding some duplicate detection also to dsync (or anywhere in its path), but I couldn't do it without impacting performance in normal operation. > The complexity and opaqueness of the mdbox format is a worrisome. > It would ease my mind quite a bit if there were a simple tool > that would just dump out the plain message contents that are > stored inside the storage files, without involving any of > dovecot's index machinery. Then I would at least know that > whatever happens, as long as the storage files stay intact, I can > always migrate my mails into some other format. By using Dovecot indexes you could use e.g. "doveadm fetch" to dump them. Also "doveadm dump" can dump the dbox files' metadata, but not the message contents themselves. It probably wouldn't be difficult to implement that though. Also alternatively you could build something based on http://dovecot.org/tools/mdbox-obfuscate.pl From tss at iki.fi Thu Feb 9 03:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:45:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? In-Reply-To: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> References: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 19.59, Lauri Alanko wrote: > When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. > > This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Fixed: http://hg.dovecot.org/dovecot-2.0/rev/d83356f1fce6 From tss at iki.fi Thu Feb 9 03:59:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:59:30 +0200 Subject: [Dovecot] LSUB and subscribed folders In-Reply-To: <201112221334.06087.p.heinlein@heinlein-support.de> References: <201112221334.06087.p.heinlein@heinlein-support.de> Message-ID: <3477C497-8041-4F0C-8FB2-FA90394A231B@iki.fi> Hi, I've finally gotten to last year's mail backlog. :) On 22.12.2011, at 14.34, Peer Heinlein wrote: > User "bar" has some folders and one shared folder from "foo" if I use the > "LIST"-command, which should list AFAIK *all* folders: Yes. > Using the LSUB-command, which should only list subscribed folders I can see > some more shared folders. AFAIK the output of LSUB must be always less then > LIST: As Michael mentioned, LSUB can show some nonexistent mailboxes also that haven't been unsubscribed. > And why is /shared/foo/INBOX listed with "HasNoChildren"? It has no visible existing children (the LSUB entries didn't exist or no longer had necessary ACLs). > And, btw: Looks like's not possible to use the "doveadm"-command to get a > list of a user's complete folder list. Maybe "doveadm mailbox" should get > some extra parameters to geht all folders / only subscribe folders / only > shared folders: Yes, that would be nice .. but maybe this is enough for now: http://hg.dovecot.org/dovecot-2.1/rev/b600af017598 From tss at iki.fi Thu Feb 9 04:18:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:18:50 +0200 Subject: [Dovecot] crash on shared folders if remote user has empty $home In-Reply-To: <201112222201.41781.p.heinlein@heinlein-support.de> References: <201112222201.41781.p.heinlein@heinlein-support.de> Message-ID: On 22.12.2011, at 23.01, Peer Heinlein wrote: > d) If foo logs in, dovecot's exploring the shared folders given from the > non-existing user bar. But bar's entry in LDAP doesn't have $home any > more. imap crashes and closes the TCP-connection immediately. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/09f964850245 From tss at iki.fi Thu Feb 9 04:29:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:29:09 +0200 Subject: [Dovecot] doveadm index In-Reply-To: <4EF218F7.2050709@gmx.de> References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: On 21.12.2011, at 19.35, e-frog wrote: >> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' > > Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. Yes, looks like it does. I changed the doveadm help output to make it clearer. > At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. > > Is there a way to examine the search index? Doveadm dump doesn't seem to like it. > > $: doveadm dump dovecot.index.search > $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. From tss at iki.fi Thu Feb 9 04:31:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:31:44 +0200 Subject: [Dovecot] Syncing shared mailboxes In-Reply-To: <4EE62AEB.2020607@graband.de> References: <4EE62AEB.2020607@graband.de> Message-ID: Hi, On 12.12.2011, at 18.25, Knud Dombrowsky wrote: > while trying to sync the mailboxes of several users who use and share > their mailboxes dsync prints this message: > > dsync-local(): Warning: Subscriptions file > /home//Maildir/subscriptions: Removing > invalid entry: shared// > > The problem is: Every user has to subscribe the shared folder again > after each run of dsync... > Is dsync capable of syncing shared mailboxes? Are there any > preliminaries to do so? Sorry for late reply. This should be fixed in v2.1 nowadays. From tss at iki.fi Thu Feb 9 04:52:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:52:08 +0200 Subject: [Dovecot] 2.0.14 doveadm proxy list segfault In-Reply-To: <20110929172134.GA25237@hawkeye.rutgers.edu> References: <20110929172134.GA25237@hawkeye.rutgers.edu> Message-ID: <142FE442-A257-4BB5-A8D6-37884C7132D0@iki.fi> Hi, On 29.9.2011, at 20.21, Tom Pawlowski wrote: > Periodic segfaults with doveadm proxy list: > > root at director1:~) doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault (core dumped) Do you still get these? gdb backtrace would be helpful. It already says "core dumped", so all that should be needed is: gdb doveadm core bt full Also it shows connection to "ipc" server dying .. Does it also crash? Anything in Dovecot's error log? From tss at iki.fi Thu Feb 9 05:02:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 05:02:02 +0200 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <20110926165651.GA20769@informatik.tu-muenchen.de> References: <20110926165651.GA20769@informatik.tu-muenchen.de> Message-ID: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Hi, On 26.9.2011, at 19.56, Christoph Bussenius wrote: > /* make sure two hierarchy separators aren't next to each others */ > for (p = storage_name+1; *p != '\0'; p++) { > if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { > client_send_tagline(cmd, "NO Invalid mailbox name."); > return NULL; > } > } > > The loop iterates over the string storage_name starting from its second > byte. However, the string may be "". This is the case if you select > the root of a namespace, like in "SELECT shared" or > "SELECT shared/user1". In that case, the code will read past the end of > the buffer from random memory until it finds a zero byte or a duplicate > separator. I finally managed to get this far in my mail backlog. :) Yes, that's a bug. Fixed now slightly differently than you: http://hg.dovecot.org/dovecot-2.0/rev/d406e376f8ee From ybhu at hk1.ibm.com Thu Feb 9 10:36:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 9 Feb 2012 16:36:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: Thanks Timo, I just tried out the Director. One question is about the re-redirection. I know director will redirect all the simultaneous requests from the same user to only a single server at the same time. The question is how to manage the time period after last connection to re-decide to redirect which machine? director_user_expire? Look like not. I did one test, set director_user_expire = 1 min, then keep sending requests to the director in 2 min interval, the result is it keeps redirect to the same back end server. Actually what i want is the "secondary load balancer layer" can redirect requests to random back end. How to manage it? Only after the files on the previous back end is expired? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 07:49 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic17213.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From hmoreno at gmv.com Thu Feb 9 10:41:34 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 09:41:34 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: Hello Timo, I can see these errors, but I'm not sure if they have something to do with my problem: ... Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe ... Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) ... Do you see anything wrong? Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. I appreciate your answers. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 0:29 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From busseniu at in.tum.de Thu Feb 9 13:31:10 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Thu, 09 Feb 2012 12:31:10 +0100 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> References: <20110926165651.GA20769@informatik.tu-muenchen.de> <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Message-ID: <4F33AE7E.5070008@in.tum.de> Hi, I'm glad to see my report finally arrive, thank you :) On 09.02.2012 04:02, Timo Sirainen wrote: > Fixed now slightly differently than you: No problem ? I agree that my code was a bit kludgy. I noticed that my original mail might be a bit unclear: > > while trying to investigate the bug I reported last week, I found that > > there is a broken boundary check So I just want to make clear that this patch does not fix the other problem that I reported at http://www.dovecot.org/list/dovecot/2011-September/061316.html (?Strange behavior from shared namespaces and INBOX, probably a bug?). Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Thu Feb 9 14:52:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:52:42 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. From tss at iki.fi Thu Feb 9 14:55:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:55:45 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. From janfrode at tanso.net Thu Feb 9 14:56:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 9 Feb 2012 13:56:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: <20120209125652.GA18552@dibs.tanso.net> On Thu, Feb 09, 2012 at 01:48:09AM +0200, Timo Sirainen wrote: > On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > > > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) > .. > > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > > other ideas for what might be causing it ? > > > The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS > (30 secs). It's actually 60 sec in v2.0 http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 > It still shouldn't have crashed of course, and that crash is already fixed in v2.1 > (in the LMTP simplification change). Do you think we should rather run v2.1-rc* on our dovecot directors (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? > Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. Thanks, well consider that option if it crashes too often... Have only seen this problem once for the last week. -jf From tss at iki.fi Thu Feb 9 15:04:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 15:04:16 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120209125652.GA18552@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> <20120209125652.GA18552@dibs.tanso.net> Message-ID: On 9.2.2012, at 14.56, Jan-Frode Myklebust wrote: >>> Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any >>> other ideas for what might be causing it ? >> >> The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> (30 secs). > > It's actually 60 sec in v2.0 > > http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 30. LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is not LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> It still shouldn't have crashed of course, and that crash is already fixed in v2.1 >> (in the LMTP simplification change). > > Do you think we should rather run v2.1-rc* on our dovecot directors > (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? Yes, I've done a lot of improvements to proxying and error handling/logging in v2.1. Also I'm planning on finishing my email backlog soon and making the last v2.1-rc before renaming it to v2.1.0. From tss at iki.fi Thu Feb 9 16:34:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 16:34:22 +0200 Subject: [Dovecot] fts (lucene): indexing of virtual mailboxes? In-Reply-To: References: Message-ID: <1328798062.11500.131.camel@innu> Hi, On Fri, 2011-09-23 at 16:49 +0200, Lutz Pre?ler wrote: > Hello, > > (recent 2.1alpha2 variant - my test setup known to Timo). > No time to diagnose in depth at the moment, but I just noticed > that SEARCHing in virtual mailboxes seems not to create lucene > index content of its own but use those of referenced mailboxes? > The problem is that no new indexing takes place. > Example: with > INBOX > INBOX.in% > all > in dovecot-virtual, for a given query I only get matches from > those mailboxes searched in previously. This was a long time ago, but I just tested and looks like it works nowadays. From apm at one.com Thu Feb 9 16:35:21 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 09 Feb 2012 15:35:21 +0100 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox Message-ID: <4F33D9A9.2070605@one.com> Hi, Considering the scenario, where you have some old account with a different POP3 UIDL format and you migrate them to dovecot. So these old UIDLs would be saved to dovecot-uidlist. At some later time you want to introduce a virtual POP3 INBOX like described on: http://wiki.dovecot.org/Plugins/Virtual So you decide to make the new UIDL format "%f" - to make them unique across folders. So far so good. But then you decide to migrate to mdbox with all your old UIDLs. The docs says that saving old UIDLs is only supported in Maildir and that %f is only supported in Maildir. So is this at all possible? Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? /Peter From hmoreno at gmv.com Thu Feb 9 17:05:09 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 16:05:09 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> Message-ID: Hello Timo, There is no other problem as far as I know. That's why I think is has something to do with dovecot. Specifically with the imap an imap-login process, as I observe in the processes status. Anyway, the system load is not high enough to cause these problems. However, imap service doesn't work properly. So, in order to enable the login process in high performance mode I add the parameter service_count = 0. Right? I'll let you know if this helps. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 13:53 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: > Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: > Error: read(imap) failed: Remote closed connection (process_limit > reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: > fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth > request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From tss at iki.fi Thu Feb 9 17:19:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 17:19:40 +0200 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox In-Reply-To: <4F33D9A9.2070605@one.com> References: <4F33D9A9.2070605@one.com> Message-ID: <1328800780.11500.135.camel@innu> On Thu, 2012-02-09 at 15:35 +0100, Peter Mogensen wrote: > Hi, > > Considering the scenario, where you have some old account with a > different POP3 UIDL format and you migrate them to dovecot. > > So these old UIDLs would be saved to dovecot-uidlist. > > At some later time you want to introduce a virtual POP3 INBOX like > described on: > http://wiki.dovecot.org/Plugins/Virtual > > So you decide to make the new UIDL format "%f" - to make them unique > across folders. > > So far so good. Assuming the messages are in the same order, so far so good. > But then you decide to migrate to mdbox with all your old UIDLs. > The docs says that saving old UIDLs is only supported in Maildir and > that %f is only supported in Maildir. > > So is this at all possible? > > Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? %g and %f are equal with Maildir. And if you migrated with dsync from maildir to mdbox, then all GUIDs and POP3 UIDLs are preserved. But test it first! The main potential problem is that although UIDLs are preserved, their order isn't and POP3 clients don't like the order changing. With Maildir uidlist you can reorder POP3 mails to different than IMAP mails, but with mdbox you can't currently. From tss at iki.fi Thu Feb 9 19:34:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:34:06 +0200 Subject: [Dovecot] [dovecot] Getting duplicates when using snarf plugin with mbox backend In-Reply-To: <4DF609B1.8020309@psu.edu> References: <4DF609B1.8020309@psu.edu> Message-ID: <1328808846.11500.136.camel@innu> On Mon, 2011-06-13 at 08:59 -0400, Jonathan SIegle wrote: > Running dovecot version 2.0.11. To reproduce, open two imap sessions and > issue a check command from each at the same time with new mail in the queue. > > 0 login testuser testpw > 1 select inbox > -- Deliver mail -- > 2 check Finally fixed: http://hg.dovecot.org/dovecot-2.0/rev/76220f2b5966 From tss at iki.fi Thu Feb 9 19:52:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:52:43 +0200 Subject: [Dovecot] BUG(?): Incorrect responses for ACL prohibited actions In-Reply-To: <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> References: <20110510124458.Horde.LnlpeIF5lbhNyYeqlCXUIdA@bigworm.curecanti.org> <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> Message-ID: <1328809963.11500.146.camel@innu> Hi, Continuing this old thread: On Tue, 2011-05-10 at 22:46 -0600, Michael M Slusarz wrote: > But as far as the NO for a non-silent STORE, it seems that RFC 4314 > [4] disagrees with you: > > STORE operation SHOULD NOT fail if the user has rights to modify > at least one flag specified in the STORE, as the tagged NO > response to a STORE command is not handled very well by deployed > clients. > > To me, the negative inference from this statement would be: "STORE > operation SHOULD fail if the user has no rights to modify at least one > flag specified in the STORE." That's not the negative of it. :) > At a minimum, a NOPERM response should be thrown, or else there is no > feedback at all why the flag was not set (without parsing ACLs). Perhaps OK [NOPERM] or some other kind of informational message about it .. But there's no way to do it with Dovecot's current API. Also RFC 3501 recommends implementing "session flags" for flags that cannot be permanently stored. So even if user doesn't have access to set any flags, a "well behaving IMAP server" (so not Dovecot :( ) would set those flags for the duration of the current session. Anyway, you can look at PERMANENTFLAGS reply to see if it's possible to set the flag, no need to look at ACLs. > >> My reading of this is that NOPERM should be returned for ANY ACL > >> prohibited action, not just for selecting or creating a mailbox. > >> Dovecot 2.0.12 does not return NOPERM for DELETE/EXPUNGE actions > >> (at a minimum) that are prohibited. > > > > I'm not really sure. Maybe for EXPUNGE a NO would be okay. For flag > > changes it's just annoying to see clients popup pointless error > > messages when trying to set a \Seen flag (or \Answered flag when > > replying). Apparently I've tried this earlier, since there's a comment in code: ret = acl_mailbox_right_lookup(_mail->box, ACL_STORAGE_RIGHT_EXPUNGE); if (ret <= 0) { /* if we don't have permission, silently return success so users won't see annoying error messages in case their clients try automatic expunging. */ From tss at iki.fi Thu Feb 9 20:00:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:00:05 +0200 Subject: [Dovecot] Crash on mail folder delete In-Reply-To: References: <4F20922C.60206@amfes.com> <54B5D728-EC26-4633-A927-7EC040043BF5@iki.fi> <4F20939E.4010903@amfes.com> Message-ID: <1328810405.11500.151.camel@innu> On Wed, 2012-01-25 at 16:04 -0800, Daniel L. Miller wrote: > On 1/25/2012 3:43 PM, Daniel L. Miller wrote: > > On 1/25/2012 3:42 PM, Timo Sirainen wrote: > >> On 26.1.2012, at 1.37, Daniel L. Miller wrote: > >> > >>> Attempting to delete a folder from within the trash folder using > >>> Thunderbird. I see the following in the log: > >> Dovecot version? > >> > > 2.1.rc3. I'm compiling rc5 now... > > > Error still there on rc5. > > Jan 25 16:03:47 bubba dovecot: imap(dmiller at amfes.com): Panic: file > mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: > (mailbox_list_is_valid_pattern(_list, name)) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/95a9428fe68b From tss at iki.fi Thu Feb 9 20:24:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:24:58 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> Message-ID: <1328811898.11500.157.camel@innu> On Sat, 2010-12-25 at 10:08 +0100, Thomas Leuxner wrote: > plugin { > quota = dict:user::file:%h/mdbox/dovecot-quota > quota_rule = *:storage=1GB > quota_rule2 = Trash:storage=+10%% > } > > Kick off a manual backup: > > $ dsync -u user at domain.tld backup mdbox://mdbox > > This results in doubling the quota for the backed up user. This is problematic. With dict quota you'll have this problem, because both source and destination uses the same file. So it would kind of make sense to disable quota for the destination dsync.. Except with Maildir++ the quota is stored in the Maildir root directory. There are no problems with dsyncing it, and you most likely wouldn't want quota disabled there. So .. I'm not really sure what I can do about this. There are some workarounds you could do, like: dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= (works only with latest 2.0/2.1 hg, but with older versions you could do e.g. -o mail_plugins=) Still, it would be nice if there was some generic solution to this. Perhaps the destination username should be something different, like "backup". In dict-sql case then it would modify "backup" user's quota. For dict-file the %h could maybe expand to backup user's homedir.. The backup username probably should be a parameter to dsync I guess.. But an extra parameter wouldn't fix this automatically.. From tss at iki.fi Thu Feb 9 20:39:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:39:57 +0200 Subject: [Dovecot] 2.1.rc1 (8a63f621bd2e): SiS permission issue + crash In-Reply-To: <4EE2D39C.2030106@localhost.localdomain.org> References: <4EE2D39C.2030106@localhost.localdomain.org> Message-ID: <1328812797.11500.159.camel@innu> On Sat, 2011-12-10 at 04:35 +0100, Pascal Volk wrote: > dsync -u test-0 at example.com mirror maildir:/tmp/Maildir > rm -rf Maildir && cp -a Maildir_org Maildir && chown -R 70010:70002 Maildir > dsync -vu test-1 at example.com mirror maildir:/tmp/Maildir > dsync(test-1 at example.com): Error: stat(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.4f06409857c627e0) failed: Permission denied > dsync(test-1 at example.com): Error: safe_mkstemp(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.) failed: Permission denied > dsync(test-1 at example.com): Panic: file dsync-worker-local.c: line 1644 (local_worker_save_msg_continue): assertion failed: (ret == -1) I couldn't reproduce this crash, but I guess this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/e29bc3eb0ba6 Also fixed a related problem where if dbox failed to save a message it still added it to index: http://hg.dovecot.org/dovecot-2.1/rev/98a59ac1f3d0 From tss at iki.fi Thu Feb 9 20:54:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:54:02 +0200 Subject: [Dovecot] Segfaul probably during dsync In-Reply-To: <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> References: <1c3f5d96e58a77359a4b56dac645eeda@127.0.0.1> <277a7ac434966ec84de651577ae7ebbe@127.0.0.1> <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> Message-ID: <1328813642.11500.161.camel@innu> On Tue, 2011-04-12 at 14:52 +0200, Matthias Rieber wrote: > Hi, > > it's caused by a virtual folder: > > INBOX.IBX.Folder1 > INBOX.Ordner.Folder1 > INBOX.Ordner.Folder1.* > OR (OR (OR HEADER FROM bar.com HEADER FROM bar.de) HEADER FROM > foo.com) HEADER FROM barfoos.net NOT HEADER FROM root@ NOT HEADER FROM > www-data@ SINCE 1-Jan-2010 > > When I delete the dovecot.index.search* files it works for a while but > fails again. Are you still getting these crashes? I tried to reproduce but couldn't. From tss at iki.fi Thu Feb 9 21:02:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:02:03 +0200 Subject: [Dovecot] Strange behavior from shared namespaces and INBOX, probably a bug In-Reply-To: <20110923121329.GA13235@informatik.tu-muenchen.de> References: <20110923121329.GA13235@informatik.tu-muenchen.de> Message-ID: <1328814123.11500.163.camel@innu> On Fri, 2011-09-23 at 14:13 +0200, Christoph Bussenius wrote: > Some folders of user1, including the INBOX, have been shared using these IMAP commands: > . login user1 XXXXXX > . setacl INBOX user2 lrwstiekx > . setacl box-a user2 lrwstiekx > > Now if we use telnet to log in as user2 and select "shared/user1", it will contain the same > mails as "shared/user1/INBOX". > > The really strange thing is that "SELECT"-ing "shared/user1" succeeds only > if it is the first command afted logging in. If it not the first > command (if e. g. the "LIST" or "SELECT" command has already been used), > then dovecot will report that the mailbox does not exist. v2.1 always fails to select "shared/user1". I don't think I'll bother figuring out why v2.0 doesn't, might not be an easy fix. Much of the code related to this was rewritten in v2.1. From tss at iki.fi Thu Feb 9 21:47:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:47:47 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328212183-sup-560@rose.makesad.us> References: <1328212183-sup-560@rose.makesad.us> Message-ID: <1328816867.11500.167.camel@innu> On Thu, 2012-02-02 at 14:59 -0500, Asheesh Laroia wrote: > I'm guessing this is some bad interaction with sdbox and partial file downloads? > > I haven't read the code for this, but I would guess the dsync process isn't being atomic > about file transfers, so it is leaving half-completed transfers in place, which results > in corrupt files when they're next examined. There were some problems related to this in dbox, although in your case it seems to be worse than what it should.. I've anyway done several fixes in v2.1. Can you try if these problems happen with it too? And in any case cleanup the dbox from the *.broken files, so that "doveadm force-resync" won't give any errors. From tss at iki.fi Thu Feb 9 23:15:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 23:15:15 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328816867.11500.167.camel@innu> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> Message-ID: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> On 9.2.2012, at 21.47, Timo Sirainen wrote: > I've anyway done several fixes in v2.1. Can you try if these problems > happen with it too? And in any case cleanup the dbox from the *.broken > files, so that "doveadm force-resync" won't give any errors. A bit more specifically: The last such dbox bug was fixed only today, so you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this week. And in general: It would be helpful to have a clean fully working dbox, and then know the *first* error(s) that gets printed about dsync corrupting it. Otherwise it's difficult to guess what are some old problems and what are new ones and which problems happens only because of another problem. From asheesh at asheesh.org Thu Feb 9 23:16:32 2012 From: asheesh at asheesh.org (Asheesh Laroia) Date: Thu, 9 Feb 2012 16:16:32 -0500 (EST) Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> Message-ID: On Thu, 9 Feb 2012, Timo Sirainen wrote: > On 9.2.2012, at 21.47, Timo Sirainen wrote: > >> I've anyway done several fixes in v2.1. Can you try if these problems >> happen with it too? And in any case cleanup the dbox from the *.broken >> files, so that "doveadm force-resync" won't give any errors. > > A bit more specifically: The last such dbox bug was fixed only today, so > you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this > week. > > And in general: It would be helpful to have a clean fully working dbox, > and then know the *first* error(s) that gets printed about dsync > corrupting it. Otherwise it's difficult to guess what are some old > problems and what are new ones and which problems happens only because > of another problem. Good to know. This weekend I can try to set up something of a 'lab' for testing dsync + (s)dbox, both to see if I can reproduce the errors with the old versions, and to see if the new versions fix them. I'll keep in mind the consideration of knowing the first error that gets printed! -- Asheesh. From ybhu at hk1.ibm.com Fri Feb 10 05:12:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 11:12:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic08408.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ybhu at hk1.ibm.com Fri Feb 10 10:08:10 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 16:08:10 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: Searched the archive, and got the doveadm -a director-admin for local doveadm access, and -a host: port for remote doveadm access. And you said: http://dovecot.org/list/dovecot/2010-July/050731.html Now in my director, I have configed the userdb passwd, but the same error occur: doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but userdbs don't support it [root at c-dev1ws--01-01 conf_director.d]$dovecot -n -c /usr/local/etc/dovecot/dovecot_director.conf # 2.0.13: /usr/local/etc/dovecot/dovecot_director.conf # OS: Linux 2.6.18-274.17.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_debug = yes auth_debug_passwords = yes auth_mechanisms = xpreauth auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot_director director_mail_servers = 9.119.7.129 director_servers = 9.119.7.60:9090 disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = mdrop mail_nfs_index = yes mail_nfs_storage = yes mail_uid = mdrop mmap_disable = yes passdb { args = proxy=y port=144 nopassword=y driver = static } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { address = 9.119.7.60 port = 10143 } } ssl = no userdb { driver = passwd } By the way, I run the director and backend in the same server. Andy YB Hu/Hong Kong/IBM at IBMHK Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/10/2012 11:12 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic21544.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ms at mike2k.de Fri Feb 10 14:13:17 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 10 Feb 2012 13:13:17 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner Message-ID: Hi, I have shared one of my mailboxes to another user providing him full rights to that mailbox. $ doveadm acl get -u user1 doc ID Global Rights user=user2 admin create delete expunge insert lookup post read write write-deleted write-seen When that user creates a new mailbox within the shared mailbox, the mailbox is successfully created but with an empty ACL. $ doveadm acl get -u user1 doc/hallo ID Global Rights According to the wiki, it should inherit the ACL from its parent, i.e. have full permissions for user2. Dovecot logs the following errors when I access the parent folder the first time after the other user created the child mailbox: dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: Recent flags state corrupted for mailbox Sent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent This is on dovecot 2.1rc5. -Michael From giles at coochey.net Fri Feb 10 16:36:53 2012 From: giles at coochey.net (Giles Coochey) Date: Fri, 10 Feb 2012 14:36:53 +0000 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! Message-ID: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> So I went and migrated a mailbox from mdbox from maildir format. All seemed ok, but I've found that I now have some missing folders. The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What can be done? (I have a backup of the maildir box). Note though that mdbox is now a live mailbox and has sent and received mails since the migration... -- Message sent via my webmail account. From alexwbaule at gmail.com Fri Feb 10 19:39:39 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 10 Feb 2012 15:39:39 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? Message-ID: Hy Everyone...and Timo ! There is a way to know if a email was expunged (deleted from hard disk) inside a plugin ? I rewrite the expunge function, but the real expunge function is void, i can't know if was really expunged. There is a way to know this ? From mildred593+dovecot at gmail.com Fri Feb 10 20:54:36 2012 From: mildred593+dovecot at gmail.com (Mildred Ki'Lya) Date: Fri, 10 Feb 2012 19:54:36 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, I am trying to set up my own mail server (to get off Google) and I have a problem with mail delivery. I'm using Postfix as a SMTP front-end, and use the LMTP server included in dovecot to receive mail from postfix. I have a setup which works with the dovecot LDA, it involves calling dovecot-lda with the -d flag. Postfix computes the username by removing what's after the recipient_delimiter and the domain from the mail address, and gives this username to dovecot. But, I'd prefer using LMTP, as it is possible to reject unknown e-mails without having to duplicate the user database (it is explained here: http://wiki2.dovecot.org/LDA/Postfix ) I set up LMTP so that postfix uses the socket provided by dovecot, and it works. Only I get the following error message in the logs: Feb 10 18:05:53 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred at mildred.fr(in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> Feb 10 18:06:52 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred-test at mildred.fr (in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> I followed the explanations here: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP The problem is that my passwd-like file only contains a user named "mildred". I tried to add a user mildred at mildred.fr and mildred%mildred.frbut it doesn't work either. Do you know how I could tell dovecot to guess the username correctly ? Thanks, Mildred -- Mildred Ki'Lya http://mildred.fr From wrosenauer at gmail.com Fri Feb 10 22:27:51 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:27:51 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, On Fri, Feb 10, 2012 at 7:54 PM, Mildred Ki'Lya wrote: > I followed the explanations here: > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > The problem is that my passwd-like file only contains a user named > "mildred". I tried to add a user mildred at mildred.fr and > mildred%mildred.frbut it doesn't work either. > > Do you know how I could tell dovecot to guess the username correctly ? I think you need to set auth_username_format = %Ln which strips the domain part from the recipient. Wolfgang From e-frog at gmx.de Fri Feb 10 22:31:45 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 10 Feb 2012 21:31:45 +0100 Subject: [Dovecot] doveadm index In-Reply-To: References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: <4F357EB1.4000103@gmx.de> On 09.02.2012 03:29, wrote Timo Sirainen: > On 21.12.2011, at 19.35, e-frog wrote: > >>> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' >> >> Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. > > Yes, looks like it does. I changed the doveadm help output to make it clearer. Thanks. >> At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. >> >> Is there a way to examine the search index? Doveadm dump doesn't seem to like it. >> >> $: doveadm dump dovecot.index.search >> $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search > > No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. Ok, I'm meanwhile running v2.1 and switched to fts-lucene. Works like a charm! From wrosenauer at gmail.com Fri Feb 10 22:46:01 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:46:01 +0100 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 Message-ID: Hi, I finally migrated my IMAP server from cyrus to dovecot2. I have been using cyrus since many years and have not much experience with dovecot yet though. I've converted my imap server using cyrus2dovecot (several times until I thought it was correct) but there is one thing left which confuses me and I'm not sure what I can or need to do to change it. With Cyrus I used unixhierarchysep: yes and subfolders within Cyrus are always below INBOX. I'm used to that in my IMAP clients and wanted to preserve it with Dovecot. So during migration I finally made sure to create the subfolder again below INBOX wich looks fine in Thunderbird for example. But my mobile devices (Android based with K9) were displaying the folders with Cyrus just with their name w/o the INBOX prefix. (Mobile clients are bad in displaying folder hierarchies so it was convenient to have them looking like they were in parallel to the Inbox.) Now I get the folders displayed as INBOX.whatever and my Android mail clients fail to map them to the special (and localized) folders like Trash. While that seems consistent to my current hierarchy it's not what I wanted. So basically my two questions are: - Can I configure Dovecot in a way that it looks like Cyrus? Having subfolders below Inbox but let clients like those on Android display them besides Inbox? - If that's not possible, can I move the subfolders up one level so they are next to the Inbox? How would I do that? Sorry for the beginner questions. I played around with Dovecot before but using it in production environment disclosed some fine differences. Thanks, Wolfgang From glenmarton at gmail.com Fri Feb 10 23:54:42 2012 From: glenmarton at gmail.com (Glen Johnson) Date: Fri, 10 Feb 2012 16:54:42 -0500 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. Message-ID: I have an IMAP mail server running Postfix, and Dovecot. I have an old Notebook computer which I can view all e-mails. The problem I am having is when I try to use any other computer on the network to view the e-mail I can not see the contents of all the folders. I can see some of the folders and their content but not all. To answer a couple of preliminary questions: dovecot version 2.0.13, postfix version 2.8.3 dovecot.conf ------------------ # 2.0.13: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 4.0.1 i386 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir/ passdb { driver = passwd } passdb { args = /usr/pkg/etc/dovecot/passwd driver = passwd-file } protocols = imap service auth { user = root } service imap-login { executable = /usr/pkg/libexec/dovecot/imap-login } ssl_cert = References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 09.02.2012 um 19:24 schrieb Timo Sirainen: > > So .. I'm not really sure what I can do about this. There are some > workarounds you could do, like: > > dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= > Thanks tried this morning. I think it has some issues with options and crashes without options :/ $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] Aborted $ dovecot --version 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sat Feb 11 10:08:29 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 09:08:29 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 11.02.2012 um 08:58 schrieb Thomas Leuxner: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] > Aborted After having some coffee, above was inflicted by having insufficient disk space. Shouldn't crash anyway I guess. > $ dovecot --version > 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cannewilson at googlemail.com Sat Feb 11 12:09:29 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Sat, 11 Feb 2012 10:09:29 +0000 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. In-Reply-To: References: Message-ID: <201202111009.29259.cannewilson@googlemail.com> On Friday 10 February 2012 21:54:42 Anne Wilson wrote: > I have an IMAP mail server running Postfix, and Dovecot. I have an old > Notebook computer which I can view all e-mails. The problem I am having is > when I try to use any other computer on the network to view the e-mail I > can not see the contents of all the folders. I can see some of the folders > and their content but not all. Does your mail client support subscriptions? Check your menus for this - you may find that only some folders are subscribed to. Anne From tlx at leuxner.net Sat Feb 11 12:59:16 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 11:59:16 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox $ doveadm -f table quota get -u tlx at leuxner.net Please look into the crash though. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mark.zealey at webfusion.com Sat Feb 11 16:33:34 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 11 Feb 2012 16:33:34 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP Message-ID: <4F367C3E.2080806@webfusion.com> Hi there, I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: 451 4.3.0 Temporary user lookup failure and in the logs: 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly I'm sure this is an obvious issue for someone who's come across it before. The /var/run/dovecot/director-userdb socket is on the system; exactly the same permissions as the standard auth-userdb Here are the relevant bits of my config (I'm setting lmtp to listen on a different port than normal). Backend password database is mysql if that makes any difference. lmtp_proxy = yes service lmtp { inet_listener lmtp_test { port = 2450 } } protocol lmtp { auth_socket_path = director-userdb } Thanks, Mark From clube03 at gmail.com Sat Feb 11 20:46:32 2012 From: clube03 at gmail.com (Gustavo) Date: Sat, 11 Feb 2012 16:46:32 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: Ok, I study the wikki and the dovecot works. But when I try to send a mail using squirrel, I get this error on logs: Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from localhost.localdomain[127.0.0.1] Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication mechanisms Feb 11 18:41:56 jubileu postfix/master[18383]: warning: process /usr/lib/postfix/smtpd pid 19522 exit status 1 Feb 11 18:41:56 jubileu postfix/master[18383]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling How can I fix this? My conf files seems to be ok: ## Diretorio Base ## base_dir = /var/run/dovecot/ ## Protocolos que ele ira executar ## protocols = imap imaps pop3 pop3s log_timestamp = '%Y-%m-%d %H:%M:%S ' syslog_facility = mail ## Dados do SSL ## ssl_listen = localhost ssl_cert_file = /etc/postfix/mail-cert.pem ssl_key_file = /etc/postfix/mail-key.pem ssl_parameters_regenerate = 168 verbose_ssl = no mail_location = maildir:/var/vmail/%d/%u mail_access_groups = mail mail_debug = no first_valid_uid = 150 last_valid_uid = 150 maildir_copy_with_hardlinks = yes protocol imap { } protocol pop3 { # Login executable location. login_executable = /usr/lib/dovecot/pop3-login mail_executable = /usr/lib/dovecot/pop3 pop3_uidl_format = %08Xu%08Xv } protocol lda { sendmail_path = /usr/lib/sendmail auth_socket_path = /tmp/auth-master } auth_verbose = no auth_debug = no auth default { mechanisms = plain login passdb pam { } passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb passwd { } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /tmp/auth-master mode = 0600 user = vmail group = mail } client { path = /tmp/auth-client mode = 0660 user = vmail group = mail } } } dict { } plugin { } ######################################### And Postfix conf: # SASL Authentication smtpd_sasl_auth_enable = yes smtpd_sasl2_auth_enable = yes smtpd_sasl_exceptions_networks = $mynetworks smtpd_sasl_security_options = noanonymous broken_sasl_auth_clients = yes smtpd_sasl_path = private/auth smtpd_sasl_application_name = smtpd smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination permit_mynetworks = 127.0.0.0/8, 184.22.103.88/32, 184.22.103.87/32 Thanks!!! > -----Mensagem Original----- From: Timo Sirainen > Sent: Saturday, January 28, 2012 3:51 PM > To: Gustavo > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Problem with Postfix + Dovecot + MySQL + > Squirrelmail > > On 27.1.2012, at 14.32, Gustavo wrote: > > #service dovecot start >> Starting IMAP/POP3 mail server: dovecotLast died with error (see error >> log for more information): Auth process died too early - shutting down >> > > No need to keep guessing the problem. "See error log for more information" > like it says. http://wiki.dovecot.org/**Logging > > -- Gustavo - clube03 at gmail.com From tss at iki.fi Sun Feb 12 02:24:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 02:24:38 +0200 Subject: [Dovecot] [PATCH] Allow verification of user's hash in doveadm-pw In-Reply-To: References: Message-ID: <4F3706C6.10909@iki.fi> On 09/01/2011 01:17 AM, Spack wrote: > Hi, > > It may be useful for testing purpose to know if a generated password > hash correspond to a given clear password. This can be useful to check > if a hash generated by another program can be verified by Dovecot > without any errors. > > This patch adds the ability the verify a password hash using `doveadm > pw` via the `-V` option. Committed, with a few changes: http://hg.dovecot.org/dovecot-2.1/rev/fffa4d53e901 > + while ((ch = getopt(argc, argv, "lp:r:s:u:V::")) != -1) { "::" isn't standard, so I added a separate -t parameter to give the hash. From tss at iki.fi Sun Feb 12 05:39:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:39:20 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <4F367C3E.2080806@webfusion.com> References: <4F367C3E.2080806@webfusion.com> Message-ID: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> On 11.2.2012, at 16.33, Mark Zealey wrote: > I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: > > 451 4.3.0 Temporary user lookup failure > > and in the logs: > 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly Aren't there any other errors logged? I think there should be some error logged from either auth or director process. Although director is missing one error, but auth should have also logged an error in that case. Anyway, fixed by http://hg.dovecot.org/dovecot-2.1/rev/9b70d164e974 From tss at iki.fi Sun Feb 12 05:42:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:42:22 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Message-ID: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> On 11.2.2012, at 12.59, Thomas Leuxner wrote: > Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: > > $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. From tss at iki.fi Sun Feb 12 05:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:45:45 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> On 11.2.2012, at 9.58, Thomas Leuxner wrote: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 From tss at iki.fi Sun Feb 12 06:02:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:02:45 +0200 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 In-Reply-To: References: Message-ID: <5BD5F73F-E9D6-4803-BA51-59CC7B0D338E@iki.fi> On 10.2.2012, at 22.46, Wolfgang Rosenauer wrote: > With Cyrus I used > unixhierarchysep: yes > and subfolders within Cyrus are always below INBOX. .. > - Can I configure Dovecot in a way that it looks like Cyrus? Having > subfolders below Inbox but let clients like those on Android display > them besides Inbox? namespace { prefix = INBOX/ separator = / } From tss at iki.fi Sun Feb 12 06:06:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:06:58 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. From tss at iki.fi Sun Feb 12 06:09:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:09:55 +0200 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! In-Reply-To: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> References: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> Message-ID: <3F5BB562-6484-43B0-8914-434694F446DB@iki.fi> On 10.2.2012, at 16.36, Giles Coochey wrote: > So I went and migrated a mailbox from mdbox from maildir format. > All seemed ok, but I've found that I now have some missing folders. > > The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What Dovecot version? And give some specifics, like outputs of: ls -a ~/Maildir/ ls ~/mdbox/mailboxes/ or something similar. From tss at iki.fi Sun Feb 12 06:16:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:16:00 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: References: Message-ID: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> On 10.2.2012, at 14.13, Michael Stilkerich wrote: > I have shared one of my mailboxes to another user providing him full rights to > that mailbox. > > $ doveadm acl get -u user1 doc Is "doc" the namespace prefix? > When that user creates a new mailbox within the shared mailbox, the mailbox is successfully > created but with an empty ACL. Creates how? With IMAP client? > According to the wiki, it should inherit the ACL from its parent, i.e. have > full permissions for user2. Works with me. dovecof -n output? > Dovecot logs the following errors when I access the parent folder the first time > after the other user created the child mailbox: > > dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 > dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX > dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Are you trying to use per-user indexes? That's not going to work. From tss at iki.fi Sun Feb 12 06:19:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:19:38 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: Message-ID: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> On 10.2.2012, at 19.39, Alex Baule wrote: > Hy Everyone...and Timo ! > > There is a way to know if a email was expunged (deleted from hard > disk) inside a plugin ? I rewrite the expunge function, but the real > expunge function is void, i can't know if was really expunged. > > There is a way to know this ? It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. From tss at iki.fi Sun Feb 12 07:05:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:05:37 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSN for overquota. In-Reply-To: References: Message-ID: <4F3748A1.3030301@iki.fi> On 12/22/2010 08:08 PM, Mark Sapiro wrote: > It appears that there is some version of the Dovecot LDA that is > sending an RFC 3798 MDN rather than an RFC 3464 DSN to report > non-delivery to a user who is over quota. Fixed, better late than never :) http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 The MDN sending was simply copied from Cyrus. So Dovecot isn't the only server sending this. From tss at iki.fi Sun Feb 12 07:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:42:28 +0200 Subject: [Dovecot] v2.1.0 status Message-ID: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. I've gone through all of my old mails. Let me know if you think something is still missing / unfixed in v2.1 hg. Preferably today. Next up: Creating dovecot-2.2 tree with great new features. :) From tss at iki.fi Sun Feb 12 07:45:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:45:12 +0200 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> On 12.2.2012, at 7.42, Timo Sirainen wrote: > I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota and Sieve's reject mails now include DSN report instead of MDN report. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From tlx at leuxner.net Sun Feb 12 08:54:41 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:54:41 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: Am 12.02.2012 um 04:42 schrieb Timo Sirainen: > On 11.2.2012, at 12.59, Thomas Leuxner wrote: > >> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: >> >> $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox > > Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. > Indeed I did as it did not like the syntax: $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 08:56:24 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:56:24 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> Message-ID: <285AA29C-A430-44C5-9E9A-BD93A34291B8@leuxner.net> Am 12.02.2012 um 04:45 schrieb Timo Sirainen: > On 11.2.2012, at 9.58, Thomas Leuxner wrote: > >> $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox >> dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 Thanks. Crash gone. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 09:02:22 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 08:02:22 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: Am 01.01.2012 um 10:31 schrieb Thomas Leuxner: > $ doveadm acl get -u tlx at leuxner.net FOO > doveadm(tlx at leuxner.net): Error: Can't open mailbox FOO: Mailbox doesn't exist: FOO > ID Global Rights > $ echo $? > 0 Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Mark.Zealey at webfusion.com Sun Feb 12 10:23:21 2012 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Sun, 12 Feb 2012 08:23:21 +0000 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: Ah I should have checked this before - line before: 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Only other relevant piece of config: service auth { user = dovecot client_limit = 15000 } Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. Mark From listen at constabel.net Sun Feb 12 13:18:09 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 12:18:09 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash Message-ID: <20120212111808.GA27677@herakles.constabel-it.de> Hi, If I delete a folder "foo" in Thunderbird, it is moved to trash. [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/ insgesamt 16K drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 8 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:51 dbox-Mails drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 foo [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxx.de/test1/mdbox/mailboxes/Trash/foo/ insgesamt 12K drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 dbox-Mails [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/foo/dbox-Mails/ insgesamt 12K drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 . drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 .. -rw------- 1 vmail vmail 224 12. Feb 11:52 dovecot.index.log [root at dovecot1][tmp:exec](~) # If I delete foo from trash, thunderbird removed it from view, but if I restart thunderbird, the folder is still in trash. No error in dovecot log, no error in thunderbird. tcpdump: 156 45.101512 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=353 Ack=1617 Win=10144 Len=0 TSV=2414467591 TSER=251091441 157 45.349600 192.168.10.1 192.168.10.117 IMAP Request: 16 list "" "Trash/*" 158 45.349717 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\HasNoChildren) "/" "Trash/foo" 159 45.349756 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=375 Ack=1727 Win=10144 Len=0 TSV=2414467653 TSER=251091503 160 45.395546 192.168.10.1 192.168.10.117 IMAP Request: 17 list "" "Trash/foo/*" 161 45.395624 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\Noselect \HasNoChildren) "/" "Trash/foo/" 162 45.395662 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=401 Ack=1802 Win=10144 Len=0 TSV=2414467665 TSER=251091515 163 45.439998 192.168.10.1 192.168.10.117 IMAP Request: 18 delete "Trash/foo/" 164 45.440067 192.168.10.117 192.168.10.1 IMAP Response: 18 NO [CANNOT] Invalid mailbox name 165 45.440104 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=425 Ack=1839 Win=10144 Len=0 TSV=2414467676 TSER=251091526 166 45.493978 192.168.10.1 192.168.10.117 IMAP Request: 19 IDLE with dovecot 1.2 and Maildir I can delete folders from trash. Something wrong in my config? Thunderbird-Bug? # 2.1.rc5 (b9e74767cd39): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.4 auth_cache_size = 1 M auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot/dovecot-debug.log dict { expire = sqlite:/etc/dovecot/dovecot-dict-expire.conf.ext } disable_plaintext_auth = no hostname = mx4.xxxx.de info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_location = mdbox:~/mdbox mail_plugins = quota notify mail_log expire mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive/2011 { auto = subscribe } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = scheme=PLAIN username_format=%u /etc/imap.userdb driver = passwd-file } plugin { expire = Trash expire2 = Trash/* expire3 = Spam expire_dict = proxy::expire quota = dict:User quota::file:/mailspool1/%d/%n/dovecot-quota quota_rule = *:storage=9000M:messages=1000000 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at localhost protocols = " imap lmtp sieve pop3" ssl_ca = References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <4F37A343.1060702@localhost.localdomain.org> On 02/12/2012 12:18 PM Mike Constabel wrote: > Hi, > > If I delete a folder "foo" in Thunderbird, it is moved to trash. > ? > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. http://hg.dovecot.org/dovecot-2.1/file/b9e74767cd39/doc/example-config/conf.d/20-imap.conf#l47 Regards, Pascal -- The trapper recommends today: deadbeef.1204312 at localdomain.org From CMarcus at Media-Brokers.com Sun Feb 12 13:33:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:33:11 -0500 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: <4F37A377.80707@Media-Brokers.com> On 2012-02-11 1:46 PM, Gustavo wrote: > Ok, I study the wikki and the dovecot works. But when I try to send a mail > using squirrel, I get this error on logs: > > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from > localhost.localdomain[127.0.0.1] > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: > xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication > mechanisms Sorry, but this has nothing to do with dovecot, unless your intention is to use dovecot-sasl with your postfix... it appears you are using cyrus-sasl and postfix on your smtp server, that is where your problem lies. Easiest would be to configure postfix to use dovecot-sasl: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL -- Best regards, Charles From mstevens at imt-systems.com Sun Feb 12 13:37:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 12:37:18 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212111808.GA27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> On 12.02.2012 12:18, Mike Constabel wrote: > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. protocol imap { ... imap_client_workarounds = tb-extra-mailbox-sep } Best regards, Morten From CMarcus at Media-Brokers.com Sun Feb 12 13:38:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:38:11 -0500 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <4F37A4A3.4000204@Media-Brokers.com> On 2012-02-12 12:42 AM, Timo Sirainen wrote: > Next up: Creating dovecot-2.2 tree with great new features.:) Can't wait to read your follow-up email outlining these 'great new features' you expect to make it into 2.2... Thanks so much for dovecot Timo! I am amazed at how far it has come in the last few years, and apparently things will only get better. -- Best regards, Charles From ms at mike2k.de Sun Feb 12 14:14:03 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 13:14:03 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> Message-ID: <4F37AD0B.9040406@mike2k.de> Hi Timo, On 12-Feb-12 5:16, Timo Sirainen wrote: > On 10.2.2012, at 14.13, Michael Stilkerich wrote: > >> I have shared one of my mailboxes to another user providing him full rights to >> that mailbox. >> >> $ doveadm acl get -u user1 doc > > Is "doc" the namespace prefix? No, 'doc' is the name of the shared parent mailbox under that the 2nd user created the new mailbox 'hallo'. To clarify: user1 shares the mailbox 'doc' with max permissions to user2. user2 then creates a new mailbox under the parent 'doc'. >> When that user creates a new mailbox within the shared mailbox, the mailbox is successfully >> created but with an empty ACL. > > Creates how? With IMAP client? Yes, using Thunderbird / Roundcube Webmail. >> According to the wiki, it should inherit the ACL from its parent, i.e. have >> full permissions for user2. > > Works with me. dovecof -n output? The config is attached. The IMAP-Client is signaled an error by dovecot, but the log file does not contain an error. At the filesystem level - The mailbox was created as a sub folder of 'doc'. - It contains nothing but an empty sub folder 'dbox-Mails'. - It does specifically not contain an dovecot-acl file. - The sharing user's (user1) dovecot-acl-list file was not updated to list the newly created subfolder. - The acl dictionary 'shared-mailboxes' was also not touched, but this is probably correct since it already contains a record that user1 shares mailboxes to user2. - When user1 accesses the new mailbox that user2 created, the below error gets logged and a dovecot.index.log file is created in the until then empty 'dbox-Mails' folder of the new mailbox. >> Dovecot logs the following errors when I access the parent folder the first time >> after the other user created the child mailbox: >> >> dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 >> dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX >> dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent > > Are you trying to use per-user indexes? That's not going to work. I don't think so. I'm using dbox and thus have not added a custom :INDEX=... to the location. -Michael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot.conf URL: From listen at constabel.net Sun Feb 12 15:27:19 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 14:27:19 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> Message-ID: <20120212132719.GB27677@herakles.constabel-it.de> Hi, On Sun, 12 Feb 2012, Morten Stevens wrote: > On 12.02.2012 12:18, Mike Constabel wrote: > > >If I delete foo from trash, thunderbird removed it from view, > >but if I restart thunderbird, the folder is still in trash. > > > >No error in dovecot log, no error in thunderbird. > > protocol imap { > ... > imap_client_workarounds = tb-extra-mailbox-sep > } Ok, this seems to work, (I thought I tried it before.. but now it works. :) Another Question to this Option: # tb-extra-mailbox-sep: # With mbox storage a mailbox can contain either mails or submailboxes, # but not both. Thunderbird separates these two by forcing server to # accept '/' suffix in mailbox names in subscriptions list. I thought this is not needed because I use mdbox. But mbox here means also mdbox and sdbox? Also "With mbox storage a mailbox can contain either mails or submailboxes, but not both" is only correct for mbox? In mdbox this is possible. This description should contain a litte bit more info for mbox/sdbox/mdbox? Should I use all 3 workarounds if I don't know which clients are in use? LAYOUT=fs and mdbox Regards, Mike From CMarcus at Media-Brokers.com Sun Feb 12 15:43:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 08:43:51 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <4F37C217.7040707@Media-Brokers.com> Hi Timo, One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? Anyway here is the thread outlining the problem with dovecot's SIS and backups: http://www.dovecot.org/list/dovecot/2011-December/062340.html Have you given this any more thought? -- Best regards, Charles From forumer at smartmobili.com Sun Feb 12 15:52:54 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 14:52:54 +0100 Subject: [Dovecot] Lost in configuration Message-ID: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Hi, I am trying to configure dovecot/postfix with virtual users and sasl auth but there are so many tutorials with mistakes and subtle differences that at the end I am lost. So I am running Ubuntu server 11.04 (natty) and when I installed my server I had followed the following guide http://workaround.org/ispmail/etch Then I have migrated to dovecot 2.1 and now I am trying to allow authenticated user to use my smtp server because for the moment I get : Relay access denied Here is my configuration (first is file hierarchy followed by file contents) and I would like to know what I need to do to allow authenticated user send email. In this tutorial http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql it seems I have to add some configuration lines inside /etc/pam.d/smtp and add database configuration inside /etc/postfix/sasl/smtpd.conf. But what I find weird is the fact I have to configure some sql queries from smtpd.conf while normally dovecot has already all the information to do this kind of query. So before to keep on modifying my config I would like to be sure I am following the right way. Thanks ########################### /etc/postfix ########################### drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ -rw-r--r-- 1 root root 373 2011-07-24 11:50 dynamicmaps.cf -rw-r--r-- 1 root root 1928 2012-02-12 12:45 main.cf -rw-r--r-- 1 root root 5762 2011-08-03 22:06 master.cf -rw-rw---- 1 root postfix 140 2011-07-24 11:43 mysql-virtual-alias-maps.cf -rw-rw---- 1 root postfix 132 2011-07-24 11:47 mysql-virtual-mailbox-domains.cf -rw-rw---- 1 root postfix 128 2011-07-24 11:47 mysql-virtual-mailbox-maps.cf -rw-r--r-- 1 root root 19509 2011-04-05 06:07 postfix-files -rwxr-xr-x 1 root root 8729 2011-04-05 06:07 postfix-script* -rwxr-xr-x 1 root root 25752 2011-04-05 06:07 post-install* drwxr-xr-x 2 root root 4096 2012-02-12 12:41 sasl/ root at xa-12345:/etc/postfix# ll sasl/ drwxr-xr-x 2 root root 4096 2012-02-12 12:41 ./ drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ../ -rw-r--r-- 1 root root 26 2012-02-12 12:41 smtpd.conf ########################### /etc/dovecot ########################### drwxr-xr-x 3 root root 4096 2011-09-24 11:42 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ drwxr-xr-x 2 root dovecot 4096 2012-02-12 12:03 conf.d/ -rw-r--r-- 1 root dovecot 3693 2011-09-23 15:11 dovecot.conf -rw-r--r-- 1 root dovecot 410 2011-09-23 15:11 dovecot-db.conf.ext -rw-r--r-- 1 root dovecot 782 2011-09-23 15:11 dovecot-dict-sql.conf.ext -rw-r--r-- 1 root dovecot 5508 2011-09-23 15:38 dovecot-sql.conf -rw-r--r-- 1 root dovecot 5348 2011-09-23 15:11 dovecot-sql.conf.ext -rw-r--r-- 1 root dovecot 116 2011-09-23 15:11 README ########################### /etc/default/saslauthd ########################### -rw-r--r-- 1 root root 2043 2012-02-12 11:57 /etc/default/saslauthd ########################### /var/spool/postfix/var/run/ ########################### drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ./ drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ../ drwxr-xr-x 2 root sasl 4096 2012-02-12 12:35 saslauthd/ I have added postfix to the sasl group (don't know if it was necessary but I found this instruction on a website) /etc/group: sasl:x:45:postfix /etc/default/saslauthd: ---------------------- START=yes PWDIR="/var/spool/postfix/var/run/saslauthd" PARAMS="-m ${PWDIR}" PIDFILE="${PWDIR}/saslauthd.pid" DESC="SASL Authentication Daemon" NAME="saslauthd" MECHANISMS="pam" MECH_OPTIONS="" THREADS=5 OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" mkdir -p /var/spool/postfix/var/run/saslauthd chown -R root.sasl /var/spool/postfix/var/run/saslauthd Then I have checked saslauth is running after restart: root at xa-12345:/home/vmail# ps -ef | grep saslauthd root 6123 1 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6124 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6125 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6127 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6128 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 Then when I check after restarting if saslauthd socket is created I don't see anything (don't know if it's normal) root at xa-12345:/home/vmail# ls -lh /var/spool/postfix/var/run/saslauthd total 0 /etc/postfix/sasl/smtpd.conf: ----------------------------- pwcheck_method: saslauthd /etc/postfix/main.cf: --------------------- myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache myhostname = mail.foobar.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = foobar.com mydestination = xa-12345.dadibox.com, localhost.dadibox.com, localhost relayhost = mynetworks = 127.0.0.0/8 80.190.190.190 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all #virtual domains virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_mailbox_base = /home/vmail virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf # Dovecot LDA virtual_transport = dovecot dovecot_destination_recipient_limit = 1 debug_peer_level = 2 debug_peer_list = 127.0.0.1 smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination smtpd_sasl_application_name = smtpd broken_sasl_auth_clients = yes /etc/dovecot/conf.d/10-auth.conf: -------------------------------------- disable_plaintext_auth = no !include auth-sql.conf.ext !include auth-static.conf.ext /etc/dovecot/conf.d/auth-static.conf.ext: ---------------------------------------- userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } /etc/dovecot/conf.d/auth-sql.conf.ext: -------------------------------------- passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } /etc/dovecot/dovecot-sql.conf: ------------------------------ driver = mysql connect = host=127.0.0.1 dbname=mailserver user=myuser password=mypassword default_pass_scheme = CRAM-MD5 password_query = SELECT email as user, password FROM view_users WHERE email='%u'; /etc/dovecot/conf.d/10-master.conf: ---------------------------------- service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener lmtp { } } service imap { } service pop3 { } service auth { unix_listener auth-userdb { mode = 0600 user = vmail #group = } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { mode = 0777 } } service auth-worker { } service dict { unix_listener dict { mode = 0600 #user = group = vmail } } From forumer at smartmobili.com Sun Feb 12 16:37:26 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 15:37:26 +0100 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <321c6c9323618a11b96bccf49ac4d135@smartmobili.com> IN my previous email ignore the information about the missing socket because it's actually created -rw------- 1 root root 5 2012-02-12 15:28 saslauthd.pid From jom at grosjo.net Sun Feb 12 16:43:26 2012 From: jom at grosjo.net (Joan Moreau) Date: Sun, 12 Feb 2012 14:43:26 +0000 Subject: [Dovecot] v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <258ebed086b260ca88432119bd0bb042@grosjo.net> Thanks for 2.1.0 Timo Looking forward to beta-test 2.2 :) Le 12/02/2012 05:45, Timo Sirainen a ?crit : > On 12.2.2012, at 7.42, Timo Sirainen wrote: > >> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. > > Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota and Sieve's reject mails now include DSN report > instead of MDN report. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. From alexwbaule at gmail.com Sun Feb 12 16:52:05 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 12:52:05 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: I want to know because i have header and body splited. To erase the body, i must have shure, that header was expunged. Some tests that i made, the client call expunge, but i don't know why (there is notting in mail log) the header was not expunged, nut my body was, because i call it before expunge the header. something like this. if (found_body){ erase_body(); super.expunge(_mail); } Because that expunge that no happen, (the client call, but the email was not expunged) i think to do like this way: if (found_body){ super.expunge(_mail); if(_mail was expunged){ erase_body(); } } TKs Timo ! Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > On 10.2.2012, at 19.39, Alex Baule wrote: > >> Hy Everyone...and Timo ! >> >> There is a way to know if a email was expunged (deleted from hard >> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> expunge function is void, i can't know if was really expunged. >> >> There is a way to know this ? > > It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. > > If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). > > Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. > From tss at iki.fi Sun Feb 12 17:42:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:42:20 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: <52A730C6-230B-42B6-8808-A9F6AE3A44D5@iki.fi> On 12.2.2012, at 8.54, Thomas Leuxner wrote: > Indeed I did as it did not like the syntax: > > $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= > backup: invalid option -- 'o' > doveadm backup [-u |-A] [-S ] [-fR] [-m ] Oh. Hm. I'm not exactly sure why it's saying that, but adding some quotes fixes that: dsync -u tlx at leuxner.net backup 'dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota=' From rob0 at gmx.co.uk Sun Feb 12 17:42:39 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 12 Feb 2012 09:42:39 -0600 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <20120212154239.GG14195@harrier.slackbuilds.org> On Sun, Feb 12, 2012 at 02:52:54PM +0100, forumer at smartmobili.com wrote: > I am trying to configure dovecot/postfix with virtual users and > sasl auth but there are so many tutorials with mistakes and subtle > differences that at the end I am lost. All the more reason to stick to each project's official documentation. The sad reality is that most such tutorials you will find are written by people who should not be writing documentation. > So I am running Ubuntu server 11.04 (natty) and when I installed my > server I had followed the following guide > http://workaround.org/ispmail/etch One of the better ones; I know the author, and he is reasonably competent. But are you sure you need and "ISP-style" mail server? Perhaps something simpler makes sense to begin with. http://www.postfix.org/BASIC_CONFIGURATION_README.html > Then I have migrated to dovecot 2.1 and now I am trying to allow > authenticated user to use my smtp server because for the moment I > get : Relay access denied A Postfix issue, or possibly somewhat on topic here if you are using Dovecot SASL. You do not seem to know whether you are or not. This might be because you are mixing and confusing different tutorials without understanding what they are doing and why. > Here is my configuration (first is file hierarchy followed by > file contents) and I would like to know what I need to do to > allow authenticated user send email. http://www.postfix.org/SASL_README.html#server_dovecot http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > In this tutorial > http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql I have not reviewed this one, but the anchor title is patently absurd. If you are using Postfix and Dovecot, there is NEVER any reason to use Cyrus SASL's saslauthd. Consider that one useless. The rest of this is not worth answering. You went off on a Cyrus tangent which is pointless (and never on topic for this list!) Start over with a better understanding of what you need. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Sun Feb 12 17:46:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:46:12 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Hi, Yeah, you shouldn't do the erasing directly in expunge(), because it may still be aborted. Do it in sync_notify() like quota plugin does. On 12.2.2012, at 16.52, Alex Baule wrote: > I want to know because i have header and body splited. To erase the > body, i must have shure, that header was expunged. Some tests that i > made, the client call expunge, but i don't know why (there is notting > in mail log) the header was not expunged, nut my body was, because i > call it before expunge the header. something like this. > > if (found_body){ > erase_body(); > super.expunge(_mail); > } > > Because that expunge that no happen, (the client call, but the email > was not expunged) i think to do like this way: > > if (found_body){ > super.expunge(_mail); > if(_mail was expunged){ > erase_body(); > } > } > > TKs Timo ! > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >>> Hy Everyone...and Timo ! >>> >>> There is a way to know if a email was expunged (deleted from hard >>> disk) inside a plugin ? I rewrite the expunge function, but the real >>> expunge function is void, i can't know if was really expunged. >>> >>> There is a way to know this ? >> >> It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. >> >> If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). >> >> Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. >> > From tss at iki.fi Sun Feb 12 17:58:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:58:20 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <4F37C217.7040707@Media-Brokers.com> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: On 12.2.2012, at 15.43, Charles Marcus wrote: > One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I'm planning on creating a new "backup" mail storage backend, where you could basically run: doveadm backup -u user at domain backup: And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. Then for restoring you can use the same input for "doveadm import". So in the backups there would be no deduplication, unless implemented by the backup software. > I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? As long as the URL is random enough it shouldn't be possible to guess the links. But if they're HTML mails and such then hyperlinks would send the referrer-url to the next site. I guess you could also add HTTP authentication. From tss at iki.fi Sun Feb 12 18:02:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 18:02:39 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: On 12.2.2012, at 10.23, Mark Zealey wrote: > Ah I should have checked this before - line before: > > 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied > > This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Yeah, the auth-userdb socket default permissions aren't usable pretty much anywhere. In v2.1 it's better, but still not as good as they could be. I'm not sure what the best solution for this is. > Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. You mean if you restart one director some of the state is lost in all of them? I think v2.0 had some bugs related to this, but they should be fixed in v2.1. If you mean that if you stop all of the directors and state is lost when restarting then yes, that does happen. You can run "doveadm director dump" to get the current state and after a restart run its output to get back to the same state. What state specifically are you talking about anyway? From CMarcus at Media-Brokers.com Sun Feb 12 18:15:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 11:15:42 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <4F37E5AE.8060104@Media-Brokers.com> On 2012-02-12 10:58 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.43, Charles Marcus wrote: >> > I'm planning on creating a new "backup" mail storage backend, where > you could basically run: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). > So it would be similar to e.g. PostgreSQL's pg_dump. Then for > restoring you can use the same input for "doveadm import". So in the > backups there would be no deduplication, unless implemented by the > backup software. Hmmm, ok thanks... loss of dedupe on the backup server may or may not be an issue I'll be concerned about, but good to know that there is an option at least. >> It looks like using mimedefang would eliminate the problem with >> backups, and would provide the added benefit of eliminating large >> file attachments for external recipients as well, however I'm >> concerned about security - ie, how do you limit access to these >> attachments to only the recipients of the message with the >> hyperlink in it? > As long as the URL is random enough it shouldn't be possible to guess > the links. But if they're HTML mails and such then hyperlinks would > send the referrer-url to the next site. I guess you could also add > HTTP authentication. Thanks for that too... I'll have to a bit of research on this, because the more I think about it, the more I like the idea, *if* it can be done securely enough... -- Best regards, Charles From alexwbaule at gmail.com Sun Feb 12 18:48:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 14:48:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Tks timo ! I will see the plugin to do like it! Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > Hi, > > Yeah, you shouldn't do the erasing directly in expunge(), because it may > still be aborted. Do it in sync_notify() like quota plugin does. > > On 12.2.2012, at 16.52, Alex Baule wrote: > > > I want to know because i have header and body splited. To erase the > > body, i must have shure, that header was expunged. Some tests that i > > made, the client call expunge, but i don't know why (there is notting > > in mail log) the header was not expunged, nut my body was, because i > > call it before expunge the header. something like this. > > > > if (found_body){ > > erase_body(); > > super.expunge(_mail); > > } > > > > Because that expunge that no happen, (the client call, but the email > > was not expunged) i think to do like this way: > > > > if (found_body){ > > super.expunge(_mail); > > if(_mail was expunged){ > > erase_body(); > > } > > } > > > > TKs Timo ! > > > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > >> On 10.2.2012, at 19.39, Alex Baule wrote: > >> > >>> Hy Everyone...and Timo ! > >>> > >>> There is a way to know if a email was expunged (deleted from hard > >>> disk) inside a plugin ? I rewrite the expunge function, but the real > >>> expunge function is void, i can't know if was really expunged. > >>> > >>> There is a way to know this ? > >> > >> It's not known until transaction_commit() what messages are expunged. > What do you need to know this for? There are a few different ways. > >> > >> If you simply need one of the sessions to definitely know that it > expunged a message, you can do it like quota plugin does. (So even if there > are multiple clients doing EXPUNGE at the same time, quota is never > decreased more than once per mail.) See quota-storage.c and most > importantly quota_mailbox_sync_notify(). > >> > >> Quota code also shows the other method of doing it: keep track of what > mail_expunge()s have been called, and then in transaction commit check if > it succeeds and if it does do what you want to do. > >> > > > > From mark at msapiro.net Sun Feb 12 18:57:36 2012 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 12 Feb 2012 08:57:36 -0800 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: <4F3748A1.3030301@iki.fi> Message-ID: Timo Sirainen wrote: >On 12/22/2010 08:08 PM, Mark Sapiro wrote: >> It appears that there is some version of the Dovecot LDA that is >> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >> non-delivery to a user who is over quota. > >Fixed, better late than never :) >http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 >http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 Thanks. >The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >server sending this. :( -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tss at iki.fi Sun Feb 12 19:00:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:00:45 +0200 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212132719.GB27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> Message-ID: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> On 12.2.2012, at 15.27, Mike Constabel wrote: > Another Question to this Option: > > # tb-extra-mailbox-sep: > # With mbox storage a mailbox can contain either mails or submailboxes, > # but not both. Thunderbird separates these two by forcing server to > # accept '/' suffix in mailbox names in subscriptions list. > > > I thought this is not needed because I use mdbox. But mbox here > means also mdbox and sdbox? Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > Should I use all 3 workarounds if I don't know which clients are > in use? I don't know if delay-newmail is still relevant. From tss at iki.fi Sun Feb 12 19:03:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:03:10 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: References: Message-ID: <83BE4415-FA94-4F23-9DB4-7F59F71D245B@iki.fi> On 12.2.2012, at 18.57, Mark Sapiro wrote: >> On 12/22/2010 08:08 PM, Mark Sapiro wrote: >>> It appears that there is some version of the Dovecot LDA that is >>> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >>> non-delivery to a user who is over quota. >> >> Fixed, better late than never :) >> The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >> server sending this. > > :( It was actually a bit more complex than I thought. Sieve "reject" is supposed to send MDN (RFC 5228), it's only the out-of-quota error that was supposed to send DSN. So now Dovecot can send both: http://hg.dovecot.org/dovecot-2.1/rev/ad5298ba3229 From tss at iki.fi Sun Feb 12 19:17:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:17:30 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <4F37AD0B.9040406@mike2k.de> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> Message-ID: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f From ms at mike2k.de Sun Feb 12 20:34:31 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 19:34:31 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> Message-ID: <4F380637.30809@mike2k.de> On 12-Feb-12 18:17, Timo Sirainen wrote: > This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f Thanks Timo, it does :) -Michael From tss at iki.fi Sun Feb 12 21:11:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 21:11:44 +0200 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: On 12.2.2012, at 9.02, Thomas Leuxner wrote: > Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 From tss at iki.fi Sun Feb 12 23:01:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:01:10 +0200 Subject: [Dovecot] v2.1.rc6 released Message-ID: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota mails now include DSN report instead of MDN. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From javierdemiguel at us.es Sun Feb 12 23:38:58 2012 From: javierdemiguel at us.es (=?UTF-8?Q?Javier_Miguel_Rodr=C3=ADguez?=) Date: Sun, 12 Feb 2012 22:38:58 +0100 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Hello Reading 2.1rc6 changelog I see this: mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. Could you elaborate more on this? Has been ported to Dovecot 2.0? Regards Javier On Sun, 12 Feb 2012 23:01:10 +0200, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [1] > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [2] > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > > I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) > > Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota mails now include DSN report instead of MDN. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. > > _______________________________________________ > Dovecot-news mailing list > Dovecot-news at dovecot.org [3] > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news [4] Links: ------ [1] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [2] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [3] mailto:Dovecot-news at dovecot.org [4] http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news From tss at iki.fi Sun Feb 12 23:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:42:17 +0200 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Message-ID: <78639407-65C6-4D89-9F8C-2E27041422F2@iki.fi> On 12.2.2012, at 23.38, Javier Miguel Rodr?guez wrote: > Reading 2.1rc6 changelog I see this: > > mdbox: When saving > to alt storage, Dovecot didn't append as much > data to m.* files as it > could have. > > Could you elaborate more on this? It means that if you had e.g. mdbox_rotate_size = 2M, the files in alt storage probably were often much smaller than 2 MB. > Has been ported to Dovecot 2.0? No. You could try how well this applies to it: http://hg.dovecot.org/dovecot-2.1/rev/6881d8c59d9f From tss at iki.fi Sun Feb 12 23:57:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:57:29 +0200 Subject: [Dovecot] v2.0.18 released Message-ID: <881CA667-9F0C-4C32-9A61-87409F7DE773@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz.sig + DIGEST-MD5 authentication supports authorization id now. Patch by Yubao Liu + Added instance_name setting which is used to prefix Dovecot processes in ps output. - LDA/LMTP: Sending a large mail via submission_host or via LMTP proxy may have caused a hang. - Fixed dbox + mail_attachment_dir + zlib problems. - Login processes weren't logging all intended messages with auth_verbose=yes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - IMAP: CONTEXT search return option wasn't handled at all. - dbox: Various error handling fixes. - snarf plugin: Keep the mailbox locked during snarfing to avoid duplicates. From mstevens at imt-systems.com Mon Feb 13 00:32:36 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 23:32:36 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole Message-ID: Hi, It is probably a problem with pigeonhole 0.2.5. ... make[3]: *** [managesieve-proxy.o] Error 1 make[3]: *** Waiting for unfinished jobs.... Here is my build log: http://oss.imt-systems.com/mstevens/temp/build.log Someone an idea? Best regards, Morten From tss at iki.fi Mon Feb 13 00:43:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 00:43:26 +0200 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: Message-ID: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> On 13.2.2012, at 0.32, Morten Stevens wrote: > It is probably a problem with pigeonhole 0.2.5. Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. From interfasys at gmail.com Mon Feb 13 02:02:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Mon, 13 Feb 2012 00:02:29 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients Message-ID: <4F385315.5010600@gmail.com> Hello Pascal, I've just tested rc6 and get almost the same results. imap still crashes, but Horde still works. I forgot to mention that this is on FreeBSD 8 using gcc 4.6 and LTO. And contrary to what I said in my initial email, it doesn't seem to be ACL plugin related. imap crashes even if it's not in the plugins list. Here is the backtrace: # gdb /usr/libexec/dovecot/imap /path/to/core Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104b4bf6 in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x0000000000416b99 in cmd_list_continue.6853 () No symbol table info available. #3 0x000000000042166d in cmd_list_full () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x8080808080808080 in ?? () No symbol table info available. #6 0x0000000000000004 in ?? () No symbol table info available. #7 0x0000000010a3f1a8 in ?? () No symbol table info available. #8 0x0000000010aa4390 in ?? () No symbol table info available. #9 0x0000000010a3f174 in ?? () No symbol table info available. #10 0x00007fffffffe9a0 in ?? () No symbol table info available. #11 0x0000000010596e4f in p_strdup () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x0000000010a22200 in ?? () No symbol table info available. #13 0x0000000010aa4458 in ?? () No symbol table info available. #14 0x0000000010aa4340 in ?? () No symbol table info available. #15 0x0000000010aa4498 in ?? () No symbol table info available. #16 0x0000000110a3f040 in ?? () No symbol table info available. #17 0x0000000010a3f128 in ?? () No symbol table info available. #18 0x0000000010a3e840 in ?? () No symbol table info available. #19 0x0000000010a3f040 in ?? () No symbol table info available. #20 0x0000000010a22200 in ?? () No symbol table info available. #21 0x0000000000000001 in ?? () No symbol table info available. #22 0x0000000010a22200 in ?? () No symbol table info available. #23 0x0000000000000000 in ?? () No symbol table info available. #24 0x000000000041d320 in client_command_input.12433 () No symbol table info available. #25 0x00000000004203ca in client_handle_input () No symbol table info available. #26 0x0000000010a574e0 in ?? () No symbol table info available. #27 0x00000000105ae219 in i_stream_read () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #28 0x000000000041d13f in client_input () No symbol table info available. #29 0x00000000105ce276 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #30 0x00000000105c9ea3 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #31 0x00000000105c9f78 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #32 0x00000000105c9fb3 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #33 0x00000000004158bb in main () No symbol table info available. Regards, Olivier > On 01/18/2012 02:03 AM interfaSys s?rl wrote: > >/ Hello, > />/ > />/ I've just noticed that when Horde is connecting to Dovecot 2.1, it > />/ crashes the imap service if Dovecot is configured to use the ACL plugin. > />/ I'm not sure what's so special about the command Horde sends, but it > />/ shouldn't make Dovecot crash. Everything is fine when using Thunderbird. > />/ > />/ Here is the message in Dovecot's logs > />/ "Fatal: master: service(imap): child 89974 killed with signal 11 (core > />/ not dumped)" > />/ > />/ The message says that the core is not dumped, even though I did add > />/ drop_priv_before_exec=yes to my config file. > / > dovecot stop > ulimit -c unlimited > dovecot > > Now connect with Horde and let it crash. > > >/ I've tried connecting to the pid using gdb, but the process just hangs > />/ as soon as I'm connected. > />/ > / > continue > [wait for the crash] > bt full > detach > quit > > > Regards, > Pascal From tss at iki.fi Mon Feb 13 02:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 02:49:07 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F385315.5010600@gmail.com> References: <4F385315.5010600@gmail.com> Message-ID: On 13.2.2012, at 2.02, interfaSys s?rl wrote: > #0 0x00000000105c64e7 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104b4bf6 in mailbox_list_iter_next () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. > #2 0x0000000000416b99 in cmd_list_continue.6853 () > No symbol table info available. > #3 0x000000000042166d in cmd_list_full () This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. From bradley.giesbrecht at gmail.com Mon Feb 13 02:51:55 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Sun, 12 Feb 2012 16:51:55 -0800 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > On 13.2.2012, at 0.32, Morten Stevens wrote: > >> It is probably a problem with pigeonhole 0.2.5. > > > Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. Confirmed. Thanks. From janfrode at tanso.net Mon Feb 13 09:49:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 08:49:54 +0100 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <20120213074954.GA13267@dibs.tanso.net> On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. So only full backups, no incremental backups? Then what's the benefit over just copying the files (of a snapshot)? -jf From tlx at leuxner.net Mon Feb 13 09:57:46 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 13 Feb 2012 08:57:46 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: <20120213075746.GA28586@nihlus.leuxner.net> On Sun, Feb 12, 2012 at 09:11:44PM +0200, Timo Sirainen wrote: > Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. > > http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 Thanks. From fabian at eideo.de Mon Feb 13 10:00:25 2012 From: fabian at eideo.de (Kuran, Fabian) Date: Mon, 13 Feb 2012 08:00:25 +0000 Subject: [Dovecot] Problems sending email direct into publich folders Message-ID: <20120213080027.1CB5B214001@mail.eideo.de> Hi, >?Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent?> to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a?> sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder.? Is there a comparable way to do this with Dovecot? FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. -- Bye, Fabian Kuran From mstevens at imt-systems.com Mon Feb 13 12:11:30 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 13 Feb 2012 11:11:30 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: <493530c2afa3ced4228a0014ac495433@imt-systems.com> On 13.02.2012 01:51, Bradley Giesbrecht wrote: > On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > >> On 13.2.2012, at 0.32, Morten Stevens wrote: >> >>> It is probably a problem with pigeonhole 0.2.5. >> >> >> Yeah, unfortunately I had to break it to fix a bug.. Although I >> guess if I really wanted to I could have worked around it also. >> Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 >> fixes it. > > Confirmed. Thanks. Okay, also confirmed. Thank you. Best regards, Morten From stephan at rename-it.nl Mon Feb 13 12:17:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 11:17:23 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <493530c2afa3ced4228a0014ac495433@imt-systems.com> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> <493530c2afa3ced4228a0014ac495433@imt-systems.com> Message-ID: <4F38E333.7030705@rename-it.nl> Op 2/13/2012 11:11 AM, Morten Stevens schreef: > On 13.02.2012 01:51, Bradley Giesbrecht wrote: >> On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: >> >>> On 13.2.2012, at 0.32, Morten Stevens wrote: >>> >>>> It is probably a problem with pigeonhole 0.2.5. >>> >>> >>> Yeah, unfortunately I had to break it to fix a bug.. Although I >>> guess if I really wanted to I could have worked around it also. >>> Anyway, >>> http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes >>> it. >> >> Confirmed. Thanks. > > Okay, also confirmed. Release will follow later today. Regards, Stephan. From tss at iki.fi Mon Feb 13 12:36:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:36:47 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <20120213074954.GA13267@dibs.tanso.net> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> <20120213074954.GA13267@dibs.tanso.net> Message-ID: On 13.2.2012, at 9.49, Jan-Frode Myklebust wrote: > On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: >> >> doveadm backup -u user at domain backup: >> >> And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. > > So only full backups, no incremental backups? I know people are doing incremental backups with PostgreSQL, so it should be possible with Dovecot too. I'm not sure about the details yet tough. > Then what's the benefit over just copying the files (of a snapshot)? If SIS is enabled, it's a little difficult to restore specific user's mails from a filesystem based backup. From tss at iki.fi Mon Feb 13 12:39:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:39:03 +0200 Subject: [Dovecot] Problems sending email direct into publich folders In-Reply-To: <20120213080027.1CB5B214001@mail.eideo.de> References: <20120213080027.1CB5B214001@mail.eideo.de> Message-ID: <2F6B2448-017D-4056-9AA6-65BB03B64593@iki.fi> On 13.2.2012, at 10.00, Kuran, Fabian wrote: >> Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent > to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a > sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). > I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder. > > Is there a comparable way to do this with Dovecot? > FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. You can set: lmtp_save_to_detail_mailbox = yes And I think then you should be able to send the mail to any user who has write access to it. From tss at iki.fi Mon Feb 13 13:47:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 13:47:06 +0200 Subject: [Dovecot] Dovecot v2.2 plans Message-ID: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: - 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. - 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) - 3. priority: Things other people want to get implemented. There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. dsync-based replication ----------------------- I'll write a separate post about this later. Besides, it's coming for Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. Shared mailbox improvements --------------------------- Support for private flags for all mailbox formats: namespace { type = public prefix = Public/ mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public } - dsync needs to be able to replicate the private flags as well as shared flags. - might as well add a common way for all mailbox formats to specify which flags are shared and which aren't. $controldir/dovecot-flags would say which is the default (private or shared) and what flags/keywords are the opposite. - easy way to configure shared mailboxes to be accessed via imapc backend, which would allow easy shared mailbox accesses across servers or simply between two system users in same server. (this may be tricky to dsync.) - global ACLs read from a single file supporting wildcards, instead of multiple different files - default ACLs for each namespace/storage root (maybe implemented using the above..) Metadata / annotations ---------------------- Add support for server, mailbox and mail annotations. These need to be dsyncable, so their changes need to be stored in various .log files: 1. Per-server metadata. This is similar to subscriptions: Add changes to dovecot.mailbox.log file, with each entry name a hash of the metadata key that was changed. 2. Per-mailbox metadata. Changes to this belong inside mailbox_transaction_context, which write the changes to mailbox's dovecot.index.log files. Each log record contains a list of changed annotation keys. This gives each change a modseq, and also allows easily finding out what changes other clients have done, so if a client has done ENABLE METADATA Dovecot can easily push metadata changes to client by only reading the dovecot.index.log file. 3. Per-mail metadata. This is pretty much equivalent to per-mailbox metadata, except changes are associated to specific message UIDs. The permanent storage is in dict. The dict keys have components: - priv/ vs. shared/ for specifying private vs. shared metadata - server/ vs mailbox// vs. mail// - the metadata key name This would be a good time to improve the dict configuration to allow things like: - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a file, while the rest goes to sql) - allow sql dict to be used in more relational way, so that mail annotations could be stored with tables: mailbox (id, guid) and mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid everywhere. Things to think through: - How to handle quota? Probably needs to be different from regular mail quota. Probably some per-user "metadata quota bytes" counter/limit. - Dict lookups should be done asynchronously and prefetched as much as possible. For per-mail annotation lookups mail_alloc() needs to include a list of annotations that are wanted. Configuration ------------- Copy all mail settings to namespaces, so it'll be possible to use per-namespace mailbox settings. Especially important for imapc_* settings, but can be useful for others as well. Those settings that aren't explicitly defined in the namespace will use the global defaults. (Should doveconf -a show all of these values, or simply the explicitly set values?) Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration. There are mainly 3 config files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. The dict-sql is something I think needs a bigger redesign (mentioned above in "Metadata" section), but the sql/ldap auth configs could be merged. One way could be: sql_db sqlmails { # most settings from dovecot-sql.conf.ext, except for queries driver = mysql connect = ... } ldap_db ldapmails { # most settings from dovecot-ldap.conf.ext, except attributes/filters } passdb { driver = sql db = sqlmails sql_query = select password from users where username = '%u' } passdb { driver = ldap db = ldapmails ldap_attributes { password = %{ldap:userPassword} } ldap_filter = ... } The sql_db {} and ldap_db {} would be generic enough to be used everywhere (e.g. dict-sql), not just for passdb/userdb. Some problems: - Similar to the per-namespace mail settings, doveconf -a would output all sql_query, ldap_attributes, ldap_filter, etc. settings for all passdbs/userdbs. Perhaps a similar solution? - The database configs contain passwords, so they should be readable only by root. This makes running dovecot-lda and maybe doveadm difficult, since they fail at "permission denied" when trying to open the config. There are probably only two solutions: a) The db configs need to be !include_try'd or b) the configs can be world-readable, but only passwords are placed to only-root-readable files by using "password = backend protocol could be improved to support moving connection to another backend. Possibly using a separate control connection to avoid making the proxying less efficient in normal operation. 3. When restarting Dovecot, move all the connections to a process that keeps the connections open for a while. When Dovecot starts up, create imap processes back to the connections. This allows changing configuration for existing client connections (which sometimes may be bad! need to add checks against client-visible config conflicts), upgrading Dovecot, etc. without being visible to clients. The only problem is SSL connections: OpenSSL doens't provide a way to save/restore state, so either you need to set shutdown_clients=no (and possibly keep some imap-login processes doing SSL proxying for a long time), or SSL connections need to be killed. Of course the SSL handling could be outsourced to some other software/hardware outside Dovecot. The IMAP state saving isn't always easy. Initially it could be implemented only for the simple cases (which are a majority) and later extended to cover more. IMAP extensions --------------- - CATENATE is already implemented by Stephan - URLAUTH is also planned to be implemented, somewhat differently than in Apple's patch. The idea is to create a separate imap-urlauth service that provides extra security. - NOTIFY extension could be implemented efficiently using mailbox list indexes, which already exists in v2.1. - FILTERS extension can be easily implemented once METADATA is implemented - There are also other missing extensions, but they're probably less important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. Backups ------- Filesystem based backups have worked well enough with Dovecot in the past. But with new features like single instance storage it's becoming more difficult. There's no 100% consistent way to even get filesystem level backups with SIS enabled, because deleting both the message file and its attachment files can't be done atomically (although usually this isn't a real problem). Restoring SIS mails is more difficult though, first you need to restore the dbox mail files and then you need to figure out what attachment files from SIS need to be restored, and finally you'll need to do doveadm import to put them into their final destination. I don't have much experience with backup software, but other people in my company do. The initial idea is to implement a Dovecot backup agent to one (commercial) backup software, which allows doing online backups and restoring mails one user/mailbox/mail at a time. I don't know the details yet how exactly this is going to be implemented, but the basic plan is probably to implement a "backup" mail storage backend, which is a PostgreSQL pg_dump-like flat file containing mails from all mailboxes. doveadm backup/import can then export/import this format via stdout/stdin. Incremental backups could possibly be done by giving a timestamp of previous backup run (I'm not sure about this yet). Once I've managed to implement the first fully functional backup agent, it should become clearer how to implement it to other backup solutions. Random things ------------- - dovecot.index.cache file writing is too complex, should be simplified - Enable auth_debug[_passwords]=yes on-the-fly for some specific users/IPs via doveadm - Optimize virtual mailboxes using mailbox list indexes. It wouldn't anymore need to keep all the backend mailboxes' index files open. - Would be nice to go forward with supporting key-value databases as mail storage backends. From janfrode at tanso.net Mon Feb 13 15:54:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 14:54:50 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend Message-ID: <20120213135450.GA31004@dibs.tanso.net> We've been collecting some stats to see what kind of benefits UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is running in High-performance mode http://wiki2.dovecot.org/LoginProcess with authentication caching http://wiki2.dovecot.org/Authentication/Caching During the weekend two servers (webmail3 and webmail4) has been running with local imapproxy and two servers without (webmail1 and webmail2). Each server has served about 1 million http requests, over 3 days. server avg. response time # requests ---------------------------------------------------- webmail1.example.net 0.370411 1092386 webmail2.example.net 0.374227 1045141 webmail3.example.net 0.378097 1043919 imapproxy webmail4.example.net 0.378593 1028653 imapproxy ONLY requests that took more than 5 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 26.048 1125 webmail2.example.net 26.2997 1080 webmail3.example.net 28.5596 808 imapproxy webmail4.example.net 27.1004 964 imapproxy ONLY requests that took more than 10 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 49.1407 516 webmail2.example.net 53.0139 459 webmail3.example.net 59.7906 333 imapproxy webmail4.example.net 58.167 384 imapproxy The responstimes are not very fast, but they do seem to support the claim that an imapproxy isn't needed for dovecot. -jf From tss at iki.fi Mon Feb 13 16:14:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:14:22 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On 13.2.2012, at 15.54, Jan-Frode Myklebust wrote: > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Other things that would be interesting to try out (both from latency and disk IO usage point of view): - maildir_very_dirty_syncs - mail_prefetch_count (Linux+maildir only, v2.1+) From busseniu at in.tum.de Mon Feb 13 16:16:29 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 15:16:29 +0100 Subject: [Dovecot] dsync, hard-links and refcounts Message-ID: <4F391B3D.8020208@in.tum.de> Hi, when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. - Is there some way to make dsync notice the hard links? (I used dovecot 2.0.17) - Alternatively, is there some tool to deduplicate the mdbox after it has been converted from maildir? - NB: I am not talking about single-instance storage for attachments. Thank you very much, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Feb 13 16:40:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:40:05 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F391B3D.8020208@in.tum.de> References: <4F391B3D.8020208@in.tum.de> Message-ID: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. > > My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. > - Is there some way to make dsync notice the hard links? (I used > dovecot 2.0.17) It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. From forumer at smartmobili.com Mon Feb 13 17:35:57 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Mon, 13 Feb 2012 16:35:57 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: Hi, Is there any plan to port dovecot to windows ? From robert at schetterer.org Mon Feb 13 17:49:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:49:14 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F3930FA.2080802@schetterer.org> Am 13.02.2012 16:35, schrieb forumer at smartmobili.com: > Hi, > > Is there any plan to port dovecot to windows ? > cant wait Timos answer *g -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Feb 13 17:51:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:51:26 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F39317E.5090300@schetterer.org> Am 13.02.2012 12:47, schrieb Timo Sirainen: > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration might a question of taste, but i never liked the splitted up config style, so i like this idea -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Feb 13 17:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 17:57:03 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <4F39317E.5090300@schetterer.org> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <4F39317E.5090300@schetterer.org> Message-ID: On 13.2.2012, at 17.51, Robert Schetterer wrote: > Am 13.02.2012 12:47, schrieb Timo Sirainen: >> Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration > > might a question of taste, but i never liked the splitted up config > style, so i like this idea Note that I said *.conf.ext, not *.conf.. From tss at iki.fi Mon Feb 13 18:11:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:11:33 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On 13.2.2012, at 17.35, forumer at smartmobili.com wrote: > Is there any plan to port dovecot to windows ? It probably works via Cygwin (although I think Cygwin has to do some kind of POSIX filesystem emulation, which slows things down and might cause some trouble if server crashes). Actually we have discussed this a little within my company.. I'm personally not interested in spending much time on it, but that's why we're hiring more coders so I won't have to do everything. :) If there is enough commercial interest, we might build something better than using Cygwin. Anyway, one thing is certain: No Dovecot for Windows questions in this mailing list. Perhaps I'll create another dovecot-windows@ mailing list. Perhaps I'll even give it a different name (dovedows? wincot? glassdove?) From joshua at hybrid.pl Mon Feb 13 18:26:38 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 17:26:38 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 Message-ID: Hi, Finally I decided to give a try to dovecot 2.0, after spending few years with 1.2. Unfortunately, it doesn't go smooth - finally I managed to cope with LMTP transport and exim, but failed to run managesieve. I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. Here is my doveconf -n. Any ideas? # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.39.4-vs2.3.1-pre9.2-nfs41-nfsd x86_64 auth_debug = yes auth_debug_passwords = yes debug_log_path = /var/log/dovecot/debug.log dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_uid = 79 info_log_path = /var/log/dovecot/info.log last_valid_uid = 79 log_path = /var/log/dovecot/main.log mail_debug = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = /var/mail/sieve/%d/%u.sieve sieve_dir = /var/mail/sieve/%d/%u sieve_global_path = /var/mail/sieve/%d/global.sieve } postmaster_address = postmaster at hybrid.pl service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 } inet_listener imaps { address = 0.0.0.0 port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { address = 127.0.0.1 port = 110 } inet_listener pop3s { address = 0.0.0.0 port = 995 ssl = yes } } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " sieve quota" } protocol lda { mail_plugins = " sieve quota" } Best regards, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Mon Feb 13 18:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:42:21 +0200 Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: References: Message-ID: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> On 13.2.2012, at 18.26, Jacek Osiecki wrote: > I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. > > Here is my doveconf -n. Any ideas? > > # 2.0.16: /etc/dovecot/dovecot.conf Try adding: protocols = imap pop3 sieve From joshua at hybrid.pl Mon Feb 13 19:11:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 18:11:11 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> References: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> Message-ID: On Mon, 13 Feb 2012, Timo Sirainen wrote: > On 13.2.2012, at 18.26, Jacek Osiecki wrote: >> I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. >> Here is my doveconf -n. Any ideas? >> >> # 2.0.16: /etc/dovecot/dovecot.conf > Try adding: > protocols = imap pop3 sieve Great, it helped :) Thanks! Now time to see what surprises will show up while migrating there people from dovecot 1.0-stable :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From alexwbaule at gmail.com Mon Feb 13 19:28:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Mon, 13 Feb 2012 15:28:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Hi Again Timo ! I Trying to port quota plugin notify_sync to my plugin, but i found some issues. I have the hooks already, to replace the mailbx_allocated, like quota plugin does. static struct mail_storage_hooks emexis_mail_storage_hooks = { .mail_user_created = Mplugin_mail_user_created, .mailbox_allocated = Mplugin_mailbox_allocated, .mail_allocated = Mplugin_mail_allocated, .mailbox_list_created = Mplugin_mailbox_list_created, .mail_namespaces_created = Mplugin_antispam_mail_namespaces_created }; inside the Mplugin_mailbox_allocated, i have: static void emexis_mailbox_allocated(struct mailbox *box) { union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = box->v; box->v.open = Mplugin_mailbox_open; box->v.sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); } The v.open i rewrite because my plugin needs. I try to call the sync_notify like this: box->v.sync_notify = Mplugin_mailbox_sync_notify; and follow quota plugin, i try to to this too: struct mailbox_vfuncs *v = box->vlast; v->sync_notify = Mplugin_mailbox_sync_notify; But in this two cases, the Mplugin_mailbox_sync_notify is never called... i missing something ?? Tks ! Em 12 de fevereiro de 2012 14:48, Alex Baule escreveu: > Tks timo ! > > I will see the plugin to do like it! > > Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > >> Hi, >> >> Yeah, you shouldn't do the erasing directly in expunge(), because it may >> still be aborted. Do it in sync_notify() like quota plugin does. >> >> On 12.2.2012, at 16.52, Alex Baule wrote: >> >> > I want to know because i have header and body splited. To erase the >> > body, i must have shure, that header was expunged. Some tests that i >> > made, the client call expunge, but i don't ?know why (there is notting >> > in mail log) the header was not expunged, nut my body was, because i >> > call it before expunge the header. something like this. >> > >> > if (found_body){ >> > ? ?erase_body(); >> > ? ?super.expunge(_mail); >> > } >> > >> > Because that expunge that no happen, (the client call, but the email >> > was not expunged) i think to do like this way: >> > >> > if (found_body){ >> > ? super.expunge(_mail); >> > ? if(_mail was expunged){ >> > ? ? ?erase_body(); >> > ?} >> > } >> > >> > TKs Timo ! >> > >> > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >> >> >>> Hy Everyone...and Timo ! >> >>> >> >>> There is a way to know if a email was expunged (deleted from hard >> >>> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> >>> expunge function is void, i can't know if was really expunged. >> >>> >> >>> There is a way to know this ? >> >> >> >> It's not known until transaction_commit() what messages are expunged. >> >> What do you need to know this for? There are a few different ways. >> >> >> >> If you simply need one of the sessions to definitely know that it >> >> expunged a message, you can do it like quota plugin does. (So even if there >> >> are multiple clients doing EXPUNGE at the same time, quota is never >> >> decreased more than once per mail.) See quota-storage.c and most importantly >> >> quota_mailbox_sync_notify(). >> >> >> >> Quota code also shows the other method of doing it: keep track of what >> >> mail_expunge()s have been called, and then in transaction commit check if it >> >> succeeds and if it does do what you want to do. >> >> >> > >> > From busseniu at in.tum.de Mon Feb 13 20:31:51 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 19:31:51 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F395717.5010504@in.tum.de> On 02/13/2012 03:40 PM, Timo Sirainen wrote: > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. I?d very much appreciate that. We will have to migrate many large mailboxes, and it would be a pity to needlessly waste space. I got the same result with some different configurations, using "mirror", "backup", or "-R backup" (though I have not tried 2.1 yet); so it should be easy to reproduce. However let me know if you need my configuration or anything. Cheers, Christoph From moseleymark at gmail.com Mon Feb 13 21:08:48 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 13 Feb 2012 11:08:48 -0800 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On Mon, Feb 13, 2012 at 5:54 AM, Jan-Frode Myklebust wrote: > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 0.370411 ? ? ? ?1092386 > webmail2.example.net ? 0.374227 ? ? ? ?1045141 > webmail3.example.net ? 0.378097 ? ? ? ?1043919 ?imapproxy > webmail4.example.net ? 0.378593 ? ? ? ?1028653 ?imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 26.048 ? ? ? ? ?1125 > webmail2.example.net ? 26.2997 ? ? ? ? 1080 > webmail3.example.net ? 28.5596 ? ? ? ? 808 ? ? ?imapproxy > webmail4.example.net ? 27.1004 ? ? ? ? 964 ? ? ?imapproxy > > ONLY requests that took more than 10 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 49.1407 ? ? ? ? 516 > webmail2.example.net ? 53.0139 ? ? ? ? 459 > webmail3.example.net ? 59.7906 ? ? ? ? 333 ? ? ?imapproxy > webmail4.example.net ? 58.167 ? ? ? ? ?384 ? ? ?imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Out of curiosity, are you running dovecot locally on those webmail servers as well, or is it talking to remote dovecot servers? I ask because I'm looking at moving our webmail from an on-box setup to a remote pool to support director and was going to look into whether running imapproxyd would help there. We don't bother with it in the local setup, since dovecot is so fast, but remote (but still on a LAN) might be different. Though imapproxyd seems to make (wait for it...) squirrelmail unhappy (complains about IMAP errors, when sniffing shows none), though I've not bothered to debug it yet. From janfrode at tanso.net Mon Feb 13 21:36:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:36:30 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213193630.GA3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 04:14:22PM +0200, Timo Sirainen wrote: > > The responstimes are not very fast, but they do seem to support > > the claim that an imapproxy isn't needed for dovecot. > > That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Yes, this is maildirs (on GPFS). > > Other things that would be interesting to try out (both from latency and disk IO usage point of view): > > - maildir_very_dirty_syncs We already have $ doveconf maildir_very_dirty_syncs maildir_very_dirty_syncs = yes but I don't think this gave the advantage I was expecting.. Was expecting this to move most iops to the index-luns, but the maildir luns seems just as busy. > - mail_prefetch_count (Linux+maildir only, v2.1+) Will look into if this works with GPFS when we upgrade to v2.1. It has it's own page cache, so I have no idea if it will respect POSIX_FADV_WILLNEED or if one will need to use it's own API's for hinting: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html -jf From janfrode at tanso.net Mon Feb 13 21:43:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:43:57 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213194356.GB3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: > > Out of curiosity, are you running dovecot locally on those webmail > servers as well, or is it talking to remote dovecot servers? The webmail servers are talking with dovecot director servers which in turn are talking with the backend dovecot servers. Each service running on different servers. Webmail-servers -> director-servers -> backend-servers > I ask because I'm looking at moving our webmail from an on-box setup to a > remote pool to support director and was going to look into whether > running imapproxyd would help there. We don't bother with it in the > local setup, since dovecot is so fast, but remote (but still on a LAN) > might be different. Doesn't seem so to us... > Though imapproxyd seems to make (wait for it...) > squirrelmail unhappy (complains about IMAP errors, when sniffing shows > none), though I've not bothered to debug it yet. :-) -jf From tss at iki.fi Mon Feb 13 21:57:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 21:57:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213193630.GA3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: On 13.2.2012, at 21.36, Jan-Frode Myklebust wrote: >> Other things that would be interesting to try out (both from latency and disk IO usage point of view): >> >> - maildir_very_dirty_syncs > > We already have > > $ doveconf maildir_very_dirty_syncs > maildir_very_dirty_syncs = yes > > but I don't think this gave the advantage I was expecting.. Was > expecting this to move most iops to the index-luns, but the maildir > luns seems just as busy. This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. >> - mail_prefetch_count (Linux+maildir only, v2.1+) > > Will look into if this works with GPFS when we upgrade to v2.1. It has > it's own page cache, so I have no idea if it will respect > POSIX_FADV_WILLNEED or if one will need to use it's own API's for > hinting: > > http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html I guess if there's an easy way to lookup filename or fd -> blockNumber that wouldn't be difficult to implement with a plugin. From chayes at afo.net Mon Feb 13 22:03:02 2012 From: chayes at afo.net (Cliff Hayes) Date: Mon, 13 Feb 2012 14:03:02 -0600 Subject: [Dovecot] Logging to messages instead of secure Message-ID: <5CDFC2DC614E436F89C10AF7F19F7DAB@ciosys> I need to log to /var/log/messages instead of /var/log/secure on a fedora system. Is this the correct entry for dovecot.conf? syslog_facility = info From tss at iki.fi Mon Feb 13 22:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 22:18:42 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> On 13.2.2012, at 19.28, Alex Baule wrote: > static void emexis_mailbox_allocated(struct mailbox *box) > { > union mailbox_module_context *zbox; > > zbox = p_new(box->pool, union mailbox_module_context, 1); > zbox->super = box->v; > box->v.open = Mplugin_mailbox_open; > box->v.sync_notify = Mplugin_mailbox_sync_notify; > > MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); > } The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > struct mailbox_vfuncs *v = box->vlast; > v->sync_notify = Mplugin_mailbox_sync_notify; > > But in this two cases, the Mplugin_mailbox_sync_notify is never > called... i missing something ?? Never called at all? What storage backend are you using as the base? Maildir? Anyway, the way you should be calling it is the exact same way quota plugin does: void Mplugin_mailbox_allocated(struct mailbox *box) { struct mailbox_vfuncs *v = box->vlast; union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = *v; box->vlast = &zbox->super; v->open = Mplugin_mailbox_open; v->sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); } From slusarz at curecanti.org Mon Feb 13 22:24:25 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 13:24:25 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 0.370411 1092386 > webmail2.example.net 0.374227 1045141 > webmail3.example.net 0.378097 1043919 imapproxy > webmail4.example.net 0.378593 1028653 imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 26.048 1125 > webmail2.example.net 26.2997 1080 > webmail3.example.net 28.5596 808 imapproxy > webmail4.example.net 27.1004 964 imapproxy > > ONLY requests that took more than 10 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 49.1407 516 > webmail2.example.net 53.0139 459 > webmail3.example.net 59.7906 333 imapproxy > webmail4.example.net 58.167 384 imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Except you are most likely NOT leveraging the truly interesting part of imapproxy - the ability to restore the IMAP connection state via the XPROXYREUSE status response. This is a significant performance improvement since it also reduces processing load on the client side (everything before/including authentication needs to be done whether using imapproxy or not, so there is no client-side savings for these commands). For further information, see, e.g.: http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. michael From janfrode at tanso.net Mon Feb 13 22:26:23 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:26:23 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: <20120213202623.GA5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 09:57:31PM +0200, Timo Sirainen wrote: > > > > $ doveconf maildir_very_dirty_syncs > > maildir_very_dirty_syncs = yes > > > > but I don't think this gave the advantage I was expecting.. Was > > expecting this to move most iops to the index-luns, but the maildir > > luns seems just as busy. > > This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. With maildir_very_dirty_syncs = yes: ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 1.5 0.0 96 92 514 73 9 7 1.2 0.0 59 43 367 18 4 76 1.7 0.0 66 61 477 67 2 6 1.2 0.0 54 50 348 31 1 145 3.0 0.0 113 90 860 59 7 8 2.9 0.0 107 99 840 58 5 11 4.0 0.0 131 101 1117 77 2 65 With maildir_very_dirty_syncs = no (same node, a few seconds later): ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 4.6 0.9 125 91 1161 1096 41 6 2.3 0.7 200 170 697 127 86 16 1.1 0.6 124 99 406 61 48 109 2.7 0.1 212 144 755 114 74 15 2.7 0.0 159 133 818 70 78 194 0.8 1.2 86 73 225 60 16 9 1.9 0.0 124 116 573 53 30 6 So it seems to be working, good :-) -jf From janfrode at tanso.net Mon Feb 13 22:35:11 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:35:11 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <20120213203511.GB5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 01:24:25PM -0700, Michael M Slusarz wrote: > > Except you are most likely NOT leveraging the truly interesting part > of imapproxy - the ability to restore the IMAP connection state via > the XPROXYREUSE status response. This is a significant performance > improvement since it also reduces processing load on the client side > (everything before/including authentication needs to be done whether > using imapproxy or not, so there is no client-side savings for these > commands). Thanks for this info, good to know. I'll check with inverse/sogo if this is something they use/intend to use.. > > additional advantage. Note that the XPROXYREUSE-enabled MUA must be > the exclusive user of the imapproxy instance for this feature to > work correctly. Not a problem. Assuming it doesn't also need to be the only imap user of the account/folder. BTW: do you also have information on the state of select caching in the up-imapproxy? I got some very negative comments when googling it, and the changelog didn't suggest there had been any improvements since.. -jf From stephan at rename-it.nl Mon Feb 13 23:01:10 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 22:01:10 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.6 for Dovecot v2.0.18 Message-ID: <4F397A16.1010107@rename-it.nl> Hello Dovecot users, Pigeonhole v0.2.5 does not compile against the new Dovecot v2.0.18 release. That is why a release a new version now. Other changes are included as well, of which the one regarding the include extension requires special attention; Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. * Matched changes regarding auth_verbose setting in Dovecot. This means that this release will only compile against Dovecot v2.0.18. - Fixed problem in ManageSieve that caused it to omit a WARNINGS response code when the uploaded script compiled with warnings. - Made sure that locations of Sieve error never report `line 0'. - Fixed potential segfault occurring when interpreter initialization fails. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Mon Feb 13 23:32:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:32:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> On 13.2.2012, at 22.24, Michael M Slusarz wrote: > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html > > These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. > > Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. Perhaps a way for (trusted) clients to be able to do this? :) a logout save * OK [SAVEDSTATE 1234567890] * BYE logged out a OK ... b login (SAVEDSTATE 1234567890) user pass * OK [RESTOREDSTATE 1234567890] b ok vs. b login (SAVEDSTATE 1234567890) user pass * NO [UNKNOWNSTATE 1234567890] b ok From tss at iki.fi Mon Feb 13 23:44:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:44:47 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: On 13.2.2012, at 23.32, Timo Sirainen wrote: > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK Actually, this could probably be safely implemented by sending all of the state to the client as a string: * OK [SAVEDSTATE base64-encoded-state] There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). From ybhu at hk1.ibm.com Tue Feb 14 04:39:01 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 10:39:01 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: Another question is about the director failover. In http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html you said: "The main complexity here comes from how to handle proxy server failures in different situations. Those are less interesting to describe and I haven't yet implemented all of it, so let's just assume that in future it all works perfectly." So currently there is no good way to handle the director failover, or any 3rd party solution like poolmon by Brad Davidson for health monitoring of backend servers? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/12/2012 12:06 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09320.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Tue Feb 14 04:46:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 04:46:21 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: On 14.2.2012, at 4.39, Andy YB Hu wrote: > Another question is about the director failover. In > http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html > you said: "The main complexity here comes from how to handle proxy server > failures in different situations. Those are less interesting to describe > and I haven't yet implemented all of it, so let's just assume that in > future it all works perfectly." So currently there is no good way to > handle the director failover, or any 3rd party solution like poolmon by > Brad Davidson for health monitoring of backend servers? Your load balancer handles director downtimes so it connects only to directors that are up. Directors themselves figure out when one of them is down (by either explicit disconnection or timeout). That's the theory, and so far no one's told me it doesn't work that way. For the backend servers there's still no automation though. You'll need to explicitly tell director to stop trying to connect some specific backend. The poolmon is intended for that. It would be possible to implement this directly in the director itself, but so far it hasn't really been a priority since the companies who have paid for it have wanted to implement it internally themselves.. From dmiller at amfes.com Tue Feb 14 04:54:20 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 18:54:20 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 2/12/2012 1:01 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > With this configure line: ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene I get this error on make: make[4]: Entering directory `/opt/dovecot-2.1.rc6/src/plugins/fts-lucene' /bin/bash ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arh -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -o lib20_doveadm_s_lucene_plugin.la -rpath /usr/local/lib/dovecot/doveadm doveadm-fts-lucene.lo -lrt /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I..//../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF eps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/l-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wraer.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory compilation terminated. make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... -- Daniel From tss at iki.fi Tue Feb 14 05:00:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:00:46 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 14.2.2012, at 4.54, Daniel L. Miller wrote: > With this configure line: > ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene > > lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ From dmiller at amfes.com Tue Feb 14 05:01:59 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:01:59 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 2/13/2012 7:00 PM, Timo Sirainen wrote: > On 14.2.2012, at 4.54, Daniel L. Miller wrote: > >> With this configure line: >> ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene >> >> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory > Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ > Ah! I would've thought configure would catch that though - before make did. Ok. -- Daniel From dmiller at amfes.com Tue Feb 14 05:04:26 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:04:26 -0800 Subject: [Dovecot] Boehm Message-ID: As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? -- Daniel From tss at iki.fi Tue Feb 14 05:05:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:05:33 +0200 Subject: [Dovecot] Boehm In-Reply-To: References: Message-ID: <4E1CD0CF-7679-4D4D-9DF1-E5B101ECE797@iki.fi> On 14.2.2012, at 5.04, Daniel L. Miller wrote: > As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? Eh. Probably not. From slusarz at curecanti.org Tue Feb 14 05:12:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:12:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203511.GB5085@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <20120213203511.GB5085@dibs.tanso.net> Message-ID: <20120213201256.Horde.k3ITIIF5lbhPOdE4A_7B9KA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > BTW: do you also have information on the state of select caching in the > up-imapproxy? I got some very negative comments when googling it, and the > changelog didn't suggest there had been any improvements since.. I wouldn't trust it. IIRC, it was added years ago and given the syntax changes to the SELECT/EXAMINE call since then (e.g. QRESYNC extentsion), it is doubtful the code has been updated to handle these situations. And in a server-caching IMAP server like dovecot, all that SELECT information is not expensive anyway, so any gains would be miniscule. michael From slusarz at curecanti.org Tue Feb 14 05:19:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:19:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 22.24, Michael M Slusarz wrote: > >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html >> >> These posts neglect the fact that you don't need to issue a >> CAPABILITY command if the connection is reused either, so that's an >> additional advantage. Note that the XPROXYREUSE-enabled MUA must be >> the exclusive user of the imapproxy instance for this feature to >> work correctly. >> >> Somewhat topical, since Timo was just mentioning support for some >> sort of IMAP state save/restore feature possibly making it into 2.2. > > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK > > ... > > b login (SAVEDSTATE 1234567890) user pass > * OK [RESTOREDSTATE 1234567890] > b ok > > vs. > > b login (SAVEDSTATE 1234567890) user pass > * NO [UNKNOWNSTATE 1234567890] > b ok I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. And you don't allow reviving the state if using the AUTHENTICATE command. The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. michael From tss at iki.fi Tue Feb 14 05:23:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:23:16 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Message-ID: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> On 14.2.2012, at 5.19, Michael M Slusarz wrote: >> b login (SAVEDSTATE 1234567890) user pass > > I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. No breaking, extending :) > And you don't allow reviving the state if using the AUTHENTICATE command. Just as easy: b authenticate (savedstate ..) plain .. > The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. Yeah. Although not bad either, since you can pipeline that command + login/authenticate. From slusarz at curecanti.org Tue Feb 14 05:30:09 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:30:09 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 23.32, Timo Sirainen wrote: > >> Perhaps a way for (trusted) clients to be able to do this? :) >> >> a logout save >> * OK [SAVEDSTATE 1234567890] >> * BYE logged out >> a OK > > Actually, this could probably be safely implemented by sending all > of the state to the client as a string: > > * OK [SAVEDSTATE base64-encoded-state] > > There isn't a whole lot of state to be saved really. Mailbox GUID, > UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have > the language/etc. states. Clients could restore their earlier state > from days ago, as long as Dovecot still has the necessary .log > records available (similar to how QRESYNC works). Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. For a disconnected client (e.g. webmail), there's going to necessarily need to be startup costs of initializing the session anyway so re-creating the state can be buried in this other work. Re-creating becomes more important when users are doing actions they expect immediate (or at least very fast) responses for. Things like listing messages in a mailbox, viewing a message, or polling mailboxes. Saving 50ms per request becomes important in these situations since re-creating state now takes 10% of the total request time. So I don't think states don't need a terribly long lifetime. I almost think of something like an internal Dovecot IDLE queue - after a logout is received, state is stored for 30 minutes after which it is discarded. (Although not knowing anything about internal Dovecot state, don't know if this is overly resource intensive.) As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if you come up with a better way to represent state. michael From dmiller at amfes.com Tue Feb 14 05:34:44 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:34:44 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: On 2/12/2012 9:00 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.27, Mike Constabel wrote: > >> Another Question to this Option: >> >> # tb-extra-mailbox-sep: >> # With mbox storage a mailbox can contain either mails or submailboxes, >> # but not both. Thunderbird separates these two by forcing server to >> # accept '/' suffix in mailbox names in subscriptions list. >> >> >> I thought this is not needed because I use mdbox. But mbox here >> means also mdbox and sdbox? > Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > >> Should I use all 3 workarounds if I don't know which clients are >> in use? > I don't know if delay-newmail is still relevant. > I'm fighting the same issue. With the current release (2.1rc6) no errors are logged - but I'm having problems deleted folders from trash. I've enabled tb-extra-mailbox-sep. Two folders that were previously in Trash were successfully removed. I deleted four more - after several Thunderbird restarts, compacts, refreshes, I have killed three of them - there is still one folder that won't "die". Thunderbird reports the mailbox doesn't exist, yet it continues to appear. -- Daniel From slusarz at curecanti.org Tue Feb 14 05:35:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:35:46 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> Message-ID: <20120213203546.Horde.TaVqCoF5lbhPOdaS8PLB9SA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 14.2.2012, at 5.19, Michael M Slusarz wrote: > >>> b login (SAVEDSTATE 1234567890) user pass >> >> I guess the drawback for this approach is that you are explicitly >> breaking the LOGIN definition. > > No breaking, extending :) Heh. I should know better after reading about 1000 RFCs last year... Anyway, to play devil's advocate, another drawback of this approach is that in the (fairly remote) chance that the LOGIN command is extended in the future by an RFC, there is no guarantee that this syntax would be used. michael From ybhu at hk1.ibm.com Tue Feb 14 05:31:54 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 11:31:54 +0800 Subject: [Dovecot] index synchronization during local delivery Message-ID: Hi there, Now our system will take dovecot as the whole imap back end, and keep the LDA of ours, But our system has our own UID generation mechanism (in dovecot it's incremental from 1), which will be done while local delivery, So when local deliveries, it will first generate the UID, move the mail into maildir, then synchronize the mailbox index, which is part of dovecot LDA does. The question is what API or something else about index synchronization I can integrate into our own LDA program easily? Seems mailbox_copy() would be the possibility? Thanks. From busseniu at in.tum.de Tue Feb 14 17:04:25 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 14 Feb 2012 16:04:25 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit Message-ID: <4F3A77F9.8000002@in.tum.de> Hi, our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. The "doveadm proxy list" would fail with an error message: $ sudo doveadm proxy list doveadm(root): Error: LIST failed: Disconnected Segmentation fault The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. Anyway, I suggest changing the log message if the failure is not actually due to the client_limit. # 2.0.18: /etc/ports/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.2 LTS auth_username_format = %n default_client_limit = 60000 default_process_limit = 10000 disable_plaintext_auth = no mail_location = maildir:/nonexistent managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/ports/dovecot/passwd driver = passwd-file } protocols = imap pop3 lmtp sieve service anvil { client_limit = 100000 } service auth { client_limit = 60000 } service ipc { client_limit = 100000 } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } shutdown_clients = no ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From cinzia.delcoiro at libero.it Tue Feb 14 17:02:07 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:02:07 +0100 (CET) Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <25869845.713261329231727832.JavaMail.root@wmail51> Hi Timo, I'm writing to alert you that even in version 2.1. we have the error: "Can't handle mixed proxy/non-proxy destinations" when we write to two users: one existent and one not. The error occurs randomly. I configure LDAP query in the file : dovecot-ldap.conf in this way: pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, =proxy_timeout=600 specifying "=proxy=y" in order to proxy every mail. As workaround, can I configure a static "host" for non-existing users? This is my dovecot -n # 2.1.rc5: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_mechanisms = plain login base_dir = /usr/local/var/run/dovecot/ disable_plaintext_auth = no listen = 127.0.0.1 lmtp_proxy = yes mail_gid = mailusers mail_location = maildir:/mnt/mbox1/mailusers/%n/Maildir mail_uid = mailusers passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User Quota } protocols = lmtp service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no syslog_facility = local5 userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { mail_plugins = quota imap_quota } protocol lmtp { postmaster_address = info at amm2.miur.it } Regards Cinzia >----Messaggio originale---- >Da: tss at iki.fi >Data: 28-gen-2012 18.32 >A: "Alexis Lelion" >Cc: "Dovecot Mailing List" >Ogg: Re: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations > >On 27.1.2012, at 12.59, Alexis Lelion wrote: > >> Jan 25 09:05:12 mail01 postfix/lmtp[23934]: A92709300DB: to=< >> user_on_mail02 at domain.com>, relay=mail01.domain.com[private/dovecot-lmtp], >> delay=0.07, delays=0.01/0/0/0.06, dsn=4.3.0, status=deferred (host >> mail01.domain.com[private/dovecot-lmtp] said: 451 4.3.0 < >> user_on_mail02 at domain.com> Can't handle mixed proxy/non-proxy destinations >> (in reply to RCPT TO command)) >> >> I was wondering if there was another way of handling this, for example >> by triggering an immediate queue lookup from postfix or forwarding a >> copy of the mail to the other server. Note that the postfix >> "queue_run_delay" was increased to 15min on purpose, so I cannot change >> that. > >It would be possible to change the code to support mixed destinations, but it's probably not a simple change and I have other things to do.. > >Maybe you could work around it so that LMTP always proxies the mails, to localhost as well, but to a different port which doesn't do proxying at all. > > From CMarcus at Media-Brokers.com Tue Feb 14 17:42:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 14 Feb 2012 10:42:12 -0500 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <4F3A80D4.4000007@Media-Brokers.com> On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it wrote: > I'm writing to alert you that even in version 2.1. we have the error: "Can't > handle mixed proxy/non-proxy destinations" when we write to two users: one > existent and one not. why are you accepting mail for non existing users? Catch-alls are a very very bad idea in 99.999% of cases, and usually when someone thinks they need a catchall, they are mistaken. -- Best regards, Charles From cinzia.delcoiro at libero.it Tue Feb 14 17:54:38 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:54:38 +0100 (CET) Subject: [Dovecot] R: Re: R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <23144211.746481329234878394.JavaMail.root@wmail51> Charles I need to accept mail also for nonexistent users because I'm delivering a particular service (Certified Mail). Regards Cinzia >----Messaggio originale---- >Da: CMarcus at Media-Brokers.com >Data: 14-feb-2012 16.42 >A: "Dovecot Mailing List" >Ogg: Re: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations > >On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it > wrote: >> I'm writing to alert you that even in version 2.1. we have the error: "Can't >> handle mixed proxy/non-proxy destinations" when we write to two users: one >> existent and one not. > >why are you accepting mail for non existing users? > >Catch-alls are a very very bad idea in 99.999% of cases, and usually >when someone thinks they need a catchall, they are mistaken. > >-- > >Best regards, > >Charles > From janfrode at tanso.net Tue Feb 14 21:34:42 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 20:34:42 +0100 Subject: [Dovecot] pop3 not autocreating directory structure Message-ID: <20120214193442.GA5156@dibs.tanso.net> We use: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail: mdbox:~/mdbox and I just noticed one of our newly provisioned users initially failed to pop her mails. I saw several of these: dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX dovecot:: pop3(new.user at example.net): Couldn't open INBOX top=0/0, retr=0/0, del=0/0, size=0 before she switched to imap and then everything looked fine: dovecot:: imap(new.user at example.net): Disconnected: Logged out bytes=11/338 Looking at the timestamps in the filesystem I see that the users home directory wasn't created before switcing to imap. Is this a know problem? -jf From wgillespie+dovecot at es2eng.com Tue Feb 14 22:06:56 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Tue, 14 Feb 2012 13:06:56 -0700 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: <4F3ABEE0.6010702@es2eng.com> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: > I'm fighting the same issue. With the current release (2.1rc6) no errors > are logged - but I'm having problems deleted folders from trash. I've > enabled tb-extra-mailbox-sep. Two folders that were previously in Trash > were successfully removed. I deleted four more - after several > Thunderbird restarts, compacts, refreshes, I have killed three of them - > there is still one folder that won't "die". Thunderbird reports the > mailbox doesn't exist, yet it continues to appear. Out of curiosity -- does the mailbox actually exist if you look on the Dovecot server? Secondly, is it listed in your subscription file? From dmiller at amfes.com Tue Feb 14 22:40:02 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 14 Feb 2012 12:40:02 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3ABEE0.6010702@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> Message-ID: On 2/14/2012 12:06 PM, Willie Gillespie wrote: > On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >> I'm fighting the same issue. With the current release (2.1rc6) no errors >> are logged - but I'm having problems deleted folders from trash. I've >> enabled tb-extra-mailbox-sep. Two folders that were previously in Trash >> were successfully removed. I deleted four more - after several >> Thunderbird restarts, compacts, refreshes, I have killed three of them - >> there is still one folder that won't "die". Thunderbird reports the >> mailbox doesn't exist, yet it continues to appear. > > Out of curiosity -- does the mailbox actually exist if you look on the > Dovecot server? > Secondly, is it listed in your subscription file? At one time - yes they were. After further playing, they do not exist in the on-disk folder structure, nor in the subscription file. But still show in the client - with errors. -- Daniel From janfrode at tanso.net Tue Feb 14 22:47:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 21:47:51 +0100 Subject: [Dovecot] doveadm director proxy Message-ID: <20120214204751.GA6279@dibs.tanso.net> I'm trying to configure a doveadm service that will proxy trough our directors, following the recipie at: http://wiki2.dovecot.org/Director#Doveadm_server So on the backends I have: service doveadm { inet_listener { port = 24245 address = * } } doveadm_proxy_port = 24245 local 192.168.42.0/24 { doveadm_password = suPerSeecret } I assume the "local" line is supposed to point at my local network..? On the directors I have the same, plus: protocol doveadm { auth_socket_path = director-userdb } When testing "doveadm quota" on the directors, it complained quota plugin not being loaded, so I added: mail_plugins=quota Then it complained about "doveadm_password not set, can't authenticate", so I added: doveadm_password = suPerSeecret in the main section. Now I get trough to my backend servers, but they complain about: dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication Any ideas for what that might be? This is with dovecot v2.0.14. -jf From as at 3a.pl Tue Feb 14 23:29:19 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 14 Feb 2012 22:29:19 +0100 Subject: [Dovecot] domain quota in sql Message-ID: <4F3AD22F.5080904@3a.pl> Hi, I have a problem with setting up quota usage tracking for domains. Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. To recreate this situation, one has only to perform: doveadm quota recalc -u USER on a user, and the domain quota takes the value of this user quota. Have anyone a clue what could cause this problem? Quota tracking is set up both for users and domains. For users quota tracking is working fine, data is saved into dbase without problems. We are using dovecot 2.0.15. Quota enforcing (rules) is NOT a part of the problem. Important parts of config: dict { quota = mysql:/etc/dovecot/dovecot-dict-used-quota.conf.ext quota2 = mysql:/etc/dovecot/dovecot-dict-used-quota-domain.conf.ext } plugin { quota = dict:User quota::proxy::quota quota2 = dict:Domain quota:%d:proxy::quota2 } /etc/dovecot/dovecot-dict-used-quota.conf.ext map { pattern = priv/quota/storage table = used_quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = used_quota username_field = username value_field = messages } /etc/dovecot/dovecot-dict-used-quota-domain.conf.ext map { pattern = priv/quota/storage table = used_quota_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = used_quota_domain username_field = domain value_field = messages } -- Adam Szpakowski From tss at iki.fi Wed Feb 15 03:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:47:11 +0200 Subject: [Dovecot] doveadm director proxy In-Reply-To: <20120214204751.GA6279@dibs.tanso.net> References: <20120214204751.GA6279@dibs.tanso.net> Message-ID: On 14.2.2012, at 22.47, Jan-Frode Myklebust wrote: > I'm trying to configure a doveadm service that will proxy trough our > directors, following the recipie at: > > http://wiki2.dovecot.org/Director#Doveadm_server > > So on the backends I have: > > service doveadm { > inet_listener { > port = 24245 > address = * > } > } > doveadm_proxy_port = 24245 > local 192.168.42.0/24 { > doveadm_password = suPerSeecret > } > > I assume the "local" line is supposed to point at my local network..? Yes. Those IPs where you allow authentication from. > in the main section. Now I get trough to my backend servers, but they > complain about: > > dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication This means doveadm was talking incorrectly. > Any ideas for what that might be? This is with dovecot v2.0.14. Some v2.0 versions had bugs related to doveadm server. So this is probably a bug in v2.0.14. And I'm not entirely sure if doveadm server works correctly in v2.0 at all, since I did some bigger changes to v2.1 related to it. From tss at iki.fi Wed Feb 15 03:49:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:49:21 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120214193442.GA5156@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: On 14.2.2012, at 21.34, Jan-Frode Myklebust wrote: > dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX > > Looking at the timestamps in the filesystem I see that the users home > directory wasn't created before switcing to imap. > > Is this a know problem? Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). From tss at iki.fi Wed Feb 15 03:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:54:01 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3AD22F.5080904@3a.pl> References: <4F3AD22F.5080904@3a.pl> Message-ID: <147CCD39-537F-455A-9B55-212340747969@iki.fi> On 14.2.2012, at 23.29, Adam Szpakowski wrote: > Hi, > I have a problem with setting up quota usage tracking for domains. > Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. > To recreate this situation, one has only to perform: > doveadm quota recalc -u USER > on a user, and the domain quota takes the value of this user quota. > > Have anyone a clue what could cause this problem? Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: - set domain's quota to 0 - iterate *@domain users via userdb - for each user get the user's quota and add it to domain quota From tss at iki.fi Wed Feb 15 03:55:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:55:57 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <147CCD39-537F-455A-9B55-212340747969@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> Message-ID: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> On 15.2.2012, at 3.54, Timo Sirainen wrote: > Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: Oh, and you could actually already do this with a doveadm script: > - set domain's quota to 0 update .. | mysql > - iterate *@domain users via userdb doveadm user '*@domain' > - for each user get the user's quota and add it to domain quota doveadm quota get -u $user | some grep+sed magic | mysql Of course it would be nice if Dovecot internally supported this. From tss at iki.fi Wed Feb 15 04:04:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:04:24 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Message-ID: <6E30EEF1-1921-4E76-88AB-8743901E5E5C@iki.fi> On 14.2.2012, at 5.30, Michael M Slusarz wrote: >> Actually, this could probably be safely implemented by sending all of the state to the client as a string: >> >> * OK [SAVEDSTATE base64-encoded-state] >> >> There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). > > Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. There wouldn't be *any* additional state stored. All the necessary state is already stored to .log files for other purposes. The base64-encoded-state would be a self-contained description of the IMAP connection's entire state. > As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if > you come up with a better way to represent state. The client doesn't need to care about the data structure. The client simply reads a string and sends it later back to server if it wants to restore that state. Only the server needs to verify that the string looks reasonable. The same string wouldn't be sent to a different server implementation, so there wouldn't be any interoperability issues. Each server can implement it in whatever way they want to (although there should be some checks in case the same string is sent to different versions of the same server). From tss at iki.fi Wed Feb 15 04:09:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:09:57 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 14.2.2012, at 5.01, Daniel L. Miller wrote: >>> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory >> Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ >> > Ah! I would've thought configure would catch that though - before make did. Ok. Checking for C++ things in autoconf seems to be a little difficult.. From tss at iki.fi Wed Feb 15 04:19:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:19:00 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: Message-ID: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. From tss at iki.fi Wed Feb 15 04:24:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:24:19 +0200 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <2AF7DAA3-ABB0-47AC-9189-E499C68DA370@iki.fi> On 14.2.2012, at 17.02, cinzia.delcoiro at libero.it wrote: > I configure LDAP query in the file : dovecot-ldap.conf in this way: > > pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, > =proxy_timeout=600 > > specifying "=proxy=y" in order to proxy every mail. In LMTP case proxy=y and proxy_maybe=y are actually exactly the same. If you want the mail always proxied, you need to also use a different port. So I guess it'll work if you: 1) Add another LMTP service listener for port 10024 2) Add =proxy_port=10024 to pass_attrs. Although if you use this same passdb for imap/pop3 lookups as well, you need to make the port change lmtp-only: protocol lmtp { passdb { driver = sql args = dovecot-ldap-lmtp.conf } } protocol !lmtp { passdb { driver = sql args = dovecot-ldap.conf } } From tss at iki.fi Wed Feb 15 04:35:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:35:22 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3A77F9.8000002@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: > > Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? > I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. Do you see some error message also about "Too many open files", or why do you think the number of open files was the problem? If a process runs out of fds there should be such an error message. > Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. Connections to anvil and ipc processes were being dropped. That doesn't necessarily mean that everything else stops. > The "doveadm proxy list" would fail with an error message: > > $ sudo doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault > > The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. From tss at iki.fi Wed Feb 15 04:37:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:37:58 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 15.2.2012, at 4.35, Timo Sirainen wrote: >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Lets see if I can reproduce this then.. From list at airstreamcomm.net Wed Feb 15 05:08:05 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 14 Feb 2012 21:08:05 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> On Mon, 13 Feb 2012 13:47:06 +0200, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot > v2.2. Probably not all of them will make it, but I'm at least interested in > working on these if I have time. > > Previously I've mostly been working on things that different companies > were paying me to work on. This is the first time I have my own company, > but the prioritization still works pretty much the same way: > > - 1. priority: If your company is highly interested in getting something > implemented, we can do it as a project via my company. This guarantees > that you'll get the feature implemented in a way that integrates well into > your system. > - 2. priority: Companies who have bought Dovecot support contract can let > me know what they're interested in getting implemented. It's not a > guarantee that it gets implemented, but it does affect my priorities. :) > - 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are > before the 2. priority above. I guess we'll see how things work out. > > dsync-based replication > ----------------------- > > I'll write a separate post about this later. Besides, it's coming for > Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. > > Shared mailbox improvements > --------------------------- > > Support for private flags for all mailbox formats: > > namespace { > type = public > prefix = Public/ > mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public > } > > - dsync needs to be able to replicate the private flags as well as shared > flags. > - might as well add a common way for all mailbox formats to specify which > flags are shared and which aren't. $controldir/dovecot-flags would say > which is the default (private or shared) and what flags/keywords are the > opposite. > - easy way to configure shared mailboxes to be accessed via imapc > backend, which would allow easy shared mailbox accesses across servers or > simply between two system users in same server. (this may be tricky to > dsync.) > - global ACLs read from a single file supporting wildcards, instead of > multiple different files > - default ACLs for each namespace/storage root (maybe implemented using > the above..) > > Metadata / annotations > ---------------------- > > Add support for server, mailbox and mail annotations. These need to be > dsyncable, so their changes need to be stored in various .log files: > > 1. Per-server metadata. This is similar to subscriptions: Add changes to > dovecot.mailbox.log file, with each entry name a hash of the metadata key > that was changed. > > 2. Per-mailbox metadata. Changes to this belong inside > mailbox_transaction_context, which write the changes to mailbox's > dovecot.index.log files. Each log record contains a list of changed > annotation keys. This gives each change a modseq, and also allows easily > finding out what changes other clients have done, so if a client has done > ENABLE METADATA Dovecot can easily push metadata changes to client by only > reading the dovecot.index.log file. > > 3. Per-mail metadata. This is pretty much equivalent to per-mailbox > metadata, except changes are associated to specific message UIDs. > > The permanent storage is in dict. The dict keys have components: > - priv/ vs. shared/ for specifying private vs. shared metadata > - server/ vs mailbox// vs. mail// > - the metadata key name > > This would be a good time to improve the dict configuration to allow > things like: > - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a > file, while the rest goes to sql) > - allow sql dict to be used in more relational way, so that mail > annotations could be stored with tables: mailbox (id, guid) and > mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid > everywhere. > > Things to think through: > - How to handle quota? Probably needs to be different from regular mail > quota. Probably some per-user "metadata quota bytes" counter/limit. > - Dict lookups should be done asynchronously and prefetched as much as > possible. For per-mail annotation lookups mail_alloc() needs to include a > list of annotations that are wanted. > > Configuration > ------------- > > Copy all mail settings to namespaces, so it'll be possible to use > per-namespace mailbox settings. Especially important for imapc_* settings, > but can be useful for others as well. Those settings that aren't explicitly > defined in the namespace will use the global defaults. (Should doveconf -a > show all of these values, or simply the explicitly set values?) > > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so > doveconf -n outputs ALL of the configuration. There are mainly 3 config > files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. > The dict-sql is something I think needs a bigger redesign (mentioned above > in "Metadata" section), but the sql/ldap auth configs could be merged. One > way could be: > > sql_db sqlmails { > # most settings from dovecot-sql.conf.ext, except for queries > driver = mysql > connect = ... > } > > ldap_db ldapmails { > # most settings from dovecot-ldap.conf.ext, except attributes/filters > } > > passdb { > driver = sql > db = sqlmails > sql_query = select password from users where username = '%u' > } > passdb { > driver = ldap > db = ldapmails > ldap_attributes { > password = %{ldap:userPassword} > } > ldap_filter = ... > } > > The sql_db {} and ldap_db {} would be generic enough to be used everywhere > (e.g. dict-sql), not just for passdb/userdb. > > Some problems: > - Similar to the per-namespace mail settings, doveconf -a would output > all sql_query, ldap_attributes, ldap_filter, etc. settings for all > passdbs/userdbs. Perhaps a similar solution? > - The database configs contain passwords, so they should be readable only > by root. This makes running dovecot-lda and maybe doveadm difficult, since > they fail at "permission denied" when trying to open the config. There are > probably only two solutions: a) The db configs need to be !include_try'd > or b) the configs can be world-readable, but only passwords are placed to > only-root-readable files by using "password = > IMAP state saving/restoring > --------------------------- > > IMAP connections are often long running. Problems with this: > > 1. Currently each connection requires a separate process (at least to work > reliably), which means each connection also uses quite a lot of memory even > when they aren't doing anything for a long time. > 2. Some clients don't handle lost connections very nicely. So Dovecot > can't be upgraded without causing some user annoyance. Also in a cluster if > you want to bring down one server, the connections have to be disconnected > before they can be moved to another server. > > If IMAP session state could be reliably saved and later restored to > another process, both of the above problems could be avoided entirely. > Typically when a connection is IDLEing there are really just 4 things that > need to be remembered: username, selected mailbox name, its UIDVALIDITY and > HIGHESTMODSEQ. With this information the IMAP session can be fully restored > in another process without losing any state. So, what we could do is: > > 1. When an IMAP connection has bee IDLEing for a while (configurable > initial time, could be dynamically adjusted): > - move the IMAP state and the connection fd to imap-idle process > - the old imap process is destroyed > - imap-idle process can handle lots of IMAP connections > - imap-idle process also uses inotify/etc. to watch for changes in the > specified mailbox > - if any mailbox changes happen or IMAP client sends a command, start up > a new imap process, restore the state and continue from where we left off > - This could save quite a lot of memory at the expense of some CPU usage > > 2. Dovecot proxy <-> backend protocol could be improved to support moving > connection to another backend. Possibly using a separate control connection > to avoid making the proxying less efficient in normal operation. > > 3. When restarting Dovecot, move all the connections to a process that > keeps the connections open for a while. When Dovecot starts up, create imap > processes back to the connections. This allows changing configuration for > existing client connections (which sometimes may be bad! need to add checks > against client-visible config conflicts), upgrading Dovecot, etc. without > being visible to clients. The only problem is SSL connections: OpenSSL > doens't provide a way to save/restore state, so either you need to set > shutdown_clients=no (and possibly keep some imap-login processes doing SSL > proxying for a long time), or SSL connections need to be killed. Of course > the SSL handling could be outsourced to some other software/hardware > outside Dovecot. > > The IMAP state saving isn't always easy. Initially it could be implemented > only for the simple cases (which are a majority) and later extended to > cover more. > > IMAP extensions > --------------- > > - CATENATE is already implemented by Stephan > - URLAUTH is also planned to be implemented, somewhat differently than in > Apple's patch. The idea is to create a separate imap-urlauth service that > provides extra security. > - NOTIFY extension could be implemented efficiently using mailbox list > indexes, which already exists in v2.1. > - FILTERS extension can be easily implemented once METADATA is implemented > - There are also other missing extensions, but they're probably less > important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, > CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. > > Backups > ------- > > Filesystem based backups have worked well enough with Dovecot in the past. > But with new features like single instance storage it's becoming more > difficult. There's no 100% consistent way to even get filesystem level > backups with SIS enabled, because deleting both the message file and its > attachment files can't be done atomically (although usually this isn't a > real problem). Restoring SIS mails is more difficult though, first you need > to restore the dbox mail files and then you need to figure out what > attachment files from SIS need to be restored, and finally you'll need to > do doveadm import to put them into their final destination. > > I don't have much experience with backup software, but other people in my > company do. The initial idea is to implement a Dovecot backup agent to one > (commercial) backup software, which allows doing online backups and > restoring mails one user/mailbox/mail at a time. I don't know the details > yet how exactly this is going to be implemented, but the basic plan is > probably to implement a "backup" mail storage backend, which is a > PostgreSQL pg_dump-like flat file containing mails from all mailboxes. > doveadm backup/import can then export/import this format via stdout/stdin. > Incremental backups could possibly be done by giving a timestamp of > previous backup run (I'm not sure about this yet). > > Once I've managed to implement the first fully functional backup agent, it > should become clearer how to implement it to other backup solutions. > > Random things > ------------- > > - dovecot.index.cache file writing is too complex, should be simplified > - Enable auth_debug[_passwords]=yes on-the-fly for some specific > users/IPs via doveadm > - Optimize virtual mailboxes using mailbox list indexes. It wouldn't > anymore need to keep all the backend mailboxes' index files open. > - Would be nice to go forward with supporting key-value databases as mail > storage backends. Timo, I know you mentioned you would cover this in a coming post, but we were curious what the new dsync replication will be capable of. Would it monitor changes to mailboxes and push automatic replication to the remote mail store, and if this is the case could it be an N-way replication setup in which any host in a cluster can participate in the replication? Do you consider this to be a high availability solution? Thanks, Michael From tss at iki.fi Wed Feb 15 05:46:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 05:46:35 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: On 13.2.2012, at 16.40, Timo Sirainen wrote: > On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > >> when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. >> >> My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. >> - Is there some way to make dsync notice the hard links? (I used >> dovecot 2.0.17) > > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2890 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:07:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:07:55 +0200 Subject: [Dovecot] v2.1.rc7 released Message-ID: http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz.sig Okay, one more RC, because I needed a new feature implemented and forgot I wasn't supposed to do any changes anymore before v2.1.0. After that I figured I'll just do a few more changes and release RC7. So, v2.1.0 will probably be released within 1-2 days. So, the one new feature is: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. Also (not really worth mentioning in NEWS) if you now connect to imap/pop3 port when auth process is misconfigured and dies, instead of hanging the client for 5 mins it gets disconnected pretty much immediately. From ybhu at hk1.ibm.com Wed Feb 15 06:13:18 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Wed, 15 Feb 2012 12:13:18 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: I can generate the UID once I create the uidlist lock, and the UID is incremental globally. Some questions: 1, Whether I can write the mail to tmp/ after creating dovecot-uidlist.lock? 2, I did the 5 steps manually, found that new mail is set to OLD flag, so that when SELECT command, it told there was no NEW mail. Looks like need to change code in maildir_uidlist_sync_next_uid()? 3, About the dovecot-uidlist.lock file creation and deletion, I checked the code, there are functions in dovecot like maildir_uidlist_lock_timeout and maildir_uidlist_unlock. If I did that via calling these functions, there are lots of extra codes need to call(like auth, mailbox open...). So the question is if I do that in my codes, how to handle the race condition correctly? (At least there would be other sessions want to lock the files also / already locked it while I want to create the dovecot-uidlist.lock) Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 10:19 Subject AM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09089.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:25:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:25:29 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h From as at 3a.pl Wed Feb 15 09:26:28 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 08:26:28 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> Message-ID: <4F3B5E24.2080407@3a.pl> On 15.02.2012 02:55, Timo Sirainen wrote: > On 15.2.2012, at 3.54, Timo Sirainen wrote: > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > Oh, and you could actually already do this with a doveadm script: [cut] I'm using something like this, but directly on the dbase level. Unfortunately such operation is quite db intensive, so I'm looking for a simpler solution. Is there a way to add domainname as another collumn in dbase? This will simplify the SQL query and the GROUP BY could be used. > Of course it would be nice if Dovecot internally supported this. I was under impression, that dovecot does support this. In examples one can see: /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: # Multiple quota roots are also possible, for example this gives each user # their own 100MB quota and one shared 1GB quota within the domain: plugin { #quota = dict:user::proxy::quota #quota2 = dict:domain:%d:proxy::quota_domain } Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): "If username is left empty, the logged in username is used (this is typically what you want). Another useful username is '%d' for supporting domain-wide quotas." -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 09:47:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 08:47:22 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <4F3B630A.4070305@schetterer.org> Am 15.02.2012 08:26, schrieb Adam Szpakowski: > On 15.02.2012 02:55, Timo Sirainen wrote: >> On 15.2.2012, at 3.54, Timo Sirainen wrote: >> >>> Domain-quota support in Dovecot is only partial. Recalculating domain >>> quota would require more code. Last time when someone asked about >>> this I wasn't even sure how it could be implemented, but nowadays it >>> actually would be possible to do with: >> Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. > > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. > >> Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: > > # Multiple quota roots are also possible, for example this gives each user > # their own 100MB quota and one shared 1GB quota within the domain: > plugin { > #quota = dict:user::proxy::quota > #quota2 = dict:domain:%d:proxy::quota_domain > } > > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): > > "If username is left empty, the logged in username is used (this is > typically what you want). Another useful username is '%d' for supporting > domain-wide quotas." > > Hi i am not sure what are trying to goal with domain quota, if you setup your accounts i.e with postfixadmin etc then its up to the gui logic , stopping postmasters to setup more accounts ( with quotas ) as you ( the superadmin ) want. For everything else user quotas usally are enough thinkable maybe calculation of all users quotas in dict from one domain in a sql postfix query table to stop deliver in more mail, but i dont think this makes real sense sorry if i missunderstand your question.. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:07:04 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:07:04 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B630A.4070305@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> Message-ID: <4F3B67A8.2030109@3a.pl> On 15.02.2012 08:47, Robert Schetterer wrote: > Hi i am not sure what are trying to goal > with domain quota, if you setup your accounts i.e with postfixadmin etc > then its up to the gui logic , stopping postmasters to setup more > accounts ( with quotas ) as you ( the superadmin ) want. My fault, I should add some "big picture" info. We are using admin panel based on iRedMail Panel, and want to show quota usage on the domain level. Quota enforcing is only on user level and this part works nicely. What I need is to have in dbase live data which allows me to get quota usage on the domain level to show it in the admin panel. The problem is, that this operation must not by db intensive. Getting entire db and the processing it with some script is fine for daily reports but not for "live" view. My example with doveadm for quota recalc was only a way to show how the problem can be triggered. We are not using doveadm for this purpose. > For everything else user quotas usally are enough > > thinkable maybe calculation of all users quotas in dict from one domain > in a sql postfix query table to stop deliver in more mail, but i dont > think this makes real sense In future the quota enforcement on domain level could be a nice add-on, but for now it is not so important for us. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 10:10:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:10:27 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F246339.708@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> Message-ID: <4F3B6873.2010601@schetterer.org> Am 28.01.2012 22:06, schrieb Robert Schetterer: > Am 28.01.2012 21:07, schrieb Timo Sirainen: >> On 13.1.2012, at 20.29, Mark Moseley wrote: >> >>> If there are multiple hosts, it seems like the most robust thing to do >>> would be to exhaust the existing connections and if none of those >>> succeed, then start a new connection to one of them. It will probably >>> result in much more convoluted logic but it'd probably match better >>> what people expect from a retry. >> >> Done: http://hg.dovecot.org/dovecot-2.0/rev/4e7676b890f1 >> > > Hi Timo > > doc/example-config/dovecot-sql.conf.ext > from hg > has something like > > # Database connection string. This is driver-specific setting. > # HA / round-robin load-balancing is supported by giving multiple host > # settings, like: host=sql1.host.org host=sql2.host.org > > but i dont find it in > http://wiki2.dovecot.org/AuthDatabase/SQL > > Hi Timo,just to make sure i have an extra dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) connect = host=192.168.123.150 dbname=.. user=... password=... map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } is it possible to have i.e connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... password=... there too? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 10:19:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:19:57 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <4F3B6AAD.7060901@schetterer.org> Am 15.02.2012 09:07, schrieb Adam Szpakowski: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. as workaround it should be possible to calculate filllevel by domain with some bash script and cron sending an info mail to you might be not exact, but enough to see whats going on ( depends on your real domain/mailbox setup...maildir etc) i.e like du -sch schetterer.org/ 1,7G schetterer.org/ or du -sch schetterer.org/* 24K schetterer.org/.... at schetterer.org 431M schetterer.org/... at schetterer.org 1,3G schetterer.org/.... at schetterer.org 1,7G sum perhaps the same works with getting it out from sql > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:37:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:37:02 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B6AAD.7060901@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> <4F3B6AAD.7060901@schetterer.org> Message-ID: <4F3B6EAE.2010201@3a.pl> On 15.02.2012 09:19, Robert Schetterer wrote: > as workaround it should be possible to calculate filllevel > by domain with some bash script and cron sending an info mail to you > might be not exact, but enough to see whats going on ( depends on your > real domain/mailbox setup...maildir etc) [cut] I'm affraid that this is not a viable option. It should be done quite often to achieve "live data" feel and the amount of I/O to do so on storage/filesystem level is staggering. Good enough solution will be to have an additional column in db table with domain for each quota record. This will allow us to do something like this: SELECT domain, sum(bytes) as sum_bytes, sum(messages) as sum_messages FROM used_quota GROUP BY domain The missing domain info can be updated regularly via some cron script. This could be a workable solution. -- Adam Szpakowski From busseniu at in.tum.de Wed Feb 15 10:51:26 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 09:51:26 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B720E.6010705@in.tum.de> Hi, thank you for the reply. On 02/15/2012 03:35 AM, Timo Sirainen wrote: > On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > >> our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: >> >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? At the moment, there are - 1118 imap-login processes, - 1136 fds in anvil according to lsof - 1123 connections to anvil according to its proctitle - 1123 fds in ipc - 1111 connections to ipc > >> I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. > > The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. > Do you see some error message also about "Too many open files", Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files - Twice: config: Error: net_accept() failed: Too many open files - Twice: ipc: Error: net_accept() failed: Too many open files - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped (Dovecot has been restarted once during this hour.) > or why do you think the number of open files was the problem? The default ulimit for open files is 1024 on this Ubuntu system (I should have noted that). The problems (log messages and "doveadm proxy list" error, as described in my last mail) appeared when there were about 1000 connections. I restarted dovecot a couple of times, and each time the problems re-appeared at ~1000 connections. Once I put "ulimit -n 6000" in the init script, the problems disappeared. I am pretty sure that this is what fixed everything for us. Since there have been no log messages about client_limit after we increased the ulimit, I think we can assume that the client_limit has never been the problem. This is why I reported the log message. As the ulimit was 1024 in the beginning, I guess there is no way the connections to anvil or ipc could possibly have reached 100.000. In fact, I never saw such a high number in the proctitle. From my very naive understanding of the source code (master/service-monitor.c): if (service->last_drop_warning + SERVICE_DROP_WARN_INTERVAL_SECS < ioloop_time) { service->last_drop_warning = ioloop_time; limit = service->process_limit > 1 ? service->process_limit : service->client_limit; i_warning("service(%s): %s (%u) reached, " "client connections are being dropped", service->set->name, service->process_limit > 1 ? "process_limit" : "client_limit", limit); } The actual number of open connections is not explicitly compared against client_limit before the warning. Maybe this leads to wrong numbers in the warning? (This part of the code looks similar in hg 2.0 and 2.1., though I have only used 2.0.18.) >> The "doveadm proxy list" would fail with an error message: >> >> $ sudo doveadm proxy list >> doveadm(root): Error: LIST failed: Disconnected >> Segmentation fault >> >> The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. > > http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. Thanks? however I am afraid I have not been able to reproduce the segfault since yesterday. Cheers, Christoph From przemek.orzechowski at makolab.pl Wed Feb 15 11:09:47 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Wed, 15 Feb 2012 10:09:47 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <9b9cf0ba2ada5b598eae8b4fc42e75e8@localhost> Hi Im not an expert in Dovecot but what prohibits you adding a field to the table Dovecot will not use it thats all Im not sure what version of database You are using but postgres,oracle and never versions of mysql have something called views (could be used to hide the additional column from dovecot if necessary) and as far as i remember triggers that could be launched on insert, update to populate that additional column Other solution (not sure if applicable for Your needs) is to enable file system quota without actually setting up any quotas/group quotas (all users from single domain must belong to same group tho) this allows quick disk usage check without excess disk io On Wed, 15 Feb 2012 09:07:04 +0100, Adam Szpakowski wrote: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. -- Pozdrawiam, Przemys?aw Orzechowski Administrator Sieci/Network Administrator e: przemek.orzechowski at makolab.net t: +48 42 683 74 96 MakoLab S.A. ul. Demokratyczna 46, 93-430 ??d? www.makolab.pl Sp??ka zarejestrowana w Krajowym Rejestrze S?dowym przez S?d Rejonowy dla ?odzi - ?r?dmie?cia w ?odzi XX Wydzia? Krajowego Rejestru S?dowego pod numerem KRS 0000289179. Wysoko?? kapita?u zak?adowego wynosi 707 473 PLN. Kapita? zak?adowy zosta? wp?acony w ca?o?ci. NIP 7250015526, REGON 471343117 Wiadomo?? ta jest przeznaczona jedynie dla osoby lub podmiotu b?d?cego jej adresatem i mo?e zawiera? poufne lub uprzywilejowane informacje. Zakazane jest przegl?danie, przesy?anie, rozpowszechnianie lub inne wykorzystywanie tych informacji, jak r?wnie? podejmowanie dzia?a? na ich podstawie, przez osoby lub podmioty inne ni? zamierzony adresat. Je?li otrzymali Pa?stwo t? wiadomo?? przez pomy?k?, prosimy o poinformowanie nadawcy i usuni?cie jej z komputera. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please notify the sender and delete the material from your computer. From busseniu at in.tum.de Wed Feb 15 11:50:15 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 10:50:15 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B7FD7.1000109@in.tum.de> On 15.02.2012 03:37, Timo Sirainen wrote: > On 15.2.2012, at 4.35, Timo Sirainen wrote: > Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Since yesterday all our users (2000 active) are being routed over this proxy, and we have not experienced any performance problems. The system load is below 0.1 most of the time. So afaict there is no reason to use high-performance mode. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 15 12:12:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 11:12:35 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F3B8513.2040806@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. Thanks. I guess it would be wise to upgrade our new Dovecot mail store to 2.1 before we migrate all our Courier maildir users to it... Cheers, Chnristoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Wed Feb 15 12:30:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 12:30:14 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3B720E.6010705@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 10.51, Christoph Bu?enius wrote: >> Do you see some error message also about "Too many open files", > > Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: > > - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files > - Twice: config: Error: net_accept() failed: Too many open files > - Twice: ipc: Error: net_accept() failed: Too many open files > - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. From janfrode at tanso.net Wed Feb 15 13:52:09 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 15 Feb 2012 12:52:09 +0100 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: <20120215115209.GA21047@dibs.tanso.net> On Wed, Feb 15, 2012 at 03:49:21AM +0200, Timo Sirainen wrote: > > > > Looking at the timestamps in the filesystem I see that the users home > > directory wasn't created before switcing to imap. > > > > Is this a know problem? > > Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). > Is it maybe changeset 11683:148fccbe9f32 you remeber: - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX This was just the updated to the todo-list, but I can't see what the fix was. Also, if it was just occationally failing, it might not be that critical.. So far it's only happended for one user for the last 36 hours, so either it's only occationally failing, or the other new users are visiting webmail/imap before pop. -jf From tss at iki.fi Wed Feb 15 15:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:01:05 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120215115209.GA21047@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> <20120215115209.GA21047@dibs.tanso.net> Message-ID: <1329310865.11500.170.camel@innu> On Wed, 2012-02-15 at 12:52 +0100, Jan-Frode Myklebust wrote: > Is it maybe changeset 11683:148fccbe9f32 you remeber: > > - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives > - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX That's probably maildir-specific. But dboxes have had same problems. > This was just the updated to the todo-list, but I can't see what the fix > was. Also, if it was just occationally failing, it might not be that > critical.. So far it's only happended for one user for the last 36 > hours, so either it's only occationally failing, or the other new users > are visiting webmail/imap before pop. Yes, it's most likely only temporary and the second login works. From tss at iki.fi Wed Feb 15 15:05:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:05:38 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <1329311138.11500.175.camel@innu> On Wed, 2012-02-15 at 08:26 +0100, Adam Szpakowski wrote: > On 15.02.2012 02:55, Timo Sirainen wrote: > > On 15.2.2012, at 3.54, Timo Sirainen wrote: > > > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > > Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. Isn't it basically one table scan? To do UPDATE .. WHERE username = '% @domain' > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. Not currently. Perhaps for v2.2 dict-sql redesign. > > Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf .. > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan a problem anyway? Does your domain quota value keep breaking? In normal operation it should work fine. From tss at iki.fi Wed Feb 15 15:07:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:07:11 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3B6873.2010601@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> Message-ID: <1329311231.11500.177.camel@innu> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: > Hi Timo,just to make sure > i have an extra > dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) > > connect = host=192.168.123.150 dbname=.. user=... password=... .. > is it possible to have i.e > > connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... > password=... > > there too? Assuming your SQL database is configured to do synchronous multi-master replication, sure. From r.zimmermann at siegnetz.de Wed Feb 15 16:55:28 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 15:55:28 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 Message-ID: <4F3BC760.7080202@siegnetz.de> Hi all, today I have compiled Dovecot-2.0.18 with Dovecot-Pigeonwhole-0.2.6. Now I have following Problem with sieve: ------------------------%<--------------------------------------------- main_script: line 249: error: included personal script 'vacation.sieve' does not exist. main_script: error: validation failed. ------------------------%<--------------------------------------------- I have a definition for 'sieve_dir' and the sieve script 'vacation.sieve' exits in the folder. # Dovecot Sieve Parameters plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same configuration I can define include :personal "vacation.sieve"; in my main sieve script. All works fine. Greets Ralf From tss at iki.fi Wed Feb 15 17:13:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 17:13:33 +0200 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <96E451B3-17B0-4C57-AAF7-D4D7D0BEF653@iki.fi> On 15.2.2012, at 16.55, Ralf Zimmermann wrote: > main_script: line 249: error: included personal script 'vacation.sieve' > does not exist. .. > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; include :personal "vacation"; From dmiller at amfes.com Wed Feb 15 17:14:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Wed, 15 Feb 2012 07:14:34 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3AFF83.4020707@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> <4F3AC6A2.5010605@amfes.com> <4F3AFF83.4020707@es2eng.com> Message-ID: On 2/14/2012 4:42 PM, Willie Gillespie wrote: > On 02/14/2012 01:40 PM, Daniel L. Miller wrote: >> On 2/14/2012 12:06 PM, Willie Gillespie wrote: >>> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >>>> I'm fighting the same issue. With the current release (2.1rc6) no >>>> errors >>>> are logged - but I'm having problems deleted folders from trash. I've >>>> enabled tb-extra-mailbox-sep. Two folders that were previously in >>>> Trash >>>> were successfully removed. I deleted four more - after several >>>> Thunderbird restarts, compacts, refreshes, I have killed three of >>>> them - >>>> there is still one folder that won't "die". Thunderbird reports the >>>> mailbox doesn't exist, yet it continues to appear. >>> >>> Out of curiosity -- does the mailbox actually exist if you look on the >>> Dovecot server? >>> Secondly, is it listed in your subscription file? >> >> At one time - yes they were. After further playing, they do not exist in >> the on-disk folder structure, nor in the subscription file. But still >> show in the client - with errors. > > Sounds like a Thunderbird bug then. With Thunderbird shut down, you > could browse into the profile. > ~\.thunderbird or > C:\Users\name\AppData\Roaming\Thunderbird\whatever\ImapMail\server.name > > Find the mailbox that is not supposed to be there and move/delete it. > The filename will usually just be the mailbox name. A .msf file is > Thunderbird's index file. It will create .sbd folders for any > sub-mailboxes. I tried going in via squirrelmail - saw no subfolders under trash. Went back in with Thunderbird - all gone. -- Daniel From tlx at leuxner.net Wed Feb 15 17:14:55 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 15 Feb 2012 16:14:55 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <20120215151455.GA1896@nihlus.leuxner.net> On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; > > in my main sieve script. All works fine. Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. Try: include :personal "vacation"; Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.zimmermann at siegnetz.de Wed Feb 15 18:06:07 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 17:06:07 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <20120215151455.GA1896@nihlus.leuxner.net> References: <4F3BC760.7080202@siegnetz.de> <20120215151455.GA1896@nihlus.leuxner.net> Message-ID: <4F3BD7EF.2070204@siegnetz.de> Thanks Thomas and Timo for the fast answer. Next time I'll read the News file ;-) It' works now. On 02/15/2012 04:14 PM, Thomas Leuxner wrote: > On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: >> If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same >> configuration I can define >> >> include :personal "vacation.sieve"; >> >> in my main sieve script. All works fine. > > Changelog v0.2.6: > > * This release fixes unintentional behavior of the include extension. > Included script names with a name like "name.sieve" would implicitly > map to a script file called "name.sieve" and not "name.sieve.sieve". > Keep in mind that the .sieve file extension has no meaning from > within the Sieve language. A Sieve script is always stored with an > appended .sieve file extension, also when the name already ends with > a .sieve suffix. > > Try: > > include :personal "vacation"; > > Regards > Thomas Regards Ralf From robert at schetterer.org Wed Feb 15 18:10:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 17:10:22 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <1329311231.11500.177.camel@innu> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> Message-ID: <4F3BD8EE.5010007@schetterer.org> Am 15.02.2012 14:07, schrieb Timo Sirainen: > On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >> Hi Timo,just to make sure >> i have an extra >> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >> >> connect = host=192.168.123.150 dbname=.. user=... password=... > .. >> is it possible to have i.e >> >> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >> password=... >> >> there too? > > Assuming your SQL database is configured to do synchronous multi-master > replication, sure. > > jep ,thx Timo -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 20:22:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 19:22:21 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BD8EE.5010007@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> Message-ID: <4F3BF7DD.8080302@schetterer.org> Am 15.02.2012 17:10, schrieb Robert Schetterer: > Am 15.02.2012 14:07, schrieb Timo Sirainen: >> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >>> Hi Timo,just to make sure >>> i have an extra >>> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >>> >>> connect = host=192.168.123.150 dbname=.. user=... password=... >> .. >>> is it possible to have i.e >>> >>> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >>> password=... >>> >>> there too? >> >> Assuming your SQL database is configured to do synchronous multi-master >> replication, sure. >> >> > jep ,thx Timo > Hi Timo, sorry for some more question what is the default behave if dict quota over sql cant be reached i.e with lmtp something like log warning and deliver anyway ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Feb 15 20:43:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:43:10 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BF7DD.8080302@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> Message-ID: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> On 15.2.2012, at 20.22, Robert Schetterer wrote: > sorry for some more question > what is the default behave if dict quota over sql cant be reached > i.e with lmtp > > something like log warning and deliver anyway ? I think it tempfails. Try. From tss at iki.fi Wed Feb 15 20:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:51:59 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On 15.2.2012, at 5.08, wrote: > I know you mentioned you would cover this in a coming post, but we were > curious what the new dsync replication will be capable of. Would it > monitor changes to mailboxes and push automatic replication to the remote > mail store, Yes. > and if this is the case could it be an N-way replication setup > in which any host in a cluster can participate in the replication? Initially 2-way, but I don't think anything prevents it being N-way. > Do you consider this to be a high availability solution? The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. The other possibility is to run Dovecot in two completely separate data centers and replicate through ssh. Here are more possibilities for how to do HA, but some of them also have downsides.. dovecot.fi mails are actually done this way, and can be accessed from either server at any time. I've been thinking about soon making half of my clients use one server and half the other one to see if I can find any dsync bugs (I've always 3-4 IMAP clients connected). From tss at iki.fi Wed Feb 15 20:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:54:01 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: <5260D0A9-A72F-48CE-8D42-50A2E818E7BA@iki.fi> On 15.2.2012, at 20.51, Timo Sirainen wrote: > The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages two replicated storages.. ugh. > are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. From moseleymark at gmail.com Wed Feb 15 21:02:42 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 15 Feb 2012 11:02:42 -0800 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On Mon, Feb 13, 2012 at 3:47 AM, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. > > Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: > > ?- 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. > ?- 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) > ?- 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. > Not to beat a dead horse, but the ability to use remote directors might be interesting. It'd make moving into a director setup probably a bit more easy. Then any server could proxy to the backend servers, but without losing the advantage of director-based locality. If a box sees one of its own IPs in the director_servers list, then it knows it's part of the ring. If it doesn't, then it could contact a randomly selected director IP. From tss at iki.fi Wed Feb 15 21:09:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 21:09:18 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <3C05A36D-F6D3-49C6-AEF1-6013E6AA01AF@iki.fi> On 15.2.2012, at 21.02, Mark Moseley wrote: > Not to beat a dead horse, but the ability to use remote directors > might be interesting. It'd make moving into a director setup probably > a bit more easy. Then any server could proxy to the backend servers, > but without losing the advantage of director-based locality. If a box > sees one of its own IPs in the director_servers list, then it knows > it's part of the ring. If it doesn't, then it could contact a randomly > selected director IP. It should already be possible to do that, although not automatically based on looking at your own IP.. Anyway, non-director servers could simply have the passdb return proxy=y host=director-servers, where director-servers expands to a round-robin list of director IPs (Dovecot uses the first one). I guess it would be possible to do this automatically if passdb lookup returns proxy=y but no host (means director isn't enabled), but if director_servers is non-empty one of the IPs would be randomly chosen. A little kludgy though.. From as at 3a.pl Wed Feb 15 21:09:18 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 20:09:18 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <1329311138.11500.175.camel@innu> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <1329311138.11500.175.camel@innu> Message-ID: <4F3C02DE.7080605@3a.pl> On 15.02.2012 14:05, Timo Sirainen wrote: >> Is there a way to add domainname as another collumn in dbase? This will >> simplify the SQL query and the GROUP BY could be used. > Not currently. Perhaps for v2.2 dict-sql redesign. It will be great feature. > Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan > a problem anyway? Does your domain quota value keep breaking? In normal > operation it should work fine. Domain quota gathering based on config examples works "funny". Its the best description I can get (yes, very descriptive ;) ). There are both entries for users (user at domain) and domains. It is not making sense for me, so I'm assuming that its the missing percentage of the working part ;) Tu sum-up the thread. I went for updating periodically the basic quota table with missing domain names. Working great. Thanks all for suggestions. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 21:10:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:10:53 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> Message-ID: <4F3C033D.9000503@schetterer.org> Am 15.02.2012 19:43, schrieb Timo Sirainen: > On 15.2.2012, at 20.22, Robert Schetterer wrote: > >> sorry for some more question >> what is the default behave if dict quota over sql cant be reached >> i.e with lmtp >> >> something like log warning and deliver anyway ? > > I think it tempfails. Try. hm, test servers arent up recently cant do it on production but i will test before i go try dove 2.1 and report, @list perhaps anyone other having it tested before? > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 21:30:46 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 14:30:46 -0500 Subject: [Dovecot] 2.1b6 imapc sync error Message-ID: <1329334246.30459.140661037015245@webmail.messagingengine.com> I received this error at the end of a migration from imapc to mdbox. Is it because I wanted doveadm 'sync' instead of 'backup'? I didn't get time to do any testing, but since you're pushing for the release, here it is: kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o imapc_password=****** -o imapc_host=*.*.*.* sync -R -u jsample_dc at mydomain.com imapc:/tmp/imapc dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2011: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2012: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox INBOX: Not supported ...etc, for all mailboxes in the account. This is my userdb entry: jsample_dc at mydomain.com:{***}password:502:6:IMAPC RECIPIENT::/bin/false: kerio2:~ adminuser$ /opt/dovecot/bin/doveconf -n # 2.1.rc6: /opt/dovecot-2.1.rc6/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_debug = yes auth_mechanisms = plain login auth_username_format = %n at mydomain.com auth_verbose = yes disable_plaintext_auth = no instance_name = dovecot-2.1.rc6 listen = 192.168.20.40 mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox mail_plugin_dir = /opt/dovecot/lib/dovecot mail_plugins = fts fts_squat # I know, this is deprecated in 2.1 mail_privileged_group = mail maildir_very_dirty_syncs = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat } postmaster_address = postmaster at mydomain.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = adminuser } } service imap { executable = imap postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = _postfix mode = 0666 user = _postfix } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_ca = References: <1329334246.30459.140661037015245@webmail.messagingengine.com> Message-ID: <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> On 15.2.2012, at 21.30, Metro Domain Admin wrote: > I received this error at the end of a migration from imapc to mdbox. Is > it because I wanted doveadm 'sync' instead of 'backup'? I didn't get > time to do any testing, but since you're pushing for the release, here > it is: > > kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o > imapc_password=****** -o imapc_host=*.*.*.* sync -R -u > jsample_dc at mydomain.com imapc:/tmp/imapc > dsync(jsample_dc at mydomain.com): Error: Can't update mailbox > Archives/2011: Not supported Yes, it's because of "sync". The "Not supported" comes from imapc backend, because Dovecot tried to do some changes to it that weren't possible with IMAP protocol (modseq changes probably). I think the migration still was successful though. From robert at schetterer.org Wed Feb 15 21:38:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:38:24 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? tests done In-Reply-To: <4F3C033D.9000503@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> <4F3C033D.9000503@schetterer.org> Message-ID: <4F3C09B0.8040500@schetterer.org> Am 15.02.2012 20:10, schrieb Robert Schetterer: > Am 15.02.2012 19:43, schrieb Timo Sirainen: >> On 15.2.2012, at 20.22, Robert Schetterer wrote: >> >>> sorry for some more question >>> what is the default behave if dict quota over sql cant be reached >>> i.e with lmtp >>> >>> something like log warning and deliver anyway ? >> >> I think it tempfails. Try. > > hm, test servers arent up recently > cant do it on production > > but i will test before i go try dove 2.1 > and report, @list perhaps anyone other having it tested before? >> > ok the test server was good enough for small tests with 2.0.13 dove private/dovecot-lmtp] said: 451 4.2.0 <.... at ....com> Internal error occurred. Refer to server log for more information. dict: Error: mysql(localhost2): Connect failed to database (postfix): Unknown MySQL server host '...' (1) - waiting for 125 seconds before retry youre right looks like default is tempfail 451 so it stays in postfix queue and is permanent retry after sql comming back , its getting delivered i.e force with Flush Queue so thats good enough for me i think -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 23:05:52 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 16:05:52 -0500 Subject: [Dovecot] 2.1b6 imapc sync error In-Reply-To: References: <1329334246.30459.140661037015245@webmail.messagingengine.com> <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> <1329338885.26216.140661037021897@webmail.messagingengine.com> Message-ID: <1329339952.472.140661037052961@webmail.messagingengine.com> On Wed, Feb 15, 2012, at 10:50 PM, Timo Sirainen wrote: > On 15.2.2012, at 22.48, Metro Domain Admin wrote: > > > On Wed, Feb 15, 2012, at 09:36 PM, Timo Sirainen wrote: > >> Yes, it's because of "sync". The "Not supported" comes from imapc > >> backend, because Dovecot tried to do some changes to it that weren't > >> possible with IMAP protocol (modseq changes probably). I think the > >> migration still was successful though. > > > > The migration went beautifully otherwise. But are the errors minor > > enough that I can continue with sync? > > Probably better to continue with "backup" just in case. :) > > > I had the idea that I could sync back to the imapc account, as a > > fallback option (and, presumably I'd want to preserve the /tmp/imapc > > directory for this?). > > That's not something that I've tested, and I don't know how well it works > (if at all). > OK, "backup" it is then. Thanks. From interfasys at gmail.com Thu Feb 16 03:12:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 01:12:07 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> Message-ID: <4F3C57E7.8070105@gmail.com> Hello, Here is a more meanigful backtrace. It happens every time I log in, using any imap client. ********************* Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00000000104e6eda in autocreate_iter_next.20443 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x00000000104dd72e in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #4 0x0000000000416351 in list_namespace_mailboxes.7882 (ctx=0x10a3f158) at cmd-list.c:408 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (struct buffer *) 0x10a04290 mutf7_name = (struct buffer *) 0x10a043e0 name = (const __unknown__ *) 0x10a10040 "INBOX" ret = 1 #5 0x000000000041c2ac in cmd_list_continue (cmd=0x10a3f040) at cmd-list.c:823 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000041c1d0 in cmd_list_full (cmd=0x10a3f040, lsub=false) at cmd-list.c:999 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = (const __unknown__ *) 0x10a04268 "INBOX" patterns_strarr = (const __unknown__ * const *) 0x10a3f1e0 str = (struct buffer *) 0x10a041f0 #7 0x000000000041bc80 in cmd_list (cmd=0x10a3f040) at cmd-list.c:1014 No locals. #8 0x0000000000411264 in command_exec (cmd=0x10a3f040) at imap-commands.c:147 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x000000000040df3c in client_command_input (cmd=0x10a3f040) at imap-client.c:673 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" __FUNCTION__ = "client_command_input" #10 0x000000000040de42 in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:740 size = 279029312 #11 0x000000000040d7b5 in client_handle_input (client=0x10a3e840) at imap-client.c:777 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" __FUNCTION__ = "client_handle_input" #12 0x000000000040d35c in client_input (client=0x10a3e840) at imap-client.c:816 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" __FUNCTION__ = "client_input" #13 0x00000000105cdffa in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #14 0x00000000105c1e16 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #15 0x00000000105c1cc4 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #16 0x00000000105c19ff in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x000000000041eda9 in main (argc=1, argv=0x7fffffffec60) at main.c:384 set_roots = {0x4230c0, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ede9 , failure_callback = 0x4190af } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x4230c0, 0x0} ****************** Cheers, Olivier On 13/02/2012 00:49, Timo Sirainen wrote: > On 13.2.2012, at 2.02, interfaSys s?rl wrote: > >> #0 0x00000000105c64e7 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104b4bf6 in mailbox_list_iter_next () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. >> #2 0x0000000000416b99 in cmd_list_continue.6853 () >> No symbol table info available. >> #3 0x000000000042166d in cmd_list_full () > > This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? > > Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. > > From ybhu at hk1.ibm.com Thu Feb 16 05:59:27 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 11:59:27 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: Tried it, found it could trigger NFS issue. The test is on one machine runs the script to lock the dovecot-uidlist, moving mail into cur/new, append one line to dovecot-uidlist, and unlock it. On another different machine, run one script to SELECT, another script to COPY. Found the similar issue in http://dovecot.org/pipermail/dovecot/2012-February/063539.html Whether it means the dovecot LDA also needs to run on the same server with the backend server? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 12:25 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09186.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From finn at kirstineslund.dk Thu Feb 16 03:04:52 2012 From: finn at kirstineslund.dk (Finn B) Date: Thu, 16 Feb 2012 02:04:52 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? Message-ID: <4F3C5634.2010906@kirstineslund.dk> Moi Timo / all. I've been using Dovecot on my Qmailtoaster production mailserver for some years - since 1.0.7 and current Version 2.0.10 - works perfectly. Now I am installing a backupsystem and a VM testsystem (2 boxes) - and no matter how and what Version I am installing on these 2 boxes I get below messages whenever I try to test connection with Telnet (localhost 110 / 995). I have tried to install all Dovecot versions from 2.010 to 2.018 as rpms, yums and as source and compiled it. It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I have done many times before) I have tried to copy all my settings from my current production system (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot 2.0.10) - nothing helps. Now I come to think of it I realize that my production system is intel based and the 2 new ones are AMD based and wonder if this may be the case ? Dovecot.log master: Error: Service(auth): command startup failed, throttling log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) pop3-login: Error: Timeout waiting for handshake from auth server. my pid=1858, input bytes=0 Regards, Finn B From tss at iki.fi Thu Feb 16 06:54:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 06:54:10 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. From ybhu at hk1.ibm.com Thu Feb 16 08:25:09 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 14:25:09 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: Sounds a little tricky. For backend server, director makes sense. But for LDA, it's also suggested to deploy the LDA on the same machine with backend server? Topologically, it's very possible the machine where LDA is running is separated from one of backend server. I will run the similar tests with pure dovecot LDA, instead of ours to see the result. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/16/2012 12:54 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic02130.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Thu Feb 16 08:29:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 08:29:32 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: <4CDB78DC-AF5D-4A70-919D-CE0BAA7D4A41@iki.fi> On 16.2.2012, at 8.25, Andy YB Hu wrote: > Sounds a little tricky. For backend server, director makes sense. But for > LDA, it's also suggested to deploy the LDA on the same machine with backend > server? Topologically, it's very possible the machine where LDA is running > is separated from one of backend server. MTA/Antivirus/Antispam is often in a separate backend server, but it can do the actual mail writing by sending the mail to Dovecot via LMTP via director. From pw at wk-serv.de Thu Feb 16 08:54:12 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 16 Feb 2012 07:54:12 +0100 Subject: [Dovecot] =?utf-8?q?Something_is_strange_-_perhaps_my_AMD_process?= =?utf-8?q?ors_=3F?= In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <79a90b1bae68f2ac4c9f57ed8ca84a99@wk-serv.de> On Thu, 16 Feb 2012 02:04:52 +0100, Finn B wrote: > Now I come to think of it I realize that my production system is > intel > based and the 2 new ones are AMD based and wonder if this may be the > case ? Dovecot 2.0.3 to 2.0.16 works fine on my DualCore Opterons (VMs). Patrick From joshua at hybrid.pl Thu Feb 16 10:41:52 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 09:41:52 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? Message-ID: Hi, I'm using finally dovecot 2.0 after quite a smooth upgrade from 1.0 :) with virtual mailboxes stored in sql database. Sometimes I need to check if a mailbox has been used recently or not. Checking for logins in dovecot logs sometimes means searching through gigabytes of info... I was just wondering if there is any possibility of running another query after successful login - just to fill some extra field like "last_login"? So far my only idea is to replace user/password query with procedure like login(user,password) that would also return a line as required by user/password query. However, some mysql implementations (for example, standard php-mysql) can't handle procedures which return values - so will it work in dovecot? Or maybe some other ideas how to do that? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 11:20:48 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 10:20:48 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: Message-ID: <20120216092048.GA10124@dibs.tanso.net> On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: > > I was just wondering if there is any possibility of running another > query after successful login - just to fill some extra field like > "last_login"? We touch a file in /var/log/activemailaccounts/$username on every successful login trough postlogin scripting: http://wiki2.dovecot.org/PostLoginScripting -jf From joshua at hybrid.pl Thu Feb 16 11:25:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 10:25:11 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting Great, that's exactly what I needed :) Thanks! Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Thu Feb 16 13:18:15 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 12:18:15 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting By the way, is such thing possible for other processess? For example, I'd like to set in mysql table information that mail has been delivered using lmtp. Would something like this work? protocol lmtp { mail_plugins = $mail_plugins sieve executable = lmtp postlmtp } Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 14:18:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 13:18:54 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: <20120216121854.GA13072@dibs.tanso.net> On Thu, Feb 16, 2012 at 12:18:15PM +0100, Jacek Osiecki wrote: > > By the way, is such thing possible for other processess? For > example, I'd like to set in mysql table information that mail has > been delivered using lmtp. Would something like this work? > > protocol lmtp { > mail_plugins = $mail_plugins sieve > executable = lmtp postlmtp > } I've been inquiring the same lately, and unfortunately that's not possible. There's no "login" involved with lmtp, and each lmtp-session can have multiple recipients.. Maybe it can be solved trough a global sieve script? -jf From joshua at hybrid.pl Thu Feb 16 17:35:14 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 16:35:14 +0100 (CET) Subject: [Dovecot] Recalculate quota when quota=dict ? Message-ID: Hi, My current setting for quota is: plugin { #quota = dirsize:User quota #quota = maildir:User quota quota = dict:User quota::proxy::quota #quota = fs:User quota } I have used mb2md.pl and copied result files manually to a desired mail folder - but quota was not recalculated. And it seems, that no matter what I do - the quota usage remains the same. I tried uncommenting line with: quota = maildir:User quota but it did not help. Any other idea how to recalculate quota? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Thu Feb 16 18:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:18:22 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3C57E7.8070105@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> Message-ID: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> On 16.2.2012, at 3.12, interfaSys s?rl wrote: > Here is a more meanigful backtrace. It happens every time I log in, > using any imap client. .. > #0 0x00000000105e0d88 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. From tss at iki.fi Thu Feb 16 18:20:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:20:08 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 16.2.2012, at 17.35, Jacek Osiecki wrote: > quota = dict:User quota::proxy::quota > > Any other idea how to recalculate quota? With v2.x you can simply do: doveadm quota recalc -u username With v1.x you can delete the user's quota row from sql. From tss at iki.fi Thu Feb 16 18:21:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:21:55 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: On 16.2.2012, at 3.04, Finn B wrote: > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. From interfasys at gmail.com Thu Feb 16 19:04:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 17:04:33 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> Message-ID: <4F3D3721.7080203@gmail.com> That patch didn't fix it, but I've now compiled Dovecot without LTO and got even more info :) Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 189 array.h: No such file or directory. in array.h (gdb) bt full #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 ctx = {inboxcase_end = 0x7fffffffe7d0 "\237\004", sep = 64 '@', inboxcase = 248} i = 0 ret = IMAP_MATCH_NO match = IMAP_MATCH_NO #1 0x000000001049a119 in autocreate_iter_autobox (ctx=0x10a0f840, autobox=0x10a0f9a8) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 match = 32767 __FUNCTION__ = "autocreate_iter_autobox" #2 0x000000001049a484 in autocreate_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 info = (const struct mailbox_info *) 0x0 autoboxes = (const struct autocreate_box *) 0x10a0f9a8 autobox = (const struct autocreate_box *) 0x10a0f9a8 count = 1 __FUNCTION__ = "autocreate_iter_next" #3 0x000000001049a561 in mailbox_list_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 info = (const struct mailbox_info *) 0x7fffffffe86c #4 0x000000000040d06a in list_namespace_mailboxes (ctx=0x10a3f158) at array.h:189 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (string_t *) 0x10a04290 mutf7_name = (string_t *) 0x10a043e0 name = 0x10a10040 "INBOX" ret = 1 #5 0x000000000040dc25 in cmd_list_continue (cmd=0x10a3f040) at array.h:189 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000040e3b7 in cmd_list_full (cmd=0x10a3f040, lsub=false) at array.h:189 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = 0x10a04268 "INBOX" patterns_strarr = (const char * const *) 0x10a3f1e0 str = (string_t *) 0x10a041f0 #7 0x000000000040e40c in cmd_list (cmd=0x10a3f040) at array.h:189 No locals. #8 0x00000000004140a0 in command_exec (cmd=0x10a3f040) at imap-commands.c:90 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x00000000004130b9 in client_command_input (cmd=0x10a3f040) at imap-client.c:94 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" #10 0x000000000041337d in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:94 size = 279029312 #11 0x00000000004134c3 in client_handle_input (client=0x10a3e840) at imap-client.c:94 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #12 0x0000000000413639 in client_input (client=0x10a3e840) at imap-client.c:94 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" #13 0x00000000105cf961 in io_loop_call_io (io=0x10a22ec0) at ioloop.c:26 ioloop = (struct ioloop *) 0x10a20040 t_id = 2 #14 0x00000000105d1382 in io_loop_handler_run (ioloop=0x10a20040) at array.h:189 ctx = (struct ioloop_handler_context *) 0x10a21080 events = (struct kevent *) 0x10a27000 event = (const struct kevent *) 0x10a27000 tv = {tv_sec = 1799, tv_usec = 999243} ts = {tv_sec = 1799, tv_nsec = 999243000} io = (struct io_file *) 0x10a22ec0 events_count = 6 ret = 1 i = 0 __FUNCTION__ = "io_loop_handler_run" #15 0x00000000105cfa04 in io_loop_run (ioloop=0x10a20040) at ioloop.c:26 No locals. #16 0x00000000105b3f1c in master_service_run (service=0x10a1a140, callback=0x41f71b ) at master-service.c:74 No locals. #17 0x000000000041fa23 in main (argc=1, argv=0x7fffffffec60) at main.c:68 set_roots = {0x422d80, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41f514 , failure_callback = 0x41f692 } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = -1 On 16/02/2012 16:18, Timo Sirainen wrote: > On 16.2.2012, at 3.12, interfaSys s?rl wrote: > >> Here is a more meanigful backtrace. It happens every time I log in, >> using any imap client. > .. >> #0 0x00000000105e0d88 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. > > Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 > > I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. > > From tss at iki.fi Thu Feb 16 19:06:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:06:59 +0200 Subject: [Dovecot] v2.1.0 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Changes since rc7: FS layout related crashfix, two man pages added. The biggest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * Temporary authentication failures sent to IMAP/POP3 clients now includes the server's hostname and timestamp. This makes it easier to find the error message from logs. * dsync was merged into doveadm. There is still "dsync" symlink pointing to "doveadm", which you can use the old way for now. The preferred ways to run dsync are "doveadm sync" (for old "dsync mirror") and "doveadm backup". + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + IMAP SPECIAL-USE extension to describe mailboxes + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. + SCRAM-SHA-1 authentication mechanism by Florian Zeitz + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + auth: Implemented support for Postfix's "TCP map" sockets for user existence lookups. - listescape plugin works perfectly now From alexwbaule at gmail.com Thu Feb 16 19:12:25 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 16 Feb 2012 15:12:25 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. You say it "old way", when it's change ? in dovecot 2.0.18 is in the old or new way ? I asking because if i update dovecot, i need to update the plugin. Tks ! Em 13 de fevereiro de 2012 18:18, Timo Sirainen escreveu: > On 13.2.2012, at 19.28, Alex Baule wrote: > >> static void emexis_mailbox_allocated(struct mailbox *box) >> { >> ? ?union mailbox_module_context *zbox; >> >> ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); >> ? ?zbox->super = box->v; >> ? ?box->v.open = Mplugin_mailbox_open; >> ? ?box->v.sync_notify = Mplugin_mailbox_sync_notify; >> >> ? ?MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); >> } > > The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > >> struct mailbox_vfuncs *v = box->vlast; >> v->sync_notify = Mplugin_mailbox_sync_notify; >> >> But in this two cases, the Mplugin_mailbox_sync_notify is never >> called... i missing something ?? > > Never called at all? What storage backend are you using as the base? Maildir? > > Anyway, the way you should be calling it is the exact same way quota plugin does: > > void Mplugin_mailbox_allocated(struct mailbox *box) > { > ? ? ? ?struct mailbox_vfuncs *v = box->vlast; > ? ? ? ?union mailbox_module_context *zbox; > > ? ? ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); > ? ? ? ?zbox->super = *v; > ? ? ? ?box->vlast = &zbox->super; > > ? ? ? ?v->open = Mplugin_mailbox_open; > ? ? ? ?v->sync_notify = Mplugin_mailbox_sync_notify; > ? ? ? ?MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); > } > From tss at iki.fi Thu Feb 16 19:15:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:15:45 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <1933A805-19A9-4476-B0C2-B331FC1033F6@iki.fi> On 16.2.2012, at 19.04, interfaSys s?rl wrote: > That patch didn't fix it, but I've now compiled Dovecot without LTO and > got even more info :) What's your doveconf -n output? From tss at iki.fi Thu Feb 16 19:21:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:21:14 +0200 Subject: [Dovecot] v2.1.0 released In-Reply-To: References: Message-ID: On 16.2.2012, at 19.06, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig > > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > Changes since rc7: FS layout related crashfix, two man pages added. Correction: I messed up by not doing a "hg pull -u" before building the v2.1.0 release, so this release is identical to rc7. Still, it's good enough that I won't bother with v2.1.1 yet (the FS layout crashfix doesn't affect any real clients). From tss at iki.fi Thu Feb 16 19:27:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:27:01 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: On 16.2.2012, at 19.12, Alex Baule wrote: > hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. > > You say it "old way", when it's change ? in dovecot 2.0.18 is in the > old or new way ? It changed before v2.0.0 release, so you should use the new way. > I asking because if i update dovecot, i need to update the plugin. It works the same in v2.1. From dluke at geeklair.net Thu Feb 16 19:42:18 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:42:18 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: References: Message-ID: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Looks like there's a problem on Mac OS X 10.5 (ppc): Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I took a cursory look at the source, but didn't notice anything obvious... -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:47:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:47:53 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> Message-ID: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> On 16.2.2012, at 19.42, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > > Looks like there's a problem on Mac OS X 10.5 (ppc): > > Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) > Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > I took a cursory look at the source, but didn't notice anything obvious... Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. Anyway, try if it works with: configure --with-ioloop=select From dluke at geeklair.net Thu Feb 16 19:49:48 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:49:48 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> Message-ID: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> On Feb 16, 2012, at 12:47 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.42, Daniel J. Luke wrote: > >> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >>> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. >> >> >> Looks like there's a problem on Mac OS X 10.5 (ppc): >> >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> I took a cursory look at the source, but didn't notice anything obvious... > > Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. > > Anyway, try if it works with: configure --with-ioloop=select 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). I'll let you know if --with-ioloop=select works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:51:59 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> Message-ID: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). OK, I bet it's the FIFOs then. See if this helps: service stats { fifo_listener stats-mail { mode = 0 } } From tss at iki.fi Thu Feb 16 20:04:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 20:04:30 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 From dluke at geeklair.net Thu Feb 16 20:20:27 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 13:20:27 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> Message-ID: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.49, Daniel J. Luke wrote: > >>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). > > OK, I bet it's the FIFOs then. See if this helps: > > service stats { > fifo_listener stats-mail { > mode = 0 > } > } That works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From bradley.giesbrecht at gmail.com Thu Feb 16 20:31:31 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Thu, 16 Feb 2012 10:31:31 -0800 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> Message-ID: <2F3C7FE5-7D31-4000-A0E4-75B1E1169332@gmail.com> On Feb 16, 2012, at 10:20 AM, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: >> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >> >>>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >>> >>> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). >> >> OK, I bet it's the FIFOs then. See if this helps: >> >> service stats { >> fifo_listener stats-mail { >> mode = 0 >> } >> } > > That works. Cool. I am surprised I missed this thread: http://dovecot.org/pipermail/dovecot/2011-November/062122.html Regards, Bradley Giesbrecht (pixilla) From interfasys at gmail.com Thu Feb 16 20:31:42 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 18:31:42 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <4F3D4B8E.2070103@gmail.com> Yep, works perfectly now. Thank you for fixing it :) On 16 February 2012 18:04:30, Timo Sirainen wrote: > OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 > > From stephan at rename-it.nl Thu Feb 16 23:29:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:29:56 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 Message-ID: <4F3D7554.1010205@rename-it.nl> Hello Dovecot users, Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most significant new features are support for the Sieve editheader extension and the sieve-filter tool. I haven't received much feedback on these new features, which can either mean that they are mostly bug-free or that not many people have tried these, so these are still considered experimental. Changelog v0.3.0: * Renamed sieve_global_path setting to sieve_default for clarity. Old name is still recognized for backwards compatibility. Support for the ancient (pre v1.1) name for this setting "global_script_path" is now dropped. * Added means to prohibit use of redirect action. Setting sieve_max_redirects to 0 now means that redirect is disallowed in stead of unlimited. Default value remains four. * Fixed interaction of Sieve include extension with ManageSieve. It is updated to match new requirements in the draft include specification. Missing included scripts are no longer an error at upload time. * Updated RFC2822 header field body verification to exclude non-printing characters (RFC5322). Only Sieve actions that can create unstructured header values (currently enotify/mailto and editheader) are affected by this change. + Completed sieve-filter tool to a useful state. The sieve-filter tool provides a means to (re)filter messages in a mailbox through a Sieve script. + Implemented the Sieve editheader extension. It is now possible to add and remove message headers from within Sieve. + ManageSieve: added support for reading quoted and literal strings as a stream. Fixes support for handing large SASL responses (analogous to similar changes in Dovecot). It is now also allowed to use a quoted string for the PUTSCRIPT script argument. + Added code to cleanup tmp directory in Sieve storage directory (sieve_dir) every once in a while. + Added support for substituting the entire message during Sieve processing. This is used for the filter action provided by the new sieve_extprograms plugin (provided separately for now). The filter action allows passing the message through an external program. + Added support for restricting certain Sieve language extensions to (admin-controled) global scripts. Restricted extensions can be configured using the new sieve_global_extensions setting. This is particularly useful for some of the Dovecot-specific (plugin-based) Sieve extensions, that can be somewhat hazardous when under direct control of users (e.g. sieve_extprograms). The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From stephan at rename-it.nl Thu Feb 16 23:35:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:35:43 +0100 Subject: [Dovecot] Sieve_extprograms plugin for Pigeonhole v0.3 Message-ID: <4F3D76AF.9060600@rename-it.nl> As indicated in the release announcement of Pigeonhole v0.3.0, there is also a new experimental plugin available for Pigeonhole v0.3: the sieve_extprograms plugin. This can be used to pipe messages to and filter messages through external programs (shell scripts). Also, string data can be retrieved from such external programs. The plugin can currently only be obtained from its Mercurial repository: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms Refer to the INSTALL and README files for more information: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/INSTALL http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/README The vendor-specific Sieve language extension provided by this plugin are documented here: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt Please test this plugin. The plan is to include it into the main Pigeonhole release eventually. Regards, Stephan. From matrixworkstation at gmail.com Thu Feb 16 23:40:32 2012 From: matrixworkstation at gmail.com (aledr) Date: Thu, 16 Feb 2012 19:40:32 -0200 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: <4F3D7554.1010205@rename-it.nl> References: <4F3D7554.1010205@rename-it.nl> Message-ID: Correct links are: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig On Thu, Feb 16, 2012 at 7:29 PM, Stephan Bosch wrote: > Hello Dovecot users, > > Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most > significant new features are support for the Sieve editheader extension and > the sieve-filter tool. I haven't received much feedback on these new > features, which can either mean that they are mostly bug-free or that not > many people have tried these, so these are still considered experimental. > > Changelog v0.3.0: > > ?* Renamed sieve_global_path setting to sieve_default for clarity. Old > ? name is still recognized for backwards compatibility. Support for > ? the ancient (pre v1.1) name for this setting "global_script_path" is > ? now dropped. > ?* Added means to prohibit use of redirect action. Setting > ? sieve_max_redirects to 0 now means that redirect is disallowed in > ? stead of unlimited. Default value remains four. > ?* Fixed interaction of Sieve include extension with ManageSieve. It is > ? updated to match new requirements in the draft include > ? specification. Missing included scripts are no longer an error at > ? upload time. > ?* Updated RFC2822 header field body verification to exclude > ? non-printing characters (RFC5322). Only Sieve actions that can > ? create unstructured header values (currently enotify/mailto and > ? editheader) are affected by this change. > ?+ Completed sieve-filter tool to a useful state. The sieve-filter tool > ? provides a means to (re)filter messages in a mailbox through a Sieve > ? script. > ?+ Implemented the Sieve editheader extension. It is now possible to > ? add and remove message headers from within Sieve. > ?+ ManageSieve: added support for reading quoted and literal strings as > ? a stream. Fixes support for handing large SASL responses (analogous > ? to similar changes in Dovecot). It is now also allowed to use a > ? quoted string for the ?PUTSCRIPT script argument. > ?+ Added code to cleanup tmp directory in Sieve storage directory > ? (sieve_dir) every once in a while. > ?+ Added support for substituting the entire message during Sieve > ? processing. This is used for the filter action provided by the new > ? sieve_extprograms plugin (provided separately for now). The filter > ? action allows passing the message through an external program. > ?+ Added support for restricting certain Sieve language extensions to > ? (admin-controled) global scripts. Restricted extensions can be > ? configured using the new sieve_global_extensions setting. This is > ? particularly useful for some of the Dovecot-specific (plugin-based) > ? Sieve extensions, that can be somewhat hazardous when under direct > ? control of users (e.g. sieve_extprograms). > > The release is available as follows: > > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig > > Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more > information. Have fun testing this new release and don't hesitate to notify > me when there are any problems. > > Regards, > > -- > Stephan Bosch > stephan at rename-it.nl > > > > > > > > > > > -- [ ]'s Aledr - Alexandre www.exatati.com.br "OpenSource Solutions for SmallBusiness Problems" From list at airstreamcomm.net Thu Feb 16 23:48:40 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 16 Feb 2012 15:48:40 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On Wed, 15 Feb 2012 20:51:59 +0200, Timo Sirainen wrote: > On 15.2.2012, at 5.08, > wrote: > >> I know you mentioned you would cover this in a coming post, but we were >> curious what the new dsync replication will be capable of. Would it >> monitor changes to mailboxes and push automatic replication to the remote >> mail store, > > Yes. > >> and if this is the case could it be an N-way replication setup >> in which any host in a cluster can participate in the replication? > > Initially 2-way, but I don't think anything prevents it being N-way. > >> Do you consider this to be a high availability solution? > > > The initial version is really about doing all of this with NFS. In NFS > setup if two replaced storages are both mounted and the primary storage > dies, Dovecot will start using the replica. So that's HA. > > The other possibility is to run Dovecot in two completely separate data > centers and replicate through ssh. Here are more possibilities for how to > do HA, but some of them also have downsides.. dovecot.fi mails are actually > done this way, and can be accessed from either server at any time. I've > been thinking about soon making half of my clients use one server and half > the other one to see if I can find any dsync bugs (I've always 3-4 IMAP > clients connected). Just to throw our thoughts into the mix, finding an open source multi-site active/active mail solution that does not require building super expensive multi-site storage systems would be a really refreshing way to purse this level of availability. Maybe the only way to accurately get this level of availability is to cluster the storage between sites? From stephan at rename-it.nl Fri Feb 17 00:03:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 23:03:07 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: References: <4F3D7554.1010205@rename-it.nl> Message-ID: <4F3D7D1B.6030404@rename-it.nl> On 2/16/2012 10:40 PM, aledr wrote: > Correct links are: > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Oops :) Regards, Stephan. From mstevens at imt-systems.com Fri Feb 17 00:27:46 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Thu, 16 Feb 2012 23:27:46 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 Message-ID: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Hi, First, Timo, Thank you for Dovecot 2.1.0. I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for RHEL6, CentOS 6 and Scientific Linux 6. If anyone is interested in testing these RPMs: http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ Feedback would be appreciated. After some testing, we will use these RPMs on our IMAP servers. Best regards, Morten From przemek.orzechowski at makolab.pl Fri Feb 17 00:39:17 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Thu, 16 Feb 2012 23:39:17 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x Message-ID: <220331515776550936318f562808607f@localhost> Hi I have already read this page but there is no info on how to delete old mails in dovecot 1.2 http://wiki1.dovecot.org/QuestionsAndAnswers#How_can_I_create_a_cron_or_something_to_delete_old_messages.3F I need to delete old mails from over 100 mailboxes with average of 10k mails / mailbox Is there a way to delete old mails (by message date not file creation date) as with doveadm in dovecot 2 ? Thanks in advance bellow my dovecot -n (not sure if it is neccessary or not so just in case) # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/makolab/star_makolab_pl.crt ssl_key_file: /etc/ssl/makolab/star_makolab_pl.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: passdb: driver: pam userdb: driver: passwd plugin: quota: fs:Limit uzytkownika:user quota2: fs:Limit domeny:group From tss at iki.fi Fri Feb 17 01:25:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:25:18 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 12.30, Timo Sirainen wrote: >>> Do you see some error message also about "Too many open files", >> >> Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: >> >> - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files >> - Twice: config: Error: net_accept() failed: Too many open files >> - Twice: ipc: Error: net_accept() failed: Too many open files >> - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped >> - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > > Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. Actually this solves it nicely: http://hg.dovecot.org/dovecot-2.1/rev/701709f472e9 It simply keeps giving net_accept() failures. From tss at iki.fi Fri Feb 17 01:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:28:28 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <220331515776550936318f562808607f@localhost> References: <220331515776550936318f562808607f@localhost> Message-ID: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > I need to delete old mails from over 100 mailboxes with average of 10k > mails / mailbox > Is there a way to delete old mails (by message date not file creation > date) as with doveadm in dovecot 2 ? You can delete them by mtime or by ctime: http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent If by "message date" you mean the Date: header, then there's no easy way. From przemek.orzechowski at makolab.pl Fri Feb 17 01:42:22 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Fri, 17 Feb 2012 00:42:22 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <6f78990c40e3c083ec43dca148edfe0d@localhost> On Fri, 17 Feb 2012 01:28:28 +0200, Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > >> I need to delete old mails from over 100 mailboxes with average of 10k >> mails / mailbox >> Is there a way to delete old mails (by message date not file creation >> date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy way. Yes Date header unfortunatly (seems i will have to write some kind of script for that) Just to be sure just deleting mail files is safe? Or do i have to run some additional commands to recreate dovecots index files? I'm using Maildir. Asking just to be sure that I will not screw up something. From slitt at troubleshooters.com Fri Feb 17 01:52:31 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 18:52:31 -0500 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <20120216185231.1eed027c@mydesk> On Fri, 17 Feb 2012 01:28:28 +0200 Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > > > I need to delete old mails from over 100 mailboxes with average of > > 10k mails / mailbox > > Is there a way to delete old mails (by message date not file > > creation date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy > way. > Would something like the following work for the date header? #!/bin/bash for msg in *; do if b4.py $msg 20080216; then rm $msg fi done #!/usr/bin/python datestring = get_dateline() ## FIND DATE HDR AND RETURN THE STRING yyyymmdd = format2yyyymmdd(datestring) ## CONVERT TO YYYYMMDD if yyyymmdd < sys.argv[1]: sys.exit(0) else: sys.exit(1) In the preceding, would the deletions mess up maildir indices? Thanks SteveT From tss at iki.fi Fri Feb 17 01:53:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:53:39 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <20120216185231.1eed027c@mydesk> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> <20120216185231.1eed027c@mydesk> Message-ID: <66CFB9DE-1A67-4EDF-B777-E482AF1226AF@iki.fi> On 17.2.2012, at 1.52, Steve Litt wrote: > In the preceding, would the deletions mess up maildir indices? no From ml at smtp.fakessh.eu Fri Feb 17 02:41:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 17 Feb 2012 01:41:06 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: Le 2012-02-16 23:27, Morten Stevens a ?crit?: > Hi, > > First, Timo, Thank you for Dovecot 2.1.0. > > I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for > RHEL6, CentOS 6 and Scientific Linux 6. > > If anyone is interested in testing these RPMs: > http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ > > Feedback would be appreciated. > > After some testing, we will use these RPMs on our IMAP servers. > > Best regards, > > Morten is there any source rpm packages for centos 5 ? best regards -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From finn at kirstineslund.dk Fri Feb 17 03:07:45 2012 From: finn at kirstineslund.dk (Finn B) Date: Fri, 17 Feb 2012 02:07:45 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3DA861.7060902@kirstineslund.dk> Moi Timo /all. Well I have looked everywhere to find any other clues - I cann't find any traces from this incident. But I have spend a lot of time figuring out what is going on so I may becom e'blind' for all the obvious so let me know if more can be found- though I have inserted more lines from Dovecot.log. Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:08 pop3-login: Error: Timeout waiting for handshake from auth serve r. my pid=29388, input bytes=0 Feb 15 01:41:38 pop3-login: Info: Disconnected: Inactivity (no auth attempts): r ip=127.0.0.1, lip=127.0.0.1, secured Feb 15 01:41:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 log: Error: service(auth): child 29450 returned error 89 (Fatal failure) Feb 15 01:42:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:43:08 imap-login: Error: Timeout waiting for handshake from auth serve r. my pid=29448, input bytes=0 This is my dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir raw SQL drivers: Passdb: checkpassword pam passwd passwd-file shadow vpopmail Userdb: checkpassword nss passwd prefetch passwd-file vpopmail and here is my Dovecot.conf's (this one is from QMT install )- further below is my Dovecot.conf taken from my production system and also used without more luck # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.18.1.el5 i686 CentOS release 5.7 (Final) auth_cache_size = 32 M auth_username_format = %Lu first_valid_uid = 89 listen = *,:: log_path = /var/log/dovecot.log login_greeting = Dovecot toaster ready. namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = cache_key=%u webmail=127.0.0.1 driver = vpopmail } plugin { quota = maildir } protocols = imap pop3 ssl_cert = method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = 89 mail_home = mail_location = maildir:~/Maildir mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = 89 mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = webmail=127.0.0.1 deny = no driver = vpopmail master = no pass = no } plugin/quota = maildir:ignore=Trash plugin/quota_rule = ?:storage=0 pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = root vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 0 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 0 } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 ssl = no } inet_listener pop3s { address = port = 995 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = On 16.2.2012, at 3.04, Finn B wrote: > >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. > > From tss at iki.fi Fri Feb 17 03:11:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 03:11:31 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3DA861.7060902@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> <4F3DA861.7060902@kirstineslund.dk> Message-ID: On 17.2.2012, at 3.07, Finn B wrote: > Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed This is your problem. Your vpopmail is somehow broken. I can't really help any further than that, since I haven't used vpopmail myself. From dovecot at r.paypc.com Fri Feb 17 04:46:23 2012 From: dovecot at r.paypc.com (Robin) Date: Thu, 16 Feb 2012 18:46:23 -0800 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> Message-ID: <4F3DBF7F.4030504@r.paypc.com> > You mean you deleted Solr index, so that it's empty? That should work too. > > Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: plugin { fts = solr fts_solr = break-imap-search url=http://solrhost:8983/solr/ } Should I be adding fts/fts_solr to the global mail_plugins setting? I have Solr up and running, without any firewalling between the hosts, and it never seems to even try to use it. The logs show, even after importing fresh mail and issuing a "search text "your" command to the server, which takes it about 5m or so to return results. I see the following in the log: Feb 16 17:51:54 indexer-worker(testuser): Info: Indexed 0 messages in INBOX2010 GET /solr/ issued to http://solrhost:8983/ via TELNET reports A-OK, and Solr Admin shows ready status when the admin console is loaded into a web browser. I can see there is ZERO traffic between the hosts during the SEARCH text command's execution, though I can see an open connection to the solr host in netstat: tcp 0 0 linuxcode:56393 solrhost:8983 ESTABLISHED =R= From slitt at troubleshooters.com Fri Feb 17 05:20:42 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 22:20:42 -0500 Subject: [Dovecot] Favorable mention in Linux Productivity Magazine Message-ID: <20120216222042.68a5a58c@mydesk> Hi all, The latest Linux Productivity Magazine, http://www.troubleshooters.com/lpm/201202/201202.htm, discusses the positive role Dovecot played in my migration away from Kmail. This is currently topical as Kmail is in the midst to transforming to Kmail2, which requires an Akonadi database and is not popular. I hope you enjoy the magazine. SteveT Steve Litt Troubleshooters.Com From klimenko.n at theitidea.ru Fri Feb 17 07:21:06 2012 From: klimenko.n at theitidea.ru (=?UTF-8?B?0JrQu9C40LzQtdC90LrQviDQndC40LrQvtC70LDQuQ==?=) Date: Fri, 17 Feb 2012 09:21:06 +0400 Subject: [Dovecot] Dovecot lda parameters Message-ID: <4F3DE3C2.1060308@theitidea.ru> hi all I have the following scheme. Postfix with virtual users with dovecot LDA. AD as database. Dovecot and Postfix get the following Idap attributes from AD: sAMAccountName, mail and mail_alias. The problem is that I can?t make dovecot lda move the user?s mail to the folder with the name from sAMAccountName attribute, as mail always moves to folder with e-mail address name. Is it posible to delivery e-mail to folder with name from ldap attributes sAMAccountName? -- From joshua at hybrid.pl Fri Feb 17 11:51:20 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 10:51:20 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: W dniu 2012-02-16 17:20, Timo Sirainen napisa?(a): > On 16.2.2012, at 17.35, Jacek Osiecki wrote: >> Any other idea how to recalculate quota? > With v2.x you can simply do: doveadm quota recalc -u username Great, That's exactly what I needed :) Sorry, it was lame - I should have found it in wiki2... By the way: what might have caused such a warning? root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Greetings, -- Jacek Osiecki From mstevens at imt-systems.com Fri Feb 17 14:40:17 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Fri, 17 Feb 2012 13:40:17 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: On 17.02.2012 01:41, ml wrote: > is there any source rpm packages for centos 5 ? No, only for CentOS6/RHEL6. Otherwise, look here: http://packages.atrpms.net/dist/el5/dovecot-2.1.x/ Best regards, Morten From joshua at hybrid.pl Fri Feb 17 15:40:32 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 14:40:32 +0100 Subject: [Dovecot] Quota warnings and LMTP Message-ID: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Hello, I have dovecot 2.0 configured with virtual users in MySQL, SMTP is handled by exim which uses dovecot LMTP transport. I'd like to test quota warning messages, but in wiki2 there is only an example for delivering it with dovecot-lda. I tried several ways of using lmtp binary, but without success. How to replace this: cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" From: postmaster at domain.com Subject: quota warning Your mailbox is now $PERCENT% full. EOF with lmtp? Greetings, -- Jacek Osiecki From as at 3a.pl Fri Feb 17 16:17:31 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 17 Feb 2012 15:17:31 +0100 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <4F3E617B.6050801@3a.pl> On 17.02.2012 14:40, joshua at hybrid.pl wrote: > Hello, Hi, > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o > "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? As far as I known, there is no specific config for LMTP transport when quota is considered. When testing quota messages, the most problems for me was with the fact, that the warning is triggered only when the limit is reached. Only once for each limit. The script we are using: #!/bin/bash PERCENT=$1 USER=$2 DOMAIN=$3 cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::noenforcing:proxy::quota" From: postmaster@$DOMAIN Subject: ostrze?enie o przekroczeniu wolnego miejsca / quota warning Twa skrzynka jest zape?niona w $PERCENT%. Your mailbox is now $PERCENT% full. EOF -- Adam Szpakowski From sanito at progresas.lt Fri Feb 17 17:39:53 2012 From: sanito at progresas.lt (sanito at progresas.lt) Date: Fri, 17 Feb 2012 17:39:53 +0200 (EET) Subject: [Dovecot] user_filter problem Message-ID: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Hello, I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 with user information stored in the LDAP database. The main problem is that email address and pop3 username are different for every mailbox (I cannot change this behaviour, because thousands of people already configured their email client for such a setup). So in my situation, user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it should be: user_filter=(cn=%u). Please, see: E-mail address: john at example.com pop3 username: 0005a pop3 password: very_secure_password And ldap ldiff for this mailbox: dn: cn=0005a,dc=example.com,ou=localmail,ou=mail uid: 0005a mail: john at example.com mailbox: /data/mail/example.com/0005a/Maildir/ sn: 0005a description: testing homeDirectory: /data/mail/example.com/0005a cn: 0005a userPassword:: e0NSWVBUfWxpOXBVaEw5SWJkRWc= objectClass: inetOrgPerson objectClass: CourierMailAccount objectClass: top I think I need to use different user_filter for lmtp and pop3, but it's not possible or I cannot find how to do it. Please, help! -- Best regards, dammit From marc at perkel.com Fri Feb 17 19:09:16 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 09:09:16 -0800 Subject: [Dovecot] 2.1 upgrade docs? Message-ID: <4F3E89BC.3070107@perkel.com> Just wondering if there's any upgrade docs for 2.1 online? From e-frog at gmx.de Fri Feb 17 19:17:03 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 17 Feb 2012 18:17:03 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E89BC.3070107@perkel.com> References: <4F3E89BC.3070107@perkel.com> Message-ID: <4F3E8B8F.1000208@gmx.de> On 17.02.2012 18:09, wrote Marc Perkel: > Just wondering if there's any upgrade docs for 2.1 online? http://wiki2.dovecot.org/Upgrading/2.1 From micah at riseup.net Fri Feb 17 19:17:53 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 12:17:53 -0500 Subject: [Dovecot] 2.1 upgrade docs? References: <4F3E89BC.3070107@perkel.com> Message-ID: <877gzlxttq.fsf@algae.riseup.net> Marc Perkel writes: > Just wondering if there's any upgrade docs for 2.1 online? yes there are. (hint: search terms "dovecot upgrade") From micah at riseup.net Fri Feb 17 20:25:44 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 13:25:44 -0500 Subject: [Dovecot] expunging Message-ID: <87r4xtwc47.fsf@algae.riseup.net> I am having trouble scaling some regular dovecot cleanup operations on our servers. On a daily basis, I'm wanting to do this on each server, which contains its own isolated set of user storage: /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d but these are a very expensive operations. For example, just doing the Spam expunge takes 30 minutes (or more, depending on the load) of heavy disk operations, on each machine it is run on. There are approximately 20k users on each machine. It seems like it does not use the iterate query, but rather it looks into the database at the expires table and rather it iterates over every user mentioned there. This is a problem because I've got multiple dovecot machines with different sets of users on them, using the same table, so that means that its doing stat() calls on each system for every user mentioned there and failing to find the user on the filesystem (because the user is on another system). In these cases it spits out an error: doveadm(user): Info: User no longer exists, skipping For users that do exist on this system, it seems to do something on the order of 15 stat() calls, at minimum. What are some things I can do to make this less of an expensive process? If I had a shared storage system that each machine used, this would cut down on the resource waste because I'd only need to run the query once, but unfortuntely, that isn't how these systems were designed. thanks for any ideas, tips etc. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From frank at moltke28.B.Shuttle.DE Fri Feb 17 20:48:17 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 17 Feb 2012 19:48:17 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? Message-ID: Hi all, are there any pitfalls known for the migration from 2.0.18 to 2.1? Can I simply stop dovecot, make install and start again? --Frank Elsner From marc at perkel.com Fri Feb 17 20:51:45 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 10:51:45 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error Message-ID: <4F3EA1C1.3080903@perkel.com> Not sure what this means: Warning: fd limit (ulimit -n) is lower than required under max. load (4096 < 20000), because of service auth { client_limit } What do I need to do? From marc at perkel.com Fri Feb 17 21:14:04 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 11:14:04 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error In-Reply-To: <4F3EA1C1.3080903@perkel.com> References: <4F3EA1C1.3080903@perkel.com> Message-ID: <4F3EA6FC.1000902@perkel.com> Never mind - I figured it out. ulimit -n 20000 On 2/17/2012 10:51 AM, Marc Perkel wrote: > Not sure what this means: > > Warning: fd limit (ulimit -n) is lower than required under max. load > (4096 < 20000), because of service auth { client_limit } > > What do I need to do? > > From eric at yorba.org Sat Feb 18 03:54:51 2012 From: eric at yorba.org (Eric Gregory) Date: Fri, 17 Feb 2012 17:54:51 -0800 Subject: [Dovecot] Order of operations in Dovecot Message-ID: Hi, I'm working on an IMAP client and had a question about the order in which commands are responded to in Dovecot. In other IMAP servers I've looked at, when you issue two commands simultaneously (i.e. pipelining) it looks something like this: a fetch something b fetch something else * 1 FETCH (result) a OK FETCH completed * 1 FETCH (result) b OK FETCH completed This works well because it's clear which results are associated with which tag; the 'a' fetch results immediately precede the a's OK response, and likewise the 'b' fetch results immediately precede the b's OK response. But in Dovecot (version 2.0.13) the results of the two simultaneous commands are ordered like this: a fetch something b fetch something else * 1 FETCH (result) * 1 FETCH (result) a OK FETCH completed b OK FETCH completed This makes associating the results of the fetch commands quite a bit more difficult, since it's not entirely clear which results are paired with which tag. My question is this: is this Dovecot's intended behavior, or did I find a bug? I realize the IMAP spec is really vague about this. If this is the intended behavior, is there any advice for clients on how to determine which results are associated with which command? Thanks! - Eric From jk at jkart.de Sat Feb 18 15:21:23 2012 From: jk at jkart.de (Jim Knuth) Date: Sat, 18 Feb 2012 14:21:23 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: References: Message-ID: <4F3FA5D3.8050101@jkart.de> am 17.02.12 19:48 schrieb Frank Elsner : > > Hi all, > > are there any pitfalls known for the migration from 2.0.18 to 2.1? > > Can I simply stop dovecot, make install and start again? THIS also interests me. > > --Frank Elsner > > > > > > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Alle M?nner hatten schon immer Angst davor, da? die Frauen ohne sie zurechtkommen k?nnten. [Margaret Mead] From tlx at leuxner.net Sat Feb 18 15:37:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 18 Feb 2012 14:37:20 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F3FA5D3.8050101@jkart.de> References: <4F3FA5D3.8050101@jkart.de> Message-ID: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> >> are there any pitfalls known for the migration from 2.0.18 to 2.1? http://wiki2.dovecot.org/Upgrading/2.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roliverio at vaslibre.org.ve Sat Feb 18 18:30:20 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:00:20 -0430 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3FD21C.6030305@vaslibre.org.ve> > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > Could you check how it's vpopmail connecting? Looks like a SSL related thing with it. HTH, -- ---------------------------------- Julio C. Ortega - VaSLibre http://www.vaslibre.org.ve Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Sat Feb 18 18:48:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:48:21 +0200 Subject: [Dovecot] Order of operations in Dovecot In-Reply-To: References: Message-ID: <4359C586-3C71-4B98-AE72-CF7BE97D2519@iki.fi> On 18.2.2012, at 3.54, Eric Gregory wrote: > But in Dovecot (version 2.0.13) the results of the two simultaneous > commands are ordered like this: > > a fetch something > b fetch something else > * 1 FETCH (result) > * 1 FETCH (result) > a OK FETCH completed > b OK FETCH completed > > This makes associating the results of the fetch commands quite a bit more > difficult, since it's not entirely clear which results are paired with > which tag. > > My question is this: is this Dovecot's intended behavior, or did I find a > bug? I realize the IMAP spec is really vague about this. If this is the > intended behavior, is there any advice for clients on how to determine > which results are associated with which command? This is intended behavior. And if you issue two long FETCH commands, their replies may even be intertwined. IMAP protocol allows all kinds of variations for this actually (e.g. it would be valid for a server to return each FETCHed field in its own untagged FETCH reply). You're not supposed to care about matching FETCH replies to FETCH commands, you just need to know that you've received all the data you asked for (after all tagged commands have replied). http://imapwiki.org/ClientImplementation From tss at iki.fi Sat Feb 18 18:51:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:51:33 +0200 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <02578EB7-C380-40E8-9D4F-79EBCBD79ECE@iki.fi> On 17.2.2012, at 15.40, joshua at hybrid.pl wrote: > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? This command is only about adding the warning mail to user's mailbox, bypassing any quota. Just because you deliver mails normally via LMTP doesn't mean that you have to do it for warning mails. This same dovecot-lda command should work just fine. From tss at iki.fi Sat Feb 18 18:52:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:52:57 +0200 Subject: [Dovecot] expunging In-Reply-To: <87r4xtwc47.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> Message-ID: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> On 17.2.2012, at 20.25, Micah Anderson wrote: > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > It seems like it does not use the iterate query, but rather it looks > into the database at the expires table and rather it iterates over every > user mentioned there. This is a problem because I've got multiple > dovecot machines with different sets of users on them, using the same > table, so that means that its doing stat() calls on each system for > every user mentioned there and failing to find the user on the > filesystem (because the user is on another system). In these cases it > spits out an error: doveadm(user): Info: User no longer exists, skipping Upgrade to v2.1: * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. From campbell at cnpapers.com Sat Feb 18 19:04:49 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 12:04:49 -0500 Subject: [Dovecot] I'm confused about my namespace parms Message-ID: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Hi, I'm new to the list and to dovecot. I'm having a strange situation that provides no errors, so I'm having a little problem diagnosing what's going on. I've been running a Centos 3 mail server with sendmail for our company for quite a few years. I finally got approval for new servers this year and put Centos 6.2 on it. I'm sticking with sendmail, but that's not important. Up to this point, I've got all working fairly well. I've found that if I configure dovecot to use the mixed format (pop and imap INBOX in /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the examples for "namespace", most works fine. Anyone who has used imap previously on our old system and work as if nothing has changed. Anyone who has never used Imap before can't seem to access the server using our webmail app (Horde/Imp). Dovecot is providing both pop and imap. An account that is set up from a normal mail client as imap has no problems as well. Sendmail is set up to create a new user in the /var/spool/mail folder. When I moved from the old server to the new server, I copied /home and /var/spool/mail to the new server - in other words, I kept the same scheme. There are no errors in the maillog, and it indicates that the user is getting logged in and out, but the return to Horde is that the user entered the wrong name or password. I'm thinking that this is probably due to how dovecot is trying to look for the imap folders, but the lack or any errors makes it a little difficult to diagnose. I'm fairly certain that what was installed on the old server was UW-Imap and was provided by the imap-2002 rpm. I tried using the examples for UW-Imap compatability but that didn't work at all. I haven't been able to figure out what's different about the old imap folder contents and what's missing in the newly created imap folders to allow Horde to access this account. I'll look at the Horde config to see if anything is there, but I'm cautious about changing anything that might break the current users' ability. Most of the accounts are set up as POP with the ability to see what's left on the server using Horde, webmail, and IMAP. Any help in tracing this down would really help. I think my biggest problem is I don't understand the "namespace" parameter as well as I should. So there it is. It seems I've got at least 3 areas to concentrate on. Steve Campbell ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From Ralf.Hildebrandt at charite.de Sat Feb 18 19:18:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:18:24 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? Message-ID: <20120218171824.GG8161@charite.de> Today I encountered this after upgrading: Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4104a) [0xb75cc04a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x410bb) [0xb75cc0bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb75cc9b7] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8052bac] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_list_full+0x4af) [0x80530df] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_lsub+0x17) [0x8053537] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](command_exec+0x32) [0x8057c92] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056c7d] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056d95] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_handle_input+0x135) [0x8056fa5] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_input+0x60) [0x80578a0] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x44) [0xb75dafb4] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xce) [0xb75dbe5e] -> /usr/local/lib/dovecot/libdovecot.so.0 a) It should not PANIC (I think) b) How can I globally "fix" the folder names (preferably in such a way that I am able to switch back and forth between 2.0 and 2.1) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From roliverio at vaslibre.org.ve Sat Feb 18 19:17:43 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:47:43 -0430 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <1329584689.4f3fda31e5952@perdition.cnpapers.net> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Message-ID: <4F3FDD37.2000903@vaslibre.org.ve> El 18/02/12 12:34, Steve Campbell escribi?: > Hi, > > I'm new to the list and to dovecot. I'm having a strange situation that provides > no errors, so I'm having a little problem diagnosing what's going on. > > I've been running a Centos 3 mail server with sendmail for our company for quite > a few years. I finally got approval for new servers this year and put Centos 6.2 > on it. I'm sticking with sendmail, but that's not important. > > Up to this point, I've got all working fairly well. I've found that if I > configure dovecot to use the mixed format (pop and imap INBOX in > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the > examples for "namespace", most works fine. Anyone who has used imap previously > on our old system and work as if nothing has changed. Anyone who has never used > Imap before can't seem to access the server using our webmail app (Horde/Imp). > Dovecot is providing both pop and imap. An account that is set up from a normal > mail client as imap has no problems as well. > > Sendmail is set up to create a new user in the /var/spool/mail folder. When I > moved from the old server to the new server, I copied /home and /var/spool/mail > to the new server - in other words, I kept the same scheme. > > There are no errors in the maillog, and it indicates that the user is getting > logged in and out, but the return to Horde is that the user entered the wrong > name or password. I'm thinking that this is probably due to how dovecot is > trying to look for the imap folders, but the lack or any errors makes it a > little difficult to diagnose. > Looks like a Mail Directory seek problem. what's your mail_location set up to?, also, does this location exist for that NEW user on it's first login? -- ---------------------------------- Julio C. Ortega - VaSLibre Usuario GNU/Linux Valencia - Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Sat Feb 18 19:25:14 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:25:14 +0100 Subject: [Dovecot] Move templates args to override_fields setting? Message-ID: <20120218172514.GH8161@charite.de> After upgrading from 2.0.x to 2.1.x I get a lot of: Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting OK, it's just a warning, but how do I fix that in a 2.0.x compatible way? The only place userdb is used is in this contaxt: # for user*masteruser logins passdb { args = /usr/local/etc/dovecot/dovecot.masteruser driver = passwd-file master = yes } # User via passwd userdb { driver = passwd args = cache_key=%u } -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Sat Feb 18 19:34:14 2012 From: micah at riseup.net (micah anderson) Date: Sat, 18 Feb 2012 12:34:14 -0500 Subject: [Dovecot] expunging In-Reply-To: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> Message-ID: <87obswvyeh.fsf@algae.riseup.net> On Sat, 18 Feb 2012 18:52:57 +0200, Timo Sirainen wrote: > On 17.2.2012, at 20.25, Micah Anderson wrote: > > > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > > > It seems like it does not use the iterate query, but rather it looks > > into the database at the expires table and rather it iterates over every > > user mentioned there. This is a problem because I've got multiple > > dovecot machines with different sets of users on them, using the same > > table, so that means that its doing stat() calls on each system for > > every user mentioned there and failing to find the user on the > > filesystem (because the user is on another system). In these cases it > > spits out an error: doveadm(user): Info: User no longer exists, skipping > > Upgrade to v2.1: > > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. Great, i'll give that version a spin, thanks! Is there a similar dict row cleanup for quota? micah From tss at iki.fi Sat Feb 18 19:48:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:48:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Today I encountered this after upgrading: > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe .. > a) It should not PANIC (I think) Right. > b) How can I globally "fix" the folder names (preferably in such a way > that I am able to switch back and forth between 2.0 and 2.1) What does this look in filesystem? Is it not in mUTF-7? If the problem is in filesystem, rename it to mUTF7 (you can use doveadm mailbox mutf7) From tss at iki.fi Sat Feb 18 19:53:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:53:22 +0200 Subject: [Dovecot] expunging In-Reply-To: <87obswvyeh.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> <87obswvyeh.fsf@algae.riseup.net> Message-ID: On 18.2.2012, at 19.34, micah anderson wrote: >> * Expire plugin: Only go through users listed by userdb iteration. >> Delete dict rows for nonexistent users, unless >> expire_keep_nonexistent_users=yes. > > Great, i'll give that version a spin, thanks! > > Is there a similar dict row cleanup for quota? No. If they're in SQL you should be able to build an SQL query for that.. From tss at iki.fi Sat Feb 18 19:56:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:56:15 +0200 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20120218172514.GH8161@charite.de> References: <20120218172514.GH8161@charite.de> Message-ID: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> On 18.2.2012, at 19.25, Ralf Hildebrandt wrote: > After upgrading from 2.0.x to 2.1.x I get a lot of: > Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting > > OK, it's just a warning, but how do I fix that in a 2.0.x compatible > way? .. > # User via passwd > userdb { > driver = passwd > args = cache_key=%u > } Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. From Ralf.Hildebrandt at charite.de Sat Feb 18 20:15:15 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:15:15 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> Message-ID: <20120218181515.GI8161@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Today I encountered this after upgrading: > > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > .. > > a) It should not PANIC (I think) > > Right. > > > b) How can I globally "fix" the folder names (preferably in such a way > > that I am able to switch back and forth between 2.0 and 2.1) > > What does this look in filesystem? Is it not in mUTF-7? If the problem > is in filesystem, rename it to mUTF7 (you can use doveadm mailbox > mutf7) # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe /home/h/e/hermel/Maildir/.Entw&APw-rfe /home/h/l/hlammert/Maildir/.Entw&APw-rfe ... /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Sat Feb 18 20:19:01 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:19:01 +0100 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> References: <20120218172514.GH8161@charite.de> <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> Message-ID: <20120218181901.GJ8161@charite.de> * Timo Sirainen : > > # User via passwd > > userdb { > > driver = passwd > > args = cache_key=%u > > } > > Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. Removed. Thanks! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From trashcan at odo.in-berlin.de Sat Feb 18 20:26:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 18 Feb 2012 19:26:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror Message-ID: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Hi -- I'm used to dsync my two mailservers running 2.0.16. Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Same error with old running command: vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here's an excerpt of my doveconf -n: mail> doveconf -n mail_home = /var/mail/homedirs/%n mail_location = mdbox:/var/mail/maildirs/%n passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } Reverted back for the time being (production servers) until I do understand the issue. Searching Google with "Fatal: All your namespaces have a location setting." only shows closed bugs back in 2010. Do I have to upgrade both servers to get syncing running? Any hint, help is highly appreciated. Thanks, Michael From Ralf.Hildebrandt at charite.de Sat Feb 18 20:49:33 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:49:33 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218181515.GI8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> Message-ID: <20120218184933.GM8161@charite.de> * Ralf Hildebrandt : > # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* > /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe > /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe > /home/h/e/hermel/Maildir/.Entw&APw-rfe > /home/h/l/hlammert/Maildir/.Entw&APw-rfe > ... > /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe > /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe > /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe Some stats: # Name =================== 2 .Entw&-APw-rfe 9 .Entw&,,0-rfe 91 .Entw&APw-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 20:59:39 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 19:59:39 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218184933.GM8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: Am 18. Februar 2012 19:49 schrieb Ralf Hildebrandt : > * Ralf Hildebrandt : > >> # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* >> /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe >> /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe >> /home/h/e/hermel/Maildir/.Entw&APw-rfe >> /home/h/l/hlammert/Maildir/.Entw&APw-rfe >> ... >> /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe >> /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe >> /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe > > Some stats: > > ? ? ?# ?Name > ? ?=================== > ? ? ?2 .Entw&-APw-rfe > ? ? ?9 .Entw&,,0-rfe > ? ? 91 .Entw&APw-rfe > Use something like "Entwuerfe" and not "Entw?rfe". -- Chris From Ralf.Hildebrandt at charite.de Sat Feb 18 21:04:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:04:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <20120218190443.GN8161@charite.de> * Chris : > > Some stats: > > > > ? ? ?# ?Name > > ? ?=================== > > ? ? ?2 .Entw&-APw-rfe > > ? ? ?9 .Entw&,,0-rfe > > ? ? 91 .Entw&APw-rfe > > > > Use something like "Entwuerfe" and not "Entw?rfe". Haha. Tell that to my users. By default, everybody gets a "Drafts" folder anyway. I'm not sure which program insists on creating "Entw?rfe" -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at luigirosa.com Sat Feb 18 21:13:13 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:13:13 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218190443.GN8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> Message-ID: <4F3FF849.8060605@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:04: > I'm not sure which program insists on creating "Entw?rfe" Outlook? ;) Ciao, luigi - -- / +--[Luigi Rosa]-- \ It is apparent to me that the possibilities of the aeroplane, which two or three years ago were thought to hold the solution to the [flying machine] problem, have been exhausted, and that we must turn elsewhere. --Thomas Edison, 1895 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8/+EUACgkQ3kWu7Tfl6ZSbogCgvIReVrSzSLg8LhtyW6d4usqh vYUAn3Br/ACxznjJ9lH0yQ5lXwLPkWvJ =OMT5 -----END PGP SIGNATURE----- From Ralf.Hildebrandt at charite.de Sat Feb 18 21:15:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:15:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: <20120218191544.GO8161@charite.de> > > I'm not sure which program insists on creating "Entw?rfe" > > Outlook? ;) Isn't it always? I hate this shit. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 21:16:03 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 20:16:03 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: 2012/2/18 Luigi Rosa : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ralf Hildebrandt said the following on 18/02/12 20:04: > >> I'm not sure which program insists on creating "Entw?rfe" > > > Outlook? ? ?;) Yes... M$ Outlook crap do this :) -- Chris From lists at luigirosa.com Sat Feb 18 21:42:45 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:42:45 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218191544.GO8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> <20120218191544.GO8161@charite.de> Message-ID: <4F3FFF35.4000600@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:15: >>> I'm not sure which program insists on creating "Entw?rfe" >> Outlook? ;) > Isn't it always? I hate this shit. Giulty by suspicion. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Airplanes are interesting toys but of no military value. --Marechal Ferdinand Foch, Professor of Strategy, Ecole Superieure de Guerre, 1904 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8//zUACgkQ3kWu7Tfl6ZRZJgCgpZJr3p0D39ssa6glFlFnH4/X WjMAnRapaPUmf9szE1iC5YajT9/oDpXq =bmTJ -----END PGP SIGNATURE----- From lists at necoro.eu Sat Feb 18 22:39:50 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Sat, 18 Feb 2012 21:39:50 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <4F400C96.4000505@necoro.eu> Am 18.02.2012 19:59, schrieb Chris: > Use something like "Entwuerfe" and not "Entw?rfe". Come on ... software having problems with non-ASCII-stuff in 2012 is just broken by design. Hence there should be no reason to use 'ue' instead of ? anywhere. - Ren? (P.S.: The intention of this mail was not to blame dovecot at all -- because at least until now, umlauts work exceptionally good here.) > -- > Chris > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From xchris89x at googlemail.com Sat Feb 18 23:31:55 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 22:31:55 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F400C96.4000505@necoro.eu> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <4F400C96.4000505@necoro.eu> Message-ID: 2012/2/18 Ren? Neumann : > Am 18.02.2012 19:59, schrieb Chris: >> Use something like "Entwuerfe" and not "Entw?rfe". > > Come on ... software having problems with non-ASCII-stuff in 2012 is > just broken by design. Hence there should be no reason to use 'ue' > instead of ? anywhere. Dovecot has no problems with non-ASCII stuff. The german language is broken by design ... ? ? ? ?... :-) -- Chris From campbell at cnpapers.com Sun Feb 19 02:35:18 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 19:35:18 -0500 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <4F3FDD37.2000903@vaslibre.org.ve> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> <4F3FDD37.2000903@vaslibre.org.ve> Message-ID: <1329611718.4f4043c6a9c6f@perdition.cnpapers.net> Quoting "Julio C. Ortega" : > El 18/02/12 12:34, Steve Campbell escribi?: > > Hi, > > > > I'm new to the list and to dovecot. I'm having a strange situation that > provides > > no errors, so I'm having a little problem diagnosing what's going on. > > > > I've been running a Centos 3 mail server with sendmail for our company for > quite > > a few years. I finally got approval for new servers this year and put > Centos 6.2 > > on it. I'm sticking with sendmail, but that's not important. > > > > Up to this point, I've got all working fairly well. I've found that if I > > configure dovecot to use the mixed format (pop and imap INBOX in > > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in > the > > examples for "namespace", most works fine. Anyone who has used imap > previously > > on our old system and work as if nothing has changed. Anyone who has never > used > > Imap before can't seem to access the server using our webmail app > (Horde/Imp). > > Dovecot is providing both pop and imap. An account that is set up from a > normal > > mail client as imap has no problems as well. > > > > Sendmail is set up to create a new user in the /var/spool/mail folder. When > I > > moved from the old server to the new server, I copied /home and > /var/spool/mail > > to the new server - in other words, I kept the same scheme. > > > > There are no errors in the maillog, and it indicates that the user is > getting > > logged in and out, but the return to Horde is that the user entered the > wrong > > name or password. I'm thinking that this is probably due to how dovecot is > > trying to look for the imap folders, but the lack or any errors makes it a > > little difficult to diagnose. > > > > Looks like a Mail Directory seek problem. what's your mail_location set > up to?, also, does this location exist for that NEW user on it's first > login? > > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > Usuario GNU/Linux > Valencia - Venezuela > > > My 10-mail.conf has the following: mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u and my namespaces are as follows: namespace { type = private separator = / prefix = "#mbox/" location = mbox:~/mail:INBOX=/var/mail/%u inbox = yes hidden = yes list = no } namespace { type = private separator = / prefix = location = maildir:~/mail } As I stated, I need to brush up on what all this really means. So far, reading the documentation has cleared up much for me. I'm sure it'll make sense later, but for now, I'm a little hazy. This machine was tested last weekend, and I had problems with reading any email. I'm using the examples found in the wiki and the above worked. Our old server is so anemic that we're getting delays of multiple hours for delivery after the server accepts it, so this weekend, I left the new server up and running. Thanks for any help and the replies. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From alessio at skye.it Sun Feb 19 12:39:02 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 19 Feb 2012 11:39:02 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL Message-ID: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Hi, I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" and SASL for postfix's SMTP-Auth, like explained ih this howto: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL Authentications over IMAP (local dovecot -> remote dovecot) works fine, and it's great. Also smtp-auth on postfix (over SASL) works fine. But in the dovecot log I found this error: Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): child 30506 killed with signal 11 (core dumps disabled) even though all works fine. This is my dovecot configuration: # 2.1.0: dovecot.conf-imapc # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_mechanisms = plain login default_login_user = nobody imapc_host = 10.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=10.0.0.1 default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl = no userdb { driver = prefetch } But if instead of "unix_listener /var/spool/postfix/private/auth" I use: service auth { unix_listener auth-userdb { } inet_listener { port = 12345 } } no errors appears in the log. In both cases all works fine. Is there any errors in my dovecot's configuration? -- Alessio Cecchi is: on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi Linux -> http://www.cecchi.biz/ Hosting Linux -> http://www.hosting-linux.it @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From mstevens at imt-systems.com Sun Feb 19 14:11:08 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 19 Feb 2012 13:11:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: On 19.02.2012 11:39, Alessio Cecchi wrote: > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: > service(auth): child 30506 killed with signal 11 (core dumps > disabled) signal 11 = segmentation fault That's probably a dovecot bug. Is it possible to reproduce this signal 11 crash? See: http://dovecot.org/bugreport.html Best regards, Morten From stephan at rename-it.nl Sun Feb 19 14:21:44 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 19 Feb 2012 13:21:44 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. Message-ID: <4F40E958.1090102@rename-it.nl> Hi, For our testing enthusiasts I've created new automatic Debian repositories for Dovecot v2.2. Repositories are available for Debian oldstable, stable and testing, on both i386 and am64. Read the following wiki item for more information: http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages The repository for Dovecot v1.2 has been defunct for a few months now and is expected to be removed soon. For the coming month or so, the builders at Xi will not be active during Dutch daytime. There are some issues with the virtual host's I/O performance that need to be resolved first. Regards, Stephan. From postmaster at mathiesen.info Sun Feb 19 14:53:25 2012 From: postmaster at mathiesen.info (postmaster) Date: Sun, 19 Feb 2012 13:53:25 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F40F0C5.2010100@mathiesen.info> I'm completely stumped by this problem and I've tried everything at present - including installing dovecot from scratch. The problem is that when trying to get email from the server -AND- there's email in the account, the pop3 or imap process freezes and no email is fetched. The crazy thing is, that it has been working perfectly, but suddenly I'm getting these freezes both when using telnet from localhost, roundcubemail on the same server and Seamonkey from another computer. Logging seems to work perfectly. For an account without email, dovecot finds the maildir++ and acts correctly. For an account -with- email, dovecot seems to find the maildir++, but also seems to try Namespace of which I haven't defined any. I've tried to get rawlog to work also, but that didn't work - the telnet process froze on login even though I'm quite sure I did the right thing and ps showed an active rawlog process Postfix is delivering the email straight into the mailboxes. Here is the information 4604ds1-ynoe:~ root# port installed | grep dovecot dovecot2 @2.0.18_0+mysql5+postgresql91 (active) dovecot2-sieve @0.2.5_1 (active) postfix @2.9.0_0+dovecot_sasl+mysql5+pcre+postgresql91 (active) 4604ds1-ynoe:~ root# 4604ds1-ynoe:log root# cat dovecot.log.txt Feb 19 11:46:52 4604ds1-ynoe dovecot[441]: master: Dovecot v2.0.18 starting up (core dumps disabled) Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=495) Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49180 resp=AGFidXNlQG1hdGhpZXNlbi5pbmZvAHRlc3R0ZXN0 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: mysql(/macports/var/run/mysql5/mysqld.sock): Connected to database infomathiesenpostfixadmin Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=abuse at mathiesen.info Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 738852865 495 1 687658e957c0e4f473891548eecc49be Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 738852865 abuse at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=503, secured Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/storage) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, alt= Feb 19 11:53:01 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Disconnected: Logged out bytes=45/750 Feb 19 11:53:06 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=512) Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49182 resp=AG1hY2ludG9zaEBtYXRoaWVzZW4uaW5mbwB0ZXN0dGVzdA== Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=macintosh at mathiesen.info Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 1866334209 512 1 332714ca2a070d7adb342d4baba4a19a Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 1866334209 macintosh at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=522, secured Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/storage) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, alt= Feb 19 11:54:13 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info: mode=0700 gid=-1 Feb 19 12:12:16 4604ds1-ynoe dovecot[441]: master: Error: service(imap): child 522 killed with signal 3 Feb 19 12:15:48 4604ds1-ynoe dovecot[441]: master: Warning: Killed with signal 15 (by pid=1086 uid=0 code=unknown 0) 4604ds1-ynoe:log root# 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 441 1 441 56804e0 0 Ss ?? 0:00.01 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 442 441 441 56804e0 0 S ?? 0:00.01 dovecot/anvil root 443 441 441 56804e0 0 S ?? 0:00.01 dovecot/log 5000 522 441 441 56804e0 0 S ?? 0:00.01 dovecot/imap root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 543 380 542 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# kill -QUIT 522 4604ds1-ynoe:mail root# dovecot stop 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 1091 380 1090 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "abuse at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1326334461] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a OK [READ-WRITE] Select completed. a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "macintosh at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox Connection closed by foreign host. 4604ds1-ynoe:~ root# 4604ds1-ynoe:~ root# dovecot -n # 2.0.18: /macports/etc/dovecot/dovecot.conf # OS: Darwin 10.8.0 i386 hfs auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /macports/var/run/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 mail_debug = yes mail_gid = 5000 mail_location = maildir:/Volumes/BrugerData/PostFix/mail/%d/%u mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 service postlogin { executable = script-login -d rawlog } ssl = no syslog_facility = local2 userdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol pop3 { pop3_client_workarounds = oe-ns-eoh outlook-no-nuls } 4604ds1-ynoe:~ root# From robert at schetterer.org Sun Feb 19 17:30:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 19 Feb 2012 16:30:37 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. In-Reply-To: <4F40E958.1090102@rename-it.nl> References: <4F40E958.1090102@rename-it.nl> Message-ID: <4F41159D.6000803@schetterer.org> Am 19.02.2012 13:21, schrieb Stephan Bosch: > Hi, > > For our testing enthusiasts I've created new automatic Debian > repositories for Dovecot v2.2. Repositories are available for Debian > oldstable, stable and testing, on both i386 and am64. Read the following > wiki item for more information: > > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > > The repository for Dovecot v1.2 has been defunct for a few months now > and is expected to be removed soon. > > For the coming month or so, the builders at Xi will not be active during > Dutch daytime. There are some issues with the virtual host's I/O > performance that need to be resolved first. > > Regards, > > Stephan. Hi Stephan, anyway, thx for keeping care of the rep ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From rob0 at gmx.co.uk Sun Feb 19 20:09:27 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 19 Feb 2012 12:09:27 -0600 Subject: [Dovecot] A Postfix/Dovecot example with SQLite backend [crossposted] Message-ID: <20120219180926.GB14195@harrier.slackbuilds.org> There are many mail howtos on the web ... can one more hurt? http://rob0.nodns4.us/howto/README http://rob0.nodns4.us/howto/ http://rob0.nodns4.us/howto/latest.tar.gz (all files) (Sorry, not HTML yet. That is on the agenda.) This is a multiple address class sample implementation of a Postfix MTA and Dovecot IMAP server using a SQLite3 data backend. Domain lookups, user maps, access and transport maps: all using a single, shared SQLite database file. What, other than the SQLite backend, distinguishes this from other mail system howtos? The Postfix high points include a complete implementation of all address classes and per-address transport(5) maps, virtual(8) UID/GID maps, and smtpd(8) recipient access(5) maps. (The latter is using smtpd_restriction_classes, which are not discussed in detail, but are implemented in an interesting way.) On the Dovecot side, it's mostly standard stuff. The SQL deny userdb implementation, and the seamless integration of system and SQL users, might be interesting. I think the database itself is the best part of this example. It's as close to "normalized" as I think it can reasonably be. A significant fact is that each revision of the system has tended to simplify the schema. That's a good sign, I think. One central Domain table lists all domains and hostnames to which the server makes reference. Likewise, a central Address table lists all addresses (with a pointer to the Domain table for each record.) The Alias table defines relationships between Address entries. (Both local(8) and virtual(5) alias maps exist in that table.) Comments and suggestions are welcome, on-list if it's topical to whichever list (please don't crosspost unless comments are relevant to both lists), or offlist to the address in the README file (or as detailed below.) Thanks for your interest. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From trashcan at odo.in-berlin.de Sun Feb 19 23:14:21 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 19 Feb 2012 22:14:21 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <622C2EAD-4DEA-4603-9C56-94CA87A2FC06@odo.in-berlin.de> Hi -- On 18.02.2012, at 19:26, Michael Grimm wrote: > Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: > > mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. [...] > Do I have to upgrade both servers to get syncing running? JFTR: 1) Upgrading both servers ends up in identical error messages. 2) After some investigation I came across a hidden upgrade info in [1], but 'doveadm dsync-server' doesn't run either :-( I'm stuck, thus I did upgrade to 2.0.18 instead ... Regards, Michael [1] http://dovecot.org/list/dovecot/2012-January/062933.html From finn_buhelt at privat.dk Mon Feb 20 01:19:17 2012 From: finn_buhelt at privat.dk (Finn_Buhelt privat) Date: Mon, 20 Feb 2012 00:19:17 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3FD21C.6030305@vaslibre.org.ve> References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <4F418375.7080609@privat.dk> Hi Julio. I'm not sure what You're asking. Can You specify and I will provide the information. I'm so puzzled since this is easy to install and I have done so before without many issues. I think I'd better get the source and go through it . Cheers, Finn B On 02/18/2012 05:30 PM, Julio C. Ortega wrote: >> Dovecot.log >> >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) >> pop3-login: Error: Timeout waiting for handshake from auth server. my >> pid=1858, input bytes=0 >> >> >> Regards, >> Finn B >> > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > From achekalin at lazurit.com Mon Feb 20 08:57:15 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 20 Feb 2012 09:57:15 +0300 Subject: [Dovecot] Homedir vs locations vs mail_location? Message-ID: <4F41EECB.2070405@lazurit.com> Hello, I use Dovecot for rather long time, but I'm still in doubt for some small things. Here they are: 1. The homedir value points to the place where everything for the user stored at, while mail_location is something (some place) where mail stored at. if I deal with pure virtual users (all users are in sql tables and no system homes for them at all), should I ever care for returning meaningful value for 'homedir' (via password_query's userdb_home), or I can simple return empty or constant ('' or '123') for it and it won't mess anything? 2. If I use single (default) namespace, should I set namespace's location (to the same value as global mail_location), and should I expect anything strange if I skip it to set? Reversely, is it possible not to set global mail_location and set only namespace's location (which would be more logical as namespace definition is compact and easy to find in config)? I've reread my questions and see it is dummy ones, but I'd like to know that for sure. Thank you in advance, Alexander From alessio at skye.it Mon Feb 20 09:26:25 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 20 Feb 2012 08:26:25 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F41F5A1.30401@skye.it> Il 16/02/2012 02:04, Finn B ha scritto: > Moi Timo / all. > > I've been using Dovecot on my Qmailtoaster production mailserver for > some years - since 1.0.7 and current Version 2.0.10 - works perfectly. > > Now I am installing a backupsystem and a VM testsystem (2 boxes) - and > no matter how and what Version I am installing on these 2 boxes I get > below messages whenever I try to test connection with Telnet (localhost > 110 / 995). > I have tried to install all Dovecot versions from 2.010 to 2.018 as > rpms, yums and as source and compiled it. > > It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I > have done many times before) > I have tried to copy all my settings from my current production system > (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot > 2.0.10) - nothing helps. > > Now I come to think of it I realize that my production system is intel > based and the 2 new ones are AMD based and wonder if this may be the case ? > > > > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > > > Hi, can you check if your vpopmail is build with "--disable-auth-logging" option? If yes first rebuild vpopmail without this options, rebuild dovecot and check again. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Mon Feb 20 11:24:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 20 Feb 2012 10:24:53 +0100 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <4F41EECB.2070405@lazurit.com> References: <4F41EECB.2070405@lazurit.com> Message-ID: <20120220092453.GA6316@dibs.tanso.net> On Mon, Feb 20, 2012 at 09:57:15AM +0300, Alexander Chekalin wrote: > > 1. The homedir value points to the place where everything for the > user stored at, while mail_location is something (some place) where > mail stored at. if I deal with pure virtual users (all users are in > sql tables and no system homes for them at all), should I ever care > for returning meaningful value for 'homedir' (via password_query's > userdb_home), or I can simple return empty or constant ('' or '123') > for it and it won't mess anything? Dovecot will store non-mailfiles in the homedir. F.ex. quota-files, sieve scripts, subscription file, .dovecot-lda.dupes, and probably more. So do yourself a favour and create a real homedir for each user :-) http://wiki2.dovecot.org/VirtualUsers/Home > > 2. If I use single (default) namespace, should I set namespace's > location (to the same value as global mail_location), and should I > expect anything strange if I skip it to set? Reversely, is it > possible not to set global mail_location and set only namespace's > location (which would be more logical as namespace definition is > compact and easy to find in config)? > We have a single namespace, with blank location: namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } But I don't really know the purpose of this location field vs. mail_location. -jf From aydin.demirel at endersys.com Mon Feb 20 11:29:43 2012 From: aydin.demirel at endersys.com (=?UTF-8?B?QXlkxLFuIERlbWlyZWw=?=) Date: Mon, 20 Feb 2012 11:29:43 +0200 Subject: [Dovecot] Feature request for maillog plugin Message-ID: <4F421287.3070101@endersys.com> Hi, Is it possible to add the modification time and file name of the mail file to maillog plugin? Thanks -- *Ayd?n Demirel Endersys Ltd. Sistem Destek M?hendisi/ System Support Engineer* * *<> Phone : +90 216 470 9423 | GSM : +90 530 401 8203 Fax : +90 216 470 9508 | Web : http://www.endersys.com Blog : http://blog.endersys.com Twitter : http://www.twitter.com/endersys LPI : The #1 Linux Certification for IT Professionals LPI (Linux Professional Institute) Turkey http://www.lpi-turkey.com From l.chelchowski at eurocar.pl Mon Feb 20 13:59:52 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 20 Feb 2012 12:59:52 +0100 Subject: [Dovecot] virtual plugin and shared namespace Message-ID: <4F4235B8.6090107@eurocar.pl> Hi, I have a problem with virtual plugin and geting mails form shared namespace. dovecot-virtual: ---- Shared/* Public/* * unseen ---- dovecot.conf: ---- namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/%d/%u/control/public:INDEX=/var/mail/vmail/%d/%u/index/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace { list = children location = maildir:%%h/:INDEX=/var/mail/vmail/%d/%u/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children prefix = Virtual/ separator = / location = virtual:~/virtual } ---- When I enter the virtual mailbox Unread I get this debug info: virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ Is it a bug or misconfig? -- ?ukasz Che?chowski From mhlavink at redhat.com Mon Feb 20 18:11:14 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 20 Feb 2012 17:11:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E8B8F.1000208@gmx.de> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> Message-ID: <4F4270A2.2040005@redhat.com> On 02/17/2012 06:17 PM, e-frog wrote: > On 17.02.2012 18:09, wrote Marc Perkel: >> Just wondering if there's any upgrade docs for 2.1 online? > > http://wiki2.dovecot.org/Upgrading/2.1 > I know about one Fedora user who had to also change: namespace { .... inbox=yes ... } to namespace inbox { .... and also in sieve (for pigeonhole): change include :personal "00-script1.sieve"; to include :personal "00-script1"; (I did not test dovecot 2.1, I still have 2.0.x on my server) From busseniu at in.tum.de Mon Feb 20 18:17:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 20 Feb 2012 17:17:28 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F427218.80709@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > On 13.2.2012, at 16.40, Timo Sirainen wrote: >> It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. > > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. After replacing "doveadm/dsync" with "dsync", the patch applied in 2.0.18 and works fine. (Is there any chance this will be in a 2.0 bugfix release?) Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From jcblanco at fi.upm.es Mon Feb 20 18:43:18 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Mon, 20 Feb 2012 17:43:18 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: <4F427826.3050506@fi.upm.es> On 20/02/2012 17:11, Michal Hlavinka wrote: > On 02/17/2012 06:17 PM, e-frog wrote: >> On 17.02.2012 18:09, wrote Marc Perkel: >>> Just wondering if there's any upgrade docs for 2.1 online? >> >> http://wiki2.dovecot.org/Upgrading/2.1 >> > > I know about one Fedora user who had to also change: > namespace { > .... > inbox=yes > ... > } > > to namespace inbox { .... > > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; > > (I did not test dovecot 2.1, I still have 2.0.x on my server) I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like user_attrs=....,ldap-attribute=quota_rule=*:storage that worked fine on version 2.0, will cause an unlimited quota for the user in version 2.1, you must adjust your configuration to look like user_attrs=....,ldap-attribute=quota_rule=*:storage=%$ as advised in the dovecot wiki http://wiki2.dovecot.org/Quota/Configuration#LDAP Regards Juan C. Blanco From metro_domain_admin at fastmail.fm Mon Feb 20 19:30:11 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Mon, 20 Feb 2012 12:30:11 -0500 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> Message-ID: <4F428323.8050404@fastmail.fm> On 2/18/12 8:37 AM, Thomas Leuxner wrote: >> are there any pitfalls known for the migration from 2.0.18 to 2.1? > http://wiki2.dovecot.org/Upgrading/2.1 Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html From campbell at cnpapers.com Mon Feb 20 20:10:10 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:10:10 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion Message-ID: <4F428C82.4010700@cnpapers.com> The more I read about all of this, the more I'm thinking about moving to maildir format. My switchover this weekend is full of holes due to the way user's imap folders were laid out. Some had folders in their home directory and others might have folders in their /home/mail directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is not doing so well. I've been reading about the conversion technique to convert to maildir, but I'm wondering if I can do this based on the way our users are set up. Most are pop accounts. Our webmail is configured to read the inbox from /var/spool/mail and their imap folders from /home or /home/mail. Some have imap accounts on their desktop defined through their mail client. RH/Centos seems to want me to switch to postfix since most of the documentation I find doesn't mention Sendmail. My filesystem is laid out to handle Inboxes in /var/spool/mail and imap folders in /home(/mail) and these aren't logical volumes but true partitions, so I've got to consider this as I make any changes. I also run pop using dovecot, so I'll need to figure all this out. I've read the technique linked to on the dovecot site that points to "Replacing UW-IMAP with Dovecot on RHEL 3" written back in 2004, and that's about the only real pointers I'm finding. I don't want to have to go around to each machine and change their clients to IMAP.. Anyone care to comment on anything similar they've done and how many pitfalls they ran into and what they were? steve campbell From CMarcus at Media-Brokers.com Mon Feb 20 20:28:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:28:49 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F4290E1.1020000@Media-Brokers.com> On 2012-02-20 1:10 PM, Steve Campbell wrote: > Our webmail is configured to read the inbox from /var/spool/mail and > their imap folders from /home or /home/mail. I can't help with your specific problem, but I do know that having mail stored directly in /home will cause problems - you will need to fix that. -- Best regards, Charles From campbell at cnpapers.com Mon Feb 20 20:34:11 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:34:11 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F4290E1.1020000@Media-Brokers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> Message-ID: <4F429223.1040202@cnpapers.com> On 2/20/2012 1:28 PM, Charles Marcus wrote: > On 2012-02-20 1:10 PM, Steve Campbell wrote: >> Our webmail is configured to read the inbox from /var/spool/mail and >> their imap folders from /home or /home/mail. > > I can't help with your specific problem, but I do know that having > mail stored directly in /home will cause problems - you will need to > fix that. That was a type. It was supposed to be ~ (user's home) or ~/mail. Thanks anyway for pointing this out. steve From CMarcus at Media-Brokers.com Mon Feb 20 20:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:45:01 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F429223.1040202@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> <4F429223.1040202@cnpapers.com> Message-ID: <4F4294AD.4020505@Media-Brokers.com> On 2012-02-20 1:34 PM, Steve Campbell wrote: > > > On 2/20/2012 1:28 PM, Charles Marcus wrote: >> On 2012-02-20 1:10 PM, Steve Campbell wrote: >>> Our webmail is configured to read the inbox from /var/spool/mail and >>> their imap folders from /home or /home/mail. >> >> I can't help with your specific problem, but I do know that having >> mail stored directly in /home will cause problems - you will need to >> fix that. > > That was a type. It was supposed to be ~ (user's home) or ~/mail. Right - that's what I meant... you do not want to ever store mail directly in the users home directory (~)... -- Best regards, Charles From mstevens at imt-systems.com Mon Feb 20 21:38:20 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 20 Feb 2012 20:38:20 +0100 Subject: [Dovecot] =?utf-8?q?2=2E1_upgrade_docs=3F?= In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: On 20.02.2012 17:11, Michal Hlavinka wrote: > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; Hi Michal, See pigeonhole changelog: v0.2.6 13-02-2012 Stephan Bosch * This release fixes unintentional behavior of the include extension. Included scriptnames with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. Best regards, Morten From jtl+dovecot at uvm.edu Mon Feb 20 21:40:29 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Mon, 20 Feb 2012 14:40:29 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F42A1AD.2000304@uvm.edu> On 2/20/12 1:10 PM, Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving > to maildir format. My switchover this weekend is full of holes due to > the way user's imap folders were laid out. Some had folders in their > home directory and others might have folders in their /home/mail > directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is > not doing so well. Not sure what problem maildir will solve for you. Large mbox mailboxes under UW-IMAP are an awful bottleneck and will bring the most powerful storage subsystems I've used to their knees, but add Dovecot indexing on top and you will be surprised how much faster everything is. We're still using mbox today and if we were to change it would be to mdbox, not maildir. > I've been reading about the conversion technique to convert to > maildir, but I'm wondering if I can do this based on the way our users > are set up. Most are pop accounts. Our webmail is configured to read > the inbox from /var/spool/mail and their imap folders from /home or > /home/mail. Some have imap accounts on their desktop defined through > their mail client. When we did our conversion from uw-imap to dovecot (about 30k accounts) we had to do a lot of clean-up work. Mail folders in the homedir needed to be moved to ~/mail. This can be scripted: if you're using mbox format, look for the first line of the file starting with "From ". (note the space) We use the uw-imap backwards compatibility option documented at http://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_UW-IMAP which allows users who have specified "mail/" as their IMAP namespace prefix to see the same message folders as people who don't. This works most of the time, but we have seen some clients (old versions of Horde/IMP) which still try to create a mail/ folder, which ends up creating ~/mail/mail, and that becomes inaccessible. Fortunately, since moving to Horde 4/IMP 5 it hasn't been a problem. You might want to keep an eye out for this if you go the backwards compatibility route. We also kept (and still keep) inboxes in /var/spool/mail (which is actually a link tree, pointing to other filesystems.) We had enough users that we needed to do "shard" directories to prevent directory lock contention from bringing everything to a crawl, e.g. jsmith's mail is in /var/spool/mail/j/s/jsmith. Dovecot nicely supports this as a config option. > > RH/Centos seems to want me to switch to postfix since most of the > documentation I find doesn't mention Sendmail. My filesystem is laid > out to handle Inboxes in /var/spool/mail and imap folders in > /home(/mail) and these aren't logical volumes but true partitions, so > I've got to consider this as I make any changes. We still use sendmail here, which calls procmail for delivery, mostly for historical reasons - many users have complex .procmailrc files. If I had to start over I'd certainly give postfix a shot. > > I also run pop using dovecot, so I'll need to figure all this out. Make sure you've read http://wiki.dovecot.org/Migration/UW , particularly for the pop3_uidl_format option, so clients do not re-download everything when you switch from UW -> Dovecot. Cheers, Jim Lawson From solo at privat.dk Tue Feb 21 01:44:30 2012 From: solo at privat.dk (solo at privat.dk) Date: Tue, 21 Feb 2012 00:44:30 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <70.4B.28132.FDAD24F4@fep46.mail.dk> Hi Julio / All. I've found the issue : the password was wrong/missing in /home/vpopmail/etc/vpopmail.mysql Now all is well. Thanks,for Your help Finn B ----- Original meddelelse ----- > Fra: Julio C. Ortega > Til: dovecot at dovecot.org > Dato: L?r, 18. feb 2012 16:30 > Emne: Re: [Dovecot] Something is strange - perhaps my AMD processors > ? > > > > Dovecot.log > > > > master: Error: Service(auth): command startup failed, throttling > > log: Error: Service(auth): child 1859 returned error 89 (Fatal > failure) > > pop3-login: Error: Timeout waiting for handshake from auth server. > my > > pid=1858, input bytes=0 > > > > > > Regards, > > Finn B > > > > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > http://www.vaslibre.org.ve > Venezuela From tss at iki.fi Tue Feb 21 02:17:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:17:20 +0200 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F427826.3050506@fi.upm.es> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: On 20.2.2012, at 18.43, Juan C. Blanco wrote: > I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. > > You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like > > user_attrs=....,ldap-attribute=quota_rule=*:storage This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 From tss at iki.fi Tue Feb 21 02:18:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:18:34 +0200 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F428323.8050404@fastmail.fm> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> Message-ID: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> On 20.2.2012, at 19.30, Metro Domain Admin wrote: > On 2/18/12 8:37 AM, Thomas Leuxner wrote: >>> are there any pitfalls known for the migration from 2.0.18 to 2.1? >> http://wiki2.dovecot.org/Upgrading/2.1 > > Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Yes, but it should still work.. From jtam.home at gmail.com Tue Feb 21 02:25:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 20 Feb 2012 16:25:46 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving to > maildir format. My switchover this weekend is full of holes due to the > way user's imap folders were laid out. Some had folders in their home > directory and others might have folders in their /home/mail directory. I had the same problem, and handled it more or less the same way as the later poster (Jim Lawson ) handled his site. I retained use of mbox format as well. I planned the migration like this - scan user home directories for mailboxes (especially "Trash", "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. This was piped into a script that Email'd users about the changes that was going to happen, what they would expect to see, and a FAQ on how to set up a mail client correctly. - during the cutover, - mailboxes left on the home directory were moved to ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid name collision for users that had a mixture of correct and blank prefixes. (I think I deleted "Trash", "Junk", etc. anyways). - Namespace aliasing was used so that prefixes "", "mail", "~/mail", and whatever darn fool settings my users used, would map to the same directory. - .subscription files were moved into the mail folder (don't have to edit prefixes since the aliasing will take care of that). - depending on what POP3 client you used (I used qpopper), you may need to configure pop3_reuse_xuidl = yes to avoid a massive re-downloading from POP3 clients after cutover. - the mail clients I control centrally (e.g. webmail, public server mail clients, etc). shouldn't need updating since they ought to have been set up properly in the first place. - after cutover, a second notification was sent for users that didn't move their mailboxes the first time around, and was it done for them during migration. And of course, test like crazy and watch the logs like a hawk. Other gotcha's: - your setup is fairly close to mine, so you may also run into the problem of user having mailboxes with group ownership that users are not part of (for example, group "mail" for INBOX set by your LDA or personal mailboxes with groups the user is no longer a member of) *and* with group permission not mode=0. You'll have to treat these (set mode=0, or change the group to something the user is part of), or the dovecot index creation will fail and they won't be able to access their mailboxes. I think my migration went pretty smoothly. Less than a handful of wazzup' Email problem reports. Joseph Tam From tss at iki.fi Tue Feb 21 02:27:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:27:28 +0200 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F40F0C5.2010100@mathiesen.info> References: <4F40F0C5.2010100@mathiesen.info> Message-ID: <3AE863BC-AEFF-478B-BBFA-99E90513F8D8@iki.fi> On 19.2.2012, at 14.53, postmaster wrote: > Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Probably won't help with your problem, but it's better if you don't fetch "mail" field. Just fetch the home and set: mail_location = maildir:~/ This avoids maildir autodetection. > a list "" * > * LIST (\HasNoChildren) "." "INBOX" > a OK List completed. > a select inbox > Connection closed by foreign host. > 4604ds1-ynoe:~ root# You mean the session hangs here until it's killed? Try attaching gdb into it: gdb -p bt full or alternatively try: gdb --args /usr/local/libexec/dovecot/imap -u user at domain run a select inbox bt full > # OS: Darwin 10.8.0 i386 hfs With HFS+ it's a good idea to set maildir_copy_with_hardlinks=no From tss at iki.fi Tue Feb 21 02:29:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:29:24 +0200 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <20120220092453.GA6316@dibs.tanso.net> References: <4F41EECB.2070405@lazurit.com> <20120220092453.GA6316@dibs.tanso.net> Message-ID: <27AB0CCC-F41B-4F99-A6BA-9EB8F1972DBE@iki.fi> On 20.2.2012, at 11.24, Jan-Frode Myklebust wrote: >> 2. If I use single (default) namespace, should I set namespace's >> location (to the same value as global mail_location), and should I >> expect anything strange if I skip it to set? Reversely, is it >> possible not to set global mail_location and set only namespace's >> location (which would be more logical as namespace definition is >> compact and easy to find in config)? >> > > We have a single namespace, with blank location: .. > But I don't really know the purpose of this location field vs. > mail_location. If namespace location is empty, it defaults to mail_location. Typically you specify namespace location only when you have more than one namespace. From tss at iki.fi Tue Feb 21 02:35:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:35:26 +0200 Subject: [Dovecot] Feature request for maillog plugin In-Reply-To: <4F421287.3070101@endersys.com> References: <4F421287.3070101@endersys.com> Message-ID: On 20.2.2012, at 11.29, Ayd?n Demirel wrote: > Is it possible to add the modification time That would be the same as IMAP INTERNALDATE / received-date. It would be possible to add this pretty easily. > and file name of the mail file to maillog plugin? This would be much more difficult, because the filename isn't known at the time when the logging is normally done. UIDs have the same problem, but they were specially implemented, and adding such support for filenames would be much more extra work. From tss at iki.fi Tue Feb 21 02:44:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:44:28 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: <4F4235B8.6090107@eurocar.pl> References: <4F4235B8.6090107@eurocar.pl> Message-ID: On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > I have a problem with virtual plugin and geting mails form shared namespace. > dovecot-virtual: > ---- > Shared/* > Public/* > * .. > When I enter the virtual mailbox Unread I get this debug info: > virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > Is it a bug or misconfig? Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 From tss at iki.fi Tue Feb 21 03:02:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:02:00 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> On 18.2.2012, at 20.26, Michael Grimm wrote: > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb From tss at iki.fi Tue Feb 21 03:09:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:09:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). From tss at iki.fi Tue Feb 21 03:11:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:11:10 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <4F3DBF7F.4030504@r.paypc.com> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> <4F3DBF7F.4030504@r.paypc.com> Message-ID: On 17.2.2012, at 4.46, Robin wrote: >> You mean you deleted Solr index, so that it's empty? That should work too. >> >> Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). > > # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts > > I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: > > plugin { > fts = solr > fts_solr = break-imap-search url=http://solrhost:8983/solr/ > } Plugin settings don't do anything if plugin isn't enabled. And if you enabled it only for imap, then it works only in imap and not in pop3/lda/doveadm/etc.. > Should I be adding fts/fts_solr to the global mail_plugins setting? Yes. Probably fixes all your troubles. From tss at iki.fi Tue Feb 21 03:13:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:13:05 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > By the way: what might have caused such a warning? > > root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. From tss at iki.fi Tue Feb 21 03:15:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:15:04 +0200 Subject: [Dovecot] user_filter problem In-Reply-To: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> References: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Message-ID: On 17.2.2012, at 17.39, sanito at progresas.lt wrote: > I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 > with user information stored in the LDAP database. The main problem is > that email address and pop3 username are different for every mailbox (I > cannot change this behaviour, because thousands of people already > configured their email client for such a setup). So in my situation, > user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it > should be: user_filter=(cn=%u). Can't it be either? user_filter = (|(mail=%u)(cn=%u)) > I think I need to use different user_filter for lmtp and pop3, but it's > not possible or I cannot find how to do it. Please, help! With v2.x you can also do something like: protocol lmtp { userdb { driver = ldap args = dovecot-ldap-lmtp.conf.ext } } protocol !lmtp { userdb { driver = ldap args = dovecot-ldap.conf.ext } } From nti at w4w.net Tue Feb 21 03:20:03 2012 From: nti at w4w.net (nti at w4w.net) Date: Tue, 21 Feb 2012 02:20:03 +0100 Subject: [Dovecot] dovecot 2 doesn't start with managesieve Message-ID: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Hi I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). Error: socket() failed: Protocol not supported Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported Fatal: Failed to start listeners I controlled the config, build pigeonhole a second time - nothing helps. Any idea ? Is there a possibility to debug the start? Nicola ... plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve ... service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service managesieve { process_limit = 1024 } ... protocol sieve { managesieve_max_compile_errors = 10 } From tss at iki.fi Tue Feb 21 03:25:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:25:26 +0200 Subject: [Dovecot] dovecot 2 doesn't start with managesieve In-Reply-To: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> References: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Message-ID: <48C8BE3A-32D9-41F7-819F-FEEB78A911E3@iki.fi> On 21.2.2012, at 3.20, nti at w4w.net wrote: > I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). > > Error: socket() failed: Protocol not supported > Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported > Fatal: Failed to start listeners > > I controlled the config, build pigeonhole a second time - nothing helps. > > Any idea ? Is there a possibility to debug the start? You don't have IPv6 supported, so listening in "::" fails. Disable it with: listen = * From javierdemiguel at us.es Tue Feb 21 09:01:11 2012 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_Miguel_Rodr=EDguez?=) Date: Tue, 21 Feb 2012 08:01:11 +0100 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: <4F434137.6070705@us.es> I have seen this behaviour with a local ext4 iSCSI filesystem. When the system is hammered by I/O (example, perfoming a full backup), I also see those messages in the log. Regards Javier > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > >> By the way: what might have caused such a warning? >> >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > From alessio at skye.it Tue Feb 21 09:46:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 08:46:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <4F434BC0.2030300@skye.it> Il 19/02/2012 13:11, Morten Stevens ha scritto: > On 19.02.2012 11:39, Alessio Cecchi wrote: >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: >> service(auth): child 30506 killed with signal 11 (core dumps disabled) > > signal 11 = segmentation fault > > That's probably a dovecot bug. > > Is it possible to reproduce this signal 11 crash? See: > http://dovecot.org/bugreport.html > > Best regards, > > Morten > Hi, I will try to provide as much as possibile information to reproduce the bug. Give me only some days to provide all informations. Thank you -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From tss at iki.fi Tue Feb 21 11:09:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:09:01 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: <4F434137.6070705@us.es> References: <4F434137.6070705@us.es> Message-ID: <1329815341.11500.223.camel@innu> Maybe http://hg.dovecot.org/dovecot-2.1/rev/2ea29ab6f10f fixes it? On Tue, 2012-02-21 at 08:01 +0100, Javier Miguel Rodr?guez wrote: > I have seen this behaviour with a local ext4 iSCSI filesystem. When the > system is hammered by I/O (example, perfoming a full backup), I also see > those messages in the log. > > Regards > > Javier > > > > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > > > >> By the way: what might have caused such a warning? > >> > >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > > > From jcblanco at fi.upm.es Tue Feb 21 11:16:14 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 21 Feb 2012 10:16:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: <4F4360DE.7070708@fi.upm.es> On 21/02/2012 1:17, Timo Sirainen wrote: > On 20.2.2012, at 18.43, Juan C. Blanco wrote: > >> I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. >> >> You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like >> >> user_attrs=....,ldap-attribute=quota_rule=*:storage > > This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 OK, thanks Timo even if the workaround was not difficult at all > -- +----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco at fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+ From tss at iki.fi Tue Feb 21 11:16:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:16:25 +0200 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <1329815785.11500.224.camel@innu> On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" > and SASL for postfix's SMTP-Auth, like explained ih this howto: > http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > > Authentications over IMAP (local dovecot -> remote dovecot) works fine, > and it's great. Also smtp-auth on postfix (over SASL) works fine. But in > the dovecot log I found this error: > > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): > child 30506 killed with signal 11 (core dumps disabled) http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. From ngoonee.talk at gmail.com Tue Feb 21 11:33:34 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 17:33:34 +0800 Subject: [Dovecot] passwd-file - lower limit for username? Message-ID: Hi, previously I could use the following in my passwd-file:- AB:{plain}password I'm not sure when it stopped working, its not an account I access with any regularity, but today it would not work (2.1.0, 64-bit Arch Linux), with an 'unknown user' error in the logs. Is there some new lower limit for username length? From Ralf.Hildebrandt at charite.de Tue Feb 21 11:43:16 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 21 Feb 2012 10:43:16 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> References: <20120218171824.GG8161@charite.de> <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> Message-ID: <20120221094316.GD11284@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > > http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). I think that's very likely. I'll update later on. Thanks again. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 21 11:54:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:54:57 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818097.11500.233.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Dovecot doesn't think the user exists. auth_debug=yes may be helpful, and doveconf -n output. > Is there some new lower limit for username length? No. From postmaster at mathiesen.info Tue Feb 21 11:58:27 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 10:58:27 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436AC3.1040800@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 On a note regarding my hardware : the computer has three ethernet-interfaces: but that doesn't really seem to be an issue as telnet from localhost also hangs indefinetely I'm seeing the issue no matter which interface I connect to, whether the two external WAN or the internal LAN IP-number . root# system_profiler SPEthernetDataType 2012-01-22 01:34:36.524 system_profiler[47377:903] Found bsdName (0x2603) for AppleUSBEthernet 2012-01-22 01:34:36.626 system_profiler[47377:903] Found bsdName (0x2c03) for AppleUSBEthernet Ethernet Cards: Apple USB Ethernet Adaptor: Type: USB BSD name: en4 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8a:f3:5a Product ID: 5122 Vendor ID: 1452 Apple USB Ethernet Adaptor: Type: USB BSD name: en3 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8b:26:c1 Product ID: 5122 Vendor ID: 1452 Marvell Yukon Gigabit Adapter 88E8053 Singleport Copper SA: Name: ethernet Type: Ethernet Controller Bus: PCI Vendor ID: 0x11ab Device ID: 0x4362 Subsystem Vendor ID: 0x11ab Subsystem ID: 0x5321 Revision ID: 0x0022 Link Width: x1 BSD name: en0 Kext name: AppleYukon2.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleYukon2.kext Version: 3.2.1b1 ethernet: Type: Ethernet Controller Bus: PCI Slot: AirPort Vendor ID: 0x168c Device ID: 0x001c Subsystem Vendor ID: 0x106b Subsystem ID: 0x0086 Revision ID: 0x0001 Link Width: x1 BSD name: en1 Kext name: AirPortAtheros21.kext Location: /System/Library/Extensions/IO80211Family.kext/Contents/PlugIns/AirPortAtheros21.kext Version: 425.14.7 root# ifconfig lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 gif0: flags=8010 mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863 mtu 1500 ether 00:16:cb:a8:7d:01 inet6 fe80::216:cbff:fea8:7d01%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.37 netmask 0xffffff00 broadcast 192.168.1.255 media: autoselect (1000baseT ) status: active en4: flags=8863 mtu 1500 ether 00:1f:f3:8a:f3:5a inet6 fe80::21f:f3ff:fe8a:f35a%en4 prefixlen 64 scopeid 0x5 inet 90.184.13.xxx netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active en3: flags=8863 mtu 1500 ether 00:1f:f3:8b:26:c1 inet6 fe80::21f:f3ff:fe8b:26c1%en3 prefixlen 64 scopeid 0x6 inet 90.184.12.yyy netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active fw0: flags=8863 mtu 2030 lladdr 00:17:f2:ff:fe:8e:e6:d2 media: autoselect status: inactive en1: flags=8863 mtu 1500 ether 00:17:f2:52:62:9e media: autoselect () status: inactive root# system_profiler SPHardwareDataType Hardware: Hardware Overview: Model Name: Mac mini Model Identifier: Macmini1,1 Processor Name: Intel Core Duo Processor Speed: 1,83 GHz Number Of Processors: 1 Total Number Of Cores: 2 L2 Cache: 2 MB Memory: 2 GB Bus Speed: 667 MHz Boot ROM Version: MM11.0055.B08 SMC Version (system): 1.3f4 root# system_profiler SPSoftwareDataType Software: System Software Overview: System Version: Mac OS X 10.6.8 (10K549) Kernel Version: Darwin 10.8.0 Boot Volume: X Boot Mode: Normal Computer Name: MiniWeb User Name: System Administrator (root) Secure Virtual Memory: Enabled 64-bit Kernel and Extensions: No Time since boot: 5 days 1:31 >On 19.2.2012, at 14.53, postmaster wrote: > >> Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: >sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM >mailbox WHERE username =3D 'abuse at mathiesen.info' AND domain =3D >'mathiesen.info' AND active =3D 1 > >Probably won't help with your problem, but it's better if you don't >fetch "mail" field. Just fetch the home and set: > >mail_location = maildir:~/ > >This avoids maildir autodetection. That seemed to fix the autodetection :-) > >> # OS: Darwin 10.8.0 i386 hfs > >With HFS+ it's a good idea to set maildir_copy_with_hardlinks = no OK - set to no as advised > >> a list "" * >> * LIST (\HasNoChildren) "." "INBOX" >> a OK List completed. >> a select inbox >> Connection closed by foreign host. >> 4604ds1-ynoe:~ root# > >You mean the session hangs here until it's killed? Yes ... It does some of what it's supposed to do : the dovecot-uidlist.lock is set email is copied from new to cur, but then nothing more seems to happen :-( 4604ds1-ynoe:mail root# ls -l mathiesen.info/macintosh\@mathiesen.info/ total 40 drwx------ 2 5000 5000 4352 Feb 21 09:58 cur -rw------- 1 5000 5000 8119 Feb 21 10:28 dovecot-uidlist -rw------- 1 5000 5000 32 Feb 21 10:28 dovecot-uidlist.lock -rw------- 1 5000 5000 8 Feb 21 10:28 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 Feb 21 10:28 dovecot-uidvalidity.4f4363ac -rw------- 1 5000 5000 40 Feb 21 10:28 dovecot.index.log drwxr-xr-x 2 5000 5000 68 Feb 19 11:18 dovecot.rawlog drwx------ 2 5000 5000 68 Feb 21 09:58 new drwx------ 2 5000 5000 68 Feb 21 09:11 tmp 4604ds1-ynoe:mail root# >Try attaching gdb into it: > >gdb -p >bt full 4604ds1-ynoe:~ root# ps -axj | grep dovecot root 40655 1 40655 4d84750 0 Ss ?? 0:00.02 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 40657 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/anvil root 40658 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/log root 40660 40655 40655 4d84750 0 S ?? 0:00.06 dovecot/config _dovecot 40662 40655 40655 4d84750 0 S ?? 0:00.02 dovecot/auth root 40663 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/auth -w 5000 40677 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40689 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40803 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 root 40618 362 40618 5680af8 1 S+ s001 0:00.01 tail -f dovecot.log root 40952 396 40951 4d85d40 2 R+ s003 0:00.00 grep dovecot 4604ds1-ynoe:~ root# gdb -p 40803 GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin". /private/var/root/40803: No such file or directory Attaching to process 40803. Reading symbols for shared libraries . done Reading symbols for shared libraries ...... done 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001f35 in login_client_connected () No symbol table info available. #22 0x000eb1a8 in master_login_auth_finish () No symbol table info available. #23 0x000eae80 in master_login_auth_callback () No symbol table info available. #24 0x000ebb3a in master_login_auth_input () No symbol table info available. #25 0x001001a7 in io_loop_call_io () No symbol table info available. #26 0x00101492 in io_loop_handler_run () No symbol table info available. #27 0x00100148 in io_loop_run () No symbol table info available. #28 0x000ed25a in master_service_run () No symbol table info available. #29 0x00001df8 in main () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from process 40803. 4604ds1-ynoe:~ root# > >or alternatively try: > >gdb --args /usr/local/libexec/dovecot/imap -u user at domain >run >a select inbox > >bt full 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/: mode=0700 gid=-1 a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001d71 in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From tss at iki.fi Tue Feb 21 11:58:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:58:41 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818321.11500.236.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Is there some new > lower limit for username length? Oh, I think it's because AB is uppercased. Either lowercase it "ab" or set this to empty: auth_username_format = I hadn't thought of this being a problem before.. Hm. Perhaps it's not a common problem, so I'll just change the Upgrading text to mention this possibility also.. From tss at iki.fi Tue Feb 21 12:08:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 12:08:06 +0200 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F436AC3.1040800@mathiesen.info> References: <4F436AC3.1040800@mathiesen.info> Message-ID: <1329818886.11500.239.camel@innu> On Tue, 2012-02-21 at 10:58 +0100, postmaster wrote: > > #0 0x90003afa in mach_msg_trap () > No symbol table info available. > #1 0x90004267 in mach_msg () > No symbol table info available. > #2 0x9001b431 in vproc_mig_look_up2 () > No symbol table info available. > #3 0x9001b26a in bootstrap_look_up2 () > No symbol table info available. > #4 0x9001b0e4 in bootstrap_look_up () > No symbol table info available. > #5 0x9001af30 in notify_register_check () > No symbol table info available. > #6 0x9001ada7 in notify_register_tz () > No symbol table info available. > #7 0x9001a621 in tzsetwall_basic () > No symbol table info available. > #8 0x90041903 in localtime () > No symbol table info available. All of the above functions are OSX's internal functions in libc. > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. From postmaster at mathiesen.info Tue Feb 21 12:11:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:11:26 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436DCE.5050001@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 arrrrgh -- used pop3 instead of imap :-( 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/imap -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/imap -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as postmaster at mathiesen.info a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x00092a2f in mail_index_update_day_headers () No symbol table info available. #10 0x0008ec95 in mail_index_transaction_finish () No symbol table info available. #11 0x0008d80a in mail_index_transaction_commit_v () No symbol table info available. #12 0x0008dacf in mail_index_transaction_commit_full () No symbol table info available. #13 0x0008db18 in mail_index_transaction_commit () No symbol table info available. #14 0x00096434 in mail_index_sync_commit () No symbol table info available. #15 0x00066f02 in maildir_sync_index_finish () No symbol table info available. #16 0x00068376 in maildir_sync_context () No symbol table info available. #17 0x000686f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000295fd in mailbox_sync () No symbol table info available. #19 0x00007833 in cmd_select_full () No symbol table info available. #20 0x00007d49 in cmd_select () No symbol table info available. #21 0x00009f63 in client_command_input () No symbol table info available. #22 0x0000a0c2 in client_command_input () No symbol table info available. #23 0x0000a1d3 in client_handle_input () No symbol table info available. #24 0x000095bd in client_input () No symbol table info available. #25 0x001171a7 in io_loop_call_io () No symbol table info available. #26 0x00118492 in io_loop_handler_run () No symbol table info available. #27 0x00117148 in io_loop_run () No symbol table info available. #28 0x0010425a in master_service_run () No symbol table info available. #29 0x00013c5a in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From postmaster at mathiesen.info Tue Feb 21 12:20:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:20:26 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436FEA.8000606@mathiesen.info> From: Timo Sirainen > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. OK - Thanks for your help :-) I'll take it up on the macports mailing lists then ! -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From ngoonee.talk at gmail.com Tue Feb 21 12:22:39 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 18:22:39 +0800 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: <1329818321.11500.236.camel@innu> References: <1329818321.11500.236.camel@innu> Message-ID: On Tue, Feb 21, 2012 at 5:58 PM, Timo Sirainen wrote: > On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: >> Hi, previously I could use the following in my passwd-file:- >> >> AB:{plain}password >> >> I'm not sure when it stopped working, its not an account I access with >> any regularity, but today it would not work (2.1.0, 64-bit Arch >> Linux), with an 'unknown user' error in the logs. Is there some new >> lower limit for username length? > > Oh, I think it's because AB is uppercased. Either lowercase it "ab" or > set this to empty: > > auth_username_format = > > I hadn't thought of this being a problem before.. Hm. Perhaps it's not a > common problem, so I'll just change the Upgrading text to mention this > possibility also.. > Yes, its probably the upper-casing. No problem, its just that it used to work and now doesn't, I've already changed the username (to something longer which isn't all uppercase) anyway. Probably SHOULD be mentioned briefly somewhere. Thanks much. From busseniu at in.tum.de Tue Feb 21 12:23:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 11:23:03 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F437087.3000407@in.tum.de> Hi, On 15.02.2012 04:46, Timo Sirainen wrote: > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. sorry to bother you again, but I think there is a problem with this patch: If a maildir contains several copies of the same message all in the same folder, dsync will not deduplicate them. While IMAP cannot directly create copies of a message in the same folder, it does still happen if you copy (or move) a message back and forth between two folders. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 13:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 13:04:25 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F437087.3000407@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> Message-ID: <1329822265.11500.254.camel@innu> On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: > Hi, > > On 15.02.2012 04:46, Timo Sirainen wrote: > > > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. > > sorry to bother you again, but I think there is a problem with this patch: > > If a maildir contains several copies of the same message all in the same > folder, dsync will not deduplicate them. Correct. I nearly finished implementing this also, but then I thought it just makes the code unnecessarily complex for no good reason. > While IMAP cannot directly create copies of a message in the same > folder, It can: SELECT INBOX, COPY 1 INBOX > it does still happen if you copy (or move) a message back and > forth between two folders. Is it common enough to be an actual problem? From joshua at hybrid.pl Tue Feb 21 13:25:51 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 21 Feb 2012 12:25:51 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: <7304de1adf6749ec476ef30957bb931c@hybrid.pl> W dniu 2012-02-21 02:13, Timo Sirainen napisa?(a): > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: >> By the way: what might have caused such a warning? >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u >> joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp >> is different than current time (1329464622 vs 1329464672): >> /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't > happen unless remote storage server's clock and local server's clock > aren't synced. OK, that's the case - /var/mail is stored on remote NFS. I'll check if I have time properly set, but if so - I'll probably have to live with it (I have no influence on NFS server). Greetings, -- Jacek Osiecki From busseniu at in.tum.de Tue Feb 21 13:55:05 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Tue, 21 Feb 2012 12:55:05 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <1329822265.11500.254.camel@innu> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> Message-ID: <4F438619.70405@in.tum.de> On 21.02.2012 12:04, Timo Sirainen wrote: > On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: >> Hi, >> >> On 15.02.2012 04:46, Timo Sirainen wrote: >>> >>> It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. >> >> sorry to bother you again, but I think there is a problem with this patch: >> >> If a maildir contains several copies of the same message all in the same >> folder, dsync will not deduplicate them. > > Correct. I nearly finished implementing this also, but then I thought it > just makes the code unnecessarily complex for no good reason. > >> While IMAP cannot directly create copies of a message in the same >> folder, > > It can: SELECT INBOX, COPY 1 INBOX Oh, mea culpa :) > >> it does still happen if you copy (or move) a message back and >> forth between two folders. > > Is it common enough to be an actual problem? Actually we have some mailboxes with massively duplicated messages in the same folder. Of course I cannot tell how common it is in general. I could imagine that some people routinely copy all INBOX messages into archive folders and do not check whether the archive already contains these messages. Apart from the waste of space, I was wondering: Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 14:15:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:15:33 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F438619.70405@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: On 21.2.2012, at 13.55, Christoph Bu?enius wrote: > Apart from the waste of space, I was wondering: > Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? No corruption. And they might even become deduplicated if you do doveadm force-resync + purge. From campbell at cnpapers.com Tue Feb 21 14:19:33 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:19:33 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F42C212.5090509@uvm.edu> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> Message-ID: <4F438BD5.4020107@cnpapers.com> On 2/20/2012 4:58 PM, Jim Lawson wrote: > On 2/20/12 3:36 PM, Steve Campbell wrote: >> >> Thanks for that input. I still think I'm missing something since I >> too used the compatibility link that you pointed to. Only thing is >> that proceeding those namespaces, I used the first example of: >> >> namespace { >> type = private >> separator = / >> prefix = "#mbox/" >> location = mbox:~/mail:INBOX=/var/mail/%u >> inbox = yes >> hidden = yes >> list = no >> } >> namespace { >> type = private >> separator = / >> prefix = >> location = maildir:~/mail >> } >> >> This seemed to get the ball rolling so that users could access >> anything at all. I still have the problem of client imap folders >> being different from webmail imap folders. That's pretty much why I'm >> thinking of using mbox as the INBOX and all other imap folders in the >> ~/mail directory in maildir format. > You ought to be able to get webmail to see the same folders as the > imap clients. > >> >> Should I remove the first two namespaces, in your opinion? Right now, >> the whole thing is kind of fragile. >> > > From your added namespaces, it looks like you are trying to support > mbox and maildir and ~/mail at the same time. Do you already have > maildir folders to support? If not, I would try to get things working > well with mbox first before I started a conversion to maildir. I > would also ask why you're thinking of moving to maildir. Yes, there > are caching benefits, but when you add the Dovecot indexing on top of > mbox, it's pretty much a wash. If you are using file-level backup, > rather than some sort of snapshot technology, maildir will be much, > much slower to back up. Your system will spend all its time walking > directories, opening and closing files. If you don't have many users > to worry about it might be OK, but make sure it's worth it. A lot of > sites went to maildir in the 1998-2004 era and have regretted the > decision as their systems get overloaded with files and they can't > back them up. No, I've got all mbox formats. Previous posts probably suggested that I wanted to move to maildir, but all of the replies I've received have convinced me that I do not want to do that. The folders in ~ and ~/mail are mboxes, so I need to see what damage I've done with a maildir namespace. The maildir reference could be part of the problems I'm seeing. I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. > >> Horde/Imp updates are probably out of the question until I can get a >> server to install the upgrade on. > For Dovecot and IMP both, you should set up an alternate server to > test out your config changes on before you put them into production. > If you are running on a bare metal single server, set up another > Dovecot instance on a different set of ports (I commonly use 20143 > (imap), 20993 (imaps), 20110 (pop3)) which you can fiddle with > freely. Once you are satisfied with the result in your various > clients, put it into production. You can do the same with Horde/IMP > by putting an installation in a different location on your webserver. > > Jim I've got a second server that is totally independent of the one I'm messing with. I've done the horde/imp alternate port/config. It all went well. The downside is that I didn't realize those secondary folders in ~/mail were being hidden. Kinda late to switch back, but the secondary server supports another domain, so I have it to test with. Again, the damage I've caused to those secondary imap folders needs to be determined to get this fixed properly. Most users are seing the secondary folders and can use them. It's just those users who have folders in ~ that are seeing problems as far as I can tell. Thanks steve > > From campbell at cnpapers.com Tue Feb 21 14:27:35 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:27:35 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <4F438DB7.5020001@cnpapers.com> On 2/20/2012 7:25 PM, Joseph Tam wrote: > > Steve Campbell wrote: > >> The more I read about all of this, the more I'm thinking about moving >> to maildir format. My switchover this weekend is full of holes due to >> the way user's imap folders were laid out. Some had folders in their >> home directory and others might have folders in their /home/mail >> directory. > > I had the same problem, and handled it more or less the same way as > the later poster (Jim Lawson ) handled his site. > I retained use of mbox format as well. > > I planned the migration like this > > - scan user home directories for mailboxes (especially "Trash", > "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. > > This was piped into a script that Email'd users about the changes > that was going to happen, what they would expect to see, and a > FAQ on how to set up a mail client correctly. Here, I'm not sure what should be done. The users with the secondary folders that are not in ~/mail can't seem to get the client configured. > > - during the cutover, > > - mailboxes left on the home directory were moved to > ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid > name collision for users that had a mixture of correct > and blank prefixes. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot is ignoring this and I'm not sure what it's looking at to determine the imap folders. > (I think I deleted "Trash", "Junk", etc. anyways). > > - Namespace aliasing was used so that prefixes "", "mail", > "~/mail", and whatever darn fool settings my users used, > would map to the same directory. > > - .subscription files were moved into the mail folder > (don't have to edit prefixes since the aliasing > will take care of that). And when would these .subscription files be created? The first time the folders would be accessed, or when? This seems to be part of my fix that I'm getting lost on. > > - depending on what POP3 client you used (I used > qpopper), you may need to configure > > pop3_reuse_xuidl = yes > > to avoid a massive re-downloading from POP3 clients > after cutover. > > - the mail clients I control centrally (e.g. webmail, > public server mail clients, etc). shouldn't need updating > since they ought to have been set up properly in the first > place. > > - after cutover, a second notification was sent for users that > didn't move their mailboxes the first time around, and was it > done for them during migration. > > And of course, test like crazy and watch the logs like a hawk. > > Other gotcha's: > > - your setup is fairly close to mine, so you may also run into > the problem of user having mailboxes with group ownership that > users are not part of (for example, group "mail" for INBOX set > by your LDA or personal mailboxes with groups the user is no > longer a member of) *and* with group permission not mode=0. Sendmail sort of requires the "mail" group, does it not? I'll take a look and see if all users are part of this or not. A crazy solution would be in order here? > > You'll have to treat these (set mode=0, or change the group to > something the user is part of), or the dovecot index creation > will fail and they won't be able to access their mailboxes. > > I think my migration went pretty smoothly. Less than a handful of > wazzup' > Email problem reports. Except for those users with the different folder locations, it seems that all is going pretty well. Maybe they're just not notifying me yet, though. > > Joseph Tam > > Thanks steve From tss at iki.fi Tue Feb 21 14:28:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:28:30 +0200 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F438BD5.4020107@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> <4F438BD5.4020107@cnpapers.com> Message-ID: <56BC47FD-F1EB-4294-A25C-5205E30AAA36@iki.fi> On 21.2.2012, at 14.19, Steve Campbell wrote: > I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. With new enough Dovecot version (2.0, maybe 1.2) you can add :SUBSCRIPTIONS=.mailboxlist to mail_location. From dag at newtech.fi Tue Feb 21 14:56:01 2012 From: dag at newtech.fi (Dag Nygren) Date: Tue, 21 Feb 2012 14:56:01 +0200 Subject: [Dovecot] Where did the message end up? Message-ID: <12808509.iPEzrCISOc@dag> Hi all! Just migrated out UW-IMAP installation to Dovecot and so far so good. Had small teething problems though which left me with one question. The scenario was that I have a set of public folders that are only needed by the main company group (gid = company). Then there are some external users as well, not directly in the company group. I set up the public mailbox with permissions 660, owner me and group "company". That is the externals didn't have access to the mail folder. At this point all looked fine, but later there was an email to an external person and this was passed on to deliver by procmail. Procmail logs says all was OK, but there is an entry about the public folder not accessible in the Dovecot log. The email is nowhere to be found. My question: Where is the delivered email? Best Dag From alessio at skye.it Tue Feb 21 15:29:53 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 14:29:53 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <1329815785.11500.224.camel@innu> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> <1329815785.11500.224.camel@innu> Message-ID: <4F439C51.3090907@skye.it> Il 21/02/2012 10:16, Timo Sirainen ha scritto: > On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > >> I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" >> and SASL for postfix's SMTP-Auth, like explained ih this howto: >> http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL >> >> Authentications over IMAP (local dovecot -> remote dovecot) works fine, >> and it's great. Also smtp-auth on postfix (over SASL) works fine. But in >> the dovecot log I found this error: >> >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): >> child 30506 killed with signal 11 (core dumps disabled) > > http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. > > Patch applied, after rebuilding all works fine. Thanks Timo -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From lists at wildgooses.com Tue Feb 21 16:33:24 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 21 Feb 2012 14:33:24 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213194356.GB3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> Message-ID: <4F43AB34.9040907@wildgooses.com> On 13/02/2012 19:43, Jan-Frode Myklebust wrote: > On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: >> Out of curiosity, are you running dovecot locally on those webmail >> servers as well, or is it talking to remote dovecot servers? > The webmail servers are talking with dovecot director servers which in > turn are talking with the backend dovecot servers. Each service running > on different servers. > > Webmail-servers -> director-servers -> backend-servers > I think the original question was still sensible. In your case it seems like the ping times are identical between: webmail -> imap-proxy webmail -> imap server I think your results show that a proxy has little (or negative) benefit in this situation, but it seems feasible that a proxy could eliminate several RTT trips in the event that the proxy is "closer" than the imap server? This might happen if say the imap server is in a different datacenter (webmail on an office server machine?) I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). However, using imap-proxy has the benefit of "clustering" logins a little and this makes log files a little easier to understand in the face of users with desktop mail clients plus webmail users. Possibly this idea useful to someone else... Thanks for measuring this! Ed W From crohmann at netcologne.de Tue Feb 21 17:05:39 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 21 Feb 2012 16:05:39 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? Message-ID: <4F43B2C3.3080006@netcologne.de> Hey dovecot-users, clients that speak telnet based protocols like http, smtp usually encode IDN (International Domain Names) containing UTF-8 characters into ACE (ASCII Compatible Encoding) punycode. I am wondering what the case was with managesieve (RFC 5804). For example if I was using a username like "user@???.example.com" with the AUTHENTICATE command. Would that need to be encoded into ACE or be left in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal Syntax" suggests that quoted or literal UTF-8 is correct. Thanks Christian From florob at babelmonkeys.de Tue Feb 21 18:11:28 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 21 Feb 2012 17:11:28 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? In-Reply-To: <4F43B2C3.3080006@netcologne.de> References: <4F43B2C3.3080006@netcologne.de> Message-ID: <4F43C230.6000408@babelmonkeys.de> Am 21.02.2012 16:05, schrieb Christian Rohmann: > Hey dovecot-users, > > clients that speak telnet based protocols like http, smtp usually encode > IDN (International Domain Names) containing UTF-8 characters into ACE > (ASCII Compatible Encoding) punycode. > > I am wondering what the case was with managesieve (RFC 5804). For > example if I was using a username like "user@???.example.com" with the > AUTHENTICATE command. Would that need to be encoded into ACE or be left > in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal > Syntax" suggests that quoted or literal UTF-8 is correct. > Hy, what you're asking for is the correct format of the simple username. This is in fact deployment specific. There is in general no requirement to include the domain at all. The only requirement for to fulfil is that it must not contain any codepoints prohibited by the SASLPrep stringprep profile. As such both the ACE and the IDN variant are valid. Which one to choose, or whether to use only "user" depends on local policies. There might be some implementation specifics in dovecot/pigeonhole that I'm not aware of though. Regards, Florian From busseniu at in.tum.de Tue Feb 21 18:36:18 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 17:36:18 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: <4F43C802.7010702@in.tum.de> On 21.02.2012 13:15, Timo Sirainen wrote: > > And they might even become deduplicated if you do doveadm force-resync + purge. > I hadn't tried that yet. Thanks for the hint, this is probably all we need. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From trashcan at odo.in-berlin.de Tue Feb 21 19:29:12 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 18:29:12 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> Message-ID: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Hi -- On 21.02.2012, at 02:02, Timo Sirainen wrote: > On 18.2.2012, at 20.26, Michael Grimm wrote: >> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb Hmm, now I cannot clients any longer: dovecot: master: Dovecot v20120221 (7d0d76df719f+) starting up dovecot: imap-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, mpid=65444, TLS dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test dovecot: imap(test): Error: Invalid user settings. Refer to server log for more information. [... repeated for every user clients are trying to connect ...] I switched back immediately (production server), thus I didn't test if dsync might run now. Here's the relevant part of my config: mail_home = /var/mail/.homedirs/%n mail_location = mdbox:/var/mail/.maildirs/%n User's individual mail storage formats are fetched from userdb (sqlite3), which normally are mdbox, except for test which is sdbox. Any idea what to test next? Thanks and regards, Michael From tss at iki.fi Tue Feb 21 22:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:36:32 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> On 21.2.2012, at 16.33, Ed W wrote: > I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: void postlogout_init(struct module *module) { } void postlogout_deinit(void) { system("/usr/local/bin/dovecot-postlogout.sh"); } Add a few missing #includes and compile and enable for imap/pop3 and that should be it. From tss at iki.fi Tue Feb 21 22:52:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:52:14 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Message-ID: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Hi, On 21.2.2012, at 19.29, Michael Grimm wrote: >>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb > > Hmm, now I cannot clients any longer: > > dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. From trashcan at odo.in-berlin.de Tue Feb 21 23:48:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 22:48:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: Hi -- On 21.02.2012, at 21:52, Timo Sirainen wrote: > On 21.2.2012, at 19.29, Michael Grimm wrote: >>>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >>> >>> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb >> >> Hmm, now I cannot connect clients any longer: >> >> dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test > > Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. Thanks, that omits the first and second error, and I believe that you solved this issue. Now, I can run my old command as used for 2.0.x and ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test ... a "newer" call like ... vmail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test ... without complaining about namespaces as listed above. But, now I do end with a message like ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test usage: dsync [-C ] [-m ] [-u ] [-frRv] mirror | [@] | dsync-local(test): Error: read() from worker server failed: EOF ... irrespectively if I use the old "dsync" or the new "doveadm sync". Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Weird. I must have missed something. I'll back to 2.0.18 for the time being. Any hints are highly welcome ... Thanks for your efforts, and thanks for dovecot! Regards, Michael From tss at iki.fi Tue Feb 21 23:57:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 23:57:21 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> On 21.2.2012, at 23.48, Michael Grimm wrote: > But, now I do end with a message like ... > > vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test > usage: dsync [-C ] [-m ] [-u ] [-frRv] > mirror | [@] | > dsync-local(test): Error: read() from worker server failed: EOF > > ... irrespectively if I use the old "dsync" or the new "doveadm sync". > > Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Added it there. From trashcan at odo.in-berlin.de Wed Feb 22 00:03:52 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 23:03:52 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 22:57, Timo Sirainen wrote: > On 21.2.2012, at 23.48, Michael Grimm wrote: >> But, now I do end with a message like ... >> >> vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test >> usage: dsync [-C ] [-m ] [-u ] [-frRv] >> mirror | [@] | >> dsync-local(test): Error: read() from worker server failed: EOF >> >> ... irrespectively if I use the old "dsync" or the new "doveadm sync". >> >> Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, > > Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). Ok. Then I will try both servers upgraded to 2.1 within the next days. Thanks for that valuable info that I really missed, sorry. >> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. > > Added it there. Great. And thanks again, Michael From as at 3a.pl Wed Feb 22 00:46:48 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 21 Feb 2012 23:46:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy Message-ID: <4F441ED8.20908@3a.pl> Hi, In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Each of this emails has the same Message-ID. Is there a way to detect those duplicates and discard them when saving copies into boss account? There was a discussion on the list a while ago (Feb 10/11, 2011) mentioning this problem, but with no clear solution. We are using Dovecot 2.0.15. Mail is delivered via LMTP (MDA/postfix is on separate machine then MAA/dovecot). -- Adam Szpakowski From tss at iki.fi Wed Feb 22 00:57:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 00:57:17 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F441ED8.20908@3a.pl> References: <4F441ED8.20908@3a.pl> Message-ID: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> On 22.2.2012, at 0.46, Adam Szpakowski wrote: > In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). > There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? From as at 3a.pl Wed Feb 22 01:02:23 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:02:23 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> Message-ID: <4F44227F.9030502@3a.pl> On 21.02.2012 23:57, Timo Sirainen wrote: > On 22.2.2012, at 0.46, Adam Szpakowski wrote: >> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. > Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. We are using dedicated sieve_before script for each user based on %u variable. -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:09:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:09:35 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F44227F.9030502@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> Message-ID: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> On 22.2.2012, at 1.02, Adam Szpakowski wrote: > On 21.02.2012 23:57, Timo Sirainen wrote: >> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? > Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. > We are using dedicated sieve_before script for each user based on %u variable. Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: fileinto :copy :x-deduplicate "boss"; Anyway, probably not going to be implemented anytime soon. From as at 3a.pl Wed Feb 22 01:15:30 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:15:30 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> Message-ID: <4F442592.608@3a.pl> On 22.02.2012 00:09, Timo Sirainen wrote: > On 22.2.2012, at 1.02, Adam Szpakowski wrote: >> On 21.02.2012 23:57, Timo Sirainen wrote: >>> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >>> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? >> Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. >> We are using dedicated sieve_before script for each user based on %u variable. > Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. > > I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: > > fileinto :copy :x-deduplicate "boss"; > > Anyway, probably not going to be implemented anytime soon. Maybe there is a way to use a procmail with something like this: :0 Wh: msgid.lock | formail -D 8192 .msgid.cache But is there a safe way to use it together with sieve? Using Pigeonhole Sieve Pipe Plugin? -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:32:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:32:56 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <12808509.iPEzrCISOc@dag> References: <12808509.iPEzrCISOc@dag> Message-ID: On 21.2.2012, at 14.56, Dag Nygren wrote: > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > Had small teething problems though which left me with one question. > > The scenario was that I have a set of public folders that are only needed by > the main company group (gid = company). Then there are some external users as > well, not directly in the company group. So this is in a public namespace? > I set up the public mailbox with permissions 660, owner me and group > "company". That is the externals didn't have access to the mail folder. > > At this point all looked fine, but later there was an email to an external > person and this was passed on to deliver by procmail. Procmail logs says all > was OK, but there is an entry about the public folder not accessible in the > Dovecot log. The email is nowhere to be found. Exact error message would be helpful. Was the error from deliver, or imap/pop3? If it was from deliver, it probably returned temporary failure. > Where is the delivered email? Have you checked what happens if deliver tempfails (e.g. make procmail call tempfail.sh script that just exits with 75), does procmail also correctly tempfail and tell MTA to retry the delivery later? From jtam.home at gmail.com Wed Feb 22 02:04:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 21 Feb 2012 16:04:30 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell writes: > > This was piped into a script that Email'd users about the changes > > that was going to happen, what they would expect to see, and a > > FAQ on how to set up a mail client correctly. > > Here, I'm not sure what should be done. The users with the secondary > folders that are not in ~/mail can't seem to get the client configured. "Great artists steal". Google around for a good set of instructions on how to set up IMAP prefix's for various mail clients, and refer your users to that. I also referred them to our webmail (with the correct mail prefix) as a reference to a list of mailboxes they ought to see. If they don't match, then that's a symptom of a bad prefix. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > is ignoring this and I'm not sure what it's looking at to determine the > imap folders. > > > - .subscription files were moved into the mail folder > > (don't have to edit prefixes since the aliasing > > will take care of that). > > And when would these .subscription files be created? The first time the > folders would be accessed, or when? This seems to be part of my fix that > I'm getting lost on. Sorry, as Timo intimated, dovecot uses .subscription, which can be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't have to edit it if you set up namespace aliasing. Just move it from the top-level into your user's mail directory during cutover: (if you use dovecot's default subscription filename) mv $USER/.mailboxlist $USER/mail/.subscription If your IMAP client is still ignoring subscriptions, it usually means the client was configured that way. > > - your setup is fairly close to mine, so you may also run into > > the problem of user having mailboxes with group ownership that > > users are not part of (for example, group "mail" for INBOX set > > by your LDA or personal mailboxes with groups the user is no > > longer a member of) *and* with group permission not mode=0. > > Sendmail sort of requires the "mail" group, does it not? I'll take a > look and see if all users are part of this or not. A crazy solution > would be in order here? I think sendmail works fine if you set your mail spool mode=1777 (like /tmp), but have each inbox mode=0600. This setting will persist. It's sendmail LDA mail.local that's the problem: it autocreates new inboxes with permissions user:mail/660 and unless the user is in group "mail", chaos will ensue. You can maybe do a cron script or some other hack, or better yet, use dovecot LDA, which sets up the perms correctly (and keeps indexes updated). A one time "chmod go-rwx" during cutover is all that is required then. The log entry that points this problem out looks like dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) failed: Operation not permitted (egid=5678(goodgroup), group based on /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not owner Joseph Tam From campbell at cnpapers.com Wed Feb 22 04:29:31 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 21:29:31 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <1329877771.4f44530b83411@perdition.cnpapers.net> Quoting Joseph Tam : > > Steve Campbell writes: > > > > This was piped into a script that Email'd users about the changes > > > that was going to happen, what they would expect to see, and a > > > FAQ on how to set up a mail client correctly. > > > > Here, I'm not sure what should be done. The users with the secondary > > folders that are not in ~/mail can't seem to get the client configured. > > "Great artists steal". Google around for a good set of instructions > on how to set up IMAP prefix's for various mail clients, and refer > your users to that. This conversion to dovecot has been a real learning experience for me. The old stuff that came with Centos 3 just worked out of the box. I never realized how much could be modified. It was always there, I just never needed it. > > I also referred them to our webmail (with the correct mail prefix) as a > reference to a list of mailboxes they ought to see. If they don't match, > then that's a symptom of a bad prefix. Our web mail, Imp, seems to be doing fine now that all of the mboxes have been moved to ~/mail. > > > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > > is ignoring this and I'm not sure what it's looking at to determine the > > imap folders. > > > > > - .subscription files were moved into the mail folder > > > (don't have to edit prefixes since the aliasing > > > will take care of that). > > > > And when would these .subscription files be created? The first time the > > folders would be accessed, or when? This seems to be part of my fix that > > I'm getting lost on. > > Sorry, as Timo intimated, dovecot uses .subscription, which can > be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't > have to edit it if you set up namespace aliasing. Just move it from > the top-level into your user's mail directory during cutover: > > (if you use dovecot's default subscription filename) > mv $USER/.mailboxlist $USER/mail/.subscription > > If your IMAP client is still ignoring subscriptions, it usually > means the client was configured that way. > > > > - your setup is fairly close to mine, so you may also run into > > > the problem of user having mailboxes with group ownership that > > > users are not part of (for example, group "mail" for INBOX set > > > by your LDA or personal mailboxes with groups the user is no > > > longer a member of) *and* with group permission not mode=0. > > > > Sendmail sort of requires the "mail" group, does it not? I'll take a > > look and see if all users are part of this or not. A crazy solution > > would be in order here? > > I think sendmail works fine if you set your mail spool mode=1777 (like > /tmp), but have each inbox mode=0600. This setting will persist. > > It's sendmail LDA mail.local that's the problem: it autocreates new > inboxes with permissions user:mail/660 and unless the user is in group > "mail", chaos will ensue. > So far, all things seem OK in this area. No complaints are being forwarded. > You can maybe do a cron script or some other hack, or better yet, > use dovecot LDA, which sets up the perms correctly (and keeps indexes > updated). A one time "chmod go-rwx" during cutover is all that is > required then. > > The log entry that points this problem out looks like > > dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) > failed: Operation not permitted (egid=5678(goodgroup), group based on > /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) > dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not > owner > > Joseph Tam For now, it looks like the only problem I'm seeing is that one user who uses Outlook Express that has a pop account for one email account and an imap account for another email account is having problems with the "Sent" folder. When she sends an email using the imap account, the "Sent" copy goes into the pop account's sent item folder. When using webmail, the sent copy goes into the sent-mail file as it's supposed to. I'm guessing for now that that google you suggested might point me to a fix for that. Thanks all for all the great help. Shared accounts are next, so wish me luck. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From dag at newtech.fi Wed Feb 22 09:34:27 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 09:34:27 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> Message-ID: <1651793.SOPdjt8lSW@eseries> onsdag 22 februari 2012 01:32:56 skrev Timo Sirainen: > On 21.2.2012, at 14.56, Dag Nygren wrote: > > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > > > Had small teething problems though which left me with one question. > > > > The scenario was that I have a set of public folders that are only > > needed by the main company group (gid = company). Then there are some > > external users as well, not directly in the company group. > > So this is in a public namespace? Yep, it was. The definition: namespace { type = public separator = / prefix = MailingLists/ location = mdbox:/usr/local/var/mail/MailingLists inbox = no subscriptions = no # v1.1+ } > > I set up the public mailbox with permissions 660, owner me and group > > "company". That is the externals didn't have access to the mail folder. > > > > At this point all looked fine, but later there was an email to an > > external person and this was passed on to deliver by procmail. Procmail > > logs says all was OK, but there is an entry about the public folder not > > accessible in the Dovecot log. The email is nowhere to be found. > > Exact error message would be helpful. Was the error from deliver, or > imap/pop3? If it was from deliver, it probably returned temporary failure. The log parts with final names and groups changed.. Deliver error: Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization failed: Namespace 'MailingLists/': stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: /usr/local/var/mail/MailingLists, euid is not dir owner) Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer to server log for more information. Procmail log: >From mailing-1341698764359 at mailpv.net Tue Feb 21 10:02:24 2012 Subject: =?CP1250?Q?P=E4=E4si=E4issuunnitelmat?= Folder: deliver -d extuser -m inbox 56979 > > Where is the delivered email? > > Have you checked what happens if deliver tempfails (e.g. make procmail call > tempfail.sh script that just exits with 75), does procmail also correctly > tempfail and tell MTA to retry the delivery later? No, haven't done that. yet. And I don't think it will pass on the failure code. Procmail settings are all from the HOWTO:s though. The .forward file looks like this: "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" And the applicable rule in .procmailrc: :0 w { :0 | $DELIVERYAGENT -d $LOGNAME -m inbox } Have to point out that this case wasn't important, but I wouldn't like the system to loose an important mail later. From tss at iki.fi Wed Feb 22 09:43:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 09:43:58 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1651793.SOPdjt8lSW@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: On 22.2.2012, at 9.34, Dag Nygren wrote: > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization > failed: Namespace 'MailingLists/': > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied > (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > /usr/local/var/mail/MailingLists, euid is not dir owner) > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer > to server log for more information. Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually v2.1.0 is the first release where you can easily configure Dovecot to simply skip the namespace when it doesn't have permission to access it: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. >>> Where is the delivered email? >> >> Have you checked what happens if deliver tempfails (e.g. make procmail call >> tempfail.sh script that just exits with 75), does procmail also correctly >> tempfail and tell MTA to retry the delivery later? > > No, haven't done that. yet. And I don't think it will pass on the failure > code. Procmail settings are all from the HOWTO:s though. > > The .forward file looks like this: > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > And the applicable rule in .procmailrc: > :0 w > { > :0 > | $DELIVERYAGENT -d $LOGNAME -m inbox > } > > Have to point out that this case wasn't important, but I wouldn't like the > system to loose an important mail later. You should definitely make sure that procmail won't lose any mails if Dovecot LDA returns tempfailure, since they do sometimes happen. You could also get rid of procmail completely by moving to sieve. From dag at newtech.fi Wed Feb 22 10:03:36 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 10:03:36 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: <1958645.10kxPAq8Bz@eseries> > On 22.2.2012, at 9.34, Dag Nygren wrote: > > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: > > Initialization failed: Namespace 'MailingLists/': > > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission > > denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > > /usr/local/var/mail/MailingLists, euid is not dir owner) > > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. > > Refer to server log for more information. > > Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually > v2.1.0 is the first release where you can easily configure Dovecot to > simply skip the namespace when it doesn't have permission to access it: > > + Added ignore_on_failure setting for namespaces. If namespace > initialization fails with this enabled (e.g. permission denied), > the namespace is silently skipped for the user. Ok. Many thanks for answering this. I am using FC14 and dovecot 2.0.4 at the moment so I don't have that option yet. It would be nice BTW if dovecot added the exit status it used in the log if it fails? > >>> Where is the delivered email? > >> > >> Have you checked what happens if deliver tempfails (e.g. make procmail > >> call tempfail.sh script that just exits with 75), does procmail also > >> correctly tempfail and tell MTA to retry the delivery later? > > > > No, haven't done that. yet. And I don't think it will pass on the > > failure > > code. Procmail settings are all from the HOWTO:s though. > > > > The .forward file looks like this: > > > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > > > And the applicable rule in .procmailrc: > > :0 w > > > > { > > > > :0 > > : > > | $DELIVERYAGENT -d $LOGNAME -m inbox > > > > } > > > > Have to point out that this case wasn't important, but I wouldn't like > > the system to loose an important mail later. > > You should definitely make sure that procmail won't lose any mails if > Dovecot LDA returns tempfailure, since they do sometimes happen. You could > also get rid of procmail completely by moving to sieve. Will check the path out. Any way of forcing dovecot to return tempfail for the tests? Preferrably only for one user :-) Not familiar with sieve yet, but will for sure look into it. Thanks again Dag From tss at iki.fi Wed Feb 22 10:18:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 10:18:00 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1958645.10kxPAq8Bz@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> <1958645.10kxPAq8Bz@eseries> Message-ID: On 22.2.2012, at 10.03, Dag Nygren wrote: > It would be nice BTW if dovecot added > the exit status it used in the log if it fails? This wouldn't be very useful normally. If you see "Fatal" prefixed error from dovecot-lda, it's nearly always tempfail (75). If it crashes, the error is different, but then it wouldn't be able to log the error anyway. > Will check the path out. Any way of forcing dovecot to return tempfail for the > tests? Preferrably only for one user :-) Run a script instead, something like: #!/bin/sh if [ $USER = "foo" ]; then exit 75 else exec .../dovecot-lda $@ fi From janfrode at tanso.net Wed Feb 22 10:25:36 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 09:25:36 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <20120222082536.GA8678@dibs.tanso.net> On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: > > I think the original question was still sensible. In your case it > seems like the ping times are identical between: > webmail -> imap-proxy > webmail -> imap server > > I think your results show that a proxy has little (or negative) > benefit in this situation, but it seems feasible that a proxy could > eliminate several RTT trips in the event that the proxy is "closer" > than the imap server? This might happen if say the imap server is > in a different datacenter (webmail on an office server machine?) The webmail/imapproxy were actually running in a different datacenter to the dovecot director/backend servers, but only about 20KM away. Ping tests: webmail->director: rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms director->backend: rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms webmail->localhost: rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms -jf From lists at wildgooses.com Wed Feb 22 11:31:55 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:31:55 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120222082536.GA8678@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> Message-ID: <4F44B60B.4000105@wildgooses.com> On 22/02/2012 08:25, Jan-Frode Myklebust wrote: > On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: >> I think the original question was still sensible. In your case it >> seems like the ping times are identical between: >> webmail -> imap-proxy >> webmail -> imap server >> >> I think your results show that a proxy has little (or negative) >> benefit in this situation, but it seems feasible that a proxy could >> eliminate several RTT trips in the event that the proxy is "closer" >> than the imap server? This might happen if say the imap server is >> in a different datacenter (webmail on an office server machine?) > The webmail/imapproxy were actually running in a different datacenter to > the dovecot director/backend servers, but only about 20KM away. > > Ping tests: > > webmail->director: > > rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms > > director->backend: > > rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms > > webmail->localhost: > > rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms > > > -jf Hmm, not sure I understand the original numbers then? It seems intuitive that the proxy installed locally could save you 2x RTT increment, which is about 0.8ms in your case. So I might expect the proxy to reduce rendering times by around 1.6ms simply because it reduces the number of round trips to login? Kind of curious why that's not achieved..? Cheers Ed W From lists at wildgooses.com Wed Feb 22 11:38:47 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:38:47 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> Message-ID: <4F44B7A7.6030303@wildgooses.com> On 21/02/2012 20:36, Timo Sirainen wrote: > On 21.2.2012, at 16.33, Ed W wrote: > >> I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). > You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: > > void postlogout_init(struct module *module) { } > void postlogout_deinit(void) { > system("/usr/local/bin/dovecot-postlogout.sh"); > } > > Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? Cheers Ed W From janfrode at tanso.net Wed Feb 22 11:48:46 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 10:48:46 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B60B.4000105@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> <4F44B60B.4000105@wildgooses.com> Message-ID: <20120222094846.GA10314@dibs.tanso.net> On Wed, Feb 22, 2012 at 09:31:55AM +0000, Ed W wrote: > > It seems intuitive that the proxy installed locally could save you > 2x RTT increment, which is about 0.8ms in your case. So I might > expect the proxy to reduce rendering times by around 1.6ms simply > because it reduces the number of round trips to login? Kind of > curious why that's not achieved..? Each http-request can probably trigger several IMAP requests. Maybe these work better in parallel directly to dovecot, than serialized (?) trough the imapproxy ? No idea if that's what's happening... or maybe the imapproxy just adds more overhead than the 2xRTT + imap logins it's supposed to save us ? -jf From tss at iki.fi Wed Feb 22 21:49:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 21:49:54 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B7A7.6030303@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> Message-ID: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> On 22.2.2012, at 11.38, Ed W wrote: >> void postlogout_init(struct module *module) { } >> void postlogout_deinit(void) { >> system("/usr/local/bin/dovecot-postlogout.sh"); >> } >> >> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. > > Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. From clube03 at gmail.com Wed Feb 22 22:49:01 2012 From: clube03 at gmail.com (Gustavo) Date: Wed, 22 Feb 2012 18:49:01 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <4F37A377.80707@Media-Brokers.com> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> <4F37A377.80707@Media-Brokers.com> Message-ID: Ok, I do some changes and now I can send mails but not recieve mail. On log I get this message: Feb 22 20:02:04 localhost dovecot: deliver(teste at clubedolimite.com.br): Fatal: setgid(12(man)) failed with euid=150(vmail), gid=8(mail), egid=8(mail): Operation not permitted (This binary should probably be called with process group set to 12(man) instead of 8(mail)) Where is the problem now? Thanks! On Sun, Feb 12, 2012 at 9:33 AM, Charles Marcus wrote: > On 2012-02-11 1:46 PM, Gustavo wrote: > >> Ok, I study the wikki and the dovecot works. But when I try to send a mail >> using squirrel, I get this error on logs: >> >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from >> localhost.localdomain[127.0.0.**1] >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: >> xsasl_cyrus_server_get_**mechanism_list: no applicable SASL mechanisms >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL >> authentication >> mechanisms >> > > Sorry, but this has nothing to do with dovecot, unless your intention is > to use dovecot-sasl with your postfix... it appears you are using > cyrus-sasl and postfix on your smtp server, that is where your problem lies. > > Easiest would be to configure postfix to use dovecot-sasl: > > http://wiki2.dovecot.org/**HowTo/PostfixAndDovecotSASL > > -- > > Best regards, > > Charles > -- Gustavo - clube03 at gmail.com From lists at wildgooses.com Thu Feb 23 00:54:53 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 22:54:53 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> Message-ID: <4F45723D.2090003@wildgooses.com> On 22/02/2012 19:49, Timo Sirainen wrote: > On 22.2.2012, at 11.38, Ed W wrote: > >>> void postlogout_init(struct module *module) { } >>> void postlogout_deinit(void) { >>> system("/usr/local/bin/dovecot-postlogout.sh"); >>> } >>> >>> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. >> Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. >> >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. Do you think you could keep something similar on your low priority backlog? Clearly parsing log files or hacking the code is possible, but I think the interest in the login scripting shows there is general interest, and having a full log of logon/logoff/bytes is clearly interesting to more minority users? >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. I think SWIG is for wrapping dovecot's api into the scripting language? (ie you could call dovecot methods from say perl/python/etc). What I had in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot normally calls a plugin method it will also run any [lua] scripts. I'm sure you know how to use google, but just so we are on the same page, top hit (below) from google shows how straight forward this is (lua has been built to be extremely fast and easy to embed, ie it's not an arbitrary choice) http://heavycoder.com/tutorials/lua_embed.php Cheers Ed W From tss at iki.fi Thu Feb 23 01:41:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 01:41:18 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F45723D.2090003@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> Message-ID: <1329954078.11500.266.camel@innu> On Wed, 2012-02-22 at 22:54 +0000, Ed W wrote: > >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > > Do you think you could keep something similar on your low priority > backlog? Clearly parsing log files or hacking the code is possible, but > I think the interest in the login scripting shows there is general > interest, and having a full log of logon/logoff/bytes is clearly > interesting to more minority users? What do you need the statistics for? I could make imap_client and pop3_client support some virtual methods, like user.destroy() initially, which would be enough for your use. I guess I could add that for v2.2. > >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. > > I think SWIG is for wrapping dovecot's api into the scripting language? > (ie you could call dovecot methods from say perl/python/etc). What I had > in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot > normally calls a plugin method it will also run any [lua] scripts. Yes, but I think SWIG can do that too. You'll need Dovecot's API implemented for scripting language if you want to do anything useful in the embedded script. That's the biggest job actually. Actually embeddeding some scripting language to Dovecot processes would be simple, they just couldn't really do anything useful. Also the nice thing about generically implementing Dovecot's APIs means that they could be used to build independent Dovecot applications, not just plugins. > I'm sure you know how to use google, but just so we are on the same > page, top hit (below) from google shows how straight forward this is > (lua has been built to be extremely fast and easy to embed, ie it's not > an arbitrary choice) > http://heavycoder.com/tutorials/lua_embed.php I've heard LUA being a commonly used embedded language, but I'd prefer to instead support several very widely used languages, such as Perl/Python. From lists at wildgooses.com Thu Feb 23 01:56:28 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 23:56:28 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <4F4580AC.5030600@wildgooses.com> On 22/02/2012 23:41, Timo Sirainen wrote: > I've heard LUA being a commonly used embedded language, but I'd prefer > to instead support several very widely used languages, such as > Perl/Python. I'm a perl/ruby fan myself, but I would still recommend a good look at lua (or python) simply because they seem to be performant, easy to use, and on the surface seem to have had some thought about making them "embeddable". My new favourite editor "Sublime Text 2" has python as it's scripting language. Lua has been used for some big name games amongst other things. Perl has some memory management issues if you leave it long running, also writing XS code looks ok on the surface, but is an exercise in hair pulling in practice.... Ruby is a beautiful language, but unsure of how easy to embed and speed + memory management is an unknown (for high performance applications) I think it has potential though. I think a lot of the current plugins on the website could easily be rewritten, likely without performance concerns, using a scripting based plugin system. I could see that some other big picture pieces could potentially benefit also Thanks for considering it Ed W From tss at iki.fi Thu Feb 23 02:56:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 02:56:29 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <1329958589.11500.273.camel@innu> On Wed, 2012-02-22 at 23:56 +0000, Ed W wrote: > On 22/02/2012 23:41, Timo Sirainen wrote: > > I've heard LUA being a commonly used embedded language, but I'd prefer > > to instead support several very widely used languages, such as > > Perl/Python. > > I'm a perl/ruby fan myself, but I would still recommend a good look at > lua (or python) simply because they seem to be performant, easy to use, > and on the surface seem to have had some thought about making them > "embeddable". SWIG appears to generate Lua bindings just as well. But yes, I noticed Lua 5.1 has a feature called FFI that makes calling C functions quite easy (but then again, SWIG is supposed to do the same thing). > Perl has some memory management issues if you leave it long running, > also writing XS code looks ok on the surface, but is an exercise in hair > pulling in practice.... I've written a Perl plugin to irssi before directly with XS, and that's my main motivation for using some kind of an easy generator this time. It's way too much trouble to write any glue functions, especially when it works for only a single scripting language. From stephan at rename-it.nl Thu Feb 23 03:15:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 02:15:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F442592.608@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> Message-ID: <4F459344.5020407@rename-it.nl> On 2/22/2012 12:15 AM, Adam Szpakowski wrote: > On 22.02.2012 00:09, Timo Sirainen wrote: >> Well, it would be possible to build a doveadm script that deletes the >> duplicates after delivery, but currently there's no implementation to >> avoid delivering duplicate Message-IDs in the first place. >> >> I don't really like such a Message-ID-based deduplication feature >> enabled by default, but something like this could be nice: >> >> fileinto :copy :x-deduplicate "boss"; >> >> Anyway, probably not going to be implemented anytime soon. > Maybe there is a way to use a procmail with something like this: > > :0 Wh: msgid.lock > | formail -D 8192 .msgid.cache > > But is there a safe way to use it together with sieve? Using > Pigeonhole Sieve Pipe Plugin? > There are a few options: * You can use Procmail as primary delivery agent and invoke dovecot-lda/sieve from within Procmail once Procmail has determined that it is not a duplicate. * Invoke procmail from Sieve using the pipe extension (i.e. the other way around). This has the disadvantage that Procmail will have to take care of final delivery, meaning the Dovecot indexes are not updated. * For Pigeonhole v0.3 there is the possibility to "filter" the message through Procmail using the sieve_extprograms plugin, but I haven't actually tested something like that. * I've just created an alternative that implements something similar to the Procmail code you posted above, but from within Sieve itself. It is a custom language extension called vnd.dovecot.duplicate and it adds the "duplicate" test. This test keeps track of which Message-IDs it has seen before in earlier deliveries and yields a true result if the message was seen before, e.g.: require "vnd.dovecot.duplicate"; if duplicate { discard; } Read the specification for details ("name" argument is not yet implemented): http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate/raw-file/4b1dbda4d3fc/doc/rfc/spec-bosch-sieve-duplicate.txt The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate This plugin is only a few hours old, experimental, and largely untested, so test it thoroughly before considering to use this. Read the INSTALL file for compile and installation instructions. Comments are welcome. Regards, Stephan. From atoth at atoth.sote.hu Thu Feb 23 07:20:26 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 06:20:26 +0100 Subject: [Dovecot] How to achieve proper privilege separation? Message-ID: I'm using a simple mbox config with regular Unix users and pam authentication. I'm also using grsecurity. That's why I see what dovecot does in which users' name. As times goes by and new versions are coming I can frustratedly see, that more and more tasks are performed as root. Why? When I used 1.x series of Dovecot, imap process started in the name of the user whose mbox was accessed. Now I can see, that nearly every task is performed by root. Why? It even tampers with the mail directories of each user as root instead of the user as it was usual long before. Please let me know how should I change the config to make dovecot stop using root privileges and revert it back to the old behavior. Thanks for your help: Dw. Here's my current config: dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = host.name last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at host.name protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = References: Message-ID: On 23.2.2012, at 7.20, T?th Attila wrote: > I'm using a simple mbox config with regular Unix users and pam > authentication. > > I'm also using grsecurity. That's why I see what dovecot does in which > users' name. As times goes by and new versions are coming I can > frustratedly see, that more and more tasks are performed as root. Why? Less tasks should be running as root now. The master process code is a lot smaller. > When I used 1.x series of Dovecot, imap process started in the name of the > user whose mbox was accessed. > Now I can see, that nearly every task is performed by root. Why? It even > tampers with the mail directories of each user as root instead of the user > as it was usual long before. The imap process starts as root, does a userdb lookup and then drops privileges to that user. It worked this way before too, only the userdb lookup code was done by master process. From atoth at atoth.sote.hu Thu Feb 23 10:03:37 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 09:03:37 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: Message-ID: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap accesses both the inbox and the mail directories of the user as root. Moreover, it creates the lock file as root. I can see no process running as the user. How could I teach dovecot to start the imap process as the user. What configuration options I should blame? Thx: Dw. dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster@ protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = On 23.2.2012, at 7.20, T?th Attila wrote: > >> I'm using a simple mbox config with regular Unix users and pam >> authentication. >> >> I'm also using grsecurity. That's why I see what dovecot does in which >> users' name. As times goes by and new versions are coming I can >> frustratedly see, that more and more tasks are performed as root. Why? > > Less tasks should be running as root now. The master process code is a lot > smaller. > >> When I used 1.x series of Dovecot, imap process started in the name of >> the >> user whose mbox was accessed. >> Now I can see, that nearly every task is performed by root. Why? It even >> tampers with the mail directories of each user as root instead of the >> user >> as it was usual long before. > > The imap process starts as root, does a userdb lookup and then drops > privileges to that user. It worked this way before too, only the userdb > lookup code was done by master process. > > From tss at iki.fi Thu Feb 23 10:15:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:15:12 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Message-ID: <1329984912.11500.277.camel@innu> On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: > Unfortunately I can see, that in my case /usr/libexec/dovecot/imap > accesses both the inbox and the mail directories of the user as root. > Moreover, it creates the lock file as root. I can see no process running > as the user. > > How could I teach dovecot to start the imap process as the user. What > configuration options I should blame? Well, that's strange. There shouldn't be any way for you to make imap access mails as root, even if you wanted to do that. If you log in as root, it'll fail with: Error: user root: Invalid settings in userdb: userdb returned 0 as uid Fatal: Invalid user settings. Refer to server log for more information. If there's a bug and it just somehow manages to get through that check, it fails with: Fatal: We couldn't drop root privileges So.. I'm not really sure what could be wrong. It makes me think maybe Gentoo's hardening features somehow mess this up, but I can't really think of how that could either. Set auth_debug=yes and mail_debug=yes. What does it log when logging in? From tss at iki.fi Thu Feb 23 10:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:54:24 +0200 Subject: [Dovecot] v2.1.1 released Message-ID: <1329987264.11500.282.camel@innu> http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz.sig v2.1.0 had a couple of bugs, but overall people have been pretty quiet about it. I'm hoping v2.1.2 won't need to be released for many more weeks. :) BTW. v2.2 branch has also been opened in hg. Not much stuff in there yet though. + dsync: If message with same GUID is saved multiple times in session, copy it instead of re-saving. - acl plugin + autocreated mailboxes crashed when listing mailboxes - doveadm force-resync: Don't skip autocreated mailboxes (especially INBOX). - If process runs out of fds, stop listening for new connections only temporarily, not permanently (avoids hangs with process_limit=1 services) - auth: passdb imap crashed for non-login authentication (e.g. smtp). From campbell at cnpapers.com Thu Feb 23 17:16:34 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:16:34 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process Message-ID: <4F465852.6030506@cnpapers.com> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. Now I see where messages are in my maillog of the type: auth: pam_unix(dovecot:auth) : authentication failure ........ Why is dovecot involved in my smtp processes and how do I fix this. I've got some very mad users. The 10-auth.conf file is pretty much stock except for allowing plain text logins. steve campbell From as at 3a.pl Thu Feb 23 17:30:12 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:30:12 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465B84.6090300@3a.pl> On 23.02.2012 16:16, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail and > sasl2 stuff seems to be proper, but the user can't use the system on > port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix this. > I've got some very mad users. The 10-auth.conf file is pretty much > stock except for allowing plain text logins. Dovecot can be used as authentication source. It has its own implementation of SASL library which can be used instead of Cyrus SASL implementation. In Dovecot2 the config for auth is in 10-master.conf, service "auth". Please see: for dovecot 1.x: http://wiki.dovecot.org/Sasl for dovecot 2.x: http://wiki2.dovecot.org/Sasl on the postfix side: http://www.postfix.org/SASL_README.html#server_dovecot -- Adam Szpakowski From as at 3a.pl Thu Feb 23 17:37:46 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:37:46 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F465D4A.1070005@3a.pl> On 23.02.2012 02:15, Stephan Bosch wrote: > On 2/22/2012 12:15 AM, Adam Szpakowski wrote: >> Maybe there is a way to use a procmail with something like this: >> >> :0 Wh: msgid.lock >> | formail -D 8192 .msgid.cache >> >> But is there a safe way to use it together with sieve? Using >> Pigeonhole Sieve Pipe Plugin? >> > There are a few options: > > * You can use Procmail as primary delivery agent and invoke > dovecot-lda/sieve from within Procmail once Procmail has determined > that it is not a duplicate. This is great option but if I understand correctly not when Postfix is using LMTP to talk to Dovecot. In this client of mine, the Postfix (MDA) and Dovecot (MAA) are in different physical locations and LMTP is tunnelled via SSH. > * I've just created an alternative that implements something similar > to the Procmail code you posted above, but from within Sieve itself. [cut] Great extension. I'll try and use it on one of the servers as soon as possible. Thanks for the work. BTW... There is another solution to my problem, which ends up to be implemented in this case. I'm using Postfix to BCC all mails on the domain level to the "archive" account. In this account I've set up all the logic which ends calling "redirect :copy". This solution eliminates copies from multiple To/CC. -- Adam Szpakowski From campbell at cnpapers.com Thu Feb 23 17:47:40 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:47:40 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465B84.6090300@3a.pl> References: <4F465852.6030506@cnpapers.com> <4F465B84.6090300@3a.pl> Message-ID: <4F465F9C.4080600@cnpapers.com> On 2/23/2012 10:30 AM, Adam Szpakowski wrote: > On 23.02.2012 16:16, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the system >> on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix this. >> I've got some very mad users. The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > Dovecot can be used as authentication source. It has its own > implementation of SASL library which can be used instead of Cyrus SASL > implementation. > > In Dovecot2 the config for auth is in 10-master.conf, service "auth". > > Please see: > for dovecot 1.x: http://wiki.dovecot.org/Sasl > for dovecot 2.x: http://wiki2.dovecot.org/Sasl > > on the postfix side: > http://www.postfix.org/SASL_README.html#server_dovecot > That's all great, but what about sendmail. What should my pam.d/dovecot file look like, what should my dovecot.conf file look like, all this just runs me around in circles. Thanks for the pointers, but I've been over that a million times and nothing seems to work. steve From CMarcus at Media-Brokers.com Thu Feb 23 17:47:25 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 10:47:25 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465F8D.2020402@Media-Brokers.com> On 2012-02-23 10:16 AM, Steve Campbell wrote: > Why is dovecot involved in my smtp processes Because you told it to? > and how do I fix this. Depends on what your intent is - what MTA you use - how it is configured - etc... > I've got some very mad users. Don't make changes to an existing and/or go live with a new system without properly testing? > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. This file is irrelevant unless you have your MTA configured to use dovecot-sasl... dovecot only *assists* an MTA in authenticating, it isn't an MTA. -- Best regards, Charles From stephan at rename-it.nl Thu Feb 23 18:14:57 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 17:14:57 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F465D4A.1070005@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> <4F465D4A.1070005@3a.pl> Message-ID: <4F466601.6000303@rename-it.nl> Op 2/23/2012 4:37 PM, Adam Szpakowski schreef: > On 23.02.2012 02:15, Stephan Bosch wrote: >> * I've just created an alternative that implements something similar >> to the Procmail code you posted above, but from within Sieve itself. > [cut] > > Great extension. I'll try and use it on one of the servers as soon as > possible. Thanks for the work. > > BTW... > There is another solution to my problem, which ends up to be > implemented in this case. > I'm using Postfix to BCC all mails on the domain level to the > "archive" account. In this account I've set up all the logic which > ends calling "redirect :copy". > This solution eliminates copies from multiple To/CC. Ok, please keep me uprised. Once it is tested a bit more I'll add it to the main Pigeonhole package. This plugin does not add any dependencies, so this is no problem whatsoever. Regards, Stephan. From campbell at cnpapers.com Thu Feb 23 18:15:21 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:15:21 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465F8D.2020402@Media-Brokers.com> References: <4F465852.6030506@cnpapers.com> <4F465F8D.2020402@Media-Brokers.com> Message-ID: <4F466619.1090103@cnpapers.com> On 2/23/2012 10:47 AM, Charles Marcus wrote: > On 2012-02-23 10:16 AM, Steve Campbell wrote: >> Why is dovecot involved in my smtp processes > > Because you told it to? Where did I tell it to do this. > >> and how do I fix this. > > Depends on what your intent is - what MTA you use - how it is > configured - etc... > Sendmail - pretty much default except I'm trying to use port submission (587). >> I've got some very mad users. > > Don't make changes to an existing and/or go live with a new system > without properly testing? > Unfortunately, switching over to a new server didn't show me most of the problems until after it had run a while. Dovecot is probably a great application, but the wiki sucks when you run sendmail, the help files are a little vague and run me around in circles. I postponed (actually move the server back to the old server) for a week to fix the problems I ran into upon first putting it into server. After fixing those, it was only later that the last batch of problems showed up. You right that I should have tested more, but that's not what I need to be hearing right now. Most of my conversions go without problems and I've been doing this for quite a few years (decades). >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > > This file is irrelevant unless you have your MTA configured to use > dovecot-sasl... dovecot only *assists* an MTA in authenticating, it > isn't an MTA. > My sendmail configs don't even list dovecot. steve From alexwbaule at gmail.com Thu Feb 23 18:19:30 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 23 Feb 2012 14:19:30 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin Message-ID: Hi Timo.... I Follow your instruction to look inside quota plugin to get the notification from dovecot that expunged a message, that's Ok. Follow the quota, it's use a array to keep all uids from a email expunged, to compare with uid that is got in notify_sync, that's OK too, but to me i need to keep uid and a char* with my body name, to erase it too. I made this: struct emexis_ids_x_uis{ uint32_t ids; char *uis_file; }; struct emexis_mailbox { union mailbox_module_context module_ctx; ARRAY_DEFINE(expunge_uids, struct emexis_ids_x_uis); }; ok, inside my expunged rewrite function i create the array like quota does, and append it to my array. struct emexis_ids_x_uis append_uis; append_uis.ids = _mail->uid; append_uis.uis_file = bodyFile; array_append(&ebox->expunge_uids, &append_uis, 1); But when i do a foreach in this Array, my ids is OK, but my uis_file has tha same value every time (the value is the lasted value inserted). Append to UID array (156) --> (/storage/gss/emexis/common/t/33/88/t33889eb1bccd28fd6643ebc3ecf548b12a1ad095) Append to UID array (157) --> (/storage/gss/emexis/common/t/e3/35/te335c20e978e12d37ad0a35ae009245cd34080c0) Append to UID array (158) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Calling Sync Expunged --> (156) Sync Expunged (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged Match (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (157 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (158 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Finish Calling Sync Expunged --> (156) Can i use a struct as a array Item ?? something is wrong with my array creation, append and etc ? Tks Timo !! From pgiessel at mac.com Thu Feb 23 18:23:28 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 07:23:28 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> > > I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. From kevingaspard at lavabit.com Thu Feb 23 18:26:45 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:26:45 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F4668C5.1020902@lavabit.com> Hi, I've a problem with Dovecot and Thunderbird. Actually I'm trying to setup a virtual host server mail with postfix + dovecot (2.0.9) on CentOS 6.2. This is my dovecot -n : http://pastebin.com/wZWAE5Cj (I guess the errors are not important, that's just saying there is some obsoletes values, I will fix that later until it's not inconvenient to setup correctly dovecot). I've a file to store password (/etc/dovecot/passwd) and when I'm trying to connect with telnet to my host on port 143 it works, after I can log myself with the command ?a login user at domain.tld passwd? and it works, this is the entire output of the telnet connection : http://pastebin.com/h10cQVqf but actually when I'm trying to connect with thunderbird I've an error about the password and the login, this is a screenshoot : http://images.koshie.fr/TB_problem.png and to finish, there is nothing into the log... The password and the user are corrects ! Of course, dovecot and postfix are started, this is the netstat -a | grep "dovecot" : http://pastebin.com/bwfsS4Tf If you need more informations tell me. PS : Excuse me for my bad english, also I'm new with dovecot. Cordially, GASPARD K?vin From rob0 at gmx.co.uk Thu Feb 23 18:33:40 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 10:33:40 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <20120223163340.GM24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail > and sasl2 stuff seems to be proper, but the user can't use the > system on port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix > this. I would question that these failures are in fact related to what Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it does not, therefore there is no way that Dovecot could possibly interfere with SMTP AUTH in Sendmail. > I've got some very mad users. And you are jumping to conclusions. I suggest that you take this matter to a Sendmail forum. When you do, provide all relevant configuration as well as complete logging to show the problem. No useful help is possible with what you posted here. > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From campbell at cnpapers.com Thu Feb 23 18:36:07 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:36:07 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> References: <4F465852.6030506@cnpapers.com> <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> Message-ID: <4F466AF7.5030009@cnpapers.com> On 2/23/2012 11:23 AM, Peter A. Giessel wrote: >> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. > My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. You're correct. This server is our smtp, imap, and pop server. All authentication is done there, supposedly through PAM, but dovecot seems to interfere with sendmail's smtp auth processes and overrides what typically was done in a non-interferred way with sasl. > From kevingaspard at lavabit.com Thu Feb 23 18:36:33 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:36:33 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F466B11.6020906@lavabit.com> I'm sorry, some one tell me my netstat (on IRC) is useless. Also my pastes will be destroyed in something like one day, that's stupid (that's originally for IRC), so this is the output of my : dovecot -n : http://pastebin.com/Tewcb7bm telnet : http://pastebin.com/6gY2puxF netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf Sorry for the error. From sm at resistor.net Thu Feb 23 18:34:46 2012 From: sm at resistor.net (SM) Date: Thu, 23 Feb 2012 08:34:46 -0800 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <6.2.5.6.2.20120223083039.09229478@resistor.net> Hi Steve, At 07:16 23-02-2012, Steve Campbell wrote: >I've been trying to get smtp auth set up for days. All my sendmail >and sasl2 stuff seems to be proper, but the user can't use the >system on port 587, which is where I require authorization. > >Now I see where messages are in my maillog of the type: > >auth: pam_unix(dovecot:auth) : authentication failure ........ > >Why is dovecot involved in my smtp processes and how do I fix this. >I've got some very If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Regards, -sm From campbell at cnpapers.com Thu Feb 23 19:10:20 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 12:10:20 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <20120223163340.GM24983@harrier.slackbuilds.org> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> Message-ID: <4F4672FC.1080802@cnpapers.com> On 2/23/2012 11:33 AM, /dev/rob0 wrote: > On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the >> system on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix >> this. > I would question that these failures are in fact related to what > Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it > does not, therefore there is no way that Dovecot could possibly > interfere with SMTP AUTH in Sendmail. Why is sendmail using Dovecot sasl when I have the regular sasl set up. If I turn off dovecot, telnet to 587, I get no failures in my logs. If I turn on dovecot and do the same, I get auth failures in my secure file. > >> I've got some very mad users. > And you are jumping to conclusions. I suggest that you take this > matter to a Sendmail forum. When you do, provide all relevant > configuration as well as complete logging to show the problem. No > useful help is possible with what you posted here. In other words, don't use sendmail if I use dovecot? > >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. I'm really having problems following the logic here. Seems that postfix and dovecot are the only way to go if I use alternate ports with smtp auth. Is that what everyone is implying? I'll try to see what sendmail guys are saying, but I don't think they'll provide much as long as it involves dovecot. Thanks for the help steve From pgiessel at mac.com Thu Feb 23 19:12:53 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 08:12:53 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <6.2.5.6.2.20120223083039.09229478@resistor.net> References: <4F465852.6030506@cnpapers.com> <6.2.5.6.2.20120223083039.09229478@resistor.net> Message-ID: <115F4A79-2861-4B1F-AB82-35B21F2C9091@mac.com> > > If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Not totally true. Typical cyrus sasl2 implementation used with sendmail uses an imap server for smtp authentication. I recall having to tweak the sasl2 configuration on my own sendmail server to talk correctly with dovecot. I don't recall the exact tweaks. I suggest reviewing the sasl2 implementation's documentation (the OP has not to my knowledge stated which sasl implementation he is using). It sounds like an sasl2 problem, not necessarily a dovecot problem. From keith1christian at gmail.com Thu Feb 23 19:31:49 2012 From: keith1christian at gmail.com (Keith Christian) Date: Thu, 23 Feb 2012 10:31:49 -0700 Subject: [Dovecot] Settings for authentication without an extra password file Message-ID: Hello, I'm new to dovecot, and, due to having an older version of an OS installed on a test server, I'll be using an older version of dovecot (dovecot-1.0-1.2) to provide IMAP services. On this wiki page (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) There are instructions for creating a separate dovecot password file: echo "$USER:{PLAIN}password" > passwd.dovecot sudo mv passwd.dovecot /etc So, it appears that two password files will be needed, one for dovecot and one for regular system logins. Is the statement above correct, or am I misreading something? Is there a setting in dovecot.conf which will authenticate IMAP using /etc/password and /etc/shadow, without a separate dovecot password file? ========Keith From rob0 at gmx.co.uk Thu Feb 23 19:41:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 11:41:55 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <20120223174155.GO24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 12:10:20PM -0500, Steve Campbell wrote: > On 2/23/2012 11:33 AM, /dev/rob0 wrote: > >On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > >>Why is dovecot involved in my smtp processes and how do I fix > >>this. > >I would question that these failures are in fact related to what > >Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK > >it does not, therefore there is no way that Dovecot could possibly > >interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set > up. Fortunately it seems that Peter has identified the issue: Cyrus SASL being configured to use IMAP for authentication. snip > In other words, don't use sendmail if I use dovecot? I didn't say that at all, and did not mean to imply it. > I'm really having problems following the logic here. Seems that > postfix and dovecot are the only way to go if I use alternate ports > with smtp auth. Is that what everyone is implying? One thing I *did* say is that what you posted was inadequate to be able to provide real help. And it seems that your issue is only tangentially related to Dovecot. > I'll try to see what sendmail guys are saying, but I don't think > they'll provide much as long as it involves dovecot. As Peter said, consult the Cyrus SASL documentation. If your SASL will be using IMAP for authentication, you need to ensure that it does so correctly for your Dovecot IMAP. As an alternative, change how Cyrus SASL is configured. The usual suggestion for Sendmail users is to use the same data backend for Cyrus SASL and Dovecot. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Thu Feb 23 20:01:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 13:01:21 -0500 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird In-Reply-To: <4F466B11.6020906@lavabit.com> References: <4F466B11.6020906@lavabit.com> Message-ID: <4F467EF1.2030803@Media-Brokers.com> On 2012-02-23 11:36 AM, GASPARD K?vin wrote: > Also my pastes will be destroyed in something like one day, that's > stupid (that's originally for IRC), so this is the output of my : > > dovecot -n : http://pastebin.com/Tewcb7bm > telnet : http://pastebin.com/6gY2puxF > netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf > > Sorry for the error. Please don't do that... just post the output in the body of the email. For one it complicates things (people reading email from terminals will not be able to help you), and as well many people simply will not click URL shortened links like those in an email. There is no reason to not simply paste the contents into the email body. -- Best regards, Charles From atoth at atoth.sote.hu Thu Feb 23 20:53:28 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 19:53:28 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <767de6d20cc46bcbccef60bf1d21ab13.squirrel@atoth.sote.hu> http://dovecot.org/list/dovecot/2009-March/038435.html -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 18:10 id?pontban Steve Campbell ezt ?rta: > > > On 2/23/2012 11:33 AM, /dev/rob0 wrote: >> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >>> I've been trying to get smtp auth set up for days. All my sendmail >>> and sasl2 stuff seems to be proper, but the user can't use the >>> system on port 587, which is where I require authorization. >>> >>> Now I see where messages are in my maillog of the type: >>> >>> auth: pam_unix(dovecot:auth) : authentication failure ........ >>> >>> Why is dovecot involved in my smtp processes and how do I fix >>> this. >> I would question that these failures are in fact related to what >> Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it >> does not, therefore there is no way that Dovecot could possibly >> interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set up. > > If I turn off dovecot, telnet to 587, I get no failures in my logs. If I > turn on dovecot and do the same, I get auth failures in my secure file. >> >>> I've got some very mad users. >> And you are jumping to conclusions. I suggest that you take this >> matter to a Sendmail forum. When you do, provide all relevant >> configuration as well as complete logging to show the problem. No >> useful help is possible with what you posted here. > > In other words, don't use sendmail if I use dovecot? >> >>> The 10-auth.conf file is pretty much >>> stock except for allowing plain text logins. > > I'm really having problems following the logic here. Seems that postfix > and dovecot are the only way to go if I use alternate ports with smtp > auth. Is that what everyone is implying? > > I'll try to see what sendmail guys are saying, but I don't think they'll > provide much as long as it involves dovecot. > > Thanks for the help > > steve > From trashcan at odo.in-berlin.de Thu Feb 23 21:55:55 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Thu, 23 Feb 2012 20:55:55 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 23:03, Michael Grimm wrote: > On 21.02.2012, at 22:57, Timo Sirainen wrote: >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > Ok. Then I will try both servers upgraded to 2.1 within the next days. JFTR: It took me some time to find a working syntax ;-) My working 2.0.18 syntax threw the following error: vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(vmail): Fatal: Unknown print formatter: -u dsync-local(test): Error: read() from worker server failed: EOF Omitting '-f' or '-v -f' results in: vmail> dsync -v -u test ssh vmail at remote-host.tld dsync -v -u test dsync: illegal option -- u dsync-local(test): Error: read() from worker server failed: EOF Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test doveadm: illegal option -- v doveadm sync [-u |-A] [-S ] [-fR] [-m ] ok, following those hints: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm sync -u test ssh: Could not resolve hostname dsync-server: hostname nor servname provided, or not known dsync-local(test): Error: read() from worker server failed: EOF dsync-local(test): Error: read() from worker server failed: EOF Hmm, 'man doveadm' doesn't show 'sync', and 'man sync' shows old syntax. Luckily, I remembered finding an mail of yours in http://www.dovecot.org/list/dovecot/2012-January/062933.html : |>>> dsync mirror -> doveadm mirror |>>> dsync backup -> doveadm backup |>>> dsync server -> doveadm dsync-server (could be hidden from the doveadm commands list) | I did this now, with mirror -> sync. After some trial by error I finally found a working syntax: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test >>> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. >> >> Added it there. Therefore you might add a warning about different syntax as well? Please, don't get me wrong, I'm not complaining. This mail is JFTR, only. And, this mail might help others to save some time after an upgrade to 2.1 and those who mirror mailboxes sitting at different servers. Thanks and regards, Michael From atoth at atoth.sote.hu Thu Feb 23 21:56:39 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 20:56:39 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <1329984912.11500.277.camel@innu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> Message-ID: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> In the mean time I've upgraded to 2.1. I've enabled debug logging and logged in. I suspect that hardening features can be blamed for my problem. After booting a previous kernel the behavior was reverted. Here is what I got. As I can make it out it uses the proper user for the imap process according to the logs. Thx: Dw. Feb 23 20:49:39 atoth dovecot: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 23 20:50:12 atoth dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16584) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50264 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 3337879553 16584 1 0a36f4227122eb3d59466523e937b25b Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 3337879553 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16587, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:12 atoth dovecot: imap(atoth): Disconnected: Logged out in=44 out=747 Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16588) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50265 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 401211393 16588 1 59b6d569049f955f31991ac3cfb1f54c Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 401211393 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16589, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=42671 out=174898 Feb 23 20:50:14 atoth dovecot: auth: Debug: auth client connected (pid=16600) Feb 23 20:50:14 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50276 resp= Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:14 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:14 atoth dovecot: auth: Debug: master in: REQUEST 3933732865 16600 1 8382f23ff412178311e55bf74162e4cd Feb 23 20:50:14 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:14 atoth dovecot: auth: Debug: master out: USER 3933732865 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16601, secured Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=405 out=9240 -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 09:15 id?pontban Timo Sirainen ezt ?rta: > On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: >> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap >> accesses both the inbox and the mail directories of the user as root. >> Moreover, it creates the lock file as root. I can see no process running >> as the user. >> >> How could I teach dovecot to start the imap process as the user. What >> configuration options I should blame? > > Well, that's strange. There shouldn't be any way for you to make imap > access mails as root, even if you wanted to do that. If you log in as > root, it'll fail with: > > Error: user root: Invalid settings in userdb: userdb returned 0 as uid > Fatal: Invalid user settings. Refer to server log for more information. > > If there's a bug and it just somehow manages to get through that check, > it fails with: > > Fatal: We couldn't drop root privileges > > So.. I'm not really sure what could be wrong. It makes me think maybe > Gentoo's hardening features somehow mess this up, but I can't really > think of how that could either. > > Set auth_debug=yes and mail_debug=yes. What does it log when logging in? > > From marc at perkel.com Thu Feb 23 22:41:51 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 23 Feb 2012 12:41:51 -0800 Subject: [Dovecot] From Maildir back to smtp Message-ID: <4F46A48F.2040304@perkel.com> Not sure how this can be done. Right now I'm running a free backup MX record service. People point their high MX records to my servers and if they go down we store the email. When they come back up it's delivered. What I want to do is have a premium service that would allow them to view through IMAP the stored email. In order to do that I would have to deliver the email locally to a dovecot server - to a single account perhaps - and they can view/forward/delete etc. But - when their server comes back online I'd like to gather up all the delivered email stored in Maildir format and run it back into smtp for delivery. Has anyone even done something like this? Or is there an easier way that I'm missing? Thanks in advance From mailing at securitylabs.it Fri Feb 24 00:12:14 2012 From: mailing at securitylabs.it (mailing at securitylabs.it) Date: Thu, 23 Feb 2012 23:12:14 +0100 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46A48F.2040304@perkel.com> References: <4F46A48F.2040304@perkel.com> Message-ID: <4F46B9BE.3000801@securitylabs.it> Il 23/02/2012 21:41, Marc Perkel ha scritto: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > Hi, store how? In a queue with smtp server? > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. > If you store the messages in a queue you can configure your smtp (your MX) server to send a copy of all message it receive also to another mailbox, see always_bcc in posfix. From wilberth.perez at uady.mx Fri Feb 24 00:33:46 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Thu, 23 Feb 2012 16:33:46 -0600 Subject: [Dovecot] How to limit max number of connections for ip address Message-ID: <4F46BECA.3020208@uady.mx> Hi List: Does any one , knows if is possible configure dovecot for limit max number of connections for IP address? I would like to prevent future fork-bombing attacks for pop3 and imap login process in my email server. Our dovecot version is : 1.2.10 Thanks, Wilberth. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From ar-dovecotlist at acrconsulting.co.uk Fri Feb 24 01:03:34 2012 From: ar-dovecotlist at acrconsulting.co.uk (Andrew Richards) Date: 23 Feb 2012 23:03:34 +0000 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <201202232303.34310.ar-dovecotlist@acrconsulting.co.uk> On Thursday 23 February 2012 22:12:14 mailing at securitylabs.it wrote: > Il 23/02/2012 21:41, Marc Perkel ha scritto: > > Not sure how this can be done. > > > > Right now I'm running a free backup MX record service. People point > > their high MX records to my servers and if they go down we store the > > email. When they come back up it's delivered. > > Hi, store how? In a queue with smtp server? > > > What I want to do is have a premium service that would allow them to > > view through IMAP the stored email. In order to do that I would have > > to deliver the email locally to a dovecot server - to a single account > > perhaps - and they can view/forward/delete etc. > > > > But - when their server comes back online I'd like to gather up all > > the delivered email stored in Maildir format and run it back into smtp > > for delivery. Sounds like a job for serialmail, a sister package to qmail: Use serialmail to forward mail from a mailbox (in Maildir format which you have) to the primary MX when it's back up. I've provided a mailbox/forwarding secondary MX service for several clients using this method (esp. 'maildirsmtp' program from serialmail with cron). More info/help on the qmail or serialmail mailing lists; URL for serialmail is, http://cr.yp.to/serialmail.html (it includes man pages within the package) A couple of particular advantages of having secondary MX email in a mailbox as you suggest for forwarding: - Unlike mail in a queue, the messages don't expire, so if the primary MX takes forever to come back up that's fine (except for your disk storage) - Since the mail is in a mailbox you can access this with IMAP or POP3 as you suggest [until it's emptied when the primary MX returns] I've not considered running serialmail in an non-qmail environment, but it would probably still work for the purpose you have in mind if you're using a different MTA. > If you store the messages in a queue you can configure your smtp (your > MX) server to send a copy of all message it receive also to another > mailbox, see always_bcc in posfix. cheers, Andrew. From tss at iki.fi Fri Feb 24 01:21:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:21:44 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F46BECA.3020208@uady.mx> References: <4F46BECA.3020208@uady.mx> Message-ID: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> On 24.2.2012, at 0.33, Wilberth Perez wrote: > Does any one , knows if is possible configure dovecot for limit max > number of connections for IP address? > > I would like to prevent future fork-bombing attacks for pop3 and imap > login process in my email server. > > Our dovecot version is : 1.2.10 There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. From tss at iki.fi Fri Feb 24 01:26:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:26:08 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> Message-ID: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> On 23.2.2012, at 21.56, T?th Attila wrote: > In the mean time I've upgraded to 2.1. > I've enabled debug logging and logged in. > > I suspect that hardening features can be blamed for my problem. After > booting a previous kernel the behavior was reverted. OK. > Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, > gid=100, home=/home/atoth This says that the IMAP is running as UID 1000. The code that produces this is: i_debug("Effective uid=%s, gid=%s, home=%s", dec2str(geteuid()), dec2str(getegid()), home); So if the process is still creating files as root, the kernel is lying.. From tss at iki.fi Fri Feb 24 01:28:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:28:47 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: On 24.2.2012, at 1.26, Timo Sirainen wrote: > On 23.2.2012, at 21.56, T?th Attila wrote: > >> In the mean time I've upgraded to 2.1. >> I've enabled debug logging and logged in. >> >> I suspect that hardening features can be blamed for my problem. After >> booting a previous kernel the behavior was reverted. > > OK. BTW. It would be interesting to know if the attached patch causes it to fail with the hardening patches. -------------- next part -------------- A non-text attachment was scrubbed... Name: euid.diff Type: application/octet-stream Size: 532 bytes Desc: not available URL: From tss at iki.fi Fri Feb 24 01:32:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:32:06 +0200 Subject: [Dovecot] Settings for authentication without an extra password file In-Reply-To: References: Message-ID: <19807B92-11DB-4901-8909-CCFA6003FEB9@iki.fi> On 23.2.2012, at 19.31, Keith Christian wrote: > Hello, I'm new to dovecot, and, due to having an older version of an > OS installed on a test server, I'll be using an older version of > dovecot (dovecot-1.0-1.2) to provide IMAP services. I think Dovecot v2.x works mostly on the same OSes as v1.x. > On this wiki page > (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) > > There are instructions for creating a separate dovecot password file: > > echo "$USER:{PLAIN}password" > passwd.dovecot > sudo mv passwd.dovecot /etc > > So, it appears that two password files will be needed, one for dovecot > and one for regular system logins. Only if you want to. > Is there a setting in dovecot.conf which will authenticate IMAP using > /etc/password and /etc/shadow, without a separate dovecot password > file? Use passdb pam + userdb passwd (not passwd-file). See password + user databases under http://wiki.dovecot.org/Authentication From jtam.home at gmail.com Fri Feb 24 01:32:50 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 23 Feb 2012 15:32:50 -0800 (PST) Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: References: Message-ID: From: "Peter A. Giessel" : > Not totally true. Typical cyrus sasl2 implementation used with > sendmail uses an imap server for smtp authentication. I recall having > to tweak the sasl2 configuration on my own sendmail server to talk > correctly with dovecot. I don't recall the exact tweaks. Just to make it clear to the OP, I have a Dovecot/Sendmail/SASL working just fine (which is independent of dovecot). I suspect Peter hit the nail on the head here. Look at the contents of $SASL_INSTALL/lib/Sendmail.conf Mine is (only) pwcheck_method:saslauthd which I think looks up my name service backend via the login and plain plugins. I'm a little hazy on how this was set up, but if you are able to successfully login without dovecot, then it appears you have some sort of authentication stack that will try IMAP first, then fallback to your the other method. Joseph Tam From tss at iki.fi Fri Feb 24 01:34:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:34:48 +0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: References: Message-ID: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> On 23.2.2012, at 18.19, Alex Baule wrote: > ok, inside my expunged rewrite function i create the array like quota > does, and append it to my array. > > struct emexis_ids_x_uis append_uis; > append_uis.ids = _mail->uid; > append_uis.uis_file = bodyFile; .. > But when i do a foreach in this Array, my ids is OK, but my uis_file > has tha same value every time (the value is the lasted value > inserted). I guess the memory isn't permanently allocated for it, so you need to do: append_uis.uis_file = i_strdup(bodyFile); Note that you'll also need to later i_free() it to avoid leaking memory. From lists at wildgooses.com Fri Feb 24 01:41:37 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 23 Feb 2012 23:41:37 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <4F46CEB1.1040604@wildgooses.com> On 22/02/2012 23:56, Ed W wrote: > I think it has potential though. I think a lot of the current plugins > on the website could easily be rewritten, likely without performance > concerns, using a scripting based plugin system. I could see that > some other big picture pieces could potentially benefit also One interesting test case for such a scripting hooks solution might be login restrictions. There seem to be regular requests for the ability to setup arbitrarily complicated restrictions on users per IP, attempts per second, etc (and my logging interest is kind of related also). Not trying to bump the item up any todo lists, just trying to chuck in some concrete ideas for actually testing a specific implementation... I guess a substantially more performance orientated area that seems to get some interest would be various spam, expunge, delete ideas and the hooks needed for those. These seem much more tricky to implement a scripting hook and still stay performant. Again just ideas for real things people might want to do? Cheers Ed W From amk at krell.zikzak.de Fri Feb 24 03:22:47 2012 From: amk at krell.zikzak.de (Andreas M. Kirchwitz) Date: Fri, 24 Feb 2012 02:22:47 +0100 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build Message-ID: <20120224012247.GA6512@krell.zikzak.de> Hello Dovecot users! There seems to be a new dependency in some modules (eg, lib-storage, libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those modules didn't require OpenSSL, but 2.1 does. For the linking process the path to the OpenSSL library isn't specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't properly used as well.) Example for lib-storage: env SSL_CFLAGS="-I/usr/local/ssl/include" SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto -lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl --with-ssldir=/usr/local/${DOVECOTVER}/etc/dovecot/certs make gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-master -I../../src/lib-settings -I../../src/lib-mail -I../../src/lib-storage -DPKG_RUNDIR=\""/usr/local/Dovecot-2.1.1/var/run/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -MT master-connection.o -MD -MP -MF .deps/master-connection.Tpo -c -o master-connection.o master-connection.c mv -f .deps/master-connection.Tpo .deps/master-connection.Po /bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -no-undefined -Wl,--as-needed -o indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o ../../src/lib-storage/libdovecot-storage.la ../../src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -Wl,--as-needed -o .libs/indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o -Wl,--export-dynamic ../../src/lib-storage/.libs/libdovecot-storage.so ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/Dovecot-2.1.1/lib/dovecot /usr/bin/ld: warning: libcrypto.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libssl.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `OBJ_txt2nid' ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `SSL_set_ex_data' [...] # ldd src/lib*/.libs/*.so src/lib-dovecot/.libs/libdovecot.so: linux-gate.so.1 => (0x00979000) libdl.so.2 => /lib/libdl.so.2 (0x00c31000) librt.so.1 => /lib/librt.so.1 (0x009ed000) libc.so.6 => /lib/libc.so.6 (0x0054d000) /lib/ld-linux.so.2 (0x00c12000) libpthread.so.0 => /lib/libpthread.so.0 (0x008f4000) src/lib-lda/.libs/libdovecot-lda.so: linux-gate.so.1 => (0x0096d000) libdovecot-storage.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-storage/.libs/libdovecot-storage.so.0 (0x00ae4000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00110000) librt.so.1 => /lib/librt.so.1 (0x00189000) libc.so.6 => /lib/libc.so.6 (0x00192000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x0031c000) libpthread.so.0 => /lib/libpthread.so.0 (0x00ebc000) /lib/ld-linux.so.2 (0x003f8000) src/lib-sql/.libs/libdovecot-sql.so: linux-gate.so.1 => (0x00a95000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x002b6000) libdl.so.2 => /lib/libdl.so.2 (0x00ab8000) librt.so.1 => /lib/librt.so.1 (0x0072a000) libc.so.6 => /lib/libc.so.6 (0x008f7000) /lib/ld-linux.so.2 (0x00ebd000) libpthread.so.0 => /lib/libpthread.so.0 (0x00619000) src/lib-ssl-iostream/.libs/libdovecot-ssl.so: linux-gate.so.1 => (0x00da6000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x00df0000) librt.so.1 => /lib/librt.so.1 (0x00112000) libc.so.6 => /lib/libc.so.6 (0x00b5e000) /lib/ld-linux.so.2 (0x00e76000) libpthread.so.0 => /lib/libpthread.so.0 (0x005a6000) src/lib-storage/.libs/libdovecot-storage.so: linux-gate.so.1 => (0x006ed000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00c66000) libdl.so.2 => /lib/libdl.so.2 (0x00491000) librt.so.1 => /lib/librt.so.1 (0x00be2000) libc.so.6 => /lib/libc.so.6 (0x00110000) /lib/ld-linux.so.2 (0x0036b000) libpthread.so.0 => /lib/libpthread.so.0 (0x0029a000) Without SSL_CFLAGS and SSL_LIBS but just PKG_CONFIG_PATH, it also won't work. Help appreciated ... Andreas From alexwbaule at gmail.com Fri Feb 24 04:32:47 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 24 Feb 2012 00:32:47 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> References: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> Message-ID: Tks Timo... Em 23 de fevereiro de 2012 21:34, Timo Sirainen escreveu: > On 23.2.2012, at 18.19, Alex Baule wrote: > >> ok, inside my expunged rewrite function i create the array like quota >> does, and append it to my array. >> >> struct emexis_ids_x_uis append_uis; >> append_uis.ids = _mail->uid; >> append_uis.uis_file = bodyFile; > .. >> But when i do a foreach in this Array, my ids is OK, but my uis_file >> has tha same value every time (the value is the lasted value >> inserted). > > I guess the memory isn't permanently allocated for it, so you need to do: > > append_uis.uis_file = i_strdup(bodyFile); > > Note that you'll also need to later i_free() it to avoid leaking memory. > From CMarcus at Media-Brokers.com Fri Feb 24 14:18:43 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 07:18:43 -0500 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <4F478023.9080706@Media-Brokers.com> On 23/02/2012 21:41, Marc Perkel wrote: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. I'm thinking it would be possible to use dsync for this? Ideally you would have to have a way to maintain a duplicate of the AUTH database, so that users could use their current passwords for accessing it (this would obviously require the approval and cooperation of whoever you are providing this service to), but this way you could store each users mail individually and provide relatively painless access (since all each user would need to know is the backup host name and their username/password). You'd have to configure a script on your end to: 1. determine a reliable way to differentiate between a temp-fail situation on the primary (in which case you just act as a normal backup MX), or an extended outage (15 minutes? 1 hour? I'd probably let the customer decide this one), which would determine when you start actually taking delivery of their mail to the 'standby' server 2. monitor for when the primary comes back online 3. wait xx interval to make sure the primary stays up (again, I'd let the customer decide this, but with a sane minimum that you define) 4. once the primary stays up for the required interval, dsync a one way 'dump' of the stored mails, being sure to test for success - and this process would obviously have to take into account the possibility of the primary going down again in the middle of the dsync 5. mv successfully restored mails on the backup mx to a temp location for xx 'safety' interval 'just in case' 6. delete the backed-up/stored mail that has been successfully dsync'd after the safety interval has elapsed -- Best regards, Charles From troy at surety-networks.com Fri Feb 24 15:55:57 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 08:55:57 -0500 Subject: [Dovecot] Apple od/ad authentication problem Message-ID: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. OS X 10.7.3, behind a Cisco asa NAT. When trying to authenticate (ssl or not) via the webmail interface I get this error... NAT IP is the public ip I assigned. It doesn't work for an AD account or an account local to the Mac server. However it DOES work on the internal LAN. Feb 22 18:15:36 mail dovecot[3501]: auth: Error: od (username,natIP): verify plain: lookup failed for user: username Feb 22 18:15:51 mail dovecot[3501]: auth: Error: od[getpwnam_ext](username,natIP): No record for user I figured I'm missing something in the dovecot config. I have followed apple's kb articles for AD webmail authentication and kerberos, and that works internally. From przemek.orzechowski at makolab.pl Fri Feb 24 16:22:13 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q? Przemys=C5=82aw.Orzechowski ?=) Date: Fri, 24 Feb 2012 15:22:13 +0100 Subject: [Dovecot] iPhone and custom imap folders problem Message-ID: Hi Im using Dovecot 1.0.10 and everything seems work ok except one client He is accessing his mailbox using iPhone There are several custom folders created and mail is directed to them using procmail The problem is iPhone mail app only sees changes in INBOX and when new mail arrives to other folders it will not see it unlers user accesses this particular folder Is there some kind of workaround for this? Im quite sure its iPhones fault as all other email clients are workin perfectly fine. From jacob at alltechservices-ia.com Fri Feb 24 16:27:14 2012 From: jacob at alltechservices-ia.com (jacob at alltechservices-ia.com) Date: Fri, 24 Feb 2012 08:27:14 -0600 Subject: [Dovecot] Virtual Folders Message-ID: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Hello, I am trying to get virtual folders to work properly, and I can't seem to. I will be as detailed as possible to remove confusion. Thank you in advance. Goal: Essentially, a gmail clone, but one that I can make work on cPanel hosts with roundcube and squirrel mail, etc. By default, cpanel creates user mail dir structure of (maildir++) inbox, drafts, sent, junk, and trash (with no dovecot specific files until first login) I would like to replace all those folders with virtual representations with more capabilities. I would like to achieve the conversation view that gmail has pulled off so well. cPanel, stock, generates the following dovecot -n: (i am still not sure how it negotiates mail_location yet) # 1.2.17: /etc/dovecot.conf # OS: Linux 2.6.18-238.5.1.el5xen x86_64 CentOS release 5.7 (Final) ssl_cert_file: /etc/dovecot/ssl/dovecot.crt ssl_key_file: /etc/dovecot/ssl/dovecot.key ssl_cipher_list: ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login login_process_per_connection: no login_processes_count: 10 login_max_processes_count: 100 login_max_connections: 500 mail_max_userip_connections: 200 maildir_copy_preserve_filename: yes mail_plugins: quota imap_quota namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at example.com auth default: mechanisms: plain login cache_size: 2048 username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+% passdb: driver: checkpassword args: /usr/local/cpanel/bin/dovecot-auth userdb: driver: prefetch socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 plugin: quota: maildir quota_rule: Trash:ignore I started a test box with Debian running the same version of dovecot (1.2.x) and on that system I run into problems every time I make a virtual folder a root folder. I assume it is trying to look up a normal folder instead of the virtual ones and I am confusing it. So I came here because it would be really really neat if I could pull this off. Again, thank you in advance. -cornfeed From CMarcus at Media-Brokers.com Fri Feb 24 16:47:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 09:47:44 -0500 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47A310.9000308@Media-Brokers.com> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski wrote: > Im using Dovecot 1.0.10 Very old - an upgrade would likeley fix your problems... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From dg at dguhl.org Fri Feb 24 17:11:58 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 24 Feb 2012 16:11:58 +0100 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Message-ID: <20120224151158.GB16307@PC211.ikt.de> On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: > Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. > > OS X 10.7.3, behind a Cisco asa NAT. Deactivate the SMTP improvements your ASA helpfully provide. Dennis [..] From radecki.rafal at gmail.com Fri Feb 24 17:20:36 2012 From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=) Date: Fri, 24 Feb 2012 16:20:36 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. Message-ID: Hi all. I am currently gathering information about migration of Maildir/ structures from Courier to Dovecot. What do you think about http://wiki.dovecot.org/Migration/Courier ? For each courier Maildir account I want to use http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce dovecot-uidlist file mv courierimapsubscribed subscriptions sed sed -i 's/INBOX\.//' subscriptions rm -r courierimaphieracl rm courierimapacl rm courierimapkeywords I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf Should I look into something else? Have I missed something? Best regards, Rafal. From troy at surety-networks.com Fri Feb 24 17:43:03 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 10:43:03 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <45684F00-E4C4-4FD3-AD71-4E21B74C471A@surety-networks.com> Looking over the running config I don't see anything relating to smtp. I'm not using the esmtp inspection either. Am I looking in the wrong place? Would this cause the error I'm getting in mailaccess log? - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From micha at krausam.de Fri Feb 24 18:26:24 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 24 Feb 2012 17:26:24 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method Message-ID: <4F47BA30.2080704@krausam.de> Hi, I want to deliver mails directly to subfolders, based on address extensions. Example: Emails sent to: micha+test at example.com Should be stored into user micha, folder INBOX.test I was able to reproduce this with exim and dovecot-lda: command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" But I was wondering if I could do this within dovecot, which would enable me to use LMTP instead of dovecot-lda. Which brings me to my second question: Which method is "better" for dovecot, lda or LMTP? Micha Krause From gfinch at ldmltd.ca Fri Feb 24 18:36:00 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Fri, 24 Feb 2012 08:36:00 -0800 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47BC70.40308@ldmltd.ca> On 2012-02-24 6:22 AM, Przemys?aw.Orzechowski wrote: > Hi > > Im using Dovecot 1.0.10 and everything seems work ok except one client > He is accessing his mailbox using iPhone > There are several custom folders created and mail is directed to them > using procmail > The problem is iPhone mail app only sees changes in INBOX and when new > mail arrives to other folders it will not see it unlers user accesses this > particular folder > Is there some kind of workaround for this? > > Im quite sure its iPhones fault as all other email clients are workin > perfectly fine. As far as I know, that is a limitation of the iPhone mail client: it only IDLEs on the currently selected folder, and it only checks the currently selected folder for new messages. You will not see new mail for the other folders unless you select them. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From as at 3a.pl Fri Feb 24 18:46:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 24 Feb 2012 17:46:02 +0100 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <4F47A310.9000308@Media-Brokers.com> References: <4F47A310.9000308@Media-Brokers.com> Message-ID: <4F47BECA.4020104@3a.pl> On 24.02.2012 15:47, Charles Marcus wrote: > On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski > wrote: >> Im using Dovecot 1.0.10 > Very old - an upgrade would likeley fix your problems... A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. Only inbox is synced automatically. -- Adam Szpakowski From wilberth.perez at uady.mx Fri Feb 24 17:07:20 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Fri, 24 Feb 2012 09:07:20 -0600 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> Message-ID: <4F47A7A8.4090009@uady.mx> Hi Timo: My question was because constantly we received brute force attack from some of ip address which uses pop3 service to affect dovecot's login proccess. For example: Error: Temporary failure in creating login processes, slowing down for now pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A auth(default): Info: shadow(best,A.B.C.D): unknown user dovecot: Error: pipe() failed: Too many open files dovecot: Error: Temporary failure in creating login processes, slowing down for now In the log above from dovecto.log file, we observed a lot of conections from IP address A.B.C.D to our email server with ip address X.Y.Z.A using pop3 login process. Is possible prevent this type of attacks with any dovecot option (maybe limit the number of max connections from one ip address or maybe upgrading my dovecot version)? Thanks for you help and time. Wilberth. El 23/02/2012 05:21 p.m., Timo Sirainen escribi?: > On 24.2.2012, at 0.33, Wilberth Perez wrote: > >> Does any one , knows if is possible configure dovecot for limit max >> number of connections for IP address? >> >> I would like to prevent future fork-bombing attacks for pop3 and imap >> login process in my email server. >> >> Our dovecot version is : 1.2.10 > > There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. > > Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From marc at perkel.com Fri Feb 24 19:09:12 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 24 Feb 2012 09:09:12 -0800 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F478023.9080706@Media-Brokers.com> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> <4F478023.9080706@Media-Brokers.com> Message-ID: <4F47C438.2090300@perkel.com> On 2/24/2012 4:18 AM, Charles Marcus wrote: > On 23/02/2012 21:41, Marc Perkel wrote: >> Not sure how this can be done. >> >> Right now I'm running a free backup MX record service. People point >> their high MX records to my servers and if they go down we store the >> email. When they come back up it's delivered. >> >> What I want to do is have a premium service that would allow them to >> view through IMAP the stored email. In order to do that I would have >> to deliver the email locally to a dovecot server - to a single account >> perhaps - and they can view/forward/delete etc. >> >> But - when their server comes back online I'd like to gather up all >> the delivered email stored in Maildir format and run it back into smtp >> for delivery. > > I'm thinking it would be possible to use dsync for this? Ideally you > would have to have a way to maintain a duplicate of the AUTH database, > so that users could use their current passwords for accessing it (this > would obviously require the approval and cooperation of whoever you > are providing this service to), but this way you could store each > users mail individually and provide relatively painless access (since > all each user would need to know is the backup host name and their > username/password). > > You'd have to configure a script on your end to: > > 1. determine a reliable way to differentiate between a temp-fail > situation on the primary (in which case you just act as a normal > backup MX), or an extended outage (15 minutes? 1 hour? I'd probably > let the customer decide this one), which would determine when you > start actually taking delivery of their mail to the 'standby' server > > 2. monitor for when the primary comes back online > > 3. wait xx interval to make sure the primary stays up (again, I'd let > the customer decide this, but with a sane minimum that you define) > > 4. once the primary stays up for the required interval, dsync a one > way 'dump' of the stored mails, being sure to test for success - and > this process would obviously have to take into account the possibility > of the primary going down again in the middle of the dsync > > 5. mv successfully restored mails on the backup mx to a temp location > for xx 'safety' interval 'just in case' > > 6. delete the backed-up/stored mail that has been successfully dsync'd > after the safety interval has elapsed I was wondering if I could just gather up the maildir files, reprocess them into batch SMTP format and run them back into Exim? From miha.vrhovnik at cordia.si Fri Feb 24 19:10:58 2012 From: miha.vrhovnik at cordia.si (Miha Vrhovnik) Date: Fri, 24 Feb 2012 18:10:58 +0100 Subject: [Dovecot] =?iso-8859-1?q?iPhone_and_custom_imap_folders_problem?= In-Reply-To: <4F47BECA.4020104@3a.pl> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> Message-ID: <20120224181059.66377BF108F@miha> "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >On 24.02.2012 15:47, Charles Marcus wrote: >> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >> wrote: >>> Im using Dovecot 1.0.10 >> Very old - an upgrade would likeley fix your problems... >A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >Only inbox is synced automatically. > >-- >Adam Szpakowski So the solution is simple go bug Apple. Oh yeah. they say if something isn't implemented on Apple platform or it doesn't work, then you don't need it, because their devs know what's best for their users. Regards, M From julioarr at fisica.uh.cu Fri Feb 24 19:44:37 2012 From: julioarr at fisica.uh.cu (julioarr at fisica.uh.cu) Date: Fri, 24 Feb 2012 12:44:37 -0500 Subject: [Dovecot] remove messages once downloaded Message-ID: I need some help with the dovecot configuration. I want to remove downloaded messages from Mail Server once the messages have been successfully downloaded by pop3-clients, even when the clients have been configured to save copy of messages in the Server. thanks, Julio From trashcan at odo.in-berlin.de Fri Feb 24 17:27:56 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 24 Feb 2012 16:27:56 +0100 Subject: [Dovecot] dsync 2.0.x versus 2.1.x field report Message-ID: <47ce619d5887f172ef96c56b24b19b0e@mx2.enfer-du-nord.net> Hi -- I used to run two mailservers with identical MX priorities (poor man's fail-over approach) with Maildir mail folders accessed by Dovecot. Synchronization has been achieved using unison, mirroring all files except dovecot index files. Once in a while some oddities could be observed, like reappearing of deleted mails, duplicates and such alike. But the all-over performance was quite good. Then I wanted to migrate to mdbox, and I became interested in dsync. I did start with 2.0.16 a couple of month ago, and recently upgraded to 2.0.18. My experience with the 2.0.x dsync version is as follows (synchronization time: every minute): -- As long as mail is not stored in corresponding mailboxes at both servers at the same time, all mailboxes become synchronized perfectly well, and no oddites like reappearing of deleted mails or generation of duplicates can be observed. -- If mail is stored into corresponding mailboxes at both servers simultaneously in between two synching runs, it gets worrying. -- If I do run dsync without '-f' those two mailboxes end up in becoming asynchronous. -- If I do run dsync with '-f' both mailboxes will become synchronized let's say in 95% of all synchronization runs. BUT in 5%, I do experience asynchronicity. Here a 'doveadm force-resync' sometimes works, but not always, what worries me a lot. -- I can see the missing mail in the mdbox storage file using UNIX' little helpers but not with the help of dovecot tools. (Or I was unable to use them properly.) -- Modifying MX priorities in a way, that I do now have a primary and backup mailserver scenario, very much reduced the risk of losing legitimate mail. -- But sometimes I do lose some syslog mail being produced at the same time. Therefore I modified runtimes in way to omit synchronicity. Ok, I could have lived with that, but I was anxious to learn, if 2.1.x dsync would perform better ;-) Here my observations sofar (synchronization time: every minute): -- No losses sofar using 'doveadm sync' with '-f' -- A *lot* of reappearing deleted mails or mails moved from one mailbox to the other. -- Quite often I am unable to delete those mails using client functionality (Mail.app, Roundcube). -- In those cases one needs to stop synchronization, run 'doveadm force-resync', delete all reappearing mail (quite a lot sometimes), empty 'Deleted' folders, and finally run 'doveadm purge'. -- Running of 'doveadm sync' without '-f' doesn't improve the situation. Still I do have to struggle with reappearing mails, and sometimes with appearing duplicates. -- I do have a test setup running for 24 hours now, where I do generate test mails in a 6 minutes interval at both servers. Mails become synchronized perfectly well, irrespectively if '-f' is set or not. This is a major improvement to 2.0x. dsync (sofar). -- But running the synchronization without '-f' still shows a *lot* of reappering mail, though. I do have the feeling that dsync became very much improved with regard to missing mail in synchronization, good. But those reappearing mails make me think about switching back to 2.0.x dsync and live with the drawback of preventing mail to arrive synchronously at both servers. Is someone out there experiencing a similar behaviour with dsync 2.1.x regarding reappearing, thus indelible mail? Regards, Michael From CMarcus at Media-Brokers.com Fri Feb 24 21:29:26 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 14:29:26 -0500 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <4F47E516.3090101@Media-Brokers.com> On 2012-02-24 10:07 AM, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. This is quickly becoming an FAQ... If this is linux, and these attacks are from the *same* IP address(es) (sounds like that is the case), then your best option is something like fail2ban. If, however, they are from bots that change IP address each time, fail2ban won't be effective... -- Best regards, Charles From atoth at atoth.sote.hu Fri Feb 24 21:35:46 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Fri, 24 Feb 2012 20:35:46 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: It was probably caused by a bug already corrected. And not by dovecot. commit 4fd554e3a097b22c5049fcdc423897477deff5ef Author: Brad Spengler Date: Mon Feb 20 09:17:57 2012 -0500 Fix wrong logic on capability checks for switching roles, broke policies Thanks to Richard Kojedzinszky for reporting I'll test it with an updated kernel containing the fix. Thx: Dw. -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 24.(P) 00:28 id?pontban Timo Sirainen ezt ?rta: > On 24.2.2012, at 1.26, Timo Sirainen wrote: > >> On 23.2.2012, at 21.56, T?th Attila wrote: >> >>> In the mean time I've upgraded to 2.1. >>> I've enabled debug logging and logged in. >>> >>> I suspect that hardening features can be blamed for my problem. After >>> booting a previous kernel the behavior was reverted. >> >> OK. > > BTW. It would be interesting to know if the attached patch causes it to > fail with the hardening patches. > From dhenderson at mediatemple.net Sat Feb 25 00:49:50 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 14:49:50 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 Message-ID: Greetings, This email is both a request for assistance/help and a heads-up. [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. dovecot: imap-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: pop3-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/pop3-login(main+0x49) [0x401b99] dovecot: master: Error: service(pop3-login): child 27603 killed with signal 6 (core not dumped - add -D parameter to service pop3-login { executable } dovecot: master: Error: service(pop3-login): command startup failed, throttling dovecot: imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: imap-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/imap-login(main+0x39) [0x402069] dovecot: master: Error: service(imap-login): child 27604 killed with signal 6 (core not dumped - add -D parameter to service imap-login { executable } Once this patch was removed, everything started working again. Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? We haven't dug deeper yet, but the error is being thrown from the method io_loop_handle_add in ioloop-epoll.c http://hg.dovecot.org/dovecot-2.0/file/aa8dfa085a99/src/lib/ioloop-epoll.c Thanks Doug From tss at iki.fi Sat Feb 25 02:27:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:27:22 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <3A91726D-8D74-4AB2-ACAA-101376543133@iki.fi> On 24.2.2012, at 17.07, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. > > For example: > > dovecot: Error: pipe() failed: Too many open files > dovecot: Error: Temporary failure in creating login processes, slowing > down for now This is your main problem. Increase ulimit -n so Dovecot won't stop creating login processes (or alternatively decrease the max number of login processes). Newer Dovecot versions warn at startup if the fd limit is too low. From tss at iki.fi Sat Feb 25 02:39:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:15 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On 25.2.2012, at 0.49, Doug Henderson wrote: > [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. > > After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument .. > Once this patch was removed, everything started working again. > > Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). From tss at iki.fi Sat Feb 25 02:39:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:41 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Not possible. From tss at iki.fi Sat Feb 25 02:47:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:47:05 +0200 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <4F47BA30.2080704@krausam.de> References: <4F47BA30.2080704@krausam.de> Message-ID: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> On 24.2.2012, at 18.26, Micha Krause wrote: > I want to deliver mails directly to subfolders, based on address extensions. .. > I was able to reproduce this with exim and dovecot-lda: > > command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" > > > But I was wondering if I could do this within dovecot, which would enable me to use > LMTP instead of dovecot-lda. lmtp_save_to_detail_mailbox = yes > Which brings me to my second question: > > Which method is "better" for dovecot, lda or LMTP? LMTP is often easier to setup and faster. From tss at iki.fi Sat Feb 25 02:50:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:50:34 +0200 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: On 24.2.2012, at 17.20, Rafa? Radecki wrote: > I am currently gathering information about migration of Maildir/ > structures from Courier to Dovecot. > What do you think about http://wiki.dovecot.org/Migration/Courier ? > For each courier Maildir account I want to use > http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce > dovecot-uidlist file > mv courierimapsubscribed subscriptions > sed sed -i 's/INBOX\.//' subscriptions The migration script migrates subscriptions, so those shouldn't be necessary. > rm -r courierimaphieracl > rm courierimapacl You don't mind losing any ACLs? > I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). From msidoric at mac.com Sat Feb 25 03:07:27 2012 From: msidoric at mac.com (Michael Sidoric) Date: Fri, 24 Feb 2012 19:07:27 -0600 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: > > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. timo is quite correct -- consider imap instead, that's why the protocol exists. From ml at smtp.fakessh.eu Sat Feb 25 04:42:27 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 03:42:27 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question Message-ID: <1330137747.13977.3.camel@localhost> hi guys hi master of "Fu" hi Timo I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I encounter fatal errors. I created it myself rpms available here http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm I can not run dovecot with this config file like this http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm i run this config ks37777 ~]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1 vsz_limit = 64 M } service imap { client_limit = 1024 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From troy at surety-networks.com Sat Feb 25 04:47:59 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 21:47:59 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <02C084C7-5EE0-4CE8-A60E-C40D880EF3E3@surety-networks.com> The correct answer was in the log file. Behind a NAT you have to set login_trusted_networks = internal/24 externalIP - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From tss at iki.fi Sat Feb 25 04:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 04:49:07 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330137747.13977.3.camel@localhost> References: <1330137747.13977.3.camel@localhost> Message-ID: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> On 25.2.2012, at 4.42, ml wrote: > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > encounter fatal errors. And what are the error messages? From ml at smtp.fakessh.eu Sat Feb 25 05:32:50 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 04:32:50 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> Message-ID: <1330140770.15576.1.camel@localhost> Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 4.42, ml wrote: > > > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > > encounter fatal errors. > > And what are the error messages? > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child 15407 killed with signal 11 (core dumps disabled) my current config ks37777 i386]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 128 vsz_limit = 64 M } service imap { client_limit = 1 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 128 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 128 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From tss at iki.fi Sat Feb 25 05:45:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:45:39 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330140770.15576.1.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> Message-ID: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> On 25.2.2012, at 5.32, ml wrote: > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : >> On 25.2.2012, at 4.42, ml wrote: >> >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I >>> encounter fatal errors. >> >> And what are the error messages? >> > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > 15407 killed with signal 11 (core dumps disabled) Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Sat Feb 25 05:54:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:54:22 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 Message-ID: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). From ml at smtp.fakessh.eu Sat Feb 25 06:51:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 05:51:06 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> Message-ID: <1330145466.15576.4.camel@localhost> Le samedi 25 f?vrier 2012 ? 05:45 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 5.32, ml wrote: > > > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > >> On 25.2.2012, at 4.42, ml wrote: > >> > >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > >>> encounter fatal errors. > >> > >> And what are the error messages? > >> > > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > > 15407 killed with signal 11 (core dumps disabled) > > Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > I have not succeeded well I got a son of fakessh I attached with gdb-p but bt full no me back stack my poor english -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From tss at iki.fi Sat Feb 25 07:17:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:17:46 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 In-Reply-To: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> References: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> Message-ID: <8AA21A89-D80B-4B82-A9A8-9E14A3A1F385@iki.fi> On 25.2.2012, at 5.54, Timo Sirainen wrote: > I just committed a couple of features that will make life easier for some types of proxying setups: Oh, and: 4. Added proxy_always=y passdb field. When used with proxy_maybe=y you can create a setup where the connections are proxied to either * when user's host matches locally, director assigns a local backend's host for the connection * when user's host doesn't match locally, Dovecot connects to that host So basically you can have a bunch of Dovecot cluster installations behind different load balancer IPs, and have all of the Dovecot proxies redirect the connection to the proper cluster's load balancer, or if it's for a local cluster then redirect to one of the backends. 5. Currently in v2.2 only, could be backported if needed: Added TTL for the proxy connections, so in case two clusters both think the other one should be handling the connection they won't end up creating thousands of connections to each others until finally failing when login process limits are reached. The hard coded TTL is 5, should be more than enough. From tss at iki.fi Sat Feb 25 07:18:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:18:54 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330145466.15576.4.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> Message-ID: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> On 25.2.2012, at 6.51, ml wrote: >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> > > I have not succeeded well I got a son of fakessh I attached with gdb-p > but bt full no me back stack Did you try without antispam plugin? My guess is that is the reason it crashes. Or if it still crashes, try disabling all plugins. From dmiller at amfes.com Sat Feb 25 08:09:09 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 24 Feb 2012 22:09:09 -0800 Subject: [Dovecot] Solr not indexing Message-ID: What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" doveadm -D fts rescan -u dmiller at amfes.com doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(dmiller at amfes.com): Debug: Added userdb setting: mail=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: Added userdb setting: plugin/acl_groups='allshared' doveadm(dmiller at amfes.com): Debug: Effective uid=5000, gid=8, home=/var/mail/amfes.com/dmiller doveadm(dmiller at amfes.com): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: fs: root=/var/mail/amfes.com/dmiller/mdbox, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: Namespace : Using permissions from /var/mail/amfes.com/dmiller/mdbox: mode=0700 gid=-1 doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 1 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls doveadm(dmiller at amfes.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=mdbox:/var/mail/%d/%n/mdbox doveadm(dmiller at amfes.com): Debug: shared: root=/usr/local/var/run/dovecot, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/' doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 0 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls -- Daniel From tss at iki.fi Sat Feb 25 08:24:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:24:20 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: Message-ID: On 25.2.2012, at 8.09, Daniel L. Miller wrote: > What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" That's intentional. "shared/%n/" namespace has no mails. Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. From dhenderson at mediatemple.net Sat Feb 25 08:32:12 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 22:32:12 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > On 25.2.2012, at 0.49, Doug Henderson wrote: > >> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >> >> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. > > I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 see: https://rhn.redhat.com/errata/RHSA-2012-0150.html "A flaw was found in the way the Linux kernel's Event Poll (epoll) subsystem handled large, nested epoll structures. A local, unprivileged user could use this flaw to cause a denial of service. (CVE-2011-1083, Moderate)" Our automated patching (ksplice) installed it at around 10am PST today. Other distributions may vary. > >> dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument > .. >> Once this patch was removed, everything started working again. >> >> Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? > > It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). > We haven't spent enough time investigating to be sure, but epoll_ctl was certainly "in the thick of it". The only outward evidence (in logs, even with debug turned on) that there was anything wrong with Dovecot at all was the Panic shown for that method. Dovecot may have been an innocent bystander in this case - but something was causing it to fail on inbound IMAP/POP3 connections, and when the patch was removed everything started working again. From tss at iki.fi Sat Feb 25 08:49:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:49:10 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <876F3440-67CC-47B2-AA33-96A067A6A22D@iki.fi> On 25.2.2012, at 8.32, Doug Henderson wrote: >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html Yes, but CVE-2011-1083 shows it was reported almost a year ago, so I'd think it was fixed in upstream kernel a long time ago. I'm running in my desktop about 2 months old kernel (from git) and I don't see any problems with it. But yeah, maybe Redhat's patches did it differently than upstream kernel and it broke because of that.. From mstevens at imt-systems.com Sat Feb 25 13:15:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sat, 25 Feb 2012 12:15:49 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> On 25.02.2012 07:32, Doug Henderson wrote: > On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > >> On 25.2.2012, at 0.49, Doug Henderson wrote: >> >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail >>> servers, most/all IMAP/POP3 connections began experiencing time-outs >>> trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so >> other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat > for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html > > "A flaw was found in the way the Linux kernel's Event Poll (epoll) > subsystem handled large, nested epoll structures. A local, > unprivileged > user could use this flaw to cause a denial of service. > (CVE-2011-1083, > Moderate)" > > Our automated patching (ksplice) installed it at around 10am PST > today. > > Other distributions may vary. Try it without ksplice. (yum update and reboot) Which kernel is running exactly? Best regards, Morten From busseniu at in.tum.de Sat Feb 25 21:16:20 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Sat, 25 Feb 2012 20:16:20 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: <4F493384.9010703@in.tum.de> On 02/25/2012 01:50 AM, Timo Sirainen wrote: > On 24.2.2012, at 17.20, Rafa? Radecki wrote: >> I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf > > That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). > The migration script takes the UIDLs from "courierpop3dsizelist" and writes them into "dovecot-uidlist". Thus Dovecot will use the same UIDL as Courier for the converted mails. %08Xu%08Xv will only be used for new mails. Cheers, Christoph From dmiller at amfes.com Sun Feb 26 03:42:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 25 Feb 2012 17:42:22 -0800 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 2/24/2012 10:24 PM, Timo Sirainen wrote: > On 25.2.2012, at 8.09, Daniel L. Miller wrote: > >> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" > That's intentional. "shared/%n/" namespace has no mails. > > Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? -- Daniel From tss at iki.fi Sun Feb 26 04:26:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 04:26:07 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 26.2.2012, at 3.42, Daniel L. Miller wrote: > On 2/24/2012 10:24 PM, Timo Sirainen wrote: >> On 25.2.2012, at 8.09, Daniel L. Miller wrote: >> >>> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" >> That's intentional. "shared/%n/" namespace has no mails. >> >> Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. >> > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? The cached "last fts indexed message" gets dropped with: doveadm force-resync INBOX From dhenderson at mediatemple.net Sun Feb 26 04:55:59 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sat, 25 Feb 2012 18:55:59 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> Message-ID: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: > > Try it without ksplice. (yum update and reboot) I don't know if I'll be permitted to do that in a production environment - possibly a test one. I'll need to get some opinions from our Ops people as to if/how they might want to go about it. > Which kernel is running exactly? 2.6.18-274.3.1.el5 > Best regards, > > Morten From f.bonnet at esiee.fr Sun Feb 26 09:22:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Sun, 26 Feb 2012 08:22:48 +0100 Subject: [Dovecot] FreeBSD maintainer ? Message-ID: <4F49DDC8.2060506@esiee.fr> Hello Does the FreeBSD Dovecot's port maintainer read this mailing-list ? Thanks From duihi77 at gmail.com Sun Feb 26 10:54:56 2012 From: duihi77 at gmail.com (Duane Hill) Date: Sun, 26 Feb 2012 08:54:56 +0000 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <1527928288.20120226085456@gmail.com> On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr confabulated: > Hello > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > Thanks Have you tried sending message to the maintainer? The email address can be found by: cd /usr/ports/mail/dovecot && make maintainer or cd /usr/ports/mail/dovecot2 && make maintainer There appears to be two different email addresses between maintainers for version 1.2.x and 2.0.x. -- If at first you don't succeed... ...so much for skydiving. From tss at iki.fi Sun Feb 26 12:30:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 12:30:43 +0200 Subject: [Dovecot] IMAP NOTIFY extension Message-ID: <7075A103-8131-491F-9496-C632418BAF67@iki.fi> http://hg.dovecot.org/dovecot-2.2-notify/ contains the initial implementation of IMAP NOTIFY extension. It's less than half-working, but the difficult part of it is done I think. If I give it a command like: x notify set (personal (messagenew messageexpunge flagchange)) and then in another session I copy a message to another mailbox, it sends: * STATUS "foobar" (MESSAGES 77 UIDNEXT 78 UIDVALIDITY 1329431281 UNSEEN 75) This requires mailbox_list_index=yes to work. It works by following the dovecot.list.index.log changes, so it's going to be pretty efficient. Some TODO that I've so far noticed: - NOTIFY is handled currently only for non-selected mailboxes. (handling the new features for selected mailbox should be simple) - initial STATUS sending isn't done with STATUS option - only MessageNew, MessageExpunge and FlagChange notifications are sent currently - MailboxName handling probably needs to be able to get "old" and "new" views, figure out what changed in them and send those changes (dsync has similar code) - mailbox list indexes seem to be adding too much stuff, which results in: - STATUS duplicates, but there should already be code to avoid these so I'm not sure what's going on - ACLs and shared mailboxes don't work right - add 0.5s delay before notifying IMAP code, so multiple changes can be bundled into one From mstevens at imt-systems.com Sun Feb 26 12:44:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 26 Feb 2012 11:44:18 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: On 26.02.2012 03:55, Doug Henderson wrote: > On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >> >> Try it without ksplice. (yum update and reboot) > > I don't know if I'll be permitted to do that in a production > environment - possibly a test one. > I'll need to get some opinions from our Ops people as to if/how they > might want to go about it. > >> Which kernel is running exactly? > > 2.6.18-274.3.1.el5 That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. Best regards, Morten From lists at wildgooses.com Sun Feb 26 13:52:37 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 11:52:37 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4A1D05.2070402@wildgooses.com> On 25/02/2012 00:39, Timo Sirainen wrote: > On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > Not possible. > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. Is it feasible to implement both of these solutions using the current plugin architecture? I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Cheers Ed W From jerry at seibercom.net Sun Feb 26 13:56:42 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 06:56:42 -0500 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <1527928288.20120226085456@gmail.com> References: <4F49DDC8.2060506@esiee.fr> <1527928288.20120226085456@gmail.com> Message-ID: <20120226065642.24deacd8@scorpio> On Sun, 26 Feb 2012 08:54:56 +0000 Duane Hill articulated: > On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr > confabulated: > > > Hello > > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > > > Thanks > > Have you tried sending message to the maintainer? The email address > can be found by: > > cd /usr/ports/mail/dovecot && make maintainer > > or > > cd /usr/ports/mail/dovecot2 && make maintainer > > There appears to be two different email addresses between maintainers > for version 1.2.x and 2.0.x. That is because there are two different maintainers: dovecot-1.2.17 == yds at CoolRat.org dovecot-2.0.16_1 == bra at fsn.hu If you are having a problem with the port, I would suggest that you post on the FreeBSD Ports forum: freebsd-ports at freebsd.org and CC the maintainer of the port in question. It works quite effectively. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Sun Feb 26 14:31:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 14:31:40 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4A1D05.2070402@wildgooses.com> References: <4F4A1D05.2070402@wildgooses.com> Message-ID: On 26.2.2012, at 13.52, Ed W wrote: > On 25/02/2012 00:39, Timo Sirainen wrote: >> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >> >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have been >>> configured to save copy of messages in the Server. >> Not possible. >> > > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) > > In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. What does the notification do? > Is it feasible to implement both of these solutions using the current plugin architecture? Yes. > I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? > > All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. From lists at wildgooses.com Sun Feb 26 15:23:59 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 13:23:59 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: <4F4A1D05.2070402@wildgooses.com> Message-ID: <4F4A326F.8000803@wildgooses.com> On 26/02/2012 12:31, Timo Sirainen wrote: > On 26.2.2012, at 13.52, Ed W wrote: > >> On 25/02/2012 00:39, Timo Sirainen wrote: >>> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >>> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. >>> Not possible. >>> >> If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) >> >> In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. > What does the notification do? Sends another email... (you know like the annoying "message read" indicators that lots of mail readers support)... (Several of our competitors have implemented these solutions very badly and we get mail loops and other nasties...) >> I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? >> >> All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? > Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. Thanks - it's off my radar for a while due to other pressures, but the hint is appreciated and I will look into it in the future - many thanks! Ed W From fxcode at me.com Sun Feb 26 15:44:05 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 13:44:05 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted Message-ID: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Hi, I am relatively new to linux, and have set up my first lamp server around a month ago. The server is housed in a data centre a few miles from my house, and is working as a web server (with apache, tomcat, mysql) just fine. I have now been trying for a week to get Postfix and Dovecot working with postfixadmin and mysql. (below I have replaced my test domain name with [***domain_name***]) The story so far; I can send 'welcome' emails from within postfix admin to an external email address... and they are received by the mail client on my computer at home no problem. I can set up alias email addresses within postfix admin, such as webmaster@[***domain_name***], and emails sent to this address are redirected to the proper address as expected. But - if I set up a standard mailbox within postfix admin - test@[***domain_name***] , and email it from my home system, nothing appear in postfixadmin (after a fetch request) or the mysql database. Here is the fail point in my mail.log file ; Feb 26 12:50:27 quantum dovecot: deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with euid=8(mail), gid=8(mail), egid=8(mail): Operation not permitted I don't know what the setguid(5000) is, or where it is being set, or what it should be! I have been changing permissions with the 'socket listen' section of my /etc/dovecot/dovecot.conf file, and when I restart dovecot it creates the auth-master file with the permissions as expected; socket listen { master { path = /var/run/dovecot/auth-master mode = 0777 user = vmail group = mail } client { path = /var/spool/postfix/private/auth mode = 432 #user = postfix #group = postfix } } my mail folders should be located at /var/vmail, but the folder is empty. I have chmod the permissions to be as follows; me at myserver: ls -ld /var/vmail drwxrwxrwx 2 vmail mail 4096 2012-02-11 08:40 /var/vmail Can anyone spot where this is going wrong? Thanks in advance for any help, it is very very much appreciated!!! My hosts file is as follows; 127.0.0.1 localhost [**server_ip***] quantum.[***domain_name***] quantum # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ...and 'dovecot -n' replies as follows; me at myserver: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:/var/vmail/%d/%n mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: log_path: /var/log/dovecot-deliver.log auth_socket_path: /var/run/dovecot/auth-master postmaster_address: fxcode at me.com mail_plugins: sieve auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 master: path: /var/run/dovecot/auth-master mode: 511 user: vmail group: mail From sk71 at gmx.de Sun Feb 26 17:54:22 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 16:54:22 +0100 Subject: [Dovecot] Deny create folders Message-ID: <4F4A55AE.1030402@gmx.de> Hi all, is it possible to configure dovecot in that way that special folders do not create automatically by some imap clients? i.e: outlook or thunderbird creates a folder junk-mail i would like that such folders are not created automatically or by users Best regards Stefan From tss at iki.fi Sun Feb 26 18:02:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 18:02:47 +0200 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: On 26.2.2012, at 17.54, S. Kremer wrote: > is it possible to configure dovecot in that way that special folders do > not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by users You could maybe create a global ACL that denies the access to create them. The clients might not be happy about it though. It's not a very good idea to go doing things like this I think. From jerry at seibercom.net Sun Feb 26 18:41:25 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 11:41:25 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: <20120226114125.59e0d407@scorpio> On Sun, 26 Feb 2012 16:54:22 +0100 S. Kremer articulated: > Hi all, > > is it possible to configure dovecot in that way that special folders > do not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by > users Exactly what problem are you attempting to alleviate? In the short term you might seriously upset some of your clients and in the long term cause a stability problem. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From fxcode at me.com Sun Feb 26 22:26:20 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 20:26:20 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted In-Reply-To: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> References: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Message-ID: <11EBA09F-36C6-4319-874A-44207A6ED435@me.com> Hi, I have fixed this fault - the following line in my /etc/dovecot/dovecot.conf file was setting the setgid to 5000, and the user id to 5000 also auth default { ... userdb static { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } ... } I changed these two values to 8 ( the id of mail on my system) and emails are now being delivered to the proper folders thanks Christy From sk71 at gmx.de Sun Feb 26 23:31:35 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 22:31:35 +0100 Subject: [Dovecot] Deny create folders In-Reply-To: <20120226114125.59e0d407@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> Message-ID: <4F4AA4B7.4030909@gmx.de> Ok, thanks for answer. Every user's mailbox has a directory called Spam and Ham. I scan this two folders by a script to learn my spamassassin database. This script does not scan folders like spam, ham, junk-email etc. But some users have such folders in their mailbox so spamassassin does not learn from these folders. My aim is to force users to use the folders Spam and Ham and not other folders for spam or ham messages. Am 26.02.2012 17:41, schrieb Jerry: > On Sun, 26 Feb 2012 16:54:22 +0100 > S. Kremer articulated: > >> Hi all, >> >> is it possible to configure dovecot in that way that special folders >> do not create automatically by some imap clients? >> >> i.e: >> outlook or thunderbird creates a folder junk-mail >> >> i would like that such folders are not created automatically or by >> users > > Exactly what problem are you attempting to alleviate? In the short term > you might seriously upset some of your clients and in the long term > cause a stability problem. > Greetings Stefan From jerry at seibercom.net Mon Feb 27 00:28:27 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 17:28:27 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4AA4B7.4030909@gmx.de> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> Message-ID: <20120226172827.112de657@scorpio> On Sun, 26 Feb 2012 22:31:35 +0100 S. Kremer articulated: > Am 26.02.2012 17:41, schrieb Jerry: > > On Sun, 26 Feb 2012 16:54:22 +0100 > > S. Kremer articulated: > > > >> Hi all, > >> > >> is it possible to configure dovecot in that way that special > >> folders do not create automatically by some imap clients? > >> > >> i.e: > >> outlook or thunderbird creates a folder junk-mail > >> > >> i would like that such folders are not created automatically or by > >> users > > > > Exactly what problem are you attempting to alleviate? In the short > > term you might seriously upset some of your clients and in the long > > term cause a stability problem. > Ok, thanks for answer. > > Every user's mailbox has a directory called Spam and Ham. I scan this > two folders by a script to learn my spamassassin database. > This script does not scan folders like spam, ham, junk-email etc. > But some users have such folders in their mailbox so spamassassin does > not learn from these folders. > > My aim is to force users to use the folders Spam and Ham and not other > folders for spam or ham messages. Please don't top post. It makes following a thread a lot harder than it needs to be. Couldn't you just make a soft link between the two folders and save yourself and your clientele a lot of trouble? I have not messed with "spamassassin" in years; however, I thought it was configurable on what it scans. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ml at smtp.fakessh.eu Mon Feb 27 00:37:35 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 26 Feb 2012 23:37:35 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330295855.14524.12.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I'm in front of a problem. despite having to disable the antispam plugin as it appears in dovecot.conf always load so I did not succeed to disable I do not know how can you tell me that I continue my tests without the antispam plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From jd.beaubien at gmail.com Mon Feb 27 01:26:02 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Sun, 26 Feb 2012 18:26:02 -0500 Subject: [Dovecot] testing fts-solr? Message-ID: hi everyone, However, how can I verify that solr is working properly? Background: - I'm trying to setup 2.1 with fts_solr and eventually test maildir/mdbox. - I used the following webpage as reference to setup solr: http://www.roessner-network-solutions.com/2012/02/19/full-text-search-with-solr-and-dovecot-on-ubuntu-10-04/ I'm looking at /var/log/tomcat6/catalina.out file, but see nothing telling me that it's being used (same for /var/log/mail.log, etc.) Any tips? Thanks From ml at smtp.fakessh.eu Mon Feb 27 01:39:58 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 00:39:58 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330299598.14524.13.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I managed dovecot 2.1. 1 works well in removing completely antispam plugin work well and fine -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From maria_arrea at gmx.com Mon Feb 27 02:07:00 2012 From: maria_arrea at gmx.com (Maria Arrea) Date: Mon, 27 Feb 2012 01:07:00 +0100 Subject: [Dovecot] Dovecot altmove questions Message-ID: <20120227000700.134750@gmx.com> Hello We are using Dovecot 2.0.17 with mdbox+zlib and we are evaluating alternative storage. These are the relevant settings mdbox_rotate_interval = 1d mdbox_rotate_size=60m [...] zlib_save_level = 9 # 1..9 zlib_save = gz # or bz2 [...] zlib plugin enabled in dovecot-lda These are my questions: 1? General mdbox question. We are happy usings zlib_save, but we would like to try bzip2 compression. Can we change zlib to bzlib without problems? In other words, can we mix & match gzipped and bzipped messages in the same mdbox file ? 2? With doveadm altmove we can move messages matching a pattern to alternate storage. I have read an older post in this mailing list about passing specific mdbox options to doveadm altmove. Should be a good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d and mdox_rotate_size=60m and an "alternate area" with bzip9, mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area should only be "old" mail (1 year older). 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is taking almost 16 hours to make a full backup. Our main problem is that indexes are backupd hours before mdboxes, and that bring us some inconsistencies. What is the "right" way of backing up for separate indexes and mdboxes? Thank you for your support. Maria From ml at smtp.fakessh.eu Mon Feb 27 03:11:22 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 02:11:22 +0100 Subject: [Dovecot] value for client_limit Message-ID: <1330305082.20912.5.camel@localhost> hello guys hello master Timo i build the last release of dovecot on my centos 5 work well and fine but what the correct value for client limit in service auth { client_limit=6000 } and service anvil { client_limit=6000 } my current dovecot -n ks37777 i386]# dovecot -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service anvil { client_limit = 6000 } service auth { client_limit = 6000 process_limit = 1 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { client_limit = 0 inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1024 service_count = 1 vsz_limit = 64 M } service imap { process_limit = 1024 process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From sven at svenhartge.de Mon Feb 27 04:26:11 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 03:26:11 +0100 Subject: [Dovecot] Dovecot altmove questions References: <20120227000700.134750@gmx.com> Message-ID: <78jhe1341pv8@mids.svenhartge.de> Maria Arrea wrote: > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). > We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? Bacula? Excellent! If you put your indexes and mdboxes on a LVM (seperate volume group obviously, because of the different storages both reside on), you can snapshot both volumes at the same time, using an FD-based script, mount them somewhere else (like /backup/data and /backup/index) and then you have (nearly) all the time you want to backup them. And because of the snapshots taken at the same time, no inconsistencies can happen. (OK, not entirely true, since both snapshots are taken with a minimal gap between them, but if you start the backup during off-hours the inconsistencies should be minimal.) If you don't use a LVM but have a storage which is able to do snapshots on its own, you can also use this. If you don't have a LVM and no storage with such functionality, then you are kind of lost, as far as I can see. My additional advise on the long time of full backups: don't do them! Switch to accurate backups and only do incremental and differential backups, replacing the full backups with virtual full ones. This will greatly reduce the load on your mail systems and speed up the backup operation at the same time. But you have to use "accurate" type backups, or your virtual full backup will grow and grow and grow, since Bacula without "accurate" is unable to know about deleted files and will include them in every virtual full backup. Gr??e, Sven. -- Sigmentation fault. Core dumped. From dhenderson at mediatemple.net Mon Feb 27 05:20:36 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sun, 26 Feb 2012 19:20:36 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: <775A9BD8-884C-47D5-BF82-2BBDF414BF56@mediatemple.net> On Feb 26, 2012, at 2:44 AM, Morten Stevens wrote: > On 26.02.2012 03:55, Doug Henderson wrote: >> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >>> >>> Try it without ksplice. (yum update and reboot) >> >> I don't know if I'll be permitted to do that in a production >> environment - possibly a test one. >> I'll need to get some opinions from our Ops people as to if/how they >> might want to go about it. >> >>> Which kernel is running exactly? >> >> 2.6.18-274.3.1.el5 > > That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. > > Best regards, > > Morten Thanks Morten, We'll install the latest kernel on a test machine tomorrow and see how things go - we'll probably also attempt to reinstall the patch (if appropriate) and see if it still breaks things. Doug From mark at ecs.vuw.ac.nz Mon Feb 27 03:17:32 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 14:17:32 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail Message-ID: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Hi, have dovecot 2.0.13 running fine on a NetBSD box, with lots of clients happily authenticating to it with GSSAPI. I've just updated an ArchLinux box to latest new version which has meant its switched from using heimdal to using mit krb5 and its kmail is now also slightly newer. Thunderbird successfully authenticates to dovecot from this machine but kmail now fails. Kmail reports: Unable to authenticate via GSSAPI. The server ... replied: SASL(-1): generic failure: GSSAPI Error: Invalid token was supplied (Token header is malformed or corrupt) The dovecot log reports: Feb 27 13:42:18 bats dovecot: auth: GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in continued response Any suggestions on how to track down what specifically has changed and how to fix it? cheers mark From bra at fsn.hu Mon Feb 27 09:07:24 2012 From: bra at fsn.hu (Attila Nagy) Date: Mon, 27 Feb 2012 08:07:24 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <4F4B2BAC.2030400@fsn.hu> On 02/26/12 08:22, Frank Bonnet wrote: > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > If you read it, you may know the answer (depending on which port do you use). From f.bonnet at esiee.fr Mon Feb 27 09:13:44 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 27 Feb 2012 08:13:44 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F4B2BAC.2030400@fsn.hu> References: <4F49DDC8.2060506@esiee.fr> <4F4B2BAC.2030400@fsn.hu> Message-ID: <4F4B2D28.5010600@esiee.fr> hello I'm wondering when you will port 2.1.1 version ? ;-) Thanks Le 27/02/2012 08:07, Attila Nagy a ?crit : > On 02/26/12 08:22, Frank Bonnet wrote: >> >> Does the FreeBSD Dovecot's port maintainer read this mailing-list ? >> > If you read it, you may know the answer (depending on which port do > you use). From busseniu at in.tum.de Mon Feb 27 09:23:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Mon, 27 Feb 2012 08:23:14 +0100 Subject: [Dovecot] [PATCH] Pop3 order in courier migration script Message-ID: <4F4B2F62.1020204@in.tum.de> Hi, I found a problem in the courier conversion script (courier-dovecot-migrate.pl). In some cases, it does not correctly preserve the order of POP3 UIDLs. Consider the following case, when courierpop3dsizelist contains: /2 9000 1327481629 AAA1:2,S 10000 21:1327481629 AAA2:2,S 10000 22:1327481629 AAA3:2,S 10000 23:1327481629 AAA4:2,S 10000 24:1327481629 AAA5:2,S 10000 25:1327481629 BBB11:2,S 10000 11:1327481629 BBB12:2,S 10000 12:1327481629 BBB13:2,S 10000 13:1327481629 BBB14:2,S 10000 14:1327481629 BBB15:2,S 10000 15:1327481629 and courierimapuiddb contains: 1 1327480919 9000 11 BBB11 12 BBB12 13 BBB13 14 BBB14 15 BBB15 16 CCC16 17 CCC17 18 CCC18 19 CCC19 20 CCC20 21 AAA1 22 AAA2 23 AAA3 24 AAA4 25 AAA5 then call courier-dovecot-migrate.pl --to-dovecot --convert and it will create dovecot-uidlist: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 :CCC17 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 :CCC16 9003 PUID15-1327481629 W10000 :BBB15:2,S 9004 :CCC20 9005 PUID14-1327481629 W10000 :BBB14:2,S 9006 PUID13-1327481629 W10000 :BBB13:2,S 9007 :CCC18 9008 PUID11-1327481629 W10000 :BBB11:2,S 9009 :CCC19 The problem here is that the BBB files are not in the same order as in courierpop3dsizelist, and will therefore have a different order in the UIDL response in POP3. Also, the CCC files are interleaved while they should appear at the end. The reason is how the script assigns new UIDs: It does not do this in the order of courierpop3dsizelist. In fact the order is random because it depends on how perl traverses hashes. The attached patch fixes this by sorting the messages: First the POP3 messages get their new UIDs in order of their appearance in courierpop3dsizelist (so that the UIDL order will really be the same as in Courier), after that, the IMAP-only messages will be checked for UID clashes. The patch also changes how the $prev_uid variable is handled because that was also needed to always get the order right. The algorithm could probably be optimized to reduce the number of UID re-assignments to a minimum, however this would complicate the code and would only be noticed in a few cases. The patched courier-dovecot-migrate.pl will create dovecot-uidlist like this: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 PUID11-1327481629 W10000 :BBB11:2,S 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 PUID13-1327481629 W10000 :BBB13:2,S 9003 PUID14-1327481629 W10000 :BBB14:2,S 9004 PUID15-1327481629 W10000 :BBB15:2,S 9005 :CCC16 9006 :CCC17 9007 :CCC18 9008 :CCC19 9009 :CCC20 Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching -------------- next part -------------- A non-text attachment was scrubbed... Name: courier-dovecot-migrate.patch Type: text/x-patch Size: 1386 bytes Desc: not available URL: From tss at iki.fi Mon Feb 27 10:34:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 10:34:42 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <1330331682.11500.291.camel@innu> On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > What do you need the statistics for? I could make imap_client and > pop3_client support some virtual methods, like user.destroy() initially, > which would be enough for your use. I guess I could add that for v2.2. http://dovecot.org/patches/2.2/imap-logout-plugin.c From Ralf.Hildebrandt at charite.de Mon Feb 27 10:39:18 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 09:39:18 +0100 Subject: [Dovecot] Error with 2.0.18 Message-ID: <20120227083917.GD16162@charite.de> Today I found this in my log: Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3d44a) [0xb771644a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3d4bb) [0xb77164bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7716d87] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x3bfa) [0xb74e1bfa] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(squat_trie_build_deinit+0x9fc) [0xb74e4d3c] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x2a95) [0xb74e0a95] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_build_deinit+0x1b) [0xb752f6bb] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x3726) [0xb7530726] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x51d3) [0xb75321d3] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x5c34) [0xb7532c34] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x3b) [0xb778b0cb] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARCH]() [0x805ce6e] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARC Feb 26 14:27:55 postamt dovecot: master: Error: service(imap): child 18468 killed with signal 6 (core dumped) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at wildgooses.com Mon Feb 27 10:46:31 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 27 Feb 2012 08:46:31 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1330331682.11500.291.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> Message-ID: <4F4B42E7.4090101@wildgooses.com> On 27/02/2012 08:34, Timo Sirainen wrote: > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > >> What do you need the statistics for? I could make imap_client and >> pop3_client support some virtual methods, like user.destroy() initially, >> which would be enough for your use. I guess I could add that for v2.2. > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > Thanks - can I assume that a pop-logout would be basically the same? Also, how might I access the bytes in/out statistics from that context? Thanks Ed W From tss at iki.fi Mon Feb 27 11:05:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:05:44 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4B42E7.4090101@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> <4F4B42E7.4090101@wildgooses.com> Message-ID: <1330333544.11500.293.camel@innu> On Mon, 2012-02-27 at 08:46 +0000, Ed W wrote: > On 27/02/2012 08:34, Timo Sirainen wrote: > > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > > > >> What do you need the statistics for? I could make imap_client and > >> pop3_client support some virtual methods, like user.destroy() initially, > >> which would be enough for your use. I guess I could add that for v2.2. > > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > > > > > Thanks - can I assume that a pop-logout would be basically the same? Yes, basically s/imap/pop3. > Also, how might I access the bytes in/out statistics from that context? input: i_stream_get_absolute_offset(client->input) output: client->output->offset From tss at iki.fi Mon Feb 27 11:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:10:03 +0200 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <20120227083917.GD16162@charite.de> References: <20120227083917.GD16162@charite.de> Message-ID: <1330333803.11500.295.camel@innu> On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > Today I found this in my log: > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Large index, process VSZ limit reached. You might need to increase service imap { vsz_limit }. > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Bugs caused by the above. I don't think I'm going to spend time on Squat anymore, with v2.1 Lucene backend works better and faster. From tss at iki.fi Mon Feb 27 12:24:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:24:27 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: Message-ID: <1330338267.11500.299.camel@innu> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > hi everyone, > > However, how can I verify that solr is working properly? You should see in logs things like: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX It should be in Dovecot's info log (see "doveadm log find"). If you can't find it, see if "doveadm index -u user at domain INBOX" does any work. Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't give an error. From tss at iki.fi Mon Feb 27 12:28:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:28:16 +0200 Subject: [Dovecot] Dovecot altmove questions In-Reply-To: <20120227000700.134750@gmx.com> References: <20120227000700.134750@gmx.com> Message-ID: <1330338496.11500.303.camel@innu> On Mon, 2012-02-27 at 01:07 +0100, Maria Arrea wrote: > 1? General mdbox question. We are happy usings zlib_save, but we > would like to try bzip2 compression. Can we change zlib to bzlib > without problems? In other words, can we mix & match gzipped and > bzipped messages in the same mdbox file ? Yes. > 2? With doveadm altmove we can move messages matching a pattern to > alternate storage. I have read an older post in this mailing list > about passing specific mdbox options to doveadm altmove. Should be a > good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d > and mdox_rotate_size=60m and an "alternate area" with bzip9, > mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area > should only be "old" mail (1 year older). I guess it makes sense. > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid > 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? The eventual solution is a Dovecot backup agent, but for now I guess there isn't anything better than taking filesystem snapshots. From tss at iki.fi Mon Feb 27 12:30:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:30:04 +0200 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <1330338604.11500.304.camel@innu> On Mon, 2012-02-27 at 02:11 +0100, ml wrote: > i build the last release of dovecot on my centos 5 > work well and fine > > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } They just need to be "high enough". Dovecot logs a warning at startup if they're not high enough. 6000 is fine. From tss at iki.fi Mon Feb 27 12:32:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:32:10 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4AD9AC.5000300@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Message-ID: <1330338730.11500.306.camel@innu> On Mon, 2012-02-27 at 14:17 +1300, Mark Davies wrote: > Feb 27 13:42:18 bats dovecot: auth: > GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in > continued response Looks like kmail is sending some kind of garbage to Dovecot. Set auth_debug_passwords=yes to make Dovecot log the auth traffic. From Ralf.Hildebrandt at charite.de Mon Feb 27 12:46:40 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 11:46:40 +0100 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <1330333803.11500.295.camel@innu> References: <20120227083917.GD16162@charite.de> <1330333803.11500.295.camel@innu> Message-ID: <20120227104640.GI16162@charite.de> * Timo Sirainen : > On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > > Today I found this in my log: > > > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory > > Large index, process VSZ limit reached. You might need to increase > service imap { vsz_limit }. Indeed. That user has quite a bit of mail (3GB) > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) > > Bugs caused by the above. I don't think I'm going to spend time on Squat > anymore, with v2.1 Lucene backend works better and faster. Too lazy to set that up (yet). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Mon Feb 27 12:50:39 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 23:50:39 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <1330338730.11500.306.camel@innu> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> Message-ID: <4F4B5FFF.9090201@ecs.vuw.ac.nz> On 02/27/12 23:32, Timo Sirainen wrote: > Looks like kmail is sending some kind of garbage to Dovecot. Set > auth_debug_passwords=yes to make Dovecot log the auth traffic. Yeah, I did a network trace and it seems kmail is not sending the full authentication request before trying to carry on. > 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" > 11 0.045486 130.195.5.13 130.195.5.88 IMAP 122 Response: 1 NO [ALERT] Invalid base64 data in continued response now to figure out why its started truncating it. cheers mark From nick at mobilia.it Mon Feb 27 13:06:04 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 12:06:04 +0100 Subject: [Dovecot] LDA alias resolution problems.. Message-ID: <4F4B639C.9040207@mobilia.it> I use postfix + dovecot LDA + mysql virtual users and up til friday, everything was working just fine. Dovecot is 2.0.beta6 (3156315704ef) Like I said, up til friday, when I added a vacation entry into postfix (which I have now removed to see if it was the reason) mails to the alias virusalert (which is aliased to me) went through just fine. Now I get these sort of errors from the LDA: Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: to=, relay=dovecot, delay=357414, delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary failure. Command output: lda: Fatal: Unknown user ) Other aliases work perfectly, the only difference is that these emails are of local origin, (they're messages from Amavisd) I'm not sure why this has started to be a problem. Can anyone give me an idea where I should start looking? Nick From tss at iki.fi Mon Feb 27 13:11:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:11:47 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4B5FFF.9090201@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: On 27.2.2012, at 12.50, Mark Davies wrote: > On 02/27/12 23:32, Timo Sirainen wrote: >> Looks like kmail is sending some kind of garbage to Dovecot. Set >> auth_debug_passwords=yes to make Dovecot log the auth traffic. > > Yeah, I did a network trace and it seems kmail is not sending the > full authentication request before trying to carry on. >> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe It uses SASL-IR to send the first seponse. >> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + Dovecot says "OK, give me more". >> 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" kmail thinks the authentication is done and sends LIST command, even though it should finish the authentication. Clearly a kmail bug. You could try what happens if you remove SASL-IR from Dovecot's imap_capability list. Maybe kmail is buggy only with SASL-IR. From tss at iki.fi Mon Feb 27 13:36:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:36:00 +0200 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: <20120224012247.GA6512@krell.zikzak.de> References: <20120224012247.GA6512@krell.zikzak.de> Message-ID: <1330342560.11500.308.camel@innu> On Fri, 2012-02-24 at 02:22 +0100, Andreas M. Kirchwitz wrote: > There seems to be a new dependency in some modules (eg, lib-storage, > libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those > modules didn't require OpenSSL, but 2.1 does. > > For the linking process the path to the OpenSSL library isn't > specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL > is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't > properly used as well.) Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes everything? You can try a new nightly snapshot from http://dovecot.org/nightly/ once it gets there in a few minutes. From tss at iki.fi Mon Feb 27 13:43:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:43:35 +0200 Subject: [Dovecot] Virtual Folders In-Reply-To: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> References: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Message-ID: <1330343015.11500.313.camel@innu> On Fri, 2012-02-24 at 08:27 -0600, jacob at alltechservices-ia.com wrote: > Essentially, a gmail clone, but one that > I can make work on cPanel hosts with roundcube and squirrel mail, etc. > > By default, cpanel creates user mail dir structure of (maildir++) > inbox, drafts, sent, junk, and trash (with no dovecot specific files > until first login) > > I would like to replace all those folders with > virtual representations with more capabilities. I would like to achieve > the conversation view that gmail has pulled off so well. It's kind of possible to do that, but Dovecot has some problems with updating the view I think. Or maybe it was only when email was deleted that it didn't properly update the view. > namespace: > type: private > prefix: INBOX. > inbox: yes > list: > yes > subscriptions: yes This is your real mail's namespace. > box with Debian running the same version of dovecot (1.2.x) and on that > system I run into problems every time I make a virtual folder a root > folder. I assume it is trying to look up a normal folder instead of the > virtual ones and I am confusing it. namespace { prefix = separator = / location = virtual:/etc/dovecot/virtual:INDEX=~/Maildir/virtual inbox = yes } namespace { prefix = RealMails/ separator = / list = no hidden = yes } This has the problem that user now can't create any mailboxes other than what you have created globally.. There's really no automatic way to do this. From micha at krausam.de Mon Feb 27 13:47:27 2012 From: micha at krausam.de (Micha Krause) Date: Mon, 27 Feb 2012 12:47:27 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> References: <4F47BA30.2080704@krausam.de> <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> Message-ID: <4F4B6D4F.2070708@krausam.de> Hi Timo, > lmtp_save_to_detail_mailbox = yes Works like a charm, thank you. Micha Krause From CMarcus at Media-Brokers.com Mon Feb 27 13:51:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:51:11 -0500 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <4F4B6E2F.7010105@Media-Brokers.com> On 2012-02-26 8:11 PM, ml wrote: > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } Timo already replied that yours should be fine, but I'd just add... You can now use doveconf -d to see what the default values for any setting should be (thanks for adding that Timo!)... and the default values for both of these are 0. Two lessons I learned a long time ago when I first installed and started using postfix: 1. never change a default value/setting without a very good reason 2. never just randomly start changing values of settings in an attempt to fix a problem unless you understand precisely what these changes do So, either you changed them, or the CentOS package that you used changed them (most likely), which is one reason (among many others) that I would never use such systems for mission critical applications. I think Timo, like Wietse (postfix's author) does his best to set sane default values which should not be changed without a good reason. -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 13:58:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:58:29 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B6FE5.3040703@Media-Brokers.com> On 2012-02-27 6:06 AM, Nick Warr wrote: > Dovecot is 2.0.beta6 (3156315704ef) Seriously? You're having problems with an ancient pre-release/beta? Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read all of the upgrading docs on the wiki), then try again... -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 14:25:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 07:25:36 -0500 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F4B7640.5070700@Media-Brokers.com> On 2012-02-22 8:15 PM, Stephan Bosch wrote: > * I've just created an alternative that implements something similar to > the Procmail code you posted above, but from within Sieve itself. It is > a custom language extension called vnd.dovecot.duplicate and it adds the > "duplicate" test. This test keeps track of which Message-IDs it has seen > before in earlier deliveries and yields a true result if the message was > seen before, e.g.: > > require "vnd.dovecot.duplicate"; > > if duplicate { > discard; > } Interesting... I'll have to look into this later to see how it might be leveraged only for deduplicating mailing list messages (suppress duplicate messages when someone is a member of multiple lists that are members of other lists - I use mailman currently). -- Best regards, Charles From tss at iki.fi Mon Feb 27 14:45:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 14:45:09 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: <1330346709.11500.324.camel@innu> On Thu, 2012-02-23 at 20:55 +0100, Michael Grimm wrote: > Hi -- > > On 21.02.2012, at 23:03, Michael Grimm wrote: > > On 21.02.2012, at 22:57, Timo Sirainen wrote: > > >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > > > Ok. Then I will try both servers upgraded to 2.1 within the next days. > > JFTR: It took me some time to find a working syntax ;-) > > My working 2.0.18 syntax threw the following error: > > vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(vmail): Fatal: Unknown print formatter: -u > dsync-local(test): Error: read() from worker server failed: EOF You left out "mirror" from that command, but after adding it the latest hg version works. > Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: > > vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test > doveadm: illegal option -- v > doveadm sync [-u |-A] [-S ] [-fR] [-m ] The -v parameter is in wrong place now, needs to be "doveadm -v sync". > After some trial by error I finally found a working syntax: > > vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test Oh, hmm. I hadn't thought about this problem, it shouldn't have been necessary to give the dsync-server parameter. But I guess there's not a whole lot of other possibilities to do this. Hmm. BTW. I think you can do this simply: doveadm sync -f -u test test at remote-host.tld From nick at mobilia.it Mon Feb 27 16:24:26 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 15:24:26 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B6FE5.3040703@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> Message-ID: <4F4B921A.3080705@mobilia.it> Il 27/02/2012 12.58, Charles Marcus ha scritto: > On 2012-02-27 6:06 AM, Nick Warr wrote: >> Dovecot is 2.0.beta6 (3156315704ef) > > Seriously? You're having problems with an ancient pre-release/beta? > > Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read > all of the upgrading docs on the wiki), then try again... > dovecot --version 2.0.9 Same exact problem. any suggestions? From tom at whyscream.net Mon Feb 27 16:26:13 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 27 Feb 2012 15:26:13 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B9285.1080004@whyscream.net> On 27/02/12 12:06, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. Usually, aliases are processed by postfix, and dovecot only gets e-mail for actual users, not aliases. Depending on how your mail processing flow works, mail generated by amavis might not visit postfix before it hits dovecot... -- Regards, Tom From CMarcus at Media-Brokers.com Mon Feb 27 16:58:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 09:58:34 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B921A.3080705@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> Message-ID: <4F4B9A1A.9070702@Media-Brokers.com> On 2012-02-27 9:24 AM, Nick Warr wrote: > Il 27/02/2012 12.58, Charles Marcus ha scritto: >> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >> all of the upgrading docs on the wiki), then try again... > dovecot --version > 2.0.9 > > Same exact problem. > > any suggestions? Update to the *latest* version - for 2.0, that would be 2.0.18, for 2.1, it is 2.1.1 I'm not necessarily saying this will solve your problems (in fact, in your case it probably won't), but upgrading to the latest version should *always* be step #1 before asking for help (at least in my book)... -- Best regards, Charles From nick at mobilia.it Mon Feb 27 17:10:09 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:10:09 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9A1A.9070702@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> Message-ID: <4F4B9CD1.8000008@mobilia.it> Il 27/02/2012 15.58, Charles Marcus ha scritto: > On 2012-02-27 9:24 AM, Nick Warr wrote: >> Il 27/02/2012 12.58, Charles Marcus ha scritto: >>> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >>> all of the upgrading docs on the wiki), then try again... > >> dovecot --version >> 2.0.9 >> >> Same exact problem. >> >> any suggestions? > > Update to the *latest* version - for 2.0, that would be 2.0.18, for > 2.1, it is 2.1.1 > > I'm not necessarily saying this will solve your problems (in fact, in > your case it probably won't), but upgrading to the latest version > should *always* be step #1 before asking for help (at least in my > book)... > Talk to the CentOS guys for me, and tell em to get on the ball :) From nick at mobilia.it Mon Feb 27 17:13:17 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:13:17 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9285.1080004@whyscream.net> References: <4F4B639C.9040207@mobilia.it> <4F4B9285.1080004@whyscream.net> Message-ID: <4F4B9D8D.7070909@mobilia.it> Il 27/02/2012 15.26, Tom Hendrikx ha scritto: > On 27/02/12 12:06, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. > > Usually, aliases are processed by postfix, and dovecot only gets > e-mail for actual users, not aliases. Depending on how your mail > processing flow works, mail generated by amavis might not visit > postfix before it hits dovecot... > > -- > Regards, > Tom That does seem to be logical, since amavis is generating the email. Normal emails go postfix -> amavis -> postfix -> dovecot These go amavis -> postfix -> dovecot if this is so, I may need to talk to the postfix list, thanks! From CMarcus at Media-Brokers.com Mon Feb 27 17:22:45 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 10:22:45 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9CD1.8000008@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> <4F4B9CD1.8000008@mobilia.it> Message-ID: <4F4B9FC5.6000702@Media-Brokers.com> On 2012-02-27 10:10 AM, Nick Warr wrote: > Il 27/02/2012 15.58, Charles Marcus ha scritto: >> Update to the *latest* version - for 2.0, that would be 2.0.18, for >> 2.1, it is 2.1.1 > Talk to the CentOS guys for me, and tell em to get on the ball :) Don't point fingers - this is why I would never use a distro that locks you into ancient/unsupported versions of mission critical software. That said, there is usually a viable solution - I'm sure there are alternative CentOS repositories that offer up to date versions of things like postfix, dovecot, apache and other mission critical software. Even Debian stable (worst offender for locking you into ancient versions of software) offers such repositories. -- Best regards, Charles From tomislav.mihalicek at gmail.com Mon Feb 27 17:30:01 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Mon, 27 Feb 2012 07:30:01 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue Message-ID: <33400491.post@talk.nabble.com> I have a proxy setup for pop/imap. The proxies are defined in login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 connections displays the ip address of proxy IP... Dovecots are both 1.2 from the debian repo deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-1.2 main thanks -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33400491.html Sent from the Dovecot mailing list archive at Nabble.com. From bind at enas.net Mon Feb 27 17:52:05 2012 From: bind at enas.net (Urban Loesch) Date: Mon, 27 Feb 2012 16:52:05 +0100 Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <33400491.post@talk.nabble.com> References: <33400491.post@talk.nabble.com> Message-ID: <4F4BA6A5.90806@enas.net> Same here on 2.0.x. But I think this is because it's only implemented for IMAP. See e-mail from Timo 2 days ago: ... Subject: Proxying improvements in v2.1.2 I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). .... regards Urban On 27.02.2012 16:30, Tomislav Mihalicek wrote: > > I have a proxy setup for pop/imap. The proxies are defined in > login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 > connections displays the ip address of proxy IP... Dovecots are both 1.2 > from the debian repo deb http://xi.rename-it.nl/debian/ > stable-auto/dovecot-1.2 main > > thanks From joshua at hybrid.pl Mon Feb 27 18:44:06 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 27 Feb 2012 17:44:06 +0100 (CET) Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <20120224181059.66377BF108F@miha> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> <20120224181059.66377BF108F@miha> Message-ID: On Fri, 24 Feb 2012, Miha Vrhovnik wrote: > "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >> On 24.02.2012 15:47, Charles Marcus wrote: >>> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >>> wrote: >>>> Im using Dovecot 1.0.10 >>> Very old - an upgrade would likeley fix your problems... >> A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >> Only inbox is synced automatically. > So the solution is simple go bug Apple. > Oh yeah. they say if something isn't implemented on Apple platform or it > doesn't work, then you don't need it, because their devs know what's > best for their users. Exactly, no hope here :( What's even more annoying, apple doesn't put those idiotic limitations on Exchange accounts - seems they like Microsoft more than free solutions :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Mon Feb 27 18:54:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 11:54:49 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... Message-ID: <4F4BB559.6050405@Media-Brokers.com> Hello all/Timo, Up until now, my main Clients office has consisted of a single location, and I have never had to deal with the situation of multiple locations for a single company. They have just told me that they are acquiring an additional floor at a building that is about 4 minutes away - but obviously far enough away that I now have to deal with supporting users in the same domain but at two disparate physical locations. These two locations will be connected via a private Gb ethernet connection, and each location will have its own internet connection (I think - still waiting on some numbers to present to the owner to see what he wants to do in that regard, but that will be my recommendation), so bandwidth for replication won't be an issue. I have a couple of months to get this done, and I am already planning on hiring Timo's new commercial support company to help with the final and actual design and implementation, but obviously first I need to know what my actual options are. Just a rough idea of what I'd like to do is: 1. Set up one dovecot server at each location (these will be VMs), so users at each location are accessing the local server for that office 2. Full replication between the two for the mail/indexes, and configure them such that each can act as a failover for the other in case one goes down for whatever reason This is my first/main question... I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? Or, if it is going to take more work than my client is willing to pay for (I'm hoping not, since you said it was on the map for 2.1, not 2.2+), maybe the notify plugin could be leveraged in some way to provide something 'close enough' until it is fully implemented in dsync? On that note (something 'close enough' until dsync fully supports this natively), would setting up a dsync cron job, say, every 5 or 10 minutes, be asking for trouble? Our mail server is not all that busy, really, so in 5 or 10 minutes, there wouldn't be many changes at all. If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? and 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Thanks to any/all for reading this far and for any thoughts, suggestions and/or ideas... -- Best regards, Charles From as at 3a.pl Mon Feb 27 19:59:14 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 18:59:14 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: <4F4BC472.4000805@3a.pl> On 27.02.2012 17:54, Charles Marcus wrote: > These two locations will be connected via a private Gb ethernet > connection, and each location will have its own internet connection (I > think - still waiting on some numbers to present to the owner to see > what he wants to do in that regard, but that will be my > recommendation), so bandwidth for replication won't be an issue. [cut] I do have a basic question... How many users will be in this new, remote location? Will the traffic be so vast, that 1GbE link will not be enough, or are you using two servers for reliability? The simpler the configuration, it is almost always the better. Maybe you can stay with one server in yours primary location? -- Adam Szpakowski From rob0 at gmx.co.uk Mon Feb 27 20:12:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Mon, 27 Feb 2012 12:12:55 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <20120227181255.GV24983@harrier.slackbuilds.org> On Mon, Feb 27, 2012 at 06:59:14PM +0100, Adam Szpakowski wrote: > On 27.02.2012 17:54, Charles Marcus wrote: > >These two locations will be connected via a private Gb ethernet > >connection, and each location will have its own internet > >connection (I think - still waiting on some numbers to present to > >the owner to see what he wants to do in that regard, but that will > >be my recommendation), so bandwidth for replication won't be an > >issue. > [cut] > > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will > not be enough, or are you using two servers for reliability? > > The simpler the configuration, it is almost always the better. > Maybe you can stay with one server in yours primary location? This was exactly my thought as reading it. If you have some control over client configuration, use "offline IMAP," where clients maintain a local copy of what's on the server. (That's a good idea anyway, distributed backups of mail which possibly is important.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Mon Feb 27 20:21:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:21:23 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <4F4BC9A3.1010404@Media-Brokers.com> On 2012-02-27 12:59 PM, Adam Szpakowski wrote: > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will not > be enough, or are you using two servers for reliability? Yeah, I guess I should have mentioned this... Each location is an entire floor of a 6 story building. The remote location has the capacity for about 60 users, the new location about 100. We only allow IMAP access to email, so if everyone is using email at the same time, that would be a lot of traffic over a single Gb link I think... > The simpler the configuration, it is almost always the better. Maybe > you can stay with one server in yours primary location? I had considered this, but like I said, this is not purely for performance - I'd also like to get redundancy in the deal (up until now, we haven't had any - admittedly - knock on wood - we haven't needed it, but I'd still like to implement it). -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 20:22:53 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:22:53 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <20120227181255.GV24983@harrier.slackbuilds.org> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <20120227181255.GV24983@harrier.slackbuilds.org> Message-ID: <4F4BC9FD.5030806@Media-Brokers.com> On 2012-02-27 1:12 PM, /dev/rob0 wrote: > If you have some control over client configuration, use "offline > IMAP," where clients maintain a local copy of what's on the server. > (That's a good idea anyway, distributed backups of mail which > possibly is important.) Most of these users have many multiple Gb of email, so keeping local cached copies of all it if is silly, in my opinion... and again, the *main* purpose for the two separate servers is for high availability (redundancy/failover)... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From sven at svenhartge.de Mon Feb 27 20:34:23 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 19:34:23 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <88jj6q741pv8@mids.svenhartge.de> Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > Yeah, I guess I should have mentioned this... > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link > I think... Naa, most clients download mails only once and then keep them cached locally (at least Thunderbird and Outlook do). Looking at the used bandwidth of the mailserver of my small university (10.000 users, about 1000 concurrently active during the daytime) shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and out. Remember: your outgoing bandwidth will be roughly the bandwidth of mail going into the server. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Mon Feb 27 20:38:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:38:39 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <88jj6q741pv8@mids.svenhartge.de> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> Message-ID: <4F4BCDAF.50003@Media-Brokers.com> On 2012-02-27 1:34 PM, Sven Hartge wrote: > Charles Marcus wrote: >> Each location is an entire floor of a 6 story building. The remote >> location has the capacity for about 60 users, the new location about >> 100. We only allow IMAP access to email, so if everyone is using email >> at the same time, that would be a lot of traffic over a single Gb link >> I think... > Naa, most clients download mails only once and then keep them cached > locally (at least Thunderbird and Outlook do). > > Looking at the used bandwidth of the mailserver of my small university > (10.000 users, about 1000 concurrently active during the daytime) > shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and > out. Interesting - thanks for the numbers... But, again, my main reason for 2 servers is not performance, it is for redundancy... -- Best regards, Charles From as at 3a.pl Mon Feb 27 20:48:55 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 19:48:55 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC9A3.1010404@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <4F4BD017.5010806@3a.pl> On 27.02.2012 19:21, Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > > Yeah, I guess I should have mentioned this... > > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link I > think... I'm not sure that the bandwidth will be a problem. One of our clients is a civic design office. Around 60 people and lots of multi megabyte files in multiple copies. Autocad 3D files are flying all around ;). All accounts are IMAP ones, there is also local SAMBA service. The server has 1GbE connection to almost all workstations and the bandwidth utilization is low, very low, on average much less then 1%. We are talking about over 50 heavy duty users. >> The simpler the configuration, it is almost always the better. Maybe >> you can stay with one server in yours primary location? > > I had considered this, but like I said, this is not purely for > performance - I'd also like to get redundancy in the deal (up until > now, we haven't had any - admittedly - knock on wood - we haven't > needed it, but I'd still like to implement it). IMHO use something simple for redundancy such us DRBD in active/passive mode in single location. Manual migration to do not have to deal with split brain problems. As a additional layer of security against local cataclysm (fire in the building) use nightly backup to the second office. You will not have automatic, 99.999% reliability, but for most clients it is ok. They do not need this. The market for highly available, redundant services is quite small. -- Adam Szpakowski From list at airstreamcomm.net Mon Feb 27 22:51:54 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 27 Feb 2012 14:51:54 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BCDAF.50003@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> Message-ID: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> On Mon, 27 Feb 2012 13:38:39 -0500, Charles Marcus wrote: > On 2012-02-27 1:34 PM, Sven Hartge wrote: >> Charles Marcus wrote: >>> Each location is an entire floor of a 6 story building. The remote >>> location has the capacity for about 60 users, the new location about >>> 100. We only allow IMAP access to email, so if everyone is using email >>> at the same time, that would be a lot of traffic over a single Gb link >>> I think... > >> Naa, most clients download mails only once and then keep them cached >> locally (at least Thunderbird and Outlook do). >> >> Looking at the used bandwidth of the mailserver of my small university >> (10.000 users, about 1000 concurrently active during the daytime) >> shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and >> out. > > Interesting - thanks for the numbers... > > But, again, my main reason for 2 servers is not performance, it is for > redundancy... I too have been tasked with multisite redundancy, and have been experimenting with GlusterFS (http://www.gluster.org/community/documentation/index.php/Main_Page), which is a distributed file system. In our network we have a dedicated 10GB link between two datacenters 100 miles apart, and I have a GlusterFS node at each site setup in Distriubted Replicated mode with 2 replicas which means the servers are mirrored. The file writes are done to all the replica servers (2 servers in this case), so depending on network latency the writes could potentially slow down. GlusterFS has it's own file serving protocol that allows automatic and immediate failover in the case that a storage node disappears, but there are some caveats to restoring a failed storage node (takes forever to resync the data). I have not put this experiment into production, but I can say that it's extremely simple to manage, and performance testing has shown that it could handle mail traffic just fine. You could also look at GPFS (http://www-03.ibm.com/systems/software/gpfs/), which is not open source but it's apparently rock solid and I believe supports multisite clustering. From pc8888 at gmail.com Mon Feb 27 23:52:26 2012 From: pc8888 at gmail.com (Pierre Chiu) Date: Mon, 27 Feb 2012 16:52:26 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4BFB1A.1010401@gmail.com> Give this a try. Find out the location of the verify cache file and delete it. This is my setting in main.cf address_verify_map = btree:$data_directory/verify_cache On 2/27/2012 6:06 AM, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. > > Can anyone give me an idea where I should start looking? > > Nick From janfrode at tanso.net Tue Feb 28 01:58:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 00:58:29 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> Message-ID: <20120227235829.GA29841@dibs.tanso.net> On Mon, Feb 27, 2012 at 02:51:54PM -0600, list at airstreamcomm.net wrote: > > You could also look at GPFS > (http://www-03.ibm.com/systems/software/gpfs/), which is not open source > but it's apparently rock solid and I believe supports multisite clustering. GPFS supports different modes of clustering. I think the appropriate solution here would be to deploy a single cluster spanning 3 sites (3. site is needed for quorum node, two sites can't work because you can't protect it from split brain). The simplest config would then be 3 nodes (but you could have any number of nodes at each site): quorum node1 on site1 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node2 on site2 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node3 on site3 The filesystem would be replicated (over IP) between the disk on site1 and site2. Should one site go down, the other site would survive as long as it could still see the quorum node on site3. After a site has been down, one would need to sync up the NSDs (mmrestripefs) to re-establish the replication of any blocks that has been changed while it was down. -jf From jd.beaubien at gmail.com Tue Feb 28 02:14:57 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 19:14:57 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <1330338267.11500.299.camel@innu> References: <1330338267.11500.299.camel@innu> Message-ID: Ok, I had to fix a few things. First of all, I had to declare mail_plugins = fts fts_solr in global file (dovecot.conf). After doing that, running doveadm fts optimize stopped giving me errors. I ran doveadm index -user INBOX (and other folders). I'm assuming this actually did something because java/tomcat6 were really busy for a few mins and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). However, whenever I run a search thru thunderbird (searching the from/to/subject/body fields), I don't see anything related to solr in the dovecot logs (I never see the line you mentioned or anything close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). After playing with the logging level, I manage to see this in the logs: Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from directory: /var/opt/dovecot/lib/dovecot Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so But nothing indicating that solr_plugin is being used. Any idea? i'd really like to be able to clearly see that I'm using solr. Regards, -JD On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > > hi everyone, > > > > However, how can I verify that solr is working properly? > > You should see in logs things like: > > indexer-worker(user at domain): Info: Indexed 1 messages in INBOX > > It should be in Dovecot's info log (see "doveadm log find"). If you > can't find it, see if "doveadm index -u user at domain INBOX" does any > work. > > Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't > give an error. > > > From jd.beaubien at gmail.com Tue Feb 28 03:37:56 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 20:37:56 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: Btw, since 2.1 is out-performing our old solution so badly, I thought I'd drop some numbers. - The old setup takes over ~4.5 seconds to perform our most common operation. - The new setup takes ~0.2 second to perform the same operation on the same dataset. - approx 22 faster, with less than half the storage; zlib is awesome, for the test account 15GB transforms into 6.5GB. Old setup: dovecot 1.0.9 (Athlon X2 processor, standard hdd) New solution: dovecot 2.1, mdbox, zlib, fts_solr (i think solr works, but not sure) (core i7-2600, standard hdd). Thank you very much for the awesome software. On Mon, Feb 27, 2012 at 7:14 PM, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I don't see anything related to solr in the > dovecot logs (I never see the line you mentioned or anything > close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). > > After playing with the logging level, I manage to see this in the logs: > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from > directory: /var/opt/dovecot/lib/dovecot > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so > > But nothing indicating that solr_plugin is being used. Any idea? i'd > really like to be able to clearly see that I'm using solr. > > Regards, > > -JD > > > > > On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > >> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: >> > hi everyone, >> > >> > However, how can I verify that solr is working properly? >> >> You should see in logs things like: >> >> indexer-worker(user at domain): Info: Indexed 1 messages in INBOX >> >> It should be in Dovecot's info log (see "doveadm log find"). If you >> can't find it, see if "doveadm index -u user at domain INBOX" does any >> work. >> >> Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't >> give an error. >> >> >> > From tss at iki.fi Tue Feb 28 04:04:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 04:04:17 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). OK, so it's indexing. > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: telnet loclahost 143 a login user pass b select inbox c search text hello From nick at mobilia.it Tue Feb 28 09:49:10 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 08:49:10 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4BFB1A.1010401@gmail.com> References: <4F4B639C.9040207@mobilia.it> <4F4BFB1A.1010401@gmail.com> Message-ID: <4F4C86F6.9010702@mobilia.it> Il 27/02/2012 22.52, Pierre Chiu ha scritto: > Give this a try. > Find out the location of the verify cache file and delete it. > > This is my setting in main.cf > address_verify_map = btree:$data_directory/verify_cache > > > > On 2/27/2012 6:06 AM, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. >> >> Can anyone give me an idea where I should start looking? >> >> Nick Thanks, though I actually understood where the problem originated after explaining where the email was injected, it was a postfix configuration issue (the vacation script has you configure the amavis line in master.cf to not expand aliases, so it doesn't send multiple vacation replies.. I hadn't considered mail not originating from postfix) , it was just that dovecot was the one who had to deal with the error. Thanks to Tom Hendrikx for the clue. From joshua at hybrid.pl Tue Feb 28 09:50:10 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 08:50:10 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Maan, that's evil :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Tue Feb 28 10:09:01 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 09:09:01 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, Michael Sidoric wrote: >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > timo is quite correct -- consider imap instead, that's why the protocol exists. This rather seems like anti-imap - kind of desperate search for free disk space... IMAP does quite the opposite thing, since users acutally never remove emails :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From nick at mobilia.it Tue Feb 28 10:17:54 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 09:17:54 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4C8DB2.9060302@mobilia.it> Il 28/02/2012 9.09, Jacek Osiecki ha scritto: > On Fri, 24 Feb 2012, Michael Sidoric wrote: > >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have >>> been >>> configured to save copy of messages in the Server. >> timo is quite correct -- consider imap instead, that's why the >> protocol exists. > > This rather seems like anti-imap - kind of desperate search for free > disk space... IMAP does quite the opposite thing, since users acutally > never remove emails :) > > Greetings, I'd suggest this: http://archivemail.sourceforge.net/ You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. From tlx at leuxner.net Tue Feb 28 10:46:00 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:46:00 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4C8DB2.9060302@mobilia.it> References: <4F4C8DB2.9060302@mobilia.it> Message-ID: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Am 28.02.2012 um 09:17 schrieb Nick Warr: > Il 28/02/2012 9.09, Jacek Osiecki ha scritto: >> On Fri, 24 Feb 2012, Michael Sidoric wrote: >> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. > I'd suggest this: > > http://archivemail.sourceforge.net/ > > You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. Assuming this is the task you could also use: $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen Play around with the scope a bit before you actually run it: $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Tue Feb 28 10:53:40 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:53:40 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> References: <4F4C8DB2.9060302@mobilia.it> <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Message-ID: Am 28.02.2012 um 09:46 schrieb Thomas Leuxner: > Assuming this is the task you could also use: > > $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen > > Play around with the scope a bit before you actually run it: > > $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l Of course this was supposed to read 'seen' :/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tomislav.mihalicek at gmail.com Tue Feb 28 11:23:18 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 01:23:18 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405413.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405413.html Sent from the Dovecot mailing list archive at Nabble.com. From dovecot at r.paypc.com Tue Feb 28 12:15:27 2012 From: dovecot at r.paypc.com (Robin) Date: Tue, 28 Feb 2012 02:15:27 -0800 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CA93F.3080100@r.paypc.com> > I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: > > telnet localhost 143 > a login user pass > b select inbox > c search text hello Yes, you definitely want to check things are being accelerated by issuing direct IMAP commands via telnet. Many clients try to "help" by performing local searches, which will only obfuscate things for you. Even with 150K+ messages, it shouldn't take fts_solr more than 20ms or so to give you results. I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. You can check your solr index data directory too. A freshly installed solr index occupies almost no space, but that grows QUICKLY once it's indexed anything. =R= From tomislav.mihalicek at gmail.com Tue Feb 28 12:18:29 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 02:18:29 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405654.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405654.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Tue Feb 28 12:36:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 12:36:18 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CA93F.3080100@r.paypc.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CA93F.3080100@r.paypc.com> Message-ID: <9DBA17BC-FA96-4DC1-A3A3-30DEA66A98B5@iki.fi> On 28.2.2012, at 12.15, Robin wrote: > I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. That reads only in wiki1 docs, in wiki2 docs it should only talk about globals.. I wonder if it would be a bad idea to show some kind of a popup in wiki1 asking if you really want to be reading it or not, and use a cookie to remember it?.. From p at state-of-mind.de Tue Feb 28 13:10:58 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 28 Feb 2012 12:10:58 +0100 Subject: [Dovecot] ANN: automx - automated mail account provisioning Message-ID: <20120228111057.GC27530@state-of-mind.de> automx I am pleased to announce automx a tool that makes setting up a mail account easy. All your users need to provide is real name, mail address and password. Their mail client and automx will safely handle the rest. Say goodbye to mistyped server settings! Put an end to endless phone calls trying to coach users to configure settings, whose dialogs they can't find. Raise user satisfaction, relieve support and regain time to work on other issues. automx unifies Microsofts and Mozillas mail account provisioning standards in one powerful Open Source tool. Choose from many backends, including LDAP and SQL, and let automx create standard and individualized profiles for multiple domains on the fly! See the details on http://automx.org. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3603 bytes Desc: not available URL: From joshua at hybrid.pl Tue Feb 28 13:14:04 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 12:14:04 +0100 (CET) Subject: [Dovecot] Deny create folders In-Reply-To: <20120226172827.112de657@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: On Sun, 26 Feb 2012, Jerry wrote: > On Sun, 26 Feb 2012 22:31:35 +0100 > S. Kremer articulated: >> Every user's mailbox has a directory called Spam and Ham. I scan this >> two folders by a script to learn my spamassassin database. >> This script does not scan folders like spam, ham, junk-email etc. >> But some users have such folders in their mailbox so spamassassin does >> not learn from these folders. [...] > Couldn't you just make a soft link between the two folders and save > yourself and your clientele a lot of trouble? I have not messed with > "spamassassin" in years; however, I thought it was configurable on what > it scans. Oh, it reminds me of one thing I would like to have - maybe it is possible? Some IMAP clients (especially outlooks) create junk/sent/etc. folders with various names, depending on national settings. Is it possible to provide a - let's say - dictionary, which would always return the same folder (like Sent) no matter if client asks for "Sent", "Sent Mail" or "Elementy wys&AUI-ane"? Hmm... Now, when I formed the question it seems that this MIGHT be a problem :) What should dovecot provide when client asks for list of subscribed folders? If I know that there is a polish version of MS Outlook, I should provide "Elementy wys&AUI-ane", but "Sent" if it is roundcube... Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Tue Feb 28 14:54:38 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 07:54:38 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CCE8E.2000103@Media-Brokers.com> On 2012-02-27 9:04 PM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: >> However, whenever I run a search thru thunderbird (searching the >> from/to/subject/body fields), > I think Thunderbird does this search internally, not via IMAP. You could test by opening the Advanced Search window (CTRL-SHIFT-F or Edit > Find > Search Messages), check the box to 'Run search on server', make sure the right account is selected, then perform a search... There are 2 bugs open for Thunderbird dealing with server side IMAP searches, please feel free to go vote for one or both of them: Main one: https://bugzilla.mozilla.org/show_bug.cgi?id=564168 and one that is tangentially related: https://bugzilla.mozilla.org/show_bug.cgi?id=543416 -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 15:28:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 08:28:06 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: <4F4CD666.3040809@Media-Brokers.com> On 2012-02-28 6:14 AM, Jacek Osiecki wrote: > Some IMAP clients (especially outlooks) create junk/sent/etc. > folders with various names, depending on national settings. > Is it possible to provide a - let's say - dictionary, which would always > return the same folder (like Sent) no matter if client asks for "Sent", > "Sent Mail" or "Elementy wys&AUI-ane"? This has been discussed before, this is probably the most recent thread about it, but it is quite long: http://www.dovecot.org/list/dovecot/2011-December/062327.html -- Best regards, Charles From kadafax at gmail.com Tue Feb 28 15:41:01 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 14:41:01 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CD96D.5010706@gmail.com> On 02/28/2012 03:04 AM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > >> Ok, I had to fix a few things. First of all, I had to declare mail_plugins >> = fts fts_solr in global file (dovecot.conf). After doing that, running >> doveadm fts optimize stopped giving me errors. >> >> I ran doveadm index -user INBOX (and other folders). I'm assuming this >> actually did something because java/tomcat6 were really busy for a few mins >> and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > OK, so it's indexing. Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the solr's data directory is growing and log shows things like this: Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages in SubFolder but thunderbird returns nothing (it says not match found for "hello" for example). I dont know what more can I do to debug this... Solr is working as the nobody user, is it OK for dovecot ? Does dovecot need to be have write access to the solr's data directory ? Thanks From janfrode at tanso.net Tue Feb 28 16:02:33 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 15:02:33 +0100 Subject: [Dovecot] fts size Message-ID: <20120228140233.GA18412@dibs.tanso.net> Does anybody have any numbers for how large storage one will need for the fts indexing server? I see the wiki says 30% of mailbox size for Squat (partial=4 full=4). Is it similar for lucene/solr? Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene will create an index for each user in his home directory? Will this be accounted for in the users' quota? -jf From tss at iki.fi Tue Feb 28 16:03:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:03:54 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication Message-ID: <1330437834.2081.2.camel@innu> This document describes a design for a dsync-replicated Dovecot cluster. This design can be used to build at least two different types of dsync clusters, which are both described here. Ville has also drawn overview pictures of these two setups, see http://www.dovecot.org/img/dsync-director-replication.png and http://www.dovecot.org/img/dsync-director-replication-ssh.png First of all, why dsync replication instead of block level filesystem replication? - dsync won't replicate filesystem corruption. - A cold restart of replication won't go through all of the data in the disks, but instead quickly finds out what has changed. - Split brain won't result in downtime or losing any data. If both sides did changes, the changes are merged without data loss. - If using more than 2 storages, the users' replicas can be divided among the other storages. So if one storage goes down, the extra load is shared by all the other storages, not just one. Replication mail plugin ----------------------- This is a simple plugin based on notify plugin. It listens for all changes that happen to mailboxes (new mails, flag changes, etc.) Once it sees a change, it sends an asynchronous (username, priority) notification to replication-notify-fifo. The priority can be either high (new mails) or low (everything else). Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP. So, what is listening in those replication-notify* sockets? It depends on if Dovecot is running on director-based setup or not. Aggregator ---------- When running in Dovecot director-based setup, all of the Dovecot backends (where replication plugin runs) also run "aggregator" process. Its job is very simple: It proxies the notifications from mail plugin and sends them via a single TCP connection to the replicator process running in Dovecot proxies. This is simply an optimization to avoid tons of short lived TCP connections directly from replication plugin to director server. When not running in Dovecot director setup (i.e. there is only a single Dovecot instance that handles all of the users), there is no point in having an aggregator proxy, because the replicator process is running on the same server. In this kind of setup the replicator process directly listens on the replication-notify* sockets. Replicator ---------- The initial design for replicator isn't very complex either: It keeps a priority queue of all users, and replicates those users at the top of the queue. Notifications about changes to user's mailboxes (may) move the user up in the priority queue. If the user at the top of the queue already has been replicated "recently enough", the replicator stops its work until new changes arrive or the "recently enough" is no longer that. dsync can do two types of syncs: quick syncs and full syncs. A quick sync trusts indexes and does the replication with the least amount of work and network traffic. A quick sync is normally enough to replicate all changes, but just in case something has gone wrong there's also the full sync option, which guarantees that the mailboxes end up being fully synced. A full sync is slower though, and uses more network traffic. The priority queue is sorted by: 1. Priority (updated by a notification from replication plugin) 2. If priority!=none: Last fast sync (those users are replicated first whose last replication time is oldest) 2. If priority=none: Last full sync (these users should already be fully synced, but do a full sync for them once in a while anyway) All users get added to the replication queue at replicator startup with "none" priority. The list of users is looked up via userdb iteration. If the previous replication state is found from a disk dump, it's used to update the priorities, last_*_sync timestamps and other replication state. Replicator process creates such dumps periodically [todo: every few mins? maybe a setting?]. Replicator starts replicating users at the top of the queue, setting their priorities to "none" before starting. This means that if another change notification arrives during replication, the priority is bumped up and no changes get lost. replication_max_conns setting specifies how many users are replicated simultaneously. If the user's last_full_sync is older than replication_full_sync_interval setting, a full sync is done instead of a fast sync. If the user at the top of the queue has "none" priority and the last_full_sync is newer than replication_full_sync_interval, the replication stops. [todo: it would be nice to prefer doing all the full syncs at night when there's hopefully less disk I/O] (A global replication_max_conns setting isn't optimal in proxy-based setup, where different backend servers are doing the replication. There it should maybe be a per-backend setting. Then again, it doesn't account for the replica servers that also need to do replication work. Also to properly handle this each backend should have its own replication queue, but this requires doing a userdb lookup for each user to find out their replication server, and this would need to be done periodically in case the backend changes, which can easily happen often with director-based setup. So all in all, none of this is being done in the initial implementation. Ideally the users are distributed in a way that a global replication queue would work well enough.) In director-based setup each director runs a replicator server, but only one of them (master) actually asks the backends to do the replication. The rest of them just keep track of what's happening, and if the master dies or hangs, one of the others becomes the new master. The server with lowest IP address is always the master. The replicators are connected to a ring like the directors, using the same director_servers setting. The communication between them is simply about notifications of what's happening to users' priorities. Preferably the aggregators would always connect to the master server, but this isn't required. In general there's not much that can go wrong, since it's not a problem if two replicators request a backend to start replication for the same user or if the replication queue states aren't identical. If the replication is running too slowly [todo: means what exactly?], log a warning and send an email to admin. So, how does the actual replication happen? Replicator connects to doveadm server and sends a "sync -u user at domain" command. In director-based setup the doveadm server redirects this command to the proper backend. doveadm sync ------------ This is an independent feature from all of the above. Even with none of it implemented, you could run this to replicate a user. Most of this is already implemented. The only problem is that currently you need to explicitly tell it where to sync. So, when the destination isn't specified, it could do a userdb lookup and use the returned "mail_replica" field as the destination. Multiple (sequentially replicated) destinations could be supported by returning "mail_replica2", "mail_replica3" etc. field. In NFS-based (or shared filesystem-based in general) setup the mail_replica setting is identical to mail_location setting. So your primary mail_location would be in /storage1/user/Maildir, while the secondary mail_replica would be in /storage2/user/Maildir. Simple. In non-NFS-based setup two Dovecot servers talk dsync protocol to each others. Currently dsync already supports SSH-based connections. It would also be easy to implement direct TCP-based connections between two doveadm servers. In future these connections could be SSL-encrypted. Initially I'm only supporting SSH-based connections, as they're already implemented. So what does the mail_replica setting look like in this kind of a setup? I'm not entirely sure. I'm thinking that it could be either "ssh:host" or "ssh:user at host", where user is the SSH login user (this is opposite of the current doveadm sync command line usage). In future then it could support also tcp:host[:port]. Both of these ssh: and tcp: prefixes would also be supported by doveadm sync command line usage (and perhaps the prefixless user at domain be deprecated). dsync can run without any long lived locking and it typically works fine. In case mailbox was modified during dsync, the replicas may not end up being identical, but nothing breaks. dsync currently usually notices this and logs a warning. When these conflicting changes was caused by imap/pop3/lda/etc. this isn't a problem, they've already notified replicator already to perform another sync that will fix it. Running two dsyncs at the same time is more problematic though, mainly related to new emails. Both dsyncs notice that mail X needs to be replicated, so both save it and it results in having a duplicate. To avoid this, there should be a dsync-lock. If this lock exists, dsync should wait until the previous dsync is done and then do it again, just in case there were more changes since the previous sync started. This should conclude everything needed for replication itself. High-availability NFS setup --------------------------- Once you have replication, it's of course nice if the system automatically recovers from a broken storage. In NFS-based setups the idea is to do soft mounts, so if the NFS server goes away things start failing with EIO errors, which Dovecot notices and switches to using the secondary storage(s). In v2.1.0 Dovecot already keeps track of mounted filesystems. Initially they're all marked as "online". When multiple I/O errors occur in a filesystem [todo: how many exactly? where are these errors checked, all around in the code or checking the log?] the mountpoint is marked as "offline" and the connections accessing that storage are killed [todo: again how exactly?]. Another job for replication plugin is to hook into namespace creation. If mail_location points to a mountpoint marked as "offline", it's replaced with mail_replica. This way the user can access mails from the secondary storage without downtime. If the replica isn't fully up to date, this means that some of the mails (or other changes) may temporarily be lost. These will come back again after the original storage has come back up and replication has finished its job. So as long as mails aren't lost in the original storage, there won't be any permanent mail loss. When an offline storage comes back online, its mountpoint's status is initially changed to "failover" (as opposed to "online"). During this state the replication plugin works a bit differently when the user's primary mail_location is in this storage: It first checks if the user is fully replicated, and if so uses the primary storage, otherwise it uses the replica storage. Long running IMAP protocesses check the replication state periodically and kill themselves once the user is replicated, to move back to primary storage. Once replicator notices that all users have been replicated, it tells the backends' to change the "failover" state to "online" (via doveadm server). High-availability non-NFS setup ------------------------------- One possibility is to use Dovecot proxies, which know which servers are down. Instead of directing users to those servers, it would direct them to replica servers. The server states could be handled similar to NFS setup's online vs. failover vs. offline states. Another possibility would be to do the same as above, except without separate proxy servers. Just make "mail.example.com" DNS point to two IP addresses, and if one Dovecot notices that it's not the user's primary server, it proxies to the secondary server, unless it's down. If one IP is down, clients hopefully connect to the other. From tss at iki.fi Tue Feb 28 16:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:16:47 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <1330438607.2081.3.camel@innu> On Tue, 2012-02-28 at 14:41 +0100, kfx wrote: > but thunderbird returns nothing (it says not match found for "hello" > for example). Test by talking IMAP protocol, not by using Thunderbird. From CMarcus at Media-Brokers.com Tue Feb 28 16:23:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:23:29 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <4F4CE361.3010705@Media-Brokers.com> On 2012-02-28 8:41 AM, kfx wrote: > Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the > solr's data directory is growing and log shows things like this: > > Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages > in SubFolder > > > but thunderbird returns nothing (it says not match found for "hello" > for example). *How* are you testing with Thunderbird? Did you enable the 'Run search on server' option in the Advanced Search window? Doing this *should* result in Thunderbird using dovecots indexes server side. -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:25:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:25:30 +0200 Subject: [Dovecot] fts size In-Reply-To: <20120228140233.GA18412@dibs.tanso.net> References: <20120228140233.GA18412@dibs.tanso.net> Message-ID: <1330439130.2081.8.camel@innu> On Tue, 2012-02-28 at 15:02 +0100, Jan-Frode Myklebust wrote: > Does anybody have any numbers for how large storage one will need for > the fts indexing server? I see the wiki says 30% of mailbox size for > Squat (partial=4 full=4). Is it similar for lucene/solr? My initial tests for CLucene were that it would take 30% of mailbox size (compared to 50% for Xapian). But this was before I actually implemented it to Dovecot.. I haven't really looked at how large the indexes actually are. Looking at Lucene indexes now for all of Dovecot and Dovecot-cvs mailing list archives, the indexes are 29.1% of the original mbox sizes. > Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene > will create an index for each user in his home directory? Yes. > Will this be accounted for in the users' quota? Only if you're using filesystem quota. From CMarcus at Media-Brokers.com Tue Feb 28 16:27:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:27:56 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330437834.2081.2.camel@innu> References: <1330437834.2081.2.camel@innu> Message-ID: <4F4CE46C.3000507@Media-Brokers.com> On 2012-02-28 9:03 AM, Timo Sirainen wrote: > This document describes a design for a dsync-replicated Dovecot cluster. Wow, talk about timing... this looks like a perfect answer to my previous question about setting up two servers in two different locations to serve two different sets of users locally... Looks like it wouldn't really matter which server they connected to externally, as any changes would simply be replicated. Timo, state above that this 'describes a design'... does this mean that this is doable right now? Which of the following terms would you use to describe it at this point in time? Potentially problematic Reliable Rock-solid Thanks!! -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:43:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:43:38 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4CE46C.3000507@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> Message-ID: <1330440218.2081.14.camel@innu> On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: > On 2012-02-28 9:03 AM, Timo Sirainen wrote: > > This document describes a design for a dsync-replicated Dovecot cluster. > > > > Wow, talk about timing... this looks like a perfect answer to my > previous question about setting up two servers in two different > locations to serve two different sets of users locally... Yeah, I was delaying answering it until I posted this. > Looks like it wouldn't really matter which server they connected to > externally, as any changes would simply be replicated. Well, preferably users would connect to the same server. If both servers have done changes at the same time, there can be some temporary (possibly user-visible) confusion. > Timo, state above that this 'describes a design'... does this mean that > this is doable right now? Which of the following terms would you use to > describe it at this point in time? > > Potentially problematic Not finished yet, so closest to this. :) I'm planning on making our own mails use this within a few weeks (currently it's a dsync in crontab every 5 minutes). From kadafax at gmail.com Tue Feb 28 16:47:53 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 15:47:53 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE361.3010705@Media-Brokers.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> Message-ID: <4F4CE919.8070505@gmail.com> On 02/28/2012 03:23 PM, Charles Marcus wrote: > On 2012-02-28 8:41 AM, kfx wrote: >> Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the >> solr's data directory is growing and log shows things like this: >> >> Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages >> in SubFolder >> >> >> but thunderbird returns nothing (it says not match found for "hello" >> for example). > > *How* are you testing with Thunderbird? > > Did you enable the 'Run search on server' option in the Advanced Search > window? Doing this *should* result in Thunderbird using dovecots indexes > server side. > Yes I did. Some more info: by telnet'ing directly and issuing: c search text pattern * SEARCH 1208 c OK Search completed (0.003 secs). But the same search in thunderbird return "No matches found" :( Here are the solr logs, first search is the "c search text pattern" second is from thunderbird: Feb 28, 2012 3:26:21 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=(hdr:"pattern"+OR+body:"pattern")&rows=108227} hits=1 status=0 QTime=1 Feb 28, 2012 3:27:49 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=body:"pattern"&rows=108227} hits=0 status=0 QTime=1 It seems thunderbird is only searching in the body and not the header unlike the "c search text" method and that the pattern was found in the header of some random email. So none of the two method is working when searching in the body of the emails but work for the headers (???). fts_solr module seems to be correctly launched: Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so [root at server log]# grep -R mail_plugins /usr/local/etc/dovecot/ | grep -v \# /usr/local/etc/dovecot/conf.d/10-mail.conf:mail_plugins = autocreate fts fts_solr quota /usr/local/etc/dovecot/conf.d/15-lda.conf: mail_plugins = $mail_plugins sieve [root at server log]# grep fts /usr/local/etc/dovecot/conf.d/90-plugin.conf fts = solr fts_solr = url=http://127.0.0.1:8983/solr/ break-imap-search Don't know where to search :( From tss at iki.fi Tue Feb 28 16:57:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:57:22 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <1330441042.2081.24.camel@innu> On Tue, 2012-02-28 at 15:47 +0100, kfx wrote: > > Did you enable the 'Run search on server' option in the Advanced Search > > window? Doing this *should* result in Thunderbird using dovecots indexes > > server side. > > > > Yes I did. > > Some more info: > > by telnet'ing directly and issuing: > c search text pattern > * SEARCH 1208 > c OK Search completed (0.003 secs). So, Solr in Dovecot works perfectly. > But the same search in thunderbird return "No matches found" :( Thunderbird problem, nothing you can do about it from Dovecot's side. From rago at lal.in2p3.fr Tue Feb 28 18:11:55 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 16:11:55 +0000 Subject: [Dovecot] Master user In-Reply-To: <4F4BB687.3020701@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> Message-ID: <4F4CFCCB.8060903@lal.in2p3.fr> Hi, I'd like to have a master user that can read the other users mailboxes. I have only one user who read and write the mail (i.e. I use the static userdb) and I use namespace for public and shared mailboxes. I can authenticate to the server but I am not able to list the mailboxes: > emiliano at dovecot:~$ telnet localhost imap > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. > 1 login user*master password > 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in > 5 LIST "" * > * LIST (\Unmarked) "." "INBOX" > * LIST (\Noselect \HasNoChildren) "." "shared" > * LIST (\Noselect \HasChildren) "." "public" > 5 OK List completed. > * BYE Server shutting down. > Connection closed by foreign host. This is my conf: > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) ext4 > auth_master_user_separator = * > disable_plaintext_auth = no > mail_location = maildir:/data/MAIL/MAILDIR/%u:INBOX=/data/MAIL/INBOX/%u:INDEX=/data/MAIL/METADATA/%u > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > list = yes > location = maildir:/data/MAIL/MAILDIR/%%u:INDEX=/data/MAIL/MAILDIR/shared/%u/%%u > prefix = shared.%%u. > separator = . > subscriptions = no > type = shared > } > namespace { > list = yes > location = maildir:/data/MAIL/SHARED > prefix = public. > separator = . > subscriptions = no > type = public > } > passdb { > driver = pam > } > passdb { > args = /etc/dovecot/master-shared > driver = passwd-file > master = yes > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_anyone = allow > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db > } > protocols = imap > service auth { > unix_listener auth-userdb { > group = mailreader > mode = 0600 > user = mailreader > } > } > ssl = no > ssl_cert = ssl_key = userdb { > args = uid=mailreader gid=mailreader home=/ allow_all_users=yes > driver = static > } > protocol lda { > mail_plugins = acl > } > protocol imap { > mail_plugins = acl imap_acl > } And in /etc/dovecot/master-shared: > master:{PLAIN}password What I did wrong? Thanks, Emiliano From tss at iki.fi Tue Feb 28 17:37:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 17:37:22 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4CFCCB.8060903@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> Message-ID: <1330443442.2081.29.camel@innu> On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > I'd like to have a master user that can read the other users mailboxes. > I have only one user who read and write the mail (i.e. I use the static > userdb) and I use namespace for public and shared mailboxes. > I can authenticate to the server but I am not able to list the mailboxes: With ACL plugin enabled, the master user's ACLs are used. And your master user has no ACLs, so nothing is shown. The proper fix for this would be to set a default ACL for the master user, but default ACLs don't exist yet. So you'll have to kludge this, by changing the master user back to the login username. Try if this works: userdb { args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u driver = static } Also don't set your home to /. Better not to set it at all. http://wiki2.dovecot.org/VirtualUsers/Home From kadafax at gmail.com Tue Feb 28 18:05:11 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 17:05:11 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <4F4CFB37.2080505@gmail.com> Ok I feel ashame... it was a third party init scrip who was the problem :( Sorry for the noise and thank you for dovecot From CMarcus at Media-Brokers.com Tue Feb 28 18:28:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:28:15 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CFB37.2080505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> <4F4CFB37.2080505@gmail.com> Message-ID: <4F4D009F.7000107@Media-Brokers.com> On 2012-02-28 11:05 AM, kfx wrote: > Ok I feel ashame... it was a third party init scrip who was the problem :( > > Sorry for the noise and thank you for dovecot So... you're saying that Thunderbird now correctly uses server side search? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 18:33:28 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:33:28 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? Message-ID: <4F4D01D8.7000704@Media-Brokers.com> Hey Timo, Subject says it all... This question is a result of an ongoing discussion on the mozilla enterprise list... Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search (with indexes)? Thanks, -- Best regards, Charles From tss at iki.fi Tue Feb 28 18:38:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:38:32 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D01D8.7000704@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.33, Charles Marcus wrote: > This question is a result of an ongoing discussion on the mozilla enterprise list... > > Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search All IMAP servers are required to support SEARCH command. Some crappy ones don't, but I think all widely used ones do. > (with indexes)? No way to know that. Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. (I don't know if there are any other servers besides Dovecot implementing FUZZY.) From tss at iki.fi Tue Feb 28 18:51:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:51:57 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.38, Timo Sirainen wrote: > Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. I meant SEARCH=FUZZY > And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. v2.1.2 will no longer advertise it unless fts=solr or fts=lucene: http://hg.dovecot.org/dovecot-2.1/rev/bdc881838b00 From steve.platt at mrc-bsu.cam.ac.uk Tue Feb 28 19:45:49 2012 From: steve.platt at mrc-bsu.cam.ac.uk (Steve Platt) Date: Tue, 28 Feb 2012 17:45:49 +0000 Subject: [Dovecot] migrating/converting from system users -> virtual users Message-ID: Hello all, We currently have a traditional mail server where all users have system accounts (ie entries in the NIS passwd map) and mbox-format mail folders in their (system) home directories. I'm trying to setup a dovecot server in which we want all users to have "virtual" accounts (in dovecot) and no entry in the passwd file at all (ie no access to the mail server). This is for security reasons, we want to imagine that users connecting to the mail server in future will not be able to access any resources on the server other than access their email messages. To this end we've set up a "vmail" user that owns all the messages in the Maildir directories. Most of this is working but I'm stuck on how to convert users' mail folders from the existing setup to the new one. I'm using the convert plugin but of course the problem is that the plugin executes as the "vmail" user and cannot access the existing mail folders that belong to the users: and I'd be worried if it could, of course! So I've converted a few users already by making copies of their old-world mailboxes, changing the ownership of the copies and pointing the convert plugin to these copies, instead of the originals. This works (modulo the dreaded "timeouted" error messages) but is very ugly. So I tried looking at the convert-tool but can't find a man page for it so am not sure what I'm doing with it! I have the idea that I should be able to run some command (as a privileged user) on the mail server and have it do the conversion for me, changing the ownership/permissions on the way. Can convert-tool do this? I'd prefer to go with the automatic (plugin) conversion if I can bodge the ownership issues somehow. Failing that, some tool or script may be the next best answer. This is Dovecot 1.2 by the way on Solaris 10. Any suggestions would be welcome! Steve Platt From list at airstreamcomm.net Tue Feb 28 20:11:29 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 12:11:29 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup Message-ID: We are considering using the post-login scripting to clear trash older than 90 days from user accounts. has anyone done this, and if so did this cause logins to slow down too much waiting for the trash to purge? One idea was to execute the trash purge script once a day by tracking their logins and seeing that it has already ran that day. Another idea was to call the trash purge script in the background and continue without acknowledging that it has finished to keep logins speedy. From robert at schetterer.org Tue Feb 28 20:26:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 28 Feb 2012 19:26:11 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D1C43.5000209@schetterer.org> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. look here if this match/solve your problem http://wiki2.dovecot.org/Plugins/Expire -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From CMarcus at Media-Brokers.com Tue Feb 28 20:29:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 13:29:32 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: <4F4D1D0C.7040708@Media-Brokers.com> On 2012-02-28 11:51 AM, Timo Sirainen wrote: > On 28.2.2012, at 18.38, Timo Sirainen wrote: > >> Well, okay, actually if server advertises FUZZY extension you can >> be quite certain that it supports indexed server side searches. > > I meant SEARCH=FUZZY Thanks for the replies Timo... Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Thanks again... hopefully something will comes of this for Thunderbird. -- Best regards, Charles From tss at iki.fi Tue Feb 28 20:31:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:31:46 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D1D0C.7040708@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> <4F4D1D0C.7040708@Media-Brokers.com> Message-ID: <70448DE8-056A-4453-BC45-79C82C781C78@iki.fi> On 28.2.2012, at 20.29, Charles Marcus wrote: > Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Neither. Probably no servers besides Dovecot. But it is less than a year old RFC.. From rago at lal.in2p3.fr Tue Feb 28 20:45:14 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 19:45:14 +0100 Subject: [Dovecot] Master user In-Reply-To: <1330443442.2081.29.camel@innu> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> Message-ID: <4F4D20BA.3060501@lal.in2p3.fr> On 02/28/2012 04:37 PM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > >> I'd like to have a master user that can read the other users mailboxes. >> I have only one user who read and write the mail (i.e. I use the static >> userdb) and I use namespace for public and shared mailboxes. >> I can authenticate to the server but I am not able to list the mailboxes: > > With ACL plugin enabled, the master user's ACLs are used. And your > master user has no ACLs, so nothing is shown. The proper fix for this > would be to set a default ACL for the master user, but default ACLs > don't exist yet. So you'll have to kludge this, by changing the master > user back to the login username. Try if this works: > > userdb { > args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u > driver = static > } Thanks, but nothing has changed. Emiliano From Ralf.Hildebrandt at charite.de Tue Feb 28 20:46:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 19:46:20 +0100 Subject: [Dovecot] 2.1.1: subscriptions: Removing invalid entry Message-ID: <20120228184619.GA3250@charite.de> Admittedly, it's not a valid folder name and dovecot is right in removing it: Feb 28 17:50:05 postamt dovecot: imap(kbaessle): Warning: Subscriptions file /home/k/b/kbaessle/Maildir/subscriptions: Removing invalid entry: Entw?rfe ...but when looking at the subscription file AFTER the removal, that file contained an empty line. I don't think that's correct. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 28 20:47:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:47:37 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D20BA.3060501@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> Message-ID: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> On 28.2.2012, at 20.45, Emiliano Rago wrote: >> userdb { >> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >> driver = static >> } > > Thanks, > but nothing has changed. Set auth_debug=yes. What does it log when you login? From Ralf.Hildebrandt at charite.de Tue Feb 28 21:02:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 20:02:24 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 Message-ID: <20120228190224.GB3250@charite.de> With 2.0.18 I was using a quota warning script like this: plugin { ... # Quota quota = maildir quota_rule = INBOX.Trash:storage=+2048M quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u } # der schickt die Quota warnmails service quota-warning { executable = script /usr/local/scripts/quota-warning2.1 user = root unix_listener quota-warning { mode = 0666 user = vmail group = users } } -rwxr-xr-x 1 root staff 3081 Feb 28 16:56 /usr/local/scripts/quota-warning2.1 But now I'm gettting this in my log: Feb 28 19:19:03 postamt quota-warning2(phxxxpsu): writing message failed Feb 28 19:19:03 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 15065 returned error 1 Feb 28 19:40:13 postamt quota-warning2(lxxxve): writing message failed Feb 28 19:40:13 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 3318 returned error 1 What is failing here? #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit some warning in german EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de -------------- next part -------------- #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Was ist los? ============ Wir haben festgestellt, dass Ihre Mailbox auf dem Server postamt.charite.de das erlaubte Gr?ssenlimit zu mindestens ${MAIL_QUOTA_PERCENT}% aussch?pft. D.h. die maximal erlaubte Gesamtgr??e der Mailbox wird bald erreicht sein oder ist es sogar schon! Was kann ich nun tun? ===================== Bitte schaffen Sie Platz in Ihrer Mailbox, indem Sie die Mails z.B. auf dem Server l?schen oder in Ihre lokale Mailbox auf Ihrem eigenen Rechner verschieben und so wieder Platz auf dem Server schaffen! Wie kann ich die F?llung meines Postfaches auf dem Server ?berpr?fen? ===================================================================== Sie k?nnen die F?llung Ihres Postfaches ?ber Webmail https://webmail.charite.de/ leicht ?berpr?fen: Nach dem Anmelden erscheint im linken Rahmen unter Ihrer Ordnerliste ein Link namens "Ordnergr??en". Sie k?nnen in Webmail ggf. auch gleich die Mails l?schen (vergessen Sie hinterher nicht, den Papierkorb / Trash Ordner zu leeren!). Warum ist mein Postfach ?berhaupt so voll? ========================================== Dies kann z.B. passieren, wenn Sie mehrere Rechner nutzen um Ihre Mail zu lesen und statt des IMAP Protokolls das POP3-Protokoll mit der aktivierten Option "Leave Mail on Server" / "Mail auf Server belassen" in Ihrem Mailprogramm verwenden. Oft wird auch vergessen, da? der Ordner mit den "Gesendeten Objekten" (Sent Folder/Sent Items) zum Gr??enlimit der Mailbox z?hlen. Besondere Platzfresser sind Mails mit gro?en Anh?ngen, wie z.B. PowerPoint-, PDF- oder Bilddateien -- insbesondere im "Sent" Ordner! Wer sind meine Ansprechpartner? =============================== Sie k?nnen uns unter postmaster at charite.de oder telefonisch unter 450-570600 erreichen. Mit freundlichen Gr??en EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log From rago at lal.in2p3.fr Tue Feb 28 21:04:23 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 20:04:23 +0100 Subject: [Dovecot] Master user In-Reply-To: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> Message-ID: <4F4D2537.7070507@lal.in2p3.fr> On 02/28/2012 07:47 PM, Timo Sirainen wrote: > On 28.2.2012, at 20.45, Emiliano Rago wrote: > >>> userdb { >>> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >>> driver = static >>> } >> >> Thanks, >> but nothing has changed. > > Set auth_debug=yes. What does it log when you login? That's the log: Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60873#011resp= Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: auth(admin,127.0.0.1,master): Master user lookup for login: rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: passwd-file(admin,127.0.0.1,master): lookup: user=admin file=/etc/dovecot/master-shared Feb 28 19:57:48 oneprivvm-239 dovecot: auth: passdb(admin,127.0.0.1,master): Master user logging in as rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client out: OK#0111#011user=rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master in: REQUEST#0113607363585#0112591#0111#0114d6e591e980edb0fabccfa9c1cd4e198 Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master out: USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Feb 28 19:57:48 oneprivvm-239 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2596, secured Thanks for answering, Emiliano From tss at iki.fi Tue Feb 28 21:09:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 21:09:02 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D2537.7070507@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> Message-ID: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> On 28.2.2012, at 21.04, Emiliano Rago wrote: > USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting From list at airstreamcomm.net Tue Feb 28 21:26:35 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 13:26:35 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <4F4D1C43.5000209@schetterer.org> References: <4F4D1C43.5000209@schetterer.org> Message-ID: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer wrote: > Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >> We are considering using the post-login scripting to clear trash older >> than 90 days from user accounts. has anyone done this, and if so did >> this >> cause logins to slow down too much waiting for the trash to purge? One >> idea was to execute the trash purge script once a day by tracking their >> logins and seeing that it has already ran that day. Another idea was to >> call the trash purge script in the background and continue without >> acknowledging that it has finished to keep logins speedy. > > look here if this match/solve your problem > > http://wiki2.dovecot.org/Plugins/Expire Expire looks to be useful, but it appears to be something that enhances expunging of messages rather than automates the process if I am reading correctly. We would like to make the process for expunging old Trash messages as automated and inline as possible. From lists at wildgooses.com Tue Feb 28 21:56:22 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 28 Feb 2012 19:56:22 +0000 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D3166.90809@wildgooses.com> On 28/02/2012 18:11, list at airstreamcomm.net wrote: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. I think you can also use doveadm to achieve this? So you could schedule something for all accounts at some out of hours period - should speed up backups also? Ed W From CMarcus at Media-Brokers.com Tue Feb 28 22:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 15:10:12 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330440218.2081.14.camel@innu> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> Message-ID: <4F4D34A4.6090106@Media-Brokers.com> On 2012-02-28 9:43 AM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: >> On 2012-02-28 9:03 AM, Timo Sirainen wrote: >>> This document describes a design for a dsync-replicated Dovecot cluster. >> >> >> >> Wow, talk about timing... this looks like a perfect answer to my >> previous question about setting up two servers in two different >> locations to serve two different sets of users locally... > Yeah, I was delaying answering it until I posted this. Cool, thanks - I look forward to any comments you might have on my current plan. >> Looks like it wouldn't really matter which server they connected to >> externally, as any changes would simply be replicated. > Well, preferably users would connect to the same server. If both servers > have done changes at the same time, there can be some temporary > (possibly user-visible) confusion. Point taken - and thinking about it, the amount of traffic would be negligible, since users accessing emails remotely is much less than internally. But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. >> Timo, state above that this 'describes a design'... does this mean that >> this is doable right now? Which of the following terms would you use to >> describe it at this point in time? >> >> Potentially problematic > Not finished yet, so closest to this. :) I'm planning on making our own > mails use this within a few weeks (currently it's a dsync in crontab > every 5 minutes). Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Thanks! -- Best regards, Charles From Ralf.Hildebrandt at charite.de Tue Feb 28 23:30:21 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 22:30:21 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 In-Reply-To: <20120228190224.GB3250@charite.de> References: <20120228190224.GB3250@charite.de> Message-ID: <20120228213021.GG3250@charite.de> * Ralf Hildebrandt : > With 2.0.18 I was using a quota warning script like this: Must have been a PEBKAC - I fixed my installation by harmonising $PREFIX (seperate $PREFIX for 2.0 and 2.1, removig an old installation along the way) as well as cleaning up some old scripts and alas: Feb 28 22:13:15 postamt quota-warning2(lscxxxk) 90%: delivered! Feb 28 22:15:40 postamt quota-warning2(sxxxwart) 85%: delivered! Feb 28 22:17:39 postamt quota-warning2.1(sxxxwart) 90%: delivered! Feb 28 22:27:55 postamt quota-warning2.1(rxxxid) 85%: delivered! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From rago at lal.in2p3.fr Wed Feb 29 03:06:37 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Wed, 29 Feb 2012 02:06:37 +0100 Subject: [Dovecot] Master user In-Reply-To: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> Message-ID: <4F4D7A1D.9040002@lal.in2p3.fr> On 02/28/2012 08:09 PM, Timo Sirainen wrote: > On 28.2.2012, at 21.04, Emiliano Rago wrote: > >> USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin > > Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting > Great, with postlogin script it works! Thanks a lot, Emiliano From amk at spamfence.net Wed Feb 29 03:03:58 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Wed, 29 Feb 2012 01:03:58 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <20120224012247.GA6512@krell.zikzak.de> <1330342560.11500.308.camel@innu.invalid> Message-ID: Timo Sirainen wrote: >> There seems to be a new dependency in some modules (eg, lib-storage, >> libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those >> modules didn't require OpenSSL, but 2.1 does. >> >> For the linking process the path to the OpenSSL library isn't >> specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL >> is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't >> properly used as well.) > > Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes > everything? That's the way to go. Makes things better, but I've found three more dependencies. This patch is against the daily snapshot 20120228. (Sorry for the changes to Makefile.in which you won't need. But this way I don't have to rebuild Makefile.in from Makefile.am when compiling. ;-) diff -ur dovecot-20120228.original/src/auth/Makefile.am dovecot-20120228/src/auth/Makefile.am --- dovecot-20120228.original/src/auth/Makefile.am 2012-02-26 04:00:02.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.am 2012-02-29 01:38:41.000000000 +0100 @@ -167,7 +167,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ -I$(top_srcdir)/src/lib-imap \ diff -ur dovecot-20120228.original/src/auth/Makefile.in dovecot-20120228/src/auth/Makefile.in --- dovecot-20120228.original/src/auth/Makefile.in 2012-02-28 04:00:12.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.in 2012-02-29 01:38:44.000000000 +0100 @@ -499,7 +499,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ diff -ur dovecot-20120228.original/src/lib-storage/Makefile.am dovecot-20120228/src/lib-storage/Makefile.am --- dovecot-20120228.original/src/lib-storage/Makefile.am 2012-02-10 04:00:02.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -85,7 +85,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/lib-storage/Makefile.in dovecot-20120228/src/lib-storage/Makefile.in --- dovecot-20120228.original/src/lib-storage/Makefile.in 2012-02-28 04:00:15.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -397,7 +397,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic test_programs = \ diff -ur dovecot-20120228.original/src/login-common/Makefile.am dovecot-20120228/src/login-common/Makefile.am --- dovecot-20120228.original/src/login-common/Makefile.am 2011-11-09 04:00:01.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -40,6 +40,6 @@ pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/login-common/Makefile.in dovecot-20120228/src/login-common/Makefile.in --- dovecot-20120228.original/src/login-common/Makefile.in 2012-02-28 04:00:18.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -320,7 +320,7 @@ pkginc_lib_HEADERS = $(headers) pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic all: all-am From robert at schetterer.org Wed Feb 29 08:44:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 29 Feb 2012 07:44:54 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> References: <4F4D1C43.5000209@schetterer.org> <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> Message-ID: <4F4DC966.4000609@schetterer.org> Am 28.02.2012 20:26, schrieb list at airstreamcomm.net: > On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer > wrote: >> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >>> We are considering using the post-login scripting to clear trash older >>> than 90 days from user accounts. has anyone done this, and if so did >>> this >>> cause logins to slow down too much waiting for the trash to purge? One >>> idea was to execute the trash purge script once a day by tracking their >>> logins and seeing that it has already ran that day. Another idea was > to >>> call the trash purge script in the background and continue without >>> acknowledging that it has finished to keep logins speedy. >> >> look here if this match/solve your problem >> >> http://wiki2.dovecot.org/Plugins/Expire > > Expire looks to be useful, but it appears to be something that enhances > expunging of messages rather than automates the process if I am reading > correctly. We would like to make the process for expunging old Trash > messages as automated and inline as possible. wouldnt doveadm expunge -A mailbox Trash savedbefore 90d as cron job do what you want ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From busseniu at in.tum.de Wed Feb 29 11:31:38 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 10:31:38 +0100 Subject: [Dovecot] doveadm fetch prints duplicate results in 2.1 Message-ID: <4F4DF07A.7020408@in.tum.de> Hi, when the private namespace has "prefix = INBOX." and you use doveadm fetch to search for "mailbox INBOX", then it prints every message twice: # doveadm fetch -u testuser 'mailbox uid guid' mailbox INBOX mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 Without "prefix = INBOX." in the configuration, the messages are only printed once. In 2.0, they were never printed twice. doveconf -n # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mdbox mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From michel at linuxint.com Wed Feb 29 09:53:05 2012 From: michel at linuxint.com (Michel Bisson) Date: Wed, 29 Feb 2012 07:53:05 +0000 (UTC) Subject: [Dovecot] per-user limit? References: <20120116091521.GA10944@gir.theapt.org> <1326814211.11500.11.camel@innu> Message-ID: Timo Sirainen iki.fi> writes: > On Mon, 2012-01-16 at 10:15 +0100, Peter Hessler wrote: > > I am seeing a problem where users are limited to 6 imap logins total. > > One of my users has a bunch of phones and computers, and wants them all > > on at the same time. > > I'm looking through my configuration, and I cannot see a limit on how > > many times a single user can connect. He is connecting from different > > IPs. > > Any ideas? My logs show the following error when they attempt to auth > > for a 7th time: Hi, I also get the same problem of the same user getting a password mismatch error when may devices are tryibg to login to the same IMAP account at the same time. I didn't check how manylogins triggers this nasty behavior but noticed that this unwanted feature started right after I upgraded the the server from Debian Lenny to Squeeze. If anybody can point me to a hint about how I coukd change this behavior it woul be very much appreciated. From busseniu at in.tum.de Wed Feb 29 12:03:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:03:35 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> Message-ID: <4F4DF7F7.8020405@in.tum.de> On 21.02.2012 01:18, Timo Sirainen wrote: > On 20.2.2012, at 19.30, Metro Domain Admin wrote: > >> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Yes, but it should still work.. As far as I can tell from my tests, squat has stopped working in 2.1. Let's take a user who does not have any mails yet, and deliver a first mail to him: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now create an IMAP session, select INBOX, and do a search: * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1330509552] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. >>> . search text test * SEARCH . OK Search completed (0.006 secs). (Should have returned 1 message.) Deliver the same message again: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now in the existing session, run the search command three more times: >>> . search text test * SEARCH * 2 EXISTS * 2 RECENT . OK Search completed (0.000 secs). >>> . search text test * SEARCH 2 . OK Search completed (0.002 secs). >>> . search text test * SEARCH . OK Search completed (0.000 secs). It found the message once, but the next time it didn't. So the squat search does not actually seem to work any more. I know it's deprecated, I just wanted to note this. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 29 12:05:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:05:28 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <4F4DF868.3090107@in.tum.de> On 29.02.2012 11:03, Christoph Bu?enius wrote: > It found the message once, but the next time it didn't. So the squat > search does not actually seem to work any more. Sorry, I forgot to include the configuration: # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = fts fts_squat mail_uid = vmail namespace { inbox = yes location = prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat fts_squat = partial=4 full=4 } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From Ralf.Hildebrandt at charite.de Wed Feb 29 12:09:57 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:09:57 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <20120229100957.GX13045@charite.de> * Christoph Bu?enius : > On 21.02.2012 01:18, Timo Sirainen wrote: > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. What else do I need for fts_lucene to work? Any 3rd party packages? Libs? Compile Time Options? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Wed Feb 29 12:15:52 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Wed, 29 Feb 2012 23:15:52 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: <4F4DFAD8.8040002@ecs.vuw.ac.nz> On 02/28/12 00:11, Timo Sirainen wrote: >>> Looks like kmail is sending some kind of garbage to Dovecot. Set >>> auth_debug_passwords=yes to make Dovecot log the auth traffic. >> >> Yeah, I did a network trace and it seems kmail is not sending the >> full authentication request before trying to carry on. > >>> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > > It uses SASL-IR to send the first seponse. > >>> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > > Dovecot says "OK, give me more". I poked some more at the kmail end of this but I cant see what its doing differently from what it used to (but clearly there is something). The new kmail sends 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n and receives + \r\n from dovecot, which it doesn't like and reports clientAuthenticate: sasl_client_step failed with: -1 an older (working kmail) sends 1 AUTHENTICATE GSSAPI YIICiAYJKoZIhvcSAQICAQBugg[...]MpPurY7cZfRSEw==\r\n and receives + YIGaBgkqhkiG9xIBAgI[...]iYoSGi9/uKVGyE64TAvkf25rCbFkNqk1D12g==\r\n and carries on. So what is it that differs in the two cases to cause dovecot to respond differently? cheers mark From mstevens at imt-systems.com Wed Feb 29 12:22:06 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 11:22:06 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: On 29.02.2012 11:09, Ralf Hildebrandt wrote: > * Christoph Bu?enius : >> On 21.02.2012 01:18, Timo Sirainen wrote: >> >On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> > >> >>Squat is apparently deprecated: >> http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Build it with lucene support. --with-lucene Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 12:22:50 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:22:50 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: <20120229102250.GY13045@charite.de> * Ralf Hildebrandt : > * Christoph Bu?enius : > > On 21.02.2012 01:18, Timo Sirainen wrote: > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Rebuilding with --with-lucene --with-stemmer, let's see what happens. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 12:34:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:34:44 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4DFAD8.8040002@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.15, Mark Davies wrote: > 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n > > and receives > > + \r\n > > from dovecot, which it doesn't like and reports > clientAuthenticate: sasl_client_step failed with: -1 Oh, yes, this is a bug in Dovecot.. From tss at iki.fi Wed Feb 29 12:41:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:41:45 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.34, Timo Sirainen wrote: > On 29.2.2012, at 12.15, Mark Davies wrote: > >> 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n >> >> and receives >> >> + \r\n >> >> from dovecot, which it doesn't like and reports >> clientAuthenticate: sasl_client_step failed with: -1 > > Oh, yes, this is a bug in Dovecot.. Hmm. Or it looked like a bug, since it replied only with "+", so I thought all auth mechanisms would have such a bug, but no.. So I'm not really sure why it's not sending more data. I don't have a Kerberos setup to test this with. v2.1's GSSAPI code is anyway identical to v2.0's. From tss at iki.fi Wed Feb 29 12:53:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:53:55 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <8F350712-55B7-457D-BB66-DD1E2CC536C8@iki.fi> On 29.2.2012, at 12.03, Christoph Bu?enius wrote: > On 21.02.2012 01:18, Timo Sirainen wrote: >> On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> >>> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html >> >> Yes, but it should still work.. > > As far as I can tell from my tests, squat has stopped working in 2.1. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ee6f3f375dd3 From piotr.tarnowski at us.edu.pl Wed Feb 29 14:57:35 2012 From: piotr.tarnowski at us.edu.pl (Piotr Tarnowski) Date: Wed, 29 Feb 2012 13:57:35 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) Message-ID: <4F4E20BF.8040707@us.edu.pl> Hi, I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but when I want to compile Sieve for this version (dovecot-2.1-pigeonhole-0.3.0) I get an error: "sieve.c", line 578: identifier redeclared: sieve_multiscript_run and I see that declaration in sieve.h is different from definition in sieve.c, so compilation process fails with SunStudio cc compiler. I have done a quick hack to correct this, and then compiled successfully. Patch is included as attachment. Full output, OS and compiler versions below: libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I/home/soft/dovecot/dovecot-2.1.1 -I/home/soft/dovecot/dovecot-2.1.1/src/lib -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dict -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dns -I/home/soft/dovecot/dovecot-2.1.1/src/lib-mail -I/home/soft/dovecot/dovecot-2.1.1/src/lib-imap -I/home/soft/dovecot/dovecot-2.1.1/src/lib-fs -I/home/soft/dovecot/dovecot-2.1.1/src/lib-charset -I/home/soft/dovecot/dovecot-2.1.1/src/lib-index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index/raw -I/home/soft/dovecot/dovecot-2.1.1/src/lib-lda -I/home/soft/dovecot/dovecot-2.1.1/src/lda -I/home/soft/dovecot/dovecot-2.1.1/src/lib-master -I/home/soft/dovecot/dovecot-2.1.1/src/lib-settings -DMODULEDIR=\"/usr/local/dovecot-2.1.1/lib/dovecot\" -I/usr/local/openldap/include -I/usr/local/ssl/include -I/usr/local/include -xO3 -xautopar -c sieve.c -KPIC -DPIC -o .libs/sieve.o "sieve-common.h", line 33: warning: useless declaration "/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "sieve.c", line 578: identifier redeclared: sieve_multiscript_run current : function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, unsigned int, _Bool) returning _Bool previous: function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, enum sieve_runtime_flags {SIEVE_RUNTIME_FLAG_NOGLOBAL(1)}, _Bool) returning _Bool : "sieve.h", line 168 cc: acomp failed for sieve.c *** Error code 1 dmake: Fatal error: Command failed for target `sieve.lo' Current working directory /home/soft/dovecot/dovecot-2.1-pigeonhole-0.3.0/src/lib-sieve *** Error code 1 The following command caused the error: fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='plugins'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && dmake $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ dmake "$target-am" || exit 1; \ fi; test -z "$fail" dmake: Fatal error: Command failed for target `all-recursive' % uname -a SunOS soft 5.10 Generic_142909-17 sun4u sparc SUNW,Sun-Fire-V890 Solaris % version Machine hardware: sun4u OS version: 5.10 Processor type: sparc Hardware: SUNW,Sun-Fire-V890 The following components are installed on your system: Sun Studio 12 update 1 Sun Studio 12 update 1 C Compiler Sun Studio 12 update 1 C++ Compiler Sun Studio 12 update 1 Tools.h++ 7.1 Sun Studio 12 update 1 C++ Standard 64-bit Class Library Sun Studio 12 update 1 Garbage Collector Sun Studio 12 update 1 Fortran 95 Sun Studio 12 update 1 Debugging Tools (including dbx) Sun Studio 12 update 1 IDE Sun Studio 12 update 1 Performance Analyzer (including collect, ...) Sun Studio 12 update 1 Performance Library Sun Studio 12 update 1 Scalapack Sun Studio 12 update 1 LockLint Sun Studio 12 update 1 Building Software (including dmake) Sun Studio 12 update 1 Documentation Set Sun Studio 12 update 1 /usr symbolic links and GNOME menu item version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/cc": Sun C 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/CC": Sun C++ 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/f90": Sun Fortran 95 8.4 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dbx": Sun DBX Debugger 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/analyzer": Sun Analyzer 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dmake": Sun Distributed Make 7.9 SunOS_sparc 2009/06/03 Regards -- [ Piotr Tarnowski piotr.tarnowski at us.edu.pl ] [ Unix Administrator at University of Silesia, Katowice, Poland ] -------------- next part -------------- A non-text attachment was scrubbed... Name: sieve.h-redeclared.patch Type: text/x-patch Size: 486 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5810 bytes Desc: S/MIME Cryptographic Signature URL: From Ralf.Hildebrandt at charite.de Wed Feb 29 15:27:19 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:27:19 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229102250.GY13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> Message-ID: <20120229132718.GN13045@charite.de> * Ralf Hildebrandt : > * Ralf Hildebrandt : > > * Christoph Bu?enius : > > > On 21.02.2012 01:18, Timo Sirainen wrote: > > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > > What else do I need for fts_lucene to work? Any 3rd party packages? > > Libs? Compile Time Options? > > Rebuilding with --with-lucene --with-stemmer, let's see what happens. The plugin doesn't build: postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/config/compiler.h:40:0, from /usr/include/CLucene/StdHeader.h:17, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/config/CompilerGcc.h:12:68: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. make: *** [lucene-wrapper.lo] Error 1 # dpkg -L libclucene-dev |grep -i clucene-config /usr/lib/CLucene/clucene-config.h That's retarded! Who's putting includes into /usr/lib/CLucene/ ??? ARGH! After symlinking, I'm getting: # make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:24:36: fatal error: CLucene/util/CLStreams.h: No such file or directory compilation terminated. Where do I get this include from (on Debian)? # dpkg -l |grep lucene ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 15:29:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 15:29:44 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229132718.GN13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> Message-ID: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > The plugin doesn't build: > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make .. > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) It requires CLucene 2.3.2. From Ralf.Hildebrandt at charite.de Wed Feb 29 15:36:55 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:36:55 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <20120229133655.GO13045@charite.de> * Timo Sirainen : > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > The plugin doesn't build: > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > .. > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > It requires CLucene 2.3.2. ARGH. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 15:54:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 14:54:49 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> On 29.02.2012 14:29, Timo Sirainen wrote: > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > >> The plugin doesn't build: >> >> postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# >> make > .. >> ii libclucene-dev 0.9.21b-2+b1 library for >> full-featured text search engine (development) >> ii libclucene0ldbl 0.9.21b-2+b1 library for >> full-featured text search engine (runtime) > > It requires CLucene 2.3.2. Hi Timo, I have the same problem. It does not build with 2.3.3. [mockbuild at x86-002 result]$ cat root.log | grep clucene DEBUG util.py:257: --> clucene-core-devel-2.3.3.4-3.fc16.x86_64 DEBUG util.py:257: clucene-core-devel x86_64 2.3.3.4-3.fc16 updates-released 258 k DEBUG util.py:257: clucene-core x86_64 2.3.3.4-3.fc16 updates-released 507 k DEBUG util.py:257: clucene-core-devel.x86_64 0:2.3.3.4-3.fc16 DEBUG util.py:257: clucene-core.x86_64 0:2.3.3.4-3.fc16 build.log: Making all in fts-lucene make[4]: Entering directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o doveadm-fts-lucene.lo doveadm-fts-lucene.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-lucene-plugin.lo fts-lucene-plugin.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-backend-lucene.lo fts-backend-lucene.c /bin/sh ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-backend-lucene.c -fPIC -DPIC -o .libs/fts-backend-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c doveadm-fts-lucene.c -fPIC -DPIC -o .libs/doveadm-fts-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-lucene-plugin.c -fPIC -DPIC -o .libs/fts-lucene-plugin.o libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/StdHeader.h:20:0, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/SharedHeader.h:18:36: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. /bin/sh ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -Wl,-z,relro -o lib20_doveadm_fts_lucene_plugin.la -rpath /usr/lib64/dovecot/doveadm doveadm-fts-lucene.lo -lrt make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... libtool: link: gcc -shared -fPIC -DPIC .libs/doveadm-fts-lucene.o -lrt -O2 -m64 -mtune=generic -Wl,-z -Wl,relro -Wl,-soname -Wl,lib20_doveadm_fts_lucene_plugin.so -o .libs/lib20_doveadm_fts_lucene_plugin.so libtool: link: ( cd ".libs" && rm -f "lib20_doveadm_fts_lucene_plugin.la" && ln -s "../lib20_doveadm_fts_lucene_plugin.la" "lib20_doveadm_fts_lucene_plugin.la" ) make[4]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' make[3]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins' make[3]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src' make[2]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1' make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 RPM build errors: error: Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Child returncode was: 1 EXCEPTION: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/mockbuild/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 352, in do raise mockbuild.exception.Error, ("Command failed. See logs for output.\n # %s" % (command,), child.returncode) Error: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] LEAVE do --> EXCEPTION RAISED Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 15:58:51 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:58:51 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> Message-ID: <20120229135851.GU13045@charite.de> * Morten Stevens : > >It requires CLucene 2.3.2. > > Hi Timo, > > I have the same problem. It does not build with 2.3.3. I'm right now trying exactly this :) > In file included from /usr/include/CLucene/StdHeader.h:20:0, > from /usr/include/CLucene.h:11, > from lucene-wrapper.cc:23: > /usr/include/CLucene/SharedHeader.h:18:36: fatal error: > CLucene/clucene-config.h: No such file or directory Did you try symlinking the file? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:09:41 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:09:41 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229135851.GU13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> Message-ID: <1be342370509d17ae81682aede00f016@imt-systems.com> On 29.02.2012 14:58, Ralf Hildebrandt wrote: > * Morten Stevens : > >> >It requires CLucene 2.3.2. >> >> Hi Timo, >> >> I have the same problem. It does not build with 2.3.3. > > I'm right now trying exactly this :) > >> In file included from /usr/include/CLucene/StdHeader.h:20:0, >> from /usr/include/CLucene.h:11, >> from lucene-wrapper.cc:23: >> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >> CLucene/clucene-config.h: No such file or directory > > Did you try symlinking the file? I'm not sure if this is possible because we use a chroot build environment to build RPM packages for Fedora and RHEL. See: http://fedoraproject.org/wiki/Projects/Mock Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 16:27:39 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:27:39 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229133655.GO13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> Message-ID: <20120229142738.GW13045@charite.de> * Ralf Hildebrandt : > * Timo Sirainen : > > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > > > The plugin doesn't build: > > > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > > .. > > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > > > It requires CLucene 2.3.2. > > ARGH. I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the info. Maybe the configure should check for a version >=2.3.2! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:29:29 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:29:29 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <1be342370509d17ae81682aede00f016@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> Message-ID: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> On 29.02.2012 15:09, Morten Stevens wrote: > On 29.02.2012 14:58, Ralf Hildebrandt wrote: >> * Morten Stevens : >> >>> >It requires CLucene 2.3.2. >>> >>> Hi Timo, >>> >>> I have the same problem. It does not build with 2.3.3. >> >> I'm right now trying exactly this :) >> >>> In file included from /usr/include/CLucene/StdHeader.h:20:0, >>> from /usr/include/CLucene.h:11, >>> from lucene-wrapper.cc:23: >>> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >>> CLucene/clucene-config.h: No such file or directory >> >> Did you try symlinking the file? > > I'm not sure if this is possible because we use a chroot build > environment to build RPM packages for Fedora and RHEL. > See: http://fedoraproject.org/wiki/Projects/Mock Update: This is a Fedora-specific problem, because clucene (build requirement) is not correctly packaged. Best regards, Morten From tss at iki.fi Wed Feb 29 16:29:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 16:29:57 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229142738.GW13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> <20120229142738.GW13045@charite.de> Message-ID: <8DB2FDBF-02E3-4E55-A817-85D62668C756@iki.fi> On 29.2.2012, at 16.27, Ralf Hildebrandt wrote: > I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the > info. Maybe the configure should check for a version >=2.3.2! I've thought about adding such checks, but autoconf seems to make it difficult to do C++ checks so I haven't bothered.. But I added that info to wiki. From Ralf.Hildebrandt at charite.de Wed Feb 29 16:30:38 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:30:38 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> References: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> Message-ID: <20120229143038.GX13045@charite.de> * Morten Stevens : > This is a Fedora-specific problem, because clucene (build > requirement) is not correctly packaged. Well, debian showed the same packaging (wrong place). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 17:51:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 17:51:47 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4D34A4.6090106@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> <4F4D34A4.6090106@Media-Brokers.com> Message-ID: <99E3DC46-0709-4DFD-A751-DCA9190E4874@iki.fi> On 28.2.2012, at 22.10, Charles Marcus wrote: > But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. Yes, there are some ways to do this. Although you would most likely want Dovecot proxy without director feature enabled. >>> Timo, state above that this 'describes a design'... does this mean that >>> this is doable right now? Which of the following terms would you use to >>> describe it at this point in time? >>> >>> Potentially problematic > >> Not finished yet, so closest to this. :) I'm planning on making our own >> mails use this within a few weeks (currently it's a dsync in crontab >> every 5 minutes). > > Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Possibly. :) From tss at iki.fi Wed Feb 29 18:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 18:10:11 +0200 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: On 27.2.2012, at 18.54, Charles Marcus wrote: > I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? I'll initially build it for only 2 locations, but I think it will be pretty simple to scale to more than 2. > If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? Synchronous drbd replication for a master/slave setup should also work, since the latency between your servers is probably quite low. I wouldn't use asynchronous replication since it can lose some of the last changes when failure happens. Then there are of course all the cluster filesystems, but I don't have much experience with them other than what I've read in this list. I think GPFS is the only one I haven't read any complains of (but it could be also that so few people have actually used it). > 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? With a clusterfs setup you could do this. With dsync-replicated setup you could assign a primary location for the user, and proxy the connection there if user got connected to wrong server, except when the primary server is down. > I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Like other mentioned, I don't think the cross-office traffic will be that much of a problem, especially for external connections from outside office. For internal connections you should be able to mostly avoid it. From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:27:34 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:27:34 +0100 Subject: [Dovecot] What's behind the message ... Message-ID: (auto-added) ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount Hello, the above message appeared when I restarted dovecot. What has dovecot to deal with my USB-Backup-Disk? Dovecot cares about filesystem without any relation to the mail system? Why this? What's bedind? Thanks in advance, Frank Elsner From tss at iki.fi Wed Feb 29 19:31:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 19:31:12 +0200 Subject: [Dovecot] What's behind the message ... In-Reply-To: (auto-added) References: (auto-added) Message-ID: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> On 29.2.2012, at 19.27, Frank Elsner wrote: > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > Hello, > the above message appeared when I restarted dovecot. > > What has dovecot to deal with my USB-Backup-Disk? > Dovecot cares about filesystem without any relation to the mail system? No, but Dovecot doesn't really know what the mail filesystems are. You can add: doveadm mount add '/media/*' ignore Perhaps this should be a default. > Why this? What's bedind? http://markmail.org/message/ad6f4fg3js6ljykg From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:59:06 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:59:06 +0100 Subject: [Dovecot] What's behind the message ... In-Reply-To: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> References: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> Message-ID: (auto-added) On Wed, 29 Feb 2012 19:31:12 +0200 Timo Sirainen wrote: > On 29.2.2012, at 19.27, Frank Elsner wrote: > > > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > > > Hello, > > the above message appeared when I restarted dovecot. > > > > What has dovecot to deal with my USB-Backup-Disk? > > Dovecot cares about filesystem without any relation to the mail system? > > No, but Dovecot doesn't really know what the mail filesystems are. You can add: > > doveadm mount add '/media/*' ignore > > Perhaps this should be a default. > > > Why this? What's bedind? > > http://markmail.org/message/ad6f4fg3js6ljykg Thanks for the pointer! --Frank Elsner From stephan at rename-it.nl Wed Feb 29 23:54:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 29 Feb 2012 22:54:15 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) In-Reply-To: <4F4E20BF.8040707@us.edu.pl> References: <4F4E20BF.8040707@us.edu.pl> Message-ID: <4F4E9E87.7020608@rename-it.nl> Hi Piotr, On 2/29/2012 1:57 PM, Piotr Tarnowski wrote: > I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but > when I want to compile Sieve for this version > (dovecot-2.1-pigeonhole-0.3.0) I get an error: > > "sieve.c", line 578: identifier redeclared: sieve_multiscript_run > > and I see that declaration in sieve.h is different from definition in > sieve.c, so compilation process fails with SunStudio cc compiler. > > I have done a quick hack to correct this, and then compiled successfully. Thanks for the report. Apparently GCC doesn't see this. Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/9da3a8398ea4 Regards, Stephan. From kurt.armistead at eeb5.eu Wed Feb 1 01:16:48 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 00:16:48 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot Message-ID: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Hi guys. Recently we've moved our old (and I mean _really_ old) mailserver to a new hardware. Since I had quite some time on my hands, I decided to switch from courier to dovecot. Transition went pretty smoothly and the users experienced just a few hours' blackout. Only trouble now is that, after re-configuring some IOS handsets and tablets, not all messages are displayed correctly. For example, I had a long existing mailbox (roughly 1600 emails in INBOX and about a dozen other folders) that showed up perfectly on my iphone. Now it's been synced to the new dovecot server via imap and a whole month of messages seems to be missing. I checked on the server, via webmail and with outlook and everything is where it should be. The problem shows up on Apple devices (same thing happened on a couple of ipads). I haven't tried rebuilding the index or wiping out a phone to see if that resolves the problem, but before I do that I'd like to know if anyone experienced the same issue and how they fixed it Thanks! Kurt From tss at iki.fi Wed Feb 1 01:33:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 01:33:05 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 1.16, Armistead, Kurt wrote: > Recently we've moved our old (and I mean _really_ old) mailserver to a new > hardware. Since I had quite some time on my hands, I decided to switch > from courier to dovecot. Transition went pretty smoothly and the users > experienced just a few hours' blackout. > > Only trouble now is that, after re-configuring some IOS handsets and > tablets, not all messages are displayed correctly. Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely related to clients not realizing that they should drop their internal caches. From f.bonnet at esiee.fr Wed Feb 1 07:55:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Wed, 01 Feb 2012 06:55:48 +0100 Subject: [Dovecot] Temporary forbid some users login ? Message-ID: <4F28D3E4.8040408@esiee.fr> hello is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? I need to move their IMAP folders to another place with more disk space but I don't want to stop dovecot IMAP service for the other users as the moving process will be a bit long ( 1 Tb to move ) thanks From arne at drlinux.no Wed Feb 1 08:58:43 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Wed, 01 Feb 2012 07:58:43 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F28E2A3.3090200@drlinux.no> Den 01. feb. 2012 06:55, skrev Frank Bonnet: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > > > This has been covered before, but if you are using SQL for auth-lookups then you can have something like this ins your query; ... AND active='y' Arne -- Arne K. Haaje - Dr Linux http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From kurt.armistead at eeb5.eu Wed Feb 1 09:48:06 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Wed, 1 Feb 2012 08:48:06 +0100 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> Message-ID: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: > On 1.2.2012, at 1.16, Armistead, Kurt wrote: > >> Recently we've moved our old (and I mean _really_ old) mailserver to a >> new >> hardware. Since I had quite some time on my hands, I decided to switch >> from courier to dovecot. Transition went pretty smoothly and the users >> experienced just a few hours' blackout. >> >> Only trouble now is that, after re-configuring some IOS handsets and >> tablets, not all messages are displayed correctly. > > Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new > IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely > related to clients not realizing that they should drop their internal > caches. > I sure did and it reported no issues. Kurt From joseba.torre at ehu.es Wed Feb 1 13:55:44 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Wed, 01 Feb 2012 12:55:44 +0100 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F292840.8080907@ehu.es> El 01/02/12 06:55, Frank Bonnet escribi?: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks Take a look to conf.d/auth-deny.conf.ext You can setup a new passdb (a passwd-file can do it) with deny = yes, and add/remove users to that passwd-file as needed. From weber at zackbummfertig.de Wed Feb 1 14:11:11 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:11:11 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: Message-ID: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Is this Howto http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL actual also for Dovecot 2.x ? Cause i read Dovecot 1.x in the Howto Thank you for any hints / tipps Marko (from cold Hamburg Harbour) From jsiegle at psu.edu Wed Feb 1 14:23:10 2012 From: jsiegle at psu.edu (Jonathan Siegle) Date: Wed, 01 Feb 2012 07:23:10 -0500 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> Message-ID: <4F292EAE.2030601@psu.edu> Marko Weber said the following on 2/1/12 7:11 AM: > > Is this Howto > http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL > actual also for Dovecot 2.x ? > Cause i read Dovecot 1.x in the Howto > > Thank you for any hints / tipps > My gut feeling is version 1.2 because it was last modified on 2010-06-29 11:37:39(see bottom of page) which is in the 1.2 days(http://www.dovecot.org/oldnews.html). -Jonathan -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4316 bytes Desc: S/MIME Cryptographic Signature URL: From janfrode at tanso.net Wed Feb 1 14:29:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 1 Feb 2012 13:29:50 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS Message-ID: <20120201122950.GA23395@dibs.tanso.net> I've been running continous dsync backups of our Maildirs for a few weeks now, with the destination dsync server using mdbox and SIS. The idea was that the destination server would act as a warm copy of all our active users data. The active servers are using Maildir, and has: $ df -h /usr/local/atmail/users/ Filesystem Size Used Avail Use% Mounted on /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users $ df -hi /usr/local/atmail/users/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users very little of this is compressed (zlib plugin enabled during christmas). I'm surprised that the destination server is so large, was expecting zlib and mdbox and SIS would compress it down to much less than what we're seeing (12TB -> 5TB): $ df -h /srv/mailbackup Filesystem Size Used Avail Use% Mounted on /dev/mapper/mailbackupvg-mailbackuplv 5.7T 4.8T 882G 85% /srv/mailbackup Lots and lots of the attachement storage is duplicated into identical files, instead of hard linked. When running "doveadm purge -u $user", we're seeing lots of Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is missing, but there are 205 other copies of this file named /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with identical sha1sum. Also we see corrupted indexes during the purge. This makes me quite uncertain if dsync is a workable backup solution.. or if we can trust mdboxes. Also on the source side, during dsync, we see too many problems. Some samples: Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX Error: command BOX-LIST failed Error: Worker server's mailbox iteration failed Error: read() from worker server failed: EOF Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Error: read() from proxy client failed: EOF Error: Unexpected finish reply: 1 596fec275888dbd89f6d1f5356c22db6 3720 0 \dsync-expunged 0 Error: Unexpected reply from server: 1 12200572a70726fca946da6f9378dc03 3721 0 \dsync-expunged 0 Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda Error: command BOX-LIST failed Error: read() failed: Broken pipe Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x367703c680] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x35) [0x367703c765] -> /usr/lib64/dovecot/libdovecot.so.0 [0x367703bb93] -> /usr/bin/dsync [0x40f48d] -> /usr/bin/dsync [0x40f589] -> /usr/bin/dsync(dsync_worker_msg_save+0x8e) [0x40eb3e] -> /usr/bin/dsync [0x40d71a] -> /usr/bin/dsync [0x40cdbf] -> /usr/bin/dsync [0x40d105] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x48) [0x3677047278] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xd5) [0x36770485c5] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x367704720d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x3677035a83] -> /usr/bin/dsync(main+0x71e) [0x406c4e] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3e3941d994] -> /usr/bin/dsync [0x406369] Do you have any idea for what our problems might be? Should we: avoid SIS ? avoid doing Maildir on one side and mdbox on the other? try other dovecot version for dsync? anything else? -jf ------------- destination server, running dovecot v2.0.14 -------- mail_attachment_dir = /srv/mailbackup/attachments mail_location = mdbox:~/mdbox mail_plugins = zlib mdbox_rotate_size = 5 M namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = static } plugin { zlib_save = gz zlib_save_level = 9 } protocols = service auth-worker { user = $default_internal_user } service auth { unix_listener auth-userdb { mode = 0600 user = mailbackup } } ssl = no userdb { args = home=/srv/mailbackup/%256Hu/%d/%n driver = static } -------------/destination server -------- -jf From weber at zackbummfertig.de Wed Feb 1 14:32:47 2012 From: weber at zackbummfertig.de (Marko Weber) Date: Wed, 01 Feb 2012 13:32:47 +0100 Subject: [Dovecot] Can anyone tell me quick, if the howto on dovecot.org is actual: In-Reply-To: <4F292DA5.2000107@Media-Brokers.com> References: <213fc8ab3290d164caa3a917591c156b@zackbummfertig.de> <4F292DA5.2000107@Media-Brokers.com> Message-ID: Thank you, i ask because in the first Part is written: At the time of writing this document we use these versions of the required software (probably newer versions also usable): CentOS 4.4 (or RHEL 4 update 4 or Fedora 6) Apache 2.0.52 PHP 5.1.6 MySQL 5.0.22 Dovecot 1.0rc15 And i think , config syntax and organisation of config changed alot from 1.x to 2.x , or ? Am 01.02.2012 13:18, schrieb Charles Marcus: > On 2012-02-01 7:11 AM, Marko Weber wrote: >> >> Is this Howto >> http://wiki2.dovecot.org/HowTo/DovecotLDAPostfixAdminMySQL >> actual also for Dovecot 2.x ? > > wiki2. stuff is for dovecot 2.x... > > wiki. stuff is for dovecot 1.x... From tompos at martos.bme.hu Wed Feb 1 14:40:58 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 13:40:58 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ Message-ID: <4F2932DA.4000600@martos.bme.hu> hi! What does patched mean in this repo? Does anybody know? Thank you, tamas From stephan at rename-it.nl Wed Feb 1 14:56:40 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 01 Feb 2012 13:56:40 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F2932DA.4000600@martos.bme.hu> References: <4F2932DA.4000600@martos.bme.hu> Message-ID: <4F293688.1030606@rename-it.nl> Op 2/1/2012 1:40 PM, Papp Tamas schreef: > hi! > > What does patched mean in this repo? > > Does anybody know? This is the v2.1 repository that includes unstable development patches: http://hg.rename-it.nl/dovecot-2.1-patches/ http://hg.rename-it.nl/pigeonhole-0.3-patches/ These are mostly meant for some side-projects I am working on. Previously, some of these patches were included in the normal automatic builds, making debugging difficult since `tainted' dovecot sources were used. The patches also caused a few rather stupid bugs. Therefore, as I announced in a mail a few months back, I started a separate builder for testing these patches. So far, there is no reason for anyone except me to install and try these. Just use the normal dovecot-2.1 repository as documented in the wiki. Regards, Stephan. From tompos at martos.bme.hu Wed Feb 1 15:01:42 2012 From: tompos at martos.bme.hu (Papp Tamas) Date: Wed, 01 Feb 2012 14:01:42 +0100 Subject: [Dovecot] http://xi.rename-it.nl/debian/dists/stable-auto/dovecot-2.1.patched/ In-Reply-To: <4F293688.1030606@rename-it.nl> References: <4F2932DA.4000600@martos.bme.hu> <4F293688.1030606@rename-it.nl> Message-ID: <4F2937B6.2010409@martos.bme.hu> On 02/01/2012 01:56 PM, Stephan Bosch wrote: > Op 2/1/2012 1:40 PM, Papp Tamas schreef: >> hi! >> >> What does patched mean in this repo? >> >> Does anybody know? > > This is the v2.1 repository that includes unstable development patches: > > http://hg.rename-it.nl/dovecot-2.1-patches/ > http://hg.rename-it.nl/pigeonhole-0.3-patches/ > > These are mostly meant for some side-projects I am working on. > Previously, some of these patches were included in the normal > automatic builds, making debugging difficult since `tainted' dovecot > sources were used. The patches also caused a few rather stupid bugs. > Therefore, as I announced in a mail a few months back, I started a > separate builder for testing these patches. So far, there is no reason > for anyone except me to install and try these. Just use the normal > dovecot-2.1 repository as documented in the wiki. OK, thank you. tamas From kowalskm at cs.moravian.edu Wed Feb 1 16:09:11 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 09:09:11 -0500 Subject: [Dovecot] locking problem Message-ID: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log I tried killing the imap daemons and removing the lock file, but it keeps coming back. Any idea what else to do? This is the dovecot file: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-28-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/certs/ssl-cert-snakeoil.pem ssl_key_file: /etc/ssl/private/ssl-cert-snakeoil.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth-client mode: 432 user: postfix group: postfix --myron ================================= Myron Kowalski MoCoSIN Network/Systems Administrator Moravian College myron at moravian.edu (610) 861-1441 From gfinch at ldmltd.ca Wed Feb 1 17:41:04 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Wed, 01 Feb 2012 07:41:04 -0800 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F28D3E4.8040408@esiee.fr> References: <4F28D3E4.8040408@esiee.fr> Message-ID: <4F295D10.5010303@ldmltd.ca> On 2012-01-31 9:55 PM, Frank Bonnet wrote: > hello > > is there a way to forbid SOME ( not all ) users's login with dovecot 2 ? > I need to move their IMAP folders to another place with more > disk space but I don't want to stop dovecot IMAP service for > the other users as the moving process will be a bit long ( 1 Tb to move ) > > > thanks > http://wiki2.dovecot.org/PasswordDatabase/ExtraFields see nologin -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From la at iki.fi Wed Feb 1 17:43:20 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 17:43:20 +0200 Subject: [Dovecot] Uncompressing dboxes Message-ID: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Hello. I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. I guess I could convert to mbox, and then back to mdbox, but for certain reasons I'm not very enthusiastic about doing anything mbox-related with dovecot any more. (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Thanks, Lauri From la at iki.fi Wed Feb 1 19:59:05 2012 From: la at iki.fi (Lauri Alanko) Date: Wed, 01 Feb 2012 19:59:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? Message-ID: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Hello again. When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Thanks again, Lauri From rob0 at gmx.co.uk Wed Feb 1 21:56:53 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Wed, 1 Feb 2012 13:56:53 -0600 Subject: [Dovecot] Temporary forbid some users login ? In-Reply-To: <4F292840.8080907@ehu.es> References: <4F28D3E4.8040408@esiee.fr> <4F292840.8080907@ehu.es> Message-ID: <20120201195653.GS14195@harrier.slackbuilds.org> On Wed, Feb 01, 2012 at 12:55:44PM +0100, Joseba Torre wrote: > El 01/02/12 06:55, Frank Bonnet escribi?: > > > >is there a way to forbid SOME ( not all ) users's login with > >dovecot 2 ? I need to move their IMAP folders to another place > >with more disk space but I don't want to stop dovecot IMAP > >service for the other users as the moving process will be a > >bit long ( 1 Tb to move ) > > Take a look to conf.d/auth-deny.conf.ext > > You can setup a new passdb (a passwd-file can do it) with deny > = yes, and add/remove users to that passwd-file as needed. Heh, funny, three different answers in this thread and AFAICT they are all correct to some extent. I think the "passdb { deny=yes }" is the best answer. I implemented this in SQL using a tri-state "active" column. Standard "active=1" means the MTA accepts mail and the user can login. "active=0" will disable both. The third state, "active=-1" has the MTA continuing to accept mail, but triggers my deny=yes passdb. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Wed Feb 1 22:11:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:11:40 +0200 Subject: [Dovecot] Uncompressing dboxes In-Reply-To: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> References: <20120201174320.111186nfno6wdzmg.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 17.43, Lauri Alanko wrote: > I have an mdbox directory that has been created with the zlib plugin active, using bzip2 compression. I would now like to convert these into uncompressed mdbox format. How do I do this? I have tried unsetting all zlib-related configuration options, but dsync still produces compressed mdbox at the other end, too. Do something like: dsync backup dsync -o plugin/zlib_save= -o mail=mdbox:/tmp/uncompressed > (Incidentally, it's a bit funny that when the zlib plugin is enabled, dsync automatically creates compressed dboxes, and even compressed maildirs (which is pretty useless), but not compressed mboxes, although that's where compression would be most beneficial.) Dovecot supports only read-only compressed mboxes. Also compressed mboxes are always pretty slow, since there's no way to quickly jump around in the file (to go to offset X, you need to uncompress the first X bytes of the mbox file, that's one reason why for mdbox I decided against using full file compression). From tss at iki.fi Wed Feb 1 22:24:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:24:54 +0200 Subject: [Dovecot] locking problem In-Reply-To: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> Message-ID: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> On 1.2.2012, at 16.09, myron wrote: > I have one user who can't read his e-mail. The /var/log/mail.log file has this statement. > > dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log > > I tried killing the imap daemons and removing the lock file, > but it keeps coming back. Any idea what else to do? .. > mail_location: mbox:~/mail:INBOX=/var/mail/%u How large is the /var/mail/ mbox file? From tss at iki.fi Wed Feb 1 22:26:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:26:39 +0200 Subject: [Dovecot] Issues w/ Apple devices after migration to dovecot In-Reply-To: <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> References: <36ef47829bc1e8ec9001b132ac442962.squirrel@webmail.vp44.net> <29aef724160eb5609c9963f705d4b0a1.squirrel@webmail.vp44.net> Message-ID: On 1.2.2012, at 9.48, Armistead, Kurt wrote: > On Wed, February 1, 2012 12:33 am, Timo Sirainen wrote: >> On 1.2.2012, at 1.16, Armistead, Kurt wrote: >> >>> Recently we've moved our old (and I mean _really_ old) mailserver to a >>> new >>> hardware. Since I had quite some time on my hands, I decided to switch >>> from courier to dovecot. Transition went pretty smoothly and the users >>> experienced just a few hours' blackout. >>> >>> Only trouble now is that, after re-configuring some IOS handsets and >>> tablets, not all messages are displayed correctly. >> >> Did you use the courier-dovecot-migrate.pl script? If not, Dovecot set new >> IMAP UIDVALIDITY and UID numbers to mails and the problems are most likely >> related to clients not realizing that they should drop their internal >> caches. > > I sure did and it reported no issues. Then I have no idea. Clients shouldn't have noticed any differences between old and new server. If you find a bug in something from Dovecot's side, let me know. From tss at iki.fi Wed Feb 1 22:29:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:29:51 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <4F27D981.7060304@noa.gr> References: <4F27D981.7060304@noa.gr> Message-ID: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> On 31.1.2012, at 14.07, Nikolaos Milas wrote: > I am running dovecot-2.0.13-1_128.el5 x86_64 RPM on CentOS 5.7. > > I would like to rename userx with email: userx at example.com to ux at example.com with a mailbox of ux (currently: userx) > > Of course the idea is that new mail will continue to be delivered to the same mailbox, although it has been renamed. > > How can I achieve it? Would it be enough (after changing the associated data in the associated LDAP entry) to simply rename the virtual user directory name, e.g. from /home/vmail/userx to /home/vmail/ux ? Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. If a mail is being delivered at the same time, it could tempfail but will be retried again. (A more complex method would be using dsync, but probably not worth the trouble.) From tss at iki.fi Wed Feb 1 22:36:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:36:11 +0200 Subject: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: References: <33BD52FA-1FE0-46D5-A1E8-9A54C406BE64@iki.fi> Message-ID: On 30.1.2012, at 11.24, Alexis Lelion wrote: >> Maybe you could work around it so that LMTP always proxies the mails, to >> localhost as well, but to a different port which doesn't do proxying at all. > Actually this was my first try, but I had proxying loops because > unlike for IMAP, the LMTP server doesn't seem to support 'proxy_maybe' > option yet, does it? Dovecot v2.1 supports it. From tss at iki.fi Wed Feb 1 22:39:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:39:58 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2656E7.8060501@esiee.fr> References: <4F2656E7.8060501@esiee.fr> Message-ID: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> On 30.1.2012, at 10.37, Frank Bonnet wrote: > We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > format to maildir format. I'd do it one user at a time with dsync, but unfortunately: > the machine use real users thru openldap ( pam_ldap + nss_ldap ) nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > another problem is disk space. The users's email data takes about > 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > the raid array of the server. With one-user-at-a-time conversion this wouldn't be an issue. From tss at iki.fi Wed Feb 1 22:41:23 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:41:23 +0200 Subject: [Dovecot] question of different type of Mailbox In-Reply-To: <1327891063.2234.4.camel@localhost.localdomain> References: <1327891063.2234.4.camel@localhost.localdomain> Message-ID: <49D113F9-09BD-41E2-94C6-0647920EA54D@iki.fi> On 30.1.2012, at 4.37, ml wrote: > I just migrate my emails from one type of Maildir to Mailbox > I did as I was having problems reading speed with my webmail. > I did it in order to optimize when do you > > my current config work for me What's the question here? Anyway, mdbox should be faster than either maildir or mbox. From kowalskm at cs.moravian.edu Wed Feb 1 22:43:38 2012 From: kowalskm at cs.moravian.edu (myron) Date: Wed, 1 Feb 2012 15:43:38 -0500 Subject: [Dovecot] locking problem In-Reply-To: <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> Message-ID: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> On Feb 1, 2012, at 3:24 PM, Timo Sirainen wrote: > On 1.2.2012, at 16.09, myron wrote: > >> I have one user who can't read his e-mail. The /var/log/mail.log >> file has this statement. >> >> dovecot: IMAP(): Timeout while waiting for lock for >> transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> >> I tried killing the imap daemons and removing the lock file, >> but it keeps coming back. Any idea what else to do? > .. >> mail_location: mbox:~/mail:INBOX=/var/mail/%u > > How large is the /var/mail/ mbox file? -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ Can I just zero out the 3 files in that directory. 6240 Jan 31 13:26 dovecot.index 165888 Jan 31 14:39 dovecot.index.cache 25100 Jan 31 14:42 dovecot.index.log From tss at iki.fi Wed Feb 1 22:46:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 22:46:03 +0200 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: <4F260EAF.4090408@gedalya.net> References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: On 30.1.2012, at 5.29, Gedalya wrote: > The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > Also, what order does dovecot return the UIDLs in? The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: + pop3: Added support for showing messages in "POP3 order", which can be different from IMAP message order. This can be useful for migrations from other servers. Implemented it for Maildir as 'O' field in dovecot-uidlist. From gedalya at gedalya.net Wed Feb 1 22:50:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 15:50:39 -0500 Subject: [Dovecot] IMAP to Maildir Migration preserving UIDs? In-Reply-To: References: <4F20D718.9010805@gedalya.net> <4F260EAF.4090408@gedalya.net> Message-ID: <4F29A59F.8080501@gedalya.net> On 02/01/2012 03:46 PM, Timo Sirainen wrote: > On 30.1.2012, at 5.29, Gedalya wrote: > >> The old system uses the message filename for UIDL, but we need to migrate via IMAP in order to preserve IMAP info and UIDs (which have nothing to do with the POP3 UIDL in this case). So I've just finished writing a script to insert X-UIDL headers, and pop3_reuse_xuidl is doing the job. > You could also store the UIDL to dovecot-uidlist file directly without keeping it in header. > >> Question: Since the system currently serves in excess of 10 pop3 connections per second, would there be any performance gain from using pop3_save_uidl? Would it be faster or slower to fetch the UIDL list from the uidlist rather than look up the X-UIDL in the index? Just wondering. > Currently with pop3_reuse_xuidl=yes setting the X-UIDL header is cached to dovecot.index.cache file. With pop3_save_uidl=yes I doubt you'd get much of a performance improvement, but if it enables you to later set pop3_reuse_xuidl=no you can delete all of the dovecot.index.cache files for pop3-only users, since they don't really need it. > >> Also, what order does dovecot return the UIDLs in? > The messages are in the same order as in dovecot-uidlist. That of course isn't necessarily the same ordering as in your POP3 server. If you directly add the UIDLs to dovecot-uidlist file in migration script you can also change the UIDL ordering in v2.0.13+: > > + pop3: Added support for showing messages in "POP3 order", which can > be different from IMAP message order. This can be useful for > migrations from other servers. Implemented it for Maildir as 'O' > field in dovecot-uidlist. > > Thank you so much! This clears up a lot of fine details. From gedalya at gedalya.net Wed Feb 1 23:03:17 2012 From: gedalya at gedalya.net (Gedalya) Date: Wed, 01 Feb 2012 16:03:17 -0500 Subject: [Dovecot] Site-wide configurable Sieve Message-ID: <4F29A895.5030007@gedalya.net> Hi, I need some sieve actions to be specified per-user, but exposing actual sieve scripts to users is of the question. For example, I need to be able to either put spam messages into a spam folder, or discard them, based on user preference. Presumably this would be done using Sieve. As I understand, it's not possible to query a database from within a sieve script. If I can't use a single global script, I can imagine automatically generating per-user scripts when settings are saved in a GUI (e.g. webmail) and store them on the dovecot machine. This would work but I would love to hear if someone has a smarter, more robust idea. Thanks, Gedalya From tss at iki.fi Wed Feb 1 23:11:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 1 Feb 2012 23:11:17 +0200 Subject: [Dovecot] locking problem In-Reply-To: <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> Message-ID: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> On 1.2.2012, at 22.43, myron wrote: >>> dovecot: IMAP(): Timeout while waiting for lock for transaction log file /users//mail/.imap/INBOX/dovecot.index.log >> How large is the /var/mail/ mbox file? > > > -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ OK, so not very big. > Can I just zero out the 3 files in that directory. > > 6240 Jan 31 13:26 dovecot.index > 165888 Jan 31 14:39 dovecot.index.cache > 25100 Jan 31 14:42 dovecot.index.log You can delete them, but somehow I doubt that helps.. That error message anyway happens when a session has opened the mbox and is "syncing" it. This normally is a pretty fast operation, but the error message indicates that another session saw the first session doing it for over 3 minutes until it aborted. Do you have hanging imap processes eating 100% CPU? That could explain it if Dovecot doesn't like something in the mbox file. Also there may have been some other reasons for this that are already fixed in newer versions. From alessio at skye.it Thu Feb 2 09:46:55 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 02 Feb 2012 08:46:55 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <4F2A3F6F.6000604@skye.it> Il 01/02/2012 13:29, Jan-Frode Myklebust ha scritto: > I've been running continous dsync backups of our Maildirs for a few > weeks now, with the destination dsync server using mdbox and SIS. The > idea was that the destination server would act as a warm copy of > all our active users data. How many users there are in this installation? > The active servers are using Maildir, and has: > > $ df -h /usr/local/atmail/users/ > Filesystem Size Used Avail Use% Mounted on > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > $ df -hi /usr/local/atmail/users/ > Filesystem Inodes IUsed IFree IUse% Mounted on > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > very little of this is compressed (zlib plugin enabled during christmas). This is the old storage in Maildir format? > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): > > $ df -h /srv/mailbackup > Filesystem Size Used Avail Use% Mounted on > /dev/mapper/mailbackupvg-mailbackuplv > 5.7T 4.8T 882G 85% /srv/mailbackup This is the new storage in mdbox format? What size you would expect? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From nmilas at noa.gr Thu Feb 2 10:17:24 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Thu, 02 Feb 2012 10:17:24 +0200 Subject: [Dovecot] Renaming user account / mailbox In-Reply-To: <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> References: <4F27D981.7060304@noa.gr> <94368F30-4C67-4D6F-8477-AD686F3CDEA2@iki.fi> Message-ID: <4F2A4694.1070904@noa.gr> On 1/2/2012 10:29 ??, Timo Sirainen wrote: > Yes, you can just kill the user's existing connections, rename the directory and update the LDAP entry. If you do all of that quickly in a script, it's unlikely that you'll run into trouble. Thanks Timo, In fact, I could have the luxury to bring down the mail server (Postfix and Dovecot) for a couple of minutes and complete the changes. There is a mail gateway device queuing for delivery when the mail server is down. Nick From f.bonnet at esiee.fr Thu Feb 2 10:53:00 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 09:53:00 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> Message-ID: <4F2A4EEC.8020207@esiee.fr> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > On 30.1.2012, at 10.37, Frank Bonnet wrote: > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >> format to maildir format. > I'd do it one user at a time with dsync, but unfortunately: > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > >> another problem is disk space. The users's email data takes about >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >> the raid array of the server. > With one-user-at-a-time conversion this wouldn't be an issue. > so is there a massive conversion method ? thanks From janfrode at tanso.net Thu Feb 2 10:53:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 09:53:51 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <4F2A3F6F.6000604@skye.it> References: <20120201122950.GA23395@dibs.tanso.net> <4F2A3F6F.6000604@skye.it> Message-ID: <20120202085351.GA18559@dibs.tanso.net> On Thu, Feb 02, 2012 at 08:46:55AM +0100, Alessio Cecchi wrote: > > How many users there are in this installation? Quite a few :-) This is for an ISP. > >The active servers are using Maildir, and has: > > > > $ df -h /usr/local/atmail/users/ > > Filesystem Size Used Avail Use% Mounted on > > /dev/atmailusers 14T 12T 2.2T 85% /usr/local/atmail/users > > $ df -hi /usr/local/atmail/users/ > > Filesystem Inodes IUsed IFree IUse% Mounted on > > /dev/atmailusers 145M 113M 33M 78% /usr/local/atmail/users > > > >very little of this is compressed (zlib plugin enabled during christmas). > > This is the old storage in Maildir format? Correct. > > >I'm surprised that the destination server is so large, was expecting zlib and > >mdbox and SIS would compress it down to much less than what we're seeing > >(12TB -> 5TB): > > > > $ df -h /srv/mailbackup > > Filesystem Size Used Avail Use% Mounted on > > /dev/mapper/mailbackupvg-mailbackuplv > > 5.7T 4.8T 882G 85% /srv/mailbackup > > This is the new storage in mdbox format? Correct. > What size you would expect? With Maildir I see message-files shrink to about 20%* of original size after turning on zlib with zlib_save_level=6. I was expecting better compression with mdbox (and zlib_save_level=9), and I would expect SIS to help even further. mdbox+SIS+zlib_save_level=9 variant taking up 40% the space of a mixed** compressed/non-compressed Maildir storage isn't very impressive to me -- and the mdbox backup isn't even complete (it's only the 25% most active users). Yes, I see there might be holes in my logic, expecting compressed messages to compress further after move to mdbox. But also I have expectation that most of the messages are not already compressed on the Maildir side. Sorry, expectations and guesses, not hard facts. [*] based on a couple of samples, not thourough research [**] Only messages saved after we enabled zlib on December 25. are compressed. -jf From tss at iki.fi Thu Feb 2 11:20:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 11:20:35 +0200 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <4F2A4EEC.8020207@esiee.fr> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> Message-ID: <1328174435.11500.105.camel@innu> On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: > Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : > > On 30.1.2012, at 10.37, Frank Bonnet wrote: > > > >> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox > >> format to maildir format. > > I'd do it one user at a time with dsync, but unfortunately: > > > >> the machine use real users thru openldap ( pam_ldap + nss_ldap ) > > nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. > > > >> another problem is disk space. The users's email data takes about > >> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on > >> the raid array of the server. > > With one-user-at-a-time conversion this wouldn't be an issue. > > > > so is there a massive conversion method ? dsync works for that too: 1. dsync backup all users somewhere 2. wait close to the time you can shut down dovecot 3. dsync backup all users the second time 4. shutdown dovecot and MTA 5. dsync backup once more (hopefully this will be relatively quick) The idea is that you can do incremental dsync backups. You'll of course need to do this in a way that you don't run out of disk space.. From f.bonnet at esiee.fr Thu Feb 2 11:33:45 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 10:33:45 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328174435.11500.105.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> Message-ID: <4F2A5879.3090509@esiee.fr> Le 02/02/2012 10:20 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 09:53 +0100, Frank Bonnet wrote: >> Le 02/01/2012 09:39 PM, Timo Sirainen a ?crit : >>> On 30.1.2012, at 10.37, Frank Bonnet wrote: >>> >>>> We are planning to convert our mailhub ( freebsd 7.4 ) from mbox >>>> format to maildir format. >>> I'd do it one user at a time with dsync, but unfortunately: >>> >>>> the machine use real users thru openldap ( pam_ldap + nss_ldap ) >>> nss_ldap can't store the mail_location override field needed to distinguish mbox user from maildir user. Maybe you could temporarily switch to some other userdb, which would allow extra fields. Also this requires that you deliver mails using dovecot-lda/lmtp. >>> >>>> another problem is disk space. The users's email data takes about >>>> 2 Terabytes of data and I cannot duplicate as I only have 3 Tb on >>>> the raid array of the server. >>> With one-user-at-a-time conversion this wouldn't be an issue. >>> >> so is there a massive conversion method ? > dsync works for that too: > > 1. dsync backup all users somewhere > 2. wait close to the time you can shut down dovecot > 3. dsync backup all users the second time > 4. shutdown dovecot and MTA > 5. dsync backup once more (hopefully this will be relatively quick) > > The idea is that you can do incremental dsync backups. You'll of course > need to do this in a way that you don't run out of disk space.. > Well I have NFS access to a Netapp filer with a LOT of disk space I need some modifications in my dovecot configuration to use NFS I think ? see below the output of doveconf -n thanks mail# doveconf -n # 2.0.16: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.4-RELEASE amd64 auth_verbose_passwords = plain auth_worker_max_count = 100 debug_log_path = /var/log/dovecot/debug.log default_client_limit = 6144 default_process_limit = 1024 default_vsz_limit = 1 G disable_plaintext_auth = no info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/dovecot.log mail_location = mbox:%h/:INDEX=/raid/cache/dovecot/index/%u:INBOX=/raid/var/mail/%u:CONTROL=%h/ mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_lock_timeout = 3 mins mbox_write_locks = fcntl mmap_disable = yes passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at esiee.fr protocols = imap pop3 sendmail_path = /usr/local/sbin/sendmail service auth { client_limit = 7169 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 10 service_count = 1 vsz_limit = 3 G } service imap { process_limit = 2048 vsz_limit = 3 G } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl_ca = /etc/ssl/xxx.pem ssl_cert = References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> Message-ID: <1328177422.11500.108.camel@innu> On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: > > dsync works for that too: > > > > 1. dsync backup all users somewhere > > 2. wait close to the time you can shut down dovecot > > 3. dsync backup all users the second time > > 4. shutdown dovecot and MTA > > 5. dsync backup once more (hopefully this will be relatively quick) > > > > The idea is that you can do incremental dsync backups. You'll of course > > need to do this in a way that you don't run out of disk space.. > > > > Well I have NFS access to a Netapp filer with a LOT of disk space > I need some modifications in my dovecot configuration to use NFS > I think ? No need for any modifications. As long as you don't have multiple Dovecots in different servers accessing the same mails via NFS, there's no trouble. > mmap_disable = yes Looks like you also have this already enabled. This is recommended (not required) with NFS, but if you're not normally using NFS at all you shouldn't enable this. From tss at iki.fi Thu Feb 2 12:23:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 12:23:01 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120201122950.GA23395@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> Message-ID: <1328178181.11500.119.camel@innu> On Wed, 2012-02-01 at 13:29 +0100, Jan-Frode Myklebust wrote: > I'm surprised that the destination server is so large, was expecting zlib and > mdbox and SIS would compress it down to much less than what we're seeing > (12TB -> 5TB): Note that with SIS the attachments aren't compressed. > Lots and lots of the attachement storage is duplicated into identical files, > instead of hard linked. Something's wrong then. > When running "doveadm purge -u $user", we're seeing lots of > > Error: unlink(/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab) failed: No such file or directory Something's wrong. > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > missing, but there are 205 other copies of this file named > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > identical sha1sum. All of them have a link count of 2, with the other link being in hashes/ directory? > Also on the source side, during dsync, we see too many problems. That is most likely related to your troubles. If the dsync runs crash, the result could leave extra files lying around etc.. > Some samples: > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX This is a little bit strange. What is the doveconf -n output of the source server? > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name Is this a namespace prefix? It shouldn't be trying to sync a mailbox named this (there's an extra "." suffix). > Error: read() from proxy client failed: EOF I guess the remote dsync crashes or otherwise aborted. > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda I guess some kind of mismatch related to namespace configuration. > Error: read() failed: Broken pipe > Panic: file dsync-worker-local.c: line 1678 (local_worker_save_msg_continue): assertion failed: (ret == -1) Probably can't handle properly when remote dsync dies. Of course it still shouldn't crash. There seems to be some bugs left when dsyncing to a remote host (instead of locally). It would help if I could reproduce the errors that you're seeing. Can you easily reproduce them with some accounts? If so, if you can give enough details for me to reproduce the problems I can fix them. (Except for the "file not found" issues, since that problems occurred earlier already. I should probably somehow make Dovecot fix those missing files though..) From janfrode at tanso.net Thu Feb 2 13:31:20 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:31:20 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <1328178181.11500.119.camel@innu> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> Message-ID: <20120202113120.GA25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:23:01PM +0200, Timo Sirainen wrote: > > Note that with SIS the attachments aren't compressed. Yes, I know. > > > "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is > > missing, but there are 205 other copies of this file named > > /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with > > identical sha1sum. > > All of them have a link count of 2, with the other link being in hashes/ > directory? No, these has link count=207. I don't know what you mean by link being in hashes directory. # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|head -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0069222e0c080f4f754a0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-00ffb9312a370e4f6b610000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-0442c5163ad3114fb4780000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-04f288390052144f012d0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-053b6c0f185a0d4fc4210000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-06c98213c3b30e4fac3c0000bf34f6ab -rw------- 207 mailbackup mailbackup 149265 Jan 9 23:31 /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-076573234fbd0b4fa8620000bf34f6ab This is just one example, I can provide tons of other examples.. Hmm, I see now that there are 206 files of that first example with the 207 links, and here's another other example with numlinks=7: # ls -l /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-*|wc -l 206 and numlinks=4: # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l 3 is dovecot somehow creating numlinks+1 copies of every file it hardlinks?? Would explain my diskusage :-) > That is most likely related to your troubles. If the dsync runs crash, > the result could leave extra files lying around etc.. If dsync backup is supposed to be a viable backup solution, I think it should fail much better. If it see errors on the target side it should clear the target and do a full sync. Manually cleaning up after it's problems is too much work. > > > Some samples: > > > > Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > This is a little bit strange. What is the doveconf -n output of the > source server? ---------------------------------------------------------------------------- # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.26.1.el5 x86_64 Red Hat Enterprise Linux Server # release 5.5 (Tikanga) auth_cache_size = 100 M auth_verbose = yes auth_verbose_passwords = sha1 disable_plaintext_auth = no login_trusted_networks = 192.168.0.0/16 mail_gid = 3000 mail_home = /srv/mailstore/%256RHu/%d/%n mail_location = maildir:~/:INDEX=/indexes/%1u/%1.1u/%u mail_max_userip_connections = 20 mail_plugins = quota zlib mail_uid = 3000 maildir_stat_dirs = yes maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { quota = dict:UserQuota::file:%h/dovecot-quota sieve = /sieve/%1u/%1.1u/%u/.dovecot.sieve sieve_dir = /sieve/%1u/%1.1u/%u sieve_max_script_size = 1M zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = imap pop3 lmtp sieve service auth-worker { user = $default_internal_user } service auth { client_limit = 4521 unix_listener auth-userdb { group = mode = 0600 user = atmail } } service imap-login { inet_listener imap { address = * port = 143 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service imap-postlogin { executable = script-login /usr/local/sbin/imap-postlogin.sh } service imap { executable = imap imap-postlogin process_limit = 2048 } service lmtp { client_limit = 1 inet_listener lmtp { address = * port = 24 } process_limit = 25 } service managesieve-login { inet_listener sieve { address = * port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { address = * port = 110 } process_min_avail = 4 service_count = 0 vsz_limit = 1 G } service pop3-postlogin { executable = script-login /usr/local/sbin/pop3-postlogin.sh } service pop3 { executable = pop3 pop3-postlogin process_limit = 2048 } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = quota zlib sieve } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota zlib imap_quota } protocol pop3 { mail_plugins = quota zlib pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = UID%u-%v } protocol sieve { managesieve_logout_format = bytes=%i/%o } ---------------------------------------------------------------------------- > > > Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > > Is this a namespace prefix? It shouldn't be trying to sync a mailbox > named this (there's an extra "." suffix). I believe it's a folder named "INBOX.ferie 2006.", with the user using the namespace separator in the folder name.. I believe dovecot allows this, so it should also handle backing it up. > > > Error: Failed to sync mailbox INBOX.INBOX.Gerda: Mailbox doesn't exist: INBOX/Gerda > > I guess some kind of mismatch related to namespace configuration. They both have same namespace config I think, but Maildir vs. mdbox maybe changes something.. > > It would help if I could reproduce the errors that you're seeing. Can > you easily reproduce them with some accounts? If so, if you can give > enough details for me to reproduce the problems I can fix them. (Except > for the "file not found" issues, since that problems occurred earlier > already. I should probably somehow make Dovecot fix those missing files > though..) I'll look trough the logs to see if there are any errors that keep repeating for the same accounts. -jf From janfrode at tanso.net Thu Feb 2 13:51:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 2 Feb 2012 12:51:03 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <20120202115102.GB25117@dibs.tanso.net> On Thu, Feb 02, 2012 at 12:31:20PM +0100, Jan-Frode Myklebust wrote: > > and numlinks=4: > > # ls -al /srv/mailbackup/attachments/c3/1b/c31beb42ef78810f7fb81a7086144034fb0fd794*|wc -l > 3 > > is dovecot somehow creating numlinks+1 copies of every file it > hardlinks?? Would explain my diskusage :-) > Sorry, brainfart.. Yes, these are hardlinks to the same inode.. # ls -i c31beb42ef78810f7fb81a7086144034fb0fd794* ../c31beb42ef78810f7fb81a7086144034fb0fd794* 2422693 c31beb42ef78810f7fb81a7086144034fb0fd794 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-13b405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-1cb405342e24284f61530000bf34f6ab 2422693 ../c31beb42ef78810f7fb81a7086144034fb0fd794-4eb405342e24284f61530000bf34f6ab -jf From tss at iki.fi Thu Feb 2 14:41:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 2 Feb 2012 14:41:11 +0200 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <20120202113120.GA25117@dibs.tanso.net> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> Message-ID: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> On 2.2.2012, at 13.31, Jan-Frode Myklebust wrote: >>> "/srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-057274283bb51f4f917e0000bf34f6ab" is >>> missing, but there are 205 other copies of this file named >>> /srv/mailbackup/attachments/c3/17/c317b32b97688c16859956f11b803e3bba434349-* with >>> identical sha1sum. >> >> All of them have a link count of 2, with the other link being in hashes/ >> directory? > > No, these has link count=207. OK, so they aren't actual copies, they are links to the same file. > I don't know what you mean by link being in hashes directory. If you have e.g. aa/bb/aabbccdd-eeee file, there should be a matching aa/bb/hashes/aabbccdd file. >> That is most likely related to your troubles. If the dsync runs crash, >> the result could leave extra files lying around etc.. > > If dsync backup is supposed to be a viable backup solution, I think it > should fail much better. If it see errors on the target side it should > clear the target and do a full sync. Manually cleaning up after it's > problems is too much work. Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX What about: doveadm mailbox status -u user at domain guid '*' in source server? in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name >> >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox >> named this (there's an extra "." suffix). > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > the namespace separator in the folder name.. I believe dovecot allows > this, so it should also handle backing it up. It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: a select foo. a NO [CANNOT] Invalid mailbox name From kowalskm at cs.moravian.edu Thu Feb 2 15:45:46 2012 From: kowalskm at cs.moravian.edu (myron) Date: Thu, 2 Feb 2012 08:45:46 -0500 Subject: [Dovecot] locking problem In-Reply-To: <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> References: <7F7F7E36-3550-4D5D-B2DB-6654D5FA314F@cs.moravian.edu> <69CDF44C-5B89-47FA-8327-366FDF09A57C@iki.fi> <78959682-0374-4F09-8565-BF514800B664@cs.moravian.edu> <1B081664-8D4B-4F85-92BB-72509B52A717@iki.fi> Message-ID: <5C6469E0-FAC8-45EB-909B-1A58FC3DBE9E@cs.moravian.edu> On Feb 1, 2012, at 4:11 PM, Timo Sirainen wrote: > On 1.2.2012, at 22.43, myron wrote: > >>>> dovecot: IMAP(): Timeout while waiting for lock for >>>> transaction log file /users//mail/.imap/INBOX/ >>>> dovecot.index.log >>> How large is the /var/mail/ mbox file? >> >> >> -rw------- 1 mail 15M 2012-02-01 14:46 /var/mail/ > > OK, so not very big. His isn't, but I have users with 500MB to a GB mailbox. Could they be the cause of the problem? > >> Can I just zero out the 3 files in that directory. >> >> 6240 Jan 31 13:26 dovecot.index >> 165888 Jan 31 14:39 dovecot.index.cache >> 25100 Jan 31 14:42 dovecot.index.log > > You can delete them, but somehow I doubt that helps.. That error > message anyway happens when a session has opened the mbox and is > "syncing" it. This normally is a pretty fast operation, but the > error message indicates that another session saw the first session > doing it for over 3 minutes until it aborted. > > Do you have hanging imap processes eating 100% CPU? Hardly anything: ps -eo pcpu,pid,user,args | sort -k 1 -r | head -20 %CPU PID USER COMMAND 0.0 9 root [events/0] 0.0 993 root /usr/sbin/sshd -D 0.0 984 root /usr/sbin/cupsd -C /etc/cups/cupsd.conf 0.0 9677 imap 0.0 9659 dovecot imap-login 0.0 959 root [ext4-dio-unwrit] 0.0 958 root [ext4-dio-unwrit] 0.0 957 root [jbd2/dm-1-8] 0.0 9442 root [lockd] 0.0 939 root [ext4-dio-unwrit] 0.0 938 root [ext4-dio-unwrit] 0.0 937 root [jbd2/dm-3-8] 0.0 930 root [ext4-dio-unwrit] 0.0 929 root [ext4-dio-unwrit] 0.0 928 root [jbd2/dm-2-8] 0.0 9192 root /usr/sbin/apache2 -k start 0.0 911 root [ext4-dio-unwrit] 0.0 910 root [ext4-dio-unwrit] 0.0 90 root [kconservative/1] > That could explain it if Dovecot doesn't like something in the mbox > file. Also there may have been some other reasons for this that are > already fixed in newer versions. I looked at the upgrade list on the server and dovecot isn't showing any upgrades via atp-get From tss at iki.fi Thu Feb 2 15:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 02 Feb 2012 15:54:24 +0200 Subject: [Dovecot] LDAP auth improvements Message-ID: <1328190864.11500.130.camel@innu> v2.1 supports now multiple LDAP fields in a template e.g.: user_attrs = \ homeDirectory=home, \ uidNumber=uid, \ gidNumber=gid, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} which is the same as: user_attrs = \ =home=%{ldap:homeDirectory}, \ =uid=%{ldap:uidNumber}, \ =gid=%{ldap:gidNumber}, \ =mail=%{ldap:mailboxFormat}:%{ldap:homeDirectory} I was also thinking about adding an alternative simplified syntax for this: user_attrs { home = %{ldap:homeDirectory} uid = %{ldap:uidNumber} gid = %{ldap:gidNumber} mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} } Any ideas for further improvements before I do this change? One thing I'm still wondering about is what I should do when LDAP returns multiple values. Like perhaps: acl_groups = %{ldap:aclGroups:,} Which would mean that all the aclGroups values would be joined together separated by "," characters. And a bit more complex with multiple gidNumbers: gid = %{ldap:gidNumber[0]} mail_access_groups = %{ldap:gidNumber[1:]:,} From f.bonnet at esiee.fr Thu Feb 2 18:22:03 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Thu, 02 Feb 2012 17:22:03 +0100 Subject: [Dovecot] converting from mbox to maildir ? In-Reply-To: <1328177422.11500.108.camel@innu> References: <4F2656E7.8060501@esiee.fr> <389FA9EB-361A-4563-914E-1D9CB860E8BC@iki.fi> <4F2A4EEC.8020207@esiee.fr> <1328174435.11500.105.camel@innu> <4F2A5879.3090509@esiee.fr> <1328177422.11500.108.camel@innu> Message-ID: <4F2AB82B.1000200@esiee.fr> Le 02/02/2012 11:10 AM, Timo Sirainen a ?crit : > On Thu, 2012-02-02 at 10:33 +0100, Frank Bonnet wrote: >>> dsync works for that too: >>> >>> 1. dsync backup all users somewhere >>> 2. wait close to the time you can shut down dovecot >>> 3. dsync backup all users the second time >>> 4. shutdown dovecot and MTA >>> 5. dsync backup once more (hopefully this will be relatively quick) >>> >>> The idea is that you can do incremental dsync backups. You'll of course >>> need to do this in a way that you don't run out of disk space.. >>> >> Well I have NFS access to a Netapp filer with a LOT of disk space >> I need some modifications in my dovecot configuration to use NFS >> I think ? > No need for any modifications. As long as you don't have multiple > Dovecots in different servers accessing the same mails via NFS, there's > no trouble. > >> mmap_disable = yes > Looks like you also have this already enabled. This is recommended (not > required) with NFS, but if you're not normally using NFS at all you > shouldn't enable this. > OK thank you From dovecot at asheesh.org Thu Feb 2 21:59:58 2012 From: dovecot at asheesh.org (Asheesh Laroia) Date: Thu, 02 Feb 2012 14:59:58 -0500 Subject: [Dovecot] dsync deleting too many emails (sdbox) Message-ID: <1328212183-sup-560@rose.makesad.us> I'm using dsync to synchronize emails on a laptop where wifi connectivity sometimes fails in the middle of a sync. I have a shell script that runs dsync, and here is one line of it including the output of dsync: + dsync -f -m realmail mirror /home/paulproteus/projects/ssh-attach/run ssh rose.makesad.us dsync dsync-local(paulproteus): Error: dbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Unexpectedly lost uid=337460 dsync-local(paulproteus): Error: msg guid lookup failed: Internal error occurred. Refer to server log for more information. [2012-02-02 11:02:12] dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Rebuilding index dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 130608.broken dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 66159.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.66159 dsync-local(paulproteus): Warning: sdbox /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails: Ignoring invalid filename 336269.broken dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.130608 dsync-remote(paulproteus): Error: proxy server timed out dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Error: link(/home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269, /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269.broken) failed: File exists dsync-local(paulproteus): Error: Corrupted dbox file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 (around offset=16): EOF reading msg header (got 0/30 bytes) dsync-local(paulproteus): Warning: sdbox: Skipping unfixable file: /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/u.336269 dsync-local(paulproteus): Warning: Transaction log file /home/paulproteus/Maildir/dbox/mailboxes/realmail/dbox-Mails/dovecot.index.log was locked for 1528 seconds That seemed problematic, but not dangerous. Then I ran a fresh sync, and found 3000 (of 60,000) messages had been deleted and expunged from the "realmail" box. I'm guessing this is some bad interaction with sdbox and partial file downloads? I haven't read the code for this, but I would guess the dsync process isn't being atomic about file transfers, so it is leaving half-completed transfers in place, which results in corrupt files when they're next examined. dovecot -n output: # 2.0.15: /etc/dovecot/dovecot.conf # OS: Linux 3.0.0-2-amd64 x86_64 Debian wheezy/sid mail_location = sdbox:~/Maildir/dbox passdb { driver = pam } protocols = " imap" ssl_cert = Hi This is my first post here so i will be as precise as i can I'm trying to setup a quota system using dovecot postfix and procmail dovecot version 1.2.9 postfix version 2.7.0 I have succesfully modified dovecot config to use fs quota system (so far imap and pop3) It reports correct quotas to imap and pop3 clients (both user and group) What im tryin to do now is to modify postfix->procmail->dovecot config in a way that if user is over quota mail delivery is delayed instead of bouncing. (is this possible?) Second thing i would like to achive is that when authenticated users close to quota/group quota for example 10MB to quota limit try sending email theyr mail is rejected and preferably an email is generated telling them to free some space for new mails first. (is such a thing possible?) MS Exchange servers have such a feature and I would like to get such a functionality on my linux mailserver I'm trying to prevent a situation when mails are lost due to quota problems. This setup was initially a typical virtualmin installation with a system on single partition. Thanks in advance for all Your suggestions From alessio at skye.it Fri Feb 3 09:20:50 2012 From: alessio at skye.it (Alessio Cecchi) Date: Fri, 03 Feb 2012 08:20:50 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <4F2B8AD2.5010905@skye.it> Il 02/02/2012 23:58, przemek.orzechowski at makolab.pl ha scritto: > > Hi > > This is my first post here so i will be as precise as i can > > I'm trying to setup a quota system using dovecot postfix and procmail > dovecot version 1.2.9 > postfix version 2.7.0 > > I have succesfully modified dovecot config to use fs quota system (so far > imap and pop3) > It reports correct quotas to imap and pop3 clients (both user and group) > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Yes, you can do it but with Dovecot LDA instead of procmail. See the options "quota_full_tempfail" for Dovecot LDA. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Fri Feb 3 09:36:04 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 08:36:04 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: References: Message-ID: <20120203073604.GA6561@dibs.tanso.net> On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: > > What im tryin to do now is to modify postfix->procmail->dovecot config > in a way that if user is over quota mail delivery is delayed instead of > bouncing. > (is this possible?) Check the quota_full_tempfail setting, http://wiki.dovecot.org/MainConfig > > Second thing i would like to achive is that when authenticated users close > to quota/group quota for example 10MB to quota limit > try sending email theyr mail is rejected and preferably an email is > generated telling them to free some space for new mails first. > (is such a thing possible?) Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 -jf From kirill at shutemov.name Fri Feb 3 13:20:49 2012 From: kirill at shutemov.name (Kirill A. Shutemov) Date: Fri, 3 Feb 2012 13:20:49 +0200 Subject: [Dovecot] Feature request - search and tag protocol - notmuch-alike approach Message-ID: <20120203112049.GA19701@shutemov.name> Hi all, notmuch[1] becomes more and more popular. At least in geek community. But notmuch is local-only (except ssh)... It looks like it's hard (if even possible) to use IMAP as transport for notmuch-alike work flow. I think it would be great to create a new protocol based global search and message tagging. All core notmuch operations (search, show, count, tag) can be mapped to the new protocol (more or less) directly. Search syntax can be reused as is, I think. The tricky part is initial tagging. It requires some kind of transaction to be able to tag new messages without racing with upcoming messages. Request (command) syntax can be similar to unix command. This way we can reuse notmuch syntax even deeper. Like: search --limit 10 --sort=oldest-first tag:to-me tag:unread Replies are often structured. I think it's reasonable to use JSON for all replies. Any ideas? I've started to write a proof-of-concept, but it moves too slow, since my knowledge of dovecot internals is poor and it's hard to find time for the project. I hope somebody else has similar idea in mind and just wait opportunity to discuss the idea and start to write code. ;) [1] http://notmuchmail.org/ -- Kirill A. Shutemov From janfrode at tanso.net Fri Feb 3 14:25:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:25:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> Message-ID: <20120203122552.GA12451@dibs.tanso.net> On Wed, Jan 18, 2012 at 09:03:18PM +0200, Timo Sirainen wrote: > > > > I think the way I originally planned LMTP proxying to work is simply too > > complex to work reliably, perhaps even if the code was bug-free. So > > instead of reading+writing DATA at the same time, this patch changes the > > DATA to be first read into memory or temp file, and then from there read > > and sent to the LMTP backends: > > > > http://hg.dovecot.org/dovecot-2.1/raw-rev/51d87deb5c26 > > > > ----8<----8<----8<-----8<-----8<-----8<----8<-----8<----8<----8<-- > > > > unfortunately I haven't tested that patch, so I have no idea if it > > fixed the issues or not... > > I'm not sure if that patch is useful or not. The important patch to fix it is http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c I now implemented this patch on our directors, and pointed postfix at them. No problem seen so far, but I'm still a bit uncertain about the LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite large delays when fsync'ing (slow IMAP APPEND). Do you think increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start seeing lmtp_proxy_output_timeout problems again ? -jf From janfrode at tanso.net Fri Feb 3 14:42:16 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Fri, 3 Feb 2012 13:42:16 +0100 Subject: [Dovecot] Doubts about dsync, mdbox, SIS In-Reply-To: <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> References: <20120201122950.GA23395@dibs.tanso.net> <1328178181.11500.119.camel@innu> <20120202113120.GA25117@dibs.tanso.net> <7B32D0F8-3102-462E-8F7E-AC5C07121017@iki.fi> Message-ID: <20120203124215.GA13495@dibs.tanso.net> On Thu, Feb 02, 2012 at 02:41:11PM +0200, Timo Sirainen wrote: > > >> That is most likely related to your troubles. If the dsync runs crash, > >> the result could leave extra files lying around etc.. > > > > If dsync backup is supposed to be a viable backup solution, I think it > > should fail much better. If it see errors on the target side it should > > clear the target and do a full sync. Manually cleaning up after it's > > problems is too much work. > > Of course. But if no one gives me enough information to reproduce problems, I can't really fix anything. I don't really have time to spend guessing ways to make it break. I've been using dsync to backup my own mails for over a year, with zero problems. I'm reducing the complexity now, removing SIS and starting the backups from scratch again. I'll start posting the problems I see over the weekend.. > > >>> Error: Mailboxes don't have unique GUIDs: 08b46439069d3d4db0490000e671bf84 is shared by INBOX and INBOX > > What about: > > doveadm mailbox status -u user at domain guid '*' > > in source server? INBOX guid=08b46439069d3d4db0490000e671bf84 INBOX.Sent guid=e8f6e431bf6e014f2d780000e671bf84 INBOX.Trash guid=c858f2234a1d5d4e1547000058d3d19f INBOX.Drafts guid=e9f6e431bf6e014f2d780000e671bf84 INBOX.Spam guid=eaf6e431bf6e014f2d780000e671bf84 INBOX.Sent Messages guid=d837512bed7d674e685c000058d3d19f INBOX.INBOX.Sent Messages guid=ebf6e431bf6e014f2d780000e671bf84 INBOX.Notes guid=c0d2250109645e4eed5c000058d3d19f > in dest server? Does one list show two INBOXes or otherwise duplicate GUIDs? Perhaps this was a bug in v2.0.14.. Scratched dest server before I replied.. sorry. > > >>> Error: Failed to sync mailbox INBOX.ferie 2006.: Invalid mailbox name > >> > >> Is this a namespace prefix? It shouldn't be trying to sync a mailbox > >> named this (there's an extra "." suffix). > > > > I believe it's a folder named "INBOX.ferie 2006.", with the user using > > the namespace separator in the folder name.. I believe dovecot allows > > this, so it should also handle backing it up. > > It has never been possible to create such folder via Dovecot. IMAP protocol itself prevents that. "CREATE foo." will end up creating "foo", not "foo." If you manually mkdir that, it's not possible to access the mailbox in any way via Dovecot. Everything will simply fail as: Oh, sorry.. then this is a problem created by @mail, which poked directly in the filesystem. Guess we'll have to clean these up manually. -jf From a23s4a2008 at yandex.ru Fri Feb 3 15:15:05 2012 From: a23s4a2008 at yandex.ru (User Max) Date: Fri, 03 Feb 2012 17:15:05 +0400 Subject: [Dovecot] openbsd and dovecot Message-ID: <725061328274905@web20.yandex.ru> I'm setting up dovecot in OpenBSD OS. The problem is that I can't set up dovecot to use kerberos authentication via bsdauth which is "similar to pam" I try to test dovecot via telnet: telnet localhost pop3 user test +OK pass password -ERR [IN-USE] Temporary authentication failure and some strings from logs Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted Feb 3 15:02:46 srv-mx-00 dovecot: pop3-login: Disconnected (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Feb 3 15:29:13 srv-mx-00 dovecot: auth: Error: bsdauth(test,127.0.0.1): getpwnam() failed: Operation not permitted What am I doing wrong? Here is the output of dovecot -n auth_krb5_keytab = /etc/kerberosV/krb5.keytab auth_mechanisms = plain login default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 2000 first_valid_uid = 2000 last_valid_gid = 2000 last_valid_uid = 2000 mail_gid = vmail mail_location = maildir:/var/vmail/%u%Ud/Maildir mail_uid = vmail mbox_write_locks = fcntl mmap_disable = yes passdb { driver = bsdauth } service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0666 user = _postfix } } ssl_cert = Hi, I'm new to Dovecot. I'm trying to start Dovecot(2.0.9) but I'm getting an error. I think the error is because I'm doing a tutorial for an old version of Devcot(1.x.x). What I get is the following: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: doveconf: Warning: NOTE: You can get a new clean config file with: doveconf -n > dovecot-new.conf doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=imaps is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:1: protocols=pop3s is no longer necessary, remove it doveconf: Warning: Obsolete setting in /etc/dovecot/dovecot.conf:5: ssl_cert_file has been replaced by ssl_cert = References: <1328190864.11500.130.camel@innu> Message-ID: <4F2BE57D.1090508@in.tum.de> On 02.02.2012 14:54, Timo Sirainen wrote: > user_attrs { > home = %{ldap:homeDirectory} > uid = %{ldap:uidNumber} > gid = %{ldap:gidNumber} > mail = %{ldap:mailboxFormat}:%{ldap:homeDirectory} > } > > Any ideas for further improvements before I do this change? One thing I was recently missing was something like this: If the user is in LDAP group "group1", then use host = imap1 otherwise use host = imap2 However this would require a separate LDAP query since most LDAP servers do not list any group information in the user table. Another nice feature: Use some sort of script to compute the lookup result, e.g. an external shell script: #!/bin/bash if [[ $ldap_uid == fred ]]; then echo "quota_rule = storage=3G" else echo "quota_rule = storage=100M" fi Stuff like this is possible for SQL user databases, however LDAP does not have such flexible queries. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From mark.zealey at webfusion.com Fri Feb 3 16:16:27 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Fri, 03 Feb 2012 16:16:27 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? Message-ID: <4F2BEC3B.2050808@webfusion.com> Hi there, I was doing some testing on sdbox yesterday. Basically I did the following procedure: 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) 2) Create a copy of the index file (no cache file created yet) 3) deliver another message to the mailbox (u.3) 4) copy back index file from stage (2) 5) deliver new mail Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Thanks, Mark From lopes80andre at gmail.com Fri Feb 3 16:42:42 2012 From: lopes80andre at gmail.com (Andre Lopes) Date: Fri, 3 Feb 2012 14:42:42 +0000 Subject: [Dovecot] What does this /var/log/maillog message means? Message-ID: Hi, I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: [code] protocols = imap pop3 lmtp log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:/home/vmail/%d/%n/Maildir ssl_cert = /etc/pki/dovecot/certs/dovecot.pem ssl_key = /etc/pki/dovecot/private/dovecot.pem namespace { type = private separator = . prefix = INBOX. inbox = yes } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster at mailserver.com } service auth { user = root } passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } [/code] I can start the service correctly: [code] [root at mailserver dovecot]# service dovecot start Starting Dovecot Imap: [ OK ] [/code] But when I go to "/var/log/maillog" I get this: [code] Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 (by pid=16757 uid=0 code=kill) Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled) [/code] What does this mean? I'm doing something that is not correct? Best Regards, From list at airstreamcomm.net Fri Feb 3 17:22:50 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Fri, 03 Feb 2012 09:22:50 -0600 Subject: [Dovecot] Vacation via database Message-ID: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> We are moving our inbound mail to use dovecot LMTP in the near future and we are looking for suggestions on how to implement a mysql based vacation system. If anyone has experience with this, good or bad please let us know. From Ralf.Hildebrandt at charite.de Fri Feb 3 17:27:59 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 16:27:59 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> Message-ID: <20120203152758.GI29889@charite.de> * list at airstreamcomm.net : > We are moving our inbound mail to use dovecot LMTP in the near future and > we are looking for suggestions on how to implement a mysql based vacation > system. If anyone has experience with this, good or bad please let us > know. Why not use sieve? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 18:41:23 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 11:41:23 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <20120203152758.GI29889@charite.de> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> Message-ID: <4F2C0E33.5070708@gedalya.net> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: > * list at airstreamcomm.net: >> We are moving our inbound mail to use dovecot LMTP in the near future and >> we are looking for suggestions on how to implement a mysql based vacation >> system. If anyone has experience with this, good or bad please let us >> know. > Why not use sieve? > I am facing a similar issue. It looks like sieve can't fetch data from a database, and dovecot can't fetch sieve scripts from a database. It would be useful to have, perhaps, only a global sieve script be able to query a DB, and it would be very useful to allow dovecot to fetch sieve scripts (actual scripts, not a file path) from a database. Any way to do this? From robert at schetterer.org Fri Feb 3 20:32:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:32:12 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C282C.4040403@schetterer.org> Am 03.02.2012 17:41, schrieb Gedalya: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near future >>> and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from a > database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able to > query a DB, and it would be very useful to allow dovecot to fetch sieve > scripts (actual scripts, not a file path) from a database. Any way to do > this? > sieve with database is on roadmap,as it was discussed before ,as far i remember info http://wiki.dovecot.org/ManageSieve http://pigeonhole.dovecot.org/ anyway it seems there is a lot to do left http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO so sql support might have to wait but snip ---- * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## * Implement extlists extension as a plugin * Enotify extension: detect use of variable values extracted from the message that are used in the method argument. RFC reports this as a security issue. * Make the sieve storage a base class with (possibly) various implementations, just like mail-storage. This aims to provide support for alternate types of script storage like LDAP or SQL database. --- looks its on the way -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Fri Feb 3 20:34:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 03 Feb 2012 19:34:18 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C282C.4040403@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> Message-ID: <4F2C28AA.90904@schetterer.org> Am 03.02.2012 19:32, schrieb Robert Schetterer: > Am 03.02.2012 17:41, schrieb Gedalya: >> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>> * list at airstreamcomm.net: >>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>> and >>>> we are looking for suggestions on how to implement a mysql based >>>> vacation >>>> system. If anyone has experience with this, good or bad please let us >>>> know. >>> Why not use sieve? >>> >> I am facing a similar issue. It looks like sieve can't fetch data from a >> database, and dovecot can't fetch sieve scripts from a database. It >> would be useful to have, perhaps, only a global sieve script be able to >> query a DB, and it would be very useful to allow dovecot to fetch sieve >> scripts (actual scripts, not a file path) from a database. Any way to do >> this? >> > > sieve with database is on roadmap,as it was discussed before ,as far i > remember > > info > > http://wiki.dovecot.org/ManageSieve > http://pigeonhole.dovecot.org/ > > anyway it seems there is a lot to do left > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO > so sql support might have to wait > > but snip > ---- > * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## > > * Implement extlists extension as a plugin > * Enotify extension: detect use of variable values extracted from the > message > that are used in the method argument. RFC reports this as a security > issue. > * Make the sieve storage a base class with (possibly) various > implementations, > just like mail-storage. This aims to provide support for alternate types > of script storage like LDAP or SQL database. > --- > > > looks its on the way by the way you can use some sql vacation script method like implemented in postfixadmin , and has support for horde webmail,squirrelmail,roundcube -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From Ralf.Hildebrandt at charite.de Fri Feb 3 20:35:06 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Fri, 3 Feb 2012 19:35:06 +0100 Subject: [Dovecot] Managesieve Build failure Message-ID: <20120203183506.GU29889@charite.de> I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, got this error today: ... make[3]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' make[2]: Leaving directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve' Making install in managesieve-login make[2]: Entering directory /usr/src/dovecot-2.0/dovecot-2.0-pigeonhole/src/managesieve-login' /bin/bash ../../libtool --tag=CC --mode=link gcc -I/usr/src/dovecot-2.0/dovecot-2.0/src/config -DPKG_LIBEXECDIR=\""/usr/local/libexec/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -module -avoid-version -g -Wl,--as-needed -o libmanagesieve_login_settings.la -rpath /usr/local/lib/dovecot/settings libmanagesieve_login_settings_la-managesieve-login-settings.lo libmanagesieve_login_settings_la-managesieve-login-settings-plugin.lo -lrt gcc -DHAVE_CONFIG_H -I. -I../.. -I/usr/src/dovecot-2.0/dovecot-2.0 -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dict -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-dns -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-mail -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-imap -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-fs -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-charset -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-storage/index/raw -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-master -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-settings -I/usr/src/dovecot-2.0/dovecot-2.0/src/lib-auth -I/usr/src/dovecot-2.0/dovecot-2.0/src/login-common -I../.. -I../../src/lib-managesieve -g -Wl,--as-needed -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wstrict-aliasing=2 -MT managesieve-proxy.o -MD -MP -MF .deps/managesieve-proxy.Tpo -c -o managesieve-proxy.o managesieve-proxy.c managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? make[2]: *** [managesieve-proxy.o] Error 1 -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From gedalya at gedalya.net Fri Feb 3 20:49:21 2012 From: gedalya at gedalya.net (Gedalya) Date: Fri, 03 Feb 2012 13:49:21 -0500 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C28AA.90904@schetterer.org> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> <4F2C282C.4040403@schetterer.org> <4F2C28AA.90904@schetterer.org> Message-ID: <4F2C2C31.4040202@gedalya.net> On 02/03/2012 01:34 PM, Robert Schetterer wrote: > Am 03.02.2012 19:32, schrieb Robert Schetterer: >> Am 03.02.2012 17:41, schrieb Gedalya: >>> On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >>>> * list at airstreamcomm.net: >>>>> We are moving our inbound mail to use dovecot LMTP in the near future >>>>> and >>>>> we are looking for suggestions on how to implement a mysql based >>>>> vacation >>>>> system. If anyone has experience with this, good or bad please let us >>>>> know. >>>> Why not use sieve? >>>> >>> I am facing a similar issue. It looks like sieve can't fetch data from a >>> database, and dovecot can't fetch sieve scripts from a database. It >>> would be useful to have, perhaps, only a global sieve script be able to >>> query a DB, and it would be very useful to allow dovecot to fetch sieve >>> scripts (actual scripts, not a file path) from a database. Any way to do >>> this? >>> >> sieve with database is on roadmap,as it was discussed before ,as far i >> remember >> >> info >> >> http://wiki.dovecot.org/ManageSieve >> http://pigeonhole.dovecot.org/ >> >> anyway it seems there is a lot to do left >> http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-file/tip/TODO >> so sql support might have to wait >> >> but snip >> ---- >> * ## MAKE A FOURTH MAIN RELEASE (0.4.x) ## >> >> * Implement extlists extension as a plugin >> * Enotify extension: detect use of variable values extracted from the >> message >> that are used in the method argument. RFC reports this as a security >> issue. >> * Make the sieve storage a base class with (possibly) various >> implementations, >> just like mail-storage. This aims to provide support for alternate types >> of script storage like LDAP or SQL database. >> --- >> >> >> looks its on the way > by the way you can use some sql vacation script method > like implemented in postfixadmin , and has support for horde > webmail,squirrelmail,roundcube > Yeap, for now, in this case, it looks like I'll be hand-coding something into exim and adapt an existing roundcube plugin to my situation. In general, it's looking like I'll be using one, global, static sieve script which will follow instructions in the form of X-Something: Action headers, which I'll be conditionally inserting per user preference on the MX end (exim lmtp transport). Since this is clumsy, I'll try to do as much as possible within the MX, and leave to sieve only things that must happen there (e.g. not necessarily vacation). From stephan at rename-it.nl Sat Feb 4 00:34:14 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:34:14 +0100 Subject: [Dovecot] Vacation via database In-Reply-To: <4F2C0E33.5070708@gedalya.net> References: <851013466e1aa68e818df0fad642c04d@mail.airstreamcomm.net> <20120203152758.GI29889@charite.de> <4F2C0E33.5070708@gedalya.net> Message-ID: <4F2C60E6.7020700@rename-it.nl> On 2/3/2012 5:41 PM, Gedalya wrote: > On 02/03/2012 10:27 AM, Ralf Hildebrandt wrote: >> * list at airstreamcomm.net: >>> We are moving our inbound mail to use dovecot LMTP in the near >>> future and >>> we are looking for suggestions on how to implement a mysql based >>> vacation >>> system. If anyone has experience with this, good or bad please let us >>> know. >> Why not use sieve? >> > I am facing a similar issue. It looks like sieve can't fetch data from > a database, and dovecot can't fetch sieve scripts from a database. It > would be useful to have, perhaps, only a global sieve script be able > to query a DB, and it would be very useful to allow dovecot to fetch > sieve scripts (actual scripts, not a file path) from a database. Any > way to do this? > Retrieving whole scripts from a DB is nice, but not strictly necessary for your application. Indeed, only fetching data from a DB is sufficient. Unfortunately, there is no standard Sieve feature that allows fetching string data from external sources. However, Pigeonhole has a few custom features: * Assuming you're using SQL, there is an experimental plugin for retrieval of external string data from a Dovecot dict lookup (there is no LDAP dict afaik): http://wiki2.dovecot.org/Pigeonhole/Sieve/Plugins/Extdata You can give that a look. Should also compile against Dovecot v2.1 and Pigeonhole v0.3. * If you were using Dovecot v2.1 and Pigeonhole v0.3 (from hg) you could also tinker something with the new extprograms plugin (using the execute command): http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/ For more info read: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/README http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt And, as you suggested, Pigeonhole v0.3 for Dovecot v2.1 can also restrict the use of these plugins to global scripts. Regards, Stephan. From stephan at rename-it.nl Sat Feb 4 00:53:49 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 03 Feb 2012 23:53:49 +0100 Subject: [Dovecot] Managesieve Build failure In-Reply-To: <20120203183506.GU29889@charite.de> References: <20120203183506.GU29889@charite.de> Message-ID: <4F2C657D.50703@rename-it.nl> On 2/3/2012 7:35 PM, Ralf Hildebrandt wrote: > I pulled the recent dovecot-2.0-pigeonhole and dovecot 2.0.x trees and rebuild, > got this error today: > > ... > > managesieve-proxy.c: In function ?managesieve_proxy_parse_line?: > managesieve-proxy.c:334:19: error: ?const struct login_settings? has no member named ?verbose_auth? > make[2]: *** [managesieve-proxy.o] Error 1 I didn't realize that this change was also performed for Dovecot v2.0 and for some reason I didn't get a failure message from the automatic builder. Fixed now: http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 Regards, Stephan. From Przemek.Orzechowski at makolab.net Fri Feb 3 16:38:05 2012 From: Przemek.Orzechowski at makolab.net (Przemek Orzechowski) Date: Fri, 3 Feb 2012 14:38:05 +0000 Subject: [Dovecot] Question about quota configuration In-Reply-To: <20120203073604.GA6561@dibs.tanso.net> References: <20120203073604.GA6561@dibs.tanso.net> Message-ID: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Thanks for quick reply >-----Original Message----- >From: Jan-Frode Myklebust >Reply-to: Jan-Frode Myklebust >To: przemek.orzechowski at makolab.pl >Cc: dovecot at dovecot.org >Subject: Re: [Dovecot] Question about quota configuration >Date: Fri, 3 Feb 2012 08:36:04 +0100 >Mailer: Mutt/1.5.20 (2009-12-10) > >On Thu, Feb 02, 2012 at 11:58:12PM +0100, przemek.orzechowski at makolab.pl wrote: >> >> What im tryin to do now is to modify postfix->procmail->dovecot config >> in a way that if user is over quota mail delivery is delayed instead of >> bouncing. >> (is this possible?) > >Check the quota_full_tempfail setting, > > > http://wiki.dovecot.org/MainConfig > For some reason when i configure dovecot to act as the final LDA in procmail and set quota_full_tempfail = yes overquota mails get lost somewhere As Virtualmin is relying for some of its functons on procmail droping it compleetly is impossible or at least very difficult. >> >> Second thing i would like to achive is that when authenticated users close >> to quota/group quota for example 10MB to quota limit >> try sending email theyr mail is rejected and preferably an email is >> generated telling them to free some space for new mails first. >> (is such a thing possible?) > >Check "Quota warnings" at http://wiki.dovecot.org/Quota/1.1 > I agree this is the solution for sending a warning email, but is there a method to actually prohibit users close to quota to send emails? Ie defining dovecot as authentication agent ? > > -jf to make things easier for everyone im posting snippets from my config of dovecot and procmail :0wi VIRTULAMIN=|/etc/webmin/virtual-server/lookup-domain.pl $LOGNAME EXITCODE=0 :0 * ?/usr/bin/test "$VIRTUALMIN" != "" { INCLUDERC=/etc/webmin/virtual-server/procmail/$VIRTUALMIN } DEFAULT=$HOME/Maildir/ ORGMAIL=$HOME/Maildir/ DELIVER="/usr/lib/dovecot/deliver" DROPPRIVS=yes dovecot config snippets regarding LDA protocol lda { mail_plugins = quota quota_full_tempfail = yes } for some reason this results in mails lost without any trace when users are over quota (i would like to requeue those emails for later delivery) Most probably its not directly dovecot related but as most people here are more or less runing mail servers so maybe someone could help me further. From CMarcus at Media-Brokers.com Sat Feb 4 15:09:13 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sat, 04 Feb 2012 08:09:13 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> Message-ID: <4F2D2DF9.1060608@Media-Brokers.com> On 2012-02-03 9:38 AM, Przemek Orzechowski wrote: > is there a method to actually prohibit users close to quota to send > emails? Ie defining dovecot as authentication agent? I wrote about a feature request on this list for something that would probably easily provide the ability for dovecot to do something like this a while back... I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. This would be a huge time saver for people who send a lot of messages with large attachments. Currently, the client has to 'send' (upload) the message to the server *twice* - once for 'sending' it, and again to 'save the sent copy'. Timo - is there a place for users to submit formal Feature Requests? And, is this idea of an LSA) something that you would even remotely consider at any point in time in the future? Maybe your new company could add a 'bounty' system, so users could submit Feature Requests and they (and others) could pledge money for the feature to be implemented? -- Best regards, Crles From gedalya at gedalya.net Sat Feb 4 15:09:39 2012 From: gedalya at gedalya.net (Gedalya) Date: Sat, 04 Feb 2012 08:09:39 -0500 Subject: [Dovecot] What does this /var/log/maillog message means? In-Reply-To: References: Message-ID: <4F2D2E13.3080904@gedalya.net> On 02/03/2012 09:42 AM, Andre Lopes wrote: > Hi, > > I'm, trying to configure Devcot(2.0.9). Here is my dovecot.conf: > > [code] > protocols = imap pop3 lmtp > > log_timestamp = "%Y-%m-%d %H:%M:%S " > mail_location = maildir:/home/vmail/%d/%n/Maildir > > ssl_cert = /etc/pki/dovecot/certs/dovecot.pem > ssl_key = /etc/pki/dovecot/private/dovecot.pem > > namespace { > type = private > separator = . > prefix = INBOX. > inbox = yes > } > > protocol lda { > log_path = /home/vmail/dovecot-deliver.log > auth_socket_path = /var/run/dovecot/auth-master > postmaster_address = postmaster at mailserver.com > } > > service auth { > user = root > } > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > userdb { > driver = static > args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes > } > [/code] > > I can start the service correctly: > > [code] > [root at mailserver dovecot]# service dovecot start > Starting Dovecot Imap: [ OK ] > [/code] > > But when I go to "/var/log/maillog" I get this: > > [code] > Feb 3 14:37:59 www dovecot: master: Warning: Killed with signal 15 > (by pid=16757 uid=0 code=kill) > Feb 3 14:38:03 www dovecot: master: Dovecot v2.0.9 starting up (core > dumps disabled) > [/code] > > What does this mean? I'm doing something that is not correct? > > Best Regards, There log entries just mean you have restarted dovecot. They are normal and don't indicate a problem. From amuehlem at gmail.com Sun Feb 5 17:00:34 2012 From: amuehlem at gmail.com (Andreas Muehlemann) Date: Sun, 05 Feb 2012 16:00:34 +0100 Subject: [Dovecot] Sieve notify messages Message-ID: <4F2E9992.9070701@gmail.com> Hi All I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. I'm trying to use the notifiy mechanism from sieve to send notifications when a mail arrives in the mailbox. The message is checked to be a 8bit message, otherwise it is replaced by the default message "Notification of new message." How can I create a 8bit message body within the sieve script that is accepted by the notification process? e.g. notify :message "Subject: ${subject}" "mailto:email at domain.com"; Do I have to put the message body into the :message string? Or how can I define a custom message body that is accepted by notify? Best regards Andreas From stephan at rename-it.nl Sun Feb 5 17:46:41 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 16:46:41 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2E9992.9070701@gmail.com> References: <4F2E9992.9070701@gmail.com> Message-ID: <4F2EA461.9000909@rename-it.nl> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: > Hi All > > I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. > > I'm trying to use the notifiy mechanism from sieve to send > notifications when a mail arrives in the mailbox. The message is > checked to be a 8bit message, otherwise it is replaced by the default > message "Notification of new message." How can I create a 8bit message > body within the sieve script that is accepted by the notification > process? > > e.g. > notify > :message "Subject: ${subject}" > "mailto:email at domain.com"; > > Do I have to put the message body into the :message string? Or how can > I define a custom message body that is accepted by notify? If I understand you correctly, the following should be the solution for your issue: This sends the following message: X-Sieve: Pigeonhole Sieve 0.3.0 Message-ID: Date: Sun, 05 Feb 2012 16:36:57 +0100 Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen From: Postmaster To: user at example.com Auto-Submitted: auto-notified; owner-email="234234234234 at ext2.domain.tld" Precedence: bulk X-Priority: 3 (Normal) Importance: Normal MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem Durchschnitt schnell. Regards, Stephan From rog7993 at web.de Sun Feb 5 18:53:50 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 17:53:50 +0100 Subject: [Dovecot] user login on behalf of another user Message-ID: <4F2EB41E.6070100@web.de> Hello, we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Example: User "user1" and "user2" shall get access to the mailbox "info". We define the accounts "info~user1" and "info~user2" with the same home directory like "info". Until now, we use a passwd-file backend. With this setup we can simply copy the password hash from "user1" to "info~user1" and from "user2" to "info~user2". But we intend to change the passdb backend from a flat file to PAM for authentication against Active Directory. This seems to be simple with pam_krb5. But then we can't simply copy password hashes anymore. Is their another possibility for configuring this? Surely the preferable alternative would be the use of ACLs to give acccess to other users mailboxes. But we started this setup with Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. And now we have too much accounts and clients, which are configured this way and can't change this for the short term. The passdb/userdb file from the above example looks like this: info:!:501:501:Info:/home/mail01/info:: info~user1:PASSWORD_USER1:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info info~user2:PASSWORD_USER2:501:501:Info:/home/mail01/info:: \ userdb_mail=maildir:~/Maildir: \ INDEX=/srv/dovecot/index/info: \ CONTROL=/srv/dovecot/control/info user1:PASSWORD_USER1:501:501:Info:/home/mail01/user1:: user2:PASSWORD_USER2:501:501:Info:/home/mail01/user2:: Ingo Rogalsky From sven at svenhartge.de Sun Feb 5 19:14:51 2012 From: sven at svenhartge.de (Sven Hartge) Date: Sun, 5 Feb 2012 18:14:51 +0100 Subject: [Dovecot] user login on behalf of another user References: <4F2EB41E.6070100@web.de> Message-ID: rog7993 at web.de wrote: > User "user1" and "user2" shall get access to the mailbox "info". We > define the accounts "info~user1" and "info~user2" with the same home > directory like "info". Isn't this the kind of scenario shared folders were made for? Gr??e, Sven. -- Sigmentation fault. Core dumped. From stephan at rename-it.nl Sun Feb 5 19:30:20 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 05 Feb 2012 18:30:20 +0100 Subject: [Dovecot] Sieve notify messages In-Reply-To: <4F2EA6E2.20700@gmail.com> References: <4F2E9992.9070701@gmail.com> <4F2EA461.9000909@rename-it.nl> <4F2EA6E2.20700@gmail.com> Message-ID: <4F2EBCAC.50403@rename-it.nl> On 2/5/2012 4:57 PM, Andreas Muehlemann wrote: > Hi Stephan > > Thank you very much, the encodeurl and ?body= are the trick! > For posterity, please reply to the mailing list as well, especially when it is a success message ;) Regards, Stephan. > > On 02/05/2012 04:46 PM, Stephan Bosch wrote: >> On 2/5/2012 4:00 PM, Andreas Muehlemann wrote: >>> Hi All >>> >>> I'm using dovecot 2.0.16 with the pigeonhole plugin 0.25. >>> >>> I'm trying to use the notifiy mechanism from sieve to send >>> notifications when a mail arrives in the mailbox. The message is >>> checked to be a 8bit message, otherwise it is replaced by the >>> default message "Notification of new message." How can I create a >>> 8bit message body within the sieve script that is accepted by the >>> notification process? >>> >>> e.g. >>> notify >>> :message "Subject: ${subject}" >>> "mailto:email at domain.com"; >>> >>> Do I have to put the message body into the :message string? Or how >>> can I define a custom message body that is accepted by notify? >> >> If I understand you correctly, the following should be the solution >> for your issue: >> >> >> >> This sends the following message: >> >> >> X-Sieve: Pigeonhole Sieve 0.3.0 >> Message-ID: >> Date: Sun, 05 Feb 2012 16:36:57 +0100 >> Subject: Herr =?utf-8?q?M=C3=BCller_m=C3=A4ht?= seinen Rasen >> From: Postmaster >> To: user at example.com >> Auto-Submitted: auto-notified; >> owner-email="234234234234 at ext2.domain.tld" >> Precedence: bulk >> X-Priority: 3 (Normal) >> Importance: Normal >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=UTF-8 >> Content-Transfer-Encoding: 8bit >> >> In ?sterreich m?ht Herr M?ller seinen Rasen ?brigens auch ?ber dem >> Durchschnitt schnell. >> >> >> Regards, >> >> Stephan > From me at junc.org Sun Feb 5 22:01:32 2012 From: me at junc.org (Benny Pedersen) Date: Sun, 05 Feb 2012 21:01:32 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: Den 2012-02-05 17:53, rog7993 at web.de skrev: > Surely the preferable alternative would be the use of ACLs to give > acccess to other users mailboxes. But we started this setup with > Dovecot 1.0 or 1.1. And with these versions, ACLs weren't available. > And now we have too much accounts and clients, which are configured > this way and can't change this for the short term. secureity wise i would also do this, had bots trying whole day here to get week passwords found to atleast find one login that works, if acl is used there is only one password found and the other user do not need to change his password, but only the acl or ask the other user to change his password, its still possible that both users is same user, imho it have no point if its pam users or not From rog7993 at web.de Sun Feb 5 23:49:54 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Sun, 05 Feb 2012 22:49:54 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: References: <4F2EB41E.6070100@web.de> Message-ID: <4F2EF982.6080309@web.de> Am 05.02.2012 18:14, schrieb Sven Hartge: > Isn't this the kind of scenario shared folders were made for? Yes, of course. Although I didn't called it shared folders in my original post, I had this in mind, when I wrote that ACLs would be the better solution. But for historical reasons we are already in the situation having these kind of accounts. And I don't want to reconfigure about 100 clients now. This would need more time than I want spend now. Ingo From janfrode at tanso.net Mon Feb 6 15:59:17 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 14:59:17 +0100 Subject: [Dovecot] doveadm purge on shared storage Message-ID: <20120206135917.GA3399@dibs.tanso.net> We've finally (!) started to put some users on mdbox instead of maildir, and now I'm wondering about the purge step. As we're running GPFS for the mailboxes (and dovecot director in front of every dovecot service), is it important to run the "doveadm purge -u $user" on the same host as $user is logged into to avoid index corruption ? If so, will we need to run the "doveadm purge" trough the dovecot director as well? -jf From tss at iki.fi Mon Feb 6 17:05:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 17:05:51 +0200 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <4F2EB41E.6070100@web.de> References: <4F2EB41E.6070100@web.de> Message-ID: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> On 5.2.2012, at 18.53, rog7993 at web.de wrote: > we are searching for a possibility to configure a user login on behalf of another user with a PAM backend. This reminds to the behavior of a master user. But a master user can access the mailboxes of all users. We need this more restricted. Master user doesn't necessarily have access to all users' mailboxes. In the passdb lookup you can decide if this master user is allowed to be this destination user. For example if you used passdb checkpassword, you could look at USER and MASTER_USER environment variables to figure out if this combination should be allowed or not. The checkpassword script can also do the actual authentication via PAM (I'd think there's a way to call it somehow). From cannewilson at googlemail.com Mon Feb 6 19:33:01 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Mon, 6 Feb 2012 17:33:01 +0000 Subject: [Dovecot] Configuration problem? Message-ID: I have a new Scientific Linux 6.1 mail server (dovecot -n below) and am seeing the following in the logs, with no idea what is happening: --------------------- Dovecot Begin ------------------------ Dovecot was killed, and not restarted afterwards. **Unmatched Entries** dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24296/89310: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=24526/76772: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=26254/75778: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=27948/134120: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=29304/116087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=34802/914087: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=4457/11719: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=5322/184434: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=61388/106618: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=6164/21347: 1 Time(s) dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) dovecot: imap: Server shutting down. bytes=14215/97371: 1 Time(s) dovecot: imap: Server shutting down. bytes=1459/9382: 1 Time(s) dovecot: imap: Server shutting down. bytes=1688/26161: 1 Time(s) dovecot: imap: Server shutting down. bytes=1705/5038: 1 Time(s) dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) ---------------------- Dovecot End ------------------------- dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) mail_location = maildir:~/Maildir/ mbox_write_locks = fcntl passdb { driver = pam } protocols = imap ssl_cert = References: Message-ID: <20120206174932.GA5904@PC211.ikt.de> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > am seeing the following in the logs, with no idea what is happening: > > --------------------- Dovecot Begin ------------------------ > > > Dovecot was killed, and not restarted afterwards. You shut dovecot down and does not restart it. > **Unmatched Entries** > dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) [..] > dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) The user closed the connection. > dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) [..] > dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) The server closed the connection due to a shutdown command. > dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) Dovecot did what it just said: it started. Your logwatch is to old to knew about the messages dovecot emits to syslog. HTH Dennis [..] From tss at iki.fi Mon Feb 6 21:58:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:58:02 +0200 Subject: [Dovecot] Multiple instances Message-ID: v2.1 now saves the base_dir and instance_name to /var/lib/dovecot/instances when starting up. If you have for example two instances with names "backend" and "proxy", you can then use: doveadm -i proxy proxy list instead of something like: doveadm -c /etc/dovecot-proxy/dovecot.conf proxy list The same -i parameter works for all dovecot programs. I also added a couple of new doveadm commands: # doveadm instance list path name last used running /var/run/dovecot proxy 2012-02-06 21:41:39 yes # doveadm instance remove proxy It would be possible to add commands to start/shutdown some/all instances using doveadm, but is it all that useful? I'd guess people would have their own init.d scripts anyway doing that. Anything else that could be useful related to this? From tss at iki.fi Mon Feb 6 21:59:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 21:59:40 +0200 Subject: [Dovecot] doveadm purge on shared storage In-Reply-To: <20120206135917.GA3399@dibs.tanso.net> References: <20120206135917.GA3399@dibs.tanso.net> Message-ID: On 6.2.2012, at 15.59, Jan-Frode Myklebust wrote: > We've finally (!) started to put some users on mdbox instead of maildir, > and now I'm wondering about the purge step. As we're running GPFS for the > mailboxes (and dovecot director in front of every dovecot service), is > it important to run the "doveadm purge -u $user" on the same host as > $user is logged into to avoid index corruption ? > > If so, will we need to run the "doveadm purge" trough the dovecot director as > well? If you run doveadm purge on the director itself (or so that it uses director's config), then doveadm automatically goes through director to the correct server to run the purge. You need to have configured http://wiki2.dovecot.org/Director#Doveadm_server From tss at iki.fi Mon Feb 6 22:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:10:11 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F2D2DF9.1060608@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> Message-ID: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> On 4.2.2012, at 15.09, Charles Marcus wrote: > I would love to see dovecot get a 'Local Submission Agent', similar to the LDA. This LSA could then most likely be leveraged (maybe via a custom plugin, or maybe even natively) to do what you are requesting. > > But the reason I requested it is I would like have the dovecot LSA be able to automatically file a copy of all Sent messages to the users designated special 'Sent' folder, similar to how GMail does this. > > This way, you could completely disable the 'Save a copy of Sent Messages to X folder' option in all mail clients like you can wit GMail accounts. Can't you already do this by configuring MTA to bcc mail to user+Sent at domain where there's a global Sieve script moving the +Sent mails to Sent folder (possibly checking that it's from localhost). > Timo - is there a place for users to submit formal Feature Requests? I wouldn't mind if they are simply sent to this list. Although would be a good idea to make it clear in the subject, I almost skipped this message. Most of the things that I want to implement end up in the TODO file in Dovecot sources, and the things that don't end up there are something I'd rather forget, at least until enough people start wanting them. From me at junc.org Mon Feb 6 22:24:56 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:24:56 +0100 Subject: [Dovecot] dovecot-antispam 1.3 and dovecot 1.2.17 fails with spool2dir Message-ID: <555d62880f4f8cab3bfb1f278f900d22@junc.org> or have i missed something in compile or configure in dovecot.conf ? running on gentoo so i can compile if need another option, i just like to have it working so each user can train bayes global db no matter what user agent users using From CMarcus at Media-Brokers.com Mon Feb 6 22:25:19 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 06 Feb 2012 15:25:19 -0500 Subject: [Dovecot] Question about quota configuration In-Reply-To: <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> Message-ID: <4F30372F.3080409@Media-Brokers.com> On 2012-02-06 3:10 PM, Timo Sirainen wrote: > On 4.2.2012, at 15.09, Charles Marcus wrote: >> This way, you could completely disable the 'Save a copy of Sent >> Messages to X folder' option in all mail clients like you can wit >> GMail accounts. > Can't you already do this by configuring MTA to bcc mail to > user+Sent at domain where there's a global Sieve script moving the > +Sent mails to Sent folder (possibly checking that it's from > localhost). You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... >> Timo - is there a place for users to submit formal Feature >> Requests? > I wouldn't mind if they are simply sent to this list. Although would > be a good idea to make it clear in the subject, I almost skipped > this message. Most of the things that I want to implement end up in > the TODO file in Dovecot sources, and the things that don't end up > there are something I'd rather forget, at least until enough people > start wanting them. Heh... understood, and no worries if my idea for a dovecot LSA doesn't make it to the TODO list... ;) -- Best regards, Charles From tss at iki.fi Mon Feb 6 22:27:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:27:05 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <725061328274905@web20.yandex.ru> References: <725061328274905@web20.yandex.ru> Message-ID: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> On 3.2.2012, at 15.15, User Max wrote: > Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted service auth { user = root } Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. From tss at iki.fi Mon Feb 6 22:29:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:29:03 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120203122552.GA12451@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> Message-ID: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > I now implemented this patch on our directors, and pointed postfix at them. > No problem seen so far, but I'm still a bit uncertain about the > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > seeing lmtp_proxy_output_timeout problems again ? Your fsyncs can run over 60 seconds? I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. From tss at iki.fi Mon Feb 6 22:33:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:33:18 +0200 Subject: [Dovecot] Question about quota configuration In-Reply-To: <4F30372F.3080409@Media-Brokers.com> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> Message-ID: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> On 6.2.2012, at 22.25, Charles Marcus wrote: > On 2012-02-06 3:10 PM, Timo Sirainen wrote: >> On 4.2.2012, at 15.09, Charles Marcus wrote: > >>> This way, you could completely disable the 'Save a copy of Sent >>> Messages to X folder' option in all mail clients like you can wit >>> GMail accounts. > >> Can't you already do this by configuring MTA to bcc mail to >> user+Sent at domain where there's a global Sieve script moving the >> +Sent mails to Sent folder (possibly checking that it's from >> localhost). > > You can, and this is the 'recommended' way by the postfix list, but they also explained how some of the envelope/headers are not properly maintained when doing it this way... Dovecot LSA would pretty much have to implement full SMTP server functionality to be able to do this (although maybe it could get away with not doing queuing). I'm not really interested in writing an SMTP server. It would be much less trouble to change Postfix to work in a way that you want. From dinh at flic.net Mon Feb 6 22:37:05 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 12:37:05 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? Message-ID: Hi, I am running dovecot version 1.2.9 on a Ubuntu 10.04 server. To obtain the message's header info from this dovecot imap server, I run the following imap command on a client side's imap session: . fetch 88:90 (envelope) which returns (in this case) ------------------------------------------------------------- * 88 FETCH (ENVELOPE ("Tue, 28 Oct 2008 15:45:35 -0500" "Linear Actuators Bring Flexibility to Packaging" (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) (("Design World" NIL "subscription" "designworldonline.com")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<490779CF.00000E7D at ak119.mta.exacttarget.com>")) * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL NIL "dinh" "flic.net")) NIL NIL NIL "<20081029154358.0C359AE169595F1B at insightmedia.info>")) * 90 FETCH (ENVELOPE ("Thu, 30 Oct 2008 04:24:31 +0800 (SGT)" "6 new products Now Online!" (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Global Sources Product Alert" NIL "paservices" "globalsources.com")) (("Dinh Le" NIL "dinh" "flic.net")) NIL NIL NIL "<352725277.1225311871753.JavaMail.web at fuchsia3.globalsources.com>")) . OK Fetch completed. -------------------------------------------------------------- Notice that message 89's Subject line Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available has quotes and this dovecot's ENVELOPE response failed to get parsed by a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). I have a hunch that this is dovecot's fault but I could be wrong. How should dovecot (or any imap server) deal with Subject lines with quotes inside ENVELOPE? ENVELOPE seems to encapsulate most of the important information and its format can be parsed cleanly and quickly. I prefer to fetch ENVELOP, at this point, rather than the other alternatives. Thanks for your help, Dinh From me at junc.org Mon Feb 6 22:39:33 2012 From: me at junc.org (Benny Pedersen) Date: Mon, 06 Feb 2012 21:39:33 +0100 Subject: [Dovecot] Question about quota configuration In-Reply-To: <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> References: <20120203073604.GA6561@dibs.tanso.net> <1328279885.14857.30.camel@PO-HP-Compaq-6530b-GW688AV> <4F2D2DF9.1060608@Media-Brokers.com> <673F5D30-5925-4B0F-8CF1-3CF66F4D3754@iki.fi> <4F30372F.3080409@Media-Brokers.com> <70943D01-DD85-4D0E-9FB6-FB5ECCE52569@iki.fi> Message-ID: <4e7f87f0748f1b70878567e9f5c07fda@junc.org> > Dovecot LSA would pretty much have to implement full SMTP server > functionality to be able to do this (although maybe it could get away > with not doing queuing). I'm not really interested in writing an SMTP > server. It would be much less trouble to change Postfix to work in a > way that you want. dovecot have lmtp, postfix have lmtp, so does quotas work in lmtp deliver ? else one need to use postfix sql to dovecot dict to get used quotas :/ From slusarz at curecanti.org Mon Feb 6 22:45:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 13:45:46 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: Message-ID: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Quoting Dinh Le : > * 89 FETCH (ENVELOPE ("29 Oct 2008 15:44:03 -0500" {65} > 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) > (("Insight Media" NIL "administrator" "insightmedia.info")) ((NIL > NIL "dinh" "flic.net")) NIL NIL NIL > "<20081029154358.0C359AE169595F1B at insightmedia.info>")) [snip] > Notice that message 89's Subject line > > Subject: 2nd Edition of Brennesholtz/Stupp "Projection Displays" Available > > has quotes and this dovecot's ENVELOPE response failed to get parsed by > a popular Perl module (Mail::IMAPClient::BodyStructure::Envelope). > I have a hunch that this is dovecot's fault but I could be wrong. > How should dovecot (or any imap server) deal with Subject lines with > quotes inside ENVELOPE? The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. Dovecot is 100% correct in its return. michael From tss at iki.fi Mon Feb 6 22:47:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:47:41 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F2BEC3B.2050808@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> Message-ID: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> On 3.2.2012, at 16.16, Mark Zealey wrote: > I was doing some testing on sdbox yesterday. Basically I did the following procedure: > > 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) > 2) Create a copy of the index file (no cache file created yet) > 3) deliver another message to the mailbox (u.3) > 4) copy back index file from stage (2) > 5) deliver new mail > > Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > Is it possible to try an open/access call on the mail file before overwriting it with the new message in case we have an issue where an older version of the index file is present (eg due to nfs latencies) ? I notice when you are expunging files you very carefully open them and read the header contents to make sure the guid is the same as in the index - any reason that this is not done when delivering? This is with lmtp on dovecot 2.0.16. Hm. Yes, I guess there should be a check to avoid overwriting files. > I also noticed that index corruption in sdbox does not get automatically repaired. I know this is because the flags are stored in the index files so you'd get some loss of flags, but in many situations for us this auto-repair with flag loss would be better than having the mailbox locked out until we manually do a force-resync on it. I'm not entirely sure what you mean by this. Does the above patch help with this problem also? > (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) Well, ideally there shouldn't be so much corruption that this matters.. From tss at iki.fi Mon Feb 6 22:55:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 22:55:43 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6.2.2012, at 22.45, Michael M Slusarz wrote: > The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. > > Dovecot is 100% correct in its return. Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) From janfrode at tanso.net Mon Feb 6 23:01:03 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 6 Feb 2012 22:01:03 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> Message-ID: <20120206210103.GA11888@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:29:03PM +0200, Timo Sirainen wrote: > On 3.2.2012, at 14.25, Jan-Frode Myklebust wrote: > > > I now implemented this patch on our directors, and pointed postfix at them. > > No problem seen so far, but I'm still a bit uncertain about the > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS. I know we're experienceing quite > > large delays when fsync'ing (slow IMAP APPEND). Do you think increasing > > LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is a sensible workaround if we start > > seeing lmtp_proxy_output_timeout problems again ? > > Your fsyncs can run over 60 seconds? Hopefully not.. maybe just me being confused by the error message about "lmtp_proxy_output_timeout". After adding http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't seen any problems so it looks like this problem is solved. But it doesn't seem unthinkable that ext3 users might see more than 60s for fsyncs... "Some stalls on the order of minutes have been reported" ref: https://lwn.net/Articles/328363/ > I think even if you increase Dovecot's timeout you'll soon reach your MTA's LMTP timeout. > My MTA's default is 10 minutes.. http://www.postfix.org/postconf.5.html#lmtp_data_done_timeout -jf From tss at iki.fi Mon Feb 6 23:16:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 6 Feb 2012 23:16:24 +0200 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> On 6.2.2012, at 22.55, Timo Sirainen wrote: > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) BTW. If anyone wants to try, the attached patch makes Dovecot return (almost) all possible fields as literals. You'll need to delete dovecot.index.cache files to get the full effect. -------------- next part -------------- A non-text attachment was scrubbed... Name: break-imap-clients.diff Type: application/octet-stream Size: 425 bytes Desc: not available URL: From dinh at flic.net Mon Feb 6 23:18:33 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:18:33 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Thanks, I will try to come up From i.thierack at iwt-promotion.com Mon Feb 6 23:26:45 2012 From: i.thierack at iwt-promotion.com (Ingo Thierack) Date: Mon, 06 Feb 2012 22:26:45 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? Message-ID: <4F304595.9030007@iwt-promotion.com> Hello, try to use the new 2.1rc and don't get any data in the searchindex. Tried first lucene, and switched than back so solr. If I do an search in an mailfolder, i get in the dovecot-log 2012-02-06 22:17:11 | dovecot: | indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot Log from solr. Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} hits=0 status=0 QTime=1 Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository yesterday) With 2.0 i see, if i start an search, solr had to work on the mail, now happens nothing. Upgrade the schema.xml, delete the old index. Regard Ingo Thierack From dinh at flic.net Mon Feb 6 23:34:31 2012 From: dinh at flic.net (Dinh Le) Date: Mon, 6 Feb 2012 13:34:31 -0800 (PST) Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On Mon, 6 Feb 2012, Timo Sirainen wrote: > On 6.2.2012, at 22.45, Michael M Slusarz wrote: > >> The subject, because it contains quotes, can't be expressed in a quoted string so it is instead sent in a literal string. > > Small correction: Quotes can be escaped in a quoted-string, but Dovecot just doesn't want to do it. Many other servers behave this way as well. > >> In a literal string, the data is exactly what appears in the original message. No post-processing mechanism should be stripping quotes or anything from this data - it is already in its canonical state. >> >> Dovecot is 100% correct in its return. > > Yes. Looks like Perl's IMAP client hasn't been built properly to expect literals in places where they are allowed. My guess is that this is only one of the 100 other places where a literal is allowed by RFC but not by Perl.. > > Maybe I should add some evil setting to Dovecot where it would use literals everywhere it can, or perhaps randomly send atoms/strings/literals. :) > Oops, my happy Alpine ^X finger accidently sent the previous reply. I meant to say thanks and will try to come up with a fix to this literal string problem for the Perl IMAPClient module. Don't want Timo's evil config setting to wreak havocs :-). From dovecot-20110531 at billmail.scconsult.com Mon Feb 6 23:47:07 2012 From: dovecot-20110531 at billmail.scconsult.com (Bill Cole) Date: Mon, 06 Feb 2012 16:47:07 -0500 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> Message-ID: On 6 Feb 2012, at 16:34, Dinh Le wrote: > On Mon, 6 Feb 2012, Timo Sirainen wrote: > >> On 6.2.2012, at 22.45, Michael M Slusarz wrote: >> >>> The subject, because it contains quotes, can't be expressed in a >>> quoted string so it is instead sent in a literal string. >> >> Small correction: Quotes can be escaped in a quoted-string, but >> Dovecot just doesn't want to do it. Many other servers behave this >> way as well. >> >>> In a literal string, the data is exactly what appears in the >>> original message. No post-processing mechanism should be stripping >>> quotes or anything from this data - it is already in its canonical >>> state. >>> >>> Dovecot is 100% correct in its return. >> >> Yes. Looks like Perl's IMAP client hasn't been built properly to >> expect literals in places where they are allowed. My guess is that >> this is only one of the 100 other places where a literal is allowed >> by RFC but not by Perl.. >> >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) >> > > Oops, my happy Alpine ^X finger accidently sent the previous reply. I > meant to say thanks and will try to come up with a fix to this literal > string problem for the Perl IMAPClient module. > > Don't want Timo's evil config setting to wreak havocs :-). The best solution for you may be to to update the version of Mail::IMAPClient you are using. There were fixes for its handling of literals claimed in the CHANGES file of v3.26 (2011-01-31). I don't know if that will fix this particular problem, but if you are running an older version then it would be worth trying the current version. From slusarz at curecanti.org Tue Feb 7 00:14:27 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 06 Feb 2012 15:14:27 -0700 Subject: [Dovecot] ENVELOPE mishandling of Subject lines with quote(s)? In-Reply-To: <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> References: <20120206134546.Horde.N1HYcoF5lbhPMDv6F1pz6BA@bigworm.curecanti.org> <53FADF89-D2C2-499C-93F5-4E8A98A73498@iki.fi> Message-ID: <20120206151427.Horde.QTJpPoF5lbhPMFDDPHZkelA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 6.2.2012, at 22.55, Timo Sirainen wrote: > >> Maybe I should add some evil setting to Dovecot where it would use >> literals everywhere it can, or perhaps randomly send >> atoms/strings/literals. :) > > BTW. If anyone wants to try, the attached patch makes Dovecot return > (almost) all possible fields as literals. You'll need to delete > dovecot.index.cache files to get the full effect. Very useful. Turns out the only thing I wasn't handling correctly was the MIME types/subtypes in BODYSTRUCTURE. This is a great tool for debugging client code. Not sure the difficulty to add as a config (or even a hidden config). Being able to easily turn on/off would increase my usage of the tool. michael From ybhu at hk1.ibm.com Tue Feb 7 08:26:43 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 7 Feb 2012 14:26:43 +0800 Subject: [Dovecot] Synchronization error in NFS Message-ID: Hi there, Not sure whether it's a bug or env setting issue. I am running some concurrent testings under NFS. The dovecot (version 2.0.13) are deployed in 2 servers (same setup), whose maildir are on the file server via NFS. Here are the output of dovecot -n: [root at MyMachine src]$dovecot -n # 2.0.13: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.3.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) nfs auth_anonymous_username = andy auth_debug = yes auth_debug_passwords = yes auth_mechanisms = anonymous plain auth_verbose = yes auth_verbose_passwords = plain disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = andy mail_location = maildir:/tmp/NFS mail_nfs_index = yes mail_nfs_storage = yes mail_uid = andy mmap_disable = yes passdb { driver = pam } ssl = no userdb { driver = passwd } Here are what I am doing: One session running loop of COPY commands (while(1) COPY...) connects to one dovecot server; The other session running loop of SELECT commands (while(1) SELECT...) connects to the other dovecot server. Both are accessing the same mail box (/tmp/NFS); After some while (not accurate duration, maybe 2 seconds, or 1 min), I found the number of EXISTS returned from SELECT command is not correct(less than the real number). Then I stop the both sessions. In the /tmp/log: Feb 07 03:44:59 imap(andy): Error: Corrupted transaction log file /tmp/NFS/dovecot.index.log seq 2: Unexpected garbage at EOF (sync_offset=2204) Feb 07 03:44:59 imap(andy): Error: Index /tmp/NFS/dovecot.index: Lost log for seq=2 offset=2204 Feb 07 03:44:59 imap(andy): Warning: fscking index file /tmp/NFS/dovecot.index Then I tried to dump the index, sometime it's failed because of the index corruption. If not, it says: [root at MyMachine src]$doveadm dump /tmp/NFS > /tmp/dump ; vi /tmp/dump doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: Log synchronization error at seq=2,offset=744 for /tmp/NFS/dovecot.index: Broken extension introduction: Headersize too large (2273345664) doveadm(root): Warning: fscking index file /tmp/NFS/dovecot.index doveadm(root): Error: fcntl(write-lock) locking failed for file /tmp/NFS/dovecot.index.log: Bad file descriptor doveadm(root): Error: mail_index_wait_lock_fd() failed with file /tmp/NFS/dovecot.index.log: Bad file descriptor And sometime from the dump output, in the RECORD part: -- RECORDS: 5 RECORD: seq=1, uid=1, flags=0x00 RECORD: seq=2, uid=2, flags=0x00 RECORD: seq=3, uid=3, flags=0x00 RECORD: seq=4, uid=4, flags=0x00 RECORD: seq=5, uid=6, flags=0x00 The uid 5 is missed. but in uidlist file, it's there.. Here are all what I found. If you need additional information, pls let me know. The clock on the 3 machines are synchronized. You can also reproduce it if the 2 sessions are APPEND and SELECT. If both sessions are running towards the same dovecot server, even the maildir are on the NFS, it works very well without any error. From cannewilson at googlemail.com Tue Feb 7 10:08:24 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 7 Feb 2012 08:08:24 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120206174932.GA5904@PC211.ikt.de> References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: On 6 February 2012 17:49, Dennis Guhl wrote: > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and >> am seeing the following in the logs, with no idea what is happening: >> >> ?--------------------- Dovecot Begin ------------------------ >> >> >> ?Dovecot was killed, and not restarted afterwards. > > You shut dovecot down and does not restart it. > It appears to be doing things without my intervention. Despite the reports that it kept shutting down, Dovecot continued to serve messages throughout the day. >> ?**Unmatched Entries** >> ? ? dovecot: imap(anne): Connection closed bytes=205614/894243: 1 Time(s) > > [..] > >> ? ? dovecot: imap(anne): Disconnected: Logged out bytes=7914/89868: 1 Time(s) > > The user closed the connection. > >> ? ? dovecot: imap: Server shutting down. bytes=1309821/4473013: 1 Time(s) > > [..] > >> ? ? dovecot: imap: Server shutting down. bytes=3146/79269: 1 Time(s) > > The server closed the connection due to a shutdown command. > >> ? ? dovecot: master: Dovecot v2.0.9 starting up (core dumps disabled): 1 Time(s) > > Dovecot did what it just said: it started. > > Your logwatch is to old to knew about the messages dovecot emits to > syslog. > > HTH > Dennis > > [..] You mean the version of logwatch is too old? I'm beginning to wonder whether running an Enterprise version is such a good idea after all. Anne From janfrode at tanso.net Tue Feb 7 10:25:10 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 7 Feb 2012 09:25:10 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120206210103.GA11888@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> Message-ID: <20120207082510.GA24432@dibs.tanso.net> On Mon, Feb 06, 2012 at 10:01:03PM +0100, Jan-Frode Myklebust wrote: > > Your fsyncs can run over 60 seconds? > > Hopefully not.. maybe just me being confused by the error message about > "lmtp_proxy_output_timeout". After adding > http://hg.dovecot.org/dovecot-2.0/rev/71084b799a6c on friday, we haven't > seen any problems so it looks like this problem is solved. Crap, saw 6 "message might be sent more than once" messages from postfix yesterday, all at the time of this crash on the director postfix/lmtp was talking with: Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d680] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193d6d6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2ab6f193cb93] -> dovecot/lmtp [0x406d75] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xcd) [0x2ab6f194859d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0x68) [0x2ab6f1949558] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x2d) [0x2ab6f194820d] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x2ab6f1936a83] -> dovecot/lmtp(main+0x144) [0x403fa4] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x35f8a1d994] -> dovecot/lmtp [0x403da9] Feb 6 16:13:10 loadbalancer2 dovecot: master: Error: service(lmtp): child 6601 killed with signal 6 (core dumps disabled) Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any other ideas for what might be causing it ? -jf From Juergen.Obermann at hrz.uni-giessen.de Tue Feb 7 11:13:24 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?UTF-8?Q?J=C3=BCrgen_Obermann?=) Date: Tue, 07 Feb 2012 10:13:24 +0100 Subject: [Dovecot] dsync error Mailbox has children, delete them first In-Reply-To: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> References: <20111213114701.185112l8l37ih0yo@webmail.hrz.uni-giessen.de> Message-ID: <16d56e5474247859b32601122db0b39f@imapproxy.hrz> Am 13.12.2011 11:47, schrieb J?rgen Obermann: > Hi, > > I use dsync to backup mailboxes from mbox format to mdbox on a remote > system. The first run for a user with dsync is OK, but during the > second there are lots of the following errors: > > dsync-remote(user): Error: Can't delete mailbox directory Example: > Mailbox has children, delete them first > > I see no way how I could influence the order dsync deletes mailboxes. > This happens with dovecot version 2.0.16 > > Thank you, > Juergen Obermann Hallo, after upgrade to dovecot 2.0.17 this problem went away. Greetings, J?rgen Obermann Hochschulrechenzentrum der Justus-Liebig-Universit?t Gie?en Heinrich-Buff-Ring 44 Tel. 0641-9913054 From nmilas at noa.gr Tue Feb 7 12:07:56 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 12:07:56 +0200 Subject: [Dovecot] Multiple userdb possible? Message-ID: <4F30F7FC.9050607@noa.gr> Hello, I am running v2.0.13. In my dovecot.conf I have: userdb { args = /etc/dovecot/dovecot-usrdb-ldap.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap.conf driver = ldap } Is it legitimate to include multiple ldap userdb's, like: userdb { args = /etc/dovecot/dovecot-usrdb-ldap1.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap1.conf driver = ldap } userdb { args = /etc/dovecot/dovecot-usrdb-ldap2.conf driver = ldap } passdb { args = /etc/dovecot/dovecot-passdb-ldap2.conf driver = ldap } If it is legitimate (in case configuration is different, please correct me), in which sequence userdb's are evaluated? Thanks, Nick From dg at dguhl.org Tue Feb 7 13:57:07 2012 From: dg at dguhl.org (Dennis Guhl) Date: Tue, 7 Feb 2012 12:57:07 +0100 Subject: [Dovecot] Configuration problem? In-Reply-To: References: <20120206174932.GA5904@PC211.ikt.de> Message-ID: <20120207115707.GA30334@laptop-dg.leere.eu> On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: > On 6 February 2012 17:49, Dennis Guhl wrote: > > On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: > >> I have a new Scientific Linux 6.1 mail server (dovecot -n below) and > >> am seeing the following in the logs, with no idea what is happening: > >> > >> ?--------------------- Dovecot Begin ------------------------ > >> > >> > >> ?Dovecot was killed, and not restarted afterwards. > > > > You shut dovecot down and does not restart it. > > > It appears to be doing things without my intervention. Despite the > reports that it kept shutting down, Dovecot continued to serve > messages throughout the day. The messages logwatch shows appeared at some time within the analysed period and are not necessarily in a time sorted order. Btw do not rely on any summary of log files but look into the log yourself. [..] > You mean the version of logwatch is too old? I'm beginning to wonder Yes, the current version is 7.4.0 from march 2011 (http://www.logwatch.org). > whether running an Enterprise version is such a good idea after all. I don't know Scientific Linux but I use Debian stable on all my server and I'm very happy with it. Nonetheless do I manually upgrade some packages wich added needed features or are maintained by upstream. It is crucial to know and understand the philosophy behind a distribution and to decide if this works for you and if you can live with the caveats resulting. Dennis From mark.zealey at webfusion.com Tue Feb 7 14:08:09 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Tue, 07 Feb 2012 14:08:09 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> Message-ID: <4F311429.2000708@webfusion.com> 06-02-2012 22:47, Timo Sirainen yazm??: > On 3.2.2012, at 16.16, Mark Zealey wrote: > >> I was doing some testing on sdbox yesterday. Basically I did the following procedure: >> >> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >> 2) Create a copy of the index file (no cache file created yet) >> 3) deliver another message to the mailbox (u.3) >> 4) copy back index file from stage (2) >> 5) deliver new mail >> >> Then the message delivered in stage 3 ie u.3 gets replaced with the message delivered in (5) also called u.3. > http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. >> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) > Well, ideally there shouldn't be so much corruption that this matters.. That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Mark From cannewilson at googlemail.com Tue Feb 7 17:20:18 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Tue, 07 Feb 2012 15:20:18 +0000 Subject: [Dovecot] Configuration problem? In-Reply-To: <20120207115707.GA30334@laptop-dg.leere.eu> References: <20120206174932.GA5904@PC211.ikt.de> <20120207115707.GA30334@laptop-dg.leere.eu> Message-ID: <4F314132.7030604@googlemail.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 07/02/12 11:57, Dennis Guhl wrote: > On Tue, Feb 07, 2012 at 08:08:24AM +0000, Anne Wilson wrote: >> On 6 February 2012 17:49, Dennis Guhl wrote: >>> On Mon, Feb 06, 2012 at 05:33:01PM +0000, Anne Wilson wrote: >>>> I have a new Scientific Linux 6.1 mail server (dovecot -n >>>> below) and am seeing the following in the logs, with no idea >>>> what is happening: >>>> >>>> --------------------- Dovecot Begin ------------------------ >>>> >>>> >>>> Dovecot was killed, and not restarted afterwards. >>> >>> You shut dovecot down and does not restart it. >>> >> It appears to be doing things without my intervention. Despite >> the reports that it kept shutting down, Dovecot continued to >> serve messages throughout the day. > > The messages logwatch shows appeared at some time within the > analysed period and are not necessarily in a time sorted order. > Btw do not rely on any summary of log files but look into the log > yourself. > > [..] > Actually, this morning there aren't the same messages, so perhaps I was restarting services while trying to get it right - in fact it seems very likely that that was so. Today there are a few like dovecot: imap(anne): Disconnected: Logged out bytes=11892/21219: 1 Time(s) I presume that refers to clients logging out of the imap connection? In which case, I can forget about that. I normally read the summary each morning and refer directly to the logs if I see something that looks unusual. Occasionally, as in this case, there are entries that I don't understand and I ask those who do :-) >> You mean the version of logwatch is too old? I'm beginning to >> wonder > > Yes, the current version is 7.4.0 from march 2011 > (http://www.logwatch.org). > >> whether running an Enterprise version is such a good idea after >> all. > > I don't know Scientific Linux but I use Debian stable on all my > server and I'm very happy with it. Nonetheless do I manually > upgrade some packages wich added needed features or are maintained > by upstream. > > It is crucial to know and understand the philosophy behind a > distribution and to decide if this works for you and if you can > live with the caveats resulting. > I've run CentOS for maybe 4 years, and it's similar to SL, both being RHEL clones, but maintained by different communities. On a server (even though this is a very "mild" server, being only file and print serving) the older packages are rarely a problem. I appreciate the time and trouble you are taking to educate me :-) Anne -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8xQSsACgkQj93fyh4cnBdynQCeO6tY/798/6YonBALxJ0baZcy qG4AoIEHcZWsjIqhz4v3+SMk76FLhjIk =/T9e -----END PGP SIGNATURE----- From rob0 at gmx.co.uk Tue Feb 7 18:00:57 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Tue, 7 Feb 2012 10:00:57 -0600 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <4F30F7FC.9050607@noa.gr> References: <4F30F7FC.9050607@noa.gr> Message-ID: <20120207160057.GX14195@harrier.slackbuilds.org> On Tue, Feb 07, 2012 at 12:07:56PM +0200, Nikolaos Milas wrote: > I am running v2.0.13. In my dovecot.conf I have: > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap.conf > driver = ldap > } > > Is it legitimate to include multiple ldap userdb's, like: Of course. The driver chosen is not significant; the query used is all that matters. Having two LDAP searches is conceptually no different than having system users and SQL users. > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap1.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap1.conf > driver = ldap > } > > userdb { > args = /etc/dovecot/dovecot-usrdb-ldap2.conf > driver = ldap > } > > passdb { > args = /etc/dovecot/dovecot-passdb-ldap2.conf > driver = ldap > } > > If it is legitimate (in case configuration is different, please > correct me), in which sequence userdb's are evaluated? In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this in mind in setting up the queries. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From chayes at afo.net Tue Feb 7 22:44:20 2012 From: chayes at afo.net (Cliff Hayes) Date: Tue, 7 Feb 2012 14:44:20 -0600 Subject: [Dovecot] Fedora 16 configuration Message-ID: Hello, I am trying to get dovecot to work on a fedora 16 install with sendmail. I have been able to get it to work in the past with dovecot.conf but not with the new conf.d directory and associated config files. I keep seeing this in maillog: Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth attempts): rip=x.x.x.x, lip=x.x.x.x And the mail client comes back with username or password invalid. Is there instructions somewhere regarding fedora 16 installs? I found this one and tried it to no avail: http://www.server-world.info/en/note?os=Fedora_16&p=mail&f=2 Thanks in advance, Cliff From nmilas at noa.gr Tue Feb 7 22:56:02 2012 From: nmilas at noa.gr (Nikolaos Milas) Date: Tue, 07 Feb 2012 22:56:02 +0200 Subject: [Dovecot] Multiple userdb possible? In-Reply-To: <20120207160057.GX14195@harrier.slackbuilds.org> References: <4F30F7FC.9050607@noa.gr> <20120207160057.GX14195@harrier.slackbuilds.org> Message-ID: <4F318FE2.9070604@noa.gr> On 7/2/2012 6:00 ??, /dev/rob0 wrote: > ... > Having two LDAP searches is conceptually no different than having > system users and SQL users. > ... > In the order specified. A /etc/dovecot/dovecot-usrdb-ldap1.conf match > prevents searching in /etc/dovecot/dovecot-usrdb-ldap2.conf; keep this > in mind in setting up the queries > ... Thank you for the clarifications! Regards, Nick From rog7993 at web.de Wed Feb 8 01:02:01 2012 From: rog7993 at web.de (rog7993 at web.de) Date: Wed, 08 Feb 2012 00:02:01 +0100 Subject: [Dovecot] user login on behalf of another user In-Reply-To: <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> References: <4F2EB41E.6070100@web.de> <34D86700-F2C1-4BAA-A102-FD4B6D582732@iki.fi> Message-ID: <4F31AD69.2000109@web.de> Hello, Am 06.02.2012 16:05, schrieb Timo Sirainen: > Master user doesn't necessarily have access to all users' mailboxes. > In the passdb lookup you can decide if this master user is allowed to > be this destination user. For example if you used passdb > checkpassword, you could look at USER and MASTER_USER environment > variables to figure out if this combination should be allowed or not. > The checkpassword script can also do the actual authentication via > PAM (I'd think there's a way to call it somehow). > Thank you. I got an idea, how I could configure this. Ingo From manuel.bertrand at gmail.com Tue Feb 7 14:20:14 2012 From: manuel.bertrand at gmail.com (manuel.bertrand at gmail.com) Date: Tue, 07 Feb 2012 13:20:14 +0100 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <4F3116FE.4080704@gmail.com> Le 06/02/12 22:26, Ingo Thierack a ?crit : > Hello, > > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. > > If I do an search in an mailfolder, i get in the dovecot-log > > 2012-02-06 22:17:11 | dovecot: | > indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot > > > Log from solr. > Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute > INFO: [] webapp=/solr path=/select > params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} > hits=0 status=0 QTime=1 > > > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. > > Regard > Ingo Thierack Same thing here. Tried with 2.1-rc1 and rc5. No results. From moseleymark at gmail.com Wed Feb 8 01:51:50 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Tue, 7 Feb 2012 15:51:50 -0800 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: On Tue, Feb 7, 2012 at 4:08 AM, Mark Zealey wrote: > 06-02-2012 22:47, Timo Sirainen yazm??: > >> On 3.2.2012, at 16.16, Mark Zealey wrote: >> >>> I was doing some testing on sdbox yesterday. Basically I did the >>> following procedure: >>> >>> 1) Create new sdbox; deliver 2 messages into it (u.1, u.2) >>> 2) Create a copy of the index file (no cache file created yet) >>> 3) deliver another message to the mailbox (u.3) >>> 4) copy back index file from stage (2) >>> 5) deliver new mail >>> >>> Then the message delivered in stage 3 ie u.3 gets replaced with the >>> message delivered in (5) also called u.3. >> >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > > I've not actually tried this patch yet, but looking at it, it is perhaps > useful for the situation I described below when the index is corrupt. In > this case I am describing however, the not is NOT corrupt - it is simply an > older version (ie it only thinks there are the first 2 mails in the > directory, not the 3rd). This could happen for example when mails are being > stored on different storage than indexes; say for example you have 2 servers > with remote NFS stored mails but local indexes that rsync between the > servers every hour. You manually fail over one server to the other and you > then have a copy of the correct indexes but only from an hour ago. The mails > are all there on the shared storage but because the indexes are out of date, > when a new message comes in it will be automatically overwritten. > >>> (speaking of which, it would be great if force-resync also rebuilt the >>> cache files if there are valid cache files around, rather than just doing >>> away with them) >> >> Well, ideally there shouldn't be so much corruption that this matters.. > > > That's true, but in our experience we usually get corruption in batches > rather than a one-off occurrence. Our most common case is something like > this: Say for example there's an issue with the NFS server (assuming we are > storing indexes on there as well now) and so we have to killall -9 dovecot > processes or similar. In that case you get a number of corrupted indexes on > the server. Rebuilding the indexes generates an IO storm (say via lmtp or a > pop3 access); then the clients log in via imap and we have to re-read all > the messages to generate the cache files which is a second IO storm. If the > caches were rebuilt at least semi-intelligently (ie you could extract from > the cache files a list of things that had previously been cached) that would > reduce the effects of rare storage level issues such as this. > > Mark What about something like: a writer to an index/cache file checks for the existence of .1. If it doesn't exist or is over a day old, if the current index/cache file is not corrupt, take a snapshot of it as .1. Then if an index/cache file is corrupt, it can check for .1 and use that as the basis for a rebuild, so at least only a day's worth of email is reverted to its previous state (instead of all of it), assuming it's been modified in less than a day. Clearly it'd take up a bit more disk space, though the various dovecot.* files are pretty modest in size, even for big mailboxes. Or it might be a decent use case for some sort of journaling, so that the actual index/cache files don't ever get written to, except during a consolidation, to roll up journals once they've reached some threshold. There'd definitely be a performance price to pay though, not to mention breaking backwards compatibility. And I'm just throwing stuff out to see if any of it sticks, so don't mistake this for even remotely well thought-out suggestions :) From hmoreno at gmv.com Wed Feb 8 10:58:30 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Wed, 8 Feb 2012 09:58:30 +0100 Subject: [Dovecot] vsz_limit Message-ID: Hello Timo and dovecot teamwork, We have dovecot 2.0.10 installed with this configuration: # dovecot -n # 2.0.10: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.1.18.el5 x86_64 Scientific Linux SL release 5.2 (Boron) auth_cache_size = 500 k auth_cache_ttl = 1000 secs disable_plaintext_auth = no first_valid_uid = 501 lock_method = dotlock mail_fsync = always mail_gid = vmail mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = quota autocreate mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes namespace { inbox = yes location = prefix = separator = . type = private } namespace { hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { location = maildir:%h/Maildir/_backup prefix = _backup. separator = . type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin/autocreate = Spam plugin/autosubscribe = Spam plugin/quota = maildir:_backup:ns= plugin/quota_warning = storage=75%% quota-warning 75 %u plugin/quota_warning2 = storage=90%% quota-warning 90 %u plugin/sieve = %h/.dovecot.sieve plugin/sieve_dir = %h/sieve protocols = imap pop3 sieve service anvil { client_limit = 5000 } service auth { client_limit = 9000 unix_listener auth-master { group = vmail mode = 0700 user = vmail } user = root vsz_limit = 2 G } service imap-login { executable = /usr/local/dovecot/libexec/dovecot/imap-login inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 2048 process_min_avail = 20 user = vmail } service imap { executable = /usr/local/dovecot/libexec/dovecot/imap process_limit = 2048 } service managesieve-login { executable = /usr/local/dovecot/libexec/dovecot/managesieve-login inet_listener sieve { address = * port = 4190 } inet_listener sieve_deprecated { address = * port = 12000 } user = vmail } service managesieve { executable = /usr/local/dovecot/libexec/dovecot/managesieve } service pop3-login { executable = /usr/local/dovecot/libexec/dovecot/pop3-login inet_listener pop3 { address = * port = 110 } inet_listener pop3s { address = * port = 995 } process_limit = 2048 process_min_avail = 20 user = vmail } service pop3 { executable = /usr/local/dovecot/libexec/dovecot/pop3 process_limit = 2048 } service quota-warning { executable = script /usr/local/dovecot/bin/quota-warning.sh unix_listener quota-warning { group = vmail mode = 0700 user = vmail } } ssl_cert = fue rechazado automaticamente por nuestro sistema :%n%r rejection_subject = Rechazado: %s sendmail_path = /usr/sbin/exim } protocol sieve { managesieve_implementation_string = dovecot Pigeonhole managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 } The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? Thank you very much in advanced. Kind regards. ________________________________ [cid:image001.gif at 01CCE647.65B8FB30] H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com [cid:image002.gif at 01CCE647.65B8FB30] [cid:image003.gif at 01CCE647.65B8FB30] [cid:image004.gif at 01CCE647.65B8FB30] [cid:image005.gif at 01CCE647.65B8FB30] ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 5711 bytes Desc: image001.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image002.gif Type: image/gif Size: 1306 bytes Desc: image002.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image003.gif Type: image/gif Size: 1309 bytes Desc: image003.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image004.gif Type: image/gif Size: 1279 bytes Desc: image004.gif URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image005.gif Type: image/gif Size: 1323 bytes Desc: image005.gif URL: From warden at geneseo.edu Wed Feb 8 18:20:31 2012 From: warden at geneseo.edu (David Warden) Date: Wed, 8 Feb 2012 11:20:31 -0500 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <86E1DCD3-F1A8-43B5-AC08-E8CAF9094C38@geneseo.edu> On Feb 8, 2012, at 3:58 AM, H?ctor Moreno Blanco wrote: -snip- > service imap-login { > executable = /usr/local/dovecot/libexec/dovecot/imap-login > inet_listener imap { > address = * > port = 143 > } > inet_listener imaps { > address = * > port = 993 > } > process_limit = 2048 > process_min_avail = 20 > user = vmail > } > service imap { > executable = /usr/local/dovecot/libexec/dovecot/imap > process_limit = 2048 > } - snip, again - > > The problem is at the moment of maximum load of the system. We can see many imap sessions but there are also many imap-login processes, which is weird. Has it something to do with any parameter of our configuration? We think is something with the vsz_limit parameter. How can we tune it? I'm not 100% sure but if this is the case but it looks like you setting process_limit and process_min_avail in the imap-login service block is putting dovecot in "High Security" mode. http://wiki2.dovecot.org/LoginProcess has good details on that. In short, I suspect your settings are creating a new imap-login process for each new IMAP connection. As I understand it, vsz_limit is more important when running in "High Performance" mode where you have just 1 imap-login process that handles all new IMAP connections. That process can grow quite large if you have a lot of simultaneous IMAP clients. > > Thank you very much in advanced. > > Kind regards. > > ________________________________ > [cid:image001.gif at 01CCE647.65B8FB30] > > H?ctor Moreno Blanco > Divisi?n de Seguridad e Infraestructuras / > Security and Infrastructures Division > > GMV > Isaac Newton, 11 > P.T.M. Tres Cantos > E-28760 Madrid > Tel. +34 91 807 21 00 > Fax +34 91 807 21 99 > www.gmv.com > [cid:image002.gif at 01CCE647.65B8FB30] > > > > [cid:image003.gif at 01CCE647.65B8FB30] > > > > [cid:image004.gif at 01CCE647.65B8FB30] > > > > [cid:image005.gif at 01CCE647.65B8FB30] > > > > > > > ______________________ > This message including any attachments may contain confidential > information, according to our Information Security Management System, > and intended solely for a specific individual to whom they are addressed. > Any unauthorised copy, disclosure or distribution of this message > is strictly forbidden. If you have received this transmission in error, > please notify the sender immediately and delete it. > > ______________________ > Este mensaje, y en su caso, cualquier fichero anexo al mismo, > puede contener informacion clasificada por su emisor como confidencial > en el marco de su Sistema de Gestion de Seguridad de la > Informacion siendo para uso exclusivo del destinatario, quedando > prohibida su divulgacion copia o distribucion a terceros sin la > autorizacion expresa del remitente. Si Vd. ha recibido este mensaje > erroneamente, se ruega lo notifique al remitente y proceda a su borrado. > Gracias por su colaboracion. > > ______________________ > Hope this helps, David Warden From a23s4a2008 at yandex.ru Wed Feb 8 20:56:40 2012 From: a23s4a2008 at yandex.ru (Maxim) Date: Wed, 08 Feb 2012 22:56:40 +0400 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> Message-ID: <4F32C568.4050202@yandex.ru> 07.02.2012 0:27, Timo Sirainen ?????: > On 3.2.2012, at 15.15, User Max wrote: > >> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted > service auth { > user = root > } > > Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. From tss at iki.fi Thu Feb 9 01:26:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:26:04 +0200 Subject: [Dovecot] Slightly more intelligent way of handling issues in sdbox? In-Reply-To: <4F311429.2000708@webfusion.com> References: <4F2BEC3B.2050808@webfusion.com> <462F63A7-B045-4174-B34B-66AB8EEC3277@iki.fi> <4F311429.2000708@webfusion.com> Message-ID: <43ABD2D0-93D3-4E00-AC16-66EC75870EC2@iki.fi> On 7.2.2012, at 14.08, Mark Zealey wrote: >> http://hg.dovecot.org/dovecot-2.1/rev/a765e0a895a9 fixes this. > > I've not actually tried this patch yet, but looking at it, it is perhaps useful for the situation I described below when the index is corrupt. In this case I am describing however, the not is NOT corrupt - it is simply an older version (ie it only thinks there are the first 2 mails in the directory, not the 3rd). This could happen for example when mails are being stored on different storage than indexes; say for example you have 2 servers with remote NFS stored mails but local indexes that rsync between the servers every hour. You manually fail over one server to the other and you then have a copy of the correct indexes but only from an hour ago. The mails are all there on the shared storage but because the indexes are out of date, when a new message comes in it will be automatically overwritten. I don't recommend using local indexes with dbox, since there is actual data loss if they're not up to date (flags, and with mdbox the user may have copied/moved the mail elsewhere). Still, better to catch this situation than not: http://hg.dovecot.org/dovecot-2.1/rev/09db0f7aa6ce >>> (speaking of which, it would be great if force-resync also rebuilt the cache files if there are valid cache files around, rather than just doing away with them) >> Well, ideally there shouldn't be so much corruption that this matters.. > > That's true, but in our experience we usually get corruption in batches rather than a one-off occurrence. Our most common case is something like this: Say for example there's an issue with the NFS server (assuming we are storing indexes on there as well now) and so we have to killall -9 dovecot processes or similar. In that case you get a number of corrupted indexes on the server. Rebuilding the indexes generates an IO storm (say via lmtp or a pop3 access); then the clients log in via imap and we have to re-read all the messages to generate the cache files which is a second IO storm. If the caches were rebuilt at least semi-intelligently (ie you could extract from the cache files a list of things that had previously been cached) that would reduce the effects of rare storage level issues such as this. Well, the decisions are now remembered: http://hg.dovecot.org/dovecot-2.1/rev/d8d214cc1936 That can't really be improved .. If nothing is deleted from cache, it might contain invalid data and doveadm force-resync wouldn't be doing its job right. If anything is added to cache, it would require reading and parsing the mail contents during rebuild, and that's not in any way better than letting the imap processes do it later when the mailbox isn't locked. From tss at iki.fi Thu Feb 9 01:29:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:29:20 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: Message-ID: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? From tss at iki.fi Thu Feb 9 01:36:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:36:50 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? In-Reply-To: <4F304595.9030007@iwt-promotion.com> References: <4F304595.9030007@iwt-promotion.com> Message-ID: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> On 6.2.2012, at 23.26, Ingo Thierack wrote: > try to use the new 2.1rc and don't get any data in the searchindex. > > Tried first lucene, and switched than back so solr. Lucene should work.. > Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository > yesterday) > With 2.0 i see, if i start an search, solr had to work on the mail, now > happens nothing. > Upgrade the schema.xml, delete the old index. You mean you deleted Solr index, so that it's empty? That should work too. Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). From tss at iki.fi Thu Feb 9 01:40:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:40:19 +0200 Subject: [Dovecot] openbsd and dovecot In-Reply-To: <4F32C568.4050202@yandex.ru> References: <725061328274905@web20.yandex.ru> <37EEB180-0DF3-4B99-8EA7-D2E46820EC82@iki.fi> <4F32C568.4050202@yandex.ru> Message-ID: On 8.2.2012, at 20.56, Maxim wrote: >>> Feb 3 15:02:37 srv-mx-00 dovecot: auth: Error: bsdauth(rodin.m,127.0.0.1): getpwnam() failed: Operation not permitted >> service auth { >> user = root >> } >> >> Does this make any difference? If it does help, there's a bug somewhere. If it doesn't help, then there's something else weird going on. >> > No, it didn't help. Might it be that I need to edit any system files in /etc directory? When I set up dovecot on debian I had to edit /etc/pam.d/dovecot file accordingly to enable kerberos authentication for mail users in Active Directory. Someone suggested that I edit /etc/login.conf file in OpenBSD to enable kerberos for dovecot. But I've no idea of how these edits should look like. The problem is that auth worker process can't do getpwnam() call. Typically that would read from /etc/passwd, but I don't know if in OpenBSD if you have configured it to read from elsewhere (AD?). Anyway, I don't know why it would give EPERM error since the process is running as root. You could try ktraceing it to see what actually fails. From tss at iki.fi Thu Feb 9 01:42:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:42:11 +0200 Subject: [Dovecot] Fedora 16 configuration In-Reply-To: References: Message-ID: <1B36032D-0D9D-4CED-9D4C-6410C6C57D86@iki.fi> On 7.2.2012, at 22.44, Cliff Hayes wrote: > Feb 7 14:28:59 sendmail dovecot: pop3-login: Aborted login (no auth > attempts): rip=x.x.x.x, lip=x.x.x.x From http://wiki2.dovecot.org/WhyDoesItNotWork: Aborted login (no auth attempts) means that the client isn't even attempting to log in. Most likely you have disable_plaintext_auth=yes (default) and the client isn't configured to use SSL/TLS (or you've also set ssl=no). From tss at iki.fi Thu Feb 9 01:48:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:48:09 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120207082510.GA24432@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) .. > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > other ideas for what might be causing it ? The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS (30 secs). It still shouldn't have crashed of course, and that crash is already fixed in v2.1 (in the LMTP simplification change). Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. From tss at iki.fi Thu Feb 9 01:49:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 01:49:38 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: Message-ID: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director From tss at iki.fi Thu Feb 9 03:04:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:04:04 +0200 Subject: [Dovecot] force-resync fails to recover all messages in mdbox In-Reply-To: <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> References: <20120131174839.13512v46jc7ur23b.lealanko@webmail.helsinki.fi> <38EB3A30-DFD5-484B-852B-327BDA5E936E@iki.fi> <20120131183445.545717eennh24eg5.lealanko@webmail.helsinki.fi> Message-ID: <4BDF4036-EE45-42CC-9A34-D565D222107B@iki.fi> On 31.1.2012, at 18.34, Lauri Alanko wrote: > Well, well, well. This is interesting. Back with the indices > created by dsync: > > $ doveadm fetch guid all | grep guid: | sort | uniq -c | sort -n | tail > 17 guid: 1b28b22d4b2ee2885b5b81221c41201d > 17 guid: 730c692395661dd62f82088804b85652 > 17 guid: 865e1537fddba6698e010d0b9dbddd02 .. http://hg.dovecot.org/dovecot-2.0/rev/4a0b7dec3a22 avoids force-resync deleting these duplicates. It also logs a warning about the duplicates. http://hg.dovecot.org/dovecot-2.1/rev/2500de8f1f51 implements mbox_md5=all setting which avoids creation of these duplicates in the first place. I thought about adding some duplicate detection also to dsync (or anywhere in its path), but I couldn't do it without impacting performance in normal operation. > The complexity and opaqueness of the mdbox format is a worrisome. > It would ease my mind quite a bit if there were a simple tool > that would just dump out the plain message contents that are > stored inside the storage files, without involving any of > dovecot's index machinery. Then I would at least know that > whatever happens, as long as the storage files stay intact, I can > always migrate my mails into some other format. By using Dovecot indexes you could use e.g. "doveadm fetch" to dump them. Also "doveadm dump" can dump the dbox files' metadata, but not the message contents themselves. It probably wouldn't be difficult to implement that though. Also alternatively you could build something based on http://dovecot.org/tools/mdbox-obfuscate.pl From tss at iki.fi Thu Feb 9 03:45:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:45:05 +0200 Subject: [Dovecot] doveadm altmove fails to pack messages? In-Reply-To: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> References: <20120201195905.405864131jovy1yh.lealanko@webmail.helsinki.fi> Message-ID: On 1.2.2012, at 19.59, Lauri Alanko wrote: > When I use doveadm altmove, I notice that it may result in some very small storage files in either the primary or the alternate storage location. I'm not quite sure, but it almost seems like every storage file that contains messages to be moved is simply split into two parts, even if a part might only contain a single message. > > This seems quite curious, since the idea of mdbox is after all to store as many messages in a single file as fits in mdbox_rotate_size, and dovecot-lda can quite successfully append messages to existing storage files. So why can't doveadm altmove? Is this the expected behavior, or is there something wrong with my setup? Fixed: http://hg.dovecot.org/dovecot-2.0/rev/d83356f1fce6 From tss at iki.fi Thu Feb 9 03:59:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 03:59:30 +0200 Subject: [Dovecot] LSUB and subscribed folders In-Reply-To: <201112221334.06087.p.heinlein@heinlein-support.de> References: <201112221334.06087.p.heinlein@heinlein-support.de> Message-ID: <3477C497-8041-4F0C-8FB2-FA90394A231B@iki.fi> Hi, I've finally gotten to last year's mail backlog. :) On 22.12.2011, at 14.34, Peer Heinlein wrote: > User "bar" has some folders and one shared folder from "foo" if I use the > "LIST"-command, which should list AFAIK *all* folders: Yes. > Using the LSUB-command, which should only list subscribed folders I can see > some more shared folders. AFAIK the output of LSUB must be always less then > LIST: As Michael mentioned, LSUB can show some nonexistent mailboxes also that haven't been unsubscribed. > And why is /shared/foo/INBOX listed with "HasNoChildren"? It has no visible existing children (the LSUB entries didn't exist or no longer had necessary ACLs). > And, btw: Looks like's not possible to use the "doveadm"-command to get a > list of a user's complete folder list. Maybe "doveadm mailbox" should get > some extra parameters to geht all folders / only subscribe folders / only > shared folders: Yes, that would be nice .. but maybe this is enough for now: http://hg.dovecot.org/dovecot-2.1/rev/b600af017598 From tss at iki.fi Thu Feb 9 04:18:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:18:50 +0200 Subject: [Dovecot] crash on shared folders if remote user has empty $home In-Reply-To: <201112222201.41781.p.heinlein@heinlein-support.de> References: <201112222201.41781.p.heinlein@heinlein-support.de> Message-ID: On 22.12.2011, at 23.01, Peer Heinlein wrote: > d) If foo logs in, dovecot's exploring the shared folders given from the > non-existing user bar. But bar's entry in LDAP doesn't have $home any > more. imap crashes and closes the TCP-connection immediately. Fixed: http://hg.dovecot.org/dovecot-2.0/rev/09f964850245 From tss at iki.fi Thu Feb 9 04:29:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:29:09 +0200 Subject: [Dovecot] doveadm index In-Reply-To: <4EF218F7.2050709@gmx.de> References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: On 21.12.2011, at 19.35, e-frog wrote: >> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' > > Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. Yes, looks like it does. I changed the doveadm help output to make it clearer. > At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. > > Is there a way to examine the search index? Doveadm dump doesn't seem to like it. > > $: doveadm dump dovecot.index.search > $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. From tss at iki.fi Thu Feb 9 04:31:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:31:44 +0200 Subject: [Dovecot] Syncing shared mailboxes In-Reply-To: <4EE62AEB.2020607@graband.de> References: <4EE62AEB.2020607@graband.de> Message-ID: Hi, On 12.12.2011, at 18.25, Knud Dombrowsky wrote: > while trying to sync the mailboxes of several users who use and share > their mailboxes dsync prints this message: > > dsync-local(): Warning: Subscriptions file > /home//Maildir/subscriptions: Removing > invalid entry: shared// > > The problem is: Every user has to subscribe the shared folder again > after each run of dsync... > Is dsync capable of syncing shared mailboxes? Are there any > preliminaries to do so? Sorry for late reply. This should be fixed in v2.1 nowadays. From tss at iki.fi Thu Feb 9 04:52:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 04:52:08 +0200 Subject: [Dovecot] 2.0.14 doveadm proxy list segfault In-Reply-To: <20110929172134.GA25237@hawkeye.rutgers.edu> References: <20110929172134.GA25237@hawkeye.rutgers.edu> Message-ID: <142FE442-A257-4BB5-A8D6-37884C7132D0@iki.fi> Hi, On 29.9.2011, at 20.21, Tom Pawlowski wrote: > Periodic segfaults with doveadm proxy list: > > root at director1:~) doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault (core dumped) Do you still get these? gdb backtrace would be helpful. It already says "core dumped", so all that should be needed is: gdb doveadm core bt full Also it shows connection to "ipc" server dying .. Does it also crash? Anything in Dovecot's error log? From tss at iki.fi Thu Feb 9 05:02:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 05:02:02 +0200 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <20110926165651.GA20769@informatik.tu-muenchen.de> References: <20110926165651.GA20769@informatik.tu-muenchen.de> Message-ID: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Hi, On 26.9.2011, at 19.56, Christoph Bussenius wrote: > /* make sure two hierarchy separators aren't next to each others */ > for (p = storage_name+1; *p != '\0'; p++) { > if (p[0] == ns->real_sep && p[-1] == ns->real_sep) { > client_send_tagline(cmd, "NO Invalid mailbox name."); > return NULL; > } > } > > The loop iterates over the string storage_name starting from its second > byte. However, the string may be "". This is the case if you select > the root of a namespace, like in "SELECT shared" or > "SELECT shared/user1". In that case, the code will read past the end of > the buffer from random memory until it finds a zero byte or a duplicate > separator. I finally managed to get this far in my mail backlog. :) Yes, that's a bug. Fixed now slightly differently than you: http://hg.dovecot.org/dovecot-2.0/rev/d406e376f8ee From ybhu at hk1.ibm.com Thu Feb 9 10:36:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 9 Feb 2012 16:36:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: Thanks Timo, I just tried out the Director. One question is about the re-redirection. I know director will redirect all the simultaneous requests from the same user to only a single server at the same time. The question is how to manage the time period after last connection to re-decide to redirect which machine? director_user_expire? Look like not. I did one test, set director_user_expire = 1 min, then keep sending requests to the director in 2 min interval, the result is it keeps redirect to the same back end server. Actually what i want is the "secondary load balancer layer" can redirect requests to random back end. How to manage it? Only after the files on the previous back end is expired? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 07:49 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 7.2.2012, at 8.26, Andy YB Hu wrote: > I am running some concurrent testings under NFS. .. > Here are what I am doing: One session running loop of COPY commands > (while(1) COPY...) connects to one dovecot server; The other session > running loop of SELECT commands (while(1) SELECT...) connects to the other > dovecot server. Both are accessing the same mail box (/tmp/NFS); I don't even attempt to support this kind of configuration anymore. Use http://wiki2.dovecot.org/Director -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic17213.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From hmoreno at gmv.com Thu Feb 9 10:41:34 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 09:41:34 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: Hello Timo, I can see these errors, but I'm not sure if they have something to do with my problem: ... Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe ... Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) ... Do you see anything wrong? Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. I appreciate your answers. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 0:29 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 8.2.2012, at 10.58, H?ctor Moreno Blanco wrote: > The problem is at the moment of maximum load of the system. What problem? Does Dovecot log any errors? ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From busseniu at in.tum.de Thu Feb 9 13:31:10 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Thu, 09 Feb 2012 12:31:10 +0100 Subject: [Dovecot] [PATCH] Bad boundary check in client_find_namespace In-Reply-To: <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> References: <20110926165651.GA20769@informatik.tu-muenchen.de> <3ABF067F-34C1-425C-B14D-80B6F23A2A78@iki.fi> Message-ID: <4F33AE7E.5070008@in.tum.de> Hi, I'm glad to see my report finally arrive, thank you :) On 09.02.2012 04:02, Timo Sirainen wrote: > Fixed now slightly differently than you: No problem ? I agree that my code was a bit kludgy. I noticed that my original mail might be a bit unclear: > > while trying to investigate the bug I reported last week, I found that > > there is a broken boundary check So I just want to make clear that this patch does not fix the other problem that I reported at http://www.dovecot.org/list/dovecot/2011-September/061316.html (?Strange behavior from shared namespaces and INBOX, probably a bug?). Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik TU M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Thu Feb 9 14:52:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:52:42 +0200 Subject: [Dovecot] vsz_limit In-Reply-To: References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> Message-ID: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Connection reset by peer > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: Remote closed connection (process_limit reached?) > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. From tss at iki.fi Thu Feb 9 14:55:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 14:55:45 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> Message-ID: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. From janfrode at tanso.net Thu Feb 9 14:56:52 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 9 Feb 2012 13:56:52 +0100 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> Message-ID: <20120209125652.GA18552@dibs.tanso.net> On Thu, Feb 09, 2012 at 01:48:09AM +0200, Timo Sirainen wrote: > On 7.2.2012, at 10.25, Jan-Frode Myklebust wrote: > > > Feb 6 16:13:10 loadbalancer2 dovecot: lmtp(6601): Panic: file lmtp-proxy.c: line 376 (lmtp_proxy_output_timeout): assertion failed: (proxy->data_input->eof) > .. > > Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any > > other ideas for what might be causing it ? > > > The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS > (30 secs). It's actually 60 sec in v2.0 http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 > It still shouldn't have crashed of course, and that crash is already fixed in v2.1 > (in the LMTP simplification change). Do you think we should rather run v2.1-rc* on our dovecot directors (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? > Anyway, you can fix this without recompiling by returning e.g. "proxy_timeout=60" passdb extra field for 60 secs timeout. Thanks, well consider that option if it crashes too often... Have only seen this problem once for the last week. -jf From tss at iki.fi Thu Feb 9 15:04:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 15:04:16 +0200 Subject: [Dovecot] Performance of Maildir vs sdbox/mdbox In-Reply-To: <20120209125652.GA18552@dibs.tanso.net> References: <7c413ccbddc8e25584311c55672a51e5@standen.id.au> <33C8A3B3-7DB5-4638-8C34-54E0C7E739A4@iki.fi> <20120118185137.GA21945@dibs.tanso.net> <23FFD99C-7D70-40BE-A4F3-FD259FFC62E9@iki.fi> <20120203122552.GA12451@dibs.tanso.net> <8CBABB6D-7D9A-4E05-ACE3-A24D221D9266@iki.fi> <20120206210103.GA11888@dibs.tanso.net> <20120207082510.GA24432@dibs.tanso.net> <20120209125652.GA18552@dibs.tanso.net> Message-ID: On 9.2.2012, at 14.56, Jan-Frode Myklebust wrote: >>> Should I try increasing LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS, or do you have any >>> other ideas for what might be causing it ? >> >> The backend server didn't reply within LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> (30 secs). > > It's actually 60 sec in v2.0 > > http://hg.dovecot.org/dovecot-2.0/file/750db4b4c7d3/src/lmtp/lmtp-proxy.c#l13 30. LMTP_PROXY_DATA_INPUT_TIMEOUT_MSECS is not LMTP_PROXY_DEFAULT_TIMEOUT_MSECS >> It still shouldn't have crashed of course, and that crash is already fixed in v2.1 >> (in the LMTP simplification change). > > Do you think we should rather run v2.1-rc* on our dovecot directors > (for IMAP, POP3 and LMTP), even if we keep the backend servers on v2.0 ? Yes, I've done a lot of improvements to proxying and error handling/logging in v2.1. Also I'm planning on finishing my email backlog soon and making the last v2.1-rc before renaming it to v2.1.0. From tss at iki.fi Thu Feb 9 16:34:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 16:34:22 +0200 Subject: [Dovecot] fts (lucene): indexing of virtual mailboxes? In-Reply-To: References: Message-ID: <1328798062.11500.131.camel@innu> Hi, On Fri, 2011-09-23 at 16:49 +0200, Lutz Pre?ler wrote: > Hello, > > (recent 2.1alpha2 variant - my test setup known to Timo). > No time to diagnose in depth at the moment, but I just noticed > that SEARCHing in virtual mailboxes seems not to create lucene > index content of its own but use those of referenced mailboxes? > The problem is that no new indexing takes place. > Example: with > INBOX > INBOX.in% > all > in dovecot-virtual, for a given query I only get matches from > those mailboxes searched in previously. This was a long time ago, but I just tested and looks like it works nowadays. From apm at one.com Thu Feb 9 16:35:21 2012 From: apm at one.com (Peter Mogensen) Date: Thu, 09 Feb 2012 15:35:21 +0100 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox Message-ID: <4F33D9A9.2070605@one.com> Hi, Considering the scenario, where you have some old account with a different POP3 UIDL format and you migrate them to dovecot. So these old UIDLs would be saved to dovecot-uidlist. At some later time you want to introduce a virtual POP3 INBOX like described on: http://wiki.dovecot.org/Plugins/Virtual So you decide to make the new UIDL format "%f" - to make them unique across folders. So far so good. But then you decide to migrate to mdbox with all your old UIDLs. The docs says that saving old UIDLs is only supported in Maildir and that %f is only supported in Maildir. So is this at all possible? Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? /Peter From hmoreno at gmv.com Thu Feb 9 17:05:09 2012 From: hmoreno at gmv.com (=?iso-8859-1?Q?H=E9ctor_Moreno_Blanco?=) Date: Thu, 9 Feb 2012 16:05:09 +0100 Subject: [Dovecot] vsz_limit In-Reply-To: <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> References: <89B82515-71F0-46F4-8169-181815562AE3@iki.fi> <1BFA54A5-CEAE-46C9-A2AE-D4774BADD975@iki.fi> Message-ID: Hello Timo, There is no other problem as far as I know. That's why I think is has something to do with dovecot. Specifically with the imap an imap-login process, as I observe in the processes status. Anyway, the system load is not high enough to cause these problems. However, imap service doesn't work properly. So, in order to enable the login process in high performance mode I add the parameter service_count = 0. Right? I'll let you know if this helps. Kind regards. H?ctor Moreno Blanco Divisi?n de Seguridad e Infraestructuras / Security and Infrastructures Division GMV Isaac Newton, 11 P.T.M. Tres Cantos E-28760 Madrid Tel. +34 91 807 21 00 Fax +34 91 807 21 99 www.gmv.com ? ? ? -----Mensaje original----- De: Timo Sirainen [mailto:tss at iki.fi] Enviado el: jueves, 09 de febrero de 2012 13:53 Para: H?ctor Moreno Blanco CC: dovecot at dovecot.org Asunto: Re: [Dovecot] vsz_limit On 9.2.2012, at 10.41, H?ctor Moreno Blanco wrote: > I can see these errors, but I'm not sure if they have something to do with my problem: > > ... > Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: read(imap) failed: > Connection reset by peer Feb 8 12:04:57 XXXXXX dovecot: imap-login: > Error: read(imap) failed: Remote closed connection (process_limit > reached?) Feb 8 12:04:57 XXXXXX dovecot: imap-login: Error: > fd_send(imap, 16) failed: Broken pipe imap service isn't responding. > Feb 8 12:08:09 XXXXXX dovecot: imap-login: Error: master(imap): Auth > request timed out (received 0/12 bytes) imap process isn't responding because auth process isn't responding. > Do you see anything wrong? Yes. Is the system load very high? That could explain this. Or do you see any other error messages? Those errors you pasted above show that something is wrong, but not the root cause of what's wrong. > Anyway, I'm going to investigate what David Warden told me about the "High Security" mode, just in case it is related to my problem. It could at least help reduce the load. Also it would be a good idea to upgrade to latest v2.0. ______________________ This message including any attachments may contain confidential information, according to our Information Security Management System, and intended solely for a specific individual to whom they are addressed. Any unauthorised copy, disclosure or distribution of this message is strictly forbidden. If you have received this transmission in error, please notify the sender immediately and delete it. ______________________ Este mensaje, y en su caso, cualquier fichero anexo al mismo, puede contener informacion clasificada por su emisor como confidencial en el marco de su Sistema de Gestion de Seguridad de la Informacion siendo para uso exclusivo del destinatario, quedando prohibida su divulgacion copia o distribucion a terceros sin la autorizacion expresa del remitente. Si Vd. ha recibido este mensaje erroneamente, se ruega lo notifique al remitente y proceda a su borrado. Gracias por su colaboracion. ______________________ From tss at iki.fi Thu Feb 9 17:19:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 17:19:40 +0200 Subject: [Dovecot] POP3 UIDLs with virtual INBOX and migration from maildir->mdbox In-Reply-To: <4F33D9A9.2070605@one.com> References: <4F33D9A9.2070605@one.com> Message-ID: <1328800780.11500.135.camel@innu> On Thu, 2012-02-09 at 15:35 +0100, Peter Mogensen wrote: > Hi, > > Considering the scenario, where you have some old account with a > different POP3 UIDL format and you migrate them to dovecot. > > So these old UIDLs would be saved to dovecot-uidlist. > > At some later time you want to introduce a virtual POP3 INBOX like > described on: > http://wiki.dovecot.org/Plugins/Virtual > > So you decide to make the new UIDL format "%f" - to make them unique > across folders. > > So far so good. Assuming the messages are in the same order, so far so good. > But then you decide to migrate to mdbox with all your old UIDLs. > The docs says that saving old UIDLs is only supported in Maildir and > that %f is only supported in Maildir. > > So is this at all possible? > > Would pop3_uidl_format = %g solve this (except for the old legacy UIDL's) ? %g and %f are equal with Maildir. And if you migrated with dsync from maildir to mdbox, then all GUIDs and POP3 UIDLs are preserved. But test it first! The main potential problem is that although UIDLs are preserved, their order isn't and POP3 clients don't like the order changing. With Maildir uidlist you can reorder POP3 mails to different than IMAP mails, but with mdbox you can't currently. From tss at iki.fi Thu Feb 9 19:34:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:34:06 +0200 Subject: [Dovecot] [dovecot] Getting duplicates when using snarf plugin with mbox backend In-Reply-To: <4DF609B1.8020309@psu.edu> References: <4DF609B1.8020309@psu.edu> Message-ID: <1328808846.11500.136.camel@innu> On Mon, 2011-06-13 at 08:59 -0400, Jonathan SIegle wrote: > Running dovecot version 2.0.11. To reproduce, open two imap sessions and > issue a check command from each at the same time with new mail in the queue. > > 0 login testuser testpw > 1 select inbox > -- Deliver mail -- > 2 check Finally fixed: http://hg.dovecot.org/dovecot-2.0/rev/76220f2b5966 From tss at iki.fi Thu Feb 9 19:52:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 19:52:43 +0200 Subject: [Dovecot] BUG(?): Incorrect responses for ACL prohibited actions In-Reply-To: <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> References: <20110510124458.Horde.LnlpeIF5lbhNyYeqlCXUIdA@bigworm.curecanti.org> <20110510224607.Horde.k4LCS4F5lbhNyhSPSJokIdA@bigworm.curecanti.org> Message-ID: <1328809963.11500.146.camel@innu> Hi, Continuing this old thread: On Tue, 2011-05-10 at 22:46 -0600, Michael M Slusarz wrote: > But as far as the NO for a non-silent STORE, it seems that RFC 4314 > [4] disagrees with you: > > STORE operation SHOULD NOT fail if the user has rights to modify > at least one flag specified in the STORE, as the tagged NO > response to a STORE command is not handled very well by deployed > clients. > > To me, the negative inference from this statement would be: "STORE > operation SHOULD fail if the user has no rights to modify at least one > flag specified in the STORE." That's not the negative of it. :) > At a minimum, a NOPERM response should be thrown, or else there is no > feedback at all why the flag was not set (without parsing ACLs). Perhaps OK [NOPERM] or some other kind of informational message about it .. But there's no way to do it with Dovecot's current API. Also RFC 3501 recommends implementing "session flags" for flags that cannot be permanently stored. So even if user doesn't have access to set any flags, a "well behaving IMAP server" (so not Dovecot :( ) would set those flags for the duration of the current session. Anyway, you can look at PERMANENTFLAGS reply to see if it's possible to set the flag, no need to look at ACLs. > >> My reading of this is that NOPERM should be returned for ANY ACL > >> prohibited action, not just for selecting or creating a mailbox. > >> Dovecot 2.0.12 does not return NOPERM for DELETE/EXPUNGE actions > >> (at a minimum) that are prohibited. > > > > I'm not really sure. Maybe for EXPUNGE a NO would be okay. For flag > > changes it's just annoying to see clients popup pointless error > > messages when trying to set a \Seen flag (or \Answered flag when > > replying). Apparently I've tried this earlier, since there's a comment in code: ret = acl_mailbox_right_lookup(_mail->box, ACL_STORAGE_RIGHT_EXPUNGE); if (ret <= 0) { /* if we don't have permission, silently return success so users won't see annoying error messages in case their clients try automatic expunging. */ From tss at iki.fi Thu Feb 9 20:00:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:00:05 +0200 Subject: [Dovecot] Crash on mail folder delete In-Reply-To: References: <4F20922C.60206@amfes.com> <54B5D728-EC26-4633-A927-7EC040043BF5@iki.fi> <4F20939E.4010903@amfes.com> Message-ID: <1328810405.11500.151.camel@innu> On Wed, 2012-01-25 at 16:04 -0800, Daniel L. Miller wrote: > On 1/25/2012 3:43 PM, Daniel L. Miller wrote: > > On 1/25/2012 3:42 PM, Timo Sirainen wrote: > >> On 26.1.2012, at 1.37, Daniel L. Miller wrote: > >> > >>> Attempting to delete a folder from within the trash folder using > >>> Thunderbird. I see the following in the log: > >> Dovecot version? > >> > > 2.1.rc3. I'm compiling rc5 now... > > > Error still there on rc5. > > Jan 25 16:03:47 bubba dovecot: imap(dmiller at amfes.com): Panic: file > mailbox-list-fs.c: line 156 (fs_list_get_path): assertion failed: > (mailbox_list_is_valid_pattern(_list, name)) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/95a9428fe68b From tss at iki.fi Thu Feb 9 20:24:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:24:58 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> Message-ID: <1328811898.11500.157.camel@innu> On Sat, 2010-12-25 at 10:08 +0100, Thomas Leuxner wrote: > plugin { > quota = dict:user::file:%h/mdbox/dovecot-quota > quota_rule = *:storage=1GB > quota_rule2 = Trash:storage=+10%% > } > > Kick off a manual backup: > > $ dsync -u user at domain.tld backup mdbox://mdbox > > This results in doubling the quota for the backed up user. This is problematic. With dict quota you'll have this problem, because both source and destination uses the same file. So it would kind of make sense to disable quota for the destination dsync.. Except with Maildir++ the quota is stored in the Maildir root directory. There are no problems with dsyncing it, and you most likely wouldn't want quota disabled there. So .. I'm not really sure what I can do about this. There are some workarounds you could do, like: dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= (works only with latest 2.0/2.1 hg, but with older versions you could do e.g. -o mail_plugins=) Still, it would be nice if there was some generic solution to this. Perhaps the destination username should be something different, like "backup". In dict-sql case then it would modify "backup" user's quota. For dict-file the %h could maybe expand to backup user's homedir.. The backup username probably should be a parameter to dsync I guess.. But an extra parameter wouldn't fix this automatically.. From tss at iki.fi Thu Feb 9 20:39:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:39:57 +0200 Subject: [Dovecot] 2.1.rc1 (8a63f621bd2e): SiS permission issue + crash In-Reply-To: <4EE2D39C.2030106@localhost.localdomain.org> References: <4EE2D39C.2030106@localhost.localdomain.org> Message-ID: <1328812797.11500.159.camel@innu> On Sat, 2011-12-10 at 04:35 +0100, Pascal Volk wrote: > dsync -u test-0 at example.com mirror maildir:/tmp/Maildir > rm -rf Maildir && cp -a Maildir_org Maildir && chown -R 70010:70002 Maildir > dsync -vu test-1 at example.com mirror maildir:/tmp/Maildir > dsync(test-1 at example.com): Error: stat(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.4f06409857c627e0) failed: Permission denied > dsync(test-1 at example.com): Error: safe_mkstemp(/srv/mail/.SiS/70002/a2/7b/.temp.blau.819.) failed: Permission denied > dsync(test-1 at example.com): Panic: file dsync-worker-local.c: line 1644 (local_worker_save_msg_continue): assertion failed: (ret == -1) I couldn't reproduce this crash, but I guess this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/e29bc3eb0ba6 Also fixed a related problem where if dbox failed to save a message it still added it to index: http://hg.dovecot.org/dovecot-2.1/rev/98a59ac1f3d0 From tss at iki.fi Thu Feb 9 20:54:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 20:54:02 +0200 Subject: [Dovecot] Segfaul probably during dsync In-Reply-To: <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> References: <1c3f5d96e58a77359a4b56dac645eeda@127.0.0.1> <277a7ac434966ec84de651577ae7ebbe@127.0.0.1> <03aed57b4a4e595de5f566a4b98d8f54@shrike.scheff32.de> Message-ID: <1328813642.11500.161.camel@innu> On Tue, 2011-04-12 at 14:52 +0200, Matthias Rieber wrote: > Hi, > > it's caused by a virtual folder: > > INBOX.IBX.Folder1 > INBOX.Ordner.Folder1 > INBOX.Ordner.Folder1.* > OR (OR (OR HEADER FROM bar.com HEADER FROM bar.de) HEADER FROM > foo.com) HEADER FROM barfoos.net NOT HEADER FROM root@ NOT HEADER FROM > www-data@ SINCE 1-Jan-2010 > > When I delete the dovecot.index.search* files it works for a while but > fails again. Are you still getting these crashes? I tried to reproduce but couldn't. From tss at iki.fi Thu Feb 9 21:02:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:02:03 +0200 Subject: [Dovecot] Strange behavior from shared namespaces and INBOX, probably a bug In-Reply-To: <20110923121329.GA13235@informatik.tu-muenchen.de> References: <20110923121329.GA13235@informatik.tu-muenchen.de> Message-ID: <1328814123.11500.163.camel@innu> On Fri, 2011-09-23 at 14:13 +0200, Christoph Bussenius wrote: > Some folders of user1, including the INBOX, have been shared using these IMAP commands: > . login user1 XXXXXX > . setacl INBOX user2 lrwstiekx > . setacl box-a user2 lrwstiekx > > Now if we use telnet to log in as user2 and select "shared/user1", it will contain the same > mails as "shared/user1/INBOX". > > The really strange thing is that "SELECT"-ing "shared/user1" succeeds only > if it is the first command afted logging in. If it not the first > command (if e. g. the "LIST" or "SELECT" command has already been used), > then dovecot will report that the mailbox does not exist. v2.1 always fails to select "shared/user1". I don't think I'll bother figuring out why v2.0 doesn't, might not be an easy fix. Much of the code related to this was rewritten in v2.1. From tss at iki.fi Thu Feb 9 21:47:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 09 Feb 2012 21:47:47 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328212183-sup-560@rose.makesad.us> References: <1328212183-sup-560@rose.makesad.us> Message-ID: <1328816867.11500.167.camel@innu> On Thu, 2012-02-02 at 14:59 -0500, Asheesh Laroia wrote: > I'm guessing this is some bad interaction with sdbox and partial file downloads? > > I haven't read the code for this, but I would guess the dsync process isn't being atomic > about file transfers, so it is leaving half-completed transfers in place, which results > in corrupt files when they're next examined. There were some problems related to this in dbox, although in your case it seems to be worse than what it should.. I've anyway done several fixes in v2.1. Can you try if these problems happen with it too? And in any case cleanup the dbox from the *.broken files, so that "doveadm force-resync" won't give any errors. From tss at iki.fi Thu Feb 9 23:15:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 9 Feb 2012 23:15:15 +0200 Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <1328816867.11500.167.camel@innu> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> Message-ID: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> On 9.2.2012, at 21.47, Timo Sirainen wrote: > I've anyway done several fixes in v2.1. Can you try if these problems > happen with it too? And in any case cleanup the dbox from the *.broken > files, so that "doveadm force-resync" won't give any errors. A bit more specifically: The last such dbox bug was fixed only today, so you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this week. And in general: It would be helpful to have a clean fully working dbox, and then know the *first* error(s) that gets printed about dsync corrupting it. Otherwise it's difficult to guess what are some old problems and what are new ones and which problems happens only because of another problem. From asheesh at asheesh.org Thu Feb 9 23:16:32 2012 From: asheesh at asheesh.org (Asheesh Laroia) Date: Thu, 9 Feb 2012 16:16:32 -0500 (EST) Subject: [Dovecot] dsync deleting too many emails (sdbox) In-Reply-To: <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> References: <1328212183-sup-560@rose.makesad.us> <1328816867.11500.167.camel@innu> <232150A7-F78C-43EA-8C58-E0841DF94491@iki.fi> Message-ID: On Thu, 9 Feb 2012, Timo Sirainen wrote: > On 9.2.2012, at 21.47, Timo Sirainen wrote: > >> I've anyway done several fixes in v2.1. Can you try if these problems >> happen with it too? And in any case cleanup the dbox from the *.broken >> files, so that "doveadm force-resync" won't give any errors. > > A bit more specifically: The last such dbox bug was fixed only today, so > you'd need v2.1 hg version or wait for v2.1.rc6 which should happen this > week. > > And in general: It would be helpful to have a clean fully working dbox, > and then know the *first* error(s) that gets printed about dsync > corrupting it. Otherwise it's difficult to guess what are some old > problems and what are new ones and which problems happens only because > of another problem. Good to know. This weekend I can try to set up something of a 'lab' for testing dsync + (s)dbox, both to see if I can reproduce the errors with the old versions, and to see if the new versions fix them. I'll keep in mind the consideration of knowing the first error that gets printed! -- Asheesh. From ybhu at hk1.ibm.com Fri Feb 10 05:12:17 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 11:12:17 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic08408.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ybhu at hk1.ibm.com Fri Feb 10 10:08:10 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Fri, 10 Feb 2012 16:08:10 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: Searched the archive, and got the doveadm -a director-admin for local doveadm access, and -a host: port for remote doveadm access. And you said: http://dovecot.org/list/dovecot/2010-July/050731.html Now in my director, I have configed the userdb passwd, but the same error occur: doveadm(root): Error: User listing returned failure doveadm(root): Error: user listing failed And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but userdbs don't support it [root at c-dev1ws--01-01 conf_director.d]$dovecot -n -c /usr/local/etc/dovecot/dovecot_director.conf # 2.0.13: /usr/local/etc/dovecot/dovecot_director.conf # OS: Linux 2.6.18-274.17.1.el5 i686 Red Hat Enterprise Linux Server release 5.7 (Tikanga) auth_debug = yes auth_debug_passwords = yes auth_mechanisms = xpreauth auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@: auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot_director director_mail_servers = 9.119.7.129 director_servers = 9.119.7.60:9090 disable_plaintext_auth = no lock_method = dotlock log_path = /tmp/log mail_debug = yes mail_fsync = always mail_gid = mdrop mail_nfs_index = yes mail_nfs_storage = yes mail_uid = mdrop mmap_disable = yes passdb { args = proxy=y port=144 nopassword=y driver = static } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director inet_listener imap { address = 9.119.7.60 port = 10143 } } ssl = no userdb { driver = passwd } By the way, I run the director and backend in the same server. Andy YB Hu/Hong Kong/IBM at IBMHK Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/10/2012 11:12 Subject AM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List OK. One more question. Now, the director and backend server are running on the same servers, I set it up according to http://wiki2.dovecot.org/RunningDovecot#Running_Multiple_Invocations_of_Dovecot . The question is how to use doveadm to manage the different instances? I know there is dovecot -c , but how about doveadm? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/09/2012 08:55 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 9.2.2012, at 10.36, Andy YB Hu wrote: > I just tried out the Director. One question is about the re-redirection. I > know director will redirect all the simultaneous requests from the same > user to only a single server at the same time. The question is how to > manage the time period after last connection to re-decide to redirect which > machine? director_user_expire? Look like not. > > I did one test, set director_user_expire = 1 min, then keep sending > requests to the director in 2 min interval, the result is it keeps redirect > to the same back end server. In normal operation the user is always redirected to the same server. http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html has some more details. If you have enough connections, it shouldn't matter that the connections aren't constantly going to random backends. In practice they get distributed well enough. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic21544.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From ms at mike2k.de Fri Feb 10 14:13:17 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Fri, 10 Feb 2012 13:13:17 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner Message-ID: Hi, I have shared one of my mailboxes to another user providing him full rights to that mailbox. $ doveadm acl get -u user1 doc ID Global Rights user=user2 admin create delete expunge insert lookup post read write write-deleted write-seen When that user creates a new mailbox within the shared mailbox, the mailbox is successfully created but with an empty ACL. $ doveadm acl get -u user1 doc/hallo ID Global Rights According to the wiki, it should inherit the ACL from its parent, i.e. have full permissions for user2. Dovecot logs the following errors when I access the parent folder the first time after the other user created the child mailbox: dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: Recent flags state corrupted for mailbox Sent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/Sent/dbox-Mails/dovecot.index reset, view is now inconsistent dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent This is on dovecot 2.1rc5. -Michael From giles at coochey.net Fri Feb 10 16:36:53 2012 From: giles at coochey.net (Giles Coochey) Date: Fri, 10 Feb 2012 14:36:53 +0000 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! Message-ID: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> So I went and migrated a mailbox from mdbox from maildir format. All seemed ok, but I've found that I now have some missing folders. The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What can be done? (I have a backup of the maildir box). Note though that mdbox is now a live mailbox and has sent and received mails since the migration... -- Message sent via my webmail account. From alexwbaule at gmail.com Fri Feb 10 19:39:39 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 10 Feb 2012 15:39:39 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? Message-ID: Hy Everyone...and Timo ! There is a way to know if a email was expunged (deleted from hard disk) inside a plugin ? I rewrite the expunge function, but the real expunge function is void, i can't know if was really expunged. There is a way to know this ? From mildred593+dovecot at gmail.com Fri Feb 10 20:54:36 2012 From: mildred593+dovecot at gmail.com (Mildred Ki'Lya) Date: Fri, 10 Feb 2012 19:54:36 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, I am trying to set up my own mail server (to get off Google) and I have a problem with mail delivery. I'm using Postfix as a SMTP front-end, and use the LMTP server included in dovecot to receive mail from postfix. I have a setup which works with the dovecot LDA, it involves calling dovecot-lda with the -d flag. Postfix computes the username by removing what's after the recipient_delimiter and the domain from the mail address, and gives this username to dovecot. But, I'd prefer using LMTP, as it is possible to reject unknown e-mails without having to duplicate the user database (it is explained here: http://wiki2.dovecot.org/LDA/Postfix ) I set up LMTP so that postfix uses the socket provided by dovecot, and it works. Only I get the following error message in the logs: Feb 10 18:05:53 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred at mildred.fr(in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> Feb 10 18:06:52 ashley postfix/smtpd[9298]: NOQUEUE: reject: RCPT from mail-wi0-f179.google.com[209.85.212.179]: 450 4.1.1 : Recipient address rejected: unverified address: host mail.mildred.fr[private/dovecot-lmtp] said: 550 5.1.1 User doesn't exist: mildred-test at mildred.fr (in reply to RCPT TO command); from=< shanti at sogilis.com> to= proto=ESMTP helo=< mail-wi0-f179.google.com> I followed the explanations here: http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP The problem is that my passwd-like file only contains a user named "mildred". I tried to add a user mildred at mildred.fr and mildred%mildred.frbut it doesn't work either. Do you know how I could tell dovecot to guess the username correctly ? Thanks, Mildred -- Mildred Ki'Lya http://mildred.fr From wrosenauer at gmail.com Fri Feb 10 22:27:51 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:27:51 +0100 Subject: [Dovecot] Postfix + LMTP with virtual users: how to strip domain from LMTP RCPT TO In-Reply-To: References: Message-ID: Hi, On Fri, Feb 10, 2012 at 7:54 PM, Mildred Ki'Lya wrote: > I followed the explanations here: > http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP > > The problem is that my passwd-like file only contains a user named > "mildred". I tried to add a user mildred at mildred.fr and > mildred%mildred.frbut it doesn't work either. > > Do you know how I could tell dovecot to guess the username correctly ? I think you need to set auth_username_format = %Ln which strips the domain part from the recipient. Wolfgang From e-frog at gmx.de Fri Feb 10 22:31:45 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 10 Feb 2012 21:31:45 +0100 Subject: [Dovecot] doveadm index In-Reply-To: References: <4EE9690A.7040308@amfes.com> <1324376325.3597.39.camel@innu> <34DB33F4-27C1-41DA-91F5-46F0AA712C78@iki.fi> <4EF218F7.2050709@gmx.de> Message-ID: <4F357EB1.4000103@gmx.de> On 09.02.2012 03:29, wrote Timo Sirainen: > On 21.12.2011, at 19.35, e-frog wrote: > >>> But some day doveadm index (and several other commands) should allow wildcards in mailbox names, so you could just do doveadm index -A '*' >> >> Actually I'm using doveadm index -A '*' in a daily cron job and it seems to work for me. > > Yes, looks like it does. I changed the doveadm help output to make it clearer. Thanks. >> At least the timestamps of the dovecot.index.search and dovecot.index.search.uids files equal or close to the cron job start time in every folder. >> >> Is there a way to examine the search index? Doveadm dump doesn't seem to like it. >> >> $: doveadm dump dovecot.index.search >> $: doveadm(vmail): Fatal: Can't autodetect file type: dovecot.index.search > > No, you can't dump Squat indexes. v2.1 can dump fts-lucene indexes though. Ok, I'm meanwhile running v2.1 and switched to fts-lucene. Works like a charm! From wrosenauer at gmail.com Fri Feb 10 22:46:01 2012 From: wrosenauer at gmail.com (Wolfgang Rosenauer) Date: Fri, 10 Feb 2012 21:46:01 +0100 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 Message-ID: Hi, I finally migrated my IMAP server from cyrus to dovecot2. I have been using cyrus since many years and have not much experience with dovecot yet though. I've converted my imap server using cyrus2dovecot (several times until I thought it was correct) but there is one thing left which confuses me and I'm not sure what I can or need to do to change it. With Cyrus I used unixhierarchysep: yes and subfolders within Cyrus are always below INBOX. I'm used to that in my IMAP clients and wanted to preserve it with Dovecot. So during migration I finally made sure to create the subfolder again below INBOX wich looks fine in Thunderbird for example. But my mobile devices (Android based with K9) were displaying the folders with Cyrus just with their name w/o the INBOX prefix. (Mobile clients are bad in displaying folder hierarchies so it was convenient to have them looking like they were in parallel to the Inbox.) Now I get the folders displayed as INBOX.whatever and my Android mail clients fail to map them to the special (and localized) folders like Trash. While that seems consistent to my current hierarchy it's not what I wanted. So basically my two questions are: - Can I configure Dovecot in a way that it looks like Cyrus? Having subfolders below Inbox but let clients like those on Android display them besides Inbox? - If that's not possible, can I move the subfolders up one level so they are next to the Inbox? How would I do that? Sorry for the beginner questions. I played around with Dovecot before but using it in production environment disclosed some fine differences. Thanks, Wolfgang From glenmarton at gmail.com Fri Feb 10 23:54:42 2012 From: glenmarton at gmail.com (Glen Johnson) Date: Fri, 10 Feb 2012 16:54:42 -0500 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. Message-ID: I have an IMAP mail server running Postfix, and Dovecot. I have an old Notebook computer which I can view all e-mails. The problem I am having is when I try to use any other computer on the network to view the e-mail I can not see the contents of all the folders. I can see some of the folders and their content but not all. To answer a couple of preliminary questions: dovecot version 2.0.13, postfix version 2.8.3 dovecot.conf ------------------ # 2.0.13: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 4.0.1 i386 auth_debug = yes auth_verbose = yes disable_plaintext_auth = no listen = * log_path = /var/log/dovecot.log mail_location = maildir:~/Maildir/ passdb { driver = passwd } passdb { args = /usr/pkg/etc/dovecot/passwd driver = passwd-file } protocols = imap service auth { user = root } service imap-login { executable = /usr/pkg/libexec/dovecot/imap-login } ssl_cert = References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 09.02.2012 um 19:24 schrieb Timo Sirainen: > > So .. I'm not really sure what I can do about this. There are some > workarounds you could do, like: > > dsync -u user at domain.tld backup dsync -o mail=mdbox://mdbox -o plugin/quota= > Thanks tried this morning. I think it has some issues with options and crashes without options :/ $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] Aborted $ dovecot --version 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sat Feb 11 10:08:29 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 09:08:29 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: Am 11.02.2012 um 08:58 schrieb Thomas Leuxner: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > dsync(tlx at leuxner.net): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43c9a) [0x7f29e7fbec9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x32) [0x7f29e7fbed82] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f29e7f963af] -> doveadm() [0x428c6a] -> doveadm() [0x428dd1] -> doveadm(dsync_worker_msg_save+0x99) [0x428299] -> doveadm() [0x423e85] -> doveadm() [0x4287bb] -> doveadm() [0x4289b8] -> doveadm(dsync_worker_msg_get+0x56) [0x427ea6] -> doveadm() [0x423c5e] -> doveadm(dsync_brain_msg_sync_more+0x2a4) [0x423524] -> doveadm(dsync_brain_sync+0x4b9) [0x421df9] -> doveadm() [0x422c79] -> doveadm(dsync_brain_sync+0x7fd) [0x42213d] -> doveadm() [0x422b0c] -> doveadm(dsync_brain_sync+0x63e) [0x421f7e] -> doveadm(dsync_brain_sync_all+0x18) [0x4229e8] -> doveadm() [0x4204b0] -> doveadm() [0x40f346] -> doveadm(doveadm_mail_single_user+0x59) [0x40f3f9] -> doveadm() [0x40f646] -> doveadm(doveadm_mail_try_run+0x141) [0x40fa91] -> doveadm(main+0x391) [0x415de1] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f29e7c37c8d] -> doveadm() [0x40ec59] > Aborted After having some coffee, above was inflicted by having insufficient disk space. Shouldn't crash anyway I guess. > $ dovecot --version > 2.1.rc5 (a927b41bf699) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cannewilson at googlemail.com Sat Feb 11 12:09:29 2012 From: cannewilson at googlemail.com (Anne Wilson) Date: Sat, 11 Feb 2012 10:09:29 +0000 Subject: [Dovecot] Can't view all sub folders or e-mail using Dovecot with Maildir. In-Reply-To: References: Message-ID: <201202111009.29259.cannewilson@googlemail.com> On Friday 10 February 2012 21:54:42 Anne Wilson wrote: > I have an IMAP mail server running Postfix, and Dovecot. I have an old > Notebook computer which I can view all e-mails. The problem I am having is > when I try to use any other computer on the network to view the e-mail I > can not see the contents of all the folders. I can see some of the folders > and their content but not all. Does your mail client support subscriptions? Check your menus for this - you may find that only some folders are subscribed to. Anne From tlx at leuxner.net Sat Feb 11 12:59:16 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 11 Feb 2012 11:59:16 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox $ doveadm -f table quota get -u tlx at leuxner.net Please look into the crash though. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mark.zealey at webfusion.com Sat Feb 11 16:33:34 2012 From: mark.zealey at webfusion.com (Mark Zealey) Date: Sat, 11 Feb 2012 16:33:34 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP Message-ID: <4F367C3E.2080806@webfusion.com> Hi there, I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: 451 4.3.0 Temporary user lookup failure and in the logs: 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly I'm sure this is an obvious issue for someone who's come across it before. The /var/run/dovecot/director-userdb socket is on the system; exactly the same permissions as the standard auth-userdb Here are the relevant bits of my config (I'm setting lmtp to listen on a different port than normal). Backend password database is mysql if that makes any difference. lmtp_proxy = yes service lmtp { inet_listener lmtp_test { port = 2450 } } protocol lmtp { auth_socket_path = director-userdb } Thanks, Mark From clube03 at gmail.com Sat Feb 11 20:46:32 2012 From: clube03 at gmail.com (Gustavo) Date: Sat, 11 Feb 2012 16:46:32 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: Ok, I study the wikki and the dovecot works. But when I try to send a mail using squirrel, I get this error on logs: Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from localhost.localdomain[127.0.0.1] Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication mechanisms Feb 11 18:41:56 jubileu postfix/master[18383]: warning: process /usr/lib/postfix/smtpd pid 19522 exit status 1 Feb 11 18:41:56 jubileu postfix/master[18383]: warning: /usr/lib/postfix/smtpd: bad command startup -- throttling How can I fix this? My conf files seems to be ok: ## Diretorio Base ## base_dir = /var/run/dovecot/ ## Protocolos que ele ira executar ## protocols = imap imaps pop3 pop3s log_timestamp = '%Y-%m-%d %H:%M:%S ' syslog_facility = mail ## Dados do SSL ## ssl_listen = localhost ssl_cert_file = /etc/postfix/mail-cert.pem ssl_key_file = /etc/postfix/mail-key.pem ssl_parameters_regenerate = 168 verbose_ssl = no mail_location = maildir:/var/vmail/%d/%u mail_access_groups = mail mail_debug = no first_valid_uid = 150 last_valid_uid = 150 maildir_copy_with_hardlinks = yes protocol imap { } protocol pop3 { # Login executable location. login_executable = /usr/lib/dovecot/pop3-login mail_executable = /usr/lib/dovecot/pop3 pop3_uidl_format = %08Xu%08Xv } protocol lda { sendmail_path = /usr/lib/sendmail auth_socket_path = /tmp/auth-master } auth_verbose = no auth_debug = no auth default { mechanisms = plain login passdb pam { } passdb sql { args = /etc/dovecot/dovecot-sql.conf } userdb passwd { } userdb sql { args = /etc/dovecot/dovecot-sql.conf } socket listen { master { path = /tmp/auth-master mode = 0600 user = vmail group = mail } client { path = /tmp/auth-client mode = 0660 user = vmail group = mail } } } dict { } plugin { } ######################################### And Postfix conf: # SASL Authentication smtpd_sasl_auth_enable = yes smtpd_sasl2_auth_enable = yes smtpd_sasl_exceptions_networks = $mynetworks smtpd_sasl_security_options = noanonymous broken_sasl_auth_clients = yes smtpd_sasl_path = private/auth smtpd_sasl_application_name = smtpd smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination permit_mynetworks = 127.0.0.0/8, 184.22.103.88/32, 184.22.103.87/32 Thanks!!! > -----Mensagem Original----- From: Timo Sirainen > Sent: Saturday, January 28, 2012 3:51 PM > To: Gustavo > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] Problem with Postfix + Dovecot + MySQL + > Squirrelmail > > On 27.1.2012, at 14.32, Gustavo wrote: > > #service dovecot start >> Starting IMAP/POP3 mail server: dovecotLast died with error (see error >> log for more information): Auth process died too early - shutting down >> > > No need to keep guessing the problem. "See error log for more information" > like it says. http://wiki.dovecot.org/**Logging > > -- Gustavo - clube03 at gmail.com From tss at iki.fi Sun Feb 12 02:24:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 02:24:38 +0200 Subject: [Dovecot] [PATCH] Allow verification of user's hash in doveadm-pw In-Reply-To: References: Message-ID: <4F3706C6.10909@iki.fi> On 09/01/2011 01:17 AM, Spack wrote: > Hi, > > It may be useful for testing purpose to know if a generated password > hash correspond to a given clear password. This can be useful to check > if a hash generated by another program can be verified by Dovecot > without any errors. > > This patch adds the ability the verify a password hash using `doveadm > pw` via the `-V` option. Committed, with a few changes: http://hg.dovecot.org/dovecot-2.1/rev/fffa4d53e901 > + while ((ch = getopt(argc, argv, "lp:r:s:u:V::")) != -1) { "::" isn't standard, so I added a separate -t parameter to give the hash. From tss at iki.fi Sun Feb 12 05:39:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:39:20 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <4F367C3E.2080806@webfusion.com> References: <4F367C3E.2080806@webfusion.com> Message-ID: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> On 11.2.2012, at 16.33, Mark Zealey wrote: > I'm running Dovecot 2.0.16 just set up the director with cut&paste from the wiki. It's working & running fine for pop/imap connections (as verified by doveadm director status user), however when I specify 'RCPT TO' for a (tcp) lmtp connection straight away I get: > > 451 4.3.0 Temporary user lookup failure > > and in the logs: > 2012-02-11T14:22:54+00:00 xxx dovecot: lmtp(27997): Error: passdb lookup(xxx at yyy.com): Disconnected unexpectedly Aren't there any other errors logged? I think there should be some error logged from either auth or director process. Although director is missing one error, but auth should have also logged an error in that case. Anyway, fixed by http://hg.dovecot.org/dovecot-2.1/rev/9b70d164e974 From tss at iki.fi Sun Feb 12 05:42:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:42:22 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> Message-ID: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> On 11.2.2012, at 12.59, Thomas Leuxner wrote: > Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: > > $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. From tss at iki.fi Sun Feb 12 05:45:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 05:45:45 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> Message-ID: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> On 11.2.2012, at 9.58, Thomas Leuxner wrote: > $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox > dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 From tss at iki.fi Sun Feb 12 06:02:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:02:45 +0200 Subject: [Dovecot] folder hierarchies with migration from cyrus to dovecot2 In-Reply-To: References: Message-ID: <5BD5F73F-E9D6-4803-BA51-59CC7B0D338E@iki.fi> On 10.2.2012, at 22.46, Wolfgang Rosenauer wrote: > With Cyrus I used > unixhierarchysep: yes > and subfolders within Cyrus are always below INBOX. .. > - Can I configure Dovecot in a way that it looks like Cyrus? Having > subfolders below Inbox but let clients like those on Android display > them besides Inbox? namespace { prefix = INBOX/ separator = / } From tss at iki.fi Sun Feb 12 06:06:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:06:58 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> Message-ID: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. From tss at iki.fi Sun Feb 12 06:09:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:09:55 +0200 Subject: [Dovecot] Migration to mdbox from Maildir, missing folders! In-Reply-To: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> References: <60c012a83a6ac9db924bc2c85cf89bb8@www.coochey.net> Message-ID: <3F5BB562-6484-43B0-8914-434694F446DB@iki.fi> On 10.2.2012, at 16.36, Giles Coochey wrote: > So I went and migrated a mailbox from mdbox from maildir format. > All seemed ok, but I've found that I now have some missing folders. > > The mailbox in maildir format had some folders which contained an '&' sign, these appear to have been omitted... What Dovecot version? And give some specifics, like outputs of: ls -a ~/Maildir/ ls ~/mdbox/mailboxes/ or something similar. From tss at iki.fi Sun Feb 12 06:16:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:16:00 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: References: Message-ID: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> On 10.2.2012, at 14.13, Michael Stilkerich wrote: > I have shared one of my mailboxes to another user providing him full rights to > that mailbox. > > $ doveadm acl get -u user1 doc Is "doc" the namespace prefix? > When that user creates a new mailbox within the shared mailbox, the mailbox is successfully > created but with an empty ACL. Creates how? With IMAP client? > According to the wiki, it should inherit the ACL from its parent, i.e. have > full permissions for user2. Works with me. dovecof -n output? > Dovecot logs the following errors when I access the parent folder the first time > after the other user created the child mailbox: > > dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 > dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX > dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent Are you trying to use per-user indexes? That's not going to work. From tss at iki.fi Sun Feb 12 06:19:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 06:19:38 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: Message-ID: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> On 10.2.2012, at 19.39, Alex Baule wrote: > Hy Everyone...and Timo ! > > There is a way to know if a email was expunged (deleted from hard > disk) inside a plugin ? I rewrite the expunge function, but the real > expunge function is void, i can't know if was really expunged. > > There is a way to know this ? It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. From tss at iki.fi Sun Feb 12 07:05:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:05:37 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSN for overquota. In-Reply-To: References: Message-ID: <4F3748A1.3030301@iki.fi> On 12/22/2010 08:08 PM, Mark Sapiro wrote: > It appears that there is some version of the Dovecot LDA that is > sending an RFC 3798 MDN rather than an RFC 3464 DSN to report > non-delivery to a user who is over quota. Fixed, better late than never :) http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 The MDN sending was simply copied from Cyrus. So Dovecot isn't the only server sending this. From tss at iki.fi Sun Feb 12 07:42:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:42:28 +0200 Subject: [Dovecot] v2.1.0 status Message-ID: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. I've gone through all of my old mails. Let me know if you think something is still missing / unfixed in v2.1 hg. Preferably today. Next up: Creating dovecot-2.2 tree with great new features. :) From tss at iki.fi Sun Feb 12 07:45:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 07:45:12 +0200 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> On 12.2.2012, at 7.42, Timo Sirainen wrote: > I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota and Sieve's reject mails now include DSN report instead of MDN report. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From tlx at leuxner.net Sun Feb 12 08:54:41 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:54:41 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: Am 12.02.2012 um 04:42 schrieb Timo Sirainen: > On 11.2.2012, at 12.59, Thomas Leuxner wrote: > >> Some more coffee later revealed doveadm/dsync being picky on the order of parameters. This works: >> >> $ doveadm -o plugin/quota= backup -u tlx at leuxner.net mdbox:/home/tlx/mdbox > > Now you have disabled quota entirely from dsync. I guess it doesn't matter in this case though, since the source mailboxes aren't modified. Anyway, my original command would have worked also, you just modified it.. > Indeed I did as it did not like the syntax: $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= backup: invalid option -- 'o' doveadm backup [-u |-A] [-S ] [-fR] [-m ] -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 08:56:24 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 07:56:24 +0100 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <86DF82B6-7395-4F0D-8726-514CDCAF08ED@iki.fi> Message-ID: <285AA29C-A430-44C5-9E9A-BD93A34291B8@leuxner.net> Am 12.02.2012 um 04:45 schrieb Timo Sirainen: > On 11.2.2012, at 9.58, Thomas Leuxner wrote: > >> $ doveadm backup -u tlx at leuxner.net mdbox:/tmp/tlx/mdbox >> dsync(tlx at leuxner.net): Panic: file dsync-worker-local.c: line 1643 (local_worker_save_msg_continue): assertion failed: (ret == -1) > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/57c735865b19 Thanks. Crash gone. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Sun Feb 12 09:02:22 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 12 Feb 2012 08:02:22 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: Am 01.01.2012 um 10:31 schrieb Thomas Leuxner: > $ doveadm acl get -u tlx at leuxner.net FOO > doveadm(tlx at leuxner.net): Error: Can't open mailbox FOO: Mailbox doesn't exist: FOO > ID Global Rights > $ echo $? > 0 Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From Mark.Zealey at webfusion.com Sun Feb 12 10:23:21 2012 From: Mark.Zealey at webfusion.com (Mark Zealey) Date: Sun, 12 Feb 2012 08:23:21 +0000 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: Ah I should have checked this before - line before: 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Only other relevant piece of config: service auth { user = dovecot client_limit = 15000 } Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. Mark From listen at constabel.net Sun Feb 12 13:18:09 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 12:18:09 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash Message-ID: <20120212111808.GA27677@herakles.constabel-it.de> Hi, If I delete a folder "foo" in Thunderbird, it is moved to trash. [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/ insgesamt 16K drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 8 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:51 dbox-Mails drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 foo [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxx.de/test1/mdbox/mailboxes/Trash/foo/ insgesamt 12K drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 . drwx------ 4 vmail vmail 4,0K 31. Jan 22:16 .. drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 dbox-Mails [root at dovecot1][tmp:exec](~) # ll /mailspool1/xxxx.de/test1/mdbox/mailboxes/Trash/foo/dbox-Mails/ insgesamt 12K drwx------ 2 vmail vmail 4,0K 12. Feb 11:52 . drwx------ 3 vmail vmail 4,0K 31. Jan 22:16 .. -rw------- 1 vmail vmail 224 12. Feb 11:52 dovecot.index.log [root at dovecot1][tmp:exec](~) # If I delete foo from trash, thunderbird removed it from view, but if I restart thunderbird, the folder is still in trash. No error in dovecot log, no error in thunderbird. tcpdump: 156 45.101512 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=353 Ack=1617 Win=10144 Len=0 TSV=2414467591 TSER=251091441 157 45.349600 192.168.10.1 192.168.10.117 IMAP Request: 16 list "" "Trash/*" 158 45.349717 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\HasNoChildren) "/" "Trash/foo" 159 45.349756 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=375 Ack=1727 Win=10144 Len=0 TSV=2414467653 TSER=251091503 160 45.395546 192.168.10.1 192.168.10.117 IMAP Request: 17 list "" "Trash/foo/*" 161 45.395624 192.168.10.117 192.168.10.1 IMAP Response: * LIST (\Noselect \HasNoChildren) "/" "Trash/foo/" 162 45.395662 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=401 Ack=1802 Win=10144 Len=0 TSV=2414467665 TSER=251091515 163 45.439998 192.168.10.1 192.168.10.117 IMAP Request: 18 delete "Trash/foo/" 164 45.440067 192.168.10.117 192.168.10.1 IMAP Response: 18 NO [CANNOT] Invalid mailbox name 165 45.440104 192.168.10.1 192.168.10.117 TCP 54094 > imap [ACK] Seq=425 Ack=1839 Win=10144 Len=0 TSV=2414467676 TSER=251091526 166 45.493978 192.168.10.1 192.168.10.117 IMAP Request: 19 IDLE with dovecot 1.2 and Maildir I can delete folders from trash. Something wrong in my config? Thunderbird-Bug? # 2.1.rc5 (b9e74767cd39): /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-xen-amd64 x86_64 Debian 6.0.4 auth_cache_size = 1 M auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login cram-md5 auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot/dovecot-debug.log dict { expire = sqlite:/etc/dovecot/dovecot-dict-expire.conf.ext } disable_plaintext_auth = no hostname = mx4.xxxx.de info_log_path = /var/log/dovecot/dovecot-info.log log_path = /var/log/dovecot/dovecot.log mail_debug = yes mail_location = mdbox:~/mdbox mail_plugins = quota notify mail_log expire mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Archive/2011 { auto = subscribe } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = scheme=PLAIN username_format=%u /etc/imap.userdb driver = passwd-file } plugin { expire = Trash expire2 = Trash/* expire3 = Spam expire_dict = proxy::expire quota = dict:User quota::file:/mailspool1/%d/%n/dovecot-quota quota_rule = *:storage=9000M:messages=1000000 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at localhost protocols = " imap lmtp sieve pop3" ssl_ca = References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <4F37A343.1060702@localhost.localdomain.org> On 02/12/2012 12:18 PM Mike Constabel wrote: > Hi, > > If I delete a folder "foo" in Thunderbird, it is moved to trash. > ? > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. http://hg.dovecot.org/dovecot-2.1/file/b9e74767cd39/doc/example-config/conf.d/20-imap.conf#l47 Regards, Pascal -- The trapper recommends today: deadbeef.1204312 at localdomain.org From CMarcus at Media-Brokers.com Sun Feb 12 13:33:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:33:11 -0500 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> Message-ID: <4F37A377.80707@Media-Brokers.com> On 2012-02-11 1:46 PM, Gustavo wrote: > Ok, I study the wikki and the dovecot works. But when I try to send a mail > using squirrel, I get this error on logs: > > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from > localhost.localdomain[127.0.0.1] > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: > xsasl_cyrus_server_get_mechanism_list: no applicable SASL mechanisms > Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL authentication > mechanisms Sorry, but this has nothing to do with dovecot, unless your intention is to use dovecot-sasl with your postfix... it appears you are using cyrus-sasl and postfix on your smtp server, that is where your problem lies. Easiest would be to configure postfix to use dovecot-sasl: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL -- Best regards, Charles From mstevens at imt-systems.com Sun Feb 12 13:37:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 12:37:18 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212111808.GA27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> Message-ID: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> On 12.02.2012 12:18, Mike Constabel wrote: > If I delete foo from trash, thunderbird removed it from view, > but if I restart thunderbird, the folder is still in trash. > > No error in dovecot log, no error in thunderbird. protocol imap { ... imap_client_workarounds = tb-extra-mailbox-sep } Best regards, Morten From CMarcus at Media-Brokers.com Sun Feb 12 13:38:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 06:38:11 -0500 Subject: [Dovecot] v2.1.0 status In-Reply-To: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> Message-ID: <4F37A4A3.4000204@Media-Brokers.com> On 2012-02-12 12:42 AM, Timo Sirainen wrote: > Next up: Creating dovecot-2.2 tree with great new features.:) Can't wait to read your follow-up email outlining these 'great new features' you expect to make it into 2.2... Thanks so much for dovecot Timo! I am amazed at how far it has come in the last few years, and apparently things will only get better. -- Best regards, Charles From ms at mike2k.de Sun Feb 12 14:14:03 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 13:14:03 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> Message-ID: <4F37AD0B.9040406@mike2k.de> Hi Timo, On 12-Feb-12 5:16, Timo Sirainen wrote: > On 10.2.2012, at 14.13, Michael Stilkerich wrote: > >> I have shared one of my mailboxes to another user providing him full rights to >> that mailbox. >> >> $ doveadm acl get -u user1 doc > > Is "doc" the namespace prefix? No, 'doc' is the name of the shared parent mailbox under that the 2nd user created the new mailbox 'hallo'. To clarify: user1 shares the mailbox 'doc' with max permissions to user2. user2 then creates a new mailbox under the parent 'doc'. >> When that user creates a new mailbox within the shared mailbox, the mailbox is successfully >> created but with an empty ACL. > > Creates how? With IMAP client? Yes, using Thunderbird / Roundcube Webmail. >> According to the wiki, it should inherit the ACL from its parent, i.e. have >> full permissions for user2. > > Works with me. dovecof -n output? The config is attached. The IMAP-Client is signaled an error by dovecot, but the log file does not contain an error. At the filesystem level - The mailbox was created as a sub folder of 'doc'. - It contains nothing but an empty sub folder 'dbox-Mails'. - It does specifically not contain an dovecot-acl file. - The sharing user's (user1) dovecot-acl-list file was not updated to list the newly created subfolder. - The acl dictionary 'shared-mailboxes' was also not touched, but this is probably correct since it already contains a record that user1 shares mailboxes to user2. - When user1 accesses the new mailbox that user2 created, the below error gets logged and a dovecot.index.log file is created in the until then empty 'dbox-Mails' folder of the new mailbox. >> Dovecot logs the following errors when I access the parent folder the first time >> after the other user created the child mailbox: >> >> dovecot: imap(user1): Error: Mailbox doc/hallo: Corrupted index, uidvalidity=0 >> dovecot: imap(user1): Error: Recent flags state corrupted for mailbox INBOX >> dovecot: imap(user1): Error: /home/dovecot/user1/mailboxes/INBOX/dbox-Mails/dovecot.index reset, view is now inconsistent > > Are you trying to use per-user indexes? That's not going to work. I don't think so. I'm using dbox and thus have not added a custom :INDEX=... to the location. -Michael -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot.conf URL: From listen at constabel.net Sun Feb 12 15:27:19 2012 From: listen at constabel.net (Mike Constabel) Date: Sun, 12 Feb 2012 14:27:19 +0100 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> Message-ID: <20120212132719.GB27677@herakles.constabel-it.de> Hi, On Sun, 12 Feb 2012, Morten Stevens wrote: > On 12.02.2012 12:18, Mike Constabel wrote: > > >If I delete foo from trash, thunderbird removed it from view, > >but if I restart thunderbird, the folder is still in trash. > > > >No error in dovecot log, no error in thunderbird. > > protocol imap { > ... > imap_client_workarounds = tb-extra-mailbox-sep > } Ok, this seems to work, (I thought I tried it before.. but now it works. :) Another Question to this Option: # tb-extra-mailbox-sep: # With mbox storage a mailbox can contain either mails or submailboxes, # but not both. Thunderbird separates these two by forcing server to # accept '/' suffix in mailbox names in subscriptions list. I thought this is not needed because I use mdbox. But mbox here means also mdbox and sdbox? Also "With mbox storage a mailbox can contain either mails or submailboxes, but not both" is only correct for mbox? In mdbox this is possible. This description should contain a litte bit more info for mbox/sdbox/mdbox? Should I use all 3 workarounds if I don't know which clients are in use? LAYOUT=fs and mdbox Regards, Mike From CMarcus at Media-Brokers.com Sun Feb 12 15:43:51 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 08:43:51 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <4F37C217.7040707@Media-Brokers.com> Hi Timo, One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? Anyway here is the thread outlining the problem with dovecot's SIS and backups: http://www.dovecot.org/list/dovecot/2011-December/062340.html Have you given this any more thought? -- Best regards, Charles From forumer at smartmobili.com Sun Feb 12 15:52:54 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 14:52:54 +0100 Subject: [Dovecot] Lost in configuration Message-ID: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Hi, I am trying to configure dovecot/postfix with virtual users and sasl auth but there are so many tutorials with mistakes and subtle differences that at the end I am lost. So I am running Ubuntu server 11.04 (natty) and when I installed my server I had followed the following guide http://workaround.org/ispmail/etch Then I have migrated to dovecot 2.1 and now I am trying to allow authenticated user to use my smtp server because for the moment I get : Relay access denied Here is my configuration (first is file hierarchy followed by file contents) and I would like to know what I need to do to allow authenticated user send email. In this tutorial http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql it seems I have to add some configuration lines inside /etc/pam.d/smtp and add database configuration inside /etc/postfix/sasl/smtpd.conf. But what I find weird is the fact I have to configure some sql queries from smtpd.conf while normally dovecot has already all the information to do this kind of query. So before to keep on modifying my config I would like to be sure I am following the right way. Thanks ########################### /etc/postfix ########################### drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ -rw-r--r-- 1 root root 373 2011-07-24 11:50 dynamicmaps.cf -rw-r--r-- 1 root root 1928 2012-02-12 12:45 main.cf -rw-r--r-- 1 root root 5762 2011-08-03 22:06 master.cf -rw-rw---- 1 root postfix 140 2011-07-24 11:43 mysql-virtual-alias-maps.cf -rw-rw---- 1 root postfix 132 2011-07-24 11:47 mysql-virtual-mailbox-domains.cf -rw-rw---- 1 root postfix 128 2011-07-24 11:47 mysql-virtual-mailbox-maps.cf -rw-r--r-- 1 root root 19509 2011-04-05 06:07 postfix-files -rwxr-xr-x 1 root root 8729 2011-04-05 06:07 postfix-script* -rwxr-xr-x 1 root root 25752 2011-04-05 06:07 post-install* drwxr-xr-x 2 root root 4096 2012-02-12 12:41 sasl/ root at xa-12345:/etc/postfix# ll sasl/ drwxr-xr-x 2 root root 4096 2012-02-12 12:41 ./ drwxr-xr-x 3 root root 4096 2012-02-12 13:33 ../ -rw-r--r-- 1 root root 26 2012-02-12 12:41 smtpd.conf ########################### /etc/dovecot ########################### drwxr-xr-x 3 root root 4096 2011-09-24 11:42 ./ drwxr-xr-x 116 root root 4096 2012-02-12 13:21 ../ drwxr-xr-x 2 root dovecot 4096 2012-02-12 12:03 conf.d/ -rw-r--r-- 1 root dovecot 3693 2011-09-23 15:11 dovecot.conf -rw-r--r-- 1 root dovecot 410 2011-09-23 15:11 dovecot-db.conf.ext -rw-r--r-- 1 root dovecot 782 2011-09-23 15:11 dovecot-dict-sql.conf.ext -rw-r--r-- 1 root dovecot 5508 2011-09-23 15:38 dovecot-sql.conf -rw-r--r-- 1 root dovecot 5348 2011-09-23 15:11 dovecot-sql.conf.ext -rw-r--r-- 1 root dovecot 116 2011-09-23 15:11 README ########################### /etc/default/saslauthd ########################### -rw-r--r-- 1 root root 2043 2012-02-12 11:57 /etc/default/saslauthd ########################### /var/spool/postfix/var/run/ ########################### drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ./ drwxr-xr-x 3 root root 4096 2012-02-12 12:35 ../ drwxr-xr-x 2 root sasl 4096 2012-02-12 12:35 saslauthd/ I have added postfix to the sasl group (don't know if it was necessary but I found this instruction on a website) /etc/group: sasl:x:45:postfix /etc/default/saslauthd: ---------------------- START=yes PWDIR="/var/spool/postfix/var/run/saslauthd" PARAMS="-m ${PWDIR}" PIDFILE="${PWDIR}/saslauthd.pid" DESC="SASL Authentication Daemon" NAME="saslauthd" MECHANISMS="pam" MECH_OPTIONS="" THREADS=5 OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd" mkdir -p /var/spool/postfix/var/run/saslauthd chown -R root.sasl /var/spool/postfix/var/run/saslauthd Then I have checked saslauth is running after restart: root at xa-12345:/home/vmail# ps -ef | grep saslauthd root 6123 1 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6124 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6125 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6127 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 root 6128 6123 0 13:21 ? 00:00:00 /usr/sbin/saslauthd -a pam -c -m /var/run/saslauthd -n 5 Then when I check after restarting if saslauthd socket is created I don't see anything (don't know if it's normal) root at xa-12345:/home/vmail# ls -lh /var/spool/postfix/var/run/saslauthd total 0 /etc/postfix/sasl/smtpd.conf: ----------------------------- pwcheck_method: saslauthd /etc/postfix/main.cf: --------------------- myorigin = /etc/mailname smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no # TLS parameters smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key smtpd_use_tls=yes smtpd_tls_session_cache_database = btree:${queue_directory}/smtpd_scache smtp_tls_session_cache_database = btree:${queue_directory}/smtp_scache myhostname = mail.foobar.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = foobar.com mydestination = xa-12345.dadibox.com, localhost.dadibox.com, localhost relayhost = mynetworks = 127.0.0.0/8 80.190.190.190 mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all #virtual domains virtual_mailbox_domains = mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf virtual_mailbox_base = /home/vmail virtual_mailbox_maps = mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf virtual_uid_maps = static:5000 virtual_gid_maps = static:5000 virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf # Dovecot LDA virtual_transport = dovecot dovecot_destination_recipient_limit = 1 debug_peer_level = 2 debug_peer_list = 127.0.0.1 smtpd_sasl_type = dovecot smtpd_sasl_path = private/auth smtpd_sasl_auth_enable = yes smtpd_recipient_restrictions = permit_mynetworks,permit_sasl_authenticated,reject_unauth_destination smtpd_sasl_application_name = smtpd broken_sasl_auth_clients = yes /etc/dovecot/conf.d/10-auth.conf: -------------------------------------- disable_plaintext_auth = no !include auth-sql.conf.ext !include auth-static.conf.ext /etc/dovecot/conf.d/auth-static.conf.ext: ---------------------------------------- userdb { driver = static args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } /etc/dovecot/conf.d/auth-sql.conf.ext: -------------------------------------- passdb { driver = sql args = /etc/dovecot/dovecot-sql.conf } /etc/dovecot/dovecot-sql.conf: ------------------------------ driver = mysql connect = host=127.0.0.1 dbname=mailserver user=myuser password=mypassword default_pass_scheme = CRAM-MD5 password_query = SELECT email as user, password FROM view_users WHERE email='%u'; /etc/dovecot/conf.d/10-master.conf: ---------------------------------- service imap-login { inet_listener imap { } inet_listener imaps { } } service pop3-login { inet_listener pop3 { } inet_listener pop3s { } } service lmtp { unix_listener lmtp { } } service imap { } service pop3 { } service auth { unix_listener auth-userdb { mode = 0600 user = vmail #group = } # Postfix smtp-auth unix_listener /var/spool/postfix/private/auth { mode = 0777 } } service auth-worker { } service dict { unix_listener dict { mode = 0600 #user = group = vmail } } From forumer at smartmobili.com Sun Feb 12 16:37:26 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Sun, 12 Feb 2012 15:37:26 +0100 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <321c6c9323618a11b96bccf49ac4d135@smartmobili.com> IN my previous email ignore the information about the missing socket because it's actually created -rw------- 1 root root 5 2012-02-12 15:28 saslauthd.pid From jom at grosjo.net Sun Feb 12 16:43:26 2012 From: jom at grosjo.net (Joan Moreau) Date: Sun, 12 Feb 2012 14:43:26 +0000 Subject: [Dovecot] v2.1.0 status In-Reply-To: <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> Message-ID: <258ebed086b260ca88432119bd0bb042@grosjo.net> Thanks for 2.1.0 Timo Looking forward to beta-test 2.2 :) Le 12/02/2012 05:45, Timo Sirainen a ?crit : > On 12.2.2012, at 7.42, Timo Sirainen wrote: > >> I'll release v2.1.rc6 in maybe 16 hours or so. If no problems are reported in it after a few days I'll just change version to 2.1.0. So now's a great time to try the latest v2.1 hg version to see if there are any problems. > > Oh, and if you want to try out some new things in v2.1.rc6, here's the biggest changes so far: > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota and Sieve's reject mails now include DSN report > instead of MDN report. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. From alexwbaule at gmail.com Sun Feb 12 16:52:05 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 12:52:05 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: I want to know because i have header and body splited. To erase the body, i must have shure, that header was expunged. Some tests that i made, the client call expunge, but i don't know why (there is notting in mail log) the header was not expunged, nut my body was, because i call it before expunge the header. something like this. if (found_body){ erase_body(); super.expunge(_mail); } Because that expunge that no happen, (the client call, but the email was not expunged) i think to do like this way: if (found_body){ super.expunge(_mail); if(_mail was expunged){ erase_body(); } } TKs Timo ! Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > On 10.2.2012, at 19.39, Alex Baule wrote: > >> Hy Everyone...and Timo ! >> >> There is a way to know if a email was expunged (deleted from hard >> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> expunge function is void, i can't know if was really expunged. >> >> There is a way to know this ? > > It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. > > If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). > > Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. > From tss at iki.fi Sun Feb 12 17:42:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:42:20 +0200 Subject: [Dovecot] Quota Calculation seems to be wrong when using dsync In-Reply-To: References: <0A9215FE-9BE5-4EBC-B568-79F67A07C8DC@leuxner.net> <1328811898.11500.157.camel@innu> <02FB218A-ACFD-4B83-97DE-CB84BC600E92@leuxner.net> <4B2E580C-A27C-4321-91F0-C45D1DB57FF3@iki.fi> Message-ID: <52A730C6-230B-42B6-8808-A9F6AE3A44D5@iki.fi> On 12.2.2012, at 8.54, Thomas Leuxner wrote: > Indeed I did as it did not like the syntax: > > $ dsync -u tlx at leuxner.net backup dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota= > backup: invalid option -- 'o' > doveadm backup [-u |-A] [-S ] [-fR] [-m ] Oh. Hm. I'm not exactly sure why it's saying that, but adding some quotes fixes that: dsync -u tlx at leuxner.net backup 'dsync -o mail=mdbox:/home/backups/tlx/mdbox -o plugin/quota=' From rob0 at gmx.co.uk Sun Feb 12 17:42:39 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 12 Feb 2012 09:42:39 -0600 Subject: [Dovecot] Lost in configuration In-Reply-To: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> References: <8c1a2b932b0489daef744270bb32f44f@smartmobili.com> Message-ID: <20120212154239.GG14195@harrier.slackbuilds.org> On Sun, Feb 12, 2012 at 02:52:54PM +0100, forumer at smartmobili.com wrote: > I am trying to configure dovecot/postfix with virtual users and > sasl auth but there are so many tutorials with mistakes and subtle > differences that at the end I am lost. All the more reason to stick to each project's official documentation. The sad reality is that most such tutorials you will find are written by people who should not be writing documentation. > So I am running Ubuntu server 11.04 (natty) and when I installed my > server I had followed the following guide > http://workaround.org/ispmail/etch One of the better ones; I know the author, and he is reasonably competent. But are you sure you need and "ISP-style" mail server? Perhaps something simpler makes sense to begin with. http://www.postfix.org/BASIC_CONFIGURATION_README.html > Then I have migrated to dovecot 2.1 and now I am trying to allow > authenticated user to use my smtp server because for the moment I > get : Relay access denied A Postfix issue, or possibly somewhat on topic here if you are using Dovecot SASL. You do not seem to know whether you are or not. This might be because you are mixing and confusing different tutorials without understanding what they are doing and why. > Here is my configuration (first is file hierarchy followed by > file contents) and I would like to know what I need to do to > allow authenticated user send email. http://www.postfix.org/SASL_README.html#server_dovecot http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > In this tutorial > http://library.linode.com/email/postfix/dovecot-mysql-ubuntu-10.04-lucid#sph_configure-saslauthd-to-use-mysql I have not reviewed this one, but the anchor title is patently absurd. If you are using Postfix and Dovecot, there is NEVER any reason to use Cyrus SASL's saslauthd. Consider that one useless. The rest of this is not worth answering. You went off on a Cyrus tangent which is pointless (and never on topic for this list!) Start over with a better understanding of what you need. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From tss at iki.fi Sun Feb 12 17:46:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:46:12 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> Message-ID: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Hi, Yeah, you shouldn't do the erasing directly in expunge(), because it may still be aborted. Do it in sync_notify() like quota plugin does. On 12.2.2012, at 16.52, Alex Baule wrote: > I want to know because i have header and body splited. To erase the > body, i must have shure, that header was expunged. Some tests that i > made, the client call expunge, but i don't know why (there is notting > in mail log) the header was not expunged, nut my body was, because i > call it before expunge the header. something like this. > > if (found_body){ > erase_body(); > super.expunge(_mail); > } > > Because that expunge that no happen, (the client call, but the email > was not expunged) i think to do like this way: > > if (found_body){ > super.expunge(_mail); > if(_mail was expunged){ > erase_body(); > } > } > > TKs Timo ! > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >>> Hy Everyone...and Timo ! >>> >>> There is a way to know if a email was expunged (deleted from hard >>> disk) inside a plugin ? I rewrite the expunge function, but the real >>> expunge function is void, i can't know if was really expunged. >>> >>> There is a way to know this ? >> >> It's not known until transaction_commit() what messages are expunged. What do you need to know this for? There are a few different ways. >> >> If you simply need one of the sessions to definitely know that it expunged a message, you can do it like quota plugin does. (So even if there are multiple clients doing EXPUNGE at the same time, quota is never decreased more than once per mail.) See quota-storage.c and most importantly quota_mailbox_sync_notify(). >> >> Quota code also shows the other method of doing it: keep track of what mail_expunge()s have been called, and then in transaction commit check if it succeeds and if it does do what you want to do. >> > From tss at iki.fi Sun Feb 12 17:58:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 17:58:20 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <4F37C217.7040707@Media-Brokers.com> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: On 12.2.2012, at 15.43, Charles Marcus wrote: > One thing that concerns me right now about my main clients site that I will soon be migrating from courier-imap to dovecot is the message/thread a while back about the problems with SIS and backups. I'm planning on creating a new "backup" mail storage backend, where you could basically run: doveadm backup -u user at domain backup: And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. Then for restoring you can use the same input for "doveadm import". So in the backups there would be no deduplication, unless implemented by the backup software. > I am trying to decide between using dovecot's SIS, or using mimedefang's ability to replace attachments with hyperlinks to them (storing them on a webserver - still not clear on exactly how it works or the security implications of doing so). It looks like using mimedefang would eliminate the problem with backups, and would provide the added benefit of eliminating large file attachments for external recipients as well, however I'm concerned about security - ie, how do you limit access to these attachments to only the recipients of the message with the hyperlink in it? As long as the URL is random enough it shouldn't be possible to guess the links. But if they're HTML mails and such then hyperlinks would send the referrer-url to the next site. I guess you could also add HTTP authentication. From tss at iki.fi Sun Feb 12 18:02:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 18:02:39 +0200 Subject: [Dovecot] Passdb disconnected unexpectedly when trying to do Director with LMTP In-Reply-To: References: <4F367C3E.2080806@webfusion.com>, <79A67FC6-7BC4-483F-83F7-D9841B275E69@iki.fi> Message-ID: On 12.2.2012, at 10.23, Mark Zealey wrote: > Ah I should have checked this before - line before: > > 2012-02-11T14:22:54+00:00 xx dovecot: director: Error: connect(/var/run/dovecot//auth-userdb) failed: Permission denied > > This is because the director is running as user dovecot; but that file is owned root.root permissions 0600. What would you suggest doing? Setting that socket 0660 is the obvious solution but perhaps there is a more secure way you would recommend eg chown dovecot? I'm guessing this would also need updating in the documentation as afaik our setup should be pretty standard. Yeah, the auth-userdb socket default permissions aren't usable pretty much anywhere. In v2.1 it's better, but still not as good as they could be. I'm not sure what the best solution for this is. > Also (perhaps this should be a new thread) but I'm not too sure the director service would work for us. I was doing some testing yesterday & discovered that when I restart a dovecot instance it doesn't have any persistence (eg resync the lookup table with the other directors or do a save/restore from disk) so you are basically starting from scratch again which would be very bad in our setup. You mean if you restart one director some of the state is lost in all of them? I think v2.0 had some bugs related to this, but they should be fixed in v2.1. If you mean that if you stop all of the directors and state is lost when restarting then yes, that does happen. You can run "doveadm director dump" to get the current state and after a restart run its output to get back to the same state. What state specifically are you talking about anyway? From CMarcus at Media-Brokers.com Sun Feb 12 18:15:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 12 Feb 2012 11:15:42 -0500 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <4F37E5AE.8060104@Media-Brokers.com> On 2012-02-12 10:58 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.43, Charles Marcus wrote: >> > I'm planning on creating a new "backup" mail storage backend, where > you could basically run: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). > So it would be similar to e.g. PostgreSQL's pg_dump. Then for > restoring you can use the same input for "doveadm import". So in the > backups there would be no deduplication, unless implemented by the > backup software. Hmmm, ok thanks... loss of dedupe on the backup server may or may not be an issue I'll be concerned about, but good to know that there is an option at least. >> It looks like using mimedefang would eliminate the problem with >> backups, and would provide the added benefit of eliminating large >> file attachments for external recipients as well, however I'm >> concerned about security - ie, how do you limit access to these >> attachments to only the recipients of the message with the >> hyperlink in it? > As long as the URL is random enough it shouldn't be possible to guess > the links. But if they're HTML mails and such then hyperlinks would > send the referrer-url to the next site. I guess you could also add > HTTP authentication. Thanks for that too... I'll have to a bit of research on this, because the more I think about it, the more I like the idea, *if* it can be done securely enough... -- Best regards, Charles From alexwbaule at gmail.com Sun Feb 12 18:48:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Sun, 12 Feb 2012 14:48:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Tks timo ! I will see the plugin to do like it! Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > Hi, > > Yeah, you shouldn't do the erasing directly in expunge(), because it may > still be aborted. Do it in sync_notify() like quota plugin does. > > On 12.2.2012, at 16.52, Alex Baule wrote: > > > I want to know because i have header and body splited. To erase the > > body, i must have shure, that header was expunged. Some tests that i > > made, the client call expunge, but i don't know why (there is notting > > in mail log) the header was not expunged, nut my body was, because i > > call it before expunge the header. something like this. > > > > if (found_body){ > > erase_body(); > > super.expunge(_mail); > > } > > > > Because that expunge that no happen, (the client call, but the email > > was not expunged) i think to do like this way: > > > > if (found_body){ > > super.expunge(_mail); > > if(_mail was expunged){ > > erase_body(); > > } > > } > > > > TKs Timo ! > > > > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: > >> On 10.2.2012, at 19.39, Alex Baule wrote: > >> > >>> Hy Everyone...and Timo ! > >>> > >>> There is a way to know if a email was expunged (deleted from hard > >>> disk) inside a plugin ? I rewrite the expunge function, but the real > >>> expunge function is void, i can't know if was really expunged. > >>> > >>> There is a way to know this ? > >> > >> It's not known until transaction_commit() what messages are expunged. > What do you need to know this for? There are a few different ways. > >> > >> If you simply need one of the sessions to definitely know that it > expunged a message, you can do it like quota plugin does. (So even if there > are multiple clients doing EXPUNGE at the same time, quota is never > decreased more than once per mail.) See quota-storage.c and most > importantly quota_mailbox_sync_notify(). > >> > >> Quota code also shows the other method of doing it: keep track of what > mail_expunge()s have been called, and then in transaction commit check if > it succeeds and if it does do what you want to do. > >> > > > > From mark at msapiro.net Sun Feb 12 18:57:36 2012 From: mark at msapiro.net (Mark Sapiro) Date: Sun, 12 Feb 2012 08:57:36 -0800 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: <4F3748A1.3030301@iki.fi> Message-ID: Timo Sirainen wrote: >On 12/22/2010 08:08 PM, Mark Sapiro wrote: >> It appears that there is some version of the Dovecot LDA that is >> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >> non-delivery to a user who is over quota. > >Fixed, better late than never :) >http://hg.dovecot.org/dovecot-2.1/rev/264821ba38a8 >http://hg.dovecot.org/dovecot-2.1/rev/ee060d756630 Thanks. >The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >server sending this. :( -- Mark Sapiro The highway is for gamblers, San Francisco Bay Area, California better use your sense - B. Dylan From tss at iki.fi Sun Feb 12 19:00:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:00:45 +0200 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <20120212132719.GB27677@herakles.constabel-it.de> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> Message-ID: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> On 12.2.2012, at 15.27, Mike Constabel wrote: > Another Question to this Option: > > # tb-extra-mailbox-sep: > # With mbox storage a mailbox can contain either mails or submailboxes, > # but not both. Thunderbird separates these two by forcing server to > # accept '/' suffix in mailbox names in subscriptions list. > > > I thought this is not needed because I use mdbox. But mbox here > means also mdbox and sdbox? Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > Should I use all 3 workarounds if I don't know which clients are > in use? I don't know if delay-newmail is still relevant. From tss at iki.fi Sun Feb 12 19:03:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:03:10 +0200 Subject: [Dovecot] Some Dovecot LDA version sends MDN rather than DSNfor overquota. In-Reply-To: References: Message-ID: <83BE4415-FA94-4F23-9DB4-7F59F71D245B@iki.fi> On 12.2.2012, at 18.57, Mark Sapiro wrote: >> On 12/22/2010 08:08 PM, Mark Sapiro wrote: >>> It appears that there is some version of the Dovecot LDA that is >>> sending an RFC 3798 MDN rather than an RFC 3464 DSN to report >>> non-delivery to a user who is over quota. >> >> Fixed, better late than never :) >> The MDN sending was simply copied from Cyrus. So Dovecot isn't the only >> server sending this. > > :( It was actually a bit more complex than I thought. Sieve "reject" is supposed to send MDN (RFC 5228), it's only the out-of-quota error that was supposed to send DSN. So now Dovecot can send both: http://hg.dovecot.org/dovecot-2.1/rev/ad5298ba3229 From tss at iki.fi Sun Feb 12 19:17:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 19:17:30 +0200 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <4F37AD0B.9040406@mike2k.de> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> Message-ID: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f From ms at mike2k.de Sun Feb 12 20:34:31 2012 From: ms at mike2k.de (Michael Stilkerich) Date: Sun, 12 Feb 2012 19:34:31 +0100 Subject: [Dovecot] Inherited ACLs for mailboxes created by not-owner In-Reply-To: <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> References: <5A332375-E451-48BD-97BE-FEA591CF0E1A@iki.fi> <4F37AD0B.9040406@mike2k.de> <97353D41-E471-41E5-8D41-A6D0DBFB7453@iki.fi> Message-ID: <4F380637.30809@mike2k.de> On 12-Feb-12 18:17, Timo Sirainen wrote: > This should fix all your problems: http://hg.dovecot.org/dovecot-2.1/rev/ccad37bc242f Thanks Timo, it does :) -Michael From tss at iki.fi Sun Feb 12 21:11:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 21:11:44 +0200 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: On 12.2.2012, at 9.02, Thomas Leuxner wrote: > Can the exit codes be revisited before rolling the final 2.1? Still not very nice/useful for scripting. Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 From tss at iki.fi Sun Feb 12 23:01:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:01:10 +0200 Subject: [Dovecot] v2.1.rc6 released Message-ID: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. * Added automatic mountpoint tracking and doveadm mount commands to manage the list. If a mountpoint is unmounted, error handling is done by assuming that the files are only temporarily lost. This is especially helpful if dbox alt storage becomes unmounted. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * LDA's out-of-quota mails now include DSN report instead of MDN. + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + doveadm log errors shows the last 1000 warnings and errors since Dovecot was started. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + doveadm mailbox delete: Added -r parameter to delete recursively + doveadm acl: Added "add" and "remove" commands. + Updated to Unicode v6.1 - mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. - dbox: Fixed error handling when saving failed or was aborted - IMAP: Using COMPRESS extension may have caused assert-crashes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - dsync: Fixed handling non-ASCII characters in mailbox names. From javierdemiguel at us.es Sun Feb 12 23:38:58 2012 From: javierdemiguel at us.es (=?UTF-8?Q?Javier_Miguel_Rodr=C3=ADguez?=) Date: Sun, 12 Feb 2012 22:38:58 +0100 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Hello Reading 2.1rc6 changelog I see this: mdbox: When saving to alt storage, Dovecot didn't append as much data to m.* files as it could have. Could you elaborate more on this? Has been ported to Dovecot 2.0? Regards Javier On Sun, 12 Feb 2012 23:01:10 +0200, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [1] > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [2] > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > > I'll also create dovecot-2.2 hg repository today and add some pending patches from Stephan there and start doing some early spring cleaning in there. :) > > Since v2.1.rc5 there have been lots of small fixes and logging improvements, but I also did a few bigger things since they really had to be done soon and I didn't want v2.2.0 release to be only a few months after v2.1.0 with barely any new features. > > * Added automatic mountpoint tracking and doveadm mount commands to > manage the list. If a mountpoint is unmounted, error handling is > done by assuming that the files are only temporarily lost. This is > especially helpful if dbox alt storage becomes unmounted. > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. > * LDA's out-of-quota mails now include DSN report instead of MDN. > > + LDAP: Allow building passdb/userdb extra fields from multiple LDAP > attributes by using %{ldap:attributeName} variables in the template. > + doveadm log errors shows the last 1000 warnings and errors since > Dovecot was started. > + Improved multi-instance support: Track automatically which instances > are started up and manage the list with doveadm instance commands. > All Dovecot commands now support -i parameter to > select the instance (instead of having to use -c ). > See instance_name setting. > + doveadm mailbox delete: Added -r parameter to delete recursively > + doveadm acl: Added "add" and "remove" commands. > + Updated to Unicode v6.1 > - mdbox: When saving to alt storage, Dovecot didn't append as much > data to m.* files as it could have. > - dbox: Fixed error handling when saving failed or was aborted > - IMAP: Using COMPRESS extension may have caused assert-crashes > - IMAP: THREAD REFS sometimes returned invalid (0) nodes. > - dsync: Fixed handling non-ASCII characters in mailbox names. > > _______________________________________________ > Dovecot-news mailing list > Dovecot-news at dovecot.org [3] > http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news [4] Links: ------ [1] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz [2] http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig [3] mailto:Dovecot-news at dovecot.org [4] http://dovecot.org/cgi-bin/mailman/listinfo/dovecot-news From tss at iki.fi Sun Feb 12 23:42:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:42:17 +0200 Subject: [Dovecot] Question about mdbox alt storage in Dovecot 2.0 In-Reply-To: <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <1c4ff7dd9b7a1c1b96022e4d6ce97c37@us.es> Message-ID: <78639407-65C6-4D89-9F8C-2E27041422F2@iki.fi> On 12.2.2012, at 23.38, Javier Miguel Rodr?guez wrote: > Reading 2.1rc6 changelog I see this: > > mdbox: When saving > to alt storage, Dovecot didn't append as much > data to m.* files as it > could have. > > Could you elaborate more on this? It means that if you had e.g. mdbox_rotate_size = 2M, the files in alt storage probably were often much smaller than 2 MB. > Has been ported to Dovecot 2.0? No. You could try how well this applies to it: http://hg.dovecot.org/dovecot-2.1/rev/6881d8c59d9f From tss at iki.fi Sun Feb 12 23:57:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 12 Feb 2012 23:57:29 +0200 Subject: [Dovecot] v2.0.18 released Message-ID: <881CA667-9F0C-4C32-9A61-87409F7DE773@iki.fi> http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz http://dovecot.org/releases/2.0/dovecot-2.0.18.tar.gz.sig + DIGEST-MD5 authentication supports authorization id now. Patch by Yubao Liu + Added instance_name setting which is used to prefix Dovecot processes in ps output. - LDA/LMTP: Sending a large mail via submission_host or via LMTP proxy may have caused a hang. - Fixed dbox + mail_attachment_dir + zlib problems. - Login processes weren't logging all intended messages with auth_verbose=yes - IMAP: THREAD REFS sometimes returned invalid (0) nodes. - IMAP: CONTEXT search return option wasn't handled at all. - dbox: Various error handling fixes. - snarf plugin: Keep the mailbox locked during snarfing to avoid duplicates. From mstevens at imt-systems.com Mon Feb 13 00:32:36 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 12 Feb 2012 23:32:36 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole Message-ID: Hi, It is probably a problem with pigeonhole 0.2.5. ... make[3]: *** [managesieve-proxy.o] Error 1 make[3]: *** Waiting for unfinished jobs.... Here is my build log: http://oss.imt-systems.com/mstevens/temp/build.log Someone an idea? Best regards, Morten From tss at iki.fi Mon Feb 13 00:43:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 00:43:26 +0200 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: Message-ID: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> On 13.2.2012, at 0.32, Morten Stevens wrote: > It is probably a problem with pigeonhole 0.2.5. Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. From interfasys at gmail.com Mon Feb 13 02:02:29 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Mon, 13 Feb 2012 00:02:29 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients Message-ID: <4F385315.5010600@gmail.com> Hello Pascal, I've just tested rc6 and get almost the same results. imap still crashes, but Horde still works. I forgot to mention that this is on FreeBSD 8 using gcc 4.6 and LTO. And contrary to what I said in my initial email, it doesn't seem to be ACL plugin related. imap crashes even if it's not in the plugins list. Here is the backtrace: # gdb /usr/libexec/dovecot/imap /path/to/core Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104b4bf6 in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x0000000000416b99 in cmd_list_continue.6853 () No symbol table info available. #3 0x000000000042166d in cmd_list_full () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x8080808080808080 in ?? () No symbol table info available. #6 0x0000000000000004 in ?? () No symbol table info available. #7 0x0000000010a3f1a8 in ?? () No symbol table info available. #8 0x0000000010aa4390 in ?? () No symbol table info available. #9 0x0000000010a3f174 in ?? () No symbol table info available. #10 0x00007fffffffe9a0 in ?? () No symbol table info available. #11 0x0000000010596e4f in p_strdup () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x0000000010a22200 in ?? () No symbol table info available. #13 0x0000000010aa4458 in ?? () No symbol table info available. #14 0x0000000010aa4340 in ?? () No symbol table info available. #15 0x0000000010aa4498 in ?? () No symbol table info available. #16 0x0000000110a3f040 in ?? () No symbol table info available. #17 0x0000000010a3f128 in ?? () No symbol table info available. #18 0x0000000010a3e840 in ?? () No symbol table info available. #19 0x0000000010a3f040 in ?? () No symbol table info available. #20 0x0000000010a22200 in ?? () No symbol table info available. #21 0x0000000000000001 in ?? () No symbol table info available. #22 0x0000000010a22200 in ?? () No symbol table info available. #23 0x0000000000000000 in ?? () No symbol table info available. #24 0x000000000041d320 in client_command_input.12433 () No symbol table info available. #25 0x00000000004203ca in client_handle_input () No symbol table info available. #26 0x0000000010a574e0 in ?? () No symbol table info available. #27 0x00000000105ae219 in i_stream_read () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #28 0x000000000041d13f in client_input () No symbol table info available. #29 0x00000000105ce276 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #30 0x00000000105c9ea3 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #31 0x00000000105c9f78 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #32 0x00000000105c9fb3 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #33 0x00000000004158bb in main () No symbol table info available. Regards, Olivier > On 01/18/2012 02:03 AM interfaSys s?rl wrote: > >/ Hello, > />/ > />/ I've just noticed that when Horde is connecting to Dovecot 2.1, it > />/ crashes the imap service if Dovecot is configured to use the ACL plugin. > />/ I'm not sure what's so special about the command Horde sends, but it > />/ shouldn't make Dovecot crash. Everything is fine when using Thunderbird. > />/ > />/ Here is the message in Dovecot's logs > />/ "Fatal: master: service(imap): child 89974 killed with signal 11 (core > />/ not dumped)" > />/ > />/ The message says that the core is not dumped, even though I did add > />/ drop_priv_before_exec=yes to my config file. > / > dovecot stop > ulimit -c unlimited > dovecot > > Now connect with Horde and let it crash. > > >/ I've tried connecting to the pid using gdb, but the process just hangs > />/ as soon as I'm connected. > />/ > / > continue > [wait for the crash] > bt full > detach > quit > > > Regards, > Pascal From tss at iki.fi Mon Feb 13 02:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 02:49:07 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F385315.5010600@gmail.com> References: <4F385315.5010600@gmail.com> Message-ID: On 13.2.2012, at 2.02, interfaSys s?rl wrote: > #0 0x00000000105c64e7 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104b4bf6 in mailbox_list_iter_next () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. > #2 0x0000000000416b99 in cmd_list_continue.6853 () > No symbol table info available. > #3 0x000000000042166d in cmd_list_full () This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. From bradley.giesbrecht at gmail.com Mon Feb 13 02:51:55 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Sun, 12 Feb 2012 16:51:55 -0800 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > On 13.2.2012, at 0.32, Morten Stevens wrote: > >> It is probably a problem with pigeonhole 0.2.5. > > > Yeah, unfortunately I had to break it to fix a bug.. Although I guess if I really wanted to I could have worked around it also. Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes it. Confirmed. Thanks. From janfrode at tanso.net Mon Feb 13 09:49:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 08:49:54 +0100 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> Message-ID: <20120213074954.GA13267@dibs.tanso.net> On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: > > doveadm backup -u user at domain backup: > > And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. So only full backups, no incremental backups? Then what's the benefit over just copying the files (of a snapshot)? -jf From tlx at leuxner.net Mon Feb 13 09:57:46 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 13 Feb 2012 08:57:46 +0100 Subject: [Dovecot] Some Doveadm Tools lack proper exit codes In-Reply-To: References: <172CBBB1-DEFD-42E6-937E-B625FB9028EF@leuxner.net> Message-ID: <20120213075746.GA28586@nihlus.leuxner.net> On Sun, Feb 12, 2012 at 09:11:44PM +0200, Timo Sirainen wrote: > Well, okay, it became a bit large change but I guess it's better now than waiting for v2.2 or breaking someone's script by changing it during v2.1.x releases. > > http://hg.dovecot.org/dovecot-2.1/rev/0a5951b08478 Thanks. From fabian at eideo.de Mon Feb 13 10:00:25 2012 From: fabian at eideo.de (Kuran, Fabian) Date: Mon, 13 Feb 2012 08:00:25 +0000 Subject: [Dovecot] Problems sending email direct into publich folders Message-ID: <20120213080027.1CB5B214001@mail.eideo.de> Hi, >?Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent?> to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a?> sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder.? Is there a comparable way to do this with Dovecot? FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. -- Bye, Fabian Kuran From mstevens at imt-systems.com Mon Feb 13 12:11:30 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 13 Feb 2012 11:11:30 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> Message-ID: <493530c2afa3ced4228a0014ac495433@imt-systems.com> On 13.02.2012 01:51, Bradley Giesbrecht wrote: > On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: > >> On 13.2.2012, at 0.32, Morten Stevens wrote: >> >>> It is probably a problem with pigeonhole 0.2.5. >> >> >> Yeah, unfortunately I had to break it to fix a bug.. Although I >> guess if I really wanted to I could have worked around it also. >> Anyway, http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 >> fixes it. > > Confirmed. Thanks. Okay, also confirmed. Thank you. Best regards, Morten From stephan at rename-it.nl Mon Feb 13 12:17:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 11:17:23 +0100 Subject: [Dovecot] Dovecot 2.0.18 build error with pigeonhole In-Reply-To: <493530c2afa3ced4228a0014ac495433@imt-systems.com> References: <876C571E-F645-4115-8FFE-67A9133637AA@iki.fi> <493530c2afa3ced4228a0014ac495433@imt-systems.com> Message-ID: <4F38E333.7030705@rename-it.nl> Op 2/13/2012 11:11 AM, Morten Stevens schreef: > On 13.02.2012 01:51, Bradley Giesbrecht wrote: >> On Feb 12, 2012, at 2:43 PM, Timo Sirainen wrote: >> >>> On 13.2.2012, at 0.32, Morten Stevens wrote: >>> >>>> It is probably a problem with pigeonhole 0.2.5. >>> >>> >>> Yeah, unfortunately I had to break it to fix a bug.. Although I >>> guess if I really wanted to I could have worked around it also. >>> Anyway, >>> http://hg.rename-it.nl/dovecot-2.0-pigeonhole/rev/58fc2f01c432 fixes >>> it. >> >> Confirmed. Thanks. > > Okay, also confirmed. Release will follow later today. Regards, Stephan. From tss at iki.fi Mon Feb 13 12:36:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:36:47 +0200 Subject: [Dovecot] Issues with SIS and Backups - was Re: v2.1.0 status In-Reply-To: <20120213074954.GA13267@dibs.tanso.net> References: <27A01FDE-0AFB-4510-9402-FF32C774E602@iki.fi> <8B8E97B4-EC9F-4B52-8A55-AD37B28DD5A1@iki.fi> <4F37C217.7040707@Media-Brokers.com> <20120213074954.GA13267@dibs.tanso.net> Message-ID: On 13.2.2012, at 9.49, Jan-Frode Myklebust wrote: > On Sun, Feb 12, 2012 at 05:58:20PM +0200, Timo Sirainen wrote: >> >> doveadm backup -u user at domain backup: >> >> And it would output the user's messages to stdout (or to some file). So it would be similar to e.g. PostgreSQL's pg_dump. > > So only full backups, no incremental backups? I know people are doing incremental backups with PostgreSQL, so it should be possible with Dovecot too. I'm not sure about the details yet tough. > Then what's the benefit over just copying the files (of a snapshot)? If SIS is enabled, it's a little difficult to restore specific user's mails from a filesystem based backup. From tss at iki.fi Mon Feb 13 12:39:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 12:39:03 +0200 Subject: [Dovecot] Problems sending email direct into publich folders In-Reply-To: <20120213080027.1CB5B214001@mail.eideo.de> References: <20120213080027.1CB5B214001@mail.eideo.de> Message-ID: <2F6B2448-017D-4056-9AA6-65BB03B64593@iki.fi> On 13.2.2012, at 10.00, Kuran, Fabian wrote: >> Depends on how you want to do this.. For example all mails intended to be put to public namespace could be sent > to a "publicuser" named user, which has write permissions to the public namespace. Then you'll simply create a > sieve script for the publicuser which redirects the mails to the wanted folder (e.g. fileinto "public/hrztest"). > I'm also migrating from cyrus-imapd to dovecot. But this solution isn't very good to me. With cyrus I could easily send an email to (for example) bb+shared/archive at mydomain.com. Then the mail will be putted to the public folder. > > Is there a comparable way to do this with Dovecot? > FYI: I'm using dovecot in a clustered and multidomain setup. The Mails will be deliviered over LMTP. You can set: lmtp_save_to_detail_mailbox = yes And I think then you should be able to send the mail to any user who has write access to it. From tss at iki.fi Mon Feb 13 13:47:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 13:47:06 +0200 Subject: [Dovecot] Dovecot v2.2 plans Message-ID: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: - 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. - 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) - 3. priority: Things other people want to get implemented. There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. dsync-based replication ----------------------- I'll write a separate post about this later. Besides, it's coming for Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. Shared mailbox improvements --------------------------- Support for private flags for all mailbox formats: namespace { type = public prefix = Public/ mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public } - dsync needs to be able to replicate the private flags as well as shared flags. - might as well add a common way for all mailbox formats to specify which flags are shared and which aren't. $controldir/dovecot-flags would say which is the default (private or shared) and what flags/keywords are the opposite. - easy way to configure shared mailboxes to be accessed via imapc backend, which would allow easy shared mailbox accesses across servers or simply between two system users in same server. (this may be tricky to dsync.) - global ACLs read from a single file supporting wildcards, instead of multiple different files - default ACLs for each namespace/storage root (maybe implemented using the above..) Metadata / annotations ---------------------- Add support for server, mailbox and mail annotations. These need to be dsyncable, so their changes need to be stored in various .log files: 1. Per-server metadata. This is similar to subscriptions: Add changes to dovecot.mailbox.log file, with each entry name a hash of the metadata key that was changed. 2. Per-mailbox metadata. Changes to this belong inside mailbox_transaction_context, which write the changes to mailbox's dovecot.index.log files. Each log record contains a list of changed annotation keys. This gives each change a modseq, and also allows easily finding out what changes other clients have done, so if a client has done ENABLE METADATA Dovecot can easily push metadata changes to client by only reading the dovecot.index.log file. 3. Per-mail metadata. This is pretty much equivalent to per-mailbox metadata, except changes are associated to specific message UIDs. The permanent storage is in dict. The dict keys have components: - priv/ vs. shared/ for specifying private vs. shared metadata - server/ vs mailbox// vs. mail// - the metadata key name This would be a good time to improve the dict configuration to allow things like: - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a file, while the rest goes to sql) - allow sql dict to be used in more relational way, so that mail annotations could be stored with tables: mailbox (id, guid) and mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid everywhere. Things to think through: - How to handle quota? Probably needs to be different from regular mail quota. Probably some per-user "metadata quota bytes" counter/limit. - Dict lookups should be done asynchronously and prefetched as much as possible. For per-mail annotation lookups mail_alloc() needs to include a list of annotations that are wanted. Configuration ------------- Copy all mail settings to namespaces, so it'll be possible to use per-namespace mailbox settings. Especially important for imapc_* settings, but can be useful for others as well. Those settings that aren't explicitly defined in the namespace will use the global defaults. (Should doveconf -a show all of these values, or simply the explicitly set values?) Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration. There are mainly 3 config files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. The dict-sql is something I think needs a bigger redesign (mentioned above in "Metadata" section), but the sql/ldap auth configs could be merged. One way could be: sql_db sqlmails { # most settings from dovecot-sql.conf.ext, except for queries driver = mysql connect = ... } ldap_db ldapmails { # most settings from dovecot-ldap.conf.ext, except attributes/filters } passdb { driver = sql db = sqlmails sql_query = select password from users where username = '%u' } passdb { driver = ldap db = ldapmails ldap_attributes { password = %{ldap:userPassword} } ldap_filter = ... } The sql_db {} and ldap_db {} would be generic enough to be used everywhere (e.g. dict-sql), not just for passdb/userdb. Some problems: - Similar to the per-namespace mail settings, doveconf -a would output all sql_query, ldap_attributes, ldap_filter, etc. settings for all passdbs/userdbs. Perhaps a similar solution? - The database configs contain passwords, so they should be readable only by root. This makes running dovecot-lda and maybe doveadm difficult, since they fail at "permission denied" when trying to open the config. There are probably only two solutions: a) The db configs need to be !include_try'd or b) the configs can be world-readable, but only passwords are placed to only-root-readable files by using "password = backend protocol could be improved to support moving connection to another backend. Possibly using a separate control connection to avoid making the proxying less efficient in normal operation. 3. When restarting Dovecot, move all the connections to a process that keeps the connections open for a while. When Dovecot starts up, create imap processes back to the connections. This allows changing configuration for existing client connections (which sometimes may be bad! need to add checks against client-visible config conflicts), upgrading Dovecot, etc. without being visible to clients. The only problem is SSL connections: OpenSSL doens't provide a way to save/restore state, so either you need to set shutdown_clients=no (and possibly keep some imap-login processes doing SSL proxying for a long time), or SSL connections need to be killed. Of course the SSL handling could be outsourced to some other software/hardware outside Dovecot. The IMAP state saving isn't always easy. Initially it could be implemented only for the simple cases (which are a majority) and later extended to cover more. IMAP extensions --------------- - CATENATE is already implemented by Stephan - URLAUTH is also planned to be implemented, somewhat differently than in Apple's patch. The idea is to create a separate imap-urlauth service that provides extra security. - NOTIFY extension could be implemented efficiently using mailbox list indexes, which already exists in v2.1. - FILTERS extension can be easily implemented once METADATA is implemented - There are also other missing extensions, but they're probably less important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. Backups ------- Filesystem based backups have worked well enough with Dovecot in the past. But with new features like single instance storage it's becoming more difficult. There's no 100% consistent way to even get filesystem level backups with SIS enabled, because deleting both the message file and its attachment files can't be done atomically (although usually this isn't a real problem). Restoring SIS mails is more difficult though, first you need to restore the dbox mail files and then you need to figure out what attachment files from SIS need to be restored, and finally you'll need to do doveadm import to put them into their final destination. I don't have much experience with backup software, but other people in my company do. The initial idea is to implement a Dovecot backup agent to one (commercial) backup software, which allows doing online backups and restoring mails one user/mailbox/mail at a time. I don't know the details yet how exactly this is going to be implemented, but the basic plan is probably to implement a "backup" mail storage backend, which is a PostgreSQL pg_dump-like flat file containing mails from all mailboxes. doveadm backup/import can then export/import this format via stdout/stdin. Incremental backups could possibly be done by giving a timestamp of previous backup run (I'm not sure about this yet). Once I've managed to implement the first fully functional backup agent, it should become clearer how to implement it to other backup solutions. Random things ------------- - dovecot.index.cache file writing is too complex, should be simplified - Enable auth_debug[_passwords]=yes on-the-fly for some specific users/IPs via doveadm - Optimize virtual mailboxes using mailbox list indexes. It wouldn't anymore need to keep all the backend mailboxes' index files open. - Would be nice to go forward with supporting key-value databases as mail storage backends. From janfrode at tanso.net Mon Feb 13 15:54:50 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 14:54:50 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend Message-ID: <20120213135450.GA31004@dibs.tanso.net> We've been collecting some stats to see what kind of benefits UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is running in High-performance mode http://wiki2.dovecot.org/LoginProcess with authentication caching http://wiki2.dovecot.org/Authentication/Caching During the weekend two servers (webmail3 and webmail4) has been running with local imapproxy and two servers without (webmail1 and webmail2). Each server has served about 1 million http requests, over 3 days. server avg. response time # requests ---------------------------------------------------- webmail1.example.net 0.370411 1092386 webmail2.example.net 0.374227 1045141 webmail3.example.net 0.378097 1043919 imapproxy webmail4.example.net 0.378593 1028653 imapproxy ONLY requests that took more than 5 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 26.048 1125 webmail2.example.net 26.2997 1080 webmail3.example.net 28.5596 808 imapproxy webmail4.example.net 27.1004 964 imapproxy ONLY requests that took more than 10 seconds to process: server avg. response time # requests ---------------------------------------------------- webmail1.example.net 49.1407 516 webmail2.example.net 53.0139 459 webmail3.example.net 59.7906 333 imapproxy webmail4.example.net 58.167 384 imapproxy The responstimes are not very fast, but they do seem to support the claim that an imapproxy isn't needed for dovecot. -jf From tss at iki.fi Mon Feb 13 16:14:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:14:22 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On 13.2.2012, at 15.54, Jan-Frode Myklebust wrote: > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Other things that would be interesting to try out (both from latency and disk IO usage point of view): - maildir_very_dirty_syncs - mail_prefetch_count (Linux+maildir only, v2.1+) From busseniu at in.tum.de Mon Feb 13 16:16:29 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 15:16:29 +0100 Subject: [Dovecot] dsync, hard-links and refcounts Message-ID: <4F391B3D.8020208@in.tum.de> Hi, when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. - Is there some way to make dsync notice the hard links? (I used dovecot 2.0.17) - Alternatively, is there some tool to deduplicate the mdbox after it has been converted from maildir? - NB: I am not talking about single-instance storage for attachments. Thank you very much, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Mon Feb 13 16:40:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 16:40:05 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F391B3D.8020208@in.tum.de> References: <4F391B3D.8020208@in.tum.de> Message-ID: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. > > My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. > - Is there some way to make dsync notice the hard links? (I used > dovecot 2.0.17) It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. From forumer at smartmobili.com Mon Feb 13 17:35:57 2012 From: forumer at smartmobili.com (forumer at smartmobili.com) Date: Mon, 13 Feb 2012 16:35:57 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: Hi, Is there any plan to port dovecot to windows ? From robert at schetterer.org Mon Feb 13 17:49:14 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:49:14 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F3930FA.2080802@schetterer.org> Am 13.02.2012 16:35, schrieb forumer at smartmobili.com: > Hi, > > Is there any plan to port dovecot to windows ? > cant wait Timos answer *g -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Mon Feb 13 17:51:26 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 13 Feb 2012 16:51:26 +0100 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <4F39317E.5090300@schetterer.org> Am 13.02.2012 12:47, schrieb Timo Sirainen: > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration might a question of taste, but i never liked the splitted up config style, so i like this idea -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Mon Feb 13 17:57:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 17:57:03 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <4F39317E.5090300@schetterer.org> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <4F39317E.5090300@schetterer.org> Message-ID: On 13.2.2012, at 17.51, Robert Schetterer wrote: > Am 13.02.2012 12:47, schrieb Timo Sirainen: >> Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration > > might a question of taste, but i never liked the splitted up config > style, so i like this idea Note that I said *.conf.ext, not *.conf.. From tss at iki.fi Mon Feb 13 18:11:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:11:33 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On 13.2.2012, at 17.35, forumer at smartmobili.com wrote: > Is there any plan to port dovecot to windows ? It probably works via Cygwin (although I think Cygwin has to do some kind of POSIX filesystem emulation, which slows things down and might cause some trouble if server crashes). Actually we have discussed this a little within my company.. I'm personally not interested in spending much time on it, but that's why we're hiring more coders so I won't have to do everything. :) If there is enough commercial interest, we might build something better than using Cygwin. Anyway, one thing is certain: No Dovecot for Windows questions in this mailing list. Perhaps I'll create another dovecot-windows@ mailing list. Perhaps I'll even give it a different name (dovedows? wincot? glassdove?) From joshua at hybrid.pl Mon Feb 13 18:26:38 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 17:26:38 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 Message-ID: Hi, Finally I decided to give a try to dovecot 2.0, after spending few years with 1.2. Unfortunately, it doesn't go smooth - finally I managed to cope with LMTP transport and exim, but failed to run managesieve. I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. Here is my doveconf -n. Any ideas? # 2.0.16: /etc/dovecot/dovecot.conf # OS: Linux 2.6.39.4-vs2.3.1-pre9.2-nfs41-nfsd x86_64 auth_debug = yes auth_debug_passwords = yes debug_log_path = /var/log/dovecot/debug.log dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_uid = 79 info_log_path = /var/log/dovecot/info.log last_valid_uid = 79 log_path = /var/log/dovecot/main.log mail_debug = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { sieve = /var/mail/sieve/%d/%u.sieve sieve_dir = /var/mail/sieve/%d/%u sieve_global_path = /var/mail/sieve/%d/global.sieve } postmaster_address = postmaster at hybrid.pl service imap-login { inet_listener imap { address = 127.0.0.1 port = 143 } inet_listener imaps { address = 0.0.0.0 port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { address = 127.0.0.1 port = 110 } inet_listener pop3s { address = 0.0.0.0 port = 995 ssl = yes } } ssl = no userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = " sieve quota" } protocol lda { mail_plugins = " sieve quota" } Best regards, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Mon Feb 13 18:42:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 18:42:21 +0200 Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: References: Message-ID: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> On 13.2.2012, at 18.26, Jacek Osiecki wrote: > I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. > > Here is my doveconf -n. Any ideas? > > # 2.0.16: /etc/dovecot/dovecot.conf Try adding: protocols = imap pop3 sieve From joshua at hybrid.pl Mon Feb 13 19:11:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 13 Feb 2012 18:11:11 +0100 (CET) Subject: [Dovecot] Managesieve doesn't listen on 2000/4190 In-Reply-To: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> References: <64303BB8-272A-4E60-9786-06E03D5E53A3@iki.fi> Message-ID: On Mon, 13 Feb 2012, Timo Sirainen wrote: > On 13.2.2012, at 18.26, Jacek Osiecki wrote: >> I have compiled pigeonhole, installed it, configured, everything seems to be done well - except the fact, that nothing is listening on port 2000 or 4190. >> Here is my doveconf -n. Any ideas? >> >> # 2.0.16: /etc/dovecot/dovecot.conf > Try adding: > protocols = imap pop3 sieve Great, it helped :) Thanks! Now time to see what surprises will show up while migrating there people from dovecot 1.0-stable :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From alexwbaule at gmail.com Mon Feb 13 19:28:37 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Mon, 13 Feb 2012 15:28:37 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: Hi Again Timo ! I Trying to port quota plugin notify_sync to my plugin, but i found some issues. I have the hooks already, to replace the mailbx_allocated, like quota plugin does. static struct mail_storage_hooks emexis_mail_storage_hooks = { .mail_user_created = Mplugin_mail_user_created, .mailbox_allocated = Mplugin_mailbox_allocated, .mail_allocated = Mplugin_mail_allocated, .mailbox_list_created = Mplugin_mailbox_list_created, .mail_namespaces_created = Mplugin_antispam_mail_namespaces_created }; inside the Mplugin_mailbox_allocated, i have: static void emexis_mailbox_allocated(struct mailbox *box) { union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = box->v; box->v.open = Mplugin_mailbox_open; box->v.sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); } The v.open i rewrite because my plugin needs. I try to call the sync_notify like this: box->v.sync_notify = Mplugin_mailbox_sync_notify; and follow quota plugin, i try to to this too: struct mailbox_vfuncs *v = box->vlast; v->sync_notify = Mplugin_mailbox_sync_notify; But in this two cases, the Mplugin_mailbox_sync_notify is never called... i missing something ?? Tks ! Em 12 de fevereiro de 2012 14:48, Alex Baule escreveu: > Tks timo ! > > I will see the plugin to do like it! > > Em 12/02/2012 13:46, "Timo Sirainen" escreveu: > >> Hi, >> >> Yeah, you shouldn't do the erasing directly in expunge(), because it may >> still be aborted. Do it in sync_notify() like quota plugin does. >> >> On 12.2.2012, at 16.52, Alex Baule wrote: >> >> > I want to know because i have header and body splited. To erase the >> > body, i must have shure, that header was expunged. Some tests that i >> > made, the client call expunge, but i don't ?know why (there is notting >> > in mail log) the header was not expunged, nut my body was, because i >> > call it before expunge the header. something like this. >> > >> > if (found_body){ >> > ? ?erase_body(); >> > ? ?super.expunge(_mail); >> > } >> > >> > Because that expunge that no happen, (the client call, but the email >> > was not expunged) i think to do like this way: >> > >> > if (found_body){ >> > ? super.expunge(_mail); >> > ? if(_mail was expunged){ >> > ? ? ?erase_body(); >> > ?} >> > } >> > >> > TKs Timo ! >> > >> > Em 12 de fevereiro de 2012 02:19, Timo Sirainen escreveu: >> >> On 10.2.2012, at 19.39, Alex Baule wrote: >> >> >> >>> Hy Everyone...and Timo ! >> >>> >> >>> There is a way to know if a email was expunged (deleted from hard >> >>> disk) ?inside a plugin ? I rewrite the expunge function, but the real >> >>> expunge function is void, i can't know if was really expunged. >> >>> >> >>> There is a way to know this ? >> >> >> >> It's not known until transaction_commit() what messages are expunged. >> >> What do you need to know this for? There are a few different ways. >> >> >> >> If you simply need one of the sessions to definitely know that it >> >> expunged a message, you can do it like quota plugin does. (So even if there >> >> are multiple clients doing EXPUNGE at the same time, quota is never >> >> decreased more than once per mail.) See quota-storage.c and most importantly >> >> quota_mailbox_sync_notify(). >> >> >> >> Quota code also shows the other method of doing it: keep track of what >> >> mail_expunge()s have been called, and then in transaction commit check if it >> >> succeeds and if it does do what you want to do. >> >> >> > >> > From busseniu at in.tum.de Mon Feb 13 20:31:51 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Mon, 13 Feb 2012 19:31:51 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F395717.5010504@in.tum.de> On 02/13/2012 03:40 PM, Timo Sirainen wrote: > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. I?d very much appreciate that. We will have to migrate many large mailboxes, and it would be a pity to needlessly waste space. I got the same result with some different configurations, using "mirror", "backup", or "-R backup" (though I have not tried 2.1 yet); so it should be easy to reproduce. However let me know if you need my configuration or anything. Cheers, Christoph From moseleymark at gmail.com Mon Feb 13 21:08:48 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Mon, 13 Feb 2012 11:08:48 -0800 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: On Mon, Feb 13, 2012 at 5:54 AM, Jan-Frode Myklebust wrote: > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 0.370411 ? ? ? ?1092386 > webmail2.example.net ? 0.374227 ? ? ? ?1045141 > webmail3.example.net ? 0.378097 ? ? ? ?1043919 ?imapproxy > webmail4.example.net ? 0.378593 ? ? ? ?1028653 ?imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 26.048 ? ? ? ? ?1125 > webmail2.example.net ? 26.2997 ? ? ? ? 1080 > webmail3.example.net ? 28.5596 ? ? ? ? 808 ? ? ?imapproxy > webmail4.example.net ? 27.1004 ? ? ? ? 964 ? ? ?imapproxy > > ONLY requests that took more than 10 seconds to process: > > server ? ? ? ? ?avg. response time ? ? ?# requests > ---------------------------------------------------- > webmail1.example.net ? 49.1407 ? ? ? ? 516 > webmail2.example.net ? 53.0139 ? ? ? ? 459 > webmail3.example.net ? 59.7906 ? ? ? ? 333 ? ? ?imapproxy > webmail4.example.net ? 58.167 ? ? ? ? ?384 ? ? ?imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Out of curiosity, are you running dovecot locally on those webmail servers as well, or is it talking to remote dovecot servers? I ask because I'm looking at moving our webmail from an on-box setup to a remote pool to support director and was going to look into whether running imapproxyd would help there. We don't bother with it in the local setup, since dovecot is so fast, but remote (but still on a LAN) might be different. Though imapproxyd seems to make (wait for it...) squirrelmail unhappy (complains about IMAP errors, when sniffing shows none), though I've not bothered to debug it yet. From janfrode at tanso.net Mon Feb 13 21:36:30 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:36:30 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213193630.GA3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 04:14:22PM +0200, Timo Sirainen wrote: > > The responstimes are not very fast, but they do seem to support > > the claim that an imapproxy isn't needed for dovecot. > > That's what I always suspected, but good to have someone actually test it. :) This is with Maildir? Yes, this is maildirs (on GPFS). > > Other things that would be interesting to try out (both from latency and disk IO usage point of view): > > - maildir_very_dirty_syncs We already have $ doveconf maildir_very_dirty_syncs maildir_very_dirty_syncs = yes but I don't think this gave the advantage I was expecting.. Was expecting this to move most iops to the index-luns, but the maildir luns seems just as busy. > - mail_prefetch_count (Linux+maildir only, v2.1+) Will look into if this works with GPFS when we upgrade to v2.1. It has it's own page cache, so I have no idea if it will respect POSIX_FADV_WILLNEED or if one will need to use it's own API's for hinting: http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html -jf From janfrode at tanso.net Mon Feb 13 21:43:57 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 20:43:57 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213194356.GB3850@dibs.tanso.net> On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: > > Out of curiosity, are you running dovecot locally on those webmail > servers as well, or is it talking to remote dovecot servers? The webmail servers are talking with dovecot director servers which in turn are talking with the backend dovecot servers. Each service running on different servers. Webmail-servers -> director-servers -> backend-servers > I ask because I'm looking at moving our webmail from an on-box setup to a > remote pool to support director and was going to look into whether > running imapproxyd would help there. We don't bother with it in the > local setup, since dovecot is so fast, but remote (but still on a LAN) > might be different. Doesn't seem so to us... > Though imapproxyd seems to make (wait for it...) > squirrelmail unhappy (complains about IMAP errors, when sniffing shows > none), though I've not bothered to debug it yet. :-) -jf From tss at iki.fi Mon Feb 13 21:57:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 21:57:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213193630.GA3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: On 13.2.2012, at 21.36, Jan-Frode Myklebust wrote: >> Other things that would be interesting to try out (both from latency and disk IO usage point of view): >> >> - maildir_very_dirty_syncs > > We already have > > $ doveconf maildir_very_dirty_syncs > maildir_very_dirty_syncs = yes > > but I don't think this gave the advantage I was expecting.. Was > expecting this to move most iops to the index-luns, but the maildir > luns seems just as busy. This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. >> - mail_prefetch_count (Linux+maildir only, v2.1+) > > Will look into if this works with GPFS when we upgrade to v2.1. It has > it's own page cache, so I have no idea if it will respect > POSIX_FADV_WILLNEED or if one will need to use it's own API's for > hinting: > > http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=%2Fcom.ibm.cluster.gpfs.v3r4.gpfs300.doc%2Fbl1adm_mlacrge.html I guess if there's an easy way to lookup filename or fd -> blockNumber that wouldn't be difficult to implement with a plugin. From chayes at afo.net Mon Feb 13 22:03:02 2012 From: chayes at afo.net (Cliff Hayes) Date: Mon, 13 Feb 2012 14:03:02 -0600 Subject: [Dovecot] Logging to messages instead of secure Message-ID: <5CDFC2DC614E436F89C10AF7F19F7DAB@ciosys> I need to log to /var/log/messages instead of /var/log/secure on a fedora system. Is this the correct entry for dovecot.conf? syslog_facility = info From tss at iki.fi Mon Feb 13 22:18:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 22:18:42 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> Message-ID: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> On 13.2.2012, at 19.28, Alex Baule wrote: > static void emexis_mailbox_allocated(struct mailbox *box) > { > union mailbox_module_context *zbox; > > zbox = p_new(box->pool, union mailbox_module_context, 1); > zbox->super = box->v; > box->v.open = Mplugin_mailbox_open; > box->v.sync_notify = Mplugin_mailbox_sync_notify; > > MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); > } The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > struct mailbox_vfuncs *v = box->vlast; > v->sync_notify = Mplugin_mailbox_sync_notify; > > But in this two cases, the Mplugin_mailbox_sync_notify is never > called... i missing something ?? Never called at all? What storage backend are you using as the base? Maildir? Anyway, the way you should be calling it is the exact same way quota plugin does: void Mplugin_mailbox_allocated(struct mailbox *box) { struct mailbox_vfuncs *v = box->vlast; union mailbox_module_context *zbox; zbox = p_new(box->pool, union mailbox_module_context, 1); zbox->super = *v; box->vlast = &zbox->super; v->open = Mplugin_mailbox_open; v->sync_notify = Mplugin_mailbox_sync_notify; MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); } From slusarz at curecanti.org Mon Feb 13 22:24:25 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 13:24:25 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213135450.GA31004@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> Message-ID: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > We've been collecting some stats to see what kind of benefits > UP/SquirrelMail's IMAP Proxy in for our SOGo webmail users. Dovecot is > running in High-performance mode http://wiki2.dovecot.org/LoginProcess > with authentication caching http://wiki2.dovecot.org/Authentication/Caching > > During the weekend two servers (webmail3 and webmail4) has been running > with local imapproxy and two servers without (webmail1 and webmail2). Each > server has served about 1 million http requests, over 3 days. > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 0.370411 1092386 > webmail2.example.net 0.374227 1045141 > webmail3.example.net 0.378097 1043919 imapproxy > webmail4.example.net 0.378593 1028653 imapproxy > > > ONLY requests that took more than 5 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 26.048 1125 > webmail2.example.net 26.2997 1080 > webmail3.example.net 28.5596 808 imapproxy > webmail4.example.net 27.1004 964 imapproxy > > ONLY requests that took more than 10 seconds to process: > > server avg. response time # requests > ---------------------------------------------------- > webmail1.example.net 49.1407 516 > webmail2.example.net 53.0139 459 > webmail3.example.net 59.7906 333 imapproxy > webmail4.example.net 58.167 384 imapproxy > > The responstimes are not very fast, but they do seem to support > the claim that an imapproxy isn't needed for dovecot. Except you are most likely NOT leveraging the truly interesting part of imapproxy - the ability to restore the IMAP connection state via the XPROXYREUSE status response. This is a significant performance improvement since it also reduces processing load on the client side (everything before/including authentication needs to be done whether using imapproxy or not, so there is no client-side savings for these commands). For further information, see, e.g.: http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. michael From janfrode at tanso.net Mon Feb 13 22:26:23 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:26:23 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213193630.GA3850@dibs.tanso.net> Message-ID: <20120213202623.GA5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 09:57:31PM +0200, Timo Sirainen wrote: > > > > $ doveconf maildir_very_dirty_syncs > > maildir_very_dirty_syncs = yes > > > > but I don't think this gave the advantage I was expecting.. Was > > expecting this to move most iops to the index-luns, but the maildir > > luns seems just as busy. > > This setting should get rid of almost all readdir() calls. If it doesn't, something's not working right. With maildir_very_dirty_syncs = yes: ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 1.5 0.0 96 92 514 73 9 7 1.2 0.0 59 43 367 18 4 76 1.7 0.0 66 61 477 67 2 6 1.2 0.0 54 50 348 31 1 145 3.0 0.0 113 90 860 59 7 8 2.9 0.0 107 99 840 58 5 11 4.0 0.0 131 101 1117 77 2 65 With maildir_very_dirty_syncs = no (same node, a few seconds later): ReadMB/s WriteMB/s F_open f_close reads writes rdir inode 4.6 0.9 125 91 1161 1096 41 6 2.3 0.7 200 170 697 127 86 16 1.1 0.6 124 99 406 61 48 109 2.7 0.1 212 144 755 114 74 15 2.7 0.0 159 133 818 70 78 194 0.8 1.2 86 73 225 60 16 9 1.9 0.0 124 116 573 53 30 6 So it seems to be working, good :-) -jf From janfrode at tanso.net Mon Feb 13 22:35:11 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 13 Feb 2012 21:35:11 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <20120213203511.GB5085@dibs.tanso.net> On Mon, Feb 13, 2012 at 01:24:25PM -0700, Michael M Slusarz wrote: > > Except you are most likely NOT leveraging the truly interesting part > of imapproxy - the ability to restore the IMAP connection state via > the XPROXYREUSE status response. This is a significant performance > improvement since it also reduces processing load on the client side > (everything before/including authentication needs to be done whether > using imapproxy or not, so there is no client-side savings for these > commands). Thanks for this info, good to know. I'll check with inverse/sogo if this is something they use/intend to use.. > > additional advantage. Note that the XPROXYREUSE-enabled MUA must be > the exclusive user of the imapproxy instance for this feature to > work correctly. Not a problem. Assuming it doesn't also need to be the only imap user of the account/folder. BTW: do you also have information on the state of select caching in the up-imapproxy? I got some very negative comments when googling it, and the changelog didn't suggest there had been any improvements since.. -jf From stephan at rename-it.nl Mon Feb 13 23:01:10 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 13 Feb 2012 22:01:10 +0100 Subject: [Dovecot] Released Pigeonhole v0.2.6 for Dovecot v2.0.18 Message-ID: <4F397A16.1010107@rename-it.nl> Hello Dovecot users, Pigeonhole v0.2.5 does not compile against the new Dovecot v2.0.18 release. That is why a release a new version now. Other changes are included as well, of which the one regarding the include extension requires special attention; Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. * Matched changes regarding auth_verbose setting in Dovecot. This means that this release will only compile against Dovecot v2.0.18. - Fixed problem in ManageSieve that caused it to omit a WARNINGS response code when the uploaded script compiled with warnings. - Made sure that locations of Sieve error never report `line 0'. - Fixed potential segfault occurring when interpreter initialization fails. The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.0-pigeonhole-0.2.6.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.0 wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From tss at iki.fi Mon Feb 13 23:32:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:32:31 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> Message-ID: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> On 13.2.2012, at 22.24, Michael M Slusarz wrote: > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html > http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html > > These posts neglect the fact that you don't need to issue a CAPABILITY command if the connection is reused either, so that's an additional advantage. Note that the XPROXYREUSE-enabled MUA must be the exclusive user of the imapproxy instance for this feature to work correctly. > > Somewhat topical, since Timo was just mentioning support for some sort of IMAP state save/restore feature possibly making it into 2.2. Perhaps a way for (trusted) clients to be able to do this? :) a logout save * OK [SAVEDSTATE 1234567890] * BYE logged out a OK ... b login (SAVEDSTATE 1234567890) user pass * OK [RESTOREDSTATE 1234567890] b ok vs. b login (SAVEDSTATE 1234567890) user pass * NO [UNKNOWNSTATE 1234567890] b ok From tss at iki.fi Mon Feb 13 23:44:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 13 Feb 2012 23:44:47 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: On 13.2.2012, at 23.32, Timo Sirainen wrote: > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK Actually, this could probably be safely implemented by sending all of the state to the client as a string: * OK [SAVEDSTATE base64-encoded-state] There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). From ybhu at hk1.ibm.com Tue Feb 14 04:39:01 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 10:39:01 +0800 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: Another question is about the director failover. In http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html you said: "The main complexity here comes from how to handle proxy server failures in different situations. Those are less interesting to describe and I haven't yet implemented all of it, so let's just assume that in future it all works perfectly." So currently there is no good way to handle the director failover, or any 3rd party solution like poolmon by Brad Davidson for health monitoring of backend servers? Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/12/2012 12:06 Subject PM Re: [Dovecot] Synchronization error in NFS Please respond to Dovecot Mailing List On 10.2.2012, at 10.08, Andy YB Hu wrote: > Searched the archive, and got the doveadm -a director-admin for local > doveadm access, and -a host: port for remote doveadm access. You can give -c parameter also to doveadm (and all other Dovecot programs as well). > And you said: http://dovecot.org/list/dovecot/2010-July/050731.html > Now in my director, I have configed the userdb passwd, but the same error > occur: > doveadm(root): Error: User listing returned failure > doveadm(root): Error: user listing failed > > And in the log: Feb 10 07:45:25 auth: Error: Trying to iterate users, but > userdbs don't support it I'm guessing that it's connecting to wrong Dovecot's auth process. Use doveadm -c instead of -a. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09320.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Tue Feb 14 04:46:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 04:46:21 +0200 Subject: [Dovecot] Synchronization error in NFS In-Reply-To: References: <3313B809-C3EA-4DF1-8B68-FE3B72A0BFCB@iki.fi> <0DE1863C-4F05-42E1-9E69-F4BB3185FF38@iki.fi> <7A654492-FE09-453E-B31B-CA519F25C11D@iki.fi> Message-ID: On 14.2.2012, at 4.39, Andy YB Hu wrote: > Another question is about the director failover. In > http://blog.dovecot.org/2010/05/new-director-service-in-v20-for-nfs.html > you said: "The main complexity here comes from how to handle proxy server > failures in different situations. Those are less interesting to describe > and I haven't yet implemented all of it, so let's just assume that in > future it all works perfectly." So currently there is no good way to > handle the director failover, or any 3rd party solution like poolmon by > Brad Davidson for health monitoring of backend servers? Your load balancer handles director downtimes so it connects only to directors that are up. Directors themselves figure out when one of them is down (by either explicit disconnection or timeout). That's the theory, and so far no one's told me it doesn't work that way. For the backend servers there's still no automation though. You'll need to explicitly tell director to stop trying to connect some specific backend. The poolmon is intended for that. It would be possible to implement this directly in the director itself, but so far it hasn't really been a priority since the companies who have paid for it have wanted to implement it internally themselves.. From dmiller at amfes.com Tue Feb 14 04:54:20 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 18:54:20 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 2/12/2012 1:01 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz > http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc6.tar.gz.sig > > I've finally finished all of my email backlog. If you haven't received an answer to some question/bugreport, resend the mail. > > This is hopefully the last v2.1 RC. If I don't receive any (serious) bug reports about this release in next few days, I'll just change the version number to v2.1.0 (and maybe update man pages, some are still missing..) > With this configure line: ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene I get this error on make: make[4]: Entering directory `/opt/dovecot-2.1.rc6/src/plugins/fts-lucene' /bin/bash ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arh -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -o lib20_doveadm_s_lucene_plugin.la -rpath /usr/local/lib/dovecot/doveadm doveadm-fts-lucene.lo -lrt /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I..//../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF eps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/l-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wraer.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory compilation terminated. make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... -- Daniel From tss at iki.fi Tue Feb 14 05:00:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:00:46 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> Message-ID: On 14.2.2012, at 4.54, Daniel L. Miller wrote: > With this configure line: > ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene > > lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ From dmiller at amfes.com Tue Feb 14 05:01:59 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:01:59 -0800 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 2/13/2012 7:00 PM, Timo Sirainen wrote: > On 14.2.2012, at 4.54, Daniel L. Miller wrote: > >> With this configure line: >> ./configure --with-ldap --with-bzlip --with-zlib --with-solr --with-lucene >> >> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory > Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ > Ah! I would've thought configure would catch that though - before make did. Ok. -- Daniel From dmiller at amfes.com Tue Feb 14 05:04:26 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:04:26 -0800 Subject: [Dovecot] Boehm Message-ID: As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? -- Daniel From tss at iki.fi Tue Feb 14 05:05:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:05:33 +0200 Subject: [Dovecot] Boehm In-Reply-To: References: Message-ID: <4E1CD0CF-7679-4D4D-9DF1-E5B101ECE797@iki.fi> On 14.2.2012, at 5.04, Daniel L. Miller wrote: > As the Boehm GC is an available option, do I take that to mean it's beneficial for some users but not most? Eh. Probably not. From slusarz at curecanti.org Tue Feb 14 05:12:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:12:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203511.GB5085@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <20120213203511.GB5085@dibs.tanso.net> Message-ID: <20120213201256.Horde.k3ITIIF5lbhPOdE4A_7B9KA@bigworm.curecanti.org> Quoting Jan-Frode Myklebust : > BTW: do you also have information on the state of select caching in the > up-imapproxy? I got some very negative comments when googling it, and the > changelog didn't suggest there had been any improvements since.. I wouldn't trust it. IIRC, it was added years ago and given the syntax changes to the SELECT/EXAMINE call since then (e.g. QRESYNC extentsion), it is doubtful the code has been updated to handle these situations. And in a server-caching IMAP server like dovecot, all that SELECT information is not expensive anyway, so any gains would be miniscule. michael From slusarz at curecanti.org Tue Feb 14 05:19:56 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:19:56 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 22.24, Michael M Slusarz wrote: > >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052316.html >> http://lists.horde.org/archives/imp/Week-of-Mon-20110523/052317.html >> >> These posts neglect the fact that you don't need to issue a >> CAPABILITY command if the connection is reused either, so that's an >> additional advantage. Note that the XPROXYREUSE-enabled MUA must be >> the exclusive user of the imapproxy instance for this feature to >> work correctly. >> >> Somewhat topical, since Timo was just mentioning support for some >> sort of IMAP state save/restore feature possibly making it into 2.2. > > Perhaps a way for (trusted) clients to be able to do this? :) > > a logout save > * OK [SAVEDSTATE 1234567890] > * BYE logged out > a OK > > ... > > b login (SAVEDSTATE 1234567890) user pass > * OK [RESTOREDSTATE 1234567890] > b ok > > vs. > > b login (SAVEDSTATE 1234567890) user pass > * NO [UNKNOWNSTATE 1234567890] > b ok I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. And you don't allow reviving the state if using the AUTHENTICATE command. The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. michael From tss at iki.fi Tue Feb 14 05:23:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 14 Feb 2012 05:23:16 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> Message-ID: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> On 14.2.2012, at 5.19, Michael M Slusarz wrote: >> b login (SAVEDSTATE 1234567890) user pass > > I guess the drawback for this approach is that you are explicitly breaking the LOGIN definition. No breaking, extending :) > And you don't allow reviving the state if using the AUTHENTICATE command. Just as easy: b authenticate (savedstate ..) plain .. > The alternative would be to have an additional pre-authentication command that sets the desired state. But that adds the overhead of sending/parsing another command. Yeah. Although not bad either, since you can pipeline that command + login/authenticate. From slusarz at curecanti.org Tue Feb 14 05:30:09 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:30:09 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> Message-ID: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Quoting Timo Sirainen : > On 13.2.2012, at 23.32, Timo Sirainen wrote: > >> Perhaps a way for (trusted) clients to be able to do this? :) >> >> a logout save >> * OK [SAVEDSTATE 1234567890] >> * BYE logged out >> a OK > > Actually, this could probably be safely implemented by sending all > of the state to the client as a string: > > * OK [SAVEDSTATE base64-encoded-state] > > There isn't a whole lot of state to be saved really. Mailbox GUID, > UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have > the language/etc. states. Clients could restore their earlier state > from days ago, as long as Dovecot still has the necessary .log > records available (similar to how QRESYNC works). Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. For a disconnected client (e.g. webmail), there's going to necessarily need to be startup costs of initializing the session anyway so re-creating the state can be buried in this other work. Re-creating becomes more important when users are doing actions they expect immediate (or at least very fast) responses for. Things like listing messages in a mailbox, viewing a message, or polling mailboxes. Saving 50ms per request becomes important in these situations since re-creating state now takes 10% of the total request time. So I don't think states don't need a terribly long lifetime. I almost think of something like an internal Dovecot IDLE queue - after a logout is received, state is stored for 30 minutes after which it is discarded. (Although not knowing anything about internal Dovecot state, don't know if this is overly resource intensive.) As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if you come up with a better way to represent state. michael From dmiller at amfes.com Tue Feb 14 05:34:44 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Mon, 13 Feb 2012 19:34:44 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: On 2/12/2012 9:00 AM, Timo Sirainen wrote: > On 12.2.2012, at 15.27, Mike Constabel wrote: > >> Another Question to this Option: >> >> # tb-extra-mailbox-sep: >> # With mbox storage a mailbox can contain either mails or submailboxes, >> # but not both. Thunderbird separates these two by forcing server to >> # accept '/' suffix in mailbox names in subscriptions list. >> >> >> I thought this is not needed because I use mdbox. But mbox here >> means also mdbox and sdbox? > Yes. I updated the comment: http://hg.dovecot.org/dovecot-2.1/rev/1a0ab868957b > >> Should I use all 3 workarounds if I don't know which clients are >> in use? > I don't know if delay-newmail is still relevant. > I'm fighting the same issue. With the current release (2.1rc6) no errors are logged - but I'm having problems deleted folders from trash. I've enabled tb-extra-mailbox-sep. Two folders that were previously in Trash were successfully removed. I deleted four more - after several Thunderbird restarts, compacts, refreshes, I have killed three of them - there is still one folder that won't "die". Thunderbird reports the mailbox doesn't exist, yet it continues to appear. -- Daniel From slusarz at curecanti.org Tue Feb 14 05:35:46 2012 From: slusarz at curecanti.org (Michael M Slusarz) Date: Mon, 13 Feb 2012 20:35:46 -0700 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213201956.Horde.CA1XE4F5lbhPOdLcjP_h9KA@bigworm.curecanti.org> <865C6CB7-DB1A-4DCA-84AD-04876B5A98D6@iki.fi> Message-ID: <20120213203546.Horde.TaVqCoF5lbhPOdaS8PLB9SA@bigworm.curecanti.org> Quoting Timo Sirainen : > On 14.2.2012, at 5.19, Michael M Slusarz wrote: > >>> b login (SAVEDSTATE 1234567890) user pass >> >> I guess the drawback for this approach is that you are explicitly >> breaking the LOGIN definition. > > No breaking, extending :) Heh. I should know better after reading about 1000 RFCs last year... Anyway, to play devil's advocate, another drawback of this approach is that in the (fairly remote) chance that the LOGIN command is extended in the future by an RFC, there is no guarantee that this syntax would be used. michael From ybhu at hk1.ibm.com Tue Feb 14 05:31:54 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Tue, 14 Feb 2012 11:31:54 +0800 Subject: [Dovecot] index synchronization during local delivery Message-ID: Hi there, Now our system will take dovecot as the whole imap back end, and keep the LDA of ours, But our system has our own UID generation mechanism (in dovecot it's incremental from 1), which will be done while local delivery, So when local deliveries, it will first generate the UID, move the mail into maildir, then synchronize the mailbox index, which is part of dovecot LDA does. The question is what API or something else about index synchronization I can integrate into our own LDA program easily? Seems mailbox_copy() would be the possibility? Thanks. From busseniu at in.tum.de Tue Feb 14 17:04:25 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 14 Feb 2012 16:04:25 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit Message-ID: <4F3A77F9.8000002@in.tum.de> Hi, our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. The "doveadm proxy list" would fail with an error message: $ sudo doveadm proxy list doveadm(root): Error: LIST failed: Disconnected Segmentation fault The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. Anyway, I suggest changing the log message if the failure is not actually due to the client_limit. # 2.0.18: /etc/ports/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.2 LTS auth_username_format = %n default_client_limit = 60000 default_process_limit = 10000 disable_plaintext_auth = no mail_location = maildir:/nonexistent managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/ports/dovecot/passwd driver = passwd-file } protocols = imap pop3 lmtp sieve service anvil { client_limit = 100000 } service auth { client_limit = 60000 } service ipc { client_limit = 100000 } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } shutdown_clients = no ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From cinzia.delcoiro at libero.it Tue Feb 14 17:02:07 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:02:07 +0100 (CET) Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <25869845.713261329231727832.JavaMail.root@wmail51> Hi Timo, I'm writing to alert you that even in version 2.1. we have the error: "Can't handle mixed proxy/non-proxy destinations" when we write to two users: one existent and one not. The error occurs randomly. I configure LDAP query in the file : dovecot-ldap.conf in this way: pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, =proxy_timeout=600 specifying "=proxy=y" in order to proxy every mail. As workaround, can I configure a static "host" for non-existing users? This is my dovecot -n # 2.1.rc5: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_mechanisms = plain login base_dir = /usr/local/var/run/dovecot/ disable_plaintext_auth = no listen = 127.0.0.1 lmtp_proxy = yes mail_gid = mailusers mail_location = maildir:/mnt/mbox1/mailusers/%n/Maildir mail_uid = mailusers passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User Quota } protocols = lmtp service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no syslog_facility = local5 userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { mail_plugins = quota imap_quota } protocol lmtp { postmaster_address = info at amm2.miur.it } Regards Cinzia >----Messaggio originale---- >Da: tss at iki.fi >Data: 28-gen-2012 18.32 >A: "Alexis Lelion" >Cc: "Dovecot Mailing List" >Ogg: Re: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations > >On 27.1.2012, at 12.59, Alexis Lelion wrote: > >> Jan 25 09:05:12 mail01 postfix/lmtp[23934]: A92709300DB: to=< >> user_on_mail02 at domain.com>, relay=mail01.domain.com[private/dovecot-lmtp], >> delay=0.07, delays=0.01/0/0/0.06, dsn=4.3.0, status=deferred (host >> mail01.domain.com[private/dovecot-lmtp] said: 451 4.3.0 < >> user_on_mail02 at domain.com> Can't handle mixed proxy/non-proxy destinations >> (in reply to RCPT TO command)) >> >> I was wondering if there was another way of handling this, for example >> by triggering an immediate queue lookup from postfix or forwarding a >> copy of the mail to the other server. Note that the postfix >> "queue_run_delay" was increased to 15min on purpose, so I cannot change >> that. > >It would be possible to change the code to support mixed destinations, but it's probably not a simple change and I have other things to do.. > >Maybe you could work around it so that LMTP always proxies the mails, to localhost as well, but to a different port which doesn't do proxying at all. > > From CMarcus at Media-Brokers.com Tue Feb 14 17:42:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 14 Feb 2012 10:42:12 -0500 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <4F3A80D4.4000007@Media-Brokers.com> On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it wrote: > I'm writing to alert you that even in version 2.1. we have the error: "Can't > handle mixed proxy/non-proxy destinations" when we write to two users: one > existent and one not. why are you accepting mail for non existing users? Catch-alls are a very very bad idea in 99.999% of cases, and usually when someone thinks they need a catchall, they are mistaken. -- Best regards, Charles From cinzia.delcoiro at libero.it Tue Feb 14 17:54:38 2012 From: cinzia.delcoiro at libero.it (cinzia.delcoiro at libero.it) Date: Tue, 14 Feb 2012 16:54:38 +0100 (CET) Subject: [Dovecot] R: Re: R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations Message-ID: <23144211.746481329234878394.JavaMail.root@wmail51> Charles I need to accept mail also for nonexistent users because I'm delivering a particular service (Certified Mail). Regards Cinzia >----Messaggio originale---- >Da: CMarcus at Media-Brokers.com >Data: 14-feb-2012 16.42 >A: "Dovecot Mailing List" >Ogg: Re: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations > >On 2012-02-14 10:02 AM, cinzia.delcoiro at libero.it > wrote: >> I'm writing to alert you that even in version 2.1. we have the error: "Can't >> handle mixed proxy/non-proxy destinations" when we write to two users: one >> existent and one not. > >why are you accepting mail for non existing users? > >Catch-alls are a very very bad idea in 99.999% of cases, and usually >when someone thinks they need a catchall, they are mistaken. > >-- > >Best regards, > >Charles > From janfrode at tanso.net Tue Feb 14 21:34:42 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 20:34:42 +0100 Subject: [Dovecot] pop3 not autocreating directory structure Message-ID: <20120214193442.GA5156@dibs.tanso.net> We use: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail: mdbox:~/mdbox and I just noticed one of our newly provisioned users initially failed to pop her mails. I saw several of these: dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX dovecot:: pop3(new.user at example.net): Couldn't open INBOX top=0/0, retr=0/0, del=0/0, size=0 before she switched to imap and then everything looked fine: dovecot:: imap(new.user at example.net): Disconnected: Logged out bytes=11/338 Looking at the timestamps in the filesystem I see that the users home directory wasn't created before switcing to imap. Is this a know problem? -jf From wgillespie+dovecot at es2eng.com Tue Feb 14 22:06:56 2012 From: wgillespie+dovecot at es2eng.com (Willie Gillespie) Date: Tue, 14 Feb 2012 13:06:56 -0700 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> Message-ID: <4F3ABEE0.6010702@es2eng.com> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: > I'm fighting the same issue. With the current release (2.1rc6) no errors > are logged - but I'm having problems deleted folders from trash. I've > enabled tb-extra-mailbox-sep. Two folders that were previously in Trash > were successfully removed. I deleted four more - after several > Thunderbird restarts, compacts, refreshes, I have killed three of them - > there is still one folder that won't "die". Thunderbird reports the > mailbox doesn't exist, yet it continues to appear. Out of curiosity -- does the mailbox actually exist if you look on the Dovecot server? Secondly, is it listed in your subscription file? From dmiller at amfes.com Tue Feb 14 22:40:02 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Tue, 14 Feb 2012 12:40:02 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3ABEE0.6010702@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> Message-ID: On 2/14/2012 12:06 PM, Willie Gillespie wrote: > On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >> I'm fighting the same issue. With the current release (2.1rc6) no errors >> are logged - but I'm having problems deleted folders from trash. I've >> enabled tb-extra-mailbox-sep. Two folders that were previously in Trash >> were successfully removed. I deleted four more - after several >> Thunderbird restarts, compacts, refreshes, I have killed three of them - >> there is still one folder that won't "die". Thunderbird reports the >> mailbox doesn't exist, yet it continues to appear. > > Out of curiosity -- does the mailbox actually exist if you look on the > Dovecot server? > Secondly, is it listed in your subscription file? At one time - yes they were. After further playing, they do not exist in the on-disk folder structure, nor in the subscription file. But still show in the client - with errors. -- Daniel From janfrode at tanso.net Tue Feb 14 22:47:51 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 14 Feb 2012 21:47:51 +0100 Subject: [Dovecot] doveadm director proxy Message-ID: <20120214204751.GA6279@dibs.tanso.net> I'm trying to configure a doveadm service that will proxy trough our directors, following the recipie at: http://wiki2.dovecot.org/Director#Doveadm_server So on the backends I have: service doveadm { inet_listener { port = 24245 address = * } } doveadm_proxy_port = 24245 local 192.168.42.0/24 { doveadm_password = suPerSeecret } I assume the "local" line is supposed to point at my local network..? On the directors I have the same, plus: protocol doveadm { auth_socket_path = director-userdb } When testing "doveadm quota" on the directors, it complained quota plugin not being loaded, so I added: mail_plugins=quota Then it complained about "doveadm_password not set, can't authenticate", so I added: doveadm_password = suPerSeecret in the main section. Now I get trough to my backend servers, but they complain about: dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication Any ideas for what that might be? This is with dovecot v2.0.14. -jf From as at 3a.pl Tue Feb 14 23:29:19 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 14 Feb 2012 22:29:19 +0100 Subject: [Dovecot] domain quota in sql Message-ID: <4F3AD22F.5080904@3a.pl> Hi, I have a problem with setting up quota usage tracking for domains. Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. To recreate this situation, one has only to perform: doveadm quota recalc -u USER on a user, and the domain quota takes the value of this user quota. Have anyone a clue what could cause this problem? Quota tracking is set up both for users and domains. For users quota tracking is working fine, data is saved into dbase without problems. We are using dovecot 2.0.15. Quota enforcing (rules) is NOT a part of the problem. Important parts of config: dict { quota = mysql:/etc/dovecot/dovecot-dict-used-quota.conf.ext quota2 = mysql:/etc/dovecot/dovecot-dict-used-quota-domain.conf.ext } plugin { quota = dict:User quota::proxy::quota quota2 = dict:Domain quota:%d:proxy::quota2 } /etc/dovecot/dovecot-dict-used-quota.conf.ext map { pattern = priv/quota/storage table = used_quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = used_quota username_field = username value_field = messages } /etc/dovecot/dovecot-dict-used-quota-domain.conf.ext map { pattern = priv/quota/storage table = used_quota_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = used_quota_domain username_field = domain value_field = messages } -- Adam Szpakowski From tss at iki.fi Wed Feb 15 03:47:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:47:11 +0200 Subject: [Dovecot] doveadm director proxy In-Reply-To: <20120214204751.GA6279@dibs.tanso.net> References: <20120214204751.GA6279@dibs.tanso.net> Message-ID: On 14.2.2012, at 22.47, Jan-Frode Myklebust wrote: > I'm trying to configure a doveadm service that will proxy trough our > directors, following the recipie at: > > http://wiki2.dovecot.org/Director#Doveadm_server > > So on the backends I have: > > service doveadm { > inet_listener { > port = 24245 > address = * > } > } > doveadm_proxy_port = 24245 > local 192.168.42.0/24 { > doveadm_password = suPerSeecret > } > > I assume the "local" line is supposed to point at my local network..? Yes. Those IPs where you allow authentication from. > in the main section. Now I get trough to my backend servers, but they > complain about: > > dovecot:: doveadm: Error: doveadm client attempted non-PLAIN authentication This means doveadm was talking incorrectly. > Any ideas for what that might be? This is with dovecot v2.0.14. Some v2.0 versions had bugs related to doveadm server. So this is probably a bug in v2.0.14. And I'm not entirely sure if doveadm server works correctly in v2.0 at all, since I did some bigger changes to v2.1 related to it. From tss at iki.fi Wed Feb 15 03:49:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:49:21 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120214193442.GA5156@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: On 14.2.2012, at 21.34, Jan-Frode Myklebust wrote: > dovecot:: pop3(new.user at example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX > > Looking at the timestamps in the filesystem I see that the users home > directory wasn't created before switcing to imap. > > Is this a know problem? Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). From tss at iki.fi Wed Feb 15 03:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:54:01 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3AD22F.5080904@3a.pl> References: <4F3AD22F.5080904@3a.pl> Message-ID: <147CCD39-537F-455A-9B55-212340747969@iki.fi> On 14.2.2012, at 23.29, Adam Szpakowski wrote: > Hi, > I have a problem with setting up quota usage tracking for domains. > Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. > To recreate this situation, one has only to perform: > doveadm quota recalc -u USER > on a user, and the domain quota takes the value of this user quota. > > Have anyone a clue what could cause this problem? Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: - set domain's quota to 0 - iterate *@domain users via userdb - for each user get the user's quota and add it to domain quota From tss at iki.fi Wed Feb 15 03:55:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 03:55:57 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <147CCD39-537F-455A-9B55-212340747969@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> Message-ID: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> On 15.2.2012, at 3.54, Timo Sirainen wrote: > Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: Oh, and you could actually already do this with a doveadm script: > - set domain's quota to 0 update .. | mysql > - iterate *@domain users via userdb doveadm user '*@domain' > - for each user get the user's quota and add it to domain quota doveadm quota get -u $user | some grep+sed magic | mysql Of course it would be nice if Dovecot internally supported this. From tss at iki.fi Wed Feb 15 04:04:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:04:24 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> References: <20120213135450.GA31004@dibs.tanso.net> <20120213132425.Horde.WhYcAIF5lbhPOXF5Gp9HdDA@bigworm.curecanti.org> <9A6F0D83-4916-42E3-A134-6CAF2C4245A9@iki.fi> <20120213203009.Horde.ifIPCIF5lbhPOdVBBENx88A@bigworm.curecanti.org> Message-ID: <6E30EEF1-1921-4E76-88AB-8743901E5E5C@iki.fi> On 14.2.2012, at 5.30, Michael M Slusarz wrote: >> Actually, this could probably be safely implemented by sending all of the state to the client as a string: >> >> * OK [SAVEDSTATE base64-encoded-state] >> >> There isn't a whole lot of state to be saved really. Mailbox GUID, UIDVALIDITY, HIGHESTMODSEQ gives the mailbox state. Then you have the language/etc. states. Clients could restore their earlier state from days ago, as long as Dovecot still has the necessary .log records available (similar to how QRESYNC works). > > Given that it is not *that* expensive to re-create the state, I don't think the ability to recreate state from several days ago would be worth the effort of storing in the log/index files. There wouldn't be *any* additional state stored. All the necessary state is already stored to .log files for other purposes. The base64-encoded-state would be a self-contained description of the IMAP connection's entire state. > As for base-64-encoded state: if other IMAP servers ever wanted to implement a similar protocol trying to coordinate the data structure would be a nightmare. Keeping it to something like a MODSEQ value would hide the state ID -> data abstraction entirely within dovecot. And would allow you to change your mind in the future if > you come up with a better way to represent state. The client doesn't need to care about the data structure. The client simply reads a string and sends it later back to server if it wants to restore that state. Only the server needs to verify that the string looks reasonable. The same string wouldn't be sent to a different server implementation, so there wouldn't be any interoperability issues. Each server can implement it in whatever way they want to (although there should be some checks in case the same string is sent to different versions of the same server). From tss at iki.fi Wed Feb 15 04:09:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:09:57 +0200 Subject: [Dovecot] v2.1.rc6 released In-Reply-To: References: <8E0D1373-1F12-42F3-AAB1-600E3D82E283@iki.fi> <4F39CCDC.9030508@amfes.com> Message-ID: On 14.2.2012, at 5.01, Daniel L. Miller wrote: >>> lucene-wrapper.cc:23: fatal error: CLucene.h: No such file or directory >> Base on this error, I suppose you don't have CLucene installed.. The 2.3.2 version is somewhere around in http://clucene.sourceforge.net/ >> > Ah! I would've thought configure would catch that though - before make did. Ok. Checking for C++ things in autoconf seems to be a little difficult.. From tss at iki.fi Wed Feb 15 04:19:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:19:00 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: Message-ID: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. From tss at iki.fi Wed Feb 15 04:24:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:24:19 +0200 Subject: [Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations In-Reply-To: <25869845.713261329231727832.JavaMail.root@wmail51> References: <25869845.713261329231727832.JavaMail.root@wmail51> Message-ID: <2AF7DAA3-ABB0-47AC-9189-E499C68DA370@iki.fi> On 14.2.2012, at 17.02, cinzia.delcoiro at libero.it wrote: > I configure LDAP query in the file : dovecot-ldap.conf in this way: > > pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, > =proxy_timeout=600 > > specifying "=proxy=y" in order to proxy every mail. In LMTP case proxy=y and proxy_maybe=y are actually exactly the same. If you want the mail always proxied, you need to also use a different port. So I guess it'll work if you: 1) Add another LMTP service listener for port 10024 2) Add =proxy_port=10024 to pass_attrs. Although if you use this same passdb for imap/pop3 lookups as well, you need to make the port change lmtp-only: protocol lmtp { passdb { driver = sql args = dovecot-ldap-lmtp.conf } } protocol !lmtp { passdb { driver = sql args = dovecot-ldap.conf } } From tss at iki.fi Wed Feb 15 04:35:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:35:22 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3A77F9.8000002@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: > > Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? > I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. Do you see some error message also about "Too many open files", or why do you think the number of open files was the problem? If a process runs out of fds there should be such an error message. > Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. Connections to anvil and ipc processes were being dropped. That doesn't necessarily mean that everything else stops. > The "doveadm proxy list" would fail with an error message: > > $ sudo doveadm proxy list > doveadm(root): Error: LIST failed: Disconnected > Segmentation fault > > The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. From tss at iki.fi Wed Feb 15 04:37:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 04:37:58 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: On 15.2.2012, at 4.35, Timo Sirainen wrote: >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Lets see if I can reproduce this then.. From list at airstreamcomm.net Wed Feb 15 05:08:05 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 14 Feb 2012 21:08:05 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> On Mon, 13 Feb 2012 13:47:06 +0200, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot > v2.2. Probably not all of them will make it, but I'm at least interested in > working on these if I have time. > > Previously I've mostly been working on things that different companies > were paying me to work on. This is the first time I have my own company, > but the prioritization still works pretty much the same way: > > - 1. priority: If your company is highly interested in getting something > implemented, we can do it as a project via my company. This guarantees > that you'll get the feature implemented in a way that integrates well into > your system. > - 2. priority: Companies who have bought Dovecot support contract can let > me know what they're interested in getting implemented. It's not a > guarantee that it gets implemented, but it does affect my priorities. :) > - 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are > before the 2. priority above. I guess we'll see how things work out. > > dsync-based replication > ----------------------- > > I'll write a separate post about this later. Besides, it's coming for > Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. > > Shared mailbox improvements > --------------------------- > > Support for private flags for all mailbox formats: > > namespace { > type = public > prefix = Public/ > mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public > } > > - dsync needs to be able to replicate the private flags as well as shared > flags. > - might as well add a common way for all mailbox formats to specify which > flags are shared and which aren't. $controldir/dovecot-flags would say > which is the default (private or shared) and what flags/keywords are the > opposite. > - easy way to configure shared mailboxes to be accessed via imapc > backend, which would allow easy shared mailbox accesses across servers or > simply between two system users in same server. (this may be tricky to > dsync.) > - global ACLs read from a single file supporting wildcards, instead of > multiple different files > - default ACLs for each namespace/storage root (maybe implemented using > the above..) > > Metadata / annotations > ---------------------- > > Add support for server, mailbox and mail annotations. These need to be > dsyncable, so their changes need to be stored in various .log files: > > 1. Per-server metadata. This is similar to subscriptions: Add changes to > dovecot.mailbox.log file, with each entry name a hash of the metadata key > that was changed. > > 2. Per-mailbox metadata. Changes to this belong inside > mailbox_transaction_context, which write the changes to mailbox's > dovecot.index.log files. Each log record contains a list of changed > annotation keys. This gives each change a modseq, and also allows easily > finding out what changes other clients have done, so if a client has done > ENABLE METADATA Dovecot can easily push metadata changes to client by only > reading the dovecot.index.log file. > > 3. Per-mail metadata. This is pretty much equivalent to per-mailbox > metadata, except changes are associated to specific message UIDs. > > The permanent storage is in dict. The dict keys have components: > - priv/ vs. shared/ for specifying private vs. shared metadata > - server/ vs mailbox// vs. mail// > - the metadata key name > > This would be a good time to improve the dict configuration to allow > things like: > - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a > file, while the rest goes to sql) > - allow sql dict to be used in more relational way, so that mail > annotations could be stored with tables: mailbox (id, guid) and > mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid > everywhere. > > Things to think through: > - How to handle quota? Probably needs to be different from regular mail > quota. Probably some per-user "metadata quota bytes" counter/limit. > - Dict lookups should be done asynchronously and prefetched as much as > possible. For per-mail annotation lookups mail_alloc() needs to include a > list of annotations that are wanted. > > Configuration > ------------- > > Copy all mail settings to namespaces, so it'll be possible to use > per-namespace mailbox settings. Especially important for imapc_* settings, > but can be useful for others as well. Those settings that aren't explicitly > defined in the namespace will use the global defaults. (Should doveconf -a > show all of these values, or simply the explicitly set values?) > > Get rid of *.conf.ext files. Make everything part of dovecot.conf, so > doveconf -n outputs ALL of the configuration. There are mainly 3 config > files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. > The dict-sql is something I think needs a bigger redesign (mentioned above > in "Metadata" section), but the sql/ldap auth configs could be merged. One > way could be: > > sql_db sqlmails { > # most settings from dovecot-sql.conf.ext, except for queries > driver = mysql > connect = ... > } > > ldap_db ldapmails { > # most settings from dovecot-ldap.conf.ext, except attributes/filters > } > > passdb { > driver = sql > db = sqlmails > sql_query = select password from users where username = '%u' > } > passdb { > driver = ldap > db = ldapmails > ldap_attributes { > password = %{ldap:userPassword} > } > ldap_filter = ... > } > > The sql_db {} and ldap_db {} would be generic enough to be used everywhere > (e.g. dict-sql), not just for passdb/userdb. > > Some problems: > - Similar to the per-namespace mail settings, doveconf -a would output > all sql_query, ldap_attributes, ldap_filter, etc. settings for all > passdbs/userdbs. Perhaps a similar solution? > - The database configs contain passwords, so they should be readable only > by root. This makes running dovecot-lda and maybe doveadm difficult, since > they fail at "permission denied" when trying to open the config. There are > probably only two solutions: a) The db configs need to be !include_try'd > or b) the configs can be world-readable, but only passwords are placed to > only-root-readable files by using "password = > IMAP state saving/restoring > --------------------------- > > IMAP connections are often long running. Problems with this: > > 1. Currently each connection requires a separate process (at least to work > reliably), which means each connection also uses quite a lot of memory even > when they aren't doing anything for a long time. > 2. Some clients don't handle lost connections very nicely. So Dovecot > can't be upgraded without causing some user annoyance. Also in a cluster if > you want to bring down one server, the connections have to be disconnected > before they can be moved to another server. > > If IMAP session state could be reliably saved and later restored to > another process, both of the above problems could be avoided entirely. > Typically when a connection is IDLEing there are really just 4 things that > need to be remembered: username, selected mailbox name, its UIDVALIDITY and > HIGHESTMODSEQ. With this information the IMAP session can be fully restored > in another process without losing any state. So, what we could do is: > > 1. When an IMAP connection has bee IDLEing for a while (configurable > initial time, could be dynamically adjusted): > - move the IMAP state and the connection fd to imap-idle process > - the old imap process is destroyed > - imap-idle process can handle lots of IMAP connections > - imap-idle process also uses inotify/etc. to watch for changes in the > specified mailbox > - if any mailbox changes happen or IMAP client sends a command, start up > a new imap process, restore the state and continue from where we left off > - This could save quite a lot of memory at the expense of some CPU usage > > 2. Dovecot proxy <-> backend protocol could be improved to support moving > connection to another backend. Possibly using a separate control connection > to avoid making the proxying less efficient in normal operation. > > 3. When restarting Dovecot, move all the connections to a process that > keeps the connections open for a while. When Dovecot starts up, create imap > processes back to the connections. This allows changing configuration for > existing client connections (which sometimes may be bad! need to add checks > against client-visible config conflicts), upgrading Dovecot, etc. without > being visible to clients. The only problem is SSL connections: OpenSSL > doens't provide a way to save/restore state, so either you need to set > shutdown_clients=no (and possibly keep some imap-login processes doing SSL > proxying for a long time), or SSL connections need to be killed. Of course > the SSL handling could be outsourced to some other software/hardware > outside Dovecot. > > The IMAP state saving isn't always easy. Initially it could be implemented > only for the simple cases (which are a majority) and later extended to > cover more. > > IMAP extensions > --------------- > > - CATENATE is already implemented by Stephan > - URLAUTH is also planned to be implemented, somewhat differently than in > Apple's patch. The idea is to create a separate imap-urlauth service that > provides extra security. > - NOTIFY extension could be implemented efficiently using mailbox list > indexes, which already exists in v2.1. > - FILTERS extension can be easily implemented once METADATA is implemented > - There are also other missing extensions, but they're probably less > important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, > CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. > > Backups > ------- > > Filesystem based backups have worked well enough with Dovecot in the past. > But with new features like single instance storage it's becoming more > difficult. There's no 100% consistent way to even get filesystem level > backups with SIS enabled, because deleting both the message file and its > attachment files can't be done atomically (although usually this isn't a > real problem). Restoring SIS mails is more difficult though, first you need > to restore the dbox mail files and then you need to figure out what > attachment files from SIS need to be restored, and finally you'll need to > do doveadm import to put them into their final destination. > > I don't have much experience with backup software, but other people in my > company do. The initial idea is to implement a Dovecot backup agent to one > (commercial) backup software, which allows doing online backups and > restoring mails one user/mailbox/mail at a time. I don't know the details > yet how exactly this is going to be implemented, but the basic plan is > probably to implement a "backup" mail storage backend, which is a > PostgreSQL pg_dump-like flat file containing mails from all mailboxes. > doveadm backup/import can then export/import this format via stdout/stdin. > Incremental backups could possibly be done by giving a timestamp of > previous backup run (I'm not sure about this yet). > > Once I've managed to implement the first fully functional backup agent, it > should become clearer how to implement it to other backup solutions. > > Random things > ------------- > > - dovecot.index.cache file writing is too complex, should be simplified > - Enable auth_debug[_passwords]=yes on-the-fly for some specific > users/IPs via doveadm > - Optimize virtual mailboxes using mailbox list indexes. It wouldn't > anymore need to keep all the backend mailboxes' index files open. > - Would be nice to go forward with supporting key-value databases as mail > storage backends. Timo, I know you mentioned you would cover this in a coming post, but we were curious what the new dsync replication will be capable of. Would it monitor changes to mailboxes and push automatic replication to the remote mail store, and if this is the case could it be an N-way replication setup in which any host in a cluster can participate in the replication? Do you consider this to be a high availability solution? Thanks, Michael From tss at iki.fi Wed Feb 15 05:46:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 05:46:35 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: On 13.2.2012, at 16.40, Timo Sirainen wrote: > On 13.2.2012, at 16.16, Christoph Bu?enius wrote: > >> when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. >> >> My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. >> - Is there some way to make dsync notice the hard links? (I used >> dovecot 2.0.17) > > It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 2890 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:07:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:07:55 +0200 Subject: [Dovecot] v2.1.rc7 released Message-ID: http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz.sig Okay, one more RC, because I needed a new feature implemented and forgot I wasn't supposed to do any changes anymore before v2.1.0. After that I figured I'll just do a few more changes and release RC7. So, v2.1.0 will probably be released within 1-2 days. So, the one new feature is: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. Also (not really worth mentioning in NEWS) if you now connect to imap/pop3 port when auth process is misconfigured and dies, instead of hanging the client for 5 mins it gets disconnected pretty much immediately. From ybhu at hk1.ibm.com Wed Feb 15 06:13:18 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Wed, 15 Feb 2012 12:13:18 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: I can generate the UID once I create the uidlist lock, and the UID is incremental globally. Some questions: 1, Whether I can write the mail to tmp/ after creating dovecot-uidlist.lock? 2, I did the 5 steps manually, found that new mail is set to OLD flag, so that when SELECT command, it told there was no NEW mail. Looks like need to change code in maildir_uidlist_sync_next_uid()? 3, About the dovecot-uidlist.lock file creation and deletion, I checked the code, there are functions in dovecot like maildir_uidlist_lock_timeout and maildir_uidlist_unlock. If I did that via calling these functions, there are lots of extra codes need to call(like auth, mailbox open...). So the question is if I do that in my codes, how to handle the race condition correctly? (At least there would be other sessions want to lock the files also / already locked it while I want to create the dovecot-uidlist.lock) Thanks. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 10:19 Subject AM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 14.2.2012, at 5.31, Andy YB Hu wrote: > Now our system will take dovecot as the whole imap back end, and keep the > LDA of ours, But our system has our own UID generation mechanism (in > dovecot it's incremental from 1), which will be done while local delivery, > So when local deliveries, it will first generate the UID, move the mail > into maildir, then synchronize the mailbox index, which is part of dovecot > LDA does. The question is what API or something else about index > synchronization I can integrate into our own LDA program easily? Seems > mailbox_copy() would be the possibility? If you want it to be free of race conditions: 1. Write the mail to tmp/ 2. Create dovecot-uidlist.lock 3. Generate the UID and move the mail to new/ or cur/ 4. Append a line to dovecot-uidlist with your UID and the Maildir filename 5. Delete dovecot-uidlist.lock Although it sounds like the UID is generated before you can even create uidlist lock? That means IMAP client can at the same time generate a conflicting UID with APPEND/COPY. Also remember that UIDs must always be increasing. If you try to add a new message with lower UID then Dovecot won't like it, and even if you fix that, some IMAP clients may never see the mail. Anyway, once you have uidlist updated you can do indexing/caching with e.g. "doveadm index" command in v2.0. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09089.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Wed Feb 15 06:25:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 06:25:29 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h From as at 3a.pl Wed Feb 15 09:26:28 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 08:26:28 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> Message-ID: <4F3B5E24.2080407@3a.pl> On 15.02.2012 02:55, Timo Sirainen wrote: > On 15.2.2012, at 3.54, Timo Sirainen wrote: > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > Oh, and you could actually already do this with a doveadm script: [cut] I'm using something like this, but directly on the dbase level. Unfortunately such operation is quite db intensive, so I'm looking for a simpler solution. Is there a way to add domainname as another collumn in dbase? This will simplify the SQL query and the GROUP BY could be used. > Of course it would be nice if Dovecot internally supported this. I was under impression, that dovecot does support this. In examples one can see: /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: # Multiple quota roots are also possible, for example this gives each user # their own 100MB quota and one shared 1GB quota within the domain: plugin { #quota = dict:user::proxy::quota #quota2 = dict:domain:%d:proxy::quota_domain } Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): "If username is left empty, the logged in username is used (this is typically what you want). Another useful username is '%d' for supporting domain-wide quotas." -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 09:47:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 08:47:22 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <4F3B630A.4070305@schetterer.org> Am 15.02.2012 08:26, schrieb Adam Szpakowski: > On 15.02.2012 02:55, Timo Sirainen wrote: >> On 15.2.2012, at 3.54, Timo Sirainen wrote: >> >>> Domain-quota support in Dovecot is only partial. Recalculating domain >>> quota would require more code. Last time when someone asked about >>> this I wasn't even sure how it could be implemented, but nowadays it >>> actually would be possible to do with: >> Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. > > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. > >> Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf: > > # Multiple quota roots are also possible, for example this gives each user > # their own 100MB quota and one shared 1GB quota within the domain: > plugin { > #quota = dict:user::proxy::quota > #quota2 = dict:domain:%d:proxy::quota_domain > } > > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): > > "If username is left empty, the logged in username is used (this is > typically what you want). Another useful username is '%d' for supporting > domain-wide quotas." > > Hi i am not sure what are trying to goal with domain quota, if you setup your accounts i.e with postfixadmin etc then its up to the gui logic , stopping postmasters to setup more accounts ( with quotas ) as you ( the superadmin ) want. For everything else user quotas usally are enough thinkable maybe calculation of all users quotas in dict from one domain in a sql postfix query table to stop deliver in more mail, but i dont think this makes real sense sorry if i missunderstand your question.. -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:07:04 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:07:04 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B630A.4070305@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> Message-ID: <4F3B67A8.2030109@3a.pl> On 15.02.2012 08:47, Robert Schetterer wrote: > Hi i am not sure what are trying to goal > with domain quota, if you setup your accounts i.e with postfixadmin etc > then its up to the gui logic , stopping postmasters to setup more > accounts ( with quotas ) as you ( the superadmin ) want. My fault, I should add some "big picture" info. We are using admin panel based on iRedMail Panel, and want to show quota usage on the domain level. Quota enforcing is only on user level and this part works nicely. What I need is to have in dbase live data which allows me to get quota usage on the domain level to show it in the admin panel. The problem is, that this operation must not by db intensive. Getting entire db and the processing it with some script is fine for daily reports but not for "live" view. My example with doveadm for quota recalc was only a way to show how the problem can be triggered. We are not using doveadm for this purpose. > For everything else user quotas usally are enough > > thinkable maybe calculation of all users quotas in dict from one domain > in a sql postfix query table to stop deliver in more mail, but i dont > think this makes real sense In future the quota enforcement on domain level could be a nice add-on, but for now it is not so important for us. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 10:10:27 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:10:27 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F246339.708@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> Message-ID: <4F3B6873.2010601@schetterer.org> Am 28.01.2012 22:06, schrieb Robert Schetterer: > Am 28.01.2012 21:07, schrieb Timo Sirainen: >> On 13.1.2012, at 20.29, Mark Moseley wrote: >> >>> If there are multiple hosts, it seems like the most robust thing to do >>> would be to exhaust the existing connections and if none of those >>> succeed, then start a new connection to one of them. It will probably >>> result in much more convoluted logic but it'd probably match better >>> what people expect from a retry. >> >> Done: http://hg.dovecot.org/dovecot-2.0/rev/4e7676b890f1 >> > > Hi Timo > > doc/example-config/dovecot-sql.conf.ext > from hg > has something like > > # Database connection string. This is driver-specific setting. > # HA / round-robin load-balancing is supported by giving multiple host > # settings, like: host=sql1.host.org host=sql2.host.org > > but i dont find it in > http://wiki2.dovecot.org/AuthDatabase/SQL > > Hi Timo,just to make sure i have an extra dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) connect = host=192.168.123.150 dbname=.. user=... password=... map { pattern = priv/quota/storage table = quota2 username_field = username value_field = bytes } map { pattern = priv/quota/messages table = quota2 username_field = username value_field = messages } is it possible to have i.e connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... password=... there too? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 10:19:57 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 09:19:57 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <4F3B6AAD.7060901@schetterer.org> Am 15.02.2012 09:07, schrieb Adam Szpakowski: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. as workaround it should be possible to calculate filllevel by domain with some bash script and cron sending an info mail to you might be not exact, but enough to see whats going on ( depends on your real domain/mailbox setup...maildir etc) i.e like du -sch schetterer.org/ 1,7G schetterer.org/ or du -sch schetterer.org/* 24K schetterer.org/.... at schetterer.org 431M schetterer.org/... at schetterer.org 1,3G schetterer.org/.... at schetterer.org 1,7G sum perhaps the same works with getting it out from sql > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From as at 3a.pl Wed Feb 15 10:37:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 09:37:02 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B6AAD.7060901@schetterer.org> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> <4F3B6AAD.7060901@schetterer.org> Message-ID: <4F3B6EAE.2010201@3a.pl> On 15.02.2012 09:19, Robert Schetterer wrote: > as workaround it should be possible to calculate filllevel > by domain with some bash script and cron sending an info mail to you > might be not exact, but enough to see whats going on ( depends on your > real domain/mailbox setup...maildir etc) [cut] I'm affraid that this is not a viable option. It should be done quite often to achieve "live data" feel and the amount of I/O to do so on storage/filesystem level is staggering. Good enough solution will be to have an additional column in db table with domain for each quota record. This will allow us to do something like this: SELECT domain, sum(bytes) as sum_bytes, sum(messages) as sum_messages FROM used_quota GROUP BY domain The missing domain info can be updated regularly via some cron script. This could be a workable solution. -- Adam Szpakowski From busseniu at in.tum.de Wed Feb 15 10:51:26 2012 From: busseniu at in.tum.de (=?windows-1252?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 09:51:26 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B720E.6010705@in.tum.de> Hi, thank you for the reply. On 02/15/2012 03:35 AM, Timo Sirainen wrote: > On 14.2.2012, at 17.04, Christoph Bu?enius wrote: > >> our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: >> >> Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped >> Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > > This indicate some kind of a connection leak.. There shouldn't be so many connections. Each login process is supposed to create only one connection to anvil and one connection to ipc. How many open fds do you see in those processes currently? At the moment, there are - 1118 imap-login processes, - 1136 fds in anvil according to lsof - 1123 connections to anvil according to its proctitle - 1123 fds in ipc - 1111 connections to ipc > >> I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. > > The number of open files is so large, because there are at least 200k fds used for connections to anvil and ipc processes. > Do you see some error message also about "Too many open files", Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files - Twice: config: Error: net_accept() failed: Too many open files - Twice: ipc: Error: net_accept() failed: Too many open files - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped (Dovecot has been restarted once during this hour.) > or why do you think the number of open files was the problem? The default ulimit for open files is 1024 on this Ubuntu system (I should have noted that). The problems (log messages and "doveadm proxy list" error, as described in my last mail) appeared when there were about 1000 connections. I restarted dovecot a couple of times, and each time the problems re-appeared at ~1000 connections. Once I put "ulimit -n 6000" in the init script, the problems disappeared. I am pretty sure that this is what fixed everything for us. Since there have been no log messages about client_limit after we increased the ulimit, I think we can assume that the client_limit has never been the problem. This is why I reported the log message. As the ulimit was 1024 in the beginning, I guess there is no way the connections to anvil or ipc could possibly have reached 100.000. In fact, I never saw such a high number in the proctitle. From my very naive understanding of the source code (master/service-monitor.c): if (service->last_drop_warning + SERVICE_DROP_WARN_INTERVAL_SECS < ioloop_time) { service->last_drop_warning = ioloop_time; limit = service->process_limit > 1 ? service->process_limit : service->client_limit; i_warning("service(%s): %s (%u) reached, " "client connections are being dropped", service->set->name, service->process_limit > 1 ? "process_limit" : "client_limit", limit); } The actual number of open connections is not explicitly compared against client_limit before the warning. Maybe this leads to wrong numbers in the warning? (This part of the code looks similar in hg 2.0 and 2.1., though I have only used 2.0.18.) >> The "doveadm proxy list" would fail with an error message: >> >> $ sudo doveadm proxy list >> doveadm(root): Error: LIST failed: Disconnected >> Segmentation fault >> >> The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. > > http://hg.dovecot.org/dovecot-2.0/rev/c402fb407501 fixes the crash I think. Thanks? however I am afraid I have not been able to reproduce the segfault since yesterday. Cheers, Christoph From przemek.orzechowski at makolab.pl Wed Feb 15 11:09:47 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Wed, 15 Feb 2012 10:09:47 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B67A8.2030109@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <4F3B630A.4070305@schetterer.org> <4F3B67A8.2030109@3a.pl> Message-ID: <9b9cf0ba2ada5b598eae8b4fc42e75e8@localhost> Hi Im not an expert in Dovecot but what prohibits you adding a field to the table Dovecot will not use it thats all Im not sure what version of database You are using but postgres,oracle and never versions of mysql have something called views (could be used to hide the additional column from dovecot if necessary) and as far as i remember triggers that could be launched on insert, update to populate that additional column Other solution (not sure if applicable for Your needs) is to enable file system quota without actually setting up any quotas/group quotas (all users from single domain must belong to same group tho) this allows quick disk usage check without excess disk io On Wed, 15 Feb 2012 09:07:04 +0100, Adam Szpakowski wrote: > On 15.02.2012 08:47, Robert Schetterer wrote: >> Hi i am not sure what are trying to goal >> with domain quota, if you setup your accounts i.e with postfixadmin etc >> then its up to the gui logic , stopping postmasters to setup more >> accounts ( with quotas ) as you ( the superadmin ) want. > My fault, I should add some "big picture" info. > We are using admin panel based on iRedMail Panel, and want to show quota > usage on the domain level. > Quota enforcing is only on user level and this part works nicely. > What I need is to have in dbase live data which allows me to get quota > usage on the domain level to show it in the admin panel. > The problem is, that this operation must not by db intensive. Getting > entire db and the processing it with some script is fine for daily > reports but not for "live" view. > > My example with doveadm for quota recalc was only a way to show how the > problem can be triggered. We are not using doveadm for this purpose. > >> For everything else user quotas usally are enough >> >> thinkable maybe calculation of all users quotas in dict from one domain >> in a sql postfix query table to stop deliver in more mail, but i dont >> think this makes real sense > In future the quota enforcement on domain level could be a nice add-on, > but for now it is not so important for us. -- Pozdrawiam, Przemys?aw Orzechowski Administrator Sieci/Network Administrator e: przemek.orzechowski at makolab.net t: +48 42 683 74 96 MakoLab S.A. ul. Demokratyczna 46, 93-430 ??d? www.makolab.pl Sp??ka zarejestrowana w Krajowym Rejestrze S?dowym przez S?d Rejonowy dla ?odzi - ?r?dmie?cia w ?odzi XX Wydzia? Krajowego Rejestru S?dowego pod numerem KRS 0000289179. Wysoko?? kapita?u zak?adowego wynosi 707 473 PLN. Kapita? zak?adowy zosta? wp?acony w ca?o?ci. NIP 7250015526, REGON 471343117 Wiadomo?? ta jest przeznaczona jedynie dla osoby lub podmiotu b?d?cego jej adresatem i mo?e zawiera? poufne lub uprzywilejowane informacje. Zakazane jest przegl?danie, przesy?anie, rozpowszechnianie lub inne wykorzystywanie tych informacji, jak r?wnie? podejmowanie dzia?a? na ich podstawie, przez osoby lub podmioty inne ni? zamierzony adresat. Je?li otrzymali Pa?stwo t? wiadomo?? przez pomy?k?, prosimy o poinformowanie nadawcy i usuni?cie jej z komputera. The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please notify the sender and delete the material from your computer. From busseniu at in.tum.de Wed Feb 15 11:50:15 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 10:50:15 +0100 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> Message-ID: <4F3B7FD7.1000109@in.tum.de> On 15.02.2012 03:37, Timo Sirainen wrote: > On 15.2.2012, at 4.35, Timo Sirainen wrote: > Oh, this is most likely related because you're not running in high-performance login process mode, unlike pretty much everyone else running proxies: http://wiki2.dovecot.org/LoginProcess Since yesterday all our users (2000 active) are being routed over this proxy, and we have not experienced any performance problems. The system load is below 0.1 most of the time. So afaict there is no reason to use high-performance mode. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 15 12:12:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 15 Feb 2012 11:12:35 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F3B8513.2040806@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. Thanks. I guess it would be wise to upgrade our new Dovecot mail store to 2.1 before we migrate all our Courier maildir users to it... Cheers, Chnristoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Wed Feb 15 12:30:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 12:30:14 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: <4F3B720E.6010705@in.tum.de> References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 10.51, Christoph Bu?enius wrote: >> Do you see some error message also about "Too many open files", > > Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: > > - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files > - Twice: config: Error: net_accept() failed: Too many open files > - Twice: ipc: Error: net_accept() failed: Too many open files > - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped > - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. From janfrode at tanso.net Wed Feb 15 13:52:09 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 15 Feb 2012 12:52:09 +0100 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: References: <20120214193442.GA5156@dibs.tanso.net> Message-ID: <20120215115209.GA21047@dibs.tanso.net> On Wed, Feb 15, 2012 at 03:49:21AM +0200, Timo Sirainen wrote: > > > > Looking at the timestamps in the filesystem I see that the users home > > directory wasn't created before switcing to imap. > > > > Is this a know problem? > > Probably again a bug in your specific Dovecot version. :) I remember doing fixes related to this (not entirely sure if it was only for v2.1). > Is it maybe changeset 11683:148fccbe9f32 you remeber: - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX This was just the updated to the todo-list, but I can't see what the fix was. Also, if it was just occationally failing, it might not be that critical.. So far it's only happended for one user for the last 36 hours, so either it's only occationally failing, or the other new users are visiting webmail/imap before pop. -jf From tss at iki.fi Wed Feb 15 15:01:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:01:05 +0200 Subject: [Dovecot] pop3 not autocreating directory structure In-Reply-To: <20120215115209.GA21047@dibs.tanso.net> References: <20120214193442.GA5156@dibs.tanso.net> <20120215115209.GA21047@dibs.tanso.net> Message-ID: <1329310865.11500.170.camel@innu> On Wed, 2012-02-15 at 12:52 +0100, Jan-Frode Myklebust wrote: > Is it maybe changeset 11683:148fccbe9f32 you remeber: > > - - maildir: sometimes rm -rf Maildir;imaptest logout=0 gives > - Error: Opening INBOX failed: Mailbox doesn't exist: INBOX That's probably maildir-specific. But dboxes have had same problems. > This was just the updated to the todo-list, but I can't see what the fix > was. Also, if it was just occationally failing, it might not be that > critical.. So far it's only happended for one user for the last 36 > hours, so either it's only occationally failing, or the other new users > are visiting webmail/imap before pop. Yes, it's most likely only temporary and the second login works. From tss at iki.fi Wed Feb 15 15:05:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:05:38 +0200 Subject: [Dovecot] domain quota in sql In-Reply-To: <4F3B5E24.2080407@3a.pl> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> Message-ID: <1329311138.11500.175.camel@innu> On Wed, 2012-02-15 at 08:26 +0100, Adam Szpakowski wrote: > On 15.02.2012 02:55, Timo Sirainen wrote: > > On 15.2.2012, at 3.54, Timo Sirainen wrote: > > > >> Domain-quota support in Dovecot is only partial. Recalculating domain quota would require more code. Last time when someone asked about this I wasn't even sure how it could be implemented, but nowadays it actually would be possible to do with: > > Oh, and you could actually already do this with a doveadm script: > [cut] > I'm using something like this, but directly on the dbase level. > Unfortunately such operation is quite db intensive, so I'm looking for a > simpler solution. Isn't it basically one table scan? To do UPDATE .. WHERE username = '% @domain' > Is there a way to add domainname as another collumn in dbase? This will > simplify the SQL query and the GROUP BY could be used. Not currently. Perhaps for v2.2 dict-sql redesign. > > Of course it would be nice if Dovecot internally supported this. > I was under impression, that dovecot does support this. In examples one > can see: > > /usr/share/doc/dovecot-core/example-config/conf.d/90-quota.conf .. > Also on dovecot2 wiki we can see (http://wiki2.dovecot.org/Quota/Dict): Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan a problem anyway? Does your domain quota value keep breaking? In normal operation it should work fine. From tss at iki.fi Wed Feb 15 15:07:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 15:07:11 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3B6873.2010601@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> Message-ID: <1329311231.11500.177.camel@innu> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: > Hi Timo,just to make sure > i have an extra > dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) > > connect = host=192.168.123.150 dbname=.. user=... password=... .. > is it possible to have i.e > > connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... > password=... > > there too? Assuming your SQL database is configured to do synchronous multi-master replication, sure. From r.zimmermann at siegnetz.de Wed Feb 15 16:55:28 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 15:55:28 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 Message-ID: <4F3BC760.7080202@siegnetz.de> Hi all, today I have compiled Dovecot-2.0.18 with Dovecot-Pigeonwhole-0.2.6. Now I have following Problem with sieve: ------------------------%<--------------------------------------------- main_script: line 249: error: included personal script 'vacation.sieve' does not exist. main_script: error: validation failed. ------------------------%<--------------------------------------------- I have a definition for 'sieve_dir' and the sieve script 'vacation.sieve' exits in the folder. # Dovecot Sieve Parameters plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same configuration I can define include :personal "vacation.sieve"; in my main sieve script. All works fine. Greets Ralf From tss at iki.fi Wed Feb 15 17:13:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 17:13:33 +0200 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6 In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <96E451B3-17B0-4C57-AAF7-D4D7D0BEF653@iki.fi> On 15.2.2012, at 16.55, Ralf Zimmermann wrote: > main_script: line 249: error: included personal script 'vacation.sieve' > does not exist. .. > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; include :personal "vacation"; From dmiller at amfes.com Wed Feb 15 17:14:34 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Wed, 15 Feb 2012 07:14:34 -0800 Subject: [Dovecot] 2.1.rc5 Cannot delete folders from trash In-Reply-To: <4F3AFF83.4020707@es2eng.com> References: <20120212111808.GA27677@herakles.constabel-it.de> <743ee2bec4494916c00f956613a4fb4e@imt-systems.com> <20120212132719.GB27677@herakles.constabel-it.de> <4FF0AF15-269F-4B22-A8D4-F74661F67D02@iki.fi> <4F39D654.2070509@amfes.com> <4F3ABEE0.6010702@es2eng.com> <4F3AC6A2.5010605@amfes.com> <4F3AFF83.4020707@es2eng.com> Message-ID: On 2/14/2012 4:42 PM, Willie Gillespie wrote: > On 02/14/2012 01:40 PM, Daniel L. Miller wrote: >> On 2/14/2012 12:06 PM, Willie Gillespie wrote: >>> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >>>> I'm fighting the same issue. With the current release (2.1rc6) no >>>> errors >>>> are logged - but I'm having problems deleted folders from trash. I've >>>> enabled tb-extra-mailbox-sep. Two folders that were previously in >>>> Trash >>>> were successfully removed. I deleted four more - after several >>>> Thunderbird restarts, compacts, refreshes, I have killed three of >>>> them - >>>> there is still one folder that won't "die". Thunderbird reports the >>>> mailbox doesn't exist, yet it continues to appear. >>> >>> Out of curiosity -- does the mailbox actually exist if you look on the >>> Dovecot server? >>> Secondly, is it listed in your subscription file? >> >> At one time - yes they were. After further playing, they do not exist in >> the on-disk folder structure, nor in the subscription file. But still >> show in the client - with errors. > > Sounds like a Thunderbird bug then. With Thunderbird shut down, you > could browse into the profile. > ~\.thunderbird or > C:\Users\name\AppData\Roaming\Thunderbird\whatever\ImapMail\server.name > > Find the mailbox that is not supposed to be there and move/delete it. > The filename will usually just be the mailbox name. A .msf file is > Thunderbird's index file. It will create .sbd folders for any > sub-mailboxes. I tried going in via squirrelmail - saw no subfolders under trash. Went back in with Thunderbird - all gone. -- Daniel From tlx at leuxner.net Wed Feb 15 17:14:55 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Wed, 15 Feb 2012 16:14:55 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <4F3BC760.7080202@siegnetz.de> References: <4F3BC760.7080202@siegnetz.de> Message-ID: <20120215151455.GA1896@nihlus.leuxner.net> On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: > If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same > configuration I can define > > include :personal "vacation.sieve"; > > in my main sieve script. All works fine. Changelog v0.2.6: * This release fixes unintentional behavior of the include extension. Included script names with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. Try: include :personal "vacation"; Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From r.zimmermann at siegnetz.de Wed Feb 15 18:06:07 2012 From: r.zimmermann at siegnetz.de (Ralf Zimmermann) Date: Wed, 15 Feb 2012 17:06:07 +0100 Subject: [Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6lL In-Reply-To: <20120215151455.GA1896@nihlus.leuxner.net> References: <4F3BC760.7080202@siegnetz.de> <20120215151455.GA1896@nihlus.leuxner.net> Message-ID: <4F3BD7EF.2070204@siegnetz.de> Thanks Thomas and Timo for the fast answer. Next time I'll read the News file ;-) It' works now. On 02/15/2012 04:14 PM, Thomas Leuxner wrote: > On Wed, Feb 15, 2012 at 03:55:28PM +0100, Ralf Zimmermann wrote: >> If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same >> configuration I can define >> >> include :personal "vacation.sieve"; >> >> in my main sieve script. All works fine. > > Changelog v0.2.6: > > * This release fixes unintentional behavior of the include extension. > Included script names with a name like "name.sieve" would implicitly > map to a script file called "name.sieve" and not "name.sieve.sieve". > Keep in mind that the .sieve file extension has no meaning from > within the Sieve language. A Sieve script is always stored with an > appended .sieve file extension, also when the name already ends with > a .sieve suffix. > > Try: > > include :personal "vacation"; > > Regards > Thomas Regards Ralf From robert at schetterer.org Wed Feb 15 18:10:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 17:10:22 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <1329311231.11500.177.camel@innu> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> Message-ID: <4F3BD8EE.5010007@schetterer.org> Am 15.02.2012 14:07, schrieb Timo Sirainen: > On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >> Hi Timo,just to make sure >> i have an extra >> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >> >> connect = host=192.168.123.150 dbname=.. user=... password=... > .. >> is it possible to have i.e >> >> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >> password=... >> >> there too? > > Assuming your SQL database is configured to do synchronous multi-master > replication, sure. > > jep ,thx Timo -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From robert at schetterer.org Wed Feb 15 20:22:21 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 19:22:21 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BD8EE.5010007@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> Message-ID: <4F3BF7DD.8080302@schetterer.org> Am 15.02.2012 17:10, schrieb Robert Schetterer: > Am 15.02.2012 14:07, schrieb Timo Sirainen: >> On Wed, 2012-02-15 at 09:10 +0100, Robert Schetterer wrote: >>> Hi Timo,just to make sure >>> i have an extra >>> dovecot-dict-quota-sql.conf.ext ( dove 2.0.18 ) >>> >>> connect = host=192.168.123.150 dbname=.. user=... password=... >> .. >>> is it possible to have i.e >>> >>> connect = host=192.168.123.150 host=127.0.0.1 dbname=.. user=... >>> password=... >>> >>> there too? >> >> Assuming your SQL database is configured to do synchronous multi-master >> replication, sure. >> >> > jep ,thx Timo > Hi Timo, sorry for some more question what is the default behave if dict quota over sql cant be reached i.e with lmtp something like log warning and deliver anyway ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From tss at iki.fi Wed Feb 15 20:43:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:43:10 +0200 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <4F3BF7DD.8080302@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> Message-ID: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> On 15.2.2012, at 20.22, Robert Schetterer wrote: > sorry for some more question > what is the default behave if dict quota over sql cant be reached > i.e with lmtp > > something like log warning and deliver anyway ? I think it tempfails. Try. From tss at iki.fi Wed Feb 15 20:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:51:59 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On 15.2.2012, at 5.08, wrote: > I know you mentioned you would cover this in a coming post, but we were > curious what the new dsync replication will be capable of. Would it > monitor changes to mailboxes and push automatic replication to the remote > mail store, Yes. > and if this is the case could it be an N-way replication setup > in which any host in a cluster can participate in the replication? Initially 2-way, but I don't think anything prevents it being N-way. > Do you consider this to be a high availability solution? The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. The other possibility is to run Dovecot in two completely separate data centers and replicate through ssh. Here are more possibilities for how to do HA, but some of them also have downsides.. dovecot.fi mails are actually done this way, and can be accessed from either server at any time. I've been thinking about soon making half of my clients use one server and half the other one to see if I can find any dsync bugs (I've always 3-4 IMAP clients connected). From tss at iki.fi Wed Feb 15 20:54:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 20:54:01 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: <5260D0A9-A72F-48CE-8D42-50A2E818E7BA@iki.fi> On 15.2.2012, at 20.51, Timo Sirainen wrote: > The initial version is really about doing all of this with NFS. In NFS setup if two replaced storages two replicated storages.. ugh. > are both mounted and the primary storage dies, Dovecot will start using the replica. So that's HA. From moseleymark at gmail.com Wed Feb 15 21:02:42 2012 From: moseleymark at gmail.com (Mark Moseley) Date: Wed, 15 Feb 2012 11:02:42 -0800 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: On Mon, Feb 13, 2012 at 3:47 AM, Timo Sirainen wrote: > Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. > > Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: > > ?- 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. > ?- 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) > ?- 3. priority: Things other people want to get implemented. > > There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. > Not to beat a dead horse, but the ability to use remote directors might be interesting. It'd make moving into a director setup probably a bit more easy. Then any server could proxy to the backend servers, but without losing the advantage of director-based locality. If a box sees one of its own IPs in the director_servers list, then it knows it's part of the ring. If it doesn't, then it could contact a randomly selected director IP. From tss at iki.fi Wed Feb 15 21:09:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 15 Feb 2012 21:09:18 +0200 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> Message-ID: <3C05A36D-F6D3-49C6-AEF1-6013E6AA01AF@iki.fi> On 15.2.2012, at 21.02, Mark Moseley wrote: > Not to beat a dead horse, but the ability to use remote directors > might be interesting. It'd make moving into a director setup probably > a bit more easy. Then any server could proxy to the backend servers, > but without losing the advantage of director-based locality. If a box > sees one of its own IPs in the director_servers list, then it knows > it's part of the ring. If it doesn't, then it could contact a randomly > selected director IP. It should already be possible to do that, although not automatically based on looking at your own IP.. Anyway, non-director servers could simply have the passdb return proxy=y host=director-servers, where director-servers expands to a round-robin list of director IPs (Dovecot uses the first one). I guess it would be possible to do this automatically if passdb lookup returns proxy=y but no host (means director isn't enabled), but if director_servers is non-empty one of the IPs would be randomly chosen. A little kludgy though.. From as at 3a.pl Wed Feb 15 21:09:18 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 15 Feb 2012 20:09:18 +0100 Subject: [Dovecot] domain quota in sql In-Reply-To: <1329311138.11500.175.camel@innu> References: <4F3AD22F.5080904@3a.pl> <147CCD39-537F-455A-9B55-212340747969@iki.fi> <762B3C26-9A4B-49F4-BE32-69BED7654240@iki.fi> <4F3B5E24.2080407@3a.pl> <1329311138.11500.175.camel@innu> Message-ID: <4F3C02DE.7080605@3a.pl> On 15.02.2012 14:05, Timo Sirainen wrote: >> Is there a way to add domainname as another collumn in dbase? This will >> simplify the SQL query and the GROUP BY could be used. > Not currently. Perhaps for v2.2 dict-sql redesign. It will be great feature. > Well, yeah, it kind of works, but not 100% :) Anyway, why is the rescan > a problem anyway? Does your domain quota value keep breaking? In normal > operation it should work fine. Domain quota gathering based on config examples works "funny". Its the best description I can get (yes, very descriptive ;) ). There are both entries for users (user at domain) and domains. It is not making sense for me, so I'm assuming that its the missing percentage of the working part ;) Tu sum-up the thread. I went for updating periodically the basic quota table with missing domain names. Working great. Thanks all for suggestions. -- Adam Szpakowski From robert at schetterer.org Wed Feb 15 21:10:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:10:53 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? In-Reply-To: <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> Message-ID: <4F3C033D.9000503@schetterer.org> Am 15.02.2012 19:43, schrieb Timo Sirainen: > On 15.2.2012, at 20.22, Robert Schetterer wrote: > >> sorry for some more question >> what is the default behave if dict quota over sql cant be reached >> i.e with lmtp >> >> something like log warning and deliver anyway ? > > I think it tempfails. Try. hm, test servers arent up recently cant do it on production but i will test before i go try dove 2.1 and report, @list perhaps anyone other having it tested before? > -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 21:30:46 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 14:30:46 -0500 Subject: [Dovecot] 2.1b6 imapc sync error Message-ID: <1329334246.30459.140661037015245@webmail.messagingengine.com> I received this error at the end of a migration from imapc to mdbox. Is it because I wanted doveadm 'sync' instead of 'backup'? I didn't get time to do any testing, but since you're pushing for the release, here it is: kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o imapc_password=****** -o imapc_host=*.*.*.* sync -R -u jsample_dc at mydomain.com imapc:/tmp/imapc dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2011: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox Archives/2012: Not supported dsync(jsample_dc at mydomain.com): Error: Can't update mailbox INBOX: Not supported ...etc, for all mailboxes in the account. This is my userdb entry: jsample_dc at mydomain.com:{***}password:502:6:IMAPC RECIPIENT::/bin/false: kerio2:~ adminuser$ /opt/dovecot/bin/doveconf -n # 2.1.rc6: /opt/dovecot-2.1.rc6/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_debug = yes auth_mechanisms = plain login auth_username_format = %n at mydomain.com auth_verbose = yes disable_plaintext_auth = no instance_name = dovecot-2.1.rc6 listen = 192.168.20.40 mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox mail_plugin_dir = /opt/dovecot/lib/dovecot mail_plugins = fts fts_squat # I know, this is deprecated in 2.1 mail_privileged_group = mail maildir_very_dirty_syncs = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat } postmaster_address = postmaster at mydomain.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = adminuser } } service imap { executable = imap postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = _postfix mode = 0666 user = _postfix } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_ca = References: <1329334246.30459.140661037015245@webmail.messagingengine.com> Message-ID: <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> On 15.2.2012, at 21.30, Metro Domain Admin wrote: > I received this error at the end of a migration from imapc to mdbox. Is > it because I wanted doveadm 'sync' instead of 'backup'? I didn't get > time to do any testing, but since you're pushing for the release, here > it is: > > kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o > imapc_password=****** -o imapc_host=*.*.*.* sync -R -u > jsample_dc at mydomain.com imapc:/tmp/imapc > dsync(jsample_dc at mydomain.com): Error: Can't update mailbox > Archives/2011: Not supported Yes, it's because of "sync". The "Not supported" comes from imapc backend, because Dovecot tried to do some changes to it that weren't possible with IMAP protocol (modseq changes probably). I think the migration still was successful though. From robert at schetterer.org Wed Feb 15 21:38:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 15 Feb 2012 20:38:24 +0100 Subject: [Dovecot] MySQL server has gone away / dict ? tests done In-Reply-To: <4F3C033D.9000503@schetterer.org> References: <02F8EC25-CB7D-4AD1-8ED8-5CB4950B6EC5@iki.fi> <4F246339.708@schetterer.org> <4F3B6873.2010601@schetterer.org> <1329311231.11500.177.camel@innu> <4F3BD8EE.5010007@schetterer.org> <4F3BF7DD.8080302@schetterer.org> <659E9BB9-01DE-486B-AE19-AB56DC05DE98@iki.fi> <4F3C033D.9000503@schetterer.org> Message-ID: <4F3C09B0.8040500@schetterer.org> Am 15.02.2012 20:10, schrieb Robert Schetterer: > Am 15.02.2012 19:43, schrieb Timo Sirainen: >> On 15.2.2012, at 20.22, Robert Schetterer wrote: >> >>> sorry for some more question >>> what is the default behave if dict quota over sql cant be reached >>> i.e with lmtp >>> >>> something like log warning and deliver anyway ? >> >> I think it tempfails. Try. > > hm, test servers arent up recently > cant do it on production > > but i will test before i go try dove 2.1 > and report, @list perhaps anyone other having it tested before? >> > ok the test server was good enough for small tests with 2.0.13 dove private/dovecot-lmtp] said: 451 4.2.0 <.... at ....com> Internal error occurred. Refer to server log for more information. dict: Error: mysql(localhost2): Connect failed to database (postfix): Unknown MySQL server host '...' (1) - waiting for 125 seconds before retry youre right looks like default is tempfail 451 so it stays in postfix queue and is permanent retry after sql comming back , its getting delivered i.e force with Flush Queue so thats good enough for me i think -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From metro_domain_admin at fastmail.fm Wed Feb 15 23:05:52 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Wed, 15 Feb 2012 16:05:52 -0500 Subject: [Dovecot] 2.1b6 imapc sync error In-Reply-To: References: <1329334246.30459.140661037015245@webmail.messagingengine.com> <4E818065-B1DE-42F7-8961-0ACD220D198A@iki.fi> <1329338885.26216.140661037021897@webmail.messagingengine.com> Message-ID: <1329339952.472.140661037052961@webmail.messagingengine.com> On Wed, Feb 15, 2012, at 10:50 PM, Timo Sirainen wrote: > On 15.2.2012, at 22.48, Metro Domain Admin wrote: > > > On Wed, Feb 15, 2012, at 09:36 PM, Timo Sirainen wrote: > >> Yes, it's because of "sync". The "Not supported" comes from imapc > >> backend, because Dovecot tried to do some changes to it that weren't > >> possible with IMAP protocol (modseq changes probably). I think the > >> migration still was successful though. > > > > The migration went beautifully otherwise. But are the errors minor > > enough that I can continue with sync? > > Probably better to continue with "backup" just in case. :) > > > I had the idea that I could sync back to the imapc account, as a > > fallback option (and, presumably I'd want to preserve the /tmp/imapc > > directory for this?). > > That's not something that I've tested, and I don't know how well it works > (if at all). > OK, "backup" it is then. Thanks. From interfasys at gmail.com Thu Feb 16 03:12:07 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 01:12:07 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> Message-ID: <4F3C57E7.8070105@gmail.com> Hello, Here is a more meanigful backtrace. It happens every time I log in, using any imap client. ********************* Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105e0d88 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x00000000104e6eda in autocreate_iter_next.20443 () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #3 0x00000000104dd72e in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #4 0x0000000000416351 in list_namespace_mailboxes.7882 (ctx=0x10a3f158) at cmd-list.c:408 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (struct buffer *) 0x10a04290 mutf7_name = (struct buffer *) 0x10a043e0 name = (const __unknown__ *) 0x10a10040 "INBOX" ret = 1 #5 0x000000000041c2ac in cmd_list_continue (cmd=0x10a3f040) at cmd-list.c:823 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000041c1d0 in cmd_list_full (cmd=0x10a3f040, lsub=false) at cmd-list.c:999 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = (const __unknown__ *) 0x10a04268 "INBOX" patterns_strarr = (const __unknown__ * const *) 0x10a3f1e0 str = (struct buffer *) 0x10a041f0 #7 0x000000000041bc80 in cmd_list (cmd=0x10a3f040) at cmd-list.c:1014 No locals. #8 0x0000000000411264 in command_exec (cmd=0x10a3f040) at imap-commands.c:147 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x000000000040df3c in client_command_input (cmd=0x10a3f040) at imap-client.c:673 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" __FUNCTION__ = "client_command_input" #10 0x000000000040de42 in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:740 size = 279029312 #11 0x000000000040d7b5 in client_handle_input (client=0x10a3e840) at imap-client.c:777 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" __FUNCTION__ = "client_handle_input" #12 0x000000000040d35c in client_input (client=0x10a3e840) at imap-client.c:816 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" __FUNCTION__ = "client_input" #13 0x00000000105cdffa in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #14 0x00000000105c1e16 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #15 0x00000000105c1cc4 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #16 0x00000000105c19ff in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #17 0x000000000041eda9 in main (argc=1, argv=0x7fffffffec60) at main.c:384 set_roots = {0x4230c0, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41ede9 , failure_callback = 0x4190af } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = (const __unknown__ *) 0x0 c = -1 set_roots = {0x4230c0, 0x0} ****************** Cheers, Olivier On 13/02/2012 00:49, Timo Sirainen wrote: > On 13.2.2012, at 2.02, interfaSys s?rl wrote: > >> #0 0x00000000105c64e7 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104b4bf6 in mailbox_list_iter_next () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. >> #2 0x0000000000416b99 in cmd_list_continue.6853 () >> No symbol table info available. >> #3 0x000000000042166d in cmd_list_full () > > This crash looks interesting, but unfortunately the libraries are stripped of debug symbols so I can't really guess why it would crash here. By default Dovecot includes them, so you must have stripped them yourself. Can you get another backtrace with debug symbols enabled? > > Or alternatively can you look at what IMAP command Horde sends that crashes Dovecot? http://wiki2.dovecot.org/Debugging/Rawlog shows one way of recording the IMAP traffic. > > From ybhu at hk1.ibm.com Thu Feb 16 05:59:27 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 11:59:27 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: Tried it, found it could trigger NFS issue. The test is on one machine runs the script to lock the dovecot-uidlist, moving mail into cur/new, append one line to dovecot-uidlist, and unlock it. On another different machine, run one script to SELECT, another script to COPY. Found the similar issue in http://dovecot.org/pipermail/dovecot/2012-February/063539.html Whether it means the dovecot LDA also needs to run on the same server with the backend server? Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/15/2012 12:25 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 15.2.2012, at 6.13, Andy YB Hu wrote: > I can generate the UID once I create the uidlist lock, and the UID is > incremental globally. > > Some questions: > 1, Whether I can write the mail to tmp/ after creating > dovecot-uidlist.lock? Yes, then you'll just keep the lock longer than necessary. > 2, I did the 5 steps manually, found that new mail is set to OLD flag, so > that when SELECT command, it told there was no NEW mail. Looks like need > to change code in maildir_uidlist_sync_next_uid()? I guess that means that the mail didn't get a \Recent flag. Wht Dovecot version is this? Are you saving the mail to new/? > 3, About the dovecot-uidlist.lock file creation and deletion, I checked the > code, there are functions in dovecot like maildir_uidlist_lock_timeout and > maildir_uidlist_unlock. If I did that via calling these functions, there > are lots of extra codes need to call(like auth, mailbox open...). Yes, way too much work to do that way. > So the > question is if I do that in my codes, how to handle the race condition > correctly? (At least there would be other sessions want to lock the files > also / already locked it while I want to create the dovecot-uidlist.lock) It's a typical dotlock file. You can create it with either open(.., O_EXCL) or creating a temp file and trying to link() it to the .lock file. Or you can also directly call the file_dotlock_create() in src/lib/file-dotlock.h -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic09186.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From finn at kirstineslund.dk Thu Feb 16 03:04:52 2012 From: finn at kirstineslund.dk (Finn B) Date: Thu, 16 Feb 2012 02:04:52 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? Message-ID: <4F3C5634.2010906@kirstineslund.dk> Moi Timo / all. I've been using Dovecot on my Qmailtoaster production mailserver for some years - since 1.0.7 and current Version 2.0.10 - works perfectly. Now I am installing a backupsystem and a VM testsystem (2 boxes) - and no matter how and what Version I am installing on these 2 boxes I get below messages whenever I try to test connection with Telnet (localhost 110 / 995). I have tried to install all Dovecot versions from 2.010 to 2.018 as rpms, yums and as source and compiled it. It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I have done many times before) I have tried to copy all my settings from my current production system (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot 2.0.10) - nothing helps. Now I come to think of it I realize that my production system is intel based and the 2 new ones are AMD based and wonder if this may be the case ? Dovecot.log master: Error: Service(auth): command startup failed, throttling log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) pop3-login: Error: Timeout waiting for handshake from auth server. my pid=1858, input bytes=0 Regards, Finn B From tss at iki.fi Thu Feb 16 06:54:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 06:54:10 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> Message-ID: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. From ybhu at hk1.ibm.com Thu Feb 16 08:25:09 2012 From: ybhu at hk1.ibm.com (Andy YB Hu) Date: Thu, 16 Feb 2012 14:25:09 +0800 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: Sounds a little tricky. For backend server, director makes sense. But for LDA, it's also suggested to deploy the LDA on the same machine with backend server? Topologically, it's very possible the machine where LDA is running is separated from one of backend server. I will run the similar tests with pure dovecot LDA, instead of ours to see the result. Timo Sirainen Sent by: To dovecot-bounces at d Dovecot Mailing List ovecot.org cc 02/16/2012 12:54 Subject PM Re: [Dovecot] index synchronization during local delivery Please respond to Dovecot Mailing List On 16.2.2012, at 5.59, Andy YB Hu wrote: > Tried it, found it could trigger NFS issue. .. > Whether it means the dovecot LDA also needs to run on the same server with > the backend server? Yes, the only 100% safe way is to do all simultaneous mailbox access via the same backend server. -------------- next part -------------- A non-text attachment was scrubbed... Name: graycol.gif Type: image/gif Size: 105 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pic02130.gif Type: image/gif Size: 1255 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ecblank.gif Type: image/gif Size: 45 bytes Desc: not available URL: From tss at iki.fi Thu Feb 16 08:29:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 08:29:32 +0200 Subject: [Dovecot] index synchronization during local delivery In-Reply-To: References: <19857FA6-7474-4F75-A32D-1272A760A1B0@iki.fi> <1F468241-3AE1-4664-90AE-FF27E342D6EB@iki.fi> Message-ID: <4CDB78DC-AF5D-4A70-919D-CE0BAA7D4A41@iki.fi> On 16.2.2012, at 8.25, Andy YB Hu wrote: > Sounds a little tricky. For backend server, director makes sense. But for > LDA, it's also suggested to deploy the LDA on the same machine with backend > server? Topologically, it's very possible the machine where LDA is running > is separated from one of backend server. MTA/Antivirus/Antispam is often in a separate backend server, but it can do the actual mail writing by sending the mail to Dovecot via LMTP via director. From pw at wk-serv.de Thu Feb 16 08:54:12 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Thu, 16 Feb 2012 07:54:12 +0100 Subject: [Dovecot] =?utf-8?q?Something_is_strange_-_perhaps_my_AMD_process?= =?utf-8?q?ors_=3F?= In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <79a90b1bae68f2ac4c9f57ed8ca84a99@wk-serv.de> On Thu, 16 Feb 2012 02:04:52 +0100, Finn B wrote: > Now I come to think of it I realize that my production system is > intel > based and the 2 new ones are AMD based and wonder if this may be the > case ? Dovecot 2.0.3 to 2.0.16 works fine on my DualCore Opterons (VMs). Patrick From joshua at hybrid.pl Thu Feb 16 10:41:52 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 09:41:52 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? Message-ID: Hi, I'm using finally dovecot 2.0 after quite a smooth upgrade from 1.0 :) with virtual mailboxes stored in sql database. Sometimes I need to check if a mailbox has been used recently or not. Checking for logins in dovecot logs sometimes means searching through gigabytes of info... I was just wondering if there is any possibility of running another query after successful login - just to fill some extra field like "last_login"? So far my only idea is to replace user/password query with procedure like login(user,password) that would also return a line as required by user/password query. However, some mysql implementations (for example, standard php-mysql) can't handle procedures which return values - so will it work in dovecot? Or maybe some other ideas how to do that? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 11:20:48 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 10:20:48 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: Message-ID: <20120216092048.GA10124@dibs.tanso.net> On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: > > I was just wondering if there is any possibility of running another > query after successful login - just to fill some extra field like > "last_login"? We touch a file in /var/log/activemailaccounts/$username on every successful login trough postlogin scripting: http://wiki2.dovecot.org/PostLoginScripting -jf From joshua at hybrid.pl Thu Feb 16 11:25:11 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 10:25:11 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting Great, that's exactly what I needed :) Thanks! Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Thu Feb 16 13:18:15 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 12:18:15 +0100 (CET) Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: <20120216092048.GA10124@dibs.tanso.net> References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: On Thu, 16 Feb 2012, Jan-Frode Myklebust wrote: > On Thu, Feb 16, 2012 at 09:41:52AM +0100, Jacek Osiecki wrote: >> I was just wondering if there is any possibility of running another >> query after successful login - just to fill some extra field like >> "last_login"? > We touch a file in /var/log/activemailaccounts/$username on every > successful login trough postlogin scripting: > > http://wiki2.dovecot.org/PostLoginScripting By the way, is such thing possible for other processess? For example, I'd like to set in mysql table information that mail has been delivered using lmtp. Would something like this work? protocol lmtp { mail_plugins = $mail_plugins sieve executable = lmtp postlmtp } Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From janfrode at tanso.net Thu Feb 16 14:18:54 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Thu, 16 Feb 2012 13:18:54 +0100 Subject: [Dovecot] Any possibility of running query after sucessful login? In-Reply-To: References: <20120216092048.GA10124@dibs.tanso.net> Message-ID: <20120216121854.GA13072@dibs.tanso.net> On Thu, Feb 16, 2012 at 12:18:15PM +0100, Jacek Osiecki wrote: > > By the way, is such thing possible for other processess? For > example, I'd like to set in mysql table information that mail has > been delivered using lmtp. Would something like this work? > > protocol lmtp { > mail_plugins = $mail_plugins sieve > executable = lmtp postlmtp > } I've been inquiring the same lately, and unfortunately that's not possible. There's no "login" involved with lmtp, and each lmtp-session can have multiple recipients.. Maybe it can be solved trough a global sieve script? -jf From joshua at hybrid.pl Thu Feb 16 17:35:14 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Thu, 16 Feb 2012 16:35:14 +0100 (CET) Subject: [Dovecot] Recalculate quota when quota=dict ? Message-ID: Hi, My current setting for quota is: plugin { #quota = dirsize:User quota #quota = maildir:User quota quota = dict:User quota::proxy::quota #quota = fs:User quota } I have used mb2md.pl and copied result files manually to a desired mail folder - but quota was not recalculated. And it seems, that no matter what I do - the quota usage remains the same. I tried uncommenting line with: quota = maildir:User quota but it did not help. Any other idea how to recalculate quota? Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From tss at iki.fi Thu Feb 16 18:18:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:18:22 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3C57E7.8070105@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> Message-ID: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> On 16.2.2012, at 3.12, interfaSys s?rl wrote: > Here is a more meanigful backtrace. It happens every time I log in, > using any imap client. .. > #0 0x00000000105e0d88 in imap_match () from > /usr/lib/dovecot/libdovecot.so.0 > No symbol table info available. > #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from > /usr/lib/dovecot/libdovecot-storage.so.0 > No symbol table info available. Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. From tss at iki.fi Thu Feb 16 18:20:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:20:08 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 16.2.2012, at 17.35, Jacek Osiecki wrote: > quota = dict:User quota::proxy::quota > > Any other idea how to recalculate quota? With v2.x you can simply do: doveadm quota recalc -u username With v1.x you can delete the user's quota row from sql. From tss at iki.fi Thu Feb 16 18:21:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 18:21:55 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: On 16.2.2012, at 3.04, Finn B wrote: > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. From interfasys at gmail.com Thu Feb 16 19:04:33 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 17:04:33 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> Message-ID: <4F3D3721.7080203@gmail.com> That patch didn't fix it, but I've now compiled Dovecot without LTO and got even more info :) Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 189 array.h: No such file or directory. in array.h (gdb) bt full #0 0x00000000105954b6 in imap_match (glob=0x0, data=0x10aa01d8 "INBOX/spam") at array.h:189 ctx = {inboxcase_end = 0x7fffffffe7d0 "\237\004", sep = 64 '@', inboxcase = 248} i = 0 ret = IMAP_MATCH_NO match = IMAP_MATCH_NO #1 0x000000001049a119 in autocreate_iter_autobox (ctx=0x10a0f840, autobox=0x10a0f9a8) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 match = 32767 __FUNCTION__ = "autocreate_iter_autobox" #2 0x000000001049a484 in autocreate_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 actx = (struct mailbox_list_autocreate_iterate_context *) 0x10a0f918 info = (const struct mailbox_info *) 0x0 autoboxes = (const struct autocreate_box *) 0x10a0f9a8 autobox = (const struct autocreate_box *) 0x10a0f9a8 count = 1 __FUNCTION__ = "autocreate_iter_next" #3 0x000000001049a561 in mailbox_list_iter_next (ctx=0x10a0f840) at mailbox-list-iter.c:88 info = (const struct mailbox_info *) 0x7fffffffe86c #4 0x000000000040d06a in list_namespace_mailboxes (ctx=0x10a3f158) at array.h:189 info = (const struct mailbox_info *) 0x10a0f920 ns = (struct mail_namespace *) 0x0 flags = 536870916 str = (string_t *) 0x10a04290 mutf7_name = (string_t *) 0x10a043e0 name = 0x10a10040 "INBOX" ret = 1 #5 0x000000000040dc25 in cmd_list_continue (cmd=0x10a3f040) at array.h:189 _data_stack_cur_id = 4 ctx = (struct cmd_list_context *) 0x10a3f158 ret = 0 #6 0x000000000040e3b7 in cmd_list_full (cmd=0x10a3f040, lsub=false) at array.h:189 client = (struct client *) 0x10a3e840 args = (const struct imap_arg *) 0x10aa4118 list_args = (const struct imap_arg *) 0x10aa4340 arg_count = 1 ctx = (struct cmd_list_context *) 0x10a3f158 patterns = {arr = {buffer = 0x10a3f1a8, element_size = 8}, v = 0x10a3f1a8, v_modifiable = 0x10a3f1a8} pattern = 0x10a04268 "INBOX" patterns_strarr = (const char * const *) 0x10a3f1e0 str = (string_t *) 0x10a041f0 #7 0x000000000040e40c in cmd_list (cmd=0x10a3f040) at array.h:189 No locals. #8 0x00000000004140a0 in command_exec (cmd=0x10a3f040) at imap-commands.c:90 hook = (const struct command_hook *) 0x10a22200 ret = false #9 0x00000000004130b9 in client_command_input (cmd=0x10a3f040) at imap-client.c:94 client = (struct client *) 0x10a3e840 command = (struct command *) 0x7fff00000001 __FUNCTION__ = "client_command_input" #10 0x000000000041337d in client_handle_next_command (client=0x10a3e840, remove_io_r=0x7fffffffea6d) at imap-client.c:94 size = 279029312 #11 0x00000000004134c3 in client_handle_input (client=0x10a3e840) at imap-client.c:94 _data_stack_cur_id = 3 ret = false remove_io = false handled_commands = false __FUNCTION__ = "client_handle_input" #12 0x0000000000413639 in client_input (client=0x10a3e840) at imap-client.c:94 cmd = (struct client_command_context *) 0x10a20178 output = (struct ostream *) 0x10a1a6c8 bytes = 2 __FUNCTION__ = "client_input" #13 0x00000000105cf961 in io_loop_call_io (io=0x10a22ec0) at ioloop.c:26 ioloop = (struct ioloop *) 0x10a20040 t_id = 2 #14 0x00000000105d1382 in io_loop_handler_run (ioloop=0x10a20040) at array.h:189 ctx = (struct ioloop_handler_context *) 0x10a21080 events = (struct kevent *) 0x10a27000 event = (const struct kevent *) 0x10a27000 tv = {tv_sec = 1799, tv_usec = 999243} ts = {tv_sec = 1799, tv_nsec = 999243000} io = (struct io_file *) 0x10a22ec0 events_count = 6 ret = 1 i = 0 __FUNCTION__ = "io_loop_handler_run" #15 0x00000000105cfa04 in io_loop_run (ioloop=0x10a20040) at ioloop.c:26 No locals. #16 0x00000000105b3f1c in master_service_run (service=0x10a1a140, callback=0x41f71b ) at master-service.c:74 No locals. #17 0x000000000041fa23 in main (argc=1, argv=0x7fffffffec60) at main.c:68 set_roots = {0x422d80, 0x0} login_set = {auth_socket_path = 0x10a04068 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x41f514 , failure_callback = 0x41f692 } service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = -1 On 16/02/2012 16:18, Timo Sirainen wrote: > On 16.2.2012, at 3.12, interfaSys s?rl wrote: > >> Here is a more meanigful backtrace. It happens every time I log in, >> using any imap client. > .. >> #0 0x00000000105e0d88 in imap_match () from >> /usr/lib/dovecot/libdovecot.so.0 >> No symbol table info available. >> #1 0x00000000104e82e5 in autocreate_iter_autobox.20436.5667 () from >> /usr/lib/dovecot/libdovecot-storage.so.0 >> No symbol table info available. > > Still debug symbols missing from libraries, but .. perhaps this fixes it? http://hg.dovecot.org/dovecot-2.1/rev/72e1e17d2e22 > > I don't know why it would happen with all imap clients though. And if that doesn't solve it, show your doveconf -n output. > > From tss at iki.fi Thu Feb 16 19:06:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:06:59 +0200 Subject: [Dovecot] v2.1.0 released Message-ID: http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Changes since rc7: FS layout related crashfix, two man pages added. The biggest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * Temporary authentication failures sent to IMAP/POP3 clients now includes the server's hostname and timestamp. This makes it easier to find the error message from logs. * dsync was merged into doveadm. There is still "dsync" symlink pointing to "doveadm", which you can use the old way for now. The preferred ways to run dsync are "doveadm sync" (for old "dsync mirror") and "doveadm backup". + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + IMAP SPECIAL-USE extension to describe mailboxes + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. + SCRAM-SHA-1 authentication mechanism by Florian Zeitz + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i parameter to select the instance (instead of having to use -c ). See instance_name setting. + auth: Implemented support for Postfix's "TCP map" sockets for user existence lookups. - listescape plugin works perfectly now From alexwbaule at gmail.com Thu Feb 16 19:12:25 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 16 Feb 2012 15:12:25 -0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. You say it "old way", when it's change ? in dovecot 2.0.18 is in the old or new way ? I asking because if i update dovecot, i need to update the plugin. Tks ! Em 13 de fevereiro de 2012 18:18, Timo Sirainen escreveu: > On 13.2.2012, at 19.28, Alex Baule wrote: > >> static void emexis_mailbox_allocated(struct mailbox *box) >> { >> ? ?union mailbox_module_context *zbox; >> >> ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); >> ? ?zbox->super = box->v; >> ? ?box->v.open = Mplugin_mailbox_open; >> ? ?box->v.sync_notify = Mplugin_mailbox_sync_notify; >> >> ? ?MODULE_CONTEXT_SET_SELF(box, emexis_storage_module, zbox); >> } > > The above is an "old way" to do it. It doesn't work well if you have any other plugins loaded. > >> struct mailbox_vfuncs *v = box->vlast; >> v->sync_notify = Mplugin_mailbox_sync_notify; >> >> But in this two cases, the Mplugin_mailbox_sync_notify is never >> called... i missing something ?? > > Never called at all? What storage backend are you using as the base? Maildir? > > Anyway, the way you should be calling it is the exact same way quota plugin does: > > void Mplugin_mailbox_allocated(struct mailbox *box) > { > ? ? ? ?struct mailbox_vfuncs *v = box->vlast; > ? ? ? ?union mailbox_module_context *zbox; > > ? ? ? ?zbox = p_new(box->pool, union mailbox_module_context, 1); > ? ? ? ?zbox->super = *v; > ? ? ? ?box->vlast = &zbox->super; > > ? ? ? ?v->open = Mplugin_mailbox_open; > ? ? ? ?v->sync_notify = Mplugin_mailbox_sync_notify; > ? ? ? ?MODULE_CONTEXT_SET_SELF(box, Mplugin_storage_module, zbox); > } > From tss at iki.fi Thu Feb 16 19:15:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:15:45 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <1933A805-19A9-4476-B0C2-B331FC1033F6@iki.fi> On 16.2.2012, at 19.04, interfaSys s?rl wrote: > That patch didn't fix it, but I've now compiled Dovecot without LTO and > got even more info :) What's your doveconf -n output? From tss at iki.fi Thu Feb 16 19:21:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:21:14 +0200 Subject: [Dovecot] v2.1.0 released In-Reply-To: References: Message-ID: On 16.2.2012, at 19.06, Timo Sirainen wrote: > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz > http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig > > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > Changes since rc7: FS layout related crashfix, two man pages added. Correction: I messed up by not doing a "hg pull -u" before building the v2.1.0 release, so this release is identical to rc7. Still, it's good enough that I won't bother with v2.1.1 yet (the FS layout crashfix doesn't affect any real clients). From tss at iki.fi Thu Feb 16 19:27:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:27:01 +0200 Subject: [Dovecot] There is a way to know if a email has been expunged ? In-Reply-To: References: <2D1E24D4-1C83-41B9-A5AD-06D98DF31110@iki.fi> <99089B82-8E3C-44B1-96AE-8F1DA893B894@iki.fi> <500520BB-9131-45EF-800D-DBD8578A6E88@iki.fi> Message-ID: On 16.2.2012, at 19.12, Alex Baule wrote: > hi Timo, now its works.. i do in the old way, my dovecot is 2.0.15. > > You say it "old way", when it's change ? in dovecot 2.0.18 is in the > old or new way ? It changed before v2.0.0 release, so you should use the new way. > I asking because if i update dovecot, i need to update the plugin. It works the same in v2.1. From dluke at geeklair.net Thu Feb 16 19:42:18 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:42:18 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: References: Message-ID: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: > Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Looks like there's a problem on Mac OS X 10.5 (ppc): Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument I took a cursory look at the source, but didn't notice anything obvious... -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:47:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:47:53 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> Message-ID: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> On 16.2.2012, at 19.42, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. > > > Looks like there's a problem on Mac OS X 10.5 (ppc): > > Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) > Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > I took a cursory look at the source, but didn't notice anything obvious... Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. Anyway, try if it works with: configure --with-ioloop=select From dluke at geeklair.net Thu Feb 16 19:49:48 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 12:49:48 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> Message-ID: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> On Feb 16, 2012, at 12:47 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.42, Daniel J. Luke wrote: > >> On Feb 16, 2012, at 12:06 PM, Timo Sirainen wrote: >>> Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. >> >> >> Looks like there's a problem on Mac OS X 10.5 (ppc): >> >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Dovecot v2.1.0 starting up (core dumps disabled) >> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> I took a cursory look at the source, but didn't notice anything obvious... > > Does v2.0 or some earlier Dovecot version work with it? I don't think I've done any changes to kqueue handling for a long long time, other than make it enabled by default in v1.1. Although maybe it's related to having FIFOs nowadays.. > > Anyway, try if it works with: configure --with-ioloop=select 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). I'll let you know if --with-ioloop=select works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From tss at iki.fi Thu Feb 16 19:51:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 19:51:59 +0200 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> Message-ID: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument > > 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). OK, I bet it's the FIFOs then. See if this helps: service stats { fifo_listener stats-mail { mode = 0 } } From tss at iki.fi Thu Feb 16 20:04:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 16 Feb 2012 20:04:30 +0200 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: <4F3D3721.7080203@gmail.com> References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 From dluke at geeklair.net Thu Feb 16 20:20:27 2012 From: dluke at geeklair.net (Daniel J. Luke) Date: Thu, 16 Feb 2012 13:20:27 -0500 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> Message-ID: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: > On 16.2.2012, at 19.49, Daniel J. Luke wrote: > >>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >> >> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). > > OK, I bet it's the FIFOs then. See if this helps: > > service stats { > fifo_listener stats-mail { > mode = 0 > } > } That works. -- Daniel J. Luke +========================================================+ | *---------------- dluke at geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+ From bradley.giesbrecht at gmail.com Thu Feb 16 20:31:31 2012 From: bradley.giesbrecht at gmail.com (Bradley Giesbrecht) Date: Thu, 16 Feb 2012 10:31:31 -0800 Subject: [Dovecot] [Dovecot-news] v2.1.0 released In-Reply-To: <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> References: <3534602C-2FC0-488F-9FD1-00DF0005496C@geeklair.net> <325B4548-C3FA-4426-A563-1CBD24FDF82C@iki.fi> <24443E00-E2DD-44C2-86C7-641CE64ABFCE@geeklair.net> <05889B21-C1AF-4573-A681-0B708B28B9D4@iki.fi> <305CFF47-25D0-4CA9-B5A3-45144270E8B8@geeklair.net> Message-ID: <2F3C7FE5-7D31-4000-A0E4-75B1E1169332@gmail.com> On Feb 16, 2012, at 10:20 AM, Daniel J. Luke wrote: > On Feb 16, 2012, at 12:51 PM, Timo Sirainen wrote: >> On 16.2.2012, at 19.49, Daniel J. Luke wrote: >> >>>>> Feb 16 12:35:42 gandalf dovecot[28047]: master: Fatal: kevent(EV_ADD, READ, 16) failed: Invalid argument >>> >>> 2.0.18 works fine (as has every other 2.0 that I've tried). I've been using the kqueue ioloop on this machine for a long time (pre 1.0, I think). >> >> OK, I bet it's the FIFOs then. See if this helps: >> >> service stats { >> fifo_listener stats-mail { >> mode = 0 >> } >> } > > That works. Cool. I am surprised I missed this thread: http://dovecot.org/pipermail/dovecot/2011-November/062122.html Regards, Bradley Giesbrecht (pixilla) From interfasys at gmail.com Thu Feb 16 20:31:42 2012 From: interfasys at gmail.com (=?UTF-8?B?aW50ZXJmYVN5cyBzw6BybA==?=) Date: Thu, 16 Feb 2012 18:31:42 +0000 Subject: [Dovecot] [Dovecot 2.1] imap service crash when using some clients In-Reply-To: References: <4F385315.5010600@gmail.com> <4F3C57E7.8070105@gmail.com> <53439CD4-ADCB-4383-9FEA-F3F17E47B2A4@iki.fi> <4F3D3721.7080203@gmail.com> Message-ID: <4F3D4B8E.2070103@gmail.com> Yep, works perfectly now. Thank you for fixing it :) On 16 February 2012 18:04:30, Timo Sirainen wrote: > OK, this should fix it: http://hg.dovecot.org/dovecot-2.1/rev/ff117a044e20 > > From stephan at rename-it.nl Thu Feb 16 23:29:56 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:29:56 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 Message-ID: <4F3D7554.1010205@rename-it.nl> Hello Dovecot users, Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most significant new features are support for the Sieve editheader extension and the sieve-filter tool. I haven't received much feedback on these new features, which can either mean that they are mostly bug-free or that not many people have tried these, so these are still considered experimental. Changelog v0.3.0: * Renamed sieve_global_path setting to sieve_default for clarity. Old name is still recognized for backwards compatibility. Support for the ancient (pre v1.1) name for this setting "global_script_path" is now dropped. * Added means to prohibit use of redirect action. Setting sieve_max_redirects to 0 now means that redirect is disallowed in stead of unlimited. Default value remains four. * Fixed interaction of Sieve include extension with ManageSieve. It is updated to match new requirements in the draft include specification. Missing included scripts are no longer an error at upload time. * Updated RFC2822 header field body verification to exclude non-printing characters (RFC5322). Only Sieve actions that can create unstructured header values (currently enotify/mailto and editheader) are affected by this change. + Completed sieve-filter tool to a useful state. The sieve-filter tool provides a means to (re)filter messages in a mailbox through a Sieve script. + Implemented the Sieve editheader extension. It is now possible to add and remove message headers from within Sieve. + ManageSieve: added support for reading quoted and literal strings as a stream. Fixes support for handing large SASL responses (analogous to similar changes in Dovecot). It is now also allowed to use a quoted string for the PUTSCRIPT script argument. + Added code to cleanup tmp directory in Sieve storage directory (sieve_dir) every once in a while. + Added support for substituting the entire message during Sieve processing. This is used for the filter action provided by the new sieve_extprograms plugin (provided separately for now). The filter action allows passing the message through an external program. + Added support for restricting certain Sieve language extensions to (admin-controled) global scripts. Restricted extensions can be configured using the new sieve_global_extensions setting. This is particularly useful for some of the Dovecot-specific (plugin-based) Sieve extensions, that can be somewhat hazardous when under direct control of users (e.g. sieve_extprograms). The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From stephan at rename-it.nl Thu Feb 16 23:35:43 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 22:35:43 +0100 Subject: [Dovecot] Sieve_extprograms plugin for Pigeonhole v0.3 Message-ID: <4F3D76AF.9060600@rename-it.nl> As indicated in the release announcement of Pigeonhole v0.3.0, there is also a new experimental plugin available for Pigeonhole v0.3: the sieve_extprograms plugin. This can be used to pipe messages to and filter messages through external programs (shell scripts). Also, string data can be retrieved from such external programs. The plugin can currently only be obtained from its Mercurial repository: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms Refer to the INSTALL and README files for more information: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/INSTALL http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/README The vendor-specific Sieve language extension provided by this plugin are documented here: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/doc/rfc/spec-bosch-sieve-extprograms.txt Please test this plugin. The plan is to include it into the main Pigeonhole release eventually. Regards, Stephan. From matrixworkstation at gmail.com Thu Feb 16 23:40:32 2012 From: matrixworkstation at gmail.com (aledr) Date: Thu, 16 Feb 2012 19:40:32 -0200 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: <4F3D7554.1010205@rename-it.nl> References: <4F3D7554.1010205@rename-it.nl> Message-ID: Correct links are: http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig On Thu, Feb 16, 2012 at 7:29 PM, Stephan Bosch wrote: > Hello Dovecot users, > > Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most > significant new features are support for the Sieve editheader extension and > the sieve-filter tool. I haven't received much feedback on these new > features, which can either mean that they are mostly bug-free or that not > many people have tried these, so these are still considered experimental. > > Changelog v0.3.0: > > ?* Renamed sieve_global_path setting to sieve_default for clarity. Old > ? name is still recognized for backwards compatibility. Support for > ? the ancient (pre v1.1) name for this setting "global_script_path" is > ? now dropped. > ?* Added means to prohibit use of redirect action. Setting > ? sieve_max_redirects to 0 now means that redirect is disallowed in > ? stead of unlimited. Default value remains four. > ?* Fixed interaction of Sieve include extension with ManageSieve. It is > ? updated to match new requirements in the draft include > ? specification. Missing included scripts are no longer an error at > ? upload time. > ?* Updated RFC2822 header field body verification to exclude > ? non-printing characters (RFC5322). Only Sieve actions that can > ? create unstructured header values (currently enotify/mailto and > ? editheader) are affected by this change. > ?+ Completed sieve-filter tool to a useful state. The sieve-filter tool > ? provides a means to (re)filter messages in a mailbox through a Sieve > ? script. > ?+ Implemented the Sieve editheader extension. It is now possible to > ? add and remove message headers from within Sieve. > ?+ ManageSieve: added support for reading quoted and literal strings as > ? a stream. Fixes support for handing large SASL responses (analogous > ? to similar changes in Dovecot). It is now also allowed to use a > ? quoted string for the ?PUTSCRIPT script argument. > ?+ Added code to cleanup tmp directory in Sieve storage directory > ? (sieve_dir) every once in a while. > ?+ Added support for substituting the entire message during Sieve > ? processing. This is used for the filter action provided by the new > ? sieve_extprograms plugin (provided separately for now). The filter > ? action allows passing the message through an external program. > ?+ Added support for restricting certain Sieve language extensions to > ? (admin-controled) global scripts. Restricted extensions can be > ? configured using the new sieve_global_extensions setting. This is > ? particularly useful for some of the Dovecot-specific (plugin-based) > ? Sieve extensions, that can be somewhat hazardous when under direct > ? control of users (e.g. sieve_extprograms). > > The release is available as follows: > > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig > > Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more > information. Have fun testing this new release and don't hesitate to notify > me when there are any problems. > > Regards, > > -- > Stephan Bosch > stephan at rename-it.nl > > > > > > > > > > > -- [ ]'s Aledr - Alexandre www.exatati.com.br "OpenSource Solutions for SmallBusiness Problems" From list at airstreamcomm.net Thu Feb 16 23:48:40 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 16 Feb 2012 15:48:40 -0600 Subject: [Dovecot] Dovecot v2.2 plans In-Reply-To: References: <9B0A121A-CFD0-439B-80FD-69AD80CBE156@iki.fi> <8a8e99845ffc85aff7dbe8d8a7790f1d@mail.airstreamcomm.net> Message-ID: On Wed, 15 Feb 2012 20:51:59 +0200, Timo Sirainen wrote: > On 15.2.2012, at 5.08, > wrote: > >> I know you mentioned you would cover this in a coming post, but we were >> curious what the new dsync replication will be capable of. Would it >> monitor changes to mailboxes and push automatic replication to the remote >> mail store, > > Yes. > >> and if this is the case could it be an N-way replication setup >> in which any host in a cluster can participate in the replication? > > Initially 2-way, but I don't think anything prevents it being N-way. > >> Do you consider this to be a high availability solution? > > > The initial version is really about doing all of this with NFS. In NFS > setup if two replaced storages are both mounted and the primary storage > dies, Dovecot will start using the replica. So that's HA. > > The other possibility is to run Dovecot in two completely separate data > centers and replicate through ssh. Here are more possibilities for how to > do HA, but some of them also have downsides.. dovecot.fi mails are actually > done this way, and can be accessed from either server at any time. I've > been thinking about soon making half of my clients use one server and half > the other one to see if I can find any dsync bugs (I've always 3-4 IMAP > clients connected). Just to throw our thoughts into the mix, finding an open source multi-site active/active mail solution that does not require building super expensive multi-site storage systems would be a really refreshing way to purse this level of availability. Maybe the only way to accurately get this level of availability is to cluster the storage between sites? From stephan at rename-it.nl Fri Feb 17 00:03:07 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 16 Feb 2012 23:03:07 +0100 Subject: [Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0 In-Reply-To: References: <4F3D7554.1010205@rename-it.nl> Message-ID: <4F3D7D1B.6030404@rename-it.nl> On 2/16/2012 10:40 PM, aledr wrote: > Correct links are: > > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz > http://www.rename-it.nl/dovecot/2.1/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Oops :) Regards, Stephan. From mstevens at imt-systems.com Fri Feb 17 00:27:46 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Thu, 16 Feb 2012 23:27:46 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 Message-ID: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Hi, First, Timo, Thank you for Dovecot 2.1.0. I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for RHEL6, CentOS 6 and Scientific Linux 6. If anyone is interested in testing these RPMs: http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ Feedback would be appreciated. After some testing, we will use these RPMs on our IMAP servers. Best regards, Morten From przemek.orzechowski at makolab.pl Fri Feb 17 00:39:17 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Thu, 16 Feb 2012 23:39:17 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x Message-ID: <220331515776550936318f562808607f@localhost> Hi I have already read this page but there is no info on how to delete old mails in dovecot 1.2 http://wiki1.dovecot.org/QuestionsAndAnswers#How_can_I_create_a_cron_or_something_to_delete_old_messages.3F I need to delete old mails from over 100 mailboxes with average of 10k mails / mailbox Is there a way to delete old mails (by message date not file creation date) as with doveadm in dovecot 2 ? Thanks in advance bellow my dovecot -n (not sure if it is neccessary or not so just in case) # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/makolab/star_makolab_pl.crt ssl_key_file: /etc/ssl/makolab/star_makolab_pl.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: passdb: driver: pam userdb: driver: passwd plugin: quota: fs:Limit uzytkownika:user quota2: fs:Limit domeny:group From tss at iki.fi Fri Feb 17 01:25:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:25:18 +0200 Subject: [Dovecot] log message about client_limit may in fact be due to ulimit In-Reply-To: References: <4F3A77F9.8000002@in.tum.de> <4F3B720E.6010705@in.tum.de> Message-ID: On 15.2.2012, at 12.30, Timo Sirainen wrote: >>> Do you see some error message also about "Too many open files", >> >> Ah, now that I grep for it, I see those messages too :) During one hour, the messages were logged with the following number of occurrences: >> >> - Twice: dovecot: anvil: Error: net_accept() failed: Too many open files >> - Twice: config: Error: net_accept() failed: Too many open files >> - Twice: ipc: Error: net_accept() failed: Too many open files >> - 43 times: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped >> - 43 times: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped > > Oh, right. The error message is wrong. There should be a difference between the client_limit having actually reached, and an error condition (e.g. out of fds) that causes a process to temporarily stop listening for new clients (= setting "available client count" to 0, which makes master think it has used up all of its clients). I'll fix it to v2.1.1. Actually this solves it nicely: http://hg.dovecot.org/dovecot-2.1/rev/701709f472e9 It simply keeps giving net_accept() failures. From tss at iki.fi Fri Feb 17 01:28:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:28:28 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <220331515776550936318f562808607f@localhost> References: <220331515776550936318f562808607f@localhost> Message-ID: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > I need to delete old mails from over 100 mailboxes with average of 10k > mails / mailbox > Is there a way to delete old mails (by message date not file creation > date) as with doveadm in dovecot 2 ? You can delete them by mtime or by ctime: http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent If by "message date" you mean the Date: header, then there's no easy way. From przemek.orzechowski at makolab.pl Fri Feb 17 01:42:22 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q?Przemys=C5=82aw_Orzechowski?=) Date: Fri, 17 Feb 2012 00:42:22 +0100 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <6f78990c40e3c083ec43dca148edfe0d@localhost> On Fri, 17 Feb 2012 01:28:28 +0200, Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > >> I need to delete old mails from over 100 mailboxes with average of 10k >> mails / mailbox >> Is there a way to delete old mails (by message date not file creation >> date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy way. Yes Date header unfortunatly (seems i will have to write some kind of script for that) Just to be sure just deleting mail files is safe? Or do i have to run some additional commands to recreate dovecots index files? I'm using Maildir. Asking just to be sure that I will not screw up something. From slitt at troubleshooters.com Fri Feb 17 01:52:31 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 18:52:31 -0500 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> Message-ID: <20120216185231.1eed027c@mydesk> On Fri, 17 Feb 2012 01:28:28 +0200 Timo Sirainen wrote: > On 17.2.2012, at 0.39, Przemys?aw Orzechowski wrote: > > > I need to delete old mails from over 100 mailboxes with average of > > 10k mails / mailbox > > Is there a way to delete old mails (by message date not file > > creation date) as with doveadm in dovecot 2 ? > > You can delete them by mtime or by ctime: > > http://wiki.dovecot.org/Plugins/Expire#v1.0_cronjob_equivalent > > If by "message date" you mean the Date: header, then there's no easy > way. > Would something like the following work for the date header? #!/bin/bash for msg in *; do if b4.py $msg 20080216; then rm $msg fi done #!/usr/bin/python datestring = get_dateline() ## FIND DATE HDR AND RETURN THE STRING yyyymmdd = format2yyyymmdd(datestring) ## CONVERT TO YYYYMMDD if yyyymmdd < sys.argv[1]: sys.exit(0) else: sys.exit(1) In the preceding, would the deletions mess up maildir indices? Thanks SteveT From tss at iki.fi Fri Feb 17 01:53:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 01:53:39 +0200 Subject: [Dovecot] How to delete old mails in dovecot 1.2.x In-Reply-To: <20120216185231.1eed027c@mydesk> References: <220331515776550936318f562808607f@localhost> <5FF0E5B5-1B55-4218-B457-74C343447540@iki.fi> <20120216185231.1eed027c@mydesk> Message-ID: <66CFB9DE-1A67-4EDF-B777-E482AF1226AF@iki.fi> On 17.2.2012, at 1.52, Steve Litt wrote: > In the preceding, would the deletions mess up maildir indices? no From ml at smtp.fakessh.eu Fri Feb 17 02:41:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Fri, 17 Feb 2012 01:41:06 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: Le 2012-02-16 23:27, Morten Stevens a ?crit?: > Hi, > > First, Timo, Thank you for Dovecot 2.1.0. > > I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for > RHEL6, CentOS 6 and Scientific Linux 6. > > If anyone is interested in testing these RPMs: > http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ > > Feedback would be appreciated. > > After some testing, we will use these RPMs on our IMAP servers. > > Best regards, > > Morten is there any source rpm packages for centos 5 ? best regards -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation From finn at kirstineslund.dk Fri Feb 17 03:07:45 2012 From: finn at kirstineslund.dk (Finn B) Date: Fri, 17 Feb 2012 02:07:45 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3DA861.7060902@kirstineslund.dk> Moi Timo /all. Well I have looked everywhere to find any other clues - I cann't find any traces from this incident. But I have spend a lot of time figuring out what is going on so I may becom e'blind' for all the obvious so let me know if more can be found- though I have inserted more lines from Dovecot.log. Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:08 pop3-login: Error: Timeout waiting for handshake from auth serve r. my pid=29388, input bytes=0 Feb 15 01:41:38 pop3-login: Info: Disconnected: Inactivity (no auth attempts): r ip=127.0.0.1, lip=127.0.0.1, secured Feb 15 01:41:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:41:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 master: Error: service(auth): command startup failed, throttling Feb 15 01:42:38 log: Error: service(auth): child 29450 returned error 89 (Fatal failure) Feb 15 01:42:38 auth: Fatal: vpopmail: vauth_open() failed Feb 15 01:43:08 imap-login: Error: Timeout waiting for handshake from auth serve r. my pid=29448, input bytes=0 This is my dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir raw SQL drivers: Passdb: checkpassword pam passwd passwd-file shadow vpopmail Userdb: checkpassword nss passwd prefetch passwd-file vpopmail and here is my Dovecot.conf's (this one is from QMT install )- further below is my Dovecot.conf taken from my production system and also used without more luck # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-274.18.1.el5 i686 CentOS release 5.7 (Final) auth_cache_size = 32 M auth_username_format = %Lu first_valid_uid = 89 listen = *,:: log_path = /var/log/dovecot.log login_greeting = Dovecot toaster ready. namespace { inbox = yes location = prefix = separator = . type = private } passdb { args = cache_key=%u webmail=127.0.0.1 driver = vpopmail } plugin { quota = maildir } protocols = imap pop3 ssl_cert = method=%m rip=%r lip=%l mpid=%e %c login_trusted_networks = mail_access_groups = mail_attachment_dir = mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k mail_cache_fields = flags mail_cache_min_mail_count = 0 mail_chroot = mail_debug = no mail_fsync = optimized mail_full_filesystem_access = no mail_gid = 89 mail_home = mail_location = maildir:~/Maildir mail_log_prefix = "%s(%u): " mail_max_keyword_length = 50 mail_max_lock_timeout = 0 mail_max_userip_connections = 10 mail_never_cache_fields = imap.envelope mail_nfs_index = no mail_nfs_storage = no mail_plugin_dir = /usr/local/lib/dovecot mail_plugins = mail_privileged_group = mail_save_crlf = no mail_temp_dir = /tmp mail_uid = 89 mailbox_idle_check_interval = 30 secs mailbox_list_index_disable = no maildir_copy_with_hardlinks = yes maildir_stat_dirs = no maildir_very_dirty_syncs = no master_user_separator = mbox_dirty_syncs = yes mbox_dotlock_change_timeout = 2 mins mbox_lazy_writes = yes mbox_lock_timeout = 5 mins mbox_min_index_size = 0 mbox_read_locks = fcntl mbox_very_dirty_syncs = no mbox_write_locks = dotlock fcntl mdbox_preallocate_space = no mdbox_rotate_interval = 0 mdbox_rotate_size = 2 M mmap_disable = no namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = webmail=127.0.0.1 deny = no driver = vpopmail master = no pass = no } plugin/quota = maildir:ignore=Trash plugin/quota_rule = ?:storage=0 pop3_client_workarounds = pop3_enable_last = no pop3_fast_size_lookups = no pop3_lock_session = no pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = no pop3_reuse_xuidl = no pop3_save_uidl = no pop3_uidl_format = %08Xu%08Xv postmaster_address = protocols = imap pop3 quota_full_tempfail = no recipient_delimiter = + rejection_reason = Your message to <%t> was automatically rejected:%n%r rejection_subject = Rejected: %s sendmail_path = /usr/sbin/sendmail service anvil { chroot = empty client_limit = 0 drop_priv_before_exec = no executable = anvil extra_groups = group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 1 protocol = service_count = 0 type = anvil unix_listener anvil-auth-penalty { group = mode = 0600 user = } unix_listener anvil { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service auth-worker { chroot = client_limit = 1 drop_priv_before_exec = no executable = auth -w extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener auth-worker { group = mode = 0600 user = $default_internal_user } user = vsz_limit = 18446744073709551615 B } service auth { chroot = client_limit = 4096 drop_priv_before_exec = no executable = auth extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener auth-client { group = mode = 0600 user = } unix_listener auth-login { group = mode = 0600 user = $default_internal_user } unix_listener auth-master { group = mode = 0600 user = } unix_listener auth-userdb { group = mode = 0600 user = } unix_listener login/login { group = mode = 0666 user = } user = root vsz_limit = 18446744073709551615 B } service config { chroot = client_limit = 0 drop_priv_before_exec = no executable = config extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = config unix_listener config { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service dict { chroot = client_limit = 1 drop_priv_before_exec = no executable = dict extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dict { group = mode = 0600 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service director { chroot = client_limit = 0 drop_priv_before_exec = no executable = director extra_groups = fifo_listener login/proxy-notify { group = mode = 00 user = } group = idle_kill = 4294967295 secs privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = unix_listener director-admin { group = mode = 0600 user = } unix_listener login/director { group = mode = 00 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service dns_client { chroot = client_limit = 1 drop_priv_before_exec = no executable = dns-client extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = unix_listener dns-client { group = mode = 0666 user = } unix_listener login/dns-client { group = mode = 0666 user = } user = $default_internal_user vsz_limit = 18446744073709551615 B } service doveadm { chroot = client_limit = 1 drop_priv_before_exec = no executable = doveadm-server extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 1 type = unix_listener doveadm-server { group = mode = 0600 user = } user = vsz_limit = 18446744073709551615 B } service imap-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = imap-login extra_groups = group = idle_kill = 0 inet_listener imap { address = port = 143 ssl = no } inet_listener imaps { address = port = 993 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = imap service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service imap { chroot = client_limit = 1 drop_priv_before_exec = no executable = imap extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = imap service_count = 1 type = unix_listener login/imap { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service lmtp { chroot = client_limit = 0 drop_priv_before_exec = no executable = lmtp extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = lmtp service_count = 0 type = unix_listener lmtp { group = mode = 0666 user = } user = vsz_limit = 0 } service log { chroot = client_limit = 0 drop_priv_before_exec = no executable = log extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1 process_min_avail = 0 protocol = service_count = 0 type = log user = vsz_limit = 18446744073709551615 B } service pop3-login { chroot = login client_limit = 0 drop_priv_before_exec = no executable = pop3-login extra_groups = group = idle_kill = 0 inet_listener pop3 { address = port = 110 ssl = no } inet_listener pop3s { address = port = 995 ssl = yes } privileged_group = process_limit = 0 process_min_avail = 0 protocol = pop3 service_count = 1 type = login user = $default_login_user vsz_limit = 64 M } service pop3 { chroot = client_limit = 1 drop_priv_before_exec = no executable = pop3 extra_groups = group = idle_kill = 0 privileged_group = process_limit = 1024 process_min_avail = 0 protocol = pop3 service_count = 1 type = unix_listener login/pop3 { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } service ssl-params { chroot = client_limit = 0 drop_priv_before_exec = no executable = ssl-params extra_groups = group = idle_kill = 0 privileged_group = process_limit = 0 process_min_avail = 0 protocol = service_count = 0 type = startup unix_listener login/ssl-params { group = mode = 0666 user = } user = vsz_limit = 18446744073709551615 B } shutdown_clients = yes ssl = yes ssl_ca = ssl_cert = On 16.2.2012, at 3.04, Finn B wrote: > >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > auth process fails in some way.. It should have logged another error about this. Isn't there anything else? Show your doveconf -n and dovecot --build-options output. > > From tss at iki.fi Fri Feb 17 03:11:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 17 Feb 2012 03:11:31 +0200 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3DA861.7060902@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> <4F3DA861.7060902@kirstineslund.dk> Message-ID: On 17.2.2012, at 3.07, Finn B wrote: > Feb 15 01:40:38 auth: Fatal: vpopmail: vauth_open() failed This is your problem. Your vpopmail is somehow broken. I can't really help any further than that, since I haven't used vpopmail myself. From dovecot at r.paypc.com Fri Feb 17 04:46:23 2012 From: dovecot at r.paypc.com (Robin) Date: Thu, 16 Feb 2012 18:46:23 -0800 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> Message-ID: <4F3DBF7F.4030504@r.paypc.com> > You mean you deleted Solr index, so that it's empty? That should work too. > > Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: plugin { fts = solr fts_solr = break-imap-search url=http://solrhost:8983/solr/ } Should I be adding fts/fts_solr to the global mail_plugins setting? I have Solr up and running, without any firewalling between the hosts, and it never seems to even try to use it. The logs show, even after importing fresh mail and issuing a "search text "your" command to the server, which takes it about 5m or so to return results. I see the following in the log: Feb 16 17:51:54 indexer-worker(testuser): Info: Indexed 0 messages in INBOX2010 GET /solr/ issued to http://solrhost:8983/ via TELNET reports A-OK, and Solr Admin shows ready status when the admin console is loaded into a web browser. I can see there is ZERO traffic between the hosts during the SEARCH text command's execution, though I can see an open connection to the solr host in netstat: tcp 0 0 linuxcode:56393 solrhost:8983 ESTABLISHED =R= From slitt at troubleshooters.com Fri Feb 17 05:20:42 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Thu, 16 Feb 2012 22:20:42 -0500 Subject: [Dovecot] Favorable mention in Linux Productivity Magazine Message-ID: <20120216222042.68a5a58c@mydesk> Hi all, The latest Linux Productivity Magazine, http://www.troubleshooters.com/lpm/201202/201202.htm, discusses the positive role Dovecot played in my migration away from Kmail. This is currently topical as Kmail is in the midst to transforming to Kmail2, which requires an Akonadi database and is not popular. I hope you enjoy the magazine. SteveT Steve Litt Troubleshooters.Com From klimenko.n at theitidea.ru Fri Feb 17 07:21:06 2012 From: klimenko.n at theitidea.ru (=?UTF-8?B?0JrQu9C40LzQtdC90LrQviDQndC40LrQvtC70LDQuQ==?=) Date: Fri, 17 Feb 2012 09:21:06 +0400 Subject: [Dovecot] Dovecot lda parameters Message-ID: <4F3DE3C2.1060308@theitidea.ru> hi all I have the following scheme. Postfix with virtual users with dovecot LDA. AD as database. Dovecot and Postfix get the following Idap attributes from AD: sAMAccountName, mail and mail_alias. The problem is that I can?t make dovecot lda move the user?s mail to the folder with the name from sAMAccountName attribute, as mail always moves to folder with e-mail address name. Is it posible to delivery e-mail to folder with name from ldap attributes sAMAccountName? -- From joshua at hybrid.pl Fri Feb 17 11:51:20 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 10:51:20 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: W dniu 2012-02-16 17:20, Timo Sirainen napisa?(a): > On 16.2.2012, at 17.35, Jacek Osiecki wrote: >> Any other idea how to recalculate quota? > With v2.x you can simply do: doveadm quota recalc -u username Great, That's exactly what I needed :) Sorry, it was lame - I should have found it in wiki2... By the way: what might have caused such a warning? root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Greetings, -- Jacek Osiecki From mstevens at imt-systems.com Fri Feb 17 14:40:17 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Fri, 17 Feb 2012 13:40:17 +0100 Subject: [Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6 In-Reply-To: References: <034a3fe6762b3ec30e097fd4172030eb@imt-systems.com> Message-ID: On 17.02.2012 01:41, ml wrote: > is there any source rpm packages for centos 5 ? No, only for CentOS6/RHEL6. Otherwise, look here: http://packages.atrpms.net/dist/el5/dovecot-2.1.x/ Best regards, Morten From joshua at hybrid.pl Fri Feb 17 15:40:32 2012 From: joshua at hybrid.pl (joshua at hybrid.pl) Date: Fri, 17 Feb 2012 14:40:32 +0100 Subject: [Dovecot] Quota warnings and LMTP Message-ID: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Hello, I have dovecot 2.0 configured with virtual users in MySQL, SMTP is handled by exim which uses dovecot LMTP transport. I'd like to test quota warning messages, but in wiki2 there is only an example for delivering it with dovecot-lda. I tried several ways of using lmtp binary, but without success. How to replace this: cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" From: postmaster at domain.com Subject: quota warning Your mailbox is now $PERCENT% full. EOF with lmtp? Greetings, -- Jacek Osiecki From as at 3a.pl Fri Feb 17 16:17:31 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 17 Feb 2012 15:17:31 +0100 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <4F3E617B.6050801@3a.pl> On 17.02.2012 14:40, joshua at hybrid.pl wrote: > Hello, Hi, > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o > "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? As far as I known, there is no specific config for LMTP transport when quota is considered. When testing quota messages, the most problems for me was with the fact, that the warning is triggered only when the limit is reached. Only once for each limit. The script we are using: #!/bin/bash PERCENT=$1 USER=$2 DOMAIN=$3 cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=dict:User quota::noenforcing:proxy::quota" From: postmaster@$DOMAIN Subject: ostrze?enie o przekroczeniu wolnego miejsca / quota warning Twa skrzynka jest zape?niona w $PERCENT%. Your mailbox is now $PERCENT% full. EOF -- Adam Szpakowski From sanito at progresas.lt Fri Feb 17 17:39:53 2012 From: sanito at progresas.lt (sanito at progresas.lt) Date: Fri, 17 Feb 2012 17:39:53 +0200 (EET) Subject: [Dovecot] user_filter problem Message-ID: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Hello, I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 with user information stored in the LDAP database. The main problem is that email address and pop3 username are different for every mailbox (I cannot change this behaviour, because thousands of people already configured their email client for such a setup). So in my situation, user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it should be: user_filter=(cn=%u). Please, see: E-mail address: john at example.com pop3 username: 0005a pop3 password: very_secure_password And ldap ldiff for this mailbox: dn: cn=0005a,dc=example.com,ou=localmail,ou=mail uid: 0005a mail: john at example.com mailbox: /data/mail/example.com/0005a/Maildir/ sn: 0005a description: testing homeDirectory: /data/mail/example.com/0005a cn: 0005a userPassword:: e0NSWVBUfWxpOXBVaEw5SWJkRWc= objectClass: inetOrgPerson objectClass: CourierMailAccount objectClass: top I think I need to use different user_filter for lmtp and pop3, but it's not possible or I cannot find how to do it. Please, help! -- Best regards, dammit From marc at perkel.com Fri Feb 17 19:09:16 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 09:09:16 -0800 Subject: [Dovecot] 2.1 upgrade docs? Message-ID: <4F3E89BC.3070107@perkel.com> Just wondering if there's any upgrade docs for 2.1 online? From e-frog at gmx.de Fri Feb 17 19:17:03 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 17 Feb 2012 18:17:03 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E89BC.3070107@perkel.com> References: <4F3E89BC.3070107@perkel.com> Message-ID: <4F3E8B8F.1000208@gmx.de> On 17.02.2012 18:09, wrote Marc Perkel: > Just wondering if there's any upgrade docs for 2.1 online? http://wiki2.dovecot.org/Upgrading/2.1 From micah at riseup.net Fri Feb 17 19:17:53 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 12:17:53 -0500 Subject: [Dovecot] 2.1 upgrade docs? References: <4F3E89BC.3070107@perkel.com> Message-ID: <877gzlxttq.fsf@algae.riseup.net> Marc Perkel writes: > Just wondering if there's any upgrade docs for 2.1 online? yes there are. (hint: search terms "dovecot upgrade") From micah at riseup.net Fri Feb 17 20:25:44 2012 From: micah at riseup.net (Micah Anderson) Date: Fri, 17 Feb 2012 13:25:44 -0500 Subject: [Dovecot] expunging Message-ID: <87r4xtwc47.fsf@algae.riseup.net> I am having trouble scaling some regular dovecot cleanup operations on our servers. On a daily basis, I'm wanting to do this on each server, which contains its own isolated set of user storage: /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d but these are a very expensive operations. For example, just doing the Spam expunge takes 30 minutes (or more, depending on the load) of heavy disk operations, on each machine it is run on. There are approximately 20k users on each machine. It seems like it does not use the iterate query, but rather it looks into the database at the expires table and rather it iterates over every user mentioned there. This is a problem because I've got multiple dovecot machines with different sets of users on them, using the same table, so that means that its doing stat() calls on each system for every user mentioned there and failing to find the user on the filesystem (because the user is on another system). In these cases it spits out an error: doveadm(user): Info: User no longer exists, skipping For users that do exist on this system, it seems to do something on the order of 15 stat() calls, at minimum. What are some things I can do to make this less of an expensive process? If I had a shared storage system that each machine used, this would cut down on the resource waste because I'd only need to run the query once, but unfortuntely, that isn't how these systems were designed. thanks for any ideas, tips etc. micah -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: From frank at moltke28.B.Shuttle.DE Fri Feb 17 20:48:17 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Fri, 17 Feb 2012 19:48:17 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? Message-ID: Hi all, are there any pitfalls known for the migration from 2.0.18 to 2.1? Can I simply stop dovecot, make install and start again? --Frank Elsner From marc at perkel.com Fri Feb 17 20:51:45 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 10:51:45 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error Message-ID: <4F3EA1C1.3080903@perkel.com> Not sure what this means: Warning: fd limit (ulimit -n) is lower than required under max. load (4096 < 20000), because of service auth { client_limit } What do I need to do? From marc at perkel.com Fri Feb 17 21:14:04 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 17 Feb 2012 11:14:04 -0800 Subject: [Dovecot] Upgraded to 2.1 - ulimit error In-Reply-To: <4F3EA1C1.3080903@perkel.com> References: <4F3EA1C1.3080903@perkel.com> Message-ID: <4F3EA6FC.1000902@perkel.com> Never mind - I figured it out. ulimit -n 20000 On 2/17/2012 10:51 AM, Marc Perkel wrote: > Not sure what this means: > > Warning: fd limit (ulimit -n) is lower than required under max. load > (4096 < 20000), because of service auth { client_limit } > > What do I need to do? > > From eric at yorba.org Sat Feb 18 03:54:51 2012 From: eric at yorba.org (Eric Gregory) Date: Fri, 17 Feb 2012 17:54:51 -0800 Subject: [Dovecot] Order of operations in Dovecot Message-ID: Hi, I'm working on an IMAP client and had a question about the order in which commands are responded to in Dovecot. In other IMAP servers I've looked at, when you issue two commands simultaneously (i.e. pipelining) it looks something like this: a fetch something b fetch something else * 1 FETCH (result) a OK FETCH completed * 1 FETCH (result) b OK FETCH completed This works well because it's clear which results are associated with which tag; the 'a' fetch results immediately precede the a's OK response, and likewise the 'b' fetch results immediately precede the b's OK response. But in Dovecot (version 2.0.13) the results of the two simultaneous commands are ordered like this: a fetch something b fetch something else * 1 FETCH (result) * 1 FETCH (result) a OK FETCH completed b OK FETCH completed This makes associating the results of the fetch commands quite a bit more difficult, since it's not entirely clear which results are paired with which tag. My question is this: is this Dovecot's intended behavior, or did I find a bug? I realize the IMAP spec is really vague about this. If this is the intended behavior, is there any advice for clients on how to determine which results are associated with which command? Thanks! - Eric From jk at jkart.de Sat Feb 18 15:21:23 2012 From: jk at jkart.de (Jim Knuth) Date: Sat, 18 Feb 2012 14:21:23 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: References: Message-ID: <4F3FA5D3.8050101@jkart.de> am 17.02.12 19:48 schrieb Frank Elsner : > > Hi all, > > are there any pitfalls known for the migration from 2.0.18 to 2.1? > > Can I simply stop dovecot, make install and start again? THIS also interests me. > > --Frank Elsner > > > > > > -- Mit freundlichen Gr??en, with kind regards, Jim Knuth --------- Alle M?nner hatten schon immer Angst davor, da? die Frauen ohne sie zurechtkommen k?nnten. [Margaret Mead] From tlx at leuxner.net Sat Feb 18 15:37:20 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Sat, 18 Feb 2012 14:37:20 +0100 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F3FA5D3.8050101@jkart.de> References: <4F3FA5D3.8050101@jkart.de> Message-ID: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> >> are there any pitfalls known for the migration from 2.0.18 to 2.1? http://wiki2.dovecot.org/Upgrading/2.1 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From roliverio at vaslibre.org.ve Sat Feb 18 18:30:20 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:00:20 -0430 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F3FD21C.6030305@vaslibre.org.ve> > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > Could you check how it's vpopmail connecting? Looks like a SSL related thing with it. HTH, -- ---------------------------------- Julio C. Ortega - VaSLibre http://www.vaslibre.org.ve Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Sat Feb 18 18:48:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:48:21 +0200 Subject: [Dovecot] Order of operations in Dovecot In-Reply-To: References: Message-ID: <4359C586-3C71-4B98-AE72-CF7BE97D2519@iki.fi> On 18.2.2012, at 3.54, Eric Gregory wrote: > But in Dovecot (version 2.0.13) the results of the two simultaneous > commands are ordered like this: > > a fetch something > b fetch something else > * 1 FETCH (result) > * 1 FETCH (result) > a OK FETCH completed > b OK FETCH completed > > This makes associating the results of the fetch commands quite a bit more > difficult, since it's not entirely clear which results are paired with > which tag. > > My question is this: is this Dovecot's intended behavior, or did I find a > bug? I realize the IMAP spec is really vague about this. If this is the > intended behavior, is there any advice for clients on how to determine > which results are associated with which command? This is intended behavior. And if you issue two long FETCH commands, their replies may even be intertwined. IMAP protocol allows all kinds of variations for this actually (e.g. it would be valid for a server to return each FETCHed field in its own untagged FETCH reply). You're not supposed to care about matching FETCH replies to FETCH commands, you just need to know that you've received all the data you asked for (after all tagged commands have replied). http://imapwiki.org/ClientImplementation From tss at iki.fi Sat Feb 18 18:51:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:51:33 +0200 Subject: [Dovecot] Quota warnings and LMTP In-Reply-To: <32d2edef634080462d4f81b99d64020f@hybrid.pl> References: <32d2edef634080462d4f81b99d64020f@hybrid.pl> Message-ID: <02578EB7-C380-40E8-9D4F-79EBCBD79ECE@iki.fi> On 17.2.2012, at 15.40, joshua at hybrid.pl wrote: > I have dovecot 2.0 configured with virtual users in MySQL, SMTP is > handled by exim which uses dovecot LMTP transport. > > I'd like to test quota warning messages, but in wiki2 there is only > an example for delivering it with dovecot-lda. I tried several ways > of using lmtp binary, but without success. > > How to replace this: > > cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" > From: postmaster at domain.com > Subject: quota warning > > Your mailbox is now $PERCENT% full. > EOF > > with lmtp? This command is only about adding the warning mail to user's mailbox, bypassing any quota. Just because you deliver mails normally via LMTP doesn't mean that you have to do it for warning mails. This same dovecot-lda command should work just fine. From tss at iki.fi Sat Feb 18 18:52:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 18:52:57 +0200 Subject: [Dovecot] expunging In-Reply-To: <87r4xtwc47.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> Message-ID: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> On 17.2.2012, at 20.25, Micah Anderson wrote: > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > It seems like it does not use the iterate query, but rather it looks > into the database at the expires table and rather it iterates over every > user mentioned there. This is a problem because I've got multiple > dovecot machines with different sets of users on them, using the same > table, so that means that its doing stat() calls on each system for > every user mentioned there and failing to find the user on the > filesystem (because the user is on another system). In these cases it > spits out an error: doveadm(user): Info: User no longer exists, skipping Upgrade to v2.1: * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. From campbell at cnpapers.com Sat Feb 18 19:04:49 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 12:04:49 -0500 Subject: [Dovecot] I'm confused about my namespace parms Message-ID: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Hi, I'm new to the list and to dovecot. I'm having a strange situation that provides no errors, so I'm having a little problem diagnosing what's going on. I've been running a Centos 3 mail server with sendmail for our company for quite a few years. I finally got approval for new servers this year and put Centos 6.2 on it. I'm sticking with sendmail, but that's not important. Up to this point, I've got all working fairly well. I've found that if I configure dovecot to use the mixed format (pop and imap INBOX in /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the examples for "namespace", most works fine. Anyone who has used imap previously on our old system and work as if nothing has changed. Anyone who has never used Imap before can't seem to access the server using our webmail app (Horde/Imp). Dovecot is providing both pop and imap. An account that is set up from a normal mail client as imap has no problems as well. Sendmail is set up to create a new user in the /var/spool/mail folder. When I moved from the old server to the new server, I copied /home and /var/spool/mail to the new server - in other words, I kept the same scheme. There are no errors in the maillog, and it indicates that the user is getting logged in and out, but the return to Horde is that the user entered the wrong name or password. I'm thinking that this is probably due to how dovecot is trying to look for the imap folders, but the lack or any errors makes it a little difficult to diagnose. I'm fairly certain that what was installed on the old server was UW-Imap and was provided by the imap-2002 rpm. I tried using the examples for UW-Imap compatability but that didn't work at all. I haven't been able to figure out what's different about the old imap folder contents and what's missing in the newly created imap folders to allow Horde to access this account. I'll look at the Horde config to see if anything is there, but I'm cautious about changing anything that might break the current users' ability. Most of the accounts are set up as POP with the ability to see what's left on the server using Horde, webmail, and IMAP. Any help in tracing this down would really help. I think my biggest problem is I don't understand the "namespace" parameter as well as I should. So there it is. It seems I've got at least 3 areas to concentrate on. Steve Campbell ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From Ralf.Hildebrandt at charite.de Sat Feb 18 19:18:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:18:24 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? Message-ID: <20120218171824.GG8161@charite.de> Today I encountered this after upgrading: Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4104a) [0xb75cc04a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x410bb) [0xb75cc0bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb75cc9b7] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8052bac] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_list_full+0x4af) [0x80530df] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_lsub+0x17) [0x8053537] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](command_exec+0x32) [0x8057c92] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056c7d] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056d95] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_handle_input+0x135) [0x8056fa5] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_input+0x60) [0x80578a0] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x44) [0xb75dafb4] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xce) [0xb75dbe5e] -> /usr/local/lib/dovecot/libdovecot.so.0 a) It should not PANIC (I think) b) How can I globally "fix" the folder names (preferably in such a way that I am able to switch back and forth between 2.0 and 2.1) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From roliverio at vaslibre.org.ve Sat Feb 18 19:17:43 2012 From: roliverio at vaslibre.org.ve (Julio C. Ortega) Date: Sat, 18 Feb 2012 12:47:43 -0430 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <1329584689.4f3fda31e5952@perdition.cnpapers.net> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> Message-ID: <4F3FDD37.2000903@vaslibre.org.ve> El 18/02/12 12:34, Steve Campbell escribi?: > Hi, > > I'm new to the list and to dovecot. I'm having a strange situation that provides > no errors, so I'm having a little problem diagnosing what's going on. > > I've been running a Centos 3 mail server with sendmail for our company for quite > a few years. I finally got approval for new servers this year and put Centos 6.2 > on it. I'm sticking with sendmail, but that's not important. > > Up to this point, I've got all working fairly well. I've found that if I > configure dovecot to use the mixed format (pop and imap INBOX in > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the > examples for "namespace", most works fine. Anyone who has used imap previously > on our old system and work as if nothing has changed. Anyone who has never used > Imap before can't seem to access the server using our webmail app (Horde/Imp). > Dovecot is providing both pop and imap. An account that is set up from a normal > mail client as imap has no problems as well. > > Sendmail is set up to create a new user in the /var/spool/mail folder. When I > moved from the old server to the new server, I copied /home and /var/spool/mail > to the new server - in other words, I kept the same scheme. > > There are no errors in the maillog, and it indicates that the user is getting > logged in and out, but the return to Horde is that the user entered the wrong > name or password. I'm thinking that this is probably due to how dovecot is > trying to look for the imap folders, but the lack or any errors makes it a > little difficult to diagnose. > Looks like a Mail Directory seek problem. what's your mail_location set up to?, also, does this location exist for that NEW user on it's first login? -- ---------------------------------- Julio C. Ortega - VaSLibre Usuario GNU/Linux Valencia - Venezuela -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 836 bytes Desc: OpenPGP digital signature URL: From Ralf.Hildebrandt at charite.de Sat Feb 18 19:25:14 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 18:25:14 +0100 Subject: [Dovecot] Move templates args to override_fields setting? Message-ID: <20120218172514.GH8161@charite.de> After upgrading from 2.0.x to 2.1.x I get a lot of: Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting OK, it's just a warning, but how do I fix that in a 2.0.x compatible way? The only place userdb is used is in this contaxt: # for user*masteruser logins passdb { args = /usr/local/etc/dovecot/dovecot.masteruser driver = passwd-file master = yes } # User via passwd userdb { driver = passwd args = cache_key=%u } -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From micah at riseup.net Sat Feb 18 19:34:14 2012 From: micah at riseup.net (micah anderson) Date: Sat, 18 Feb 2012 12:34:14 -0500 Subject: [Dovecot] expunging In-Reply-To: <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> Message-ID: <87obswvyeh.fsf@algae.riseup.net> On Sat, 18 Feb 2012 18:52:57 +0200, Timo Sirainen wrote: > On 17.2.2012, at 20.25, Micah Anderson wrote: > > > /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d > > /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d > > /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d > > > > It seems like it does not use the iterate query, but rather it looks > > into the database at the expires table and rather it iterates over every > > user mentioned there. This is a problem because I've got multiple > > dovecot machines with different sets of users on them, using the same > > table, so that means that its doing stat() calls on each system for > > every user mentioned there and failing to find the user on the > > filesystem (because the user is on another system). In these cases it > > spits out an error: doveadm(user): Info: User no longer exists, skipping > > Upgrade to v2.1: > > * Expire plugin: Only go through users listed by userdb iteration. > Delete dict rows for nonexistent users, unless > expire_keep_nonexistent_users=yes. Great, i'll give that version a spin, thanks! Is there a similar dict row cleanup for quota? micah From tss at iki.fi Sat Feb 18 19:48:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:48:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Today I encountered this after upgrading: > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe .. > a) It should not PANIC (I think) Right. > b) How can I globally "fix" the folder names (preferably in such a way > that I am able to switch back and forth between 2.0 and 2.1) What does this look in filesystem? Is it not in mUTF-7? If the problem is in filesystem, rename it to mUTF7 (you can use doveadm mailbox mutf7) From tss at iki.fi Sat Feb 18 19:53:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:53:22 +0200 Subject: [Dovecot] expunging In-Reply-To: <87obswvyeh.fsf@algae.riseup.net> References: <87r4xtwc47.fsf@algae.riseup.net> <2B3B320A-3C45-4220-9421-117536819D0F@iki.fi> <87obswvyeh.fsf@algae.riseup.net> Message-ID: On 18.2.2012, at 19.34, micah anderson wrote: >> * Expire plugin: Only go through users listed by userdb iteration. >> Delete dict rows for nonexistent users, unless >> expire_keep_nonexistent_users=yes. > > Great, i'll give that version a spin, thanks! > > Is there a similar dict row cleanup for quota? No. If they're in SQL you should be able to build an SQL query for that.. From tss at iki.fi Sat Feb 18 19:56:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 18 Feb 2012 19:56:15 +0200 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20120218172514.GH8161@charite.de> References: <20120218172514.GH8161@charite.de> Message-ID: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> On 18.2.2012, at 19.25, Ralf Hildebrandt wrote: > After upgrading from 2.0.x to 2.1.x I get a lot of: > Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting > > OK, it's just a warning, but how do I fix that in a 2.0.x compatible > way? .. > # User via passwd > userdb { > driver = passwd > args = cache_key=%u > } Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. From Ralf.Hildebrandt at charite.de Sat Feb 18 20:15:15 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:15:15 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> Message-ID: <20120218181515.GI8161@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Today I encountered this after upgrading: > > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > .. > > a) It should not PANIC (I think) > > Right. > > > b) How can I globally "fix" the folder names (preferably in such a way > > that I am able to switch back and forth between 2.0 and 2.1) > > What does this look in filesystem? Is it not in mUTF-7? If the problem > is in filesystem, rename it to mUTF7 (you can use doveadm mailbox > mutf7) # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe /home/h/e/hermel/Maildir/.Entw&APw-rfe /home/h/l/hlammert/Maildir/.Entw&APw-rfe ... /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Sat Feb 18 20:19:01 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:19:01 +0100 Subject: [Dovecot] Move templates args to override_fields setting? In-Reply-To: <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> References: <20120218172514.GH8161@charite.de> <20F5C754-A32B-46D1-86E7-096A7D116B7F@iki.fi> Message-ID: <20120218181901.GJ8161@charite.de> * Timo Sirainen : > > # User via passwd > > userdb { > > driver = passwd > > args = cache_key=%u > > } > > Remove the cache_key, because 1) it's already %u by default and 2) cache_key setting has never existed for passwd. Removed. Thanks! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From trashcan at odo.in-berlin.de Sat Feb 18 20:26:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 18 Feb 2012 19:26:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror Message-ID: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Hi -- I'm used to dsync my two mailservers running 2.0.16. Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Same error with old running command: vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Here's an excerpt of my doveconf -n: mail> doveconf -n mail_home = /var/mail/homedirs/%n mail_location = mdbox:/var/mail/maildirs/%n passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf driver = sql } Reverted back for the time being (production servers) until I do understand the issue. Searching Google with "Fatal: All your namespaces have a location setting." only shows closed bugs back in 2010. Do I have to upgrade both servers to get syncing running? Any hint, help is highly appreciated. Thanks, Michael From Ralf.Hildebrandt at charite.de Sat Feb 18 20:49:33 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 19:49:33 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218181515.GI8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> Message-ID: <20120218184933.GM8161@charite.de> * Ralf Hildebrandt : > # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* > /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe > /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe > /home/h/e/hermel/Maildir/.Entw&APw-rfe > /home/h/l/hlammert/Maildir/.Entw&APw-rfe > ... > /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe > /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe > /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe Some stats: # Name =================== 2 .Entw&-APw-rfe 9 .Entw&,,0-rfe 91 .Entw&APw-rfe -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 20:59:39 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 19:59:39 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218184933.GM8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: Am 18. Februar 2012 19:49 schrieb Ralf Hildebrandt : > * Ralf Hildebrandt : > >> # find /home -mindepth 5 -maxdepth 5 -type d -name .Entw* >> /home/h/o/hoffmanb/Maildir/.Entw&APw-rfe >> /home/h/d/hdaebrit/Maildir/.Entw&APw-rfe >> /home/h/e/hermel/Maildir/.Entw&APw-rfe >> /home/h/l/hlammert/Maildir/.Entw&APw-rfe >> ... >> /home/p/d/pdeindl/Maildir/.Entw&,,0-rfe >> /home/m/v/mviehweg/Maildir/.Entw&,,0-rfe >> /home/m/k/mkoehn/Maildir/.Entw&,,0-rfe > > Some stats: > > ? ? ?# ?Name > ? ?=================== > ? ? ?2 .Entw&-APw-rfe > ? ? ?9 .Entw&,,0-rfe > ? ? 91 .Entw&APw-rfe > Use something like "Entwuerfe" and not "Entw?rfe". -- Chris From Ralf.Hildebrandt at charite.de Sat Feb 18 21:04:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:04:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <20120218190443.GN8161@charite.de> * Chris : > > Some stats: > > > > ? ? ?# ?Name > > ? ?=================== > > ? ? ?2 .Entw&-APw-rfe > > ? ? ?9 .Entw&,,0-rfe > > ? ? 91 .Entw&APw-rfe > > > > Use something like "Entwuerfe" and not "Entw?rfe". Haha. Tell that to my users. By default, everybody gets a "Drafts" folder anyway. I'm not sure which program insists on creating "Entw?rfe" -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at luigirosa.com Sat Feb 18 21:13:13 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:13:13 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218190443.GN8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> Message-ID: <4F3FF849.8060605@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:04: > I'm not sure which program insists on creating "Entw?rfe" Outlook? ;) Ciao, luigi - -- / +--[Luigi Rosa]-- \ It is apparent to me that the possibilities of the aeroplane, which two or three years ago were thought to hold the solution to the [flying machine] problem, have been exhausted, and that we must turn elsewhere. --Thomas Edison, 1895 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8/+EUACgkQ3kWu7Tfl6ZSbogCgvIReVrSzSLg8LhtyW6d4usqh vYUAn3Br/ACxznjJ9lH0yQ5lXwLPkWvJ =OMT5 -----END PGP SIGNATURE----- From Ralf.Hildebrandt at charite.de Sat Feb 18 21:15:44 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Sat, 18 Feb 2012 20:15:44 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: <20120218191544.GO8161@charite.de> > > I'm not sure which program insists on creating "Entw?rfe" > > Outlook? ;) Isn't it always? I hate this shit. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Sat Feb 18 21:16:03 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 20:16:03 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F3FF849.8060605@luigirosa.com> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> Message-ID: 2012/2/18 Luigi Rosa : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Ralf Hildebrandt said the following on 18/02/12 20:04: > >> I'm not sure which program insists on creating "Entw?rfe" > > > Outlook? ? ?;) Yes... M$ Outlook crap do this :) -- Chris From lists at luigirosa.com Sat Feb 18 21:42:45 2012 From: lists at luigirosa.com (Luigi Rosa) Date: Sat, 18 Feb 2012 20:42:45 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218191544.GO8161@charite.de> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <20120218190443.GN8161@charite.de> <4F3FF849.8060605@luigirosa.com> <20120218191544.GO8161@charite.de> Message-ID: <4F3FFF35.4000600@luigirosa.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ralf Hildebrandt said the following on 18/02/12 20:15: >>> I'm not sure which program insists on creating "Entw?rfe" >> Outlook? ;) > Isn't it always? I hate this shit. Giulty by suspicion. Ciao, luigi - -- / +--[Luigi Rosa]-- \ Airplanes are interesting toys but of no military value. --Marechal Ferdinand Foch, Professor of Strategy, Ecole Superieure de Guerre, 1904 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk8//zUACgkQ3kWu7Tfl6ZRZJgCgpZJr3p0D39ssa6glFlFnH4/X WjMAnRapaPUmf9szE1iC5YajT9/oDpXq =bmTJ -----END PGP SIGNATURE----- From lists at necoro.eu Sat Feb 18 22:39:50 2012 From: lists at necoro.eu (=?UTF-8?B?UmVuw6kgTmV1bWFubg==?=) Date: Sat, 18 Feb 2012 21:39:50 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> Message-ID: <4F400C96.4000505@necoro.eu> Am 18.02.2012 19:59, schrieb Chris: > Use something like "Entwuerfe" and not "Entw?rfe". Come on ... software having problems with non-ASCII-stuff in 2012 is just broken by design. Hence there should be no reason to use 'ue' instead of ? anywhere. - Ren? (P.S.: The intention of this mail was not to blame dovecot at all -- because at least until now, umlauts work exceptionally good here.) > -- > Chris > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From xchris89x at googlemail.com Sat Feb 18 23:31:55 2012 From: xchris89x at googlemail.com (Chris) Date: Sat, 18 Feb 2012 22:31:55 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <4F400C96.4000505@necoro.eu> References: <20120218171824.GG8161@charite.de> <479FC06A-BD3D-4F89-A877-472A192464D7@iki.fi> <20120218181515.GI8161@charite.de> <20120218184933.GM8161@charite.de> <4F400C96.4000505@necoro.eu> Message-ID: 2012/2/18 Ren? Neumann : > Am 18.02.2012 19:59, schrieb Chris: >> Use something like "Entwuerfe" and not "Entw?rfe". > > Come on ... software having problems with non-ASCII-stuff in 2012 is > just broken by design. Hence there should be no reason to use 'ue' > instead of ? anywhere. Dovecot has no problems with non-ASCII stuff. The german language is broken by design ... ? ? ? ?... :-) -- Chris From campbell at cnpapers.com Sun Feb 19 02:35:18 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Sat, 18 Feb 2012 19:35:18 -0500 Subject: [Dovecot] I'm confused about my namespace parms In-Reply-To: <4F3FDD37.2000903@vaslibre.org.ve> References: <1329584689.4f3fda31e5952@perdition.cnpapers.net> <4F3FDD37.2000903@vaslibre.org.ve> Message-ID: <1329611718.4f4043c6a9c6f@perdition.cnpapers.net> Quoting "Julio C. Ortega" : > El 18/02/12 12:34, Steve Campbell escribi?: > > Hi, > > > > I'm new to the list and to dovecot. I'm having a strange situation that > provides > > no errors, so I'm having a little problem diagnosing what's going on. > > > > I've been running a Centos 3 mail server with sendmail for our company for > quite > > a few years. I finally got approval for new servers this year and put > Centos 6.2 > > on it. I'm sticking with sendmail, but that's not important. > > > > Up to this point, I've got all working fairly well. I've found that if I > > configure dovecot to use the mixed format (pop and imap INBOX in > > /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in > the > > examples for "namespace", most works fine. Anyone who has used imap > previously > > on our old system and work as if nothing has changed. Anyone who has never > used > > Imap before can't seem to access the server using our webmail app > (Horde/Imp). > > Dovecot is providing both pop and imap. An account that is set up from a > normal > > mail client as imap has no problems as well. > > > > Sendmail is set up to create a new user in the /var/spool/mail folder. When > I > > moved from the old server to the new server, I copied /home and > /var/spool/mail > > to the new server - in other words, I kept the same scheme. > > > > There are no errors in the maillog, and it indicates that the user is > getting > > logged in and out, but the return to Horde is that the user entered the > wrong > > name or password. I'm thinking that this is probably due to how dovecot is > > trying to look for the imap folders, but the lack or any errors makes it a > > little difficult to diagnose. > > > > Looks like a Mail Directory seek problem. what's your mail_location set > up to?, also, does this location exist for that NEW user on it's first > login? > > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > Usuario GNU/Linux > Valencia - Venezuela > > > My 10-mail.conf has the following: mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u and my namespaces are as follows: namespace { type = private separator = / prefix = "#mbox/" location = mbox:~/mail:INBOX=/var/mail/%u inbox = yes hidden = yes list = no } namespace { type = private separator = / prefix = location = maildir:~/mail } As I stated, I need to brush up on what all this really means. So far, reading the documentation has cleared up much for me. I'm sure it'll make sense later, but for now, I'm a little hazy. This machine was tested last weekend, and I had problems with reading any email. I'm using the examples found in the wiki and the above worked. Our old server is so anemic that we're getting delays of multiple hours for delivery after the server accepts it, so this weekend, I left the new server up and running. Thanks for any help and the replies. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From alessio at skye.it Sun Feb 19 12:39:02 2012 From: alessio at skye.it (Alessio Cecchi) Date: Sun, 19 Feb 2012 11:39:02 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL Message-ID: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Hi, I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" and SASL for postfix's SMTP-Auth, like explained ih this howto: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL Authentications over IMAP (local dovecot -> remote dovecot) works fine, and it's great. Also smtp-auth on postfix (over SASL) works fine. But in the dovecot log I found this error: Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): child 30506 killed with signal 11 (core dumps disabled) even though all works fine. This is my dovecot configuration: # 2.1.0: dovecot.conf-imapc # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_mechanisms = plain login default_login_user = nobody imapc_host = 10.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=10.0.0.1 default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl = no userdb { driver = prefetch } But if instead of "unix_listener /var/spool/postfix/private/auth" I use: service auth { unix_listener auth-userdb { } inet_listener { port = 12345 } } no errors appears in the log. In both cases all works fine. Is there any errors in my dovecot's configuration? -- Alessio Cecchi is: on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi Linux -> http://www.cecchi.biz/ Hosting Linux -> http://www.hosting-linux.it @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From mstevens at imt-systems.com Sun Feb 19 14:11:08 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 19 Feb 2012 13:11:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: On 19.02.2012 11:39, Alessio Cecchi wrote: > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: > service(auth): child 30506 killed with signal 11 (core dumps > disabled) signal 11 = segmentation fault That's probably a dovecot bug. Is it possible to reproduce this signal 11 crash? See: http://dovecot.org/bugreport.html Best regards, Morten From stephan at rename-it.nl Sun Feb 19 14:21:44 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 19 Feb 2012 13:21:44 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. Message-ID: <4F40E958.1090102@rename-it.nl> Hi, For our testing enthusiasts I've created new automatic Debian repositories for Dovecot v2.2. Repositories are available for Debian oldstable, stable and testing, on both i386 and am64. Read the following wiki item for more information: http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages The repository for Dovecot v1.2 has been defunct for a few months now and is expected to be removed soon. For the coming month or so, the builders at Xi will not be active during Dutch daytime. There are some issues with the virtual host's I/O performance that need to be resolved first. Regards, Stephan. From postmaster at mathiesen.info Sun Feb 19 14:53:25 2012 From: postmaster at mathiesen.info (postmaster) Date: Sun, 19 Feb 2012 13:53:25 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F40F0C5.2010100@mathiesen.info> I'm completely stumped by this problem and I've tried everything at present - including installing dovecot from scratch. The problem is that when trying to get email from the server -AND- there's email in the account, the pop3 or imap process freezes and no email is fetched. The crazy thing is, that it has been working perfectly, but suddenly I'm getting these freezes both when using telnet from localhost, roundcubemail on the same server and Seamonkey from another computer. Logging seems to work perfectly. For an account without email, dovecot finds the maildir++ and acts correctly. For an account -with- email, dovecot seems to find the maildir++, but also seems to try Namespace of which I haven't defined any. I've tried to get rawlog to work also, but that didn't work - the telnet process froze on login even though I'm quite sure I did the right thing and ps showed an active rawlog process Postfix is delivering the email straight into the mailboxes. Here is the information 4604ds1-ynoe:~ root# port installed | grep dovecot dovecot2 @2.0.18_0+mysql5+postgresql91 (active) dovecot2-sieve @0.2.5_1 (active) postfix @2.9.0_0+dovecot_sasl+mysql5+pcre+postgresql91 (active) 4604ds1-ynoe:~ root# 4604ds1-ynoe:log root# cat dovecot.log.txt Feb 19 11:46:52 4604ds1-ynoe dovecot[441]: master: Dovecot v2.0.18 starting up (core dumps disabled) Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=495) Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49180 resp=AGFidXNlQG1hdGhpZXNlbi5pbmZvAHRlc3R0ZXN0 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: mysql(/macports/var/run/mysql5/mysqld.sock): Connected to database infomathiesenpostfixadmin Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=abuse at mathiesen.info Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 738852865 495 1 687658e957c0e4f473891548eecc49be Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 738852865 abuse at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=503, secured Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/storage) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse at mathiesen.info, alt= Feb 19 11:53:01 4604ds1-ynoe dovecot[443]: imap(abuse at mathiesen.info): Disconnected: Logged out bytes=45/750 Feb 19 11:53:06 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=512) Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49182 resp=AG1hY2ludG9zaEBtYXRoaWVzZW4uaW5mbwB0ZXN0dGVzdA== Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=macintosh at mathiesen.info Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 1866334209 512 1 332714ca2a070d7adb342d4baba4a19a Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'macintosh at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 1866334209 macintosh at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=522, secured Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/storage) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info, alt= Feb 19 11:54:13 4604ds1-ynoe dovecot[443]: imap(macintosh at mathiesen.info): Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh at mathiesen.info: mode=0700 gid=-1 Feb 19 12:12:16 4604ds1-ynoe dovecot[441]: master: Error: service(imap): child 522 killed with signal 3 Feb 19 12:15:48 4604ds1-ynoe dovecot[441]: master: Warning: Killed with signal 15 (by pid=1086 uid=0 code=unknown 0) 4604ds1-ynoe:log root# 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 441 1 441 56804e0 0 Ss ?? 0:00.01 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 442 441 441 56804e0 0 S ?? 0:00.01 dovecot/anvil root 443 441 441 56804e0 0 S ?? 0:00.01 dovecot/log 5000 522 441 441 56804e0 0 S ?? 0:00.01 dovecot/imap root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 543 380 542 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# kill -QUIT 522 4604ds1-ynoe:mail root# dovecot stop 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 1091 380 1090 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "abuse at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1326334461] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a OK [READ-WRITE] Select completed. a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "macintosh at mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox Connection closed by foreign host. 4604ds1-ynoe:~ root# 4604ds1-ynoe:~ root# dovecot -n # 2.0.18: /macports/etc/dovecot/dovecot.conf # OS: Darwin 10.8.0 i386 hfs auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /macports/var/run/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 mail_debug = yes mail_gid = 5000 mail_location = maildir:/Volumes/BrugerData/PostFix/mail/%d/%u mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 service postlogin { executable = script-login -d rawlog } ssl = no syslog_facility = local2 userdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol pop3 { pop3_client_workarounds = oe-ns-eoh outlook-no-nuls } 4604ds1-ynoe:~ root# From robert at schetterer.org Sun Feb 19 17:30:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 19 Feb 2012 16:30:37 +0100 Subject: [Dovecot] Automatic Debian packages for Dovecot v2.2. In-Reply-To: <4F40E958.1090102@rename-it.nl> References: <4F40E958.1090102@rename-it.nl> Message-ID: <4F41159D.6000803@schetterer.org> Am 19.02.2012 13:21, schrieb Stephan Bosch: > Hi, > > For our testing enthusiasts I've created new automatic Debian > repositories for Dovecot v2.2. Repositories are available for Debian > oldstable, stable and testing, on both i386 and am64. Read the following > wiki item for more information: > > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > > The repository for Dovecot v1.2 has been defunct for a few months now > and is expected to be removed soon. > > For the coming month or so, the builders at Xi will not be active during > Dutch daytime. There are some issues with the virtual host's I/O > performance that need to be resolved first. > > Regards, > > Stephan. Hi Stephan, anyway, thx for keeping care of the rep ! -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From rob0 at gmx.co.uk Sun Feb 19 20:09:27 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Sun, 19 Feb 2012 12:09:27 -0600 Subject: [Dovecot] A Postfix/Dovecot example with SQLite backend [crossposted] Message-ID: <20120219180926.GB14195@harrier.slackbuilds.org> There are many mail howtos on the web ... can one more hurt? http://rob0.nodns4.us/howto/README http://rob0.nodns4.us/howto/ http://rob0.nodns4.us/howto/latest.tar.gz (all files) (Sorry, not HTML yet. That is on the agenda.) This is a multiple address class sample implementation of a Postfix MTA and Dovecot IMAP server using a SQLite3 data backend. Domain lookups, user maps, access and transport maps: all using a single, shared SQLite database file. What, other than the SQLite backend, distinguishes this from other mail system howtos? The Postfix high points include a complete implementation of all address classes and per-address transport(5) maps, virtual(8) UID/GID maps, and smtpd(8) recipient access(5) maps. (The latter is using smtpd_restriction_classes, which are not discussed in detail, but are implemented in an interesting way.) On the Dovecot side, it's mostly standard stuff. The SQL deny userdb implementation, and the seamless integration of system and SQL users, might be interesting. I think the database itself is the best part of this example. It's as close to "normalized" as I think it can reasonably be. A significant fact is that each revision of the system has tended to simplify the schema. That's a good sign, I think. One central Domain table lists all domains and hostnames to which the server makes reference. Likewise, a central Address table lists all addresses (with a pointer to the Domain table for each record.) The Alias table defines relationships between Address entries. (Both local(8) and virtual(5) alias maps exist in that table.) Comments and suggestions are welcome, on-list if it's topical to whichever list (please don't crosspost unless comments are relevant to both lists), or offlist to the address in the README file (or as detailed below.) Thanks for your interest. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From trashcan at odo.in-berlin.de Sun Feb 19 23:14:21 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 19 Feb 2012 22:14:21 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <622C2EAD-4DEA-4603-9C56-94CA87A2FC06@odo.in-berlin.de> Hi -- On 18.02.2012, at 19:26, Michael Grimm wrote: > Today, I did upgrade one server to 2.1, and doveadm sync is throwing to following error: > > mail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. [...] > Do I have to upgrade both servers to get syncing running? JFTR: 1) Upgrading both servers ends up in identical error messages. 2) After some investigation I came across a hidden upgrade info in [1], but 'doveadm dsync-server' doesn't run either :-( I'm stuck, thus I did upgrade to 2.0.18 instead ... Regards, Michael [1] http://dovecot.org/list/dovecot/2012-January/062933.html From finn_buhelt at privat.dk Mon Feb 20 01:19:17 2012 From: finn_buhelt at privat.dk (Finn_Buhelt privat) Date: Mon, 20 Feb 2012 00:19:17 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3FD21C.6030305@vaslibre.org.ve> References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <4F418375.7080609@privat.dk> Hi Julio. I'm not sure what You're asking. Can You specify and I will provide the information. I'm so puzzled since this is easy to install and I have done so before without many issues. I think I'd better get the source and go through it . Cheers, Finn B On 02/18/2012 05:30 PM, Julio C. Ortega wrote: >> Dovecot.log >> >> master: Error: Service(auth): command startup failed, throttling >> log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) >> pop3-login: Error: Timeout waiting for handshake from auth server. my >> pid=1858, input bytes=0 >> >> >> Regards, >> Finn B >> > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > From achekalin at lazurit.com Mon Feb 20 08:57:15 2012 From: achekalin at lazurit.com (Alexander Chekalin) Date: Mon, 20 Feb 2012 09:57:15 +0300 Subject: [Dovecot] Homedir vs locations vs mail_location? Message-ID: <4F41EECB.2070405@lazurit.com> Hello, I use Dovecot for rather long time, but I'm still in doubt for some small things. Here they are: 1. The homedir value points to the place where everything for the user stored at, while mail_location is something (some place) where mail stored at. if I deal with pure virtual users (all users are in sql tables and no system homes for them at all), should I ever care for returning meaningful value for 'homedir' (via password_query's userdb_home), or I can simple return empty or constant ('' or '123') for it and it won't mess anything? 2. If I use single (default) namespace, should I set namespace's location (to the same value as global mail_location), and should I expect anything strange if I skip it to set? Reversely, is it possible not to set global mail_location and set only namespace's location (which would be more logical as namespace definition is compact and easy to find in config)? I've reread my questions and see it is dummy ones, but I'd like to know that for sure. Thank you in advance, Alexander From alessio at skye.it Mon Feb 20 09:26:25 2012 From: alessio at skye.it (Alessio Cecchi) Date: Mon, 20 Feb 2012 08:26:25 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? In-Reply-To: <4F3C5634.2010906@kirstineslund.dk> References: <4F3C5634.2010906@kirstineslund.dk> Message-ID: <4F41F5A1.30401@skye.it> Il 16/02/2012 02:04, Finn B ha scritto: > Moi Timo / all. > > I've been using Dovecot on my Qmailtoaster production mailserver for > some years - since 1.0.7 and current Version 2.0.10 - works perfectly. > > Now I am installing a backupsystem and a VM testsystem (2 boxes) - and > no matter how and what Version I am installing on these 2 boxes I get > below messages whenever I try to test connection with Telnet (localhost > 110 / 995). > I have tried to install all Dovecot versions from 2.010 to 2.018 as > rpms, yums and as source and compiled it. > > It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I > have done many times before) > I have tried to copy all my settings from my current production system > (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot > 2.0.10) - nothing helps. > > Now I come to think of it I realize that my production system is intel > based and the 2 new ones are AMD based and wonder if this may be the case ? > > > > Dovecot.log > > master: Error: Service(auth): command startup failed, throttling > log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) > pop3-login: Error: Timeout waiting for handshake from auth server. my > pid=1858, input bytes=0 > > > Regards, > Finn B > > > Hi, can you check if your vpopmail is build with "--disable-auth-logging" option? If yes first rebuild vpopmail without this options, rebuild dovecot and check again. Bye -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From janfrode at tanso.net Mon Feb 20 11:24:53 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Mon, 20 Feb 2012 10:24:53 +0100 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <4F41EECB.2070405@lazurit.com> References: <4F41EECB.2070405@lazurit.com> Message-ID: <20120220092453.GA6316@dibs.tanso.net> On Mon, Feb 20, 2012 at 09:57:15AM +0300, Alexander Chekalin wrote: > > 1. The homedir value points to the place where everything for the > user stored at, while mail_location is something (some place) where > mail stored at. if I deal with pure virtual users (all users are in > sql tables and no system homes for them at all), should I ever care > for returning meaningful value for 'homedir' (via password_query's > userdb_home), or I can simple return empty or constant ('' or '123') > for it and it won't mess anything? Dovecot will store non-mailfiles in the homedir. F.ex. quota-files, sieve scripts, subscription file, .dovecot-lda.dupes, and probably more. So do yourself a favour and create a real homedir for each user :-) http://wiki2.dovecot.org/VirtualUsers/Home > > 2. If I use single (default) namespace, should I set namespace's > location (to the same value as global mail_location), and should I > expect anything strange if I skip it to set? Reversely, is it > possible not to set global mail_location and set only namespace's > location (which would be more logical as namespace definition is > compact and easy to find in config)? > We have a single namespace, with blank location: namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } But I don't really know the purpose of this location field vs. mail_location. -jf From aydin.demirel at endersys.com Mon Feb 20 11:29:43 2012 From: aydin.demirel at endersys.com (=?UTF-8?B?QXlkxLFuIERlbWlyZWw=?=) Date: Mon, 20 Feb 2012 11:29:43 +0200 Subject: [Dovecot] Feature request for maillog plugin Message-ID: <4F421287.3070101@endersys.com> Hi, Is it possible to add the modification time and file name of the mail file to maillog plugin? Thanks -- *Ayd?n Demirel Endersys Ltd. Sistem Destek M?hendisi/ System Support Engineer* * *<> Phone : +90 216 470 9423 | GSM : +90 530 401 8203 Fax : +90 216 470 9508 | Web : http://www.endersys.com Blog : http://blog.endersys.com Twitter : http://www.twitter.com/endersys LPI : The #1 Linux Certification for IT Professionals LPI (Linux Professional Institute) Turkey http://www.lpi-turkey.com From l.chelchowski at eurocar.pl Mon Feb 20 13:59:52 2012 From: l.chelchowski at eurocar.pl (l.chelchowski at eurocar.pl) Date: Mon, 20 Feb 2012 12:59:52 +0100 Subject: [Dovecot] virtual plugin and shared namespace Message-ID: <4F4235B8.6090107@eurocar.pl> Hi, I have a problem with virtual plugin and geting mails form shared namespace. dovecot-virtual: ---- Shared/* Public/* * unseen ---- dovecot.conf: ---- namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/%d/%u/control/public:INDEX=/var/mail/vmail/%d/%u/index/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace { list = children location = maildir:%%h/:INDEX=/var/mail/vmail/%d/%u/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children prefix = Virtual/ separator = / location = virtual:~/virtual } ---- When I enter the virtual mailbox Unread I get this debug info: virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ Is it a bug or misconfig? -- ?ukasz Che?chowski From mhlavink at redhat.com Mon Feb 20 18:11:14 2012 From: mhlavink at redhat.com (Michal Hlavinka) Date: Mon, 20 Feb 2012 17:11:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F3E8B8F.1000208@gmx.de> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> Message-ID: <4F4270A2.2040005@redhat.com> On 02/17/2012 06:17 PM, e-frog wrote: > On 17.02.2012 18:09, wrote Marc Perkel: >> Just wondering if there's any upgrade docs for 2.1 online? > > http://wiki2.dovecot.org/Upgrading/2.1 > I know about one Fedora user who had to also change: namespace { .... inbox=yes ... } to namespace inbox { .... and also in sieve (for pigeonhole): change include :personal "00-script1.sieve"; to include :personal "00-script1"; (I did not test dovecot 2.1, I still have 2.0.x on my server) From busseniu at in.tum.de Mon Feb 20 18:17:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Mon, 20 Feb 2012 17:17:28 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F427218.80709@in.tum.de> On 15.02.2012 04:46, Timo Sirainen wrote: > On 13.2.2012, at 16.40, Timo Sirainen wrote: >> It should deduplicate when GUIDs are the same.. I guess I'll have to look into why it's not working. > > The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. After replacing "doveadm/dsync" with "dsync", the patch applied in 2.0.18 and works fine. (Is there any chance this will be in a 2.0 bugfix release?) Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From jcblanco at fi.upm.es Mon Feb 20 18:43:18 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Mon, 20 Feb 2012 17:43:18 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: <4F427826.3050506@fi.upm.es> On 20/02/2012 17:11, Michal Hlavinka wrote: > On 02/17/2012 06:17 PM, e-frog wrote: >> On 17.02.2012 18:09, wrote Marc Perkel: >>> Just wondering if there's any upgrade docs for 2.1 online? >> >> http://wiki2.dovecot.org/Upgrading/2.1 >> > > I know about one Fedora user who had to also change: > namespace { > .... > inbox=yes > ... > } > > to namespace inbox { .... > > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; > > (I did not test dovecot 2.1, I still have 2.0.x on my server) I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like user_attrs=....,ldap-attribute=quota_rule=*:storage that worked fine on version 2.0, will cause an unlimited quota for the user in version 2.1, you must adjust your configuration to look like user_attrs=....,ldap-attribute=quota_rule=*:storage=%$ as advised in the dovecot wiki http://wiki2.dovecot.org/Quota/Configuration#LDAP Regards Juan C. Blanco From metro_domain_admin at fastmail.fm Mon Feb 20 19:30:11 2012 From: metro_domain_admin at fastmail.fm (Metro Domain Admin) Date: Mon, 20 Feb 2012 12:30:11 -0500 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> Message-ID: <4F428323.8050404@fastmail.fm> On 2/18/12 8:37 AM, Thomas Leuxner wrote: >> are there any pitfalls known for the migration from 2.0.18 to 2.1? > http://wiki2.dovecot.org/Upgrading/2.1 Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html From campbell at cnpapers.com Mon Feb 20 20:10:10 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:10:10 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion Message-ID: <4F428C82.4010700@cnpapers.com> The more I read about all of this, the more I'm thinking about moving to maildir format. My switchover this weekend is full of holes due to the way user's imap folders were laid out. Some had folders in their home directory and others might have folders in their /home/mail directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is not doing so well. I've been reading about the conversion technique to convert to maildir, but I'm wondering if I can do this based on the way our users are set up. Most are pop accounts. Our webmail is configured to read the inbox from /var/spool/mail and their imap folders from /home or /home/mail. Some have imap accounts on their desktop defined through their mail client. RH/Centos seems to want me to switch to postfix since most of the documentation I find doesn't mention Sendmail. My filesystem is laid out to handle Inboxes in /var/spool/mail and imap folders in /home(/mail) and these aren't logical volumes but true partitions, so I've got to consider this as I make any changes. I also run pop using dovecot, so I'll need to figure all this out. I've read the technique linked to on the dovecot site that points to "Replacing UW-IMAP with Dovecot on RHEL 3" written back in 2004, and that's about the only real pointers I'm finding. I don't want to have to go around to each machine and change their clients to IMAP.. Anyone care to comment on anything similar they've done and how many pitfalls they ran into and what they were? steve campbell From CMarcus at Media-Brokers.com Mon Feb 20 20:28:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:28:49 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F4290E1.1020000@Media-Brokers.com> On 2012-02-20 1:10 PM, Steve Campbell wrote: > Our webmail is configured to read the inbox from /var/spool/mail and > their imap folders from /home or /home/mail. I can't help with your specific problem, but I do know that having mail stored directly in /home will cause problems - you will need to fix that. -- Best regards, Charles From campbell at cnpapers.com Mon Feb 20 20:34:11 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 20 Feb 2012 13:34:11 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F4290E1.1020000@Media-Brokers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> Message-ID: <4F429223.1040202@cnpapers.com> On 2/20/2012 1:28 PM, Charles Marcus wrote: > On 2012-02-20 1:10 PM, Steve Campbell wrote: >> Our webmail is configured to read the inbox from /var/spool/mail and >> their imap folders from /home or /home/mail. > > I can't help with your specific problem, but I do know that having > mail stored directly in /home will cause problems - you will need to > fix that. That was a type. It was supposed to be ~ (user's home) or ~/mail. Thanks anyway for pointing this out. steve From CMarcus at Media-Brokers.com Mon Feb 20 20:45:01 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 20 Feb 2012 13:45:01 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F429223.1040202@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F4290E1.1020000@Media-Brokers.com> <4F429223.1040202@cnpapers.com> Message-ID: <4F4294AD.4020505@Media-Brokers.com> On 2012-02-20 1:34 PM, Steve Campbell wrote: > > > On 2/20/2012 1:28 PM, Charles Marcus wrote: >> On 2012-02-20 1:10 PM, Steve Campbell wrote: >>> Our webmail is configured to read the inbox from /var/spool/mail and >>> their imap folders from /home or /home/mail. >> >> I can't help with your specific problem, but I do know that having >> mail stored directly in /home will cause problems - you will need to >> fix that. > > That was a type. It was supposed to be ~ (user's home) or ~/mail. Right - that's what I meant... you do not want to ever store mail directly in the users home directory (~)... -- Best regards, Charles From mstevens at imt-systems.com Mon Feb 20 21:38:20 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Mon, 20 Feb 2012 20:38:20 +0100 Subject: [Dovecot] =?utf-8?q?2=2E1_upgrade_docs=3F?= In-Reply-To: <4F4270A2.2040005@redhat.com> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> Message-ID: On 20.02.2012 17:11, Michal Hlavinka wrote: > and also in sieve (for pigeonhole): > change > include :personal "00-script1.sieve"; > to > include :personal "00-script1"; Hi Michal, See pigeonhole changelog: v0.2.6 13-02-2012 Stephan Bosch * This release fixes unintentional behavior of the include extension. Included scriptnames with a name like "name.sieve" would implicitly map to a script file called "name.sieve" and not "name.sieve.sieve". Keep in mind that the .sieve file extension has no meaning from within the Sieve language. A Sieve script is always stored with an appended .sieve file extension, also when the name already ends with a .sieve suffix. IMPORTANT: Some installations have relied on this unintentional feature, so check your script includes for issues before upgrading. Best regards, Morten From jtl+dovecot at uvm.edu Mon Feb 20 21:40:29 2012 From: jtl+dovecot at uvm.edu (Jim Lawson) Date: Mon, 20 Feb 2012 14:40:29 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F428C82.4010700@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> Message-ID: <4F42A1AD.2000304@uvm.edu> On 2/20/12 1:10 PM, Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving > to maildir format. My switchover this weekend is full of holes due to > the way user's imap folders were laid out. Some had folders in their > home directory and others might have folders in their /home/mail > directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is > not doing so well. Not sure what problem maildir will solve for you. Large mbox mailboxes under UW-IMAP are an awful bottleneck and will bring the most powerful storage subsystems I've used to their knees, but add Dovecot indexing on top and you will be surprised how much faster everything is. We're still using mbox today and if we were to change it would be to mdbox, not maildir. > I've been reading about the conversion technique to convert to > maildir, but I'm wondering if I can do this based on the way our users > are set up. Most are pop accounts. Our webmail is configured to read > the inbox from /var/spool/mail and their imap folders from /home or > /home/mail. Some have imap accounts on their desktop defined through > their mail client. When we did our conversion from uw-imap to dovecot (about 30k accounts) we had to do a lot of clean-up work. Mail folders in the homedir needed to be moved to ~/mail. This can be scripted: if you're using mbox format, look for the first line of the file starting with "From ". (note the space) We use the uw-imap backwards compatibility option documented at http://wiki.dovecot.org/Namespaces#Backwards_Compatibility:_UW-IMAP which allows users who have specified "mail/" as their IMAP namespace prefix to see the same message folders as people who don't. This works most of the time, but we have seen some clients (old versions of Horde/IMP) which still try to create a mail/ folder, which ends up creating ~/mail/mail, and that becomes inaccessible. Fortunately, since moving to Horde 4/IMP 5 it hasn't been a problem. You might want to keep an eye out for this if you go the backwards compatibility route. We also kept (and still keep) inboxes in /var/spool/mail (which is actually a link tree, pointing to other filesystems.) We had enough users that we needed to do "shard" directories to prevent directory lock contention from bringing everything to a crawl, e.g. jsmith's mail is in /var/spool/mail/j/s/jsmith. Dovecot nicely supports this as a config option. > > RH/Centos seems to want me to switch to postfix since most of the > documentation I find doesn't mention Sendmail. My filesystem is laid > out to handle Inboxes in /var/spool/mail and imap folders in > /home(/mail) and these aren't logical volumes but true partitions, so > I've got to consider this as I make any changes. We still use sendmail here, which calls procmail for delivery, mostly for historical reasons - many users have complex .procmailrc files. If I had to start over I'd certainly give postfix a shot. > > I also run pop using dovecot, so I'll need to figure all this out. Make sure you've read http://wiki.dovecot.org/Migration/UW , particularly for the pop3_uidl_format option, so clients do not re-download everything when you switch from UW -> Dovecot. Cheers, Jim Lawson From solo at privat.dk Tue Feb 21 01:44:30 2012 From: solo at privat.dk (solo at privat.dk) Date: Tue, 21 Feb 2012 00:44:30 +0100 Subject: [Dovecot] Something is strange - perhaps my AMD processors ? References: <4F3C5634.2010906@kirstineslund.dk> <4F3FD21C.6030305@vaslibre.org.ve> Message-ID: <70.4B.28132.FDAD24F4@fep46.mail.dk> Hi Julio / All. I've found the issue : the password was wrong/missing in /home/vpopmail/etc/vpopmail.mysql Now all is well. Thanks,for Your help Finn B ----- Original meddelelse ----- > Fra: Julio C. Ortega > Til: dovecot at dovecot.org > Dato: L?r, 18. feb 2012 16:30 > Emne: Re: [Dovecot] Something is strange - perhaps my AMD processors > ? > > > > Dovecot.log > > > > master: Error: Service(auth): command startup failed, throttling > > log: Error: Service(auth): child 1859 returned error 89 (Fatal > failure) > > pop3-login: Error: Timeout waiting for handshake from auth server. > my > > pid=1858, input bytes=0 > > > > > > Regards, > > Finn B > > > > Could you check how it's vpopmail connecting? > > Looks like a SSL related thing with it. > > HTH, > > -- > ---------------------------------- > Julio C. Ortega - VaSLibre > http://www.vaslibre.org.ve > Venezuela From tss at iki.fi Tue Feb 21 02:17:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:17:20 +0200 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: <4F427826.3050506@fi.upm.es> References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: On 20.2.2012, at 18.43, Juan C. Blanco wrote: > I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. > > You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like > > user_attrs=....,ldap-attribute=quota_rule=*:storage This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 From tss at iki.fi Tue Feb 21 02:18:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:18:34 +0200 Subject: [Dovecot] 2.0.X -> 2.1 pitfalls? In-Reply-To: <4F428323.8050404@fastmail.fm> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> Message-ID: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> On 20.2.2012, at 19.30, Metro Domain Admin wrote: > On 2/18/12 8:37 AM, Thomas Leuxner wrote: >>> are there any pitfalls known for the migration from 2.0.18 to 2.1? >> http://wiki2.dovecot.org/Upgrading/2.1 > > Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Yes, but it should still work.. From jtam.home at gmail.com Tue Feb 21 02:25:46 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 20 Feb 2012 16:25:46 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving to > maildir format. My switchover this weekend is full of holes due to the > way user's imap folders were laid out. Some had folders in their home > directory and others might have folders in their /home/mail directory. I had the same problem, and handled it more or less the same way as the later poster (Jim Lawson ) handled his site. I retained use of mbox format as well. I planned the migration like this - scan user home directories for mailboxes (especially "Trash", "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. This was piped into a script that Email'd users about the changes that was going to happen, what they would expect to see, and a FAQ on how to set up a mail client correctly. - during the cutover, - mailboxes left on the home directory were moved to ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid name collision for users that had a mixture of correct and blank prefixes. (I think I deleted "Trash", "Junk", etc. anyways). - Namespace aliasing was used so that prefixes "", "mail", "~/mail", and whatever darn fool settings my users used, would map to the same directory. - .subscription files were moved into the mail folder (don't have to edit prefixes since the aliasing will take care of that). - depending on what POP3 client you used (I used qpopper), you may need to configure pop3_reuse_xuidl = yes to avoid a massive re-downloading from POP3 clients after cutover. - the mail clients I control centrally (e.g. webmail, public server mail clients, etc). shouldn't need updating since they ought to have been set up properly in the first place. - after cutover, a second notification was sent for users that didn't move their mailboxes the first time around, and was it done for them during migration. And of course, test like crazy and watch the logs like a hawk. Other gotcha's: - your setup is fairly close to mine, so you may also run into the problem of user having mailboxes with group ownership that users are not part of (for example, group "mail" for INBOX set by your LDA or personal mailboxes with groups the user is no longer a member of) *and* with group permission not mode=0. You'll have to treat these (set mode=0, or change the group to something the user is part of), or the dovecot index creation will fail and they won't be able to access their mailboxes. I think my migration went pretty smoothly. Less than a handful of wazzup' Email problem reports. Joseph Tam From tss at iki.fi Tue Feb 21 02:27:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:27:28 +0200 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F40F0C5.2010100@mathiesen.info> References: <4F40F0C5.2010100@mathiesen.info> Message-ID: <3AE863BC-AEFF-478B-BBFA-99E90513F8D8@iki.fi> On 19.2.2012, at 14.53, postmaster wrote: > Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse at mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Probably won't help with your problem, but it's better if you don't fetch "mail" field. Just fetch the home and set: mail_location = maildir:~/ This avoids maildir autodetection. > a list "" * > * LIST (\HasNoChildren) "." "INBOX" > a OK List completed. > a select inbox > Connection closed by foreign host. > 4604ds1-ynoe:~ root# You mean the session hangs here until it's killed? Try attaching gdb into it: gdb -p bt full or alternatively try: gdb --args /usr/local/libexec/dovecot/imap -u user at domain run a select inbox bt full > # OS: Darwin 10.8.0 i386 hfs With HFS+ it's a good idea to set maildir_copy_with_hardlinks=no From tss at iki.fi Tue Feb 21 02:29:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:29:24 +0200 Subject: [Dovecot] Homedir vs locations vs mail_location? In-Reply-To: <20120220092453.GA6316@dibs.tanso.net> References: <4F41EECB.2070405@lazurit.com> <20120220092453.GA6316@dibs.tanso.net> Message-ID: <27AB0CCC-F41B-4F99-A6BA-9EB8F1972DBE@iki.fi> On 20.2.2012, at 11.24, Jan-Frode Myklebust wrote: >> 2. If I use single (default) namespace, should I set namespace's >> location (to the same value as global mail_location), and should I >> expect anything strange if I skip it to set? Reversely, is it >> possible not to set global mail_location and set only namespace's >> location (which would be more logical as namespace definition is >> compact and easy to find in config)? >> > > We have a single namespace, with blank location: .. > But I don't really know the purpose of this location field vs. > mail_location. If namespace location is empty, it defaults to mail_location. Typically you specify namespace location only when you have more than one namespace. From tss at iki.fi Tue Feb 21 02:35:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:35:26 +0200 Subject: [Dovecot] Feature request for maillog plugin In-Reply-To: <4F421287.3070101@endersys.com> References: <4F421287.3070101@endersys.com> Message-ID: On 20.2.2012, at 11.29, Ayd?n Demirel wrote: > Is it possible to add the modification time That would be the same as IMAP INTERNALDATE / received-date. It would be possible to add this pretty easily. > and file name of the mail file to maillog plugin? This would be much more difficult, because the filename isn't known at the time when the logging is normally done. UIDs have the same problem, but they were specially implemented, and adding such support for filenames would be much more extra work. From tss at iki.fi Tue Feb 21 02:44:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 02:44:28 +0200 Subject: [Dovecot] virtual plugin and shared namespace In-Reply-To: <4F4235B8.6090107@eurocar.pl> References: <4F4235B8.6090107@eurocar.pl> Message-ID: On 20.2.2012, at 13.59, l.chelchowski at eurocar.pl wrote: > I have a problem with virtual plugin and geting mails form shared namespace. > dovecot-virtual: > ---- > Shared/* > Public/* > * .. > When I enter the virtual mailbox Unread I get this debug info: > virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test at domain.pl/ * matches only within namespace. Shared/* matches only the entries directly under Shared/, i.e. Shared/test at domain.pl itself. Shared/test at domain.pl/* would again be a separate namespace. > Is it a bug or misconfig? Maybe a "missing feature" :) I guess Shared/* should match all autocreated child namespaces.. This should help: http://hg.dovecot.org/dovecot-2.1/rev/10994a5b64e1 From tss at iki.fi Tue Feb 21 03:02:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:02:00 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> Message-ID: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> On 18.2.2012, at 20.26, Michael Grimm wrote: > doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb From tss at iki.fi Tue Feb 21 03:09:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:09:13 +0200 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <20120218171824.GG8161@charite.de> References: <20120218171824.GG8161@charite.de> Message-ID: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). From tss at iki.fi Tue Feb 21 03:11:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:11:10 +0200 Subject: [Dovecot] Possible broken indexer(lucene/solr)? (Updated: also present in 2.1rc7 perhaps?) In-Reply-To: <4F3DBF7F.4030504@r.paypc.com> References: <4F304595.9030007@iwt-promotion.com> <828FBB24-8FEB-4706-9570-9499D7B462B2@iki.fi> <4F3DBF7F.4030504@r.paypc.com> Message-ID: On 17.2.2012, at 4.46, Robin wrote: >> You mean you deleted Solr index, so that it's empty? That should work too. >> >> Anyway, in v2.1 Dovecot keeps track of what is the last indexed message in dovecot.index files. So if you're switching between backends or have messed things up in tests, you need to run "doveadm fts rescan" (for each user). > > # doveadm(root): Fatal: Unknown command 'fts', but plugin fts exists. Try to set mail_plugins=fts > > I get this, despite having fts + fts_solr defined in 20-imap.conf as recommended with the following plugin format stanza: > > plugin { > fts = solr > fts_solr = break-imap-search url=http://solrhost:8983/solr/ > } Plugin settings don't do anything if plugin isn't enabled. And if you enabled it only for imap, then it works only in imap and not in pop3/lda/doveadm/etc.. > Should I be adding fts/fts_solr to the global mail_plugins setting? Yes. Probably fixes all your troubles. From tss at iki.fi Tue Feb 21 03:13:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:13:05 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > By the way: what might have caused such a warning? > > root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. From tss at iki.fi Tue Feb 21 03:15:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:15:04 +0200 Subject: [Dovecot] user_filter problem In-Reply-To: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> References: <31591.193.219.14.35.1329493193.squirrel@h1.progresas.lt> Message-ID: On 17.2.2012, at 17.39, sanito at progresas.lt wrote: > I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 > with user information stored in the LDAP database. The main problem is > that email address and pop3 username are different for every mailbox (I > cannot change this behaviour, because thousands of people already > configured their email client for such a setup). So in my situation, > user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it > should be: user_filter=(cn=%u). Can't it be either? user_filter = (|(mail=%u)(cn=%u)) > I think I need to use different user_filter for lmtp and pop3, but it's > not possible or I cannot find how to do it. Please, help! With v2.x you can also do something like: protocol lmtp { userdb { driver = ldap args = dovecot-ldap-lmtp.conf.ext } } protocol !lmtp { userdb { driver = ldap args = dovecot-ldap.conf.ext } } From nti at w4w.net Tue Feb 21 03:20:03 2012 From: nti at w4w.net (nti at w4w.net) Date: Tue, 21 Feb 2012 02:20:03 +0100 Subject: [Dovecot] dovecot 2 doesn't start with managesieve Message-ID: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Hi I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). Error: socket() failed: Protocol not supported Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported Fatal: Failed to start listeners I controlled the config, build pigeonhole a second time - nothing helps. Any idea ? Is there a possibility to debug the start? Nicola ... plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve ... service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service managesieve { process_limit = 1024 } ... protocol sieve { managesieve_max_compile_errors = 10 } From tss at iki.fi Tue Feb 21 03:25:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 03:25:26 +0200 Subject: [Dovecot] dovecot 2 doesn't start with managesieve In-Reply-To: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> References: <8C76FC7F-3F9D-4B48-A026-9FC6AE583F71@w4w.net> Message-ID: <48C8BE3A-32D9-41F7-819F-FEEB78A911E3@iki.fi> On 21.2.2012, at 3.20, nti at w4w.net wrote: > I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). > > Error: socket() failed: Protocol not supported > Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported > Fatal: Failed to start listeners > > I controlled the config, build pigeonhole a second time - nothing helps. > > Any idea ? Is there a possibility to debug the start? You don't have IPv6 supported, so listening in "::" fails. Disable it with: listen = * From javierdemiguel at us.es Tue Feb 21 09:01:11 2012 From: javierdemiguel at us.es (=?ISO-8859-1?Q?Javier_Miguel_Rodr=EDguez?=) Date: Tue, 21 Feb 2012 08:01:11 +0100 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: References: Message-ID: <4F434137.6070705@us.es> I have seen this behaviour with a local ext4 iSCSI filesystem. When the system is hammered by I/O (example, perfoming a full backup), I also see those messages in the log. Regards Javier > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > >> By the way: what might have caused such a warning? >> >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > From alessio at skye.it Tue Feb 21 09:46:08 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 08:46:08 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <4F434BC0.2030300@skye.it> Il 19/02/2012 13:11, Morten Stevens ha scritto: > On 19.02.2012 11:39, Alessio Cecchi wrote: >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: >> service(auth): child 30506 killed with signal 11 (core dumps disabled) > > signal 11 = segmentation fault > > That's probably a dovecot bug. > > Is it possible to reproduce this signal 11 crash? See: > http://dovecot.org/bugreport.html > > Best regards, > > Morten > Hi, I will try to provide as much as possibile information to reproduce the bug. Give me only some days to provide all informations. Thank you -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From tss at iki.fi Tue Feb 21 11:09:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:09:01 +0200 Subject: [Dovecot] Recalculate quota when quota=dict ? In-Reply-To: <4F434137.6070705@us.es> References: <4F434137.6070705@us.es> Message-ID: <1329815341.11500.223.camel@innu> Maybe http://hg.dovecot.org/dovecot-2.1/rev/2ea29ab6f10f fixes it? On Tue, 2012-02-21 at 08:01 +0100, Javier Miguel Rodr?guez wrote: > I have seen this behaviour with a local ext4 iSCSI filesystem. When the > system is hammered by I/O (example, perfoming a full backup), I also see > those messages in the log. > > Regards > > Javier > > > > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: > > > >> By the way: what might have caused such a warning? > >> > >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u joshua at hybrid.pl > >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp is different than current time (1329464622 vs 1329464672): /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't happen unless remote storage server's clock and local server's clock aren't synced. > > > From jcblanco at fi.upm.es Tue Feb 21 11:16:14 2012 From: jcblanco at fi.upm.es (Juan C. Blanco) Date: Tue, 21 Feb 2012 10:16:14 +0100 Subject: [Dovecot] 2.1 upgrade docs? In-Reply-To: References: <4F3E89BC.3070107@perkel.com> <4F3E8B8F.1000208@gmx.de> <4F4270A2.2040005@redhat.com> <4F427826.3050506@fi.upm.es> Message-ID: <4F4360DE.7070708@fi.upm.es> On 21/02/2012 1:17, Timo Sirainen wrote: > On 20.2.2012, at 18.43, Juan C. Blanco wrote: > >> I'm just migrating one of our servers to 2.1 and noticed a possible configuration problem that I've missed to detect with the RCs. >> >> You ca have a quota problema if you have a per-user quota based on an ldap attribute and if your configuration file dovecot-ldap.conf.ext uses an old format; if the user attributes line looks like >> >> user_attrs=....,ldap-attribute=quota_rule=*:storage > > This was unintentional. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/e8da2675da41 OK, thanks Timo even if the workaround was not difficult at all > -- +----------------------------------------------------------------+ | Juan C. Blanco | | | | Centro de Calculo | | | Facultad de Informatica U.P.M. | E-mail: jcblanco at fi.upm.es | | Campus de Montegancedo | | | Boadilla del Monte | Tel.: (+34) 91 336 7466 | | 28660 MADRID (Spain) | Fax : (+34) 91 336 6913 | +----------------------------------------------------------------+ From tss at iki.fi Tue Feb 21 11:16:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:16:25 +0200 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> Message-ID: <1329815785.11500.224.camel@innu> On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" > and SASL for postfix's SMTP-Auth, like explained ih this howto: > http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL > > Authentications over IMAP (local dovecot -> remote dovecot) works fine, > and it's great. Also smtp-auth on postfix (over SASL) works fine. But in > the dovecot log I found this error: > > > Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): > child 30506 killed with signal 11 (core dumps disabled) http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. From ngoonee.talk at gmail.com Tue Feb 21 11:33:34 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 17:33:34 +0800 Subject: [Dovecot] passwd-file - lower limit for username? Message-ID: Hi, previously I could use the following in my passwd-file:- AB:{plain}password I'm not sure when it stopped working, its not an account I access with any regularity, but today it would not work (2.1.0, 64-bit Arch Linux), with an 'unknown user' error in the logs. Is there some new lower limit for username length? From Ralf.Hildebrandt at charite.de Tue Feb 21 11:43:16 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 21 Feb 2012 10:43:16 +0100 Subject: [Dovecot] Folder name migration when moving from 2.0.x to 2.1.x? In-Reply-To: <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> References: <20120218171824.GG8161@charite.de> <2BFDDA89-EF7B-4030-90DA-5265489DDBA4@iki.fi> Message-ID: <20120221094316.GD11284@charite.de> * Timo Sirainen : > On 18.2.2012, at 19.18, Ralf Hildebrandt wrote: > > > Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw?rfe > > http://hg.dovecot.org/dovecot-2.1/rev/7d0d76df719f should fix this by removing the invalid entry from subscriptions file (probably broken client added it in some earlier Dovecot version that didn't disallow it). I think that's very likely. I'll update later on. Thanks again. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 21 11:54:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:54:57 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818097.11500.233.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Dovecot doesn't think the user exists. auth_debug=yes may be helpful, and doveconf -n output. > Is there some new lower limit for username length? No. From postmaster at mathiesen.info Tue Feb 21 11:58:27 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 10:58:27 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436AC3.1040800@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 On a note regarding my hardware : the computer has three ethernet-interfaces: but that doesn't really seem to be an issue as telnet from localhost also hangs indefinetely I'm seeing the issue no matter which interface I connect to, whether the two external WAN or the internal LAN IP-number . root# system_profiler SPEthernetDataType 2012-01-22 01:34:36.524 system_profiler[47377:903] Found bsdName (0x2603) for AppleUSBEthernet 2012-01-22 01:34:36.626 system_profiler[47377:903] Found bsdName (0x2c03) for AppleUSBEthernet Ethernet Cards: Apple USB Ethernet Adaptor: Type: USB BSD name: en4 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8a:f3:5a Product ID: 5122 Vendor ID: 1452 Apple USB Ethernet Adaptor: Type: USB BSD name: en3 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8b:26:c1 Product ID: 5122 Vendor ID: 1452 Marvell Yukon Gigabit Adapter 88E8053 Singleport Copper SA: Name: ethernet Type: Ethernet Controller Bus: PCI Vendor ID: 0x11ab Device ID: 0x4362 Subsystem Vendor ID: 0x11ab Subsystem ID: 0x5321 Revision ID: 0x0022 Link Width: x1 BSD name: en0 Kext name: AppleYukon2.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleYukon2.kext Version: 3.2.1b1 ethernet: Type: Ethernet Controller Bus: PCI Slot: AirPort Vendor ID: 0x168c Device ID: 0x001c Subsystem Vendor ID: 0x106b Subsystem ID: 0x0086 Revision ID: 0x0001 Link Width: x1 BSD name: en1 Kext name: AirPortAtheros21.kext Location: /System/Library/Extensions/IO80211Family.kext/Contents/PlugIns/AirPortAtheros21.kext Version: 425.14.7 root# ifconfig lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 gif0: flags=8010 mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863 mtu 1500 ether 00:16:cb:a8:7d:01 inet6 fe80::216:cbff:fea8:7d01%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.37 netmask 0xffffff00 broadcast 192.168.1.255 media: autoselect (1000baseT ) status: active en4: flags=8863 mtu 1500 ether 00:1f:f3:8a:f3:5a inet6 fe80::21f:f3ff:fe8a:f35a%en4 prefixlen 64 scopeid 0x5 inet 90.184.13.xxx netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active en3: flags=8863 mtu 1500 ether 00:1f:f3:8b:26:c1 inet6 fe80::21f:f3ff:fe8b:26c1%en3 prefixlen 64 scopeid 0x6 inet 90.184.12.yyy netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX ) status: active fw0: flags=8863 mtu 2030 lladdr 00:17:f2:ff:fe:8e:e6:d2 media: autoselect status: inactive en1: flags=8863 mtu 1500 ether 00:17:f2:52:62:9e media: autoselect () status: inactive root# system_profiler SPHardwareDataType Hardware: Hardware Overview: Model Name: Mac mini Model Identifier: Macmini1,1 Processor Name: Intel Core Duo Processor Speed: 1,83 GHz Number Of Processors: 1 Total Number Of Cores: 2 L2 Cache: 2 MB Memory: 2 GB Bus Speed: 667 MHz Boot ROM Version: MM11.0055.B08 SMC Version (system): 1.3f4 root# system_profiler SPSoftwareDataType Software: System Software Overview: System Version: Mac OS X 10.6.8 (10K549) Kernel Version: Darwin 10.8.0 Boot Volume: X Boot Mode: Normal Computer Name: MiniWeb User Name: System Administrator (root) Secure Virtual Memory: Enabled 64-bit Kernel and Extensions: No Time since boot: 5 days 1:31 >On 19.2.2012, at 14.53, postmaster wrote: > >> Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: >sql(abuse at mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM >mailbox WHERE username =3D 'abuse at mathiesen.info' AND domain =3D >'mathiesen.info' AND active =3D 1 > >Probably won't help with your problem, but it's better if you don't >fetch "mail" field. Just fetch the home and set: > >mail_location = maildir:~/ > >This avoids maildir autodetection. That seemed to fix the autodetection :-) > >> # OS: Darwin 10.8.0 i386 hfs > >With HFS+ it's a good idea to set maildir_copy_with_hardlinks = no OK - set to no as advised > >> a list "" * >> * LIST (\HasNoChildren) "." "INBOX" >> a OK List completed. >> a select inbox >> Connection closed by foreign host. >> 4604ds1-ynoe:~ root# > >You mean the session hangs here until it's killed? Yes ... It does some of what it's supposed to do : the dovecot-uidlist.lock is set email is copied from new to cur, but then nothing more seems to happen :-( 4604ds1-ynoe:mail root# ls -l mathiesen.info/macintosh\@mathiesen.info/ total 40 drwx------ 2 5000 5000 4352 Feb 21 09:58 cur -rw------- 1 5000 5000 8119 Feb 21 10:28 dovecot-uidlist -rw------- 1 5000 5000 32 Feb 21 10:28 dovecot-uidlist.lock -rw------- 1 5000 5000 8 Feb 21 10:28 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 Feb 21 10:28 dovecot-uidvalidity.4f4363ac -rw------- 1 5000 5000 40 Feb 21 10:28 dovecot.index.log drwxr-xr-x 2 5000 5000 68 Feb 19 11:18 dovecot.rawlog drwx------ 2 5000 5000 68 Feb 21 09:58 new drwx------ 2 5000 5000 68 Feb 21 09:11 tmp 4604ds1-ynoe:mail root# >Try attaching gdb into it: > >gdb -p >bt full 4604ds1-ynoe:~ root# ps -axj | grep dovecot root 40655 1 40655 4d84750 0 Ss ?? 0:00.02 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 40657 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/anvil root 40658 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/log root 40660 40655 40655 4d84750 0 S ?? 0:00.06 dovecot/config _dovecot 40662 40655 40655 4d84750 0 S ?? 0:00.02 dovecot/auth root 40663 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/auth -w 5000 40677 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40689 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40803 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 root 40618 362 40618 5680af8 1 S+ s001 0:00.01 tail -f dovecot.log root 40952 396 40951 4d85d40 2 R+ s003 0:00.00 grep dovecot 4604ds1-ynoe:~ root# gdb -p 40803 GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin". /private/var/root/40803: No such file or directory Attaching to process 40803. Reading symbols for shared libraries . done Reading symbols for shared libraries ...... done 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001f35 in login_client_connected () No symbol table info available. #22 0x000eb1a8 in master_login_auth_finish () No symbol table info available. #23 0x000eae80 in master_login_auth_callback () No symbol table info available. #24 0x000ebb3a in master_login_auth_input () No symbol table info available. #25 0x001001a7 in io_loop_call_io () No symbol table info available. #26 0x00101492 in io_loop_handler_run () No symbol table info available. #27 0x00100148 in io_loop_run () No symbol table info available. #28 0x000ed25a in master_service_run () No symbol table info available. #29 0x00001df8 in main () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from process 40803. 4604ds1-ynoe:~ root# > >or alternatively try: > >gdb --args /usr/local/libexec/dovecot/imap -u user at domain >run >a select inbox > >bt full 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/pop3 -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/: mode=0700 gid=-1 a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001d71 in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From tss at iki.fi Tue Feb 21 11:58:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 11:58:41 +0200 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: References: Message-ID: <1329818321.11500.236.camel@innu> On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: > Hi, previously I could use the following in my passwd-file:- > > AB:{plain}password > > I'm not sure when it stopped working, its not an account I access with > any regularity, but today it would not work (2.1.0, 64-bit Arch > Linux), with an 'unknown user' error in the logs. Is there some new > lower limit for username length? Oh, I think it's because AB is uppercased. Either lowercase it "ab" or set this to empty: auth_username_format = I hadn't thought of this being a problem before.. Hm. Perhaps it's not a common problem, so I'll just change the Upgrading text to mention this possibility also.. From tss at iki.fi Tue Feb 21 12:08:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 12:08:06 +0200 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail In-Reply-To: <4F436AC3.1040800@mathiesen.info> References: <4F436AC3.1040800@mathiesen.info> Message-ID: <1329818886.11500.239.camel@innu> On Tue, 2012-02-21 at 10:58 +0100, postmaster wrote: > > #0 0x90003afa in mach_msg_trap () > No symbol table info available. > #1 0x90004267 in mach_msg () > No symbol table info available. > #2 0x9001b431 in vproc_mig_look_up2 () > No symbol table info available. > #3 0x9001b26a in bootstrap_look_up2 () > No symbol table info available. > #4 0x9001b0e4 in bootstrap_look_up () > No symbol table info available. > #5 0x9001af30 in notify_register_check () > No symbol table info available. > #6 0x9001ada7 in notify_register_tz () > No symbol table info available. > #7 0x9001a621 in tzsetwall_basic () > No symbol table info available. > #8 0x90041903 in localtime () > No symbol table info available. All of the above functions are OSX's internal functions in libc. > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. From postmaster at mathiesen.info Tue Feb 21 12:11:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:11:26 +0100 Subject: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436DCE.5050001@mathiesen.info> [ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 arrrrgh -- used pop3 instead of imap :-( 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/imap -u postmaster at mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/imap -u postmaster at mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster at mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster at mathiesen.info/, alt= * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as postmaster at mathiesen.info a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x00092a2f in mail_index_update_day_headers () No symbol table info available. #10 0x0008ec95 in mail_index_transaction_finish () No symbol table info available. #11 0x0008d80a in mail_index_transaction_commit_v () No symbol table info available. #12 0x0008dacf in mail_index_transaction_commit_full () No symbol table info available. #13 0x0008db18 in mail_index_transaction_commit () No symbol table info available. #14 0x00096434 in mail_index_sync_commit () No symbol table info available. #15 0x00066f02 in maildir_sync_index_finish () No symbol table info available. #16 0x00068376 in maildir_sync_context () No symbol table info available. #17 0x000686f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000295fd in mailbox_sync () No symbol table info available. #19 0x00007833 in cmd_select_full () No symbol table info available. #20 0x00007d49 in cmd_select () No symbol table info available. #21 0x00009f63 in client_command_input () No symbol table info available. #22 0x0000a0c2 in client_command_input () No symbol table info available. #23 0x0000a1d3 in client_handle_input () No symbol table info available. #24 0x000095bd in client_input () No symbol table info available. #25 0x001171a7 in io_loop_call_io () No symbol table info available. #26 0x00118492 in io_loop_handler_run () No symbol table info available. #27 0x00117148 in io_loop_run () No symbol table info available. #28 0x0010425a in master_service_run () No symbol table info available. #29 0x00013c5a in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root# From postmaster at mathiesen.info Tue Feb 21 12:20:26 2012 From: postmaster at mathiesen.info (postmaster) Date: Tue, 21 Feb 2012 11:20:26 +0100 Subject: [Dovecot] dovecot freezes when trying to get mail from maildir with mail Message-ID: <4F436FEA.8000606@mathiesen.info> From: Timo Sirainen > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. OK - Thanks for your help :-) I'll take it up on the macports mailing lists then ! -- Bjarne D Mathiesen K?benhavn N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit milj? MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 From ngoonee.talk at gmail.com Tue Feb 21 12:22:39 2012 From: ngoonee.talk at gmail.com (Oon-Ee Ng) Date: Tue, 21 Feb 2012 18:22:39 +0800 Subject: [Dovecot] passwd-file - lower limit for username? In-Reply-To: <1329818321.11500.236.camel@innu> References: <1329818321.11500.236.camel@innu> Message-ID: On Tue, Feb 21, 2012 at 5:58 PM, Timo Sirainen wrote: > On Tue, 2012-02-21 at 17:33 +0800, Oon-Ee Ng wrote: >> Hi, previously I could use the following in my passwd-file:- >> >> AB:{plain}password >> >> I'm not sure when it stopped working, its not an account I access with >> any regularity, but today it would not work (2.1.0, 64-bit Arch >> Linux), with an 'unknown user' error in the logs. Is there some new >> lower limit for username length? > > Oh, I think it's because AB is uppercased. Either lowercase it "ab" or > set this to empty: > > auth_username_format = > > I hadn't thought of this being a problem before.. Hm. Perhaps it's not a > common problem, so I'll just change the Upgrading text to mention this > possibility also.. > Yes, its probably the upper-casing. No problem, its just that it used to work and now doesn't, I've already changed the username (to something longer which isn't all uppercase) anyway. Probably SHOULD be mentioned briefly somewhere. Thanks much. From busseniu at in.tum.de Tue Feb 21 12:23:03 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 11:23:03 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> Message-ID: <4F437087.3000407@in.tum.de> Hi, On 15.02.2012 04:46, Timo Sirainen wrote: > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. sorry to bother you again, but I think there is a problem with this patch: If a maildir contains several copies of the same message all in the same folder, dsync will not deduplicate them. While IMAP cannot directly create copies of a message in the same folder, it does still happen if you copy (or move) a message back and forth between two folders. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 13:04:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 13:04:25 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F437087.3000407@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> Message-ID: <1329822265.11500.254.camel@innu> On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: > Hi, > > On 15.02.2012 04:46, Timo Sirainen wrote: > > > > It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. > > sorry to bother you again, but I think there is a problem with this patch: > > If a maildir contains several copies of the same message all in the same > folder, dsync will not deduplicate them. Correct. I nearly finished implementing this also, but then I thought it just makes the code unnecessarily complex for no good reason. > While IMAP cannot directly create copies of a message in the same > folder, It can: SELECT INBOX, COPY 1 INBOX > it does still happen if you copy (or move) a message back and > forth between two folders. Is it common enough to be an actual problem? From joshua at hybrid.pl Tue Feb 21 13:25:51 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 21 Feb 2012 12:25:51 +0100 Subject: [Dovecot] =?utf-8?q?Recalculate_quota_when_quota=3Ddict_=3F?= In-Reply-To: References: Message-ID: <7304de1adf6749ec476ef30957bb931c@hybrid.pl> W dniu 2012-02-21 02:13, Timo Sirainen napisa?(a): > On 17.2.2012, at 11.51, joshua at hybrid.pl wrote: >> By the way: what might have caused such a warning? >> root at mail2.hybrid.pl /tmp/transfer> doveadm quota recalc -u >> joshua at hybrid.pl >> doveadm(joshua at hybrid.pl): Warning: Created dotlock file's timestamp >> is different than current time (1329464622 vs 1329464672): >> /var/mail/mail/hybrid.pl/joshua/.mailing.ekg/dovecot-uidlist > > Does it keep happening? Is this a local filesystem or NFS? Shouldn't > happen unless remote storage server's clock and local server's clock > aren't synced. OK, that's the case - /var/mail is stored on remote NFS. I'll check if I have time properly set, but if so - I'll probably have to live with it (I have no influence on NFS server). Greetings, -- Jacek Osiecki From busseniu at in.tum.de Tue Feb 21 13:55:05 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Tue, 21 Feb 2012 12:55:05 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <1329822265.11500.254.camel@innu> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> Message-ID: <4F438619.70405@in.tum.de> On 21.02.2012 12:04, Timo Sirainen wrote: > On Tue, 2012-02-21 at 11:23 +0100, Christoph Bu?enius wrote: >> Hi, >> >> On 15.02.2012 04:46, Timo Sirainen wrote: >>> >>> It worked when the GUID already existed somewhere in destination, but not if it was added only during the same session. The attached patch fixes it. I'll commit it to v2.1 hg after I'll release v2.1.0.. >> >> sorry to bother you again, but I think there is a problem with this patch: >> >> If a maildir contains several copies of the same message all in the same >> folder, dsync will not deduplicate them. > > Correct. I nearly finished implementing this also, but then I thought it > just makes the code unnecessarily complex for no good reason. > >> While IMAP cannot directly create copies of a message in the same >> folder, > > It can: SELECT INBOX, COPY 1 INBOX Oh, mea culpa :) > >> it does still happen if you copy (or move) a message back and >> forth between two folders. > > Is it common enough to be an actual problem? Actually we have some mailboxes with massively duplicated messages in the same folder. Of course I cannot tell how common it is in general. I could imagine that some people routinely copy all INBOX messages into archive folders and do not check whether the archive already contains these messages. Apart from the waste of space, I was wondering: Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From tss at iki.fi Tue Feb 21 14:15:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:15:33 +0200 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: <4F438619.70405@in.tum.de> References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: On 21.2.2012, at 13.55, Christoph Bu?enius wrote: > Apart from the waste of space, I was wondering: > Is it okay for an mdbox to have several duplicate instances of a message with the same GUID? Might some kind of corruption arise from this? No corruption. And they might even become deduplicated if you do doveadm force-resync + purge. From campbell at cnpapers.com Tue Feb 21 14:19:33 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:19:33 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F42C212.5090509@uvm.edu> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> Message-ID: <4F438BD5.4020107@cnpapers.com> On 2/20/2012 4:58 PM, Jim Lawson wrote: > On 2/20/12 3:36 PM, Steve Campbell wrote: >> >> Thanks for that input. I still think I'm missing something since I >> too used the compatibility link that you pointed to. Only thing is >> that proceeding those namespaces, I used the first example of: >> >> namespace { >> type = private >> separator = / >> prefix = "#mbox/" >> location = mbox:~/mail:INBOX=/var/mail/%u >> inbox = yes >> hidden = yes >> list = no >> } >> namespace { >> type = private >> separator = / >> prefix = >> location = maildir:~/mail >> } >> >> This seemed to get the ball rolling so that users could access >> anything at all. I still have the problem of client imap folders >> being different from webmail imap folders. That's pretty much why I'm >> thinking of using mbox as the INBOX and all other imap folders in the >> ~/mail directory in maildir format. > You ought to be able to get webmail to see the same folders as the > imap clients. > >> >> Should I remove the first two namespaces, in your opinion? Right now, >> the whole thing is kind of fragile. >> > > From your added namespaces, it looks like you are trying to support > mbox and maildir and ~/mail at the same time. Do you already have > maildir folders to support? If not, I would try to get things working > well with mbox first before I started a conversion to maildir. I > would also ask why you're thinking of moving to maildir. Yes, there > are caching benefits, but when you add the Dovecot indexing on top of > mbox, it's pretty much a wash. If you are using file-level backup, > rather than some sort of snapshot technology, maildir will be much, > much slower to back up. Your system will spend all its time walking > directories, opening and closing files. If you don't have many users > to worry about it might be OK, but make sure it's worth it. A lot of > sites went to maildir in the 1998-2004 era and have regretted the > decision as their systems get overloaded with files and they can't > back them up. No, I've got all mbox formats. Previous posts probably suggested that I wanted to move to maildir, but all of the replies I've received have convinced me that I do not want to do that. The folders in ~ and ~/mail are mboxes, so I need to see what damage I've done with a maildir namespace. The maildir reference could be part of the problems I'm seeing. I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. > >> Horde/Imp updates are probably out of the question until I can get a >> server to install the upgrade on. > For Dovecot and IMP both, you should set up an alternate server to > test out your config changes on before you put them into production. > If you are running on a bare metal single server, set up another > Dovecot instance on a different set of ports (I commonly use 20143 > (imap), 20993 (imaps), 20110 (pop3)) which you can fiddle with > freely. Once you are satisfied with the result in your various > clients, put it into production. You can do the same with Horde/IMP > by putting an installation in a different location on your webserver. > > Jim I've got a second server that is totally independent of the one I'm messing with. I've done the horde/imp alternate port/config. It all went well. The downside is that I didn't realize those secondary folders in ~/mail were being hidden. Kinda late to switch back, but the secondary server supports another domain, so I have it to test with. Again, the damage I've caused to those secondary imap folders needs to be determined to get this fixed properly. Most users are seing the secondary folders and can use them. It's just those users who have folders in ~ that are seeing problems as far as I can tell. Thanks steve > > From campbell at cnpapers.com Tue Feb 21 14:27:35 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 07:27:35 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <4F438DB7.5020001@cnpapers.com> On 2/20/2012 7:25 PM, Joseph Tam wrote: > > Steve Campbell wrote: > >> The more I read about all of this, the more I'm thinking about moving >> to maildir format. My switchover this weekend is full of holes due to >> the way user's imap folders were laid out. Some had folders in their >> home directory and others might have folders in their /home/mail >> directory. > > I had the same problem, and handled it more or less the same way as > the later poster (Jim Lawson ) handled his site. > I retained use of mbox format as well. > > I planned the migration like this > > - scan user home directories for mailboxes (especially "Trash", > "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. > > This was piped into a script that Email'd users about the changes > that was going to happen, what they would expect to see, and a > FAQ on how to set up a mail client correctly. Here, I'm not sure what should be done. The users with the secondary folders that are not in ~/mail can't seem to get the client configured. > > - during the cutover, > > - mailboxes left on the home directory were moved to > ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid > name collision for users that had a mixture of correct > and blank prefixes. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot is ignoring this and I'm not sure what it's looking at to determine the imap folders. > (I think I deleted "Trash", "Junk", etc. anyways). > > - Namespace aliasing was used so that prefixes "", "mail", > "~/mail", and whatever darn fool settings my users used, > would map to the same directory. > > - .subscription files were moved into the mail folder > (don't have to edit prefixes since the aliasing > will take care of that). And when would these .subscription files be created? The first time the folders would be accessed, or when? This seems to be part of my fix that I'm getting lost on. > > - depending on what POP3 client you used (I used > qpopper), you may need to configure > > pop3_reuse_xuidl = yes > > to avoid a massive re-downloading from POP3 clients > after cutover. > > - the mail clients I control centrally (e.g. webmail, > public server mail clients, etc). shouldn't need updating > since they ought to have been set up properly in the first > place. > > - after cutover, a second notification was sent for users that > didn't move their mailboxes the first time around, and was it > done for them during migration. > > And of course, test like crazy and watch the logs like a hawk. > > Other gotcha's: > > - your setup is fairly close to mine, so you may also run into > the problem of user having mailboxes with group ownership that > users are not part of (for example, group "mail" for INBOX set > by your LDA or personal mailboxes with groups the user is no > longer a member of) *and* with group permission not mode=0. Sendmail sort of requires the "mail" group, does it not? I'll take a look and see if all users are part of this or not. A crazy solution would be in order here? > > You'll have to treat these (set mode=0, or change the group to > something the user is part of), or the dovecot index creation > will fail and they won't be able to access their mailboxes. > > I think my migration went pretty smoothly. Less than a handful of > wazzup' > Email problem reports. Except for those users with the different folder locations, it seems that all is going pretty well. Maybe they're just not notifying me yet, though. > > Joseph Tam > > Thanks steve From tss at iki.fi Tue Feb 21 14:28:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 14:28:30 +0200 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: <4F438BD5.4020107@cnpapers.com> References: <4F428C82.4010700@cnpapers.com> <4F42A1AD.2000304@uvm.edu> <4F42AEBC.6000300@cnpapers.com> <4F42C212.5090509@uvm.edu> <4F438BD5.4020107@cnpapers.com> Message-ID: <56BC47FD-F1EB-4294-A25C-5205E30AAA36@iki.fi> On 21.2.2012, at 14.19, Steve Campbell wrote: > I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. With new enough Dovecot version (2.0, maybe 1.2) you can add :SUBSCRIPTIONS=.mailboxlist to mail_location. From dag at newtech.fi Tue Feb 21 14:56:01 2012 From: dag at newtech.fi (Dag Nygren) Date: Tue, 21 Feb 2012 14:56:01 +0200 Subject: [Dovecot] Where did the message end up? Message-ID: <12808509.iPEzrCISOc@dag> Hi all! Just migrated out UW-IMAP installation to Dovecot and so far so good. Had small teething problems though which left me with one question. The scenario was that I have a set of public folders that are only needed by the main company group (gid = company). Then there are some external users as well, not directly in the company group. I set up the public mailbox with permissions 660, owner me and group "company". That is the externals didn't have access to the mail folder. At this point all looked fine, but later there was an email to an external person and this was passed on to deliver by procmail. Procmail logs says all was OK, but there is an entry about the public folder not accessible in the Dovecot log. The email is nowhere to be found. My question: Where is the delivered email? Best Dag From alessio at skye.it Tue Feb 21 15:29:53 2012 From: alessio at skye.it (Alessio Cecchi) Date: Tue, 21 Feb 2012 14:29:53 +0100 Subject: [Dovecot] Some trouble with dovecot 2.1 and SASL In-Reply-To: <1329815785.11500.224.camel@innu> References: <32f6a0bcefd32de3c2cb56857b452b75@skye.it> <1329815785.11500.224.camel@innu> Message-ID: <4F439C51.3090907@skye.it> Il 21/02/2012 10:16, Timo Sirainen ha scritto: > On Sun, 2012-02-19 at 11:39 +0100, Alessio Cecchi wrote: > >> I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" >> and SASL for postfix's SMTP-Auth, like explained ih this howto: >> http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL >> >> Authentications over IMAP (local dovecot -> remote dovecot) works fine, >> and it's great. Also smtp-auth on postfix (over SASL) works fine. But in >> the dovecot log I found this error: >> >> >> Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): >> child 30506 killed with signal 11 (core dumps disabled) > > http://hg.dovecot.org/dovecot-2.1/rev/a6a58512f0df should fix this. > > Patch applied, after rebuilding all works fine. Thanks Timo -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it @ LOLUG -> Socio http://www.lolug.net From lists at wildgooses.com Tue Feb 21 16:33:24 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 21 Feb 2012 14:33:24 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120213194356.GB3850@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> Message-ID: <4F43AB34.9040907@wildgooses.com> On 13/02/2012 19:43, Jan-Frode Myklebust wrote: > On Mon, Feb 13, 2012 at 11:08:48AM -0800, Mark Moseley wrote: >> Out of curiosity, are you running dovecot locally on those webmail >> servers as well, or is it talking to remote dovecot servers? > The webmail servers are talking with dovecot director servers which in > turn are talking with the backend dovecot servers. Each service running > on different servers. > > Webmail-servers -> director-servers -> backend-servers > I think the original question was still sensible. In your case it seems like the ping times are identical between: webmail -> imap-proxy webmail -> imap server I think your results show that a proxy has little (or negative) benefit in this situation, but it seems feasible that a proxy could eliminate several RTT trips in the event that the proxy is "closer" than the imap server? This might happen if say the imap server is in a different datacenter (webmail on an office server machine?) I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). However, using imap-proxy has the benefit of "clustering" logins a little and this makes log files a little easier to understand in the face of users with desktop mail clients plus webmail users. Possibly this idea useful to someone else... Thanks for measuring this! Ed W From crohmann at netcologne.de Tue Feb 21 17:05:39 2012 From: crohmann at netcologne.de (Christian Rohmann) Date: Tue, 21 Feb 2012 16:05:39 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? Message-ID: <4F43B2C3.3080006@netcologne.de> Hey dovecot-users, clients that speak telnet based protocols like http, smtp usually encode IDN (International Domain Names) containing UTF-8 characters into ACE (ASCII Compatible Encoding) punycode. I am wondering what the case was with managesieve (RFC 5804). For example if I was using a username like "user@???.example.com" with the AUTHENTICATE command. Would that need to be encoded into ACE or be left in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal Syntax" suggests that quoted or literal UTF-8 is correct. Thanks Christian From florob at babelmonkeys.de Tue Feb 21 18:11:28 2012 From: florob at babelmonkeys.de (Florian Zeitz) Date: Tue, 21 Feb 2012 17:11:28 +0100 Subject: [Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE? In-Reply-To: <4F43B2C3.3080006@netcologne.de> References: <4F43B2C3.3080006@netcologne.de> Message-ID: <4F43C230.6000408@babelmonkeys.de> Am 21.02.2012 16:05, schrieb Christian Rohmann: > Hey dovecot-users, > > clients that speak telnet based protocols like http, smtp usually encode > IDN (International Domain Names) containing UTF-8 characters into ACE > (ASCII Compatible Encoding) punycode. > > I am wondering what the case was with managesieve (RFC 5804). For > example if I was using a username like "user@???.example.com" with the > AUTHENTICATE command. Would that need to be encoded into ACE or be left > in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal > Syntax" suggests that quoted or literal UTF-8 is correct. > Hy, what you're asking for is the correct format of the simple username. This is in fact deployment specific. There is in general no requirement to include the domain at all. The only requirement for to fulfil is that it must not contain any codepoints prohibited by the SASLPrep stringprep profile. As such both the ACE and the IDN variant are valid. Which one to choose, or whether to use only "user" depends on local policies. There might be some implementation specifics in dovecot/pigeonhole that I'm not aware of though. Regards, Florian From busseniu at in.tum.de Tue Feb 21 18:36:18 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Tue, 21 Feb 2012 17:36:18 +0100 Subject: [Dovecot] dsync, hard-links and refcounts In-Reply-To: References: <4F391B3D.8020208@in.tum.de> <360FD93D-F43F-480C-9A6A-842E041EDE4F@iki.fi> <4F437087.3000407@in.tum.de> <1329822265.11500.254.camel@innu> <4F438619.70405@in.tum.de> Message-ID: <4F43C802.7010702@in.tum.de> On 21.02.2012 13:15, Timo Sirainen wrote: > > And they might even become deduplicated if you do doveadm force-resync + purge. > I hadn't tried that yet. Thanks for the hint, this is probably all we need. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From trashcan at odo.in-berlin.de Tue Feb 21 19:29:12 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 18:29:12 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> Message-ID: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Hi -- On 21.02.2012, at 02:02, Timo Sirainen wrote: > On 18.2.2012, at 20.26, Michael Grimm wrote: >> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb Hmm, now I cannot clients any longer: dovecot: master: Dovecot v20120221 (7d0d76df719f+) starting up dovecot: imap-login: Login: user=, method=PLAIN, rip=xxx.xxx.xxx.xxx, lip=yyy.yyy.yyy.yyy, mpid=65444, TLS dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test dovecot: imap(test): Error: Invalid user settings. Refer to server log for more information. [... repeated for every user clients are trying to connect ...] I switched back immediately (production server), thus I didn't test if dsync might run now. Here's the relevant part of my config: mail_home = /var/mail/.homedirs/%n mail_location = mdbox:/var/mail/.maildirs/%n User's individual mail storage formats are fetched from userdb (sqlite3), which normally are mdbox, except for test which is sdbox. Any idea what to test next? Thanks and regards, Michael From tss at iki.fi Tue Feb 21 22:36:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:36:32 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> On 21.2.2012, at 16.33, Ed W wrote: > I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: void postlogout_init(struct module *module) { } void postlogout_deinit(void) { system("/usr/local/bin/dovecot-postlogout.sh"); } Add a few missing #includes and compile and enable for imap/pop3 and that should be it. From tss at iki.fi Tue Feb 21 22:52:14 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 22:52:14 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> Message-ID: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Hi, On 21.2.2012, at 19.29, Michael Grimm wrote: >>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb > > Hmm, now I cannot clients any longer: > > dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. From trashcan at odo.in-berlin.de Tue Feb 21 23:48:37 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 22:48:37 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: Hi -- On 21.02.2012, at 21:52, Timo Sirainen wrote: > On 21.2.2012, at 19.29, Michael Grimm wrote: >>>> doveadm(test): Fatal: All your namespaces have a location setting. It should be empty (default mail_location) in the namespace to be converted. >>> >>> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/488fe9d417eb >> >> Hmm, now I cannot connect clients any longer: >> >> dovecot: imap(test): Error: user test: Initialization failed: mail_location not set and autodetection failed: Mail storage autodetection failed with home=/var/mail/.homedirs/test > > Sorry, that part of the code is a bit weird. http://hg.dovecot.org/dovecot-2.1/rev/3007994141b6 is another attempt at this. Thanks, that omits the first and second error, and I believe that you solved this issue. Now, I can run my old command as used for 2.0.x and ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test ... a "newer" call like ... vmail> doveadm sync -f -u test ssh vmail at remote-host.tld dsync -v -f -u test ... without complaining about namespaces as listed above. But, now I do end with a message like ... vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test usage: dsync [-C ] [-m ] [-u ] [-frRv] mirror | [@] | dsync-local(test): Error: read() from worker server failed: EOF ... irrespectively if I use the old "dsync" or the new "doveadm sync". Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Weird. I must have missed something. I'll back to 2.0.18 for the time being. Any hints are highly welcome ... Thanks for your efforts, and thanks for dovecot! Regards, Michael From tss at iki.fi Tue Feb 21 23:57:21 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 21 Feb 2012 23:57:21 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> Message-ID: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> On 21.2.2012, at 23.48, Michael Grimm wrote: > But, now I do end with a message like ... > > vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test > usage: dsync [-C ] [-m ] [-u ] [-frRv] > mirror | [@] | > dsync-local(test): Error: read() from worker server failed: EOF > > ... irrespectively if I use the old "dsync" or the new "doveadm sync". > > Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. Added it there. From trashcan at odo.in-berlin.de Wed Feb 22 00:03:52 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Tue, 21 Feb 2012 23:03:52 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 22:57, Timo Sirainen wrote: > On 21.2.2012, at 23.48, Michael Grimm wrote: >> But, now I do end with a message like ... >> >> vmail> dsync -v -f -u test mirror ssh vmail at remote-host.tld dsync -v -f -u test >> usage: dsync [-C ] [-m ] [-u ] [-frRv] >> mirror | [@] | >> dsync-local(test): Error: read() from worker server failed: EOF >> >> ... irrespectively if I use the old "dsync" or the new "doveadm sync". >> >> Ok, this is still a mixed 2.0.18 <--remote host--> 2.1.0 setup, > > Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). Ok. Then I will try both servers upgraded to 2.1 within the next days. Thanks for that valuable info that I really missed, sorry. >> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. > > Added it there. Great. And thanks again, Michael From as at 3a.pl Wed Feb 22 00:46:48 2012 From: as at 3a.pl (Adam Szpakowski) Date: Tue, 21 Feb 2012 23:46:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy Message-ID: <4F441ED8.20908@3a.pl> Hi, In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Each of this emails has the same Message-ID. Is there a way to detect those duplicates and discard them when saving copies into boss account? There was a discussion on the list a while ago (Feb 10/11, 2011) mentioning this problem, but with no clear solution. We are using Dovecot 2.0.15. Mail is delivered via LMTP (MDA/postfix is on separate machine then MAA/dovecot). -- Adam Szpakowski From tss at iki.fi Wed Feb 22 00:57:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 00:57:17 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F441ED8.20908@3a.pl> References: <4F441ED8.20908@3a.pl> Message-ID: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> On 22.2.2012, at 0.46, Adam Szpakowski wrote: > In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). > There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? From as at 3a.pl Wed Feb 22 01:02:23 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:02:23 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> Message-ID: <4F44227F.9030502@3a.pl> On 21.02.2012 23:57, Timo Sirainen wrote: > On 22.2.2012, at 0.46, Adam Szpakowski wrote: >> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. > Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. We are using dedicated sieve_before script for each user based on %u variable. -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:09:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:09:35 +0200 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F44227F.9030502@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> Message-ID: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> On 22.2.2012, at 1.02, Adam Szpakowski wrote: > On 21.02.2012 23:57, Timo Sirainen wrote: >> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? > Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. > We are using dedicated sieve_before script for each user based on %u variable. Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: fileinto :copy :x-deduplicate "boss"; Anyway, probably not going to be implemented anytime soon. From as at 3a.pl Wed Feb 22 01:15:30 2012 From: as at 3a.pl (Adam Szpakowski) Date: Wed, 22 Feb 2012 00:15:30 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> Message-ID: <4F442592.608@3a.pl> On 22.02.2012 00:09, Timo Sirainen wrote: > On 22.2.2012, at 1.02, Adam Szpakowski wrote: >> On 21.02.2012 23:57, Timo Sirainen wrote: >>> On 22.2.2012, at 0.46, Adam Szpakowski wrote: >>>> In one installation we are using sieve "redirect :copy" to create copy of the incoming email (local boss gets copy of its employees mails). >>>> There is a problem when the incoming email has multiple To/CC entries with local users. Multiple copy/redirects are created (one for each employee) and there are multiple copies in "boss" account. >>> Could this instead be implemented by MTA as automatic bcc: boss of all incoming mails? >> Unfortunately no. The employee/boss structure is quite complicated. Some employees have multiple "bosses", there are also some "project" emails when the structure is even more complicated. >> We are using dedicated sieve_before script for each user based on %u variable. > Well, it would be possible to build a doveadm script that deletes the duplicates after delivery, but currently there's no implementation to avoid delivering duplicate Message-IDs in the first place. > > I don't really like such a Message-ID-based deduplication feature enabled by default, but something like this could be nice: > > fileinto :copy :x-deduplicate "boss"; > > Anyway, probably not going to be implemented anytime soon. Maybe there is a way to use a procmail with something like this: :0 Wh: msgid.lock | formail -D 8192 .msgid.cache But is there a safe way to use it together with sieve? Using Pigeonhole Sieve Pipe Plugin? -- Adam Szpakowski From tss at iki.fi Wed Feb 22 01:32:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 01:32:56 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <12808509.iPEzrCISOc@dag> References: <12808509.iPEzrCISOc@dag> Message-ID: On 21.2.2012, at 14.56, Dag Nygren wrote: > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > Had small teething problems though which left me with one question. > > The scenario was that I have a set of public folders that are only needed by > the main company group (gid = company). Then there are some external users as > well, not directly in the company group. So this is in a public namespace? > I set up the public mailbox with permissions 660, owner me and group > "company". That is the externals didn't have access to the mail folder. > > At this point all looked fine, but later there was an email to an external > person and this was passed on to deliver by procmail. Procmail logs says all > was OK, but there is an entry about the public folder not accessible in the > Dovecot log. The email is nowhere to be found. Exact error message would be helpful. Was the error from deliver, or imap/pop3? If it was from deliver, it probably returned temporary failure. > Where is the delivered email? Have you checked what happens if deliver tempfails (e.g. make procmail call tempfail.sh script that just exits with 75), does procmail also correctly tempfail and tell MTA to retry the delivery later? From jtam.home at gmail.com Wed Feb 22 02:04:30 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 21 Feb 2012 16:04:30 -0800 (PST) Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: Steve Campbell writes: > > This was piped into a script that Email'd users about the changes > > that was going to happen, what they would expect to see, and a > > FAQ on how to set up a mail client correctly. > > Here, I'm not sure what should be done. The users with the secondary > folders that are not in ~/mail can't seem to get the client configured. "Great artists steal". Google around for a good set of instructions on how to set up IMAP prefix's for various mail clients, and refer your users to that. I also referred them to our webmail (with the correct mail prefix) as a reference to a list of mailboxes they ought to see. If they don't match, then that's a symptom of a bad prefix. > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > is ignoring this and I'm not sure what it's looking at to determine the > imap folders. > > > - .subscription files were moved into the mail folder > > (don't have to edit prefixes since the aliasing > > will take care of that). > > And when would these .subscription files be created? The first time the > folders would be accessed, or when? This seems to be part of my fix that > I'm getting lost on. Sorry, as Timo intimated, dovecot uses .subscription, which can be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't have to edit it if you set up namespace aliasing. Just move it from the top-level into your user's mail directory during cutover: (if you use dovecot's default subscription filename) mv $USER/.mailboxlist $USER/mail/.subscription If your IMAP client is still ignoring subscriptions, it usually means the client was configured that way. > > - your setup is fairly close to mine, so you may also run into > > the problem of user having mailboxes with group ownership that > > users are not part of (for example, group "mail" for INBOX set > > by your LDA or personal mailboxes with groups the user is no > > longer a member of) *and* with group permission not mode=0. > > Sendmail sort of requires the "mail" group, does it not? I'll take a > look and see if all users are part of this or not. A crazy solution > would be in order here? I think sendmail works fine if you set your mail spool mode=1777 (like /tmp), but have each inbox mode=0600. This setting will persist. It's sendmail LDA mail.local that's the problem: it autocreates new inboxes with permissions user:mail/660 and unless the user is in group "mail", chaos will ensue. You can maybe do a cron script or some other hack, or better yet, use dovecot LDA, which sets up the perms correctly (and keeps indexes updated). A one time "chmod go-rwx" during cutover is all that is required then. The log entry that points this problem out looks like dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) failed: Operation not permitted (egid=5678(goodgroup), group based on /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not owner Joseph Tam From campbell at cnpapers.com Wed Feb 22 04:29:31 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Tue, 21 Feb 2012 21:29:31 -0500 Subject: [Dovecot] Questions still about a uw-imap -> dovecot conversion In-Reply-To: References: Message-ID: <1329877771.4f44530b83411@perdition.cnpapers.net> Quoting Joseph Tam : > > Steve Campbell writes: > > > > This was piped into a script that Email'd users about the changes > > > that was going to happen, what they would expect to see, and a > > > FAQ on how to set up a mail client correctly. > > > > Here, I'm not sure what should be done. The users with the secondary > > folders that are not in ~/mail can't seem to get the client configured. > > "Great artists steal". Google around for a good set of instructions > on how to set up IMAP prefix's for various mail clients, and refer > your users to that. This conversion to dovecot has been a real learning experience for me. The old stuff that came with Centos 3 just worked out of the box. I never realized how much could be modified. It was always there, I just never needed it. > > I also referred them to our webmail (with the correct mail prefix) as a > reference to a list of mailboxes they ought to see. If they don't match, > then that's a symptom of a bad prefix. Our web mail, Imp, seems to be doing fine now that all of the mboxes have been moved to ~/mail. > > > I've tried this and modified the .mailboxlist, but I'm thinking Dovecot > > is ignoring this and I'm not sure what it's looking at to determine the > > imap folders. > > > > > - .subscription files were moved into the mail folder > > > (don't have to edit prefixes since the aliasing > > > will take care of that). > > > > And when would these .subscription files be created? The first time the > > folders would be accessed, or when? This seems to be part of my fix that > > I'm getting lost on. > > Sorry, as Timo intimated, dovecot uses .subscription, which can > be adjusted back to .mailboxlist, which uw-imapd uses. You shouldn't > have to edit it if you set up namespace aliasing. Just move it from > the top-level into your user's mail directory during cutover: > > (if you use dovecot's default subscription filename) > mv $USER/.mailboxlist $USER/mail/.subscription > > If your IMAP client is still ignoring subscriptions, it usually > means the client was configured that way. > > > > - your setup is fairly close to mine, so you may also run into > > > the problem of user having mailboxes with group ownership that > > > users are not part of (for example, group "mail" for INBOX set > > > by your LDA or personal mailboxes with groups the user is no > > > longer a member of) *and* with group permission not mode=0. > > > > Sendmail sort of requires the "mail" group, does it not? I'll take a > > look and see if all users are part of this or not. A crazy solution > > would be in order here? > > I think sendmail works fine if you set your mail spool mode=1777 (like > /tmp), but have each inbox mode=0600. This setting will persist. > > It's sendmail LDA mail.local that's the problem: it autocreates new > inboxes with permissions user:mail/660 and unless the user is in group > "mail", chaos will ensue. > So far, all things seem OK in this area. No complaints are being forwarded. > You can maybe do a cron script or some other hack, or better yet, > use dovecot LDA, which sets up the perms correctly (and keeps indexes > updated). A one time "chmod go-rwx" during cutover is all that is > required then. > > The log entry that points this problem out looks like > > dovecot: imap(smith): Error: chown(/dc-cache/smith/.imap/xx, group=6(mail)) > failed: Operation not permitted (egid=5678(goodgroup), group based on > /home/smith/mail/xx - see http://wiki2.dovecot.org/Errors/ChgrpNoPerm) > dovecot: imap(smith): Error: mkdir(/dc-cache/smith/.imap/xx) failed: Not > owner > > Joseph Tam For now, it looks like the only problem I'm seeing is that one user who uses Outlook Express that has a pop account for one email account and an imap account for another email account is having problems with the "Sent" folder. When she sends an email using the imap account, the "Sent" copy goes into the pop account's sent item folder. When using webmail, the sent copy goes into the sent-mail file as it's supposed to. I'm guessing for now that that google you suggested might point me to a fix for that. Thanks all for all the great help. Shared accounts are next, so wish me luck. steve > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/ From dag at newtech.fi Wed Feb 22 09:34:27 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 09:34:27 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> Message-ID: <1651793.SOPdjt8lSW@eseries> onsdag 22 februari 2012 01:32:56 skrev Timo Sirainen: > On 21.2.2012, at 14.56, Dag Nygren wrote: > > Just migrated out UW-IMAP installation to Dovecot and so far so good. > > > > Had small teething problems though which left me with one question. > > > > The scenario was that I have a set of public folders that are only > > needed by the main company group (gid = company). Then there are some > > external users as well, not directly in the company group. > > So this is in a public namespace? Yep, it was. The definition: namespace { type = public separator = / prefix = MailingLists/ location = mdbox:/usr/local/var/mail/MailingLists inbox = no subscriptions = no # v1.1+ } > > I set up the public mailbox with permissions 660, owner me and group > > "company". That is the externals didn't have access to the mail folder. > > > > At this point all looked fine, but later there was an email to an > > external person and this was passed on to deliver by procmail. Procmail > > logs says all was OK, but there is an entry about the public folder not > > accessible in the Dovecot log. The email is nowhere to be found. > > Exact error message would be helpful. Was the error from deliver, or > imap/pop3? If it was from deliver, it probably returned temporary failure. The log parts with final names and groups changed.. Deliver error: Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization failed: Namespace 'MailingLists/': stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: /usr/local/var/mail/MailingLists, euid is not dir owner) Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer to server log for more information. Procmail log: >From mailing-1341698764359 at mailpv.net Tue Feb 21 10:02:24 2012 Subject: =?CP1250?Q?P=E4=E4si=E4issuunnitelmat?= Folder: deliver -d extuser -m inbox 56979 > > Where is the delivered email? > > Have you checked what happens if deliver tempfails (e.g. make procmail call > tempfail.sh script that just exits with 75), does procmail also correctly > tempfail and tell MTA to retry the delivery later? No, haven't done that. yet. And I don't think it will pass on the failure code. Procmail settings are all from the HOWTO:s though. The .forward file looks like this: "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" And the applicable rule in .procmailrc: :0 w { :0 | $DELIVERYAGENT -d $LOGNAME -m inbox } Have to point out that this case wasn't important, but I wouldn't like the system to loose an important mail later. From tss at iki.fi Wed Feb 22 09:43:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 09:43:58 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1651793.SOPdjt8lSW@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: On 22.2.2012, at 9.34, Dag Nygren wrote: > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: Initialization > failed: Namespace 'MailingLists/': > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission denied > (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > /usr/local/var/mail/MailingLists, euid is not dir owner) > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. Refer > to server log for more information. Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually v2.1.0 is the first release where you can easily configure Dovecot to simply skip the namespace when it doesn't have permission to access it: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. >>> Where is the delivered email? >> >> Have you checked what happens if deliver tempfails (e.g. make procmail call >> tempfail.sh script that just exits with 75), does procmail also correctly >> tempfail and tell MTA to retry the delivery later? > > No, haven't done that. yet. And I don't think it will pass on the failure > code. Procmail settings are all from the HOWTO:s though. > > The .forward file looks like this: > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > And the applicable rule in .procmailrc: > :0 w > { > :0 > | $DELIVERYAGENT -d $LOGNAME -m inbox > } > > Have to point out that this case wasn't important, but I wouldn't like the > system to loose an important mail later. You should definitely make sure that procmail won't lose any mails if Dovecot LDA returns tempfailure, since they do sometimes happen. You could also get rid of procmail completely by moving to sieve. From dag at newtech.fi Wed Feb 22 10:03:36 2012 From: dag at newtech.fi (Dag Nygren) Date: Wed, 22 Feb 2012 10:03:36 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> Message-ID: <1958645.10kxPAq8Bz@eseries> > On 22.2.2012, at 9.34, Dag Nygren wrote: > > Feb 21 10:02:24 dag dovecot: lda(extuser): Error: user extuser: > > Initialization failed: Namespace 'MailingLists/': > > stat(/usr/local/var/mail/MailingLists/mailboxes) failed: Permission > > denied (euid=2002(extuser) egid=2002(extgroup) missing +x perm: > > /usr/local/var/mail/MailingLists, euid is not dir owner) > > Feb 21 10:02:24 dag dovecot: lda(extuser): Fatal: Invalid user settings. > > Refer to server log for more information. > > Right, the mail wasn't delivered, LDA simply exited with tempfail. Actually > v2.1.0 is the first release where you can easily configure Dovecot to > simply skip the namespace when it doesn't have permission to access it: > > + Added ignore_on_failure setting for namespaces. If namespace > initialization fails with this enabled (e.g. permission denied), > the namespace is silently skipped for the user. Ok. Many thanks for answering this. I am using FC14 and dovecot 2.0.4 at the moment so I don't have that option yet. It would be nice BTW if dovecot added the exit status it used in the log if it fails? > >>> Where is the delivered email? > >> > >> Have you checked what happens if deliver tempfails (e.g. make procmail > >> call tempfail.sh script that just exits with 75), does procmail also > >> correctly tempfail and tell MTA to retry the delivery later? > > > > No, haven't done that. yet. And I don't think it will pass on the > > failure > > code. Procmail settings are all from the HOWTO:s though. > > > > The .forward file looks like this: > > > > "|IFS=' ' && exec /usr/bin/procmail -f- || exit 75 #extuser" > > > > And the applicable rule in .procmailrc: > > :0 w > > > > { > > > > :0 > > : > > | $DELIVERYAGENT -d $LOGNAME -m inbox > > > > } > > > > Have to point out that this case wasn't important, but I wouldn't like > > the system to loose an important mail later. > > You should definitely make sure that procmail won't lose any mails if > Dovecot LDA returns tempfailure, since they do sometimes happen. You could > also get rid of procmail completely by moving to sieve. Will check the path out. Any way of forcing dovecot to return tempfail for the tests? Preferrably only for one user :-) Not familiar with sieve yet, but will for sure look into it. Thanks again Dag From tss at iki.fi Wed Feb 22 10:18:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 10:18:00 +0200 Subject: [Dovecot] Where did the message end up? In-Reply-To: <1958645.10kxPAq8Bz@eseries> References: <12808509.iPEzrCISOc@dag> <1651793.SOPdjt8lSW@eseries> <1958645.10kxPAq8Bz@eseries> Message-ID: On 22.2.2012, at 10.03, Dag Nygren wrote: > It would be nice BTW if dovecot added > the exit status it used in the log if it fails? This wouldn't be very useful normally. If you see "Fatal" prefixed error from dovecot-lda, it's nearly always tempfail (75). If it crashes, the error is different, but then it wouldn't be able to log the error anyway. > Will check the path out. Any way of forcing dovecot to return tempfail for the > tests? Preferrably only for one user :-) Run a script instead, something like: #!/bin/sh if [ $USER = "foo" ]; then exit 75 else exec .../dovecot-lda $@ fi From janfrode at tanso.net Wed Feb 22 10:25:36 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 09:25:36 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F43AB34.9040907@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> Message-ID: <20120222082536.GA8678@dibs.tanso.net> On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: > > I think the original question was still sensible. In your case it > seems like the ping times are identical between: > webmail -> imap-proxy > webmail -> imap server > > I think your results show that a proxy has little (or negative) > benefit in this situation, but it seems feasible that a proxy could > eliminate several RTT trips in the event that the proxy is "closer" > than the imap server? This might happen if say the imap server is > in a different datacenter (webmail on an office server machine?) The webmail/imapproxy were actually running in a different datacenter to the dovecot director/backend servers, but only about 20KM away. Ping tests: webmail->director: rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms director->backend: rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms webmail->localhost: rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms -jf From lists at wildgooses.com Wed Feb 22 11:31:55 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:31:55 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <20120222082536.GA8678@dibs.tanso.net> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> Message-ID: <4F44B60B.4000105@wildgooses.com> On 22/02/2012 08:25, Jan-Frode Myklebust wrote: > On Tue, Feb 21, 2012 at 02:33:24PM +0000, Ed W wrote: >> I think the original question was still sensible. In your case it >> seems like the ping times are identical between: >> webmail -> imap-proxy >> webmail -> imap server >> >> I think your results show that a proxy has little (or negative) >> benefit in this situation, but it seems feasible that a proxy could >> eliminate several RTT trips in the event that the proxy is "closer" >> than the imap server? This might happen if say the imap server is >> in a different datacenter (webmail on an office server machine?) > The webmail/imapproxy were actually running in a different datacenter to > the dovecot director/backend servers, but only about 20KM away. > > Ping tests: > > webmail->director: > > rtt min/avg/max/mdev = 0.933/1.061/2.034/0.183 ms > > director->backend: > > rtt min/avg/max/mdev = 0.104/0.108/0.127/0.005 ms > > webmail->localhost: > > rtt min/avg/max/mdev = 0.020/0.062/1.866/0.257 ms > > > -jf Hmm, not sure I understand the original numbers then? It seems intuitive that the proxy installed locally could save you 2x RTT increment, which is about 0.8ms in your case. So I might expect the proxy to reduce rendering times by around 1.6ms simply because it reduces the number of round trips to login? Kind of curious why that's not achieved..? Cheers Ed W From lists at wildgooses.com Wed Feb 22 11:38:47 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 09:38:47 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> Message-ID: <4F44B7A7.6030303@wildgooses.com> On 21/02/2012 20:36, Timo Sirainen wrote: > On 21.2.2012, at 16.33, Ed W wrote: > >> I'm also pleased to see that there is little negative cost in using a proxy... I recently added imap-proxy to our webmail setup because I wanted to log "last login + logout" times. I haven't quite figured out how to best log "logout time" (Timo, any chance of a post logout script? Or perhaps it's possible with the current login scripting?). > You could of course grep the logs, but other than that you'd need to write a Dovecot plugin. Luckily it's really simple to write a plugin. Basically: > > void postlogout_init(struct module *module) { } > void postlogout_deinit(void) { > system("/usr/local/bin/dovecot-postlogout.sh"); > } > > Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? Cheers Ed W From janfrode at tanso.net Wed Feb 22 11:48:46 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Wed, 22 Feb 2012 10:48:46 +0100 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B60B.4000105@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <20120222082536.GA8678@dibs.tanso.net> <4F44B60B.4000105@wildgooses.com> Message-ID: <20120222094846.GA10314@dibs.tanso.net> On Wed, Feb 22, 2012 at 09:31:55AM +0000, Ed W wrote: > > It seems intuitive that the proxy installed locally could save you > 2x RTT increment, which is about 0.8ms in your case. So I might > expect the proxy to reduce rendering times by around 1.6ms simply > because it reduces the number of round trips to login? Kind of > curious why that's not achieved..? Each http-request can probably trigger several IMAP requests. Maybe these work better in parallel directly to dovecot, than serialized (?) trough the imapproxy ? No idea if that's what's happening... or maybe the imapproxy just adds more overhead than the 2xRTT + imap logins it's supposed to save us ? -jf From tss at iki.fi Wed Feb 22 21:49:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 22 Feb 2012 21:49:54 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F44B7A7.6030303@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> Message-ID: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> On 22.2.2012, at 11.38, Ed W wrote: >> void postlogout_init(struct module *module) { } >> void postlogout_deinit(void) { >> system("/usr/local/bin/dovecot-postlogout.sh"); >> } >> >> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. > > Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. > > Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. From clube03 at gmail.com Wed Feb 22 22:49:01 2012 From: clube03 at gmail.com (Gustavo) Date: Wed, 22 Feb 2012 18:49:01 -0200 Subject: [Dovecot] Problem with Postfix + Dovecot + MySQL + Squirrelmail In-Reply-To: <4F37A377.80707@Media-Brokers.com> References: <2C09275EB540478192BDFAE5052597F9@VOSTRO1510> <4F37A377.80707@Media-Brokers.com> Message-ID: Ok, I do some changes and now I can send mails but not recieve mail. On log I get this message: Feb 22 20:02:04 localhost dovecot: deliver(teste at clubedolimite.com.br): Fatal: setgid(12(man)) failed with euid=150(vmail), gid=8(mail), egid=8(mail): Operation not permitted (This binary should probably be called with process group set to 12(man) instead of 8(mail)) Where is the problem now? Thanks! On Sun, Feb 12, 2012 at 9:33 AM, Charles Marcus wrote: > On 2012-02-11 1:46 PM, Gustavo wrote: > >> Ok, I study the wikki and the dovecot works. But when I try to send a mail >> using squirrel, I get this error on logs: >> >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: connect from >> localhost.localdomain[127.0.0.**1] >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: warning: >> xsasl_cyrus_server_get_**mechanism_list: no applicable SASL mechanisms >> Feb 11 18:41:55 jubileu postfix/smtpd[19522]: fatal: no SASL >> authentication >> mechanisms >> > > Sorry, but this has nothing to do with dovecot, unless your intention is > to use dovecot-sasl with your postfix... it appears you are using > cyrus-sasl and postfix on your smtp server, that is where your problem lies. > > Easiest would be to configure postfix to use dovecot-sasl: > > http://wiki2.dovecot.org/**HowTo/PostfixAndDovecotSASL > > -- > > Best regards, > > Charles > -- Gustavo - clube03 at gmail.com From lists at wildgooses.com Thu Feb 23 00:54:53 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 22:54:53 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> Message-ID: <4F45723D.2090003@wildgooses.com> On 22/02/2012 19:49, Timo Sirainen wrote: > On 22.2.2012, at 11.38, Ed W wrote: > >>> void postlogout_init(struct module *module) { } >>> void postlogout_deinit(void) { >>> system("/usr/local/bin/dovecot-postlogout.sh"); >>> } >>> >>> Add a few missing #includes and compile and enable for imap/pop3 and that should be it. >> Thanks - that's really obvious and quite interesting. I guess a simple "log" plugin makes sense. >> >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. Do you think you could keep something similar on your low priority backlog? Clearly parsing log files or hacking the code is possible, but I think the interest in the login scripting shows there is general interest, and having a full log of logon/logoff/bytes is clearly interesting to more minority users? >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. I think SWIG is for wrapping dovecot's api into the scripting language? (ie you could call dovecot methods from say perl/python/etc). What I had in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot normally calls a plugin method it will also run any [lua] scripts. I'm sure you know how to use google, but just so we are on the same page, top hit (below) from google shows how straight forward this is (lua has been built to be extremely fast and easy to embed, ie it's not an arbitrary choice) http://heavycoder.com/tutorials/lua_embed.php Cheers Ed W From tss at iki.fi Thu Feb 23 01:41:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 01:41:18 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F45723D.2090003@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> Message-ID: <1329954078.11500.266.camel@innu> On Wed, 2012-02-22 at 22:54 +0000, Ed W wrote: > >> Quick followup question - the logout log file currently logs a bunch of statistics such as mails read/deleted, bytes sent/received. How might I access these from the _deinit context as above? Apologies if this is a RTFM question? > > You'd have to build separate plugins for POP3 and IMAP, and even then it becomes tricky since there's no simple hook for catching when client gets destroyed. > > Do you think you could keep something similar on your low priority > backlog? Clearly parsing log files or hacking the code is possible, but > I think the interest in the login scripting shows there is general > interest, and having a full log of logon/logoff/bytes is clearly > interesting to more minority users? What do you need the statistics for? I could make imap_client and pop3_client support some virtual methods, like user.destroy() initially, which would be enough for your use. I guess I could add that for v2.2. > >> Finally, do you see it feasible to offer a "scriptable" plugin interface, eg perhaps using some high performance scripting language such as lua? Such a plugin might itself be simply a standard plugin..? The motivation being to offer the ability to create plugins to those who are nervous of using a compiler, and of course to reduce the ability of a badly written plugin to kill dovecot? > > I've been thinking about adding a scripting language plugin to Dovecot. Perhaps even using one of the existing generators that are supposed to make this easy for multiple languages, such as SWIG. But this is pretty low priority currently.. > > I think SWIG is for wrapping dovecot's api into the scripting language? > (ie you could call dovecot methods from say perl/python/etc). What I had > in mind was the reverse, ie embed LUA inside dovecot. Whenever dovecot > normally calls a plugin method it will also run any [lua] scripts. Yes, but I think SWIG can do that too. You'll need Dovecot's API implemented for scripting language if you want to do anything useful in the embedded script. That's the biggest job actually. Actually embeddeding some scripting language to Dovecot processes would be simple, they just couldn't really do anything useful. Also the nice thing about generically implementing Dovecot's APIs means that they could be used to build independent Dovecot applications, not just plugins. > I'm sure you know how to use google, but just so we are on the same > page, top hit (below) from google shows how straight forward this is > (lua has been built to be extremely fast and easy to embed, ie it's not > an arbitrary choice) > http://heavycoder.com/tutorials/lua_embed.php I've heard LUA being a commonly used embedded language, but I'd prefer to instead support several very widely used languages, such as Perl/Python. From lists at wildgooses.com Thu Feb 23 01:56:28 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 22 Feb 2012 23:56:28 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <4F4580AC.5030600@wildgooses.com> On 22/02/2012 23:41, Timo Sirainen wrote: > I've heard LUA being a commonly used embedded language, but I'd prefer > to instead support several very widely used languages, such as > Perl/Python. I'm a perl/ruby fan myself, but I would still recommend a good look at lua (or python) simply because they seem to be performant, easy to use, and on the surface seem to have had some thought about making them "embeddable". My new favourite editor "Sublime Text 2" has python as it's scripting language. Lua has been used for some big name games amongst other things. Perl has some memory management issues if you leave it long running, also writing XS code looks ok on the surface, but is an exercise in hair pulling in practice.... Ruby is a beautiful language, but unsure of how easy to embed and speed + memory management is an unknown (for high performance applications) I think it has potential though. I think a lot of the current plugins on the website could easily be rewritten, likely without performance concerns, using a scripting based plugin system. I could see that some other big picture pieces could potentially benefit also Thanks for considering it Ed W From tss at iki.fi Thu Feb 23 02:56:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 02:56:29 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <1329958589.11500.273.camel@innu> On Wed, 2012-02-22 at 23:56 +0000, Ed W wrote: > On 22/02/2012 23:41, Timo Sirainen wrote: > > I've heard LUA being a commonly used embedded language, but I'd prefer > > to instead support several very widely used languages, such as > > Perl/Python. > > I'm a perl/ruby fan myself, but I would still recommend a good look at > lua (or python) simply because they seem to be performant, easy to use, > and on the surface seem to have had some thought about making them > "embeddable". SWIG appears to generate Lua bindings just as well. But yes, I noticed Lua 5.1 has a feature called FFI that makes calling C functions quite easy (but then again, SWIG is supposed to do the same thing). > Perl has some memory management issues if you leave it long running, > also writing XS code looks ok on the surface, but is an exercise in hair > pulling in practice.... I've written a Perl plugin to irssi before directly with XS, and that's my main motivation for using some kind of an easy generator this time. It's way too much trouble to write any glue functions, especially when it works for only a single scripting language. From stephan at rename-it.nl Thu Feb 23 03:15:48 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 02:15:48 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F442592.608@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> Message-ID: <4F459344.5020407@rename-it.nl> On 2/22/2012 12:15 AM, Adam Szpakowski wrote: > On 22.02.2012 00:09, Timo Sirainen wrote: >> Well, it would be possible to build a doveadm script that deletes the >> duplicates after delivery, but currently there's no implementation to >> avoid delivering duplicate Message-IDs in the first place. >> >> I don't really like such a Message-ID-based deduplication feature >> enabled by default, but something like this could be nice: >> >> fileinto :copy :x-deduplicate "boss"; >> >> Anyway, probably not going to be implemented anytime soon. > Maybe there is a way to use a procmail with something like this: > > :0 Wh: msgid.lock > | formail -D 8192 .msgid.cache > > But is there a safe way to use it together with sieve? Using > Pigeonhole Sieve Pipe Plugin? > There are a few options: * You can use Procmail as primary delivery agent and invoke dovecot-lda/sieve from within Procmail once Procmail has determined that it is not a duplicate. * Invoke procmail from Sieve using the pipe extension (i.e. the other way around). This has the disadvantage that Procmail will have to take care of final delivery, meaning the Dovecot indexes are not updated. * For Pigeonhole v0.3 there is the possibility to "filter" the message through Procmail using the sieve_extprograms plugin, but I haven't actually tested something like that. * I've just created an alternative that implements something similar to the Procmail code you posted above, but from within Sieve itself. It is a custom language extension called vnd.dovecot.duplicate and it adds the "duplicate" test. This test keeps track of which Message-IDs it has seen before in earlier deliveries and yields a true result if the message was seen before, e.g.: require "vnd.dovecot.duplicate"; if duplicate { discard; } Read the specification for details ("name" argument is not yet implemented): http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate/raw-file/4b1dbda4d3fc/doc/rfc/spec-bosch-sieve-duplicate.txt The repository is at: http://hg.rename-it.nl/pigeonhole-0.3-sieve-duplicate This plugin is only a few hours old, experimental, and largely untested, so test it thoroughly before considering to use this. Read the INSTALL file for compile and installation instructions. Comments are welcome. Regards, Stephan. From atoth at atoth.sote.hu Thu Feb 23 07:20:26 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 06:20:26 +0100 Subject: [Dovecot] How to achieve proper privilege separation? Message-ID: I'm using a simple mbox config with regular Unix users and pam authentication. I'm also using grsecurity. That's why I see what dovecot does in which users' name. As times goes by and new versions are coming I can frustratedly see, that more and more tasks are performed as root. Why? When I used 1.x series of Dovecot, imap process started in the name of the user whose mbox was accessed. Now I can see, that nearly every task is performed by root. Why? It even tampers with the mail directories of each user as root instead of the user as it was usual long before. Please let me know how should I change the config to make dovecot stop using root privileges and revert it back to the old behavior. Thanks for your help: Dw. Here's my current config: dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = host.name last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster at host.name protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = References: Message-ID: On 23.2.2012, at 7.20, T?th Attila wrote: > I'm using a simple mbox config with regular Unix users and pam > authentication. > > I'm also using grsecurity. That's why I see what dovecot does in which > users' name. As times goes by and new versions are coming I can > frustratedly see, that more and more tasks are performed as root. Why? Less tasks should be running as root now. The master process code is a lot smaller. > When I used 1.x series of Dovecot, imap process started in the name of the > user whose mbox was accessed. > Now I can see, that nearly every task is performed by root. Why? It even > tampers with the mail directories of each user as root instead of the user > as it was usual long before. The imap process starts as root, does a userdb lookup and then drops privileges to that user. It worked this way before too, only the userdb lookup code was done by master process. From atoth at atoth.sote.hu Thu Feb 23 10:03:37 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 09:03:37 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: Message-ID: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap accesses both the inbox and the mail directories of the user as root. Moreover, it creates the lock file as root. I can see no process running as the user. How could I teach dovecot to start the imap process as the user. What configuration options I should blame? Thx: Dw. dovecot -n # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: Linux 3.2.6-hardened i686 Gentoo Base System release 2.0.3 auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_worker_max_count = 16 base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_gid = 99 first_valid_uid = 1000 hostname = last_valid_gid = 65533 last_valid_uid = 1003 listen = * mail_access_groups = mail mail_full_filesystem_access = yes mail_gid = mail mail_location = mbox:~/mail/:INBOX=/var/spool/mail/%u mail_max_keyword_length = 150 mail_privileged_group = mail mail_uid = mail passdb { args = * driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } postmaster_address = postmaster@ protocols = imap service auth-worker { user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 vsz_limit = 16 M } service imap { process_limit = 4 vsz_limit = 64 M } ssl_cert = On 23.2.2012, at 7.20, T?th Attila wrote: > >> I'm using a simple mbox config with regular Unix users and pam >> authentication. >> >> I'm also using grsecurity. That's why I see what dovecot does in which >> users' name. As times goes by and new versions are coming I can >> frustratedly see, that more and more tasks are performed as root. Why? > > Less tasks should be running as root now. The master process code is a lot > smaller. > >> When I used 1.x series of Dovecot, imap process started in the name of >> the >> user whose mbox was accessed. >> Now I can see, that nearly every task is performed by root. Why? It even >> tampers with the mail directories of each user as root instead of the >> user >> as it was usual long before. > > The imap process starts as root, does a userdb lookup and then drops > privileges to that user. It worked this way before too, only the userdb > lookup code was done by master process. > > From tss at iki.fi Thu Feb 23 10:15:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:15:12 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> Message-ID: <1329984912.11500.277.camel@innu> On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: > Unfortunately I can see, that in my case /usr/libexec/dovecot/imap > accesses both the inbox and the mail directories of the user as root. > Moreover, it creates the lock file as root. I can see no process running > as the user. > > How could I teach dovecot to start the imap process as the user. What > configuration options I should blame? Well, that's strange. There shouldn't be any way for you to make imap access mails as root, even if you wanted to do that. If you log in as root, it'll fail with: Error: user root: Invalid settings in userdb: userdb returned 0 as uid Fatal: Invalid user settings. Refer to server log for more information. If there's a bug and it just somehow manages to get through that check, it fails with: Fatal: We couldn't drop root privileges So.. I'm not really sure what could be wrong. It makes me think maybe Gentoo's hardening features somehow mess this up, but I can't really think of how that could either. Set auth_debug=yes and mail_debug=yes. What does it log when logging in? From tss at iki.fi Thu Feb 23 10:54:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 23 Feb 2012 10:54:24 +0200 Subject: [Dovecot] v2.1.1 released Message-ID: <1329987264.11500.282.camel@innu> http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.1.tar.gz.sig v2.1.0 had a couple of bugs, but overall people have been pretty quiet about it. I'm hoping v2.1.2 won't need to be released for many more weeks. :) BTW. v2.2 branch has also been opened in hg. Not much stuff in there yet though. + dsync: If message with same GUID is saved multiple times in session, copy it instead of re-saving. - acl plugin + autocreated mailboxes crashed when listing mailboxes - doveadm force-resync: Don't skip autocreated mailboxes (especially INBOX). - If process runs out of fds, stop listening for new connections only temporarily, not permanently (avoids hangs with process_limit=1 services) - auth: passdb imap crashed for non-login authentication (e.g. smtp). From campbell at cnpapers.com Thu Feb 23 17:16:34 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:16:34 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process Message-ID: <4F465852.6030506@cnpapers.com> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. Now I see where messages are in my maillog of the type: auth: pam_unix(dovecot:auth) : authentication failure ........ Why is dovecot involved in my smtp processes and how do I fix this. I've got some very mad users. The 10-auth.conf file is pretty much stock except for allowing plain text logins. steve campbell From as at 3a.pl Thu Feb 23 17:30:12 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:30:12 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465B84.6090300@3a.pl> On 23.02.2012 16:16, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail and > sasl2 stuff seems to be proper, but the user can't use the system on > port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix this. > I've got some very mad users. The 10-auth.conf file is pretty much > stock except for allowing plain text logins. Dovecot can be used as authentication source. It has its own implementation of SASL library which can be used instead of Cyrus SASL implementation. In Dovecot2 the config for auth is in 10-master.conf, service "auth". Please see: for dovecot 1.x: http://wiki.dovecot.org/Sasl for dovecot 2.x: http://wiki2.dovecot.org/Sasl on the postfix side: http://www.postfix.org/SASL_README.html#server_dovecot -- Adam Szpakowski From as at 3a.pl Thu Feb 23 17:37:46 2012 From: as at 3a.pl (Adam Szpakowski) Date: Thu, 23 Feb 2012 16:37:46 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F465D4A.1070005@3a.pl> On 23.02.2012 02:15, Stephan Bosch wrote: > On 2/22/2012 12:15 AM, Adam Szpakowski wrote: >> Maybe there is a way to use a procmail with something like this: >> >> :0 Wh: msgid.lock >> | formail -D 8192 .msgid.cache >> >> But is there a safe way to use it together with sieve? Using >> Pigeonhole Sieve Pipe Plugin? >> > There are a few options: > > * You can use Procmail as primary delivery agent and invoke > dovecot-lda/sieve from within Procmail once Procmail has determined > that it is not a duplicate. This is great option but if I understand correctly not when Postfix is using LMTP to talk to Dovecot. In this client of mine, the Postfix (MDA) and Dovecot (MAA) are in different physical locations and LMTP is tunnelled via SSH. > * I've just created an alternative that implements something similar > to the Procmail code you posted above, but from within Sieve itself. [cut] Great extension. I'll try and use it on one of the servers as soon as possible. Thanks for the work. BTW... There is another solution to my problem, which ends up to be implemented in this case. I'm using Postfix to BCC all mails on the domain level to the "archive" account. In this account I've set up all the logic which ends calling "redirect :copy". This solution eliminates copies from multiple To/CC. -- Adam Szpakowski From campbell at cnpapers.com Thu Feb 23 17:47:40 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 10:47:40 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465B84.6090300@3a.pl> References: <4F465852.6030506@cnpapers.com> <4F465B84.6090300@3a.pl> Message-ID: <4F465F9C.4080600@cnpapers.com> On 2/23/2012 10:30 AM, Adam Szpakowski wrote: > On 23.02.2012 16:16, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the system >> on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix this. >> I've got some very mad users. The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > Dovecot can be used as authentication source. It has its own > implementation of SASL library which can be used instead of Cyrus SASL > implementation. > > In Dovecot2 the config for auth is in 10-master.conf, service "auth". > > Please see: > for dovecot 1.x: http://wiki.dovecot.org/Sasl > for dovecot 2.x: http://wiki2.dovecot.org/Sasl > > on the postfix side: > http://www.postfix.org/SASL_README.html#server_dovecot > That's all great, but what about sendmail. What should my pam.d/dovecot file look like, what should my dovecot.conf file look like, all this just runs me around in circles. Thanks for the pointers, but I've been over that a million times and nothing seems to work. steve From CMarcus at Media-Brokers.com Thu Feb 23 17:47:25 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 10:47:25 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <4F465F8D.2020402@Media-Brokers.com> On 2012-02-23 10:16 AM, Steve Campbell wrote: > Why is dovecot involved in my smtp processes Because you told it to? > and how do I fix this. Depends on what your intent is - what MTA you use - how it is configured - etc... > I've got some very mad users. Don't make changes to an existing and/or go live with a new system without properly testing? > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. This file is irrelevant unless you have your MTA configured to use dovecot-sasl... dovecot only *assists* an MTA in authenticating, it isn't an MTA. -- Best regards, Charles From stephan at rename-it.nl Thu Feb 23 18:14:57 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 23 Feb 2012 17:14:57 +0100 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F465D4A.1070005@3a.pl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> <4F465D4A.1070005@3a.pl> Message-ID: <4F466601.6000303@rename-it.nl> Op 2/23/2012 4:37 PM, Adam Szpakowski schreef: > On 23.02.2012 02:15, Stephan Bosch wrote: >> * I've just created an alternative that implements something similar >> to the Procmail code you posted above, but from within Sieve itself. > [cut] > > Great extension. I'll try and use it on one of the servers as soon as > possible. Thanks for the work. > > BTW... > There is another solution to my problem, which ends up to be > implemented in this case. > I'm using Postfix to BCC all mails on the domain level to the > "archive" account. In this account I've set up all the logic which > ends calling "redirect :copy". > This solution eliminates copies from multiple To/CC. Ok, please keep me uprised. Once it is tested a bit more I'll add it to the main Pigeonhole package. This plugin does not add any dependencies, so this is no problem whatsoever. Regards, Stephan. From campbell at cnpapers.com Thu Feb 23 18:15:21 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:15:21 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465F8D.2020402@Media-Brokers.com> References: <4F465852.6030506@cnpapers.com> <4F465F8D.2020402@Media-Brokers.com> Message-ID: <4F466619.1090103@cnpapers.com> On 2/23/2012 10:47 AM, Charles Marcus wrote: > On 2012-02-23 10:16 AM, Steve Campbell wrote: >> Why is dovecot involved in my smtp processes > > Because you told it to? Where did I tell it to do this. > >> and how do I fix this. > > Depends on what your intent is - what MTA you use - how it is > configured - etc... > Sendmail - pretty much default except I'm trying to use port submission (587). >> I've got some very mad users. > > Don't make changes to an existing and/or go live with a new system > without properly testing? > Unfortunately, switching over to a new server didn't show me most of the problems until after it had run a while. Dovecot is probably a great application, but the wiki sucks when you run sendmail, the help files are a little vague and run me around in circles. I postponed (actually move the server back to the old server) for a week to fix the problems I ran into upon first putting it into server. After fixing those, it was only later that the last batch of problems showed up. You right that I should have tested more, but that's not what I need to be hearing right now. Most of my conversions go without problems and I've been doing this for quite a few years (decades). >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. > > This file is irrelevant unless you have your MTA configured to use > dovecot-sasl... dovecot only *assists* an MTA in authenticating, it > isn't an MTA. > My sendmail configs don't even list dovecot. steve From alexwbaule at gmail.com Thu Feb 23 18:19:30 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Thu, 23 Feb 2012 14:19:30 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin Message-ID: Hi Timo.... I Follow your instruction to look inside quota plugin to get the notification from dovecot that expunged a message, that's Ok. Follow the quota, it's use a array to keep all uids from a email expunged, to compare with uid that is got in notify_sync, that's OK too, but to me i need to keep uid and a char* with my body name, to erase it too. I made this: struct emexis_ids_x_uis{ uint32_t ids; char *uis_file; }; struct emexis_mailbox { union mailbox_module_context module_ctx; ARRAY_DEFINE(expunge_uids, struct emexis_ids_x_uis); }; ok, inside my expunged rewrite function i create the array like quota does, and append it to my array. struct emexis_ids_x_uis append_uis; append_uis.ids = _mail->uid; append_uis.uis_file = bodyFile; array_append(&ebox->expunge_uids, &append_uis, 1); But when i do a foreach in this Array, my ids is OK, but my uis_file has tha same value every time (the value is the lasted value inserted). Append to UID array (156) --> (/storage/gss/emexis/common/t/33/88/t33889eb1bccd28fd6643ebc3ecf548b12a1ad095) Append to UID array (157) --> (/storage/gss/emexis/common/t/e3/35/te335c20e978e12d37ad0a35ae009245cd34080c0) Append to UID array (158) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Calling Sync Expunged --> (156) Sync Expunged (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged Match (156 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (157 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Sync Expunged (158 == 156) --> (/storage/gss/emexis/common/t/0b/83/t0b83efc253c34db35843fc89b655586d628cccaf) Finish Calling Sync Expunged --> (156) Can i use a struct as a array Item ?? something is wrong with my array creation, append and etc ? Tks Timo !! From pgiessel at mac.com Thu Feb 23 18:23:28 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 07:23:28 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> > > I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. From kevingaspard at lavabit.com Thu Feb 23 18:26:45 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:26:45 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F4668C5.1020902@lavabit.com> Hi, I've a problem with Dovecot and Thunderbird. Actually I'm trying to setup a virtual host server mail with postfix + dovecot (2.0.9) on CentOS 6.2. This is my dovecot -n : http://pastebin.com/wZWAE5Cj (I guess the errors are not important, that's just saying there is some obsoletes values, I will fix that later until it's not inconvenient to setup correctly dovecot). I've a file to store password (/etc/dovecot/passwd) and when I'm trying to connect with telnet to my host on port 143 it works, after I can log myself with the command ?a login user at domain.tld passwd? and it works, this is the entire output of the telnet connection : http://pastebin.com/h10cQVqf but actually when I'm trying to connect with thunderbird I've an error about the password and the login, this is a screenshoot : http://images.koshie.fr/TB_problem.png and to finish, there is nothing into the log... The password and the user are corrects ! Of course, dovecot and postfix are started, this is the netstat -a | grep "dovecot" : http://pastebin.com/bwfsS4Tf If you need more informations tell me. PS : Excuse me for my bad english, also I'm new with dovecot. Cordially, GASPARD K?vin From rob0 at gmx.co.uk Thu Feb 23 18:33:40 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 10:33:40 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <20120223163340.GM24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > I've been trying to get smtp auth set up for days. All my sendmail > and sasl2 stuff seems to be proper, but the user can't use the > system on port 587, which is where I require authorization. > > Now I see where messages are in my maillog of the type: > > auth: pam_unix(dovecot:auth) : authentication failure ........ > > Why is dovecot involved in my smtp processes and how do I fix > this. I would question that these failures are in fact related to what Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it does not, therefore there is no way that Dovecot could possibly interfere with SMTP AUTH in Sendmail. > I've got some very mad users. And you are jumping to conclusions. I suggest that you take this matter to a Sendmail forum. When you do, provide all relevant configuration as well as complete logging to show the problem. No useful help is possible with what you posted here. > The 10-auth.conf file is pretty much > stock except for allowing plain text logins. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From campbell at cnpapers.com Thu Feb 23 18:36:07 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 11:36:07 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> References: <4F465852.6030506@cnpapers.com> <44383448-F920-4D0D-B3FB-0952D8B3DEAC@mac.com> Message-ID: <4F466AF7.5030009@cnpapers.com> On 2/23/2012 11:23 AM, Peter A. Giessel wrote: >> I've been trying to get smtp auth set up for days. All my sendmail and sasl2 stuff seems to be proper, but the user can't use the system on port 587, which is where I require authorization. > My guess is that your sasl2 configuration is configured to use your imap server to authenticate users. Check your sasl2 software's documentation. You're correct. This server is our smtp, imap, and pop server. All authentication is done there, supposedly through PAM, but dovecot seems to interfere with sendmail's smtp auth processes and overrides what typically was done in a non-interferred way with sasl. > From kevingaspard at lavabit.com Thu Feb 23 18:36:33 2012 From: kevingaspard at lavabit.com (=?ISO-8859-1?Q?GASPARD_K=E9vin?=) Date: Thu, 23 Feb 2012 17:36:33 +0100 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird Message-ID: <4F466B11.6020906@lavabit.com> I'm sorry, some one tell me my netstat (on IRC) is useless. Also my pastes will be destroyed in something like one day, that's stupid (that's originally for IRC), so this is the output of my : dovecot -n : http://pastebin.com/Tewcb7bm telnet : http://pastebin.com/6gY2puxF netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf Sorry for the error. From sm at resistor.net Thu Feb 23 18:34:46 2012 From: sm at resistor.net (SM) Date: Thu, 23 Feb 2012 08:34:46 -0800 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F465852.6030506@cnpapers.com> References: <4F465852.6030506@cnpapers.com> Message-ID: <6.2.5.6.2.20120223083039.09229478@resistor.net> Hi Steve, At 07:16 23-02-2012, Steve Campbell wrote: >I've been trying to get smtp auth set up for days. All my sendmail >and sasl2 stuff seems to be proper, but the user can't use the >system on port 587, which is where I require authorization. > >Now I see where messages are in my maillog of the type: > >auth: pam_unix(dovecot:auth) : authentication failure ........ > >Why is dovecot involved in my smtp processes and how do I fix this. >I've got some very If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Regards, -sm From campbell at cnpapers.com Thu Feb 23 19:10:20 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Thu, 23 Feb 2012 12:10:20 -0500 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <20120223163340.GM24983@harrier.slackbuilds.org> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> Message-ID: <4F4672FC.1080802@cnpapers.com> On 2/23/2012 11:33 AM, /dev/rob0 wrote: > On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >> I've been trying to get smtp auth set up for days. All my sendmail >> and sasl2 stuff seems to be proper, but the user can't use the >> system on port 587, which is where I require authorization. >> >> Now I see where messages are in my maillog of the type: >> >> auth: pam_unix(dovecot:auth) : authentication failure ........ >> >> Why is dovecot involved in my smtp processes and how do I fix >> this. > I would question that these failures are in fact related to what > Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it > does not, therefore there is no way that Dovecot could possibly > interfere with SMTP AUTH in Sendmail. Why is sendmail using Dovecot sasl when I have the regular sasl set up. If I turn off dovecot, telnet to 587, I get no failures in my logs. If I turn on dovecot and do the same, I get auth failures in my secure file. > >> I've got some very mad users. > And you are jumping to conclusions. I suggest that you take this > matter to a Sendmail forum. When you do, provide all relevant > configuration as well as complete logging to show the problem. No > useful help is possible with what you posted here. In other words, don't use sendmail if I use dovecot? > >> The 10-auth.conf file is pretty much >> stock except for allowing plain text logins. I'm really having problems following the logic here. Seems that postfix and dovecot are the only way to go if I use alternate ports with smtp auth. Is that what everyone is implying? I'll try to see what sendmail guys are saying, but I don't think they'll provide much as long as it involves dovecot. Thanks for the help steve From pgiessel at mac.com Thu Feb 23 19:12:53 2012 From: pgiessel at mac.com (Peter A. Giessel) Date: Thu, 23 Feb 2012 08:12:53 -0900 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <6.2.5.6.2.20120223083039.09229478@resistor.net> References: <4F465852.6030506@cnpapers.com> <6.2.5.6.2.20120223083039.09229478@resistor.net> Message-ID: <115F4A79-2861-4B1F-AB82-35B21F2C9091@mac.com> > > If you are using sendmail as the MTA, Dovecot is not involved for authentication on mail submission. The maillog line you provided does not contain enough information to tell what triggered the error. Not totally true. Typical cyrus sasl2 implementation used with sendmail uses an imap server for smtp authentication. I recall having to tweak the sasl2 configuration on my own sendmail server to talk correctly with dovecot. I don't recall the exact tweaks. I suggest reviewing the sasl2 implementation's documentation (the OP has not to my knowledge stated which sasl implementation he is using). It sounds like an sasl2 problem, not necessarily a dovecot problem. From keith1christian at gmail.com Thu Feb 23 19:31:49 2012 From: keith1christian at gmail.com (Keith Christian) Date: Thu, 23 Feb 2012 10:31:49 -0700 Subject: [Dovecot] Settings for authentication without an extra password file Message-ID: Hello, I'm new to dovecot, and, due to having an older version of an OS installed on a test server, I'll be using an older version of dovecot (dovecot-1.0-1.2) to provide IMAP services. On this wiki page (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) There are instructions for creating a separate dovecot password file: echo "$USER:{PLAIN}password" > passwd.dovecot sudo mv passwd.dovecot /etc So, it appears that two password files will be needed, one for dovecot and one for regular system logins. Is the statement above correct, or am I misreading something? Is there a setting in dovecot.conf which will authenticate IMAP using /etc/password and /etc/shadow, without a separate dovecot password file? ========Keith From rob0 at gmx.co.uk Thu Feb 23 19:41:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Thu, 23 Feb 2012 11:41:55 -0600 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <20120223174155.GO24983@harrier.slackbuilds.org> On Thu, Feb 23, 2012 at 12:10:20PM -0500, Steve Campbell wrote: > On 2/23/2012 11:33 AM, /dev/rob0 wrote: > >On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: > >>Why is dovecot involved in my smtp processes and how do I fix > >>this. > >I would question that these failures are in fact related to what > >Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK > >it does not, therefore there is no way that Dovecot could possibly > >interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set > up. Fortunately it seems that Peter has identified the issue: Cyrus SASL being configured to use IMAP for authentication. snip > In other words, don't use sendmail if I use dovecot? I didn't say that at all, and did not mean to imply it. > I'm really having problems following the logic here. Seems that > postfix and dovecot are the only way to go if I use alternate ports > with smtp auth. Is that what everyone is implying? One thing I *did* say is that what you posted was inadequate to be able to provide real help. And it seems that your issue is only tangentially related to Dovecot. > I'll try to see what sendmail guys are saying, but I don't think > they'll provide much as long as it involves dovecot. As Peter said, consult the Cyrus SASL documentation. If your SASL will be using IMAP for authentication, you need to ensure that it does so correctly for your Dovecot IMAP. As an alternative, change how Cyrus SASL is configured. The usual suggestion for Sendmail users is to use the same data backend for Cyrus SASL and Dovecot. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Thu Feb 23 20:01:21 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 23 Feb 2012 13:01:21 -0500 Subject: [Dovecot] Can't connect to my mail server (postfix + dovecot, IMAP) with Thunderbird In-Reply-To: <4F466B11.6020906@lavabit.com> References: <4F466B11.6020906@lavabit.com> Message-ID: <4F467EF1.2030803@Media-Brokers.com> On 2012-02-23 11:36 AM, GASPARD K?vin wrote: > Also my pastes will be destroyed in something like one day, that's > stupid (that's originally for IRC), so this is the output of my : > > dovecot -n : http://pastebin.com/Tewcb7bm > telnet : http://pastebin.com/6gY2puxF > netstat -Nnap | grep "dovecot" : http://pastebin.com/yx2wxfWf > > Sorry for the error. Please don't do that... just post the output in the body of the email. For one it complicates things (people reading email from terminals will not be able to help you), and as well many people simply will not click URL shortened links like those in an email. There is no reason to not simply paste the contents into the email body. -- Best regards, Charles From atoth at atoth.sote.hu Thu Feb 23 20:53:28 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 19:53:28 +0100 Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: <4F4672FC.1080802@cnpapers.com> References: <4F465852.6030506@cnpapers.com> <20120223163340.GM24983@harrier.slackbuilds.org> <4F4672FC.1080802@cnpapers.com> Message-ID: <767de6d20cc46bcbccef60bf1d21ab13.squirrel@atoth.sote.hu> http://dovecot.org/list/dovecot/2009-March/038435.html -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 18:10 id?pontban Steve Campbell ezt ?rta: > > > On 2/23/2012 11:33 AM, /dev/rob0 wrote: >> On Thu, Feb 23, 2012 at 10:16:34AM -0500, Steve Campbell wrote: >>> I've been trying to get smtp auth set up for days. All my sendmail >>> and sasl2 stuff seems to be proper, but the user can't use the >>> system on port 587, which is where I require authorization. >>> >>> Now I see where messages are in my maillog of the type: >>> >>> auth: pam_unix(dovecot:auth) : authentication failure ........ >>> >>> Why is dovecot involved in my smtp processes and how do I fix >>> this. >> I would question that these failures are in fact related to what >> Sendmail is doing. Does Sendmail even support Dovecot SASL? AFAIK it >> does not, therefore there is no way that Dovecot could possibly >> interfere with SMTP AUTH in Sendmail. > Why is sendmail using Dovecot sasl when I have the regular sasl set up. > > If I turn off dovecot, telnet to 587, I get no failures in my logs. If I > turn on dovecot and do the same, I get auth failures in my secure file. >> >>> I've got some very mad users. >> And you are jumping to conclusions. I suggest that you take this >> matter to a Sendmail forum. When you do, provide all relevant >> configuration as well as complete logging to show the problem. No >> useful help is possible with what you posted here. > > In other words, don't use sendmail if I use dovecot? >> >>> The 10-auth.conf file is pretty much >>> stock except for allowing plain text logins. > > I'm really having problems following the logic here. Seems that postfix > and dovecot are the only way to go if I use alternate ports with smtp > auth. Is that what everyone is implying? > > I'll try to see what sendmail guys are saying, but I don't think they'll > provide much as long as it involves dovecot. > > Thanks for the help > > steve > From trashcan at odo.in-berlin.de Thu Feb 23 21:55:55 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Thu, 23 Feb 2012 20:55:55 +0100 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: Hi -- On 21.02.2012, at 23:03, Michael Grimm wrote: > On 21.02.2012, at 22:57, Timo Sirainen wrote: >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > Ok. Then I will try both servers upgraded to 2.1 within the next days. JFTR: It took me some time to find a working syntax ;-) My working 2.0.18 syntax threw the following error: vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test doveadm(vmail): Fatal: Unknown print formatter: -u dsync-local(test): Error: read() from worker server failed: EOF Omitting '-f' or '-v -f' results in: vmail> dsync -v -u test ssh vmail at remote-host.tld dsync -v -u test dsync: illegal option -- u dsync-local(test): Error: read() from worker server failed: EOF Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test doveadm: illegal option -- v doveadm sync [-u |-A] [-S ] [-fR] [-m ] ok, following those hints: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm sync -u test ssh: Could not resolve hostname dsync-server: hostname nor servname provided, or not known dsync-local(test): Error: read() from worker server failed: EOF dsync-local(test): Error: read() from worker server failed: EOF Hmm, 'man doveadm' doesn't show 'sync', and 'man sync' shows old syntax. Luckily, I remembered finding an mail of yours in http://www.dovecot.org/list/dovecot/2012-January/062933.html : |>>> dsync mirror -> doveadm mirror |>>> dsync backup -> doveadm backup |>>> dsync server -> doveadm dsync-server (could be hidden from the doveadm commands list) | I did this now, with mirror -> sync. After some trial by error I finally found a working syntax: vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test >>> but as long I didn't get your http://wiki2.dovecot.org/Upgrading/2.1 wrong, it *should* work that way. >> >> Added it there. Therefore you might add a warning about different syntax as well? Please, don't get me wrong, I'm not complaining. This mail is JFTR, only. And, this mail might help others to save some time after an upgrade to 2.1 and those who mirror mailboxes sitting at different servers. Thanks and regards, Michael From atoth at atoth.sote.hu Thu Feb 23 21:56:39 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Thu, 23 Feb 2012 20:56:39 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <1329984912.11500.277.camel@innu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> Message-ID: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> In the mean time I've upgraded to 2.1. I've enabled debug logging and logged in. I suspect that hardening features can be blamed for my problem. After booting a previous kernel the behavior was reverted. Here is what I got. As I can make it out it uses the proper user for the imap process according to the logs. Thx: Dw. Feb 23 20:49:39 atoth dovecot: master: Dovecot v2.1.0 starting up (core dumps disabled) Feb 23 20:50:12 atoth dovecot: auth: Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16584) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50264 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: Loading modules from directory: /usr/lib/dovecot/auth Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 3337879553 16584 1 0a36f4227122eb3d59466523e937b25b Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 3337879553 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16587, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:12 atoth dovecot: imap(atoth): Disconnected: Logged out in=44 out=747 Feb 23 20:50:12 atoth dovecot: auth: Debug: auth client connected (pid=16588) Feb 23 20:50:12 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50265 resp= Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:12 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:12 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:12 atoth dovecot: auth: Debug: master in: REQUEST 401211393 16588 1 59b6d569049f955f31991ac3cfb1f54c Feb 23 20:50:12 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:12 atoth dovecot: auth: Debug: master out: USER 401211393 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16589, secured Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=42671 out=174898 Feb 23 20:50:14 atoth dovecot: auth: Debug: auth client connected (pid=16600) Feb 23 20:50:14 atoth dovecot: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=50276 resp= Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): lookup service=imap Feb 23 20:50:14 atoth dovecot: auth-worker(16586): Debug: pam(atoth,127.0.0.1): #1/1 style=1 msg=Password: Feb 23 20:50:14 atoth dovecot: auth: Debug: client out: OK 1 user=atoth Feb 23 20:50:14 atoth dovecot: auth: Debug: master in: REQUEST 3933732865 16600 1 8382f23ff412178311e55bf74162e4cd Feb 23 20:50:14 atoth dovecot: auth: Debug: passwd(atoth,127.0.0.1): lookup Feb 23 20:50:14 atoth dovecot: auth: Debug: master out: USER 3933732865 atoth system_groups_user=atoth uid=1000 gid=100 home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=16601, secured Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Effective uid=1000, gid=100, home=/home/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail/:INBOX=/var/spool/mail/atoth Feb 23 20:50:14 atoth dovecot: imap(atoth): Debug: fs: root=/home/atoth/mail, index=, control=, inbox=/var/spool/mail/atoth, alt= Feb 23 20:50:14 atoth dovecot: imap(atoth): Disconnected: Logged out in=405 out=9240 -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 23.(Cs) 09:15 id?pontban Timo Sirainen ezt ?rta: > On Thu, 2012-02-23 at 09:03 +0100, "T?th Attila" wrote: >> Unfortunately I can see, that in my case /usr/libexec/dovecot/imap >> accesses both the inbox and the mail directories of the user as root. >> Moreover, it creates the lock file as root. I can see no process running >> as the user. >> >> How could I teach dovecot to start the imap process as the user. What >> configuration options I should blame? > > Well, that's strange. There shouldn't be any way for you to make imap > access mails as root, even if you wanted to do that. If you log in as > root, it'll fail with: > > Error: user root: Invalid settings in userdb: userdb returned 0 as uid > Fatal: Invalid user settings. Refer to server log for more information. > > If there's a bug and it just somehow manages to get through that check, > it fails with: > > Fatal: We couldn't drop root privileges > > So.. I'm not really sure what could be wrong. It makes me think maybe > Gentoo's hardening features somehow mess this up, but I can't really > think of how that could either. > > Set auth_debug=yes and mail_debug=yes. What does it log when logging in? > > From marc at perkel.com Thu Feb 23 22:41:51 2012 From: marc at perkel.com (Marc Perkel) Date: Thu, 23 Feb 2012 12:41:51 -0800 Subject: [Dovecot] From Maildir back to smtp Message-ID: <4F46A48F.2040304@perkel.com> Not sure how this can be done. Right now I'm running a free backup MX record service. People point their high MX records to my servers and if they go down we store the email. When they come back up it's delivered. What I want to do is have a premium service that would allow them to view through IMAP the stored email. In order to do that I would have to deliver the email locally to a dovecot server - to a single account perhaps - and they can view/forward/delete etc. But - when their server comes back online I'd like to gather up all the delivered email stored in Maildir format and run it back into smtp for delivery. Has anyone even done something like this? Or is there an easier way that I'm missing? Thanks in advance From mailing at securitylabs.it Fri Feb 24 00:12:14 2012 From: mailing at securitylabs.it (mailing at securitylabs.it) Date: Thu, 23 Feb 2012 23:12:14 +0100 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46A48F.2040304@perkel.com> References: <4F46A48F.2040304@perkel.com> Message-ID: <4F46B9BE.3000801@securitylabs.it> Il 23/02/2012 21:41, Marc Perkel ha scritto: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > Hi, store how? In a queue with smtp server? > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. > If you store the messages in a queue you can configure your smtp (your MX) server to send a copy of all message it receive also to another mailbox, see always_bcc in posfix. From wilberth.perez at uady.mx Fri Feb 24 00:33:46 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Thu, 23 Feb 2012 16:33:46 -0600 Subject: [Dovecot] How to limit max number of connections for ip address Message-ID: <4F46BECA.3020208@uady.mx> Hi List: Does any one , knows if is possible configure dovecot for limit max number of connections for IP address? I would like to prevent future fork-bombing attacks for pop3 and imap login process in my email server. Our dovecot version is : 1.2.10 Thanks, Wilberth. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From ar-dovecotlist at acrconsulting.co.uk Fri Feb 24 01:03:34 2012 From: ar-dovecotlist at acrconsulting.co.uk (Andrew Richards) Date: 23 Feb 2012 23:03:34 +0000 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <201202232303.34310.ar-dovecotlist@acrconsulting.co.uk> On Thursday 23 February 2012 22:12:14 mailing at securitylabs.it wrote: > Il 23/02/2012 21:41, Marc Perkel ha scritto: > > Not sure how this can be done. > > > > Right now I'm running a free backup MX record service. People point > > their high MX records to my servers and if they go down we store the > > email. When they come back up it's delivered. > > Hi, store how? In a queue with smtp server? > > > What I want to do is have a premium service that would allow them to > > view through IMAP the stored email. In order to do that I would have > > to deliver the email locally to a dovecot server - to a single account > > perhaps - and they can view/forward/delete etc. > > > > But - when their server comes back online I'd like to gather up all > > the delivered email stored in Maildir format and run it back into smtp > > for delivery. Sounds like a job for serialmail, a sister package to qmail: Use serialmail to forward mail from a mailbox (in Maildir format which you have) to the primary MX when it's back up. I've provided a mailbox/forwarding secondary MX service for several clients using this method (esp. 'maildirsmtp' program from serialmail with cron). More info/help on the qmail or serialmail mailing lists; URL for serialmail is, http://cr.yp.to/serialmail.html (it includes man pages within the package) A couple of particular advantages of having secondary MX email in a mailbox as you suggest for forwarding: - Unlike mail in a queue, the messages don't expire, so if the primary MX takes forever to come back up that's fine (except for your disk storage) - Since the mail is in a mailbox you can access this with IMAP or POP3 as you suggest [until it's emptied when the primary MX returns] I've not considered running serialmail in an non-qmail environment, but it would probably still work for the purpose you have in mind if you're using a different MTA. > If you store the messages in a queue you can configure your smtp (your > MX) server to send a copy of all message it receive also to another > mailbox, see always_bcc in posfix. cheers, Andrew. From tss at iki.fi Fri Feb 24 01:21:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:21:44 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F46BECA.3020208@uady.mx> References: <4F46BECA.3020208@uady.mx> Message-ID: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> On 24.2.2012, at 0.33, Wilberth Perez wrote: > Does any one , knows if is possible configure dovecot for limit max > number of connections for IP address? > > I would like to prevent future fork-bombing attacks for pop3 and imap > login process in my email server. > > Our dovecot version is : 1.2.10 There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. From tss at iki.fi Fri Feb 24 01:26:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:26:08 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> Message-ID: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> On 23.2.2012, at 21.56, T?th Attila wrote: > In the mean time I've upgraded to 2.1. > I've enabled debug logging and logged in. > > I suspect that hardening features can be blamed for my problem. After > booting a previous kernel the behavior was reverted. OK. > Feb 23 20:50:12 atoth dovecot: imap(atoth): Debug: Effective uid=1000, > gid=100, home=/home/atoth This says that the IMAP is running as UID 1000. The code that produces this is: i_debug("Effective uid=%s, gid=%s, home=%s", dec2str(geteuid()), dec2str(getegid()), home); So if the process is still creating files as root, the kernel is lying.. From tss at iki.fi Fri Feb 24 01:28:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:28:47 +0200 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: On 24.2.2012, at 1.26, Timo Sirainen wrote: > On 23.2.2012, at 21.56, T?th Attila wrote: > >> In the mean time I've upgraded to 2.1. >> I've enabled debug logging and logged in. >> >> I suspect that hardening features can be blamed for my problem. After >> booting a previous kernel the behavior was reverted. > > OK. BTW. It would be interesting to know if the attached patch causes it to fail with the hardening patches. -------------- next part -------------- A non-text attachment was scrubbed... Name: euid.diff Type: application/octet-stream Size: 532 bytes Desc: not available URL: From tss at iki.fi Fri Feb 24 01:32:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:32:06 +0200 Subject: [Dovecot] Settings for authentication without an extra password file In-Reply-To: References: Message-ID: <19807B92-11DB-4901-8909-CCFA6003FEB9@iki.fi> On 23.2.2012, at 19.31, Keith Christian wrote: > Hello, I'm new to dovecot, and, due to having an older version of an > OS installed on a test server, I'll be using an older version of > dovecot (dovecot-1.0-1.2) to provide IMAP services. I think Dovecot v2.x works mostly on the same OSes as v1.x. > On this wiki page > (http://wiki.dovecot.org/BasicConfiguration?highlight=%28passwd.dovecot%29) > > There are instructions for creating a separate dovecot password file: > > echo "$USER:{PLAIN}password" > passwd.dovecot > sudo mv passwd.dovecot /etc > > So, it appears that two password files will be needed, one for dovecot > and one for regular system logins. Only if you want to. > Is there a setting in dovecot.conf which will authenticate IMAP using > /etc/password and /etc/shadow, without a separate dovecot password > file? Use passdb pam + userdb passwd (not passwd-file). See password + user databases under http://wiki.dovecot.org/Authentication From jtam.home at gmail.com Fri Feb 24 01:32:50 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Thu, 23 Feb 2012 15:32:50 -0800 (PST) Subject: [Dovecot] Why is dovecot involved in my smtp process In-Reply-To: References: Message-ID: From: "Peter A. Giessel" : > Not totally true. Typical cyrus sasl2 implementation used with > sendmail uses an imap server for smtp authentication. I recall having > to tweak the sasl2 configuration on my own sendmail server to talk > correctly with dovecot. I don't recall the exact tweaks. Just to make it clear to the OP, I have a Dovecot/Sendmail/SASL working just fine (which is independent of dovecot). I suspect Peter hit the nail on the head here. Look at the contents of $SASL_INSTALL/lib/Sendmail.conf Mine is (only) pwcheck_method:saslauthd which I think looks up my name service backend via the login and plain plugins. I'm a little hazy on how this was set up, but if you are able to successfully login without dovecot, then it appears you have some sort of authentication stack that will try IMAP first, then fallback to your the other method. Joseph Tam From tss at iki.fi Fri Feb 24 01:34:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Fri, 24 Feb 2012 01:34:48 +0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: References: Message-ID: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> On 23.2.2012, at 18.19, Alex Baule wrote: > ok, inside my expunged rewrite function i create the array like quota > does, and append it to my array. > > struct emexis_ids_x_uis append_uis; > append_uis.ids = _mail->uid; > append_uis.uis_file = bodyFile; .. > But when i do a foreach in this Array, my ids is OK, but my uis_file > has tha same value every time (the value is the lasted value > inserted). I guess the memory isn't permanently allocated for it, so you need to do: append_uis.uis_file = i_strdup(bodyFile); Note that you'll also need to later i_free() it to avoid leaking memory. From lists at wildgooses.com Fri Feb 24 01:41:37 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 23 Feb 2012 23:41:37 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4580AC.5030600@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <4F4580AC.5030600@wildgooses.com> Message-ID: <4F46CEB1.1040604@wildgooses.com> On 22/02/2012 23:56, Ed W wrote: > I think it has potential though. I think a lot of the current plugins > on the website could easily be rewritten, likely without performance > concerns, using a scripting based plugin system. I could see that > some other big picture pieces could potentially benefit also One interesting test case for such a scripting hooks solution might be login restrictions. There seem to be regular requests for the ability to setup arbitrarily complicated restrictions on users per IP, attempts per second, etc (and my logging interest is kind of related also). Not trying to bump the item up any todo lists, just trying to chuck in some concrete ideas for actually testing a specific implementation... I guess a substantially more performance orientated area that seems to get some interest would be various spam, expunge, delete ideas and the hooks needed for those. These seem much more tricky to implement a scripting hook and still stay performant. Again just ideas for real things people might want to do? Cheers Ed W From amk at krell.zikzak.de Fri Feb 24 03:22:47 2012 From: amk at krell.zikzak.de (Andreas M. Kirchwitz) Date: Fri, 24 Feb 2012 02:22:47 +0100 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build Message-ID: <20120224012247.GA6512@krell.zikzak.de> Hello Dovecot users! There seems to be a new dependency in some modules (eg, lib-storage, libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those modules didn't require OpenSSL, but 2.1 does. For the linking process the path to the OpenSSL library isn't specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't properly used as well.) Example for lib-storage: env SSL_CFLAGS="-I/usr/local/ssl/include" SSL_LIBS="-L/usr/local/ssl/lib -Wl,-R/usr/local/ssl/lib -lcrypto -lssl" ./configure --prefix=/usr/local/${DOVECOTVER} --with-ssl=openssl --with-ssldir=/usr/local/${DOVECOTVER}/etc/dovecot/certs make gcc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-master -I../../src/lib-settings -I../../src/lib-mail -I../../src/lib-storage -DPKG_RUNDIR=\""/usr/local/Dovecot-2.1.1/var/run/dovecot"\" -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -MT master-connection.o -MD -MP -MF .deps/master-connection.Tpo -c -o master-connection.o master-connection.c mv -f .deps/master-connection.Tpo .deps/master-connection.Po /bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -no-undefined -Wl,--as-needed -o indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o ../../src/lib-storage/libdovecot-storage.la ../../src/lib-dovecot/libdovecot.la -export-dynamic -ldl -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/local/ssl/include -Wl,--as-needed -o .libs/indexer-worker indexer-worker.o indexer-worker-settings.o master-connection.o -Wl,--export-dynamic ../../src/lib-storage/.libs/libdovecot-storage.so ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/Dovecot-2.1.1/lib/dovecot /usr/bin/ld: warning: libcrypto.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libssl.so.1.0.0, needed by ../../src/lib-storage/.libs/libdovecot-storage.so, not found (try using -rpath or -rpath-link) ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `OBJ_txt2nid' ../../src/lib-storage/.libs/libdovecot-storage.so: undefined reference to `SSL_set_ex_data' [...] # ldd src/lib*/.libs/*.so src/lib-dovecot/.libs/libdovecot.so: linux-gate.so.1 => (0x00979000) libdl.so.2 => /lib/libdl.so.2 (0x00c31000) librt.so.1 => /lib/librt.so.1 (0x009ed000) libc.so.6 => /lib/libc.so.6 (0x0054d000) /lib/ld-linux.so.2 (0x00c12000) libpthread.so.0 => /lib/libpthread.so.0 (0x008f4000) src/lib-lda/.libs/libdovecot-lda.so: linux-gate.so.1 => (0x0096d000) libdovecot-storage.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-storage/.libs/libdovecot-storage.so.0 (0x00ae4000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00110000) librt.so.1 => /lib/librt.so.1 (0x00189000) libc.so.6 => /lib/libc.so.6 (0x00192000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x0031c000) libpthread.so.0 => /lib/libpthread.so.0 (0x00ebc000) /lib/ld-linux.so.2 (0x003f8000) src/lib-sql/.libs/libdovecot-sql.so: linux-gate.so.1 => (0x00a95000) libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x002b6000) libdl.so.2 => /lib/libdl.so.2 (0x00ab8000) librt.so.1 => /lib/librt.so.1 (0x0072a000) libc.so.6 => /lib/libc.so.6 (0x008f7000) /lib/ld-linux.so.2 (0x00ebd000) libpthread.so.0 => /lib/libpthread.so.0 (0x00619000) src/lib-ssl-iostream/.libs/libdovecot-ssl.so: linux-gate.so.1 => (0x00da6000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdl.so.2 => /lib/libdl.so.2 (0x00df0000) librt.so.1 => /lib/librt.so.1 (0x00112000) libc.so.6 => /lib/libc.so.6 (0x00b5e000) /lib/ld-linux.so.2 (0x00e76000) libpthread.so.0 => /lib/libpthread.so.0 (0x005a6000) src/lib-storage/.libs/libdovecot-storage.so: linux-gate.so.1 => (0x006ed000) libcrypto.so.1.0.0 => not found libssl.so.1.0.0 => not found libdovecot.so.0 => /usr/local/src/dovecot-2.1.1/src/lib-dovecot/.libs/libdovecot.so.0 (0x00c66000) libdl.so.2 => /lib/libdl.so.2 (0x00491000) librt.so.1 => /lib/librt.so.1 (0x00be2000) libc.so.6 => /lib/libc.so.6 (0x00110000) /lib/ld-linux.so.2 (0x0036b000) libpthread.so.0 => /lib/libpthread.so.0 (0x0029a000) Without SSL_CFLAGS and SSL_LIBS but just PKG_CONFIG_PATH, it also won't work. Help appreciated ... Andreas From alexwbaule at gmail.com Fri Feb 24 04:32:47 2012 From: alexwbaule at gmail.com (Alex Baule) Date: Fri, 24 Feb 2012 00:32:47 -0200 Subject: [Dovecot] Creating and interacting with array inside plugin In-Reply-To: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> References: <018C87FC-548B-464A-A02E-5ADD31BA6BD4@iki.fi> Message-ID: Tks Timo... Em 23 de fevereiro de 2012 21:34, Timo Sirainen escreveu: > On 23.2.2012, at 18.19, Alex Baule wrote: > >> ok, inside my expunged rewrite function i create the array like quota >> does, and append it to my array. >> >> struct emexis_ids_x_uis append_uis; >> append_uis.ids = _mail->uid; >> append_uis.uis_file = bodyFile; > .. >> But when i do a foreach in this Array, my ids is OK, but my uis_file >> has tha same value every time (the value is the lasted value >> inserted). > > I guess the memory isn't permanently allocated for it, so you need to do: > > append_uis.uis_file = i_strdup(bodyFile); > > Note that you'll also need to later i_free() it to avoid leaking memory. > From CMarcus at Media-Brokers.com Fri Feb 24 14:18:43 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 07:18:43 -0500 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F46B9BE.3000801@securitylabs.it> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> Message-ID: <4F478023.9080706@Media-Brokers.com> On 23/02/2012 21:41, Marc Perkel wrote: > Not sure how this can be done. > > Right now I'm running a free backup MX record service. People point > their high MX records to my servers and if they go down we store the > email. When they come back up it's delivered. > > What I want to do is have a premium service that would allow them to > view through IMAP the stored email. In order to do that I would have > to deliver the email locally to a dovecot server - to a single account > perhaps - and they can view/forward/delete etc. > > But - when their server comes back online I'd like to gather up all > the delivered email stored in Maildir format and run it back into smtp > for delivery. I'm thinking it would be possible to use dsync for this? Ideally you would have to have a way to maintain a duplicate of the AUTH database, so that users could use their current passwords for accessing it (this would obviously require the approval and cooperation of whoever you are providing this service to), but this way you could store each users mail individually and provide relatively painless access (since all each user would need to know is the backup host name and their username/password). You'd have to configure a script on your end to: 1. determine a reliable way to differentiate between a temp-fail situation on the primary (in which case you just act as a normal backup MX), or an extended outage (15 minutes? 1 hour? I'd probably let the customer decide this one), which would determine when you start actually taking delivery of their mail to the 'standby' server 2. monitor for when the primary comes back online 3. wait xx interval to make sure the primary stays up (again, I'd let the customer decide this, but with a sane minimum that you define) 4. once the primary stays up for the required interval, dsync a one way 'dump' of the stored mails, being sure to test for success - and this process would obviously have to take into account the possibility of the primary going down again in the middle of the dsync 5. mv successfully restored mails on the backup mx to a temp location for xx 'safety' interval 'just in case' 6. delete the backed-up/stored mail that has been successfully dsync'd after the safety interval has elapsed -- Best regards, Charles From troy at surety-networks.com Fri Feb 24 15:55:57 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 08:55:57 -0500 Subject: [Dovecot] Apple od/ad authentication problem Message-ID: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. OS X 10.7.3, behind a Cisco asa NAT. When trying to authenticate (ssl or not) via the webmail interface I get this error... NAT IP is the public ip I assigned. It doesn't work for an AD account or an account local to the Mac server. However it DOES work on the internal LAN. Feb 22 18:15:36 mail dovecot[3501]: auth: Error: od (username,natIP): verify plain: lookup failed for user: username Feb 22 18:15:51 mail dovecot[3501]: auth: Error: od[getpwnam_ext](username,natIP): No record for user I figured I'm missing something in the dovecot config. I have followed apple's kb articles for AD webmail authentication and kerberos, and that works internally. From przemek.orzechowski at makolab.pl Fri Feb 24 16:22:13 2012 From: przemek.orzechowski at makolab.pl (=?UTF-8?Q? Przemys=C5=82aw.Orzechowski ?=) Date: Fri, 24 Feb 2012 15:22:13 +0100 Subject: [Dovecot] iPhone and custom imap folders problem Message-ID: Hi Im using Dovecot 1.0.10 and everything seems work ok except one client He is accessing his mailbox using iPhone There are several custom folders created and mail is directed to them using procmail The problem is iPhone mail app only sees changes in INBOX and when new mail arrives to other folders it will not see it unlers user accesses this particular folder Is there some kind of workaround for this? Im quite sure its iPhones fault as all other email clients are workin perfectly fine. From jacob at alltechservices-ia.com Fri Feb 24 16:27:14 2012 From: jacob at alltechservices-ia.com (jacob at alltechservices-ia.com) Date: Fri, 24 Feb 2012 08:27:14 -0600 Subject: [Dovecot] Virtual Folders Message-ID: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Hello, I am trying to get virtual folders to work properly, and I can't seem to. I will be as detailed as possible to remove confusion. Thank you in advance. Goal: Essentially, a gmail clone, but one that I can make work on cPanel hosts with roundcube and squirrel mail, etc. By default, cpanel creates user mail dir structure of (maildir++) inbox, drafts, sent, junk, and trash (with no dovecot specific files until first login) I would like to replace all those folders with virtual representations with more capabilities. I would like to achieve the conversation view that gmail has pulled off so well. cPanel, stock, generates the following dovecot -n: (i am still not sure how it negotiates mail_location yet) # 1.2.17: /etc/dovecot.conf # OS: Linux 2.6.18-238.5.1.el5xen x86_64 CentOS release 5.7 (Final) ssl_cert_file: /etc/dovecot/ssl/dovecot.crt ssl_key_file: /etc/dovecot/ssl/dovecot.key ssl_cipher_list: ALL:!ADH:RC4+RSA:+HIGH:+MEDIUM:-LOW:-SSLv2:-EXP login_dir: /var/run/dovecot/login login_executable: /usr/libexec/dovecot/imap-login login_process_per_connection: no login_processes_count: 10 login_max_processes_count: 100 login_max_connections: 500 mail_max_userip_connections: 200 maildir_copy_preserve_filename: yes mail_plugins: quota imap_quota namespace: type: private prefix: INBOX. inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster at example.com auth default: mechanisms: plain login cache_size: 2048 username_chars: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!#$-=?^_{}~./@+% passdb: driver: checkpassword args: /usr/local/cpanel/bin/dovecot-auth userdb: driver: prefetch socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 plugin: quota: maildir quota_rule: Trash:ignore I started a test box with Debian running the same version of dovecot (1.2.x) and on that system I run into problems every time I make a virtual folder a root folder. I assume it is trying to look up a normal folder instead of the virtual ones and I am confusing it. So I came here because it would be really really neat if I could pull this off. Again, thank you in advance. -cornfeed From CMarcus at Media-Brokers.com Fri Feb 24 16:47:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 09:47:44 -0500 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47A310.9000308@Media-Brokers.com> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski wrote: > Im using Dovecot 1.0.10 Very old - an upgrade would likeley fix your problems... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From dg at dguhl.org Fri Feb 24 17:11:58 2012 From: dg at dguhl.org (Dennis Guhl) Date: Fri, 24 Feb 2012 16:11:58 +0100 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> Message-ID: <20120224151158.GB16307@PC211.ikt.de> On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: > Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. > > OS X 10.7.3, behind a Cisco asa NAT. Deactivate the SMTP improvements your ASA helpfully provide. Dennis [..] From radecki.rafal at gmail.com Fri Feb 24 17:20:36 2012 From: radecki.rafal at gmail.com (=?ISO-8859-2?Q?Rafa=B3_Radecki?=) Date: Fri, 24 Feb 2012 16:20:36 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. Message-ID: Hi all. I am currently gathering information about migration of Maildir/ structures from Courier to Dovecot. What do you think about http://wiki.dovecot.org/Migration/Courier ? For each courier Maildir account I want to use http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce dovecot-uidlist file mv courierimapsubscribed subscriptions sed sed -i 's/INBOX\.//' subscriptions rm -r courierimaphieracl rm courierimapacl rm courierimapkeywords I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf Should I look into something else? Have I missed something? Best regards, Rafal. From troy at surety-networks.com Fri Feb 24 17:43:03 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 10:43:03 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <45684F00-E4C4-4FD3-AD71-4E21B74C471A@surety-networks.com> Looking over the running config I don't see anything relating to smtp. I'm not using the esmtp inspection either. Am I looking in the wrong place? Would this cause the error I'm getting in mailaccess log? - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From micha at krausam.de Fri Feb 24 18:26:24 2012 From: micha at krausam.de (Micha Krause) Date: Fri, 24 Feb 2012 17:26:24 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method Message-ID: <4F47BA30.2080704@krausam.de> Hi, I want to deliver mails directly to subfolders, based on address extensions. Example: Emails sent to: micha+test at example.com Should be stored into user micha, folder INBOX.test I was able to reproduce this with exim and dovecot-lda: command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" But I was wondering if I could do this within dovecot, which would enable me to use LMTP instead of dovecot-lda. Which brings me to my second question: Which method is "better" for dovecot, lda or LMTP? Micha Krause From gfinch at ldmltd.ca Fri Feb 24 18:36:00 2012 From: gfinch at ldmltd.ca (Gregory Finch) Date: Fri, 24 Feb 2012 08:36:00 -0800 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: References: Message-ID: <4F47BC70.40308@ldmltd.ca> On 2012-02-24 6:22 AM, Przemys?aw.Orzechowski wrote: > Hi > > Im using Dovecot 1.0.10 and everything seems work ok except one client > He is accessing his mailbox using iPhone > There are several custom folders created and mail is directed to them > using procmail > The problem is iPhone mail app only sees changes in INBOX and when new > mail arrives to other folders it will not see it unlers user accesses this > particular folder > Is there some kind of workaround for this? > > Im quite sure its iPhones fault as all other email clients are workin > perfectly fine. As far as I know, that is a limitation of the iPhone mail client: it only IDLEs on the currently selected folder, and it only checks the currently selected folder for new messages. You will not see new mail for the other folders unless you select them. -Greg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 260 bytes Desc: OpenPGP digital signature URL: From as at 3a.pl Fri Feb 24 18:46:02 2012 From: as at 3a.pl (Adam Szpakowski) Date: Fri, 24 Feb 2012 17:46:02 +0100 Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <4F47A310.9000308@Media-Brokers.com> References: <4F47A310.9000308@Media-Brokers.com> Message-ID: <4F47BECA.4020104@3a.pl> On 24.02.2012 15:47, Charles Marcus wrote: > On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski > wrote: >> Im using Dovecot 1.0.10 > Very old - an upgrade would likeley fix your problems... A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. Only inbox is synced automatically. -- Adam Szpakowski From wilberth.perez at uady.mx Fri Feb 24 17:07:20 2012 From: wilberth.perez at uady.mx (Wilberth Perez) Date: Fri, 24 Feb 2012 09:07:20 -0600 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> Message-ID: <4F47A7A8.4090009@uady.mx> Hi Timo: My question was because constantly we received brute force attack from some of ip address which uses pop3 service to affect dovecot's login proccess. For example: Error: Temporary failure in creating login processes, slowing down for now pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A pop3-login: Info: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=A.B.C.D, lip=X.Y.Z.A auth(default): Info: shadow(best,A.B.C.D): unknown user dovecot: Error: pipe() failed: Too many open files dovecot: Error: Temporary failure in creating login processes, slowing down for now In the log above from dovecto.log file, we observed a lot of conections from IP address A.B.C.D to our email server with ip address X.Y.Z.A using pop3 login process. Is possible prevent this type of attacks with any dovecot option (maybe limit the number of max connections from one ip address or maybe upgrading my dovecot version)? Thanks for you help and time. Wilberth. El 23/02/2012 05:21 p.m., Timo Sirainen escribi?: > On 24.2.2012, at 0.33, Wilberth Perez wrote: > >> Does any one , knows if is possible configure dovecot for limit max >> number of connections for IP address? >> >> I would like to prevent future fork-bombing attacks for pop3 and imap >> login process in my email server. >> >> Our dovecot version is : 1.2.10 > > There is mail_max_userip_connections setting which limits IP+username combination. Typically that should be enough to prevent fork bombing, because users normally don't have more than one account. > > Or you mean when some IP keeps connecting even without actually loggin in? http://wiki.dovecot.org/LoginProcess has some settings related to this, which should normally be quite helpful if the limits are right. > > -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2570 bytes Desc: Firma criptogr??fica S/MIME URL: From marc at perkel.com Fri Feb 24 19:09:12 2012 From: marc at perkel.com (Marc Perkel) Date: Fri, 24 Feb 2012 09:09:12 -0800 Subject: [Dovecot] From Maildir back to smtp In-Reply-To: <4F478023.9080706@Media-Brokers.com> References: <4F46A48F.2040304@perkel.com> <4F46B9BE.3000801@securitylabs.it> <4F478023.9080706@Media-Brokers.com> Message-ID: <4F47C438.2090300@perkel.com> On 2/24/2012 4:18 AM, Charles Marcus wrote: > On 23/02/2012 21:41, Marc Perkel wrote: >> Not sure how this can be done. >> >> Right now I'm running a free backup MX record service. People point >> their high MX records to my servers and if they go down we store the >> email. When they come back up it's delivered. >> >> What I want to do is have a premium service that would allow them to >> view through IMAP the stored email. In order to do that I would have >> to deliver the email locally to a dovecot server - to a single account >> perhaps - and they can view/forward/delete etc. >> >> But - when their server comes back online I'd like to gather up all >> the delivered email stored in Maildir format and run it back into smtp >> for delivery. > > I'm thinking it would be possible to use dsync for this? Ideally you > would have to have a way to maintain a duplicate of the AUTH database, > so that users could use their current passwords for accessing it (this > would obviously require the approval and cooperation of whoever you > are providing this service to), but this way you could store each > users mail individually and provide relatively painless access (since > all each user would need to know is the backup host name and their > username/password). > > You'd have to configure a script on your end to: > > 1. determine a reliable way to differentiate between a temp-fail > situation on the primary (in which case you just act as a normal > backup MX), or an extended outage (15 minutes? 1 hour? I'd probably > let the customer decide this one), which would determine when you > start actually taking delivery of their mail to the 'standby' server > > 2. monitor for when the primary comes back online > > 3. wait xx interval to make sure the primary stays up (again, I'd let > the customer decide this, but with a sane minimum that you define) > > 4. once the primary stays up for the required interval, dsync a one > way 'dump' of the stored mails, being sure to test for success - and > this process would obviously have to take into account the possibility > of the primary going down again in the middle of the dsync > > 5. mv successfully restored mails on the backup mx to a temp location > for xx 'safety' interval 'just in case' > > 6. delete the backed-up/stored mail that has been successfully dsync'd > after the safety interval has elapsed I was wondering if I could just gather up the maildir files, reprocess them into batch SMTP format and run them back into Exim? From miha.vrhovnik at cordia.si Fri Feb 24 19:10:58 2012 From: miha.vrhovnik at cordia.si (Miha Vrhovnik) Date: Fri, 24 Feb 2012 18:10:58 +0100 Subject: [Dovecot] =?iso-8859-1?q?iPhone_and_custom_imap_folders_problem?= In-Reply-To: <4F47BECA.4020104@3a.pl> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> Message-ID: <20120224181059.66377BF108F@miha> "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >On 24.02.2012 15:47, Charles Marcus wrote: >> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >> wrote: >>> Im using Dovecot 1.0.10 >> Very old - an upgrade would likeley fix your problems... >A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >Only inbox is synced automatically. > >-- >Adam Szpakowski So the solution is simple go bug Apple. Oh yeah. they say if something isn't implemented on Apple platform or it doesn't work, then you don't need it, because their devs know what's best for their users. Regards, M From julioarr at fisica.uh.cu Fri Feb 24 19:44:37 2012 From: julioarr at fisica.uh.cu (julioarr at fisica.uh.cu) Date: Fri, 24 Feb 2012 12:44:37 -0500 Subject: [Dovecot] remove messages once downloaded Message-ID: I need some help with the dovecot configuration. I want to remove downloaded messages from Mail Server once the messages have been successfully downloaded by pop3-clients, even when the clients have been configured to save copy of messages in the Server. thanks, Julio From trashcan at odo.in-berlin.de Fri Feb 24 17:27:56 2012 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 24 Feb 2012 16:27:56 +0100 Subject: [Dovecot] dsync 2.0.x versus 2.1.x field report Message-ID: <47ce619d5887f172ef96c56b24b19b0e@mx2.enfer-du-nord.net> Hi -- I used to run two mailservers with identical MX priorities (poor man's fail-over approach) with Maildir mail folders accessed by Dovecot. Synchronization has been achieved using unison, mirroring all files except dovecot index files. Once in a while some oddities could be observed, like reappearing of deleted mails, duplicates and such alike. But the all-over performance was quite good. Then I wanted to migrate to mdbox, and I became interested in dsync. I did start with 2.0.16 a couple of month ago, and recently upgraded to 2.0.18. My experience with the 2.0.x dsync version is as follows (synchronization time: every minute): -- As long as mail is not stored in corresponding mailboxes at both servers at the same time, all mailboxes become synchronized perfectly well, and no oddites like reappearing of deleted mails or generation of duplicates can be observed. -- If mail is stored into corresponding mailboxes at both servers simultaneously in between two synching runs, it gets worrying. -- If I do run dsync without '-f' those two mailboxes end up in becoming asynchronous. -- If I do run dsync with '-f' both mailboxes will become synchronized let's say in 95% of all synchronization runs. BUT in 5%, I do experience asynchronicity. Here a 'doveadm force-resync' sometimes works, but not always, what worries me a lot. -- I can see the missing mail in the mdbox storage file using UNIX' little helpers but not with the help of dovecot tools. (Or I was unable to use them properly.) -- Modifying MX priorities in a way, that I do now have a primary and backup mailserver scenario, very much reduced the risk of losing legitimate mail. -- But sometimes I do lose some syslog mail being produced at the same time. Therefore I modified runtimes in way to omit synchronicity. Ok, I could have lived with that, but I was anxious to learn, if 2.1.x dsync would perform better ;-) Here my observations sofar (synchronization time: every minute): -- No losses sofar using 'doveadm sync' with '-f' -- A *lot* of reappearing deleted mails or mails moved from one mailbox to the other. -- Quite often I am unable to delete those mails using client functionality (Mail.app, Roundcube). -- In those cases one needs to stop synchronization, run 'doveadm force-resync', delete all reappearing mail (quite a lot sometimes), empty 'Deleted' folders, and finally run 'doveadm purge'. -- Running of 'doveadm sync' without '-f' doesn't improve the situation. Still I do have to struggle with reappearing mails, and sometimes with appearing duplicates. -- I do have a test setup running for 24 hours now, where I do generate test mails in a 6 minutes interval at both servers. Mails become synchronized perfectly well, irrespectively if '-f' is set or not. This is a major improvement to 2.0x. dsync (sofar). -- But running the synchronization without '-f' still shows a *lot* of reappering mail, though. I do have the feeling that dsync became very much improved with regard to missing mail in synchronization, good. But those reappearing mails make me think about switching back to 2.0.x dsync and live with the drawback of preventing mail to arrive synchronously at both servers. Is someone out there experiencing a similar behaviour with dsync 2.1.x regarding reappearing, thus indelible mail? Regards, Michael From CMarcus at Media-Brokers.com Fri Feb 24 21:29:26 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 24 Feb 2012 14:29:26 -0500 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <4F47E516.3090101@Media-Brokers.com> On 2012-02-24 10:07 AM, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. This is quickly becoming an FAQ... If this is linux, and these attacks are from the *same* IP address(es) (sounds like that is the case), then your best option is something like fail2ban. If, however, they are from bots that change IP address each time, fail2ban won't be effective... -- Best regards, Charles From atoth at atoth.sote.hu Fri Feb 24 21:35:46 2012 From: atoth at atoth.sote.hu (=?utf-8?B?IlTDs3RoIEF0dGlsYSI=?=) Date: Fri, 24 Feb 2012 20:35:46 +0100 Subject: [Dovecot] How to achieve proper privilege separation? In-Reply-To: References: <4620e1233657d1ee997f1bf6f5f4ec49.squirrel@atoth.sote.hu> <1329984912.11500.277.camel@innu> <7b95d5d0f96ecde13953a8e1fe847c3a.squirrel@atoth.sote.hu> <4088D5C0-111F-4781-B74A-103117D374E6@iki.fi> Message-ID: It was probably caused by a bug already corrected. And not by dovecot. commit 4fd554e3a097b22c5049fcdc423897477deff5ef Author: Brad Spengler Date: Mon Feb 20 09:17:57 2012 -0500 Fix wrong logic on capability checks for switching roles, broke policies Thanks to Richard Kojedzinszky for reporting I'll test it with an updated kernel containing the fix. Thx: Dw. -- dr T?th Attila, Radiol?gus, 06-20-825-8057 Attila Toth MD, Radiologist, +36-20-825-8057 2012.Febru?r 24.(P) 00:28 id?pontban Timo Sirainen ezt ?rta: > On 24.2.2012, at 1.26, Timo Sirainen wrote: > >> On 23.2.2012, at 21.56, T?th Attila wrote: >> >>> In the mean time I've upgraded to 2.1. >>> I've enabled debug logging and logged in. >>> >>> I suspect that hardening features can be blamed for my problem. After >>> booting a previous kernel the behavior was reverted. >> >> OK. > > BTW. It would be interesting to know if the attached patch causes it to > fail with the hardening patches. > From dhenderson at mediatemple.net Sat Feb 25 00:49:50 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 14:49:50 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 Message-ID: Greetings, This email is both a request for assistance/help and a heads-up. [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. dovecot: imap-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Disconnected (no auth attempts): rip=a.a.a.a, lip=b.b.b.b, TLS handshaking: Disconnected dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: pop3-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/pop3-login(main+0x49) [0x401b99] dovecot: master: Error: service(pop3-login): child 27603 killed with signal 6 (core not dumped - add -D parameter to service pop3-login { executable } dovecot: master: Error: service(pop3-login): command startup failed, throttling dovecot: imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument dovecot: imap-login: Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baa0] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543baf6] -> /usr/lib64/dovecot/libdovecot.so.0 [0x3cb543afb3] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handle_add+0x118) [0x3cb5447708] -> /usr/lib64/dovecot/libdovecot.so.0(io_add+0xa5) [0x3cb5446e15] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_init_finish+0x1c6) [0x3cb54355a6] -> /usr/lib64/dovecot/libdovecot-login.so.0(main+0x136) [0x37a000bdf6] -> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3cb301d994] -> dovecot/imap-login(main+0x39) [0x402069] dovecot: master: Error: service(imap-login): child 27604 killed with signal 6 (core not dumped - add -D parameter to service imap-login { executable } Once this patch was removed, everything started working again. Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? We haven't dug deeper yet, but the error is being thrown from the method io_loop_handle_add in ioloop-epoll.c http://hg.dovecot.org/dovecot-2.0/file/aa8dfa085a99/src/lib/ioloop-epoll.c Thanks Doug From tss at iki.fi Sat Feb 25 02:27:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:27:22 +0200 Subject: [Dovecot] How to limit max number of connections for ip address In-Reply-To: <4F47A7A8.4090009@uady.mx> References: <4F46BECA.3020208@uady.mx> <064BAE74-EF29-4225-9B6A-68E3EB4708ED@iki.fi> <4F47A7A8.4090009@uady.mx> Message-ID: <3A91726D-8D74-4AB2-ACAA-101376543133@iki.fi> On 24.2.2012, at 17.07, Wilberth Perez wrote: > My question was because constantly we received brute force attack from > some of ip address which uses pop3 service to affect dovecot's login > proccess. > > For example: > > dovecot: Error: pipe() failed: Too many open files > dovecot: Error: Temporary failure in creating login processes, slowing > down for now This is your main problem. Increase ulimit -n so Dovecot won't stop creating login processes (or alternatively decrease the max number of login processes). Newer Dovecot versions warn at startup if the fd limit is too low. From tss at iki.fi Sat Feb 25 02:39:15 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:15 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On 25.2.2012, at 0.49, Doug Henderson wrote: > [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. > > After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument .. > Once this patch was removed, everything started working again. > > Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). From tss at iki.fi Sat Feb 25 02:39:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:39:41 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Not possible. From tss at iki.fi Sat Feb 25 02:47:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:47:05 +0200 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <4F47BA30.2080704@krausam.de> References: <4F47BA30.2080704@krausam.de> Message-ID: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> On 24.2.2012, at 18.26, Micha Krause wrote: > I want to deliver mails directly to subfolders, based on address extensions. .. > I was able to reproduce this with exim and dovecot-lda: > > command = /usr/lib/dovecot/dovecot-lda -e -m "INBOX.${substr_1:$local_part_suffix}" > > > But I was wondering if I could do this within dovecot, which would enable me to use > LMTP instead of dovecot-lda. lmtp_save_to_detail_mailbox = yes > Which brings me to my second question: > > Which method is "better" for dovecot, lda or LMTP? LMTP is often easier to setup and faster. From tss at iki.fi Sat Feb 25 02:50:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 02:50:34 +0200 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: On 24.2.2012, at 17.20, Rafa? Radecki wrote: > I am currently gathering information about migration of Maildir/ > structures from Courier to Dovecot. > What do you think about http://wiki.dovecot.org/Migration/Courier ? > For each courier Maildir account I want to use > http://www.dovecot.org/tools/courier-dovecot-migrate.pl to produce > dovecot-uidlist file > mv courierimapsubscribed subscriptions > sed sed -i 's/INBOX\.//' subscriptions The migration script migrates subscriptions, so those shouldn't be necessary. > rm -r courierimaphieracl > rm courierimapacl You don't mind losing any ACLs? > I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). From msidoric at mac.com Sat Feb 25 03:07:27 2012 From: msidoric at mac.com (Michael Sidoric) Date: Fri, 24 Feb 2012 19:07:27 -0600 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: > > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. timo is quite correct -- consider imap instead, that's why the protocol exists. From ml at smtp.fakessh.eu Sat Feb 25 04:42:27 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 03:42:27 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question Message-ID: <1330137747.13977.3.camel@localhost> hi guys hi master of "Fu" hi Timo I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I encounter fatal errors. I created it myself rpms available here http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm I can not run dovecot with this config file like this http://ns.fakessh.eu/rpms/dovecot-2.1.1-1.centme.el5.src.rpm i run this config ks37777 ~]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1 vsz_limit = 64 M } service imap { client_limit = 1024 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From troy at surety-networks.com Sat Feb 25 04:47:59 2012 From: troy at surety-networks.com (Troy Shafer) Date: Fri, 24 Feb 2012 21:47:59 -0500 Subject: [Dovecot] Apple od/ad authentication problem In-Reply-To: <20120224151158.GB16307@PC211.ikt.de> References: <7CA2EE5C-7A77-4E5A-9F96-BCEA5BDDA164@surety-networks.com> <20120224151158.GB16307@PC211.ikt.de> Message-ID: <02C084C7-5EE0-4CE8-A60E-C40D880EF3E3@surety-networks.com> The correct answer was in the log file. Behind a NAT you have to set login_trusted_networks = internal/24 externalIP - Troy On Feb 24, 2012, at 10:11 AM, Dennis Guhl wrote: > On Fri, Feb 24, 2012 at 08:55:57AM -0500, Troy Shafer wrote: >> Hello all. Im pretty new to dovecot and postfix world. I'm completely stumped. >> >> OS X 10.7.3, behind a Cisco asa NAT. > > Deactivate the SMTP improvements your ASA helpfully provide. > > Dennis > > [..] From tss at iki.fi Sat Feb 25 04:49:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 04:49:07 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330137747.13977.3.camel@localhost> References: <1330137747.13977.3.camel@localhost> Message-ID: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> On 25.2.2012, at 4.42, ml wrote: > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > encounter fatal errors. And what are the error messages? From ml at smtp.fakessh.eu Sat Feb 25 05:32:50 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 04:32:50 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> Message-ID: <1330140770.15576.1.camel@localhost> Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 4.42, ml wrote: > > > I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > > encounter fatal errors. > > And what are the error messages? > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child 15407 killed with signal 11 (core dumps disabled) my current config ks37777 i386]# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items;Deleted Messages autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 128 vsz_limit = 64 M } service imap { client_limit = 1 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 128 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 128 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From tss at iki.fi Sat Feb 25 05:45:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:45:39 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330140770.15576.1.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> Message-ID: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> On 25.2.2012, at 5.32, ml wrote: > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : >> On 25.2.2012, at 4.42, ml wrote: >> >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I >>> encounter fatal errors. >> >> And what are the error messages? >> > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > 15407 killed with signal 11 (core dumps disabled) Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html From tss at iki.fi Sat Feb 25 05:54:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 05:54:22 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 Message-ID: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). From ml at smtp.fakessh.eu Sat Feb 25 06:51:06 2012 From: ml at smtp.fakessh.eu (ml) Date: Sat, 25 Feb 2012 05:51:06 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> Message-ID: <1330145466.15576.4.camel@localhost> Le samedi 25 f?vrier 2012 ? 05:45 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 5.32, ml wrote: > > > Le samedi 25 f?vrier 2012 ? 04:49 +0200, Timo Sirainen a ?crit : > >> On 25.2.2012, at 4.42, ml wrote: > >> > >>> I am having problems to migrate to dovecot dovecot 2.0.18 to 2.1.1 I > >>> encounter fatal errors. > >> > >> And what are the error messages? > >> > > 2012-02-25 04:29:44imap(fakessh): Fatal: master: service(imap): child > > 15407 killed with signal 11 (core dumps disabled) > > Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > I have not succeeded well I got a son of fakessh I attached with gdb-p but bt full no me back stack my poor english -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From tss at iki.fi Sat Feb 25 07:17:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:17:46 +0200 Subject: [Dovecot] Proxying improvements in v2.1.2 In-Reply-To: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> References: <77FC9C82-EAB0-4A0A-87BD-F6BEC1D0E342@iki.fi> Message-ID: <8AA21A89-D80B-4B82-A9A8-9E14A3A1F385@iki.fi> On 25.2.2012, at 5.54, Timo Sirainen wrote: > I just committed a couple of features that will make life easier for some types of proxying setups: Oh, and: 4. Added proxy_always=y passdb field. When used with proxy_maybe=y you can create a setup where the connections are proxied to either * when user's host matches locally, director assigns a local backend's host for the connection * when user's host doesn't match locally, Dovecot connects to that host So basically you can have a bunch of Dovecot cluster installations behind different load balancer IPs, and have all of the Dovecot proxies redirect the connection to the proper cluster's load balancer, or if it's for a local cluster then redirect to one of the backends. 5. Currently in v2.2 only, could be backported if needed: Added TTL for the proxy connections, so in case two clusters both think the other one should be handling the connection they won't end up creating thousands of connections to each others until finally failing when login process limits are reached. The hard coded TTL is 5, should be more than enough. From tss at iki.fi Sat Feb 25 07:18:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 07:18:54 +0200 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <1330145466.15576.4.camel@localhost> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> Message-ID: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> On 25.2.2012, at 6.51, ml wrote: >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html >> > > I have not succeeded well I got a son of fakessh I attached with gdb-p > but bt full no me back stack Did you try without antispam plugin? My guess is that is the reason it crashes. Or if it still crashes, try disabling all plugins. From dmiller at amfes.com Sat Feb 25 08:09:09 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Fri, 24 Feb 2012 22:09:09 -0800 Subject: [Dovecot] Solr not indexing Message-ID: What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" doveadm -D fts rescan -u dmiller at amfes.com doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib01_acl_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/local/lib/dovecot/doveadm doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib10_doveadm_acl_plugin.so doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/local/lib/dovecot/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/local/lib/dovecot/doveadm/lib20_doveadm_fts_plugin.so doveadm(dmiller at amfes.com): Debug: Added userdb setting: mail=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: Added userdb setting: plugin/acl_groups='allshared' doveadm(dmiller at amfes.com): Debug: Effective uid=5000, gid=8, home=/var/mail/amfes.com/dmiller doveadm(dmiller at amfes.com): Debug: Namespace : type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:/var/mail/amfes.com/dmiller/mdbox doveadm(dmiller at amfes.com): Debug: fs: root=/var/mail/amfes.com/dmiller/mdbox, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: Namespace : Using permissions from /var/mail/amfes.com/dmiller/mdbox: mode=0700 gid=-1 doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 1 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls doveadm(dmiller at amfes.com): Debug: Namespace : type=shared, prefix=shared/%n/, sep=/, inbox=no, hidden=no, list=children, subscriptions=no location=mdbox:/var/mail/%d/%n/mdbox doveadm(dmiller at amfes.com): Debug: shared: root=/usr/local/var/run/dovecot, index=, control=, inbox=, alt= doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/' doveadm(dmiller at amfes.com): Debug: acl: initializing backend with data: vfile:/usr/local/etc/dovecot/global-acls:cache_secs=300 doveadm(dmiller at amfes.com): Debug: acl: acl username = dmiller at amfes.com doveadm(dmiller at amfes.com): Debug: acl: owner = 0 doveadm(dmiller at amfes.com): Debug: acl vfile: Global ACL directory: /usr/local/etc/dovecot/global-acls -- Daniel From tss at iki.fi Sat Feb 25 08:24:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:24:20 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: Message-ID: On 25.2.2012, at 8.09, Daniel L. Miller wrote: > What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" That's intentional. "shared/%n/" namespace has no mails. Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. From dhenderson at mediatemple.net Sat Feb 25 08:32:12 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Fri, 24 Feb 2012 22:32:12 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > On 25.2.2012, at 0.49, Doug Henderson wrote: > >> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >> >> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. > > I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 see: https://rhn.redhat.com/errata/RHSA-2012-0150.html "A flaw was found in the way the Linux kernel's Event Poll (epoll) subsystem handled large, nested epoll structures. A local, unprivileged user could use this flaw to cause a denial of service. (CVE-2011-1083, Moderate)" Our automated patching (ksplice) installed it at around 10am PST today. Other distributions may vary. > >> dovecot: pop3-login: Panic: epoll_ctl(add, 6) failed: Invalid argument > .. >> Once this patch was removed, everything started working again. >> >> Is it possible that dovecot is trying to re-add already-added connections to the polling list - which this specific 'patch' prevents? > > It shouldn't be possible .. EPOLL_CTL_ADD is done only once, EPOLL_CTL_MOD is done afterwards. And if the same fd is attempted to be added/modded twice, Dovecot should assert-crash first in ioloop_iolist_add(). > We haven't spent enough time investigating to be sure, but epoll_ctl was certainly "in the thick of it". The only outward evidence (in logs, even with debug turned on) that there was anything wrong with Dovecot at all was the Panic shown for that method. Dovecot may have been an innocent bystander in this case - but something was causing it to fail on inbound IMAP/POP3 connections, and when the patch was removed everything started working again. From tss at iki.fi Sat Feb 25 08:49:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 25 Feb 2012 08:49:10 +0200 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <876F3440-67CC-47B2-AA33-96A067A6A22D@iki.fi> On 25.2.2012, at 8.32, Doug Henderson wrote: >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail servers, most/all IMAP/POP3 connections began experiencing time-outs trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html Yes, but CVE-2011-1083 shows it was reported almost a year ago, so I'd think it was fixed in upstream kernel a long time ago. I'm running in my desktop about 2 months old kernel (from git) and I don't see any problems with it. But yeah, maybe Redhat's patches did it differently than upstream kernel and it broke because of that.. From mstevens at imt-systems.com Sat Feb 25 13:15:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sat, 25 Feb 2012 12:15:49 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: Message-ID: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> On 25.02.2012 07:32, Doug Henderson wrote: > On Feb 24, 2012, at 4:39 PM, Timo Sirainen wrote: > >> On 25.2.2012, at 0.49, Doug Henderson wrote: >> >>> [8irgehuq] CVE-2011-1083: Algorithmic denial of service in epoll. >>> >>> After ksplice automatically installed the above patch on our mail >>> servers, most/all IMAP/POP3 connections began experiencing time-outs >>> trying to connect, or extreme timeouts in the auth procedure. >> >> I'd guess this patch is already in new Linux kernel versions, so >> other people should have seen any problems caused by it? > > Actually, it was only released a couple of days ago (2/21) by redhat > for EL 5.8 > see: https://rhn.redhat.com/errata/RHSA-2012-0150.html > > "A flaw was found in the way the Linux kernel's Event Poll (epoll) > subsystem handled large, nested epoll structures. A local, > unprivileged > user could use this flaw to cause a denial of service. > (CVE-2011-1083, > Moderate)" > > Our automated patching (ksplice) installed it at around 10am PST > today. > > Other distributions may vary. Try it without ksplice. (yum update and reboot) Which kernel is running exactly? Best regards, Morten From busseniu at in.tum.de Sat Feb 25 21:16:20 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Sat, 25 Feb 2012 20:16:20 +0100 Subject: [Dovecot] Maildir migration: Courier-imap 4.4.3 to Dovecot 1.2. In-Reply-To: References: Message-ID: <4F493384.9010703@in.tum.de> On 02/25/2012 01:50 AM, Timo Sirainen wrote: > On 24.2.2012, at 17.20, Rafa? Radecki wrote: >> I want also to leave default pop3_uidl_format (%08Xu%08Xv) in dovecot.conf > > That might cause your POP3 clients to redownload mails (or maybe not, I can't remember exactly how the script works). > The migration script takes the UIDLs from "courierpop3dsizelist" and writes them into "dovecot-uidlist". Thus Dovecot will use the same UIDL as Courier for the converted mails. %08Xu%08Xv will only be used for new mails. Cheers, Christoph From dmiller at amfes.com Sun Feb 26 03:42:22 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 25 Feb 2012 17:42:22 -0800 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 2/24/2012 10:24 PM, Timo Sirainen wrote: > On 25.2.2012, at 8.09, Daniel L. Miller wrote: > >> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" > That's intentional. "shared/%n/" namespace has no mails. > > Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? -- Daniel From tss at iki.fi Sun Feb 26 04:26:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 04:26:07 +0200 Subject: [Dovecot] Solr not indexing In-Reply-To: References: <4F487B05.5000002@amfes.com> Message-ID: On 26.2.2012, at 3.42, Daniel L. Miller wrote: > On 2/24/2012 10:24 PM, Timo Sirainen wrote: >> On 25.2.2012, at 8.09, Daniel L. Miller wrote: >> >>> What did I do wrong to make this happen? "doveadm(dmiller at amfes.com): Debug: fts: Indexes disabled for namespace 'shared/%n/'" >> That's intentional. "shared/%n/" namespace has no mails. >> >> Anyway, looks like "doveadm fts rescan" works only for fts-lucene currently.. >> > Solr is currently not indexing at all. I've tried a few different settings and commands - nothing is re-generating Solr indexes. I have deleted the Solr index completely - it gets generated empty, and is not filling. Does Dovecot think everything is indexed so nothing needs to be added? It SEEMS new messages, after a search, WILL be indexed - but not my existing mail store. How can I reset the internal Dovecot fts log? The cached "last fts indexed message" gets dropped with: doveadm force-resync INBOX From dhenderson at mediatemple.net Sun Feb 26 04:55:59 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sat, 25 Feb 2012 18:55:59 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> Message-ID: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: > > Try it without ksplice. (yum update and reboot) I don't know if I'll be permitted to do that in a production environment - possibly a test one. I'll need to get some opinions from our Ops people as to if/how they might want to go about it. > Which kernel is running exactly? 2.6.18-274.3.1.el5 > Best regards, > > Morten From f.bonnet at esiee.fr Sun Feb 26 09:22:48 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Sun, 26 Feb 2012 08:22:48 +0100 Subject: [Dovecot] FreeBSD maintainer ? Message-ID: <4F49DDC8.2060506@esiee.fr> Hello Does the FreeBSD Dovecot's port maintainer read this mailing-list ? Thanks From duihi77 at gmail.com Sun Feb 26 10:54:56 2012 From: duihi77 at gmail.com (Duane Hill) Date: Sun, 26 Feb 2012 08:54:56 +0000 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <1527928288.20120226085456@gmail.com> On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr confabulated: > Hello > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > Thanks Have you tried sending message to the maintainer? The email address can be found by: cd /usr/ports/mail/dovecot && make maintainer or cd /usr/ports/mail/dovecot2 && make maintainer There appears to be two different email addresses between maintainers for version 1.2.x and 2.0.x. -- If at first you don't succeed... ...so much for skydiving. From tss at iki.fi Sun Feb 26 12:30:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 12:30:43 +0200 Subject: [Dovecot] IMAP NOTIFY extension Message-ID: <7075A103-8131-491F-9496-C632418BAF67@iki.fi> http://hg.dovecot.org/dovecot-2.2-notify/ contains the initial implementation of IMAP NOTIFY extension. It's less than half-working, but the difficult part of it is done I think. If I give it a command like: x notify set (personal (messagenew messageexpunge flagchange)) and then in another session I copy a message to another mailbox, it sends: * STATUS "foobar" (MESSAGES 77 UIDNEXT 78 UIDVALIDITY 1329431281 UNSEEN 75) This requires mailbox_list_index=yes to work. It works by following the dovecot.list.index.log changes, so it's going to be pretty efficient. Some TODO that I've so far noticed: - NOTIFY is handled currently only for non-selected mailboxes. (handling the new features for selected mailbox should be simple) - initial STATUS sending isn't done with STATUS option - only MessageNew, MessageExpunge and FlagChange notifications are sent currently - MailboxName handling probably needs to be able to get "old" and "new" views, figure out what changed in them and send those changes (dsync has similar code) - mailbox list indexes seem to be adding too much stuff, which results in: - STATUS duplicates, but there should already be code to avoid these so I'm not sure what's going on - ACLs and shared mailboxes don't work right - add 0.5s delay before notifying IMAP code, so multiple changes can be bundled into one From mstevens at imt-systems.com Sun Feb 26 12:44:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Sun, 26 Feb 2012 11:44:18 +0100 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: On 26.02.2012 03:55, Doug Henderson wrote: > On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >> >> Try it without ksplice. (yum update and reboot) > > I don't know if I'll be permitted to do that in a production > environment - possibly a test one. > I'll need to get some opinions from our Ops people as to if/how they > might want to go about it. > >> Which kernel is running exactly? > > 2.6.18-274.3.1.el5 That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. Best regards, Morten From lists at wildgooses.com Sun Feb 26 13:52:37 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 11:52:37 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4A1D05.2070402@wildgooses.com> On 25/02/2012 00:39, Timo Sirainen wrote: > On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: > >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > Not possible. > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. Is it feasible to implement both of these solutions using the current plugin architecture? I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Cheers Ed W From jerry at seibercom.net Sun Feb 26 13:56:42 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 06:56:42 -0500 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <1527928288.20120226085456@gmail.com> References: <4F49DDC8.2060506@esiee.fr> <1527928288.20120226085456@gmail.com> Message-ID: <20120226065642.24deacd8@scorpio> On Sun, 26 Feb 2012 08:54:56 +0000 Duane Hill articulated: > On Sunday, February 26, 2012 at 07:22:48 UTC, f.bonnet at esiee.fr > confabulated: > > > Hello > > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > > > Thanks > > Have you tried sending message to the maintainer? The email address > can be found by: > > cd /usr/ports/mail/dovecot && make maintainer > > or > > cd /usr/ports/mail/dovecot2 && make maintainer > > There appears to be two different email addresses between maintainers > for version 1.2.x and 2.0.x. That is because there are two different maintainers: dovecot-1.2.17 == yds at CoolRat.org dovecot-2.0.16_1 == bra at fsn.hu If you are having a problem with the port, I would suggest that you post on the FreeBSD Ports forum: freebsd-ports at freebsd.org and CC the maintainer of the port in question. It works quite effectively. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From tss at iki.fi Sun Feb 26 14:31:40 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 14:31:40 +0200 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4A1D05.2070402@wildgooses.com> References: <4F4A1D05.2070402@wildgooses.com> Message-ID: On 26.2.2012, at 13.52, Ed W wrote: > On 25/02/2012 00:39, Timo Sirainen wrote: >> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >> >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have been >>> configured to save copy of messages in the Server. >> Not possible. >> > > If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) > > In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. What does the notification do? > Is it feasible to implement both of these solutions using the current plugin architecture? Yes. > I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? > > All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. From lists at wildgooses.com Sun Feb 26 15:23:59 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 26 Feb 2012 13:23:59 +0000 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: <4F4A1D05.2070402@wildgooses.com> Message-ID: <4F4A326F.8000803@wildgooses.com> On 26/02/2012 12:31, Timo Sirainen wrote: > On 26.2.2012, at 13.52, Ed W wrote: > >> On 25/02/2012 00:39, Timo Sirainen wrote: >>> On 24.2.2012, at 19.44, julioarr at fisica.uh.cu wrote: >>> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. >>> Not possible. >>> >> If you were thinking about longer term TODOs then I have a similar problem (just adding a "me too"...) >> >> In my industry, competing solutions offer a kind of server side "been downloaded" notification when customers have actually downloaded (ie "read" via POP) the message. The customers are all on the far side of expensive satellite links, so this serves as an inexpensive proxy for "message read" notifications. > What does the notification do? Sends another email... (you know like the annoying "message read" indicators that lots of mail readers support)... (Several of our competitors have implemented these solutions very badly and we get mail loops and other nasties...) >> I think our competition implement such features because they are Exchange based and I believe you can write server side hooks in various scripting languages quite easily (I personally don't like the idea, but someone obviously did it once and it rolled from there...) - this obviously harking to the "is it feasible to imagine some higher level hook solution for simpler plugin creation" suggestion from a few days ago? >> >> All these "do something when it's accessed" or "do something when it's deleted" problems all feel kind of related to me (ie we need some hook which runs on a per message basis). Perhaps someone smarter than me can think of a better way to unify them? > Dovecot has a "notify" plugin that makes things like this pretty easy to implement, but it still needs C coding. Thanks - it's off my radar for a while due to other pressures, but the hint is appreciated and I will look into it in the future - many thanks! Ed W From fxcode at me.com Sun Feb 26 15:44:05 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 13:44:05 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted Message-ID: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Hi, I am relatively new to linux, and have set up my first lamp server around a month ago. The server is housed in a data centre a few miles from my house, and is working as a web server (with apache, tomcat, mysql) just fine. I have now been trying for a week to get Postfix and Dovecot working with postfixadmin and mysql. (below I have replaced my test domain name with [***domain_name***]) The story so far; I can send 'welcome' emails from within postfix admin to an external email address... and they are received by the mail client on my computer at home no problem. I can set up alias email addresses within postfix admin, such as webmaster@[***domain_name***], and emails sent to this address are redirected to the proper address as expected. But - if I set up a standard mailbox within postfix admin - test@[***domain_name***] , and email it from my home system, nothing appear in postfixadmin (after a fetch request) or the mysql database. Here is the fail point in my mail.log file ; Feb 26 12:50:27 quantum dovecot: deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with euid=8(mail), gid=8(mail), egid=8(mail): Operation not permitted I don't know what the setguid(5000) is, or where it is being set, or what it should be! I have been changing permissions with the 'socket listen' section of my /etc/dovecot/dovecot.conf file, and when I restart dovecot it creates the auth-master file with the permissions as expected; socket listen { master { path = /var/run/dovecot/auth-master mode = 0777 user = vmail group = mail } client { path = /var/spool/postfix/private/auth mode = 432 #user = postfix #group = postfix } } my mail folders should be located at /var/vmail, but the folder is empty. I have chmod the permissions to be as follows; me at myserver: ls -ld /var/vmail drwxrwxrwx 2 vmail mail 4096 2012-02-11 08:40 /var/vmail Can anyone spot where this is going wrong? Thanks in advance for any help, it is very very much appreciated!!! My hosts file is as follows; 127.0.0.1 localhost [**server_ip***] quantum.[***domain_name***] quantum # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ...and 'dovecot -n' replies as follows; me at myserver: dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.4 LTS ext4 log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_location: maildir:/var/vmail/%d/%n mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 lda: log_path: /var/log/dovecot-deliver.log auth_socket_path: /var/run/dovecot/auth-master postmaster_address: fxcode at me.com mail_plugins: sieve auth default: mechanisms: plain login passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: static args: uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 master: path: /var/run/dovecot/auth-master mode: 511 user: vmail group: mail From sk71 at gmx.de Sun Feb 26 17:54:22 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 16:54:22 +0100 Subject: [Dovecot] Deny create folders Message-ID: <4F4A55AE.1030402@gmx.de> Hi all, is it possible to configure dovecot in that way that special folders do not create automatically by some imap clients? i.e: outlook or thunderbird creates a folder junk-mail i would like that such folders are not created automatically or by users Best regards Stefan From tss at iki.fi Sun Feb 26 18:02:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 26 Feb 2012 18:02:47 +0200 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: On 26.2.2012, at 17.54, S. Kremer wrote: > is it possible to configure dovecot in that way that special folders do > not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by users You could maybe create a global ACL that denies the access to create them. The clients might not be happy about it though. It's not a very good idea to go doing things like this I think. From jerry at seibercom.net Sun Feb 26 18:41:25 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 11:41:25 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4A55AE.1030402@gmx.de> References: <4F4A55AE.1030402@gmx.de> Message-ID: <20120226114125.59e0d407@scorpio> On Sun, 26 Feb 2012 16:54:22 +0100 S. Kremer articulated: > Hi all, > > is it possible to configure dovecot in that way that special folders > do not create automatically by some imap clients? > > i.e: > outlook or thunderbird creates a folder junk-mail > > i would like that such folders are not created automatically or by > users Exactly what problem are you attempting to alleviate? In the short term you might seriously upset some of your clients and in the long term cause a stability problem. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From fxcode at me.com Sun Feb 26 22:26:20 2012 From: fxcode at me.com (fxCode) Date: Sun, 26 Feb 2012 20:26:20 +0000 Subject: [Dovecot] deliver(test@[***domain_name***]): Fatal: setgid(5000) failed with ... Operation not permitted In-Reply-To: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> References: <5DEDBE1C-8215-4157-8CF0-412499BCED32@me.com> Message-ID: <11EBA09F-36C6-4319-874A-44207A6ED435@me.com> Hi, I have fixed this fault - the following line in my /etc/dovecot/dovecot.conf file was setting the setgid to 5000, and the user id to 5000 also auth default { ... userdb static { args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } ... } I changed these two values to 8 ( the id of mail on my system) and emails are now being delivered to the proper folders thanks Christy From sk71 at gmx.de Sun Feb 26 23:31:35 2012 From: sk71 at gmx.de (S. Kremer) Date: Sun, 26 Feb 2012 22:31:35 +0100 Subject: [Dovecot] Deny create folders In-Reply-To: <20120226114125.59e0d407@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> Message-ID: <4F4AA4B7.4030909@gmx.de> Ok, thanks for answer. Every user's mailbox has a directory called Spam and Ham. I scan this two folders by a script to learn my spamassassin database. This script does not scan folders like spam, ham, junk-email etc. But some users have such folders in their mailbox so spamassassin does not learn from these folders. My aim is to force users to use the folders Spam and Ham and not other folders for spam or ham messages. Am 26.02.2012 17:41, schrieb Jerry: > On Sun, 26 Feb 2012 16:54:22 +0100 > S. Kremer articulated: > >> Hi all, >> >> is it possible to configure dovecot in that way that special folders >> do not create automatically by some imap clients? >> >> i.e: >> outlook or thunderbird creates a folder junk-mail >> >> i would like that such folders are not created automatically or by >> users > > Exactly what problem are you attempting to alleviate? In the short term > you might seriously upset some of your clients and in the long term > cause a stability problem. > Greetings Stefan From jerry at seibercom.net Mon Feb 27 00:28:27 2012 From: jerry at seibercom.net (Jerry) Date: Sun, 26 Feb 2012 17:28:27 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: <4F4AA4B7.4030909@gmx.de> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> Message-ID: <20120226172827.112de657@scorpio> On Sun, 26 Feb 2012 22:31:35 +0100 S. Kremer articulated: > Am 26.02.2012 17:41, schrieb Jerry: > > On Sun, 26 Feb 2012 16:54:22 +0100 > > S. Kremer articulated: > > > >> Hi all, > >> > >> is it possible to configure dovecot in that way that special > >> folders do not create automatically by some imap clients? > >> > >> i.e: > >> outlook or thunderbird creates a folder junk-mail > >> > >> i would like that such folders are not created automatically or by > >> users > > > > Exactly what problem are you attempting to alleviate? In the short > > term you might seriously upset some of your clients and in the long > > term cause a stability problem. > Ok, thanks for answer. > > Every user's mailbox has a directory called Spam and Ham. I scan this > two folders by a script to learn my spamassassin database. > This script does not scan folders like spam, ham, junk-email etc. > But some users have such folders in their mailbox so spamassassin does > not learn from these folders. > > My aim is to force users to use the folders Spam and Ham and not other > folders for spam or ham messages. Please don't top post. It makes following a thread a lot harder than it needs to be. Couldn't you just make a soft link between the two folders and save yourself and your clientele a lot of trouble? I have not messed with "spamassassin" in years; however, I thought it was configurable on what it scans. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From ml at smtp.fakessh.eu Mon Feb 27 00:37:35 2012 From: ml at smtp.fakessh.eu (ml) Date: Sun, 26 Feb 2012 23:37:35 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330295855.14524.12.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I'm in front of a problem. despite having to disable the antispam plugin as it appears in dovecot.conf always load so I did not succeed to disable I do not know how can you tell me that I continue my tests without the antispam plugin -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From jd.beaubien at gmail.com Mon Feb 27 01:26:02 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Sun, 26 Feb 2012 18:26:02 -0500 Subject: [Dovecot] testing fts-solr? Message-ID: hi everyone, However, how can I verify that solr is working properly? Background: - I'm trying to setup 2.1 with fts_solr and eventually test maildir/mdbox. - I used the following webpage as reference to setup solr: http://www.roessner-network-solutions.com/2012/02/19/full-text-search-with-solr-and-dovecot-on-ubuntu-10-04/ I'm looking at /var/log/tomcat6/catalina.out file, but see nothing telling me that it's being used (same for /var/log/mail.log, etc.) Any tips? Thanks From ml at smtp.fakessh.eu Mon Feb 27 01:39:58 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 00:39:58 +0100 Subject: [Dovecot] upgrade to dovecot 2.1.1 any question In-Reply-To: <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> References: <1330137747.13977.3.camel@localhost> <82E09A63-1362-4A33-BE55-CDE78BC178EC@iki.fi> <1330140770.15576.1.camel@localhost> <282ABE42-78AC-4350-9609-F22DD76C6B2F@iki.fi> <1330145466.15576.4.camel@localhost> <98CF2A5E-163C-44AC-A23C-60F1E7CDE6F2@iki.fi> Message-ID: <1330299598.14524.13.camel@localhost> Le samedi 25 f?vrier 2012 ? 07:18 +0200, Timo Sirainen a ?crit : > On 25.2.2012, at 6.51, ml wrote: > > >> Try if it starts up without antispam plugin enabled? If not, gdb backtrace would be helpful: http://dovecot.org/bugreport.html > >> > > > > I have not succeeded well I got a son of fakessh I attached with gdb-p > > but bt full no me back stack > > Did you try without antispam plugin? My guess is that is the reason it crashes. > > Or if it still crashes, try disabling all plugins. > I managed dovecot 2.1. 1 works well in removing completely antispam plugin work well and fine -- http://pgp.mit.edu:11371/pks/lookup?op=get&search=0xC2626742 gpg --keyserver pgp.mit.edu --recv-key C2626742 http://urlshort.eu fakessh @ http://gplus.to/sshfake http://gplus.to/sshswilting http://gplus.to/john.swilting https://lists.fakessh.eu/mailman/ This list is moderated by me, but all applications will be accepted provided they receive a note of presentation -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Ceci est une partie de message num?riquement sign?e URL: From maria_arrea at gmx.com Mon Feb 27 02:07:00 2012 From: maria_arrea at gmx.com (Maria Arrea) Date: Mon, 27 Feb 2012 01:07:00 +0100 Subject: [Dovecot] Dovecot altmove questions Message-ID: <20120227000700.134750@gmx.com> Hello We are using Dovecot 2.0.17 with mdbox+zlib and we are evaluating alternative storage. These are the relevant settings mdbox_rotate_interval = 1d mdbox_rotate_size=60m [...] zlib_save_level = 9 # 1..9 zlib_save = gz # or bz2 [...] zlib plugin enabled in dovecot-lda These are my questions: 1? General mdbox question. We are happy usings zlib_save, but we would like to try bzip2 compression. Can we change zlib to bzlib without problems? In other words, can we mix & match gzipped and bzipped messages in the same mdbox file ? 2? With doveadm altmove we can move messages matching a pattern to alternate storage. I have read an older post in this mailing list about passing specific mdbox options to doveadm altmove. Should be a good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d and mdox_rotate_size=60m and an "alternate area" with bzip9, mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area should only be "old" mail (1 year older). 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is taking almost 16 hours to make a full backup. Our main problem is that indexes are backupd hours before mdboxes, and that bring us some inconsistencies. What is the "right" way of backing up for separate indexes and mdboxes? Thank you for your support. Maria From ml at smtp.fakessh.eu Mon Feb 27 03:11:22 2012 From: ml at smtp.fakessh.eu (ml) Date: Mon, 27 Feb 2012 02:11:22 +0100 Subject: [Dovecot] value for client_limit Message-ID: <1330305082.20912.5.camel@localhost> hello guys hello master Timo i build the last release of dovecot on my centos 5 work well and fine but what the correct value for client limit in service auth { client_limit=6000 } and service anvil { client_limit=6000 } my current dovecot -n ks37777 i386]# dovecot -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31.5-xxxx-rt14-ipv6-32 i686 CentOS release 5.7 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_debug = yes mail_location = maildir:~/Maildir mail_max_userip_connections = 30 mail_plugins = " quota trash zlib" managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { autocreate = Trash autocreate2 = Junk autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Junk autosubscribe3 = Sent autosubscribe4 = Drafts deleted_to_trash_folder = Trash plugin = $mail_plugins autocreate managesieve sieve quota quota = maildir:User quota quota_exceeded_message = Quota exceeded, please go to http://www.fakessh.eu/over_quota_help.html for instructions on how to fix this. quota_rule = *:storage=10GB quota_rule2 = Trash:storage=+10% quota_rule3 = Spam:storage=+20% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever trash = /etc/dovecot/dovecot-trash.conf.ext zlib_save = bz2 zlib_save_level = 9 } protocols = sieve imap pop3 service anvil { client_limit = 6000 } service auth { client_limit = 6000 process_limit = 1 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0600 user = postfix } unix_listener auth-master { mode = 0666 } unix_listener auth-userdb { mode = 0666 } vsz_limit = 64 M } service imap-login { client_limit = 0 inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 1024 service_count = 1 vsz_limit = 64 M } service imap { process_limit = 1024 process_min_avail = 0 service_count = 1 vsz_limit = 64 M } service managesieve-login { inet_listener managesieve-login { address = * , [::] port = 2000 } process_limit = 1 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 1 vsz_limit = 64 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = mail } user = dovecot } ssl_ca = From sven at svenhartge.de Mon Feb 27 04:26:11 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 03:26:11 +0100 Subject: [Dovecot] Dovecot altmove questions References: <20120227000700.134750@gmx.com> Message-ID: <78jhe1341pv8@mids.svenhartge.de> Maria Arrea wrote: > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid 5). > We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? Bacula? Excellent! If you put your indexes and mdboxes on a LVM (seperate volume group obviously, because of the different storages both reside on), you can snapshot both volumes at the same time, using an FD-based script, mount them somewhere else (like /backup/data and /backup/index) and then you have (nearly) all the time you want to backup them. And because of the snapshots taken at the same time, no inconsistencies can happen. (OK, not entirely true, since both snapshots are taken with a minimal gap between them, but if you start the backup during off-hours the inconsistencies should be minimal.) If you don't use a LVM but have a storage which is able to do snapshots on its own, you can also use this. If you don't have a LVM and no storage with such functionality, then you are kind of lost, as far as I can see. My additional advise on the long time of full backups: don't do them! Switch to accurate backups and only do incremental and differential backups, replacing the full backups with virtual full ones. This will greatly reduce the load on your mail systems and speed up the backup operation at the same time. But you have to use "accurate" type backups, or your virtual full backup will grow and grow and grow, since Bacula without "accurate" is unable to know about deleted files and will include them in every virtual full backup. Gr??e, Sven. -- Sigmentation fault. Core dumped. From dhenderson at mediatemple.net Mon Feb 27 05:20:36 2012 From: dhenderson at mediatemple.net (Doug Henderson) Date: Sun, 26 Feb 2012 19:20:36 -0800 Subject: [Dovecot] v2.0.13 problems after kernel patch for CVE-2011-1083 applied on Centos 5 In-Reply-To: References: <50b432725fe83ebd00d29dddf18fa77d@imt-systems.com> <3C08373C-6693-46D7-A09D-116B280B9385@mediatemple.net> Message-ID: <775A9BD8-884C-47D5-BF82-2BBDF414BF56@mediatemple.net> On Feb 26, 2012, at 2:44 AM, Morten Stevens wrote: > On 26.02.2012 03:55, Doug Henderson wrote: >> On Feb 25, 2012, at 3:15 AM, Morten Stevens wrote: >>> >>> Try it without ksplice. (yum update and reboot) >> >> I don't know if I'll be permitted to do that in a production >> environment - possibly a test one. >> I'll need to get some opinions from our Ops people as to if/how they >> might want to go about it. >> >>> Which kernel is running exactly? >> >> 2.6.18-274.3.1.el5 > > That is probably the problem. The current RHEL 5.8 kernel is 2.6.18-308.el5. There are many changes between 2.6.18-274 (EL 5.7) and 2.6.18-308 (EL 5.8). So I do not know if it is a good idea to apply ksplice patches between minor 5.x releases. > > Best regards, > > Morten Thanks Morten, We'll install the latest kernel on a test machine tomorrow and see how things go - we'll probably also attempt to reinstall the patch (if appropriate) and see if it still breaks things. Doug From mark at ecs.vuw.ac.nz Mon Feb 27 03:17:32 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 14:17:32 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail Message-ID: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Hi, have dovecot 2.0.13 running fine on a NetBSD box, with lots of clients happily authenticating to it with GSSAPI. I've just updated an ArchLinux box to latest new version which has meant its switched from using heimdal to using mit krb5 and its kmail is now also slightly newer. Thunderbird successfully authenticates to dovecot from this machine but kmail now fails. Kmail reports: Unable to authenticate via GSSAPI. The server ... replied: SASL(-1): generic failure: GSSAPI Error: Invalid token was supplied (Token header is malformed or corrupt) The dovecot log reports: Feb 27 13:42:18 bats dovecot: auth: GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in continued response Any suggestions on how to track down what specifically has changed and how to fix it? cheers mark From bra at fsn.hu Mon Feb 27 09:07:24 2012 From: bra at fsn.hu (Attila Nagy) Date: Mon, 27 Feb 2012 08:07:24 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F49DDC8.2060506@esiee.fr> References: <4F49DDC8.2060506@esiee.fr> Message-ID: <4F4B2BAC.2030400@fsn.hu> On 02/26/12 08:22, Frank Bonnet wrote: > > Does the FreeBSD Dovecot's port maintainer read this mailing-list ? > If you read it, you may know the answer (depending on which port do you use). From f.bonnet at esiee.fr Mon Feb 27 09:13:44 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Mon, 27 Feb 2012 08:13:44 +0100 Subject: [Dovecot] FreeBSD maintainer ? In-Reply-To: <4F4B2BAC.2030400@fsn.hu> References: <4F49DDC8.2060506@esiee.fr> <4F4B2BAC.2030400@fsn.hu> Message-ID: <4F4B2D28.5010600@esiee.fr> hello I'm wondering when you will port 2.1.1 version ? ;-) Thanks Le 27/02/2012 08:07, Attila Nagy a ?crit : > On 02/26/12 08:22, Frank Bonnet wrote: >> >> Does the FreeBSD Dovecot's port maintainer read this mailing-list ? >> > If you read it, you may know the answer (depending on which port do > you use). From busseniu at in.tum.de Mon Feb 27 09:23:14 2012 From: busseniu at in.tum.de (=?UTF-8?B?Q2hyaXN0b3BoIEJ1w59lbml1cw==?=) Date: Mon, 27 Feb 2012 08:23:14 +0100 Subject: [Dovecot] [PATCH] Pop3 order in courier migration script Message-ID: <4F4B2F62.1020204@in.tum.de> Hi, I found a problem in the courier conversion script (courier-dovecot-migrate.pl). In some cases, it does not correctly preserve the order of POP3 UIDLs. Consider the following case, when courierpop3dsizelist contains: /2 9000 1327481629 AAA1:2,S 10000 21:1327481629 AAA2:2,S 10000 22:1327481629 AAA3:2,S 10000 23:1327481629 AAA4:2,S 10000 24:1327481629 AAA5:2,S 10000 25:1327481629 BBB11:2,S 10000 11:1327481629 BBB12:2,S 10000 12:1327481629 BBB13:2,S 10000 13:1327481629 BBB14:2,S 10000 14:1327481629 BBB15:2,S 10000 15:1327481629 and courierimapuiddb contains: 1 1327480919 9000 11 BBB11 12 BBB12 13 BBB13 14 BBB14 15 BBB15 16 CCC16 17 CCC17 18 CCC18 19 CCC19 20 CCC20 21 AAA1 22 AAA2 23 AAA3 24 AAA4 25 AAA5 then call courier-dovecot-migrate.pl --to-dovecot --convert and it will create dovecot-uidlist: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 :CCC17 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 :CCC16 9003 PUID15-1327481629 W10000 :BBB15:2,S 9004 :CCC20 9005 PUID14-1327481629 W10000 :BBB14:2,S 9006 PUID13-1327481629 W10000 :BBB13:2,S 9007 :CCC18 9008 PUID11-1327481629 W10000 :BBB11:2,S 9009 :CCC19 The problem here is that the BBB files are not in the same order as in courierpop3dsizelist, and will therefore have a different order in the UIDL response in POP3. Also, the CCC files are interleaved while they should appear at the end. The reason is how the script assigns new UIDs: It does not do this in the order of courierpop3dsizelist. In fact the order is random because it depends on how perl traverses hashes. The attached patch fixes this by sorting the messages: First the POP3 messages get their new UIDs in order of their appearance in courierpop3dsizelist (so that the UIDL order will really be the same as in Courier), after that, the IMAP-only messages will be checked for UID clashes. The patch also changes how the $prev_uid variable is handled because that was also needed to always get the order right. The algorithm could probably be optimized to reduce the number of UID re-assignments to a minimum, however this would complicate the code and would only be noticed in a few cases. The patched courier-dovecot-migrate.pl will create dovecot-uidlist like this: 3 V1327480919 N9010 21 PUID21-1327481629 W10000 :AAA1:2,S 22 PUID22-1327481629 W10000 :AAA2:2,S 23 PUID23-1327481629 W10000 :AAA3:2,S 24 PUID24-1327481629 W10000 :AAA4:2,S 25 PUID25-1327481629 W10000 :AAA5:2,S 9000 PUID11-1327481629 W10000 :BBB11:2,S 9001 PUID12-1327481629 W10000 :BBB12:2,S 9002 PUID13-1327481629 W10000 :BBB13:2,S 9003 PUID14-1327481629 W10000 :BBB14:2,S 9004 PUID15-1327481629 W10000 :BBB15:2,S 9005 :CCC16 9006 :CCC17 9007 :CCC18 9008 :CCC19 9009 :CCC20 Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching -------------- next part -------------- A non-text attachment was scrubbed... Name: courier-dovecot-migrate.patch Type: text/x-patch Size: 1386 bytes Desc: not available URL: From tss at iki.fi Mon Feb 27 10:34:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 10:34:42 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1329954078.11500.266.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> Message-ID: <1330331682.11500.291.camel@innu> On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > What do you need the statistics for? I could make imap_client and > pop3_client support some virtual methods, like user.destroy() initially, > which would be enough for your use. I guess I could add that for v2.2. http://dovecot.org/patches/2.2/imap-logout-plugin.c From Ralf.Hildebrandt at charite.de Mon Feb 27 10:39:18 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 09:39:18 +0100 Subject: [Dovecot] Error with 2.0.18 Message-ID: <20120227083917.GD16162@charite.de> Today I found this in my log: Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x3d44a) [0xb771644a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3d4bb) [0xb77164bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb7716d87] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x3bfa) [0xb74e1bfa] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(squat_trie_build_deinit+0x9fc) [0xb74e4d3c] -> /usr/local/lib/dovecot/lib21_fts_squat_plugin.so(+0x2a95) [0xb74e0a95] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(fts_backend_build_deinit+0x1b) [0xb752f6bb] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x3726) [0xb7530726] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x51d3) [0xb75321d3] -> /usr/local/lib/dovecot/lib20_fts_plugin.so(+0x5c34) [0xb7532c34] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x3b) [0xb778b0cb] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARCH]() [0x805ce6e] -> dovecot/imap [tscxxxf 141.42.206.38 SELECT UID SEARC Feb 26 14:27:55 postamt dovecot: master: Error: service(imap): child 18468 killed with signal 6 (core dumped) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From lists at wildgooses.com Mon Feb 27 10:46:31 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 27 Feb 2012 08:46:31 +0000 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <1330331682.11500.291.camel@innu> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> Message-ID: <4F4B42E7.4090101@wildgooses.com> On 27/02/2012 08:34, Timo Sirainen wrote: > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > >> What do you need the statistics for? I could make imap_client and >> pop3_client support some virtual methods, like user.destroy() initially, >> which would be enough for your use. I guess I could add that for v2.2. > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > Thanks - can I assume that a pop-logout would be basically the same? Also, how might I access the bytes in/out statistics from that context? Thanks Ed W From tss at iki.fi Mon Feb 27 11:05:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:05:44 +0200 Subject: [Dovecot] IMAP-proxy or not with sogo webmail and dovecot backend In-Reply-To: <4F4B42E7.4090101@wildgooses.com> References: <20120213135450.GA31004@dibs.tanso.net> <20120213194356.GB3850@dibs.tanso.net> <4F43AB34.9040907@wildgooses.com> <8570FCE0-E7A5-4C0A-8723-5C734207809C@iki.fi> <4F44B7A7.6030303@wildgooses.com> <0A032ACD-A6B8-4D01-A4E7-E7167A8A8681@iki.fi> <4F45723D.2090003@wildgooses.com> <1329954078.11500.266.camel@innu> <1330331682.11500.291.camel@innu> <4F4B42E7.4090101@wildgooses.com> Message-ID: <1330333544.11500.293.camel@innu> On Mon, 2012-02-27 at 08:46 +0000, Ed W wrote: > On 27/02/2012 08:34, Timo Sirainen wrote: > > On Thu, 2012-02-23 at 01:41 +0200, Timo Sirainen wrote: > > > >> What do you need the statistics for? I could make imap_client and > >> pop3_client support some virtual methods, like user.destroy() initially, > >> which would be enough for your use. I guess I could add that for v2.2. > > http://dovecot.org/patches/2.2/imap-logout-plugin.c > > > > > > Thanks - can I assume that a pop-logout would be basically the same? Yes, basically s/imap/pop3. > Also, how might I access the bytes in/out statistics from that context? input: i_stream_get_absolute_offset(client->input) output: client->output->offset From tss at iki.fi Mon Feb 27 11:10:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 11:10:03 +0200 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <20120227083917.GD16162@charite.de> References: <20120227083917.GD16162@charite.de> Message-ID: <1330333803.11500.295.camel@innu> On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > Today I found this in my log: > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory Large index, process VSZ limit reached. You might need to increase service imap { vsz_limit }. > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) Bugs caused by the above. I don't think I'm going to spend time on Squat anymore, with v2.1 Lucene backend works better and faster. From tss at iki.fi Mon Feb 27 12:24:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:24:27 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: Message-ID: <1330338267.11500.299.camel@innu> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > hi everyone, > > However, how can I verify that solr is working properly? You should see in logs things like: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX It should be in Dovecot's info log (see "doveadm log find"). If you can't find it, see if "doveadm index -u user at domain INBOX" does any work. Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't give an error. From tss at iki.fi Mon Feb 27 12:28:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:28:16 +0200 Subject: [Dovecot] Dovecot altmove questions In-Reply-To: <20120227000700.134750@gmx.com> References: <20120227000700.134750@gmx.com> Message-ID: <1330338496.11500.303.camel@innu> On Mon, 2012-02-27 at 01:07 +0100, Maria Arrea wrote: > 1? General mdbox question. We are happy usings zlib_save, but we > would like to try bzip2 compression. Can we change zlib to bzlib > without problems? In other words, can we mix & match gzipped and > bzipped messages in the same mdbox file ? Yes. > 2? With doveadm altmove we can move messages matching a pattern to > alternate storage. I have read an older post in this mailing list > about passing specific mdbox options to doveadm altmove. Should be a > good idea to have a "main" mdbox area with zlib 9 and mdbox_rotate=1d > and mdox_rotate_size=60m and an "alternate area" with bzip9, > mdbox_rotate=1 week and rotate_size > 60 MB? In the alternate area > should only be "old" mail (1 year older). I guess it makes sense. > 3? We have separate raids for indexes (raid 1+0) and mdboxes (raid > 5). We have almost 3 TB of gzipped mdboxes, and our backup (bacula) is > taking almost 16 hours to make a full backup. Our main problem is that > indexes are backupd hours before mdboxes, and that bring us some > inconsistencies. What is the "right" way of backing up for separate > indexes and mdboxes? The eventual solution is a Dovecot backup agent, but for now I guess there isn't anything better than taking filesystem snapshots. From tss at iki.fi Mon Feb 27 12:30:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:30:04 +0200 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <1330338604.11500.304.camel@innu> On Mon, 2012-02-27 at 02:11 +0100, ml wrote: > i build the last release of dovecot on my centos 5 > work well and fine > > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } They just need to be "high enough". Dovecot logs a warning at startup if they're not high enough. 6000 is fine. From tss at iki.fi Mon Feb 27 12:32:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 12:32:10 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4AD9AC.5000300@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> Message-ID: <1330338730.11500.306.camel@innu> On Mon, 2012-02-27 at 14:17 +1300, Mark Davies wrote: > Feb 27 13:42:18 bats dovecot: auth: > GSSAPI(mark at ECS.VUW.AC.NZ,130.195.5.88): Invalid base64 data in > continued response Looks like kmail is sending some kind of garbage to Dovecot. Set auth_debug_passwords=yes to make Dovecot log the auth traffic. From Ralf.Hildebrandt at charite.de Mon Feb 27 12:46:40 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Mon, 27 Feb 2012 11:46:40 +0100 Subject: [Dovecot] Error with 2.0.18 In-Reply-To: <1330333803.11500.295.camel@innu> References: <20120227083917.GD16162@charite.de> <1330333803.11500.295.camel@innu> Message-ID: <20120227104640.GI16162@charite.de> * Timo Sirainen : > On Mon, 2012-02-27 at 09:39 +0100, Ralf Hildebrandt wrote: > > Today I found this in my log: > > > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: mmap(/home/t/s/tscxxxf/Maildir/dovecot.index.search.uids) failed: Cannot allocate memory > > Large index, process VSZ limit reached. You might need to increase > service imap { vsz_limit }. Indeed. That user has quite a bit of mail (3GB) > > Feb 26 14:27:51 postamt dovecot: imap(tscxxxf): Error: Corrupted squat uidlist file /home/t/s/tscxxxf/Maildir/dovecot.index.search.uids: uidlist not found > > Feb 26 14:27:52 postamt dovecot: imap(tscxxxf): Panic: file squat-trie.c: line 289 (squat_trie_lock): assertion failed: (trie->fd != -1) > > Bugs caused by the above. I don't think I'm going to spend time on Squat > anymore, with v2.1 Lucene backend works better and faster. Too lazy to set that up (yet). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Mon Feb 27 12:50:39 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Mon, 27 Feb 2012 23:50:39 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <1330338730.11500.306.camel@innu> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> Message-ID: <4F4B5FFF.9090201@ecs.vuw.ac.nz> On 02/27/12 23:32, Timo Sirainen wrote: > Looks like kmail is sending some kind of garbage to Dovecot. Set > auth_debug_passwords=yes to make Dovecot log the auth traffic. Yeah, I did a network trace and it seems kmail is not sending the full authentication request before trying to carry on. > 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" > 11 0.045486 130.195.5.13 130.195.5.88 IMAP 122 Response: 1 NO [ALERT] Invalid base64 data in continued response now to figure out why its started truncating it. cheers mark From nick at mobilia.it Mon Feb 27 13:06:04 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 12:06:04 +0100 Subject: [Dovecot] LDA alias resolution problems.. Message-ID: <4F4B639C.9040207@mobilia.it> I use postfix + dovecot LDA + mysql virtual users and up til friday, everything was working just fine. Dovecot is 2.0.beta6 (3156315704ef) Like I said, up til friday, when I added a vacation entry into postfix (which I have now removed to see if it was the reason) mails to the alias virusalert (which is aliased to me) went through just fine. Now I get these sort of errors from the LDA: Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: to=, relay=dovecot, delay=357414, delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary failure. Command output: lda: Fatal: Unknown user ) Other aliases work perfectly, the only difference is that these emails are of local origin, (they're messages from Amavisd) I'm not sure why this has started to be a problem. Can anyone give me an idea where I should start looking? Nick From tss at iki.fi Mon Feb 27 13:11:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:11:47 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4B5FFF.9090201@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: On 27.2.2012, at 12.50, Mark Davies wrote: > On 02/27/12 23:32, Timo Sirainen wrote: >> Looks like kmail is sending some kind of garbage to Dovecot. Set >> auth_debug_passwords=yes to make Dovecot log the auth traffic. > > Yeah, I did a network trace and it seems kmail is not sending the > full authentication request before trying to carry on. >> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe It uses SASL-IR to send the first seponse. >> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + Dovecot says "OK, give me more". >> 10 0.045413 130.195.5.88 130.195.5.13 IMAP 80 Request: 2 LIST "" "" kmail thinks the authentication is done and sends LIST command, even though it should finish the authentication. Clearly a kmail bug. You could try what happens if you remove SASL-IR from Dovecot's imap_capability list. Maybe kmail is buggy only with SASL-IR. From tss at iki.fi Mon Feb 27 13:36:00 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:36:00 +0200 Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build In-Reply-To: <20120224012247.GA6512@krell.zikzak.de> References: <20120224012247.GA6512@krell.zikzak.de> Message-ID: <1330342560.11500.308.camel@innu> On Fri, 2012-02-24 at 02:22 +0100, Andreas M. Kirchwitz wrote: > There seems to be a new dependency in some modules (eg, lib-storage, > libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those > modules didn't require OpenSSL, but 2.1 does. > > For the linking process the path to the OpenSSL library isn't > specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL > is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't > properly used as well.) Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes everything? You can try a new nightly snapshot from http://dovecot.org/nightly/ once it gets there in a few minutes. From tss at iki.fi Mon Feb 27 13:43:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 13:43:35 +0200 Subject: [Dovecot] Virtual Folders In-Reply-To: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> References: <1790de88e8c0e301647df330a46cf952@alltechservices-ia.com> Message-ID: <1330343015.11500.313.camel@innu> On Fri, 2012-02-24 at 08:27 -0600, jacob at alltechservices-ia.com wrote: > Essentially, a gmail clone, but one that > I can make work on cPanel hosts with roundcube and squirrel mail, etc. > > By default, cpanel creates user mail dir structure of (maildir++) > inbox, drafts, sent, junk, and trash (with no dovecot specific files > until first login) > > I would like to replace all those folders with > virtual representations with more capabilities. I would like to achieve > the conversation view that gmail has pulled off so well. It's kind of possible to do that, but Dovecot has some problems with updating the view I think. Or maybe it was only when email was deleted that it didn't properly update the view. > namespace: > type: private > prefix: INBOX. > inbox: yes > list: > yes > subscriptions: yes This is your real mail's namespace. > box with Debian running the same version of dovecot (1.2.x) and on that > system I run into problems every time I make a virtual folder a root > folder. I assume it is trying to look up a normal folder instead of the > virtual ones and I am confusing it. namespace { prefix = separator = / location = virtual:/etc/dovecot/virtual:INDEX=~/Maildir/virtual inbox = yes } namespace { prefix = RealMails/ separator = / list = no hidden = yes } This has the problem that user now can't create any mailboxes other than what you have created globally.. There's really no automatic way to do this. From micha at krausam.de Mon Feb 27 13:47:27 2012 From: micha at krausam.de (Micha Krause) Date: Mon, 27 Feb 2012 12:47:27 +0100 Subject: [Dovecot] Mail delivery to subfolders via address extension, and prefered lda method In-Reply-To: <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> References: <4F47BA30.2080704@krausam.de> <8AC4E802-6381-4F97-A023-28293C0F77CF@iki.fi> Message-ID: <4F4B6D4F.2070708@krausam.de> Hi Timo, > lmtp_save_to_detail_mailbox = yes Works like a charm, thank you. Micha Krause From CMarcus at Media-Brokers.com Mon Feb 27 13:51:11 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:51:11 -0500 Subject: [Dovecot] value for client_limit In-Reply-To: <1330305082.20912.5.camel@localhost> References: <1330305082.20912.5.camel@localhost> Message-ID: <4F4B6E2F.7010105@Media-Brokers.com> On 2012-02-26 8:11 PM, ml wrote: > but what the correct value for client limit > in > service auth { client_limit=6000 } > and > service anvil { client_limit=6000 } Timo already replied that yours should be fine, but I'd just add... You can now use doveconf -d to see what the default values for any setting should be (thanks for adding that Timo!)... and the default values for both of these are 0. Two lessons I learned a long time ago when I first installed and started using postfix: 1. never change a default value/setting without a very good reason 2. never just randomly start changing values of settings in an attempt to fix a problem unless you understand precisely what these changes do So, either you changed them, or the CentOS package that you used changed them (most likely), which is one reason (among many others) that I would never use such systems for mission critical applications. I think Timo, like Wietse (postfix's author) does his best to set sane default values which should not be changed without a good reason. -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 13:58:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 06:58:29 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B6FE5.3040703@Media-Brokers.com> On 2012-02-27 6:06 AM, Nick Warr wrote: > Dovecot is 2.0.beta6 (3156315704ef) Seriously? You're having problems with an ancient pre-release/beta? Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read all of the upgrading docs on the wiki), then try again... -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 14:25:36 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 07:25:36 -0500 Subject: [Dovecot] duplicates with multiple To/CC and sieve redirect copy In-Reply-To: <4F459344.5020407@rename-it.nl> References: <4F441ED8.20908@3a.pl> <673D2924-344E-4E9E-9BBC-9AF4E92C5BE2@iki.fi> <4F44227F.9030502@3a.pl> <1287D4B6-BF86-4A96-9963-8029CADDBB13@iki.fi> <4F442592.608@3a.pl> <4F459344.5020407@rename-it.nl> Message-ID: <4F4B7640.5070700@Media-Brokers.com> On 2012-02-22 8:15 PM, Stephan Bosch wrote: > * I've just created an alternative that implements something similar to > the Procmail code you posted above, but from within Sieve itself. It is > a custom language extension called vnd.dovecot.duplicate and it adds the > "duplicate" test. This test keeps track of which Message-IDs it has seen > before in earlier deliveries and yields a true result if the message was > seen before, e.g.: > > require "vnd.dovecot.duplicate"; > > if duplicate { > discard; > } Interesting... I'll have to look into this later to see how it might be leveraged only for deduplicating mailing list messages (suppress duplicate messages when someone is a member of multiple lists that are members of other lists - I use mailman currently). -- Best regards, Charles From tss at iki.fi Mon Feb 27 14:45:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 27 Feb 2012 14:45:09 +0200 Subject: [Dovecot] Failing: doveadm sync <--remote host--> dsync mirror In-Reply-To: References: <0F2AC8D9-E7D0-455F-BB2A-ACC6AA32422F@odo.in-berlin.de> <973D6AE7-4330-4589-970D-F94CA12A6C91@iki.fi> <09FCAE83-5985-49B8-9445-B99157571418@odo.in-berlin.de> <3CD953C4-BCCB-4137-BA9F-6BEE5C2081FA@iki.fi> <8EB87965-B01A-4E4C-A45F-49F94200749E@iki.fi> Message-ID: <1330346709.11500.324.camel@innu> On Thu, 2012-02-23 at 20:55 +0100, Michael Grimm wrote: > Hi -- > > On 21.02.2012, at 23:03, Michael Grimm wrote: > > On 21.02.2012, at 22:57, Timo Sirainen wrote: > > >> Probably intentional. Even if it didn't give that error message, it would fail because v2.0 and v2.1 uses a slightly different protocol (I thought about making it backwards compatible, but that seemed like too much work and too much ugliness to code). > > > > Ok. Then I will try both servers upgraded to 2.1 within the next days. > > JFTR: It took me some time to find a working syntax ;-) > > My working 2.0.18 syntax threw the following error: > > vmail> dsync -v -f -u test ssh vmail at remote-host.tld dsync -v -f -u test > doveadm(vmail): Fatal: Unknown print formatter: -u > dsync-local(test): Error: read() from worker server failed: EOF You left out "mirror" from that command, but after adding it the latest hg version works. > Now I switched to the recommended new syntax as stated in http://wiki2.dovecot.org/Upgrading/2.1: > > vmail> doveadm sync -v -f -u test ssh vmail at remote-host.tld doveadm sync -v -f -u test > doveadm: illegal option -- v > doveadm sync [-u |-A] [-S ] [-fR] [-m ] The -v parameter is in wrong place now, needs to be "doveadm -v sync". > After some trial by error I finally found a working syntax: > > vmail> doveadm sync -u test -f ssh vmail at remote-host.tld doveadm dsync-server -u test Oh, hmm. I hadn't thought about this problem, it shouldn't have been necessary to give the dsync-server parameter. But I guess there's not a whole lot of other possibilities to do this. Hmm. BTW. I think you can do this simply: doveadm sync -f -u test test at remote-host.tld From nick at mobilia.it Mon Feb 27 16:24:26 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 15:24:26 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B6FE5.3040703@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> Message-ID: <4F4B921A.3080705@mobilia.it> Il 27/02/2012 12.58, Charles Marcus ha scritto: > On 2012-02-27 6:06 AM, Nick Warr wrote: >> Dovecot is 2.0.beta6 (3156315704ef) > > Seriously? You're having problems with an ancient pre-release/beta? > > Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read > all of the upgrading docs on the wiki), then try again... > dovecot --version 2.0.9 Same exact problem. any suggestions? From tom at whyscream.net Mon Feb 27 16:26:13 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 27 Feb 2012 15:26:13 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4B9285.1080004@whyscream.net> On 27/02/12 12:06, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. Usually, aliases are processed by postfix, and dovecot only gets e-mail for actual users, not aliases. Depending on how your mail processing flow works, mail generated by amavis might not visit postfix before it hits dovecot... -- Regards, Tom From CMarcus at Media-Brokers.com Mon Feb 27 16:58:34 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 09:58:34 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B921A.3080705@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> Message-ID: <4F4B9A1A.9070702@Media-Brokers.com> On 2012-02-27 9:24 AM, Nick Warr wrote: > Il 27/02/2012 12.58, Charles Marcus ha scritto: >> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >> all of the upgrading docs on the wiki), then try again... > dovecot --version > 2.0.9 > > Same exact problem. > > any suggestions? Update to the *latest* version - for 2.0, that would be 2.0.18, for 2.1, it is 2.1.1 I'm not necessarily saying this will solve your problems (in fact, in your case it probably won't), but upgrading to the latest version should *always* be step #1 before asking for help (at least in my book)... -- Best regards, Charles From nick at mobilia.it Mon Feb 27 17:10:09 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:10:09 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9A1A.9070702@Media-Brokers.com> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> Message-ID: <4F4B9CD1.8000008@mobilia.it> Il 27/02/2012 15.58, Charles Marcus ha scritto: > On 2012-02-27 9:24 AM, Nick Warr wrote: >> Il 27/02/2012 12.58, Charles Marcus ha scritto: >>> Upgrade to the most recent 2.0, or better yet, 2.1.1 (be sure to read >>> all of the upgrading docs on the wiki), then try again... > >> dovecot --version >> 2.0.9 >> >> Same exact problem. >> >> any suggestions? > > Update to the *latest* version - for 2.0, that would be 2.0.18, for > 2.1, it is 2.1.1 > > I'm not necessarily saying this will solve your problems (in fact, in > your case it probably won't), but upgrading to the latest version > should *always* be step #1 before asking for help (at least in my > book)... > Talk to the CentOS guys for me, and tell em to get on the ball :) From nick at mobilia.it Mon Feb 27 17:13:17 2012 From: nick at mobilia.it (Nick Warr) Date: Mon, 27 Feb 2012 16:13:17 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9285.1080004@whyscream.net> References: <4F4B639C.9040207@mobilia.it> <4F4B9285.1080004@whyscream.net> Message-ID: <4F4B9D8D.7070909@mobilia.it> Il 27/02/2012 15.26, Tom Hendrikx ha scritto: > On 27/02/12 12:06, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. > > Usually, aliases are processed by postfix, and dovecot only gets > e-mail for actual users, not aliases. Depending on how your mail > processing flow works, mail generated by amavis might not visit > postfix before it hits dovecot... > > -- > Regards, > Tom That does seem to be logical, since amavis is generating the email. Normal emails go postfix -> amavis -> postfix -> dovecot These go amavis -> postfix -> dovecot if this is so, I may need to talk to the postfix list, thanks! From CMarcus at Media-Brokers.com Mon Feb 27 17:22:45 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 10:22:45 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B9CD1.8000008@mobilia.it> References: <4F4B639C.9040207@mobilia.it> <4F4B6FE5.3040703@Media-Brokers.com> <4F4B921A.3080705@mobilia.it> <4F4B9A1A.9070702@Media-Brokers.com> <4F4B9CD1.8000008@mobilia.it> Message-ID: <4F4B9FC5.6000702@Media-Brokers.com> On 2012-02-27 10:10 AM, Nick Warr wrote: > Il 27/02/2012 15.58, Charles Marcus ha scritto: >> Update to the *latest* version - for 2.0, that would be 2.0.18, for >> 2.1, it is 2.1.1 > Talk to the CentOS guys for me, and tell em to get on the ball :) Don't point fingers - this is why I would never use a distro that locks you into ancient/unsupported versions of mission critical software. That said, there is usually a viable solution - I'm sure there are alternative CentOS repositories that offer up to date versions of things like postfix, dovecot, apache and other mission critical software. Even Debian stable (worst offender for locking you into ancient versions of software) offers such repositories. -- Best regards, Charles From tomislav.mihalicek at gmail.com Mon Feb 27 17:30:01 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Mon, 27 Feb 2012 07:30:01 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue Message-ID: <33400491.post@talk.nabble.com> I have a proxy setup for pop/imap. The proxies are defined in login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 connections displays the ip address of proxy IP... Dovecots are both 1.2 from the debian repo deb http://xi.rename-it.nl/debian/ stable-auto/dovecot-1.2 main thanks -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33400491.html Sent from the Dovecot mailing list archive at Nabble.com. From bind at enas.net Mon Feb 27 17:52:05 2012 From: bind at enas.net (Urban Loesch) Date: Mon, 27 Feb 2012 16:52:05 +0100 Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <33400491.post@talk.nabble.com> References: <33400491.post@talk.nabble.com> Message-ID: <4F4BA6A5.90806@enas.net> Same here on 2.0.x. But I think this is because it's only implemented for IMAP. See e-mail from Timo 2 days ago: ... Subject: Proxying improvements in v2.1.2 I just committed a couple of features that will make life easier for some types of proxying setups: 1. IMAP proxying has already for a while supported sending local/remote IP/port to backend server, which can use it for logging and other purposes. I've now implemented this for POP3 as well, although only the remote IP/port is forwarded, not local IP/port. I implemented this also for LMTP in v2.2 tree, but haven't bothered to backport that change. Both POP3 and LMTP uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 PORT=110). 2. proxy_maybe=yes + host=host.example.com actually works now. As long as host.example.com DNS lookup returns one IP that belongs to the current server the proxying is skipped. 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and host=1.2.3.4 then Dovecot assumes that this is a local login and won't proxy it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the host field contains load balancer's IP address instead of the server's. You can add more than one IP (space separated) and of course everything related to this works just as well with hostnames as with IPs (even when hostname expands to multiple IPs). .... regards Urban On 27.02.2012 16:30, Tomislav Mihalicek wrote: > > I have a proxy setup for pop/imap. The proxies are defined in > login_trusted_networks = x.x.x.x and for the imap it works fine but for pop3 > connections displays the ip address of proxy IP... Dovecots are both 1.2 > from the debian repo deb http://xi.rename-it.nl/debian/ > stable-auto/dovecot-1.2 main > > thanks From joshua at hybrid.pl Mon Feb 27 18:44:06 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Mon, 27 Feb 2012 17:44:06 +0100 (CET) Subject: [Dovecot] iPhone and custom imap folders problem In-Reply-To: <20120224181059.66377BF108F@miha> References: <4F47A310.9000308@Media-Brokers.com> <4F47BECA.4020104@3a.pl> <20120224181059.66377BF108F@miha> Message-ID: On Fri, 24 Feb 2012, Miha Vrhovnik wrote: > "Adam Szpakowski" wrote on 24.2.2012 17:46:02: >> On 24.02.2012 15:47, Charles Marcus wrote: >>> On 2012-02-24 9:22 AM, Przemys?aw.Orzechowski >>> wrote: >>>> Im using Dovecot 1.0.10 >>> Very old - an upgrade would likeley fix your problems... >> A client of mine using iPhone1 has the same problem on Dovecot 2.0.15. >> Only inbox is synced automatically. > So the solution is simple go bug Apple. > Oh yeah. they say if something isn't implemented on Apple platform or it > doesn't work, then you don't need it, because their devs know what's > best for their users. Exactly, no hope here :( What's even more annoying, apple doesn't put those idiotic limitations on Exchange accounts - seems they like Microsoft more than free solutions :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Mon Feb 27 18:54:49 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 11:54:49 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... Message-ID: <4F4BB559.6050405@Media-Brokers.com> Hello all/Timo, Up until now, my main Clients office has consisted of a single location, and I have never had to deal with the situation of multiple locations for a single company. They have just told me that they are acquiring an additional floor at a building that is about 4 minutes away - but obviously far enough away that I now have to deal with supporting users in the same domain but at two disparate physical locations. These two locations will be connected via a private Gb ethernet connection, and each location will have its own internet connection (I think - still waiting on some numbers to present to the owner to see what he wants to do in that regard, but that will be my recommendation), so bandwidth for replication won't be an issue. I have a couple of months to get this done, and I am already planning on hiring Timo's new commercial support company to help with the final and actual design and implementation, but obviously first I need to know what my actual options are. Just a rough idea of what I'd like to do is: 1. Set up one dovecot server at each location (these will be VMs), so users at each location are accessing the local server for that office 2. Full replication between the two for the mail/indexes, and configure them such that each can act as a failover for the other in case one goes down for whatever reason This is my first/main question... I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? Or, if it is going to take more work than my client is willing to pay for (I'm hoping not, since you said it was on the map for 2.1, not 2.2+), maybe the notify plugin could be leveraged in some way to provide something 'close enough' until it is fully implemented in dsync? On that note (something 'close enough' until dsync fully supports this natively), would setting up a dsync cron job, say, every 5 or 10 minutes, be asking for trouble? Our mail server is not all that busy, really, so in 5 or 10 minutes, there wouldn't be many changes at all. If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? and 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Thanks to any/all for reading this far and for any thoughts, suggestions and/or ideas... -- Best regards, Charles From as at 3a.pl Mon Feb 27 19:59:14 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 18:59:14 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: <4F4BC472.4000805@3a.pl> On 27.02.2012 17:54, Charles Marcus wrote: > These two locations will be connected via a private Gb ethernet > connection, and each location will have its own internet connection (I > think - still waiting on some numbers to present to the owner to see > what he wants to do in that regard, but that will be my > recommendation), so bandwidth for replication won't be an issue. [cut] I do have a basic question... How many users will be in this new, remote location? Will the traffic be so vast, that 1GbE link will not be enough, or are you using two servers for reliability? The simpler the configuration, it is almost always the better. Maybe you can stay with one server in yours primary location? -- Adam Szpakowski From rob0 at gmx.co.uk Mon Feb 27 20:12:55 2012 From: rob0 at gmx.co.uk (/dev/rob0) Date: Mon, 27 Feb 2012 12:12:55 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <20120227181255.GV24983@harrier.slackbuilds.org> On Mon, Feb 27, 2012 at 06:59:14PM +0100, Adam Szpakowski wrote: > On 27.02.2012 17:54, Charles Marcus wrote: > >These two locations will be connected via a private Gb ethernet > >connection, and each location will have its own internet > >connection (I think - still waiting on some numbers to present to > >the owner to see what he wants to do in that regard, but that will > >be my recommendation), so bandwidth for replication won't be an > >issue. > [cut] > > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will > not be enough, or are you using two servers for reliability? > > The simpler the configuration, it is almost always the better. > Maybe you can stay with one server in yours primary location? This was exactly my thought as reading it. If you have some control over client configuration, use "offline IMAP," where clients maintain a local copy of what's on the server. (That's a good idea anyway, distributed backups of mail which possibly is important.) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject: From CMarcus at Media-Brokers.com Mon Feb 27 20:21:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:21:23 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC472.4000805@3a.pl> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> Message-ID: <4F4BC9A3.1010404@Media-Brokers.com> On 2012-02-27 12:59 PM, Adam Szpakowski wrote: > I do have a basic question... How many users will be in this new, > remote location? Will the traffic be so vast, that 1GbE link will not > be enough, or are you using two servers for reliability? Yeah, I guess I should have mentioned this... Each location is an entire floor of a 6 story building. The remote location has the capacity for about 60 users, the new location about 100. We only allow IMAP access to email, so if everyone is using email at the same time, that would be a lot of traffic over a single Gb link I think... > The simpler the configuration, it is almost always the better. Maybe > you can stay with one server in yours primary location? I had considered this, but like I said, this is not purely for performance - I'd also like to get redundancy in the deal (up until now, we haven't had any - admittedly - knock on wood - we haven't needed it, but I'd still like to implement it). -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Feb 27 20:22:53 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:22:53 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <20120227181255.GV24983@harrier.slackbuilds.org> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <20120227181255.GV24983@harrier.slackbuilds.org> Message-ID: <4F4BC9FD.5030806@Media-Brokers.com> On 2012-02-27 1:12 PM, /dev/rob0 wrote: > If you have some control over client configuration, use "offline > IMAP," where clients maintain a local copy of what's on the server. > (That's a good idea anyway, distributed backups of mail which > possibly is important.) Most of these users have many multiple Gb of email, so keeping local cached copies of all it if is silly, in my opinion... and again, the *main* purpose for the two separate servers is for high availability (redundancy/failover)... -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From sven at svenhartge.de Mon Feb 27 20:34:23 2012 From: sven at svenhartge.de (Sven Hartge) Date: Mon, 27 Feb 2012 19:34:23 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <88jj6q741pv8@mids.svenhartge.de> Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > Yeah, I guess I should have mentioned this... > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link > I think... Naa, most clients download mails only once and then keep them cached locally (at least Thunderbird and Outlook do). Looking at the used bandwidth of the mailserver of my small university (10.000 users, about 1000 concurrently active during the daytime) shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and out. Remember: your outgoing bandwidth will be roughly the bandwidth of mail going into the server. Gr??e, Sven. -- Sigmentation fault. Core dumped. From CMarcus at Media-Brokers.com Mon Feb 27 20:38:39 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 27 Feb 2012 13:38:39 -0500 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <88jj6q741pv8@mids.svenhartge.de> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> Message-ID: <4F4BCDAF.50003@Media-Brokers.com> On 2012-02-27 1:34 PM, Sven Hartge wrote: > Charles Marcus wrote: >> Each location is an entire floor of a 6 story building. The remote >> location has the capacity for about 60 users, the new location about >> 100. We only allow IMAP access to email, so if everyone is using email >> at the same time, that would be a lot of traffic over a single Gb link >> I think... > Naa, most clients download mails only once and then keep them cached > locally (at least Thunderbird and Outlook do). > > Looking at the used bandwidth of the mailserver of my small university > (10.000 users, about 1000 concurrently active during the daytime) > shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and > out. Interesting - thanks for the numbers... But, again, my main reason for 2 servers is not performance, it is for redundancy... -- Best regards, Charles From as at 3a.pl Mon Feb 27 20:48:55 2012 From: as at 3a.pl (Adam Szpakowski) Date: Mon, 27 Feb 2012 19:48:55 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BC9A3.1010404@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> Message-ID: <4F4BD017.5010806@3a.pl> On 27.02.2012 19:21, Charles Marcus wrote: > On 2012-02-27 12:59 PM, Adam Szpakowski wrote: >> I do have a basic question... How many users will be in this new, >> remote location? Will the traffic be so vast, that 1GbE link will not >> be enough, or are you using two servers for reliability? > > Yeah, I guess I should have mentioned this... > > Each location is an entire floor of a 6 story building. The remote > location has the capacity for about 60 users, the new location about > 100. We only allow IMAP access to email, so if everyone is using email > at the same time, that would be a lot of traffic over a single Gb link I > think... I'm not sure that the bandwidth will be a problem. One of our clients is a civic design office. Around 60 people and lots of multi megabyte files in multiple copies. Autocad 3D files are flying all around ;). All accounts are IMAP ones, there is also local SAMBA service. The server has 1GbE connection to almost all workstations and the bandwidth utilization is low, very low, on average much less then 1%. We are talking about over 50 heavy duty users. >> The simpler the configuration, it is almost always the better. Maybe >> you can stay with one server in yours primary location? > > I had considered this, but like I said, this is not purely for > performance - I'd also like to get redundancy in the deal (up until > now, we haven't had any - admittedly - knock on wood - we haven't > needed it, but I'd still like to implement it). IMHO use something simple for redundancy such us DRBD in active/passive mode in single location. Manual migration to do not have to deal with split brain problems. As a additional layer of security against local cataclysm (fire in the building) use nightly backup to the second office. You will not have automatic, 99.999% reliability, but for most clients it is ok. They do not need this. The market for highly available, redundant services is quite small. -- Adam Szpakowski From list at airstreamcomm.net Mon Feb 27 22:51:54 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Mon, 27 Feb 2012 14:51:54 -0600 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BCDAF.50003@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> Message-ID: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> On Mon, 27 Feb 2012 13:38:39 -0500, Charles Marcus wrote: > On 2012-02-27 1:34 PM, Sven Hartge wrote: >> Charles Marcus wrote: >>> Each location is an entire floor of a 6 story building. The remote >>> location has the capacity for about 60 users, the new location about >>> 100. We only allow IMAP access to email, so if everyone is using email >>> at the same time, that would be a lot of traffic over a single Gb link >>> I think... > >> Naa, most clients download mails only once and then keep them cached >> locally (at least Thunderbird and Outlook do). >> >> Looking at the used bandwidth of the mailserver of my small university >> (10.000 users, about 1000 concurrently active during the daytime) >> shows a steady amount of roughly 5MBit/s with peaks to 10MBit/s in and >> out. > > Interesting - thanks for the numbers... > > But, again, my main reason for 2 servers is not performance, it is for > redundancy... I too have been tasked with multisite redundancy, and have been experimenting with GlusterFS (http://www.gluster.org/community/documentation/index.php/Main_Page), which is a distributed file system. In our network we have a dedicated 10GB link between two datacenters 100 miles apart, and I have a GlusterFS node at each site setup in Distriubted Replicated mode with 2 replicas which means the servers are mirrored. The file writes are done to all the replica servers (2 servers in this case), so depending on network latency the writes could potentially slow down. GlusterFS has it's own file serving protocol that allows automatic and immediate failover in the case that a storage node disappears, but there are some caveats to restoring a failed storage node (takes forever to resync the data). I have not put this experiment into production, but I can say that it's extremely simple to manage, and performance testing has shown that it could handle mail traffic just fine. You could also look at GPFS (http://www-03.ibm.com/systems/software/gpfs/), which is not open source but it's apparently rock solid and I believe supports multisite clustering. From pc8888 at gmail.com Mon Feb 27 23:52:26 2012 From: pc8888 at gmail.com (Pierre Chiu) Date: Mon, 27 Feb 2012 16:52:26 -0500 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4B639C.9040207@mobilia.it> References: <4F4B639C.9040207@mobilia.it> Message-ID: <4F4BFB1A.1010401@gmail.com> Give this a try. Find out the location of the verify cache file and delete it. This is my setting in main.cf address_verify_map = btree:$data_directory/verify_cache On 2/27/2012 6:06 AM, Nick Warr wrote: > I use postfix + dovecot LDA + mysql virtual users and up til friday, > everything was working just fine. > > Dovecot is 2.0.beta6 (3156315704ef) > > Like I said, up til friday, when I added a vacation entry into postfix > (which I have now removed to see if it was the reason) mails to the > alias virusalert (which is aliased to me) went through just fine. > > Now I get these sort of errors from the LDA: > > Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: > to=, relay=dovecot, delay=357414, > delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary > failure. Command output: lda: Fatal: Unknown user ) > > > Other aliases work perfectly, the only difference is that these emails > are of local origin, (they're messages from Amavisd) I'm not sure why > this has started to be a problem. > > Can anyone give me an idea where I should start looking? > > Nick From janfrode at tanso.net Tue Feb 28 01:58:29 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 00:58:29 +0100 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> References: <4F4BB559.6050405@Media-Brokers.com> <4F4BC472.4000805@3a.pl> <4F4BC9A3.1010404@Media-Brokers.com> <88jj6q741pv8@mids.svenhartge.de> <4F4BCDAF.50003@Media-Brokers.com> <6e72b4f11020e305e8e163e998a8d6bd@mail.airstreamcomm.net> Message-ID: <20120227235829.GA29841@dibs.tanso.net> On Mon, Feb 27, 2012 at 02:51:54PM -0600, list at airstreamcomm.net wrote: > > You could also look at GPFS > (http://www-03.ibm.com/systems/software/gpfs/), which is not open source > but it's apparently rock solid and I believe supports multisite clustering. GPFS supports different modes of clustering. I think the appropriate solution here would be to deploy a single cluster spanning 3 sites (3. site is needed for quorum node, two sites can't work because you can't protect it from split brain). The simplest config would then be 3 nodes (but you could have any number of nodes at each site): quorum node1 on site1 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node2 on site2 with a local disk (or local SAN-disk) as Network Shared Disk (NSD) quorum node3 on site3 The filesystem would be replicated (over IP) between the disk on site1 and site2. Should one site go down, the other site would survive as long as it could still see the quorum node on site3. After a site has been down, one would need to sync up the NSDs (mmrestripefs) to re-establish the replication of any blocks that has been changed while it was down. -jf From jd.beaubien at gmail.com Tue Feb 28 02:14:57 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 19:14:57 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <1330338267.11500.299.camel@innu> References: <1330338267.11500.299.camel@innu> Message-ID: Ok, I had to fix a few things. First of all, I had to declare mail_plugins = fts fts_solr in global file (dovecot.conf). After doing that, running doveadm fts optimize stopped giving me errors. I ran doveadm index -user INBOX (and other folders). I'm assuming this actually did something because java/tomcat6 were really busy for a few mins and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). However, whenever I run a search thru thunderbird (searching the from/to/subject/body fields), I don't see anything related to solr in the dovecot logs (I never see the line you mentioned or anything close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). After playing with the logging level, I manage to see this in the logs: Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from directory: /var/opt/dovecot/lib/dovecot Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so But nothing indicating that solr_plugin is being used. Any idea? i'd really like to be able to clearly see that I'm using solr. Regards, -JD On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: > > hi everyone, > > > > However, how can I verify that solr is working properly? > > You should see in logs things like: > > indexer-worker(user at domain): Info: Indexed 1 messages in INBOX > > It should be in Dovecot's info log (see "doveadm log find"). If you > can't find it, see if "doveadm index -u user at domain INBOX" does any > work. > > Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't > give an error. > > > From jd.beaubien at gmail.com Tue Feb 28 03:37:56 2012 From: jd.beaubien at gmail.com (Jean-Daniel Beaubien) Date: Mon, 27 Feb 2012 20:37:56 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: Btw, since 2.1 is out-performing our old solution so badly, I thought I'd drop some numbers. - The old setup takes over ~4.5 seconds to perform our most common operation. - The new setup takes ~0.2 second to perform the same operation on the same dataset. - approx 22 faster, with less than half the storage; zlib is awesome, for the test account 15GB transforms into 6.5GB. Old setup: dovecot 1.0.9 (Athlon X2 processor, standard hdd) New solution: dovecot 2.1, mdbox, zlib, fts_solr (i think solr works, but not sure) (core i7-2600, standard hdd). Thank you very much for the awesome software. On Mon, Feb 27, 2012 at 7:14 PM, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I don't see anything related to solr in the > dovecot logs (I never see the line you mentioned or anything > close: indexer-worker(user at domain): Info: Indexed 1 messages in INBOX). > > After playing with the logging level, I manage to see this in the logs: > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Loading modules from > directory: /var/opt/dovecot/lib/dovecot > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_fts_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib20_zlib_plugin.so > Feb 27 19:12:19 mba-server dovecot: imap: Debug: Module loaded: > /var/opt/dovecot/lib/dovecot/lib21_fts_solr_plugin.so > > But nothing indicating that solr_plugin is being used. Any idea? i'd > really like to be able to clearly see that I'm using solr. > > Regards, > > -JD > > > > > On Mon, Feb 27, 2012 at 5:24 AM, Timo Sirainen wrote: > >> On Sun, 2012-02-26 at 18:26 -0500, Jean-Daniel Beaubien wrote: >> > hi everyone, >> > >> > However, how can I verify that solr is working properly? >> >> You should see in logs things like: >> >> indexer-worker(user at domain): Info: Indexed 1 messages in INBOX >> >> It should be in Dovecot's info log (see "doveadm log find"). If you >> can't find it, see if "doveadm index -u user at domain INBOX" does any >> work. >> >> Make sure the fts plugin is enabled, "doveadm fts optimize" shouldn't >> give an error. >> >> >> > From tss at iki.fi Tue Feb 28 04:04:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 04:04:17 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: References: <1330338267.11500.299.camel@innu> Message-ID: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > Ok, I had to fix a few things. First of all, I had to declare mail_plugins > = fts fts_solr in global file (dovecot.conf). After doing that, running > doveadm fts optimize stopped giving me errors. > > I ran doveadm index -user INBOX (and other folders). I'm assuming this > actually did something because java/tomcat6 were really busy for a few mins > and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). OK, so it's indexing. > However, whenever I run a search thru thunderbird (searching the > from/to/subject/body fields), I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: telnet loclahost 143 a login user pass b select inbox c search text hello From nick at mobilia.it Tue Feb 28 09:49:10 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 08:49:10 +0100 Subject: [Dovecot] LDA alias resolution problems.. In-Reply-To: <4F4BFB1A.1010401@gmail.com> References: <4F4B639C.9040207@mobilia.it> <4F4BFB1A.1010401@gmail.com> Message-ID: <4F4C86F6.9010702@mobilia.it> Il 27/02/2012 22.52, Pierre Chiu ha scritto: > Give this a try. > Find out the location of the verify cache file and delete it. > > This is my setting in main.cf > address_verify_map = btree:$data_directory/verify_cache > > > > On 2/27/2012 6:06 AM, Nick Warr wrote: >> I use postfix + dovecot LDA + mysql virtual users and up til friday, >> everything was working just fine. >> >> Dovecot is 2.0.beta6 (3156315704ef) >> >> Like I said, up til friday, when I added a vacation entry into postfix >> (which I have now removed to see if it was the reason) mails to the >> alias virusalert (which is aliased to me) went through just fine. >> >> Now I get these sort of errors from the LDA: >> >> Feb 27 11:54:03 cluster2 postfix/pipe[31267]: 866F0A41173: >> to=, relay=dovecot, delay=357414, >> delays=357413/0.79/0/0.05, dsn=4.3.0, status=deferred (temporary >> failure. Command output: lda: Fatal: Unknown user ) >> >> >> Other aliases work perfectly, the only difference is that these emails >> are of local origin, (they're messages from Amavisd) I'm not sure why >> this has started to be a problem. >> >> Can anyone give me an idea where I should start looking? >> >> Nick Thanks, though I actually understood where the problem originated after explaining where the email was injected, it was a postfix configuration issue (the vacation script has you configure the amavis line in master.cf to not expand aliases, so it doesn't send multiple vacation replies.. I hadn't considered mail not originating from postfix) , it was just that dovecot was the one who had to deal with the error. Thanks to Tom Hendrikx for the clue. From joshua at hybrid.pl Tue Feb 28 09:50:10 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 08:50:10 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, julioarr at fisica.uh.cu wrote: > I need some help with the dovecot configuration. I want to remove > downloaded messages from Mail Server once the messages have been > successfully downloaded by pop3-clients, even when the clients have been > configured to save copy of messages in the Server. Maan, that's evil :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From joshua at hybrid.pl Tue Feb 28 10:09:01 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 09:09:01 +0100 (CET) Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: On Fri, 24 Feb 2012, Michael Sidoric wrote: >> I need some help with the dovecot configuration. I want to remove >> downloaded messages from Mail Server once the messages have been >> successfully downloaded by pop3-clients, even when the clients have been >> configured to save copy of messages in the Server. > timo is quite correct -- consider imap instead, that's why the protocol exists. This rather seems like anti-imap - kind of desperate search for free disk space... IMAP does quite the opposite thing, since users acutally never remove emails :) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From nick at mobilia.it Tue Feb 28 10:17:54 2012 From: nick at mobilia.it (Nick Warr) Date: Tue, 28 Feb 2012 09:17:54 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: References: Message-ID: <4F4C8DB2.9060302@mobilia.it> Il 28/02/2012 9.09, Jacek Osiecki ha scritto: > On Fri, 24 Feb 2012, Michael Sidoric wrote: > >>> I need some help with the dovecot configuration. I want to remove >>> downloaded messages from Mail Server once the messages have been >>> successfully downloaded by pop3-clients, even when the clients have >>> been >>> configured to save copy of messages in the Server. >> timo is quite correct -- consider imap instead, that's why the >> protocol exists. > > This rather seems like anti-imap - kind of desperate search for free > disk space... IMAP does quite the opposite thing, since users acutally > never remove emails :) > > Greetings, I'd suggest this: http://archivemail.sourceforge.net/ You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. From tlx at leuxner.net Tue Feb 28 10:46:00 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:46:00 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <4F4C8DB2.9060302@mobilia.it> References: <4F4C8DB2.9060302@mobilia.it> Message-ID: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Am 28.02.2012 um 09:17 schrieb Nick Warr: > Il 28/02/2012 9.09, Jacek Osiecki ha scritto: >> On Fri, 24 Feb 2012, Michael Sidoric wrote: >> >>>> I need some help with the dovecot configuration. I want to remove >>>> downloaded messages from Mail Server once the messages have been >>>> successfully downloaded by pop3-clients, even when the clients have been >>>> configured to save copy of messages in the Server. > I'd suggest this: > > http://archivemail.sourceforge.net/ > > You can chose to skip unread email, and decide when read mail gets archived/deleted, it suports mbox/maildir, etc.. Assuming this is the task you could also use: $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen Play around with the scope a bit before you actually run it: $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tlx at leuxner.net Tue Feb 28 10:53:40 2012 From: tlx at leuxner.net (Thomas Leuxner) Date: Tue, 28 Feb 2012 09:53:40 +0100 Subject: [Dovecot] remove messages once downloaded In-Reply-To: <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> References: <4F4C8DB2.9060302@mobilia.it> <6477DAB9-6840-410D-875D-5885A7E9962D@leuxner.net> Message-ID: Am 28.02.2012 um 09:46 schrieb Thomas Leuxner: > Assuming this is the task you could also use: > > $ doveadm expunge -u jane.doe at example.org before 2012-02-01 unseen > > Play around with the scope a bit before you actually run it: > > $ doveadm search -u jane.doe at example.org before 2012-02-01 unseen | wc -l Of course this was supposed to read 'seen' :/ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From tomislav.mihalicek at gmail.com Tue Feb 28 11:23:18 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 01:23:18 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405413.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405413.html Sent from the Dovecot mailing list archive at Nabble.com. From dovecot at r.paypc.com Tue Feb 28 12:15:27 2012 From: dovecot at r.paypc.com (Robin) Date: Tue, 28 Feb 2012 02:15:27 -0800 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CA93F.3080100@r.paypc.com> > I think Thunderbird does this search internally, not via IMAP. You can test this by talking IMAP protocol directly: > > telnet localhost 143 > a login user pass > b select inbox > c search text hello Yes, you definitely want to check things are being accelerated by issuing direct IMAP commands via telnet. Many clients try to "help" by performing local searches, which will only obfuscate things for you. Even with 150K+ messages, it shouldn't take fts_solr more than 20ms or so to give you results. I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. You can check your solr index data directory too. A freshly installed solr index occupies almost no space, but that grows QUICKLY once it's indexed anything. =R= From tomislav.mihalicek at gmail.com Tue Feb 28 12:18:29 2012 From: tomislav.mihalicek at gmail.com (Tomislav Mihalicek) Date: Tue, 28 Feb 2012 02:18:29 -0800 (PST) Subject: [Dovecot] POP/IMAP on proxy rip issue In-Reply-To: <4F4BA6A5.90806@enas.net> References: <33400491.post@talk.nabble.com> <4F4BA6A5.90806@enas.net> Message-ID: <33405654.post@talk.nabble.com> Mhm, thanks for the answer. I wanted to upgrade to 2.1 anyway because of the LMTP. t. Urban Loesch-2 wrote: > > Same here on 2.0.x. > But I think this is because it's only implemented for IMAP. > > See e-mail from Timo 2 days ago: > > ... > > Subject: Proxying improvements in v2.1.2 > > I just committed a couple of features that will make life easier for some > types of proxying setups: > > 1. IMAP proxying has already for a while supported sending local/remote > IP/port to backend server, which can use it for logging and other > purposes. > I've now implemented this for POP3 as well, although only the remote > IP/port is forwarded, not local IP/port. I implemented this also for LMTP > in v2.2 > tree, but haven't bothered to backport that change. Both POP3 and LMTP > uses XCLIENT command that is compatible to Postfix's (XCLIENT ADDR=1.2.3.4 > PORT=110). > > 2. proxy_maybe=yes + host=host.example.com actually works now. As long as > host.example.com DNS lookup returns one IP that belongs to the current > server the proxying is skipped. > > 3. auth_proxy_self = 1.2.3.4 setting means that if proxy_maybe=yes and > host=1.2.3.4 then Dovecot assumes that this is a local login and won't > proxy > it, even if 1.2.3.4 isn't the actual local IP. This can be helpful if the > host field contains load balancer's IP address instead of the server's. > You > can add more than one IP (space separated) and of course everything > related to this works just as well with hostnames as with IPs (even when > hostname > expands to multiple IPs). > .... > > > regards > Urban > > > On 27.02.2012 16:30, Tomislav Mihalicek wrote: >> >> I have a proxy setup for pop/imap. The proxies are defined in >> login_trusted_networks = x.x.x.x and for the imap it works fine but for >> pop3 >> connections displays the ip address of proxy IP... Dovecots are both 1.2 >> from the debian repo deb http://xi.rename-it.nl/debian/ >> stable-auto/dovecot-1.2 main >> >> thanks > > -- View this message in context: http://old.nabble.com/POP-IMAP-on-proxy-rip-issue-tp33400491p33405654.html Sent from the Dovecot mailing list archive at Nabble.com. From tss at iki.fi Tue Feb 28 12:36:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 12:36:18 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CA93F.3080100@r.paypc.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CA93F.3080100@r.paypc.com> Message-ID: <9DBA17BC-FA96-4DC1-A3A3-30DEA66A98B5@iki.fi> On 28.2.2012, at 12.15, Robin wrote: > I too was bitten by the configuration issue. The wiki/docs suggest that you only need to put the fts fts_solr plugin spec into imap "section", which never worked for me, unlike fts_squat which did). Putting it into the "global" plugin list made it all work for me. That reads only in wiki1 docs, in wiki2 docs it should only talk about globals.. I wonder if it would be a bad idea to show some kind of a popup in wiki1 asking if you really want to be reading it or not, and use a cookie to remember it?.. From p at state-of-mind.de Tue Feb 28 13:10:58 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 28 Feb 2012 12:10:58 +0100 Subject: [Dovecot] ANN: automx - automated mail account provisioning Message-ID: <20120228111057.GC27530@state-of-mind.de> automx I am pleased to announce automx a tool that makes setting up a mail account easy. All your users need to provide is real name, mail address and password. Their mail client and automx will safely handle the rest. Say goodbye to mistyped server settings! Put an end to endless phone calls trying to coach users to configure settings, whose dialogs they can't find. Raise user satisfaction, relieve support and regain time to work on other issues. automx unifies Microsofts and Mozillas mail account provisioning standards in one powerful Open Source tool. Choose from many backends, including LDAP and SQL, and let automx create standard and individualized profiles for multiple domains on the fly! See the details on http://automx.org. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 3603 bytes Desc: not available URL: From joshua at hybrid.pl Tue Feb 28 13:14:04 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 28 Feb 2012 12:14:04 +0100 (CET) Subject: [Dovecot] Deny create folders In-Reply-To: <20120226172827.112de657@scorpio> References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: On Sun, 26 Feb 2012, Jerry wrote: > On Sun, 26 Feb 2012 22:31:35 +0100 > S. Kremer articulated: >> Every user's mailbox has a directory called Spam and Ham. I scan this >> two folders by a script to learn my spamassassin database. >> This script does not scan folders like spam, ham, junk-email etc. >> But some users have such folders in their mailbox so spamassassin does >> not learn from these folders. [...] > Couldn't you just make a soft link between the two folders and save > yourself and your clientele a lot of trouble? I have not messed with > "spamassassin" in years; however, I thought it was configurable on what > it scans. Oh, it reminds me of one thing I would like to have - maybe it is possible? Some IMAP clients (especially outlooks) create junk/sent/etc. folders with various names, depending on national settings. Is it possible to provide a - let's say - dictionary, which would always return the same folder (like Sent) no matter if client asks for "Sent", "Sent Mail" or "Elementy wys&AUI-ane"? Hmm... Now, when I formed the question it seems that this MIGHT be a problem :) What should dovecot provide when client asks for list of subscribed folders? If I know that there is a polish version of MS Outlook, I should provide "Elementy wys&AUI-ane", but "Sent" if it is roundcube... Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From CMarcus at Media-Brokers.com Tue Feb 28 14:54:38 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 07:54:38 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CCE8E.2000103@Media-Brokers.com> On 2012-02-27 9:04 PM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: >> However, whenever I run a search thru thunderbird (searching the >> from/to/subject/body fields), > I think Thunderbird does this search internally, not via IMAP. You could test by opening the Advanced Search window (CTRL-SHIFT-F or Edit > Find > Search Messages), check the box to 'Run search on server', make sure the right account is selected, then perform a search... There are 2 bugs open for Thunderbird dealing with server side IMAP searches, please feel free to go vote for one or both of them: Main one: https://bugzilla.mozilla.org/show_bug.cgi?id=564168 and one that is tangentially related: https://bugzilla.mozilla.org/show_bug.cgi?id=543416 -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 15:28:06 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 08:28:06 -0500 Subject: [Dovecot] Deny create folders In-Reply-To: References: <4F4A55AE.1030402@gmx.de> <20120226114125.59e0d407@scorpio> <4F4AA4B7.4030909@gmx.de> <20120226172827.112de657@scorpio> Message-ID: <4F4CD666.3040809@Media-Brokers.com> On 2012-02-28 6:14 AM, Jacek Osiecki wrote: > Some IMAP clients (especially outlooks) create junk/sent/etc. > folders with various names, depending on national settings. > Is it possible to provide a - let's say - dictionary, which would always > return the same folder (like Sent) no matter if client asks for "Sent", > "Sent Mail" or "Elementy wys&AUI-ane"? This has been discussed before, this is probably the most recent thread about it, but it is quite long: http://www.dovecot.org/list/dovecot/2011-December/062327.html -- Best regards, Charles From kadafax at gmail.com Tue Feb 28 15:41:01 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 14:41:01 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> Message-ID: <4F4CD96D.5010706@gmail.com> On 02/28/2012 03:04 AM, Timo Sirainen wrote: > On 28.2.2012, at 2.14, Jean-Daniel Beaubien wrote: > >> Ok, I had to fix a few things. First of all, I had to declare mail_plugins >> = fts fts_solr in global file (dovecot.conf). After doing that, running >> doveadm fts optimize stopped giving me errors. >> >> I ran doveadm index -user INBOX (and other folders). I'm assuming this >> actually did something because java/tomcat6 were really busy for a few mins >> and I could see the solr logs going nuts (/var/log/tomcat6/catalina.out). > > OK, so it's indexing. Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the solr's data directory is growing and log shows things like this: Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages in SubFolder but thunderbird returns nothing (it says not match found for "hello" for example). I dont know what more can I do to debug this... Solr is working as the nobody user, is it OK for dovecot ? Does dovecot need to be have write access to the solr's data directory ? Thanks From janfrode at tanso.net Tue Feb 28 16:02:33 2012 From: janfrode at tanso.net (Jan-Frode Myklebust) Date: Tue, 28 Feb 2012 15:02:33 +0100 Subject: [Dovecot] fts size Message-ID: <20120228140233.GA18412@dibs.tanso.net> Does anybody have any numbers for how large storage one will need for the fts indexing server? I see the wiki says 30% of mailbox size for Squat (partial=4 full=4). Is it similar for lucene/solr? Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene will create an index for each user in his home directory? Will this be accounted for in the users' quota? -jf From tss at iki.fi Tue Feb 28 16:03:54 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:03:54 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication Message-ID: <1330437834.2081.2.camel@innu> This document describes a design for a dsync-replicated Dovecot cluster. This design can be used to build at least two different types of dsync clusters, which are both described here. Ville has also drawn overview pictures of these two setups, see http://www.dovecot.org/img/dsync-director-replication.png and http://www.dovecot.org/img/dsync-director-replication-ssh.png First of all, why dsync replication instead of block level filesystem replication? - dsync won't replicate filesystem corruption. - A cold restart of replication won't go through all of the data in the disks, but instead quickly finds out what has changed. - Split brain won't result in downtime or losing any data. If both sides did changes, the changes are merged without data loss. - If using more than 2 storages, the users' replicas can be divided among the other storages. So if one storage goes down, the extra load is shared by all the other storages, not just one. Replication mail plugin ----------------------- This is a simple plugin based on notify plugin. It listens for all changes that happen to mailboxes (new mails, flag changes, etc.) Once it sees a change, it sends an asynchronous (username, priority) notification to replication-notify-fifo. The priority can be either high (new mails) or low (everything else). Optionally the replication plugin can also support synchronous replication of new mail deliveries. In this way it connects to replication-notify UNIX socket, tells it to replicate the user with sync (=highest) priority and waits until it is done or replication_sync_timeout occurs. The IMAP/LMTP client won't see an "OK" reply until the mail is replicated (or the replication has failed). The synchronous replication probably adds a noticeable delay, so it might not be acceptable for IMAP, but might be for LMTP. So, what is listening in those replication-notify* sockets? It depends on if Dovecot is running on director-based setup or not. Aggregator ---------- When running in Dovecot director-based setup, all of the Dovecot backends (where replication plugin runs) also run "aggregator" process. Its job is very simple: It proxies the notifications from mail plugin and sends them via a single TCP connection to the replicator process running in Dovecot proxies. This is simply an optimization to avoid tons of short lived TCP connections directly from replication plugin to director server. When not running in Dovecot director setup (i.e. there is only a single Dovecot instance that handles all of the users), there is no point in having an aggregator proxy, because the replicator process is running on the same server. In this kind of setup the replicator process directly listens on the replication-notify* sockets. Replicator ---------- The initial design for replicator isn't very complex either: It keeps a priority queue of all users, and replicates those users at the top of the queue. Notifications about changes to user's mailboxes (may) move the user up in the priority queue. If the user at the top of the queue already has been replicated "recently enough", the replicator stops its work until new changes arrive or the "recently enough" is no longer that. dsync can do two types of syncs: quick syncs and full syncs. A quick sync trusts indexes and does the replication with the least amount of work and network traffic. A quick sync is normally enough to replicate all changes, but just in case something has gone wrong there's also the full sync option, which guarantees that the mailboxes end up being fully synced. A full sync is slower though, and uses more network traffic. The priority queue is sorted by: 1. Priority (updated by a notification from replication plugin) 2. If priority!=none: Last fast sync (those users are replicated first whose last replication time is oldest) 2. If priority=none: Last full sync (these users should already be fully synced, but do a full sync for them once in a while anyway) All users get added to the replication queue at replicator startup with "none" priority. The list of users is looked up via userdb iteration. If the previous replication state is found from a disk dump, it's used to update the priorities, last_*_sync timestamps and other replication state. Replicator process creates such dumps periodically [todo: every few mins? maybe a setting?]. Replicator starts replicating users at the top of the queue, setting their priorities to "none" before starting. This means that if another change notification arrives during replication, the priority is bumped up and no changes get lost. replication_max_conns setting specifies how many users are replicated simultaneously. If the user's last_full_sync is older than replication_full_sync_interval setting, a full sync is done instead of a fast sync. If the user at the top of the queue has "none" priority and the last_full_sync is newer than replication_full_sync_interval, the replication stops. [todo: it would be nice to prefer doing all the full syncs at night when there's hopefully less disk I/O] (A global replication_max_conns setting isn't optimal in proxy-based setup, where different backend servers are doing the replication. There it should maybe be a per-backend setting. Then again, it doesn't account for the replica servers that also need to do replication work. Also to properly handle this each backend should have its own replication queue, but this requires doing a userdb lookup for each user to find out their replication server, and this would need to be done periodically in case the backend changes, which can easily happen often with director-based setup. So all in all, none of this is being done in the initial implementation. Ideally the users are distributed in a way that a global replication queue would work well enough.) In director-based setup each director runs a replicator server, but only one of them (master) actually asks the backends to do the replication. The rest of them just keep track of what's happening, and if the master dies or hangs, one of the others becomes the new master. The server with lowest IP address is always the master. The replicators are connected to a ring like the directors, using the same director_servers setting. The communication between them is simply about notifications of what's happening to users' priorities. Preferably the aggregators would always connect to the master server, but this isn't required. In general there's not much that can go wrong, since it's not a problem if two replicators request a backend to start replication for the same user or if the replication queue states aren't identical. If the replication is running too slowly [todo: means what exactly?], log a warning and send an email to admin. So, how does the actual replication happen? Replicator connects to doveadm server and sends a "sync -u user at domain" command. In director-based setup the doveadm server redirects this command to the proper backend. doveadm sync ------------ This is an independent feature from all of the above. Even with none of it implemented, you could run this to replicate a user. Most of this is already implemented. The only problem is that currently you need to explicitly tell it where to sync. So, when the destination isn't specified, it could do a userdb lookup and use the returned "mail_replica" field as the destination. Multiple (sequentially replicated) destinations could be supported by returning "mail_replica2", "mail_replica3" etc. field. In NFS-based (or shared filesystem-based in general) setup the mail_replica setting is identical to mail_location setting. So your primary mail_location would be in /storage1/user/Maildir, while the secondary mail_replica would be in /storage2/user/Maildir. Simple. In non-NFS-based setup two Dovecot servers talk dsync protocol to each others. Currently dsync already supports SSH-based connections. It would also be easy to implement direct TCP-based connections between two doveadm servers. In future these connections could be SSL-encrypted. Initially I'm only supporting SSH-based connections, as they're already implemented. So what does the mail_replica setting look like in this kind of a setup? I'm not entirely sure. I'm thinking that it could be either "ssh:host" or "ssh:user at host", where user is the SSH login user (this is opposite of the current doveadm sync command line usage). In future then it could support also tcp:host[:port]. Both of these ssh: and tcp: prefixes would also be supported by doveadm sync command line usage (and perhaps the prefixless user at domain be deprecated). dsync can run without any long lived locking and it typically works fine. In case mailbox was modified during dsync, the replicas may not end up being identical, but nothing breaks. dsync currently usually notices this and logs a warning. When these conflicting changes was caused by imap/pop3/lda/etc. this isn't a problem, they've already notified replicator already to perform another sync that will fix it. Running two dsyncs at the same time is more problematic though, mainly related to new emails. Both dsyncs notice that mail X needs to be replicated, so both save it and it results in having a duplicate. To avoid this, there should be a dsync-lock. If this lock exists, dsync should wait until the previous dsync is done and then do it again, just in case there were more changes since the previous sync started. This should conclude everything needed for replication itself. High-availability NFS setup --------------------------- Once you have replication, it's of course nice if the system automatically recovers from a broken storage. In NFS-based setups the idea is to do soft mounts, so if the NFS server goes away things start failing with EIO errors, which Dovecot notices and switches to using the secondary storage(s). In v2.1.0 Dovecot already keeps track of mounted filesystems. Initially they're all marked as "online". When multiple I/O errors occur in a filesystem [todo: how many exactly? where are these errors checked, all around in the code or checking the log?] the mountpoint is marked as "offline" and the connections accessing that storage are killed [todo: again how exactly?]. Another job for replication plugin is to hook into namespace creation. If mail_location points to a mountpoint marked as "offline", it's replaced with mail_replica. This way the user can access mails from the secondary storage without downtime. If the replica isn't fully up to date, this means that some of the mails (or other changes) may temporarily be lost. These will come back again after the original storage has come back up and replication has finished its job. So as long as mails aren't lost in the original storage, there won't be any permanent mail loss. When an offline storage comes back online, its mountpoint's status is initially changed to "failover" (as opposed to "online"). During this state the replication plugin works a bit differently when the user's primary mail_location is in this storage: It first checks if the user is fully replicated, and if so uses the primary storage, otherwise it uses the replica storage. Long running IMAP protocesses check the replication state periodically and kill themselves once the user is replicated, to move back to primary storage. Once replicator notices that all users have been replicated, it tells the backends' to change the "failover" state to "online" (via doveadm server). High-availability non-NFS setup ------------------------------- One possibility is to use Dovecot proxies, which know which servers are down. Instead of directing users to those servers, it would direct them to replica servers. The server states could be handled similar to NFS setup's online vs. failover vs. offline states. Another possibility would be to do the same as above, except without separate proxy servers. Just make "mail.example.com" DNS point to two IP addresses, and if one Dovecot notices that it's not the user's primary server, it proxies to the secondary server, unless it's down. If one IP is down, clients hopefully connect to the other. From tss at iki.fi Tue Feb 28 16:16:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:16:47 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <1330438607.2081.3.camel@innu> On Tue, 2012-02-28 at 14:41 +0100, kfx wrote: > but thunderbird returns nothing (it says not match found for "hello" > for example). Test by talking IMAP protocol, not by using Thunderbird. From CMarcus at Media-Brokers.com Tue Feb 28 16:23:29 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:23:29 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CD96D.5010706@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> Message-ID: <4F4CE361.3010705@Media-Brokers.com> On 2012-02-28 8:41 AM, kfx wrote: > Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the > solr's data directory is growing and log shows things like this: > > Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages > in SubFolder > > > but thunderbird returns nothing (it says not match found for "hello" > for example). *How* are you testing with Thunderbird? Did you enable the 'Run search on server' option in the Advanced Search window? Doing this *should* result in Thunderbird using dovecots indexes server side. -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:25:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:25:30 +0200 Subject: [Dovecot] fts size In-Reply-To: <20120228140233.GA18412@dibs.tanso.net> References: <20120228140233.GA18412@dibs.tanso.net> Message-ID: <1330439130.2081.8.camel@innu> On Tue, 2012-02-28 at 15:02 +0100, Jan-Frode Myklebust wrote: > Does anybody have any numbers for how large storage one will need for > the fts indexing server? I see the wiki says 30% of mailbox size for > Squat (partial=4 full=4). Is it similar for lucene/solr? My initial tests for CLucene were that it would take 30% of mailbox size (compared to 50% for Xapian). But this was before I actually implemented it to Dovecot.. I haven't really looked at how large the indexes actually are. Looking at Lucene indexes now for all of Dovecot and Dovecot-cvs mailing list archives, the indexes are 29.1% of the original mbox sizes. > Do I understand correctly if I think http://wiki2.dovecot.org/Plugins/FTS/Lucene > will create an index for each user in his home directory? Yes. > Will this be accounted for in the users' quota? Only if you're using filesystem quota. From CMarcus at Media-Brokers.com Tue Feb 28 16:27:56 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 09:27:56 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330437834.2081.2.camel@innu> References: <1330437834.2081.2.camel@innu> Message-ID: <4F4CE46C.3000507@Media-Brokers.com> On 2012-02-28 9:03 AM, Timo Sirainen wrote: > This document describes a design for a dsync-replicated Dovecot cluster. Wow, talk about timing... this looks like a perfect answer to my previous question about setting up two servers in two different locations to serve two different sets of users locally... Looks like it wouldn't really matter which server they connected to externally, as any changes would simply be replicated. Timo, state above that this 'describes a design'... does this mean that this is doable right now? Which of the following terms would you use to describe it at this point in time? Potentially problematic Reliable Rock-solid Thanks!! -- Best regards, Charles From tss at iki.fi Tue Feb 28 16:43:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:43:38 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4CE46C.3000507@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> Message-ID: <1330440218.2081.14.camel@innu> On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: > On 2012-02-28 9:03 AM, Timo Sirainen wrote: > > This document describes a design for a dsync-replicated Dovecot cluster. > > > > Wow, talk about timing... this looks like a perfect answer to my > previous question about setting up two servers in two different > locations to serve two different sets of users locally... Yeah, I was delaying answering it until I posted this. > Looks like it wouldn't really matter which server they connected to > externally, as any changes would simply be replicated. Well, preferably users would connect to the same server. If both servers have done changes at the same time, there can be some temporary (possibly user-visible) confusion. > Timo, state above that this 'describes a design'... does this mean that > this is doable right now? Which of the following terms would you use to > describe it at this point in time? > > Potentially problematic Not finished yet, so closest to this. :) I'm planning on making our own mails use this within a few weeks (currently it's a dsync in crontab every 5 minutes). From kadafax at gmail.com Tue Feb 28 16:47:53 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 15:47:53 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE361.3010705@Media-Brokers.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> Message-ID: <4F4CE919.8070505@gmail.com> On 02/28/2012 03:23 PM, Charles Marcus wrote: > On 2012-02-28 8:41 AM, kfx wrote: >> Dovecot 2.1.1 and latest solr. Indexing seems ok because the size of the >> solr's data directory is growing and log shows things like this: >> >> Feb 28 14:30:45 indexer-worker(username): Info: Indexed 12239 messages >> in SubFolder >> >> >> but thunderbird returns nothing (it says not match found for "hello" >> for example). > > *How* are you testing with Thunderbird? > > Did you enable the 'Run search on server' option in the Advanced Search > window? Doing this *should* result in Thunderbird using dovecots indexes > server side. > Yes I did. Some more info: by telnet'ing directly and issuing: c search text pattern * SEARCH 1208 c OK Search completed (0.003 secs). But the same search in thunderbird return "No matches found" :( Here are the solr logs, first search is the "c search text pattern" second is from thunderbird: Feb 28, 2012 3:26:21 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=(hdr:"pattern"+OR+body:"pattern")&rows=108227} hits=1 status=0 QTime=1 Feb 28, 2012 3:27:49 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:d8f5c7250ed14c4f1a5d00003ea484a8+%2Buser:"username"&q=body:"pattern"&rows=108227} hits=0 status=0 QTime=1 It seems thunderbird is only searching in the body and not the header unlike the "c search text" method and that the pattern was found in the header of some random email. So none of the two method is working when searching in the body of the emails but work for the headers (???). fts_solr module seems to be correctly launched: Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib20_fts_plugin.so Feb 28 15:07:53 imap: Debug: Module loaded: /usr/local/lib/dovecot/lib21_fts_solr_plugin.so [root at server log]# grep -R mail_plugins /usr/local/etc/dovecot/ | grep -v \# /usr/local/etc/dovecot/conf.d/10-mail.conf:mail_plugins = autocreate fts fts_solr quota /usr/local/etc/dovecot/conf.d/15-lda.conf: mail_plugins = $mail_plugins sieve [root at server log]# grep fts /usr/local/etc/dovecot/conf.d/90-plugin.conf fts = solr fts_solr = url=http://127.0.0.1:8983/solr/ break-imap-search Don't know where to search :( From tss at iki.fi Tue Feb 28 16:57:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 16:57:22 +0200 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <1330441042.2081.24.camel@innu> On Tue, 2012-02-28 at 15:47 +0100, kfx wrote: > > Did you enable the 'Run search on server' option in the Advanced Search > > window? Doing this *should* result in Thunderbird using dovecots indexes > > server side. > > > > Yes I did. > > Some more info: > > by telnet'ing directly and issuing: > c search text pattern > * SEARCH 1208 > c OK Search completed (0.003 secs). So, Solr in Dovecot works perfectly. > But the same search in thunderbird return "No matches found" :( Thunderbird problem, nothing you can do about it from Dovecot's side. From rago at lal.in2p3.fr Tue Feb 28 18:11:55 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 16:11:55 +0000 Subject: [Dovecot] Master user In-Reply-To: <4F4BB687.3020701@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> Message-ID: <4F4CFCCB.8060903@lal.in2p3.fr> Hi, I'd like to have a master user that can read the other users mailboxes. I have only one user who read and write the mail (i.e. I use the static userdb) and I use namespace for public and shared mailboxes. I can authenticate to the server but I am not able to list the mailboxes: > emiliano at dovecot:~$ telnet localhost imap > Trying 127.0.0.1... > Connected to localhost. > Escape character is '^]'. > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. > 1 login user*master password > 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in > 5 LIST "" * > * LIST (\Unmarked) "." "INBOX" > * LIST (\Noselect \HasNoChildren) "." "shared" > * LIST (\Noselect \HasChildren) "." "public" > 5 OK List completed. > * BYE Server shutting down. > Connection closed by foreign host. This is my conf: > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-131.0.15.el6.x86_64 x86_64 Scientific Linux release 6.1 (Carbon) ext4 > auth_master_user_separator = * > disable_plaintext_auth = no > mail_location = maildir:/data/MAIL/MAILDIR/%u:INBOX=/data/MAIL/INBOX/%u:INDEX=/data/MAIL/METADATA/%u > mbox_write_locks = fcntl > namespace { > inbox = yes > location = > prefix = > separator = . > type = private > } > namespace { > list = yes > location = maildir:/data/MAIL/MAILDIR/%%u:INDEX=/data/MAIL/MAILDIR/shared/%u/%%u > prefix = shared.%%u. > separator = . > subscriptions = no > type = shared > } > namespace { > list = yes > location = maildir:/data/MAIL/SHARED > prefix = public. > separator = . > subscriptions = no > type = public > } > passdb { > driver = pam > } > passdb { > args = /etc/dovecot/master-shared > driver = passwd-file > master = yes > } > plugin { > acl = vfile:/etc/dovecot/global-acls:cache_secs=300 > acl_anyone = allow > acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db > } > protocols = imap > service auth { > unix_listener auth-userdb { > group = mailreader > mode = 0600 > user = mailreader > } > } > ssl = no > ssl_cert = ssl_key = userdb { > args = uid=mailreader gid=mailreader home=/ allow_all_users=yes > driver = static > } > protocol lda { > mail_plugins = acl > } > protocol imap { > mail_plugins = acl imap_acl > } And in /etc/dovecot/master-shared: > master:{PLAIN}password What I did wrong? Thanks, Emiliano From tss at iki.fi Tue Feb 28 17:37:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 17:37:22 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4CFCCB.8060903@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> Message-ID: <1330443442.2081.29.camel@innu> On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > I'd like to have a master user that can read the other users mailboxes. > I have only one user who read and write the mail (i.e. I use the static > userdb) and I use namespace for public and shared mailboxes. > I can authenticate to the server but I am not able to list the mailboxes: With ACL plugin enabled, the master user's ACLs are used. And your master user has no ACLs, so nothing is shown. The proper fix for this would be to set a default ACL for the master user, but default ACLs don't exist yet. So you'll have to kludge this, by changing the master user back to the login username. Try if this works: userdb { args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u driver = static } Also don't set your home to /. Better not to set it at all. http://wiki2.dovecot.org/VirtualUsers/Home From kadafax at gmail.com Tue Feb 28 18:05:11 2012 From: kadafax at gmail.com (kfx) Date: Tue, 28 Feb 2012 17:05:11 +0100 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CE919.8070505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> Message-ID: <4F4CFB37.2080505@gmail.com> Ok I feel ashame... it was a third party init scrip who was the problem :( Sorry for the noise and thank you for dovecot From CMarcus at Media-Brokers.com Tue Feb 28 18:28:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:28:15 -0500 Subject: [Dovecot] testing fts-solr? In-Reply-To: <4F4CFB37.2080505@gmail.com> References: <1330338267.11500.299.camel@innu> <6B4DF036-BCC0-408D-9839-D7F9F27AA040@iki.fi> <4F4CD96D.5010706@gmail.com> <4F4CE361.3010705@Media-Brokers.com> <4F4CE919.8070505@gmail.com> <4F4CFB37.2080505@gmail.com> Message-ID: <4F4D009F.7000107@Media-Brokers.com> On 2012-02-28 11:05 AM, kfx wrote: > Ok I feel ashame... it was a third party init scrip who was the problem :( > > Sorry for the noise and thank you for dovecot So... you're saying that Thunderbird now correctly uses server side search? -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Feb 28 18:33:28 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 11:33:28 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? Message-ID: <4F4D01D8.7000704@Media-Brokers.com> Hey Timo, Subject says it all... This question is a result of an ongoing discussion on the mozilla enterprise list... Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search (with indexes)? Thanks, -- Best regards, Charles From tss at iki.fi Tue Feb 28 18:38:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:38:32 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D01D8.7000704@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.33, Charles Marcus wrote: > This question is a result of an ongoing discussion on the mozilla enterprise list... > > Is there a standard/reliable way for an IMAP client to determine that an IMAP server supports server side search All IMAP servers are required to support SEARCH command. Some crappy ones don't, but I think all widely used ones do. > (with indexes)? No way to know that. Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. (I don't know if there are any other servers besides Dovecot implementing FUZZY.) From tss at iki.fi Tue Feb 28 18:51:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 18:51:57 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: On 28.2.2012, at 18.38, Timo Sirainen wrote: > Well, okay, actually if server advertises FUZZY extension you can be quite certain that it supports indexed server side searches. I meant SEARCH=FUZZY > And that reminds me, I should hide that extension when FTS isn't enabled in Dovecot.. v2.1.2 will no longer advertise it unless fts=solr or fts=lucene: http://hg.dovecot.org/dovecot-2.1/rev/bdc881838b00 From steve.platt at mrc-bsu.cam.ac.uk Tue Feb 28 19:45:49 2012 From: steve.platt at mrc-bsu.cam.ac.uk (Steve Platt) Date: Tue, 28 Feb 2012 17:45:49 +0000 Subject: [Dovecot] migrating/converting from system users -> virtual users Message-ID: Hello all, We currently have a traditional mail server where all users have system accounts (ie entries in the NIS passwd map) and mbox-format mail folders in their (system) home directories. I'm trying to setup a dovecot server in which we want all users to have "virtual" accounts (in dovecot) and no entry in the passwd file at all (ie no access to the mail server). This is for security reasons, we want to imagine that users connecting to the mail server in future will not be able to access any resources on the server other than access their email messages. To this end we've set up a "vmail" user that owns all the messages in the Maildir directories. Most of this is working but I'm stuck on how to convert users' mail folders from the existing setup to the new one. I'm using the convert plugin but of course the problem is that the plugin executes as the "vmail" user and cannot access the existing mail folders that belong to the users: and I'd be worried if it could, of course! So I've converted a few users already by making copies of their old-world mailboxes, changing the ownership of the copies and pointing the convert plugin to these copies, instead of the originals. This works (modulo the dreaded "timeouted" error messages) but is very ugly. So I tried looking at the convert-tool but can't find a man page for it so am not sure what I'm doing with it! I have the idea that I should be able to run some command (as a privileged user) on the mail server and have it do the conversion for me, changing the ownership/permissions on the way. Can convert-tool do this? I'd prefer to go with the automatic (plugin) conversion if I can bodge the ownership issues somehow. Failing that, some tool or script may be the next best answer. This is Dovecot 1.2 by the way on Solaris 10. Any suggestions would be welcome! Steve Platt From list at airstreamcomm.net Tue Feb 28 20:11:29 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 12:11:29 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup Message-ID: We are considering using the post-login scripting to clear trash older than 90 days from user accounts. has anyone done this, and if so did this cause logins to slow down too much waiting for the trash to purge? One idea was to execute the trash purge script once a day by tracking their logins and seeing that it has already ran that day. Another idea was to call the trash purge script in the background and continue without acknowledging that it has finished to keep logins speedy. From robert at schetterer.org Tue Feb 28 20:26:11 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 28 Feb 2012 19:26:11 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D1C43.5000209@schetterer.org> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. look here if this match/solve your problem http://wiki2.dovecot.org/Plugins/Expire -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From CMarcus at Media-Brokers.com Tue Feb 28 20:29:32 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 13:29:32 -0500 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: References: <4F4D01D8.7000704@Media-Brokers.com> Message-ID: <4F4D1D0C.7040708@Media-Brokers.com> On 2012-02-28 11:51 AM, Timo Sirainen wrote: > On 28.2.2012, at 18.38, Timo Sirainen wrote: > >> Well, okay, actually if server advertises FUZZY extension you can >> be quite certain that it supports indexed server side searches. > > I meant SEARCH=FUZZY Thanks for the replies Timo... Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Thanks again... hopefully something will comes of this for Thunderbird. -- Best regards, Charles From tss at iki.fi Tue Feb 28 20:31:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:31:46 +0200 Subject: [Dovecot] Question: Is Server Side Search announced somehow via the IMAP CAPABILITY string? In-Reply-To: <4F4D1D0C.7040708@Media-Brokers.com> References: <4F4D01D8.7000704@Media-Brokers.com> <4F4D1D0C.7040708@Media-Brokers.com> Message-ID: <70448DE8-056A-4453-BC45-79C82C781C78@iki.fi> On 28.2.2012, at 20.29, Charles Marcus wrote: > Off the top of your head, do you know if Cyrus or GMail (I guess the two other most popular IMAP servers that support server side indexes) advertise SEARCH=FUZZY? Neither. Probably no servers besides Dovecot. But it is less than a year old RFC.. From rago at lal.in2p3.fr Tue Feb 28 20:45:14 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 19:45:14 +0100 Subject: [Dovecot] Master user In-Reply-To: <1330443442.2081.29.camel@innu> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> Message-ID: <4F4D20BA.3060501@lal.in2p3.fr> On 02/28/2012 04:37 PM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 16:11 +0000, Emiliano Rago wrote: > >> I'd like to have a master user that can read the other users mailboxes. >> I have only one user who read and write the mail (i.e. I use the static >> userdb) and I use namespace for public and shared mailboxes. >> I can authenticate to the server but I am not able to list the mailboxes: > > With ACL plugin enabled, the master user's ACLs are used. And your > master user has no ACLs, so nothing is shown. The proper fix for this > would be to set a default ACL for the master user, but default ACLs > don't exist yet. So you'll have to kludge this, by changing the master > user back to the login username. Try if this works: > > userdb { > args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u > driver = static > } Thanks, but nothing has changed. Emiliano From Ralf.Hildebrandt at charite.de Tue Feb 28 20:46:20 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 19:46:20 +0100 Subject: [Dovecot] 2.1.1: subscriptions: Removing invalid entry Message-ID: <20120228184619.GA3250@charite.de> Admittedly, it's not a valid folder name and dovecot is right in removing it: Feb 28 17:50:05 postamt dovecot: imap(kbaessle): Warning: Subscriptions file /home/k/b/kbaessle/Maildir/subscriptions: Removing invalid entry: Entw?rfe ...but when looking at the subscription file AFTER the removal, that file contained an empty line. I don't think that's correct. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Tue Feb 28 20:47:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 20:47:37 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D20BA.3060501@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> Message-ID: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> On 28.2.2012, at 20.45, Emiliano Rago wrote: >> userdb { >> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >> driver = static >> } > > Thanks, > but nothing has changed. Set auth_debug=yes. What does it log when you login? From Ralf.Hildebrandt at charite.de Tue Feb 28 21:02:24 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 20:02:24 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 Message-ID: <20120228190224.GB3250@charite.de> With 2.0.18 I was using a quota warning script like this: plugin { ... # Quota quota = maildir quota_rule = INBOX.Trash:storage=+2048M quota_warning = storage=99%% quota-warning 99 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=85%% quota-warning 85 %u } # der schickt die Quota warnmails service quota-warning { executable = script /usr/local/scripts/quota-warning2.1 user = root unix_listener quota-warning { mode = 0666 user = vmail group = users } } -rwxr-xr-x 1 root staff 3081 Feb 28 16:56 /usr/local/scripts/quota-warning2.1 But now I'm gettting this in my log: Feb 28 19:19:03 postamt quota-warning2(phxxxpsu): writing message failed Feb 28 19:19:03 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 15065 returned error 1 Feb 28 19:40:13 postamt quota-warning2(lxxxve): writing message failed Feb 28 19:40:13 postamt dovecot: quota-warning: Fatal: master: service(quota-warning): child 3318 returned error 1 What is failing here? #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit some warning in german EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de -------------- next part -------------- #!/bin/sh # # Should be run by the Local Delivery Agent # #set -x SCRIPT_NAME="quota-warning2" MAIL_TO=$2 MAIL_FROM="postmaster at charite.de" MAIL_QUOTA_PERCENT=$1 MAIL_SUBJECT="Ihre Mailbox ist zu "$1"% voll!" BIN_CAT="/bin/cat" BIN_DATE="/bin/date" BIN_HOSTNAME="/bin/hostname" BIN_LOGGER="/usr/bin/logger" BIN_MV="/bin/mv" BIN_STAT="/usr/bin/stat" #set -x # set misc values HOSTNAME="`${BIN_HOSTNAME}`" # set mail value MAIL_DATE="`${BIN_DATE} '+%a, %d %b %Y %T %z (%Z)'`" MAIL_MESSAGE_ID="`${BIN_DATE} '+%Y%m%d%H%M%S'`@${HOSTNAME}" #To: ${MAIL_TO} # write out message ${BIN_CAT} << EOF | /usr/local/dovecot-2.1/libexec/dovecot/dovecot-lda -d $MAIL_TO -o "plugin/quota=maildir::noenforcing" From: ${MAIL_FROM} Subject: ${MAIL_SUBJECT} Message-Id: <${MAIL_MESSAGE_ID}> Date: ${MAIL_DATE} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Was ist los? ============ Wir haben festgestellt, dass Ihre Mailbox auf dem Server postamt.charite.de das erlaubte Gr?ssenlimit zu mindestens ${MAIL_QUOTA_PERCENT}% aussch?pft. D.h. die maximal erlaubte Gesamtgr??e der Mailbox wird bald erreicht sein oder ist es sogar schon! Was kann ich nun tun? ===================== Bitte schaffen Sie Platz in Ihrer Mailbox, indem Sie die Mails z.B. auf dem Server l?schen oder in Ihre lokale Mailbox auf Ihrem eigenen Rechner verschieben und so wieder Platz auf dem Server schaffen! Wie kann ich die F?llung meines Postfaches auf dem Server ?berpr?fen? ===================================================================== Sie k?nnen die F?llung Ihres Postfaches ?ber Webmail https://webmail.charite.de/ leicht ?berpr?fen: Nach dem Anmelden erscheint im linken Rahmen unter Ihrer Ordnerliste ein Link namens "Ordnergr??en". Sie k?nnen in Webmail ggf. auch gleich die Mails l?schen (vergessen Sie hinterher nicht, den Papierkorb / Trash Ordner zu leeren!). Warum ist mein Postfach ?berhaupt so voll? ========================================== Dies kann z.B. passieren, wenn Sie mehrere Rechner nutzen um Ihre Mail zu lesen und statt des IMAP Protokolls das POP3-Protokoll mit der aktivierten Option "Leave Mail on Server" / "Mail auf Server belassen" in Ihrem Mailprogramm verwenden. Oft wird auch vergessen, da? der Ordner mit den "Gesendeten Objekten" (Sent Folder/Sent Items) zum Gr??enlimit der Mailbox z?hlen. Besondere Platzfresser sind Mails mit gro?en Anh?ngen, wie z.B. PowerPoint-, PDF- oder Bilddateien -- insbesondere im "Sent" Ordner! Wer sind meine Ansprechpartner? =============================== Sie k?nnen uns unter postmaster at charite.de oder telefonisch unter 450-570600 erreichen. Mit freundlichen Gr??en EOF [ "$?" -gt 0 ] && \ ${BIN_LOGGER} -p mail.error -t "${SCRIPT_NAME}(${MAIL_TO})" \ "writing message failed" && exit 1 # log warning attempt ${BIN_LOGGER} -p mail.info -t "${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}%" "delivered!" echo "${MAIL_DATE}: ${SCRIPT_NAME}(${MAIL_TO}) ${MAIL_QUOTA_PERCENT}% delivered!" >> /home/${SCRIPT_NAME}.log From rago at lal.in2p3.fr Tue Feb 28 21:04:23 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Tue, 28 Feb 2012 20:04:23 +0100 Subject: [Dovecot] Master user In-Reply-To: <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> Message-ID: <4F4D2537.7070507@lal.in2p3.fr> On 02/28/2012 07:47 PM, Timo Sirainen wrote: > On 28.2.2012, at 20.45, Emiliano Rago wrote: > >>> userdb { >>> args = uid=mailreader gid=mailreader allow_all_users=yes master_user=%u >>> driver = static >>> } >> >> Thanks, >> but nothing has changed. > > Set auth_debug=yes. What does it log when you login? That's the log: Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011lip=127.0.0.1#011rip=127.0.0.1#011lport=143#011rport=60873#011resp= Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: auth(admin,127.0.0.1,master): Master user lookup for login: rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: passwd-file(admin,127.0.0.1,master): lookup: user=admin file=/etc/dovecot/master-shared Feb 28 19:57:48 oneprivvm-239 dovecot: auth: passdb(admin,127.0.0.1,master): Master user logging in as rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: client out: OK#0111#011user=rago Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master in: REQUEST#0113607363585#0112591#0111#0114d6e591e980edb0fabccfa9c1cd4e198 Feb 28 19:57:48 oneprivvm-239 dovecot: auth: Debug: master out: USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Feb 28 19:57:48 oneprivvm-239 dovecot: imap-login: Login: user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=2596, secured Thanks for answering, Emiliano From tss at iki.fi Tue Feb 28 21:09:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 28 Feb 2012 21:09:02 +0200 Subject: [Dovecot] Master user In-Reply-To: <4F4D2537.7070507@lal.in2p3.fr> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> Message-ID: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> On 28.2.2012, at 21.04, Emiliano Rago wrote: > USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting From list at airstreamcomm.net Tue Feb 28 21:26:35 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Tue, 28 Feb 2012 13:26:35 -0600 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <4F4D1C43.5000209@schetterer.org> References: <4F4D1C43.5000209@schetterer.org> Message-ID: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer wrote: > Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >> We are considering using the post-login scripting to clear trash older >> than 90 days from user accounts. has anyone done this, and if so did >> this >> cause logins to slow down too much waiting for the trash to purge? One >> idea was to execute the trash purge script once a day by tracking their >> logins and seeing that it has already ran that day. Another idea was to >> call the trash purge script in the background and continue without >> acknowledging that it has finished to keep logins speedy. > > look here if this match/solve your problem > > http://wiki2.dovecot.org/Plugins/Expire Expire looks to be useful, but it appears to be something that enhances expunging of messages rather than automates the process if I am reading correctly. We would like to make the process for expunging old Trash messages as automated and inline as possible. From lists at wildgooses.com Tue Feb 28 21:56:22 2012 From: lists at wildgooses.com (Ed W) Date: Tue, 28 Feb 2012 19:56:22 +0000 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: References: Message-ID: <4F4D3166.90809@wildgooses.com> On 28/02/2012 18:11, list at airstreamcomm.net wrote: > We are considering using the post-login scripting to clear trash older > than 90 days from user accounts. has anyone done this, and if so did this > cause logins to slow down too much waiting for the trash to purge? One > idea was to execute the trash purge script once a day by tracking their > logins and seeing that it has already ran that day. Another idea was to > call the trash purge script in the background and continue without > acknowledging that it has finished to keep logins speedy. I think you can also use doveadm to achieve this? So you could schedule something for all accounts at some out of hours period - should speed up backups also? Ed W From CMarcus at Media-Brokers.com Tue Feb 28 22:10:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 28 Feb 2012 15:10:12 -0500 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <1330440218.2081.14.camel@innu> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> Message-ID: <4F4D34A4.6090106@Media-Brokers.com> On 2012-02-28 9:43 AM, Timo Sirainen wrote: > On Tue, 2012-02-28 at 09:27 -0500, Charles Marcus wrote: >> On 2012-02-28 9:03 AM, Timo Sirainen wrote: >>> This document describes a design for a dsync-replicated Dovecot cluster. >> >> >> >> Wow, talk about timing... this looks like a perfect answer to my >> previous question about setting up two servers in two different >> locations to serve two different sets of users locally... > Yeah, I was delaying answering it until I posted this. Cool, thanks - I look forward to any comments you might have on my current plan. >> Looks like it wouldn't really matter which server they connected to >> externally, as any changes would simply be replicated. > Well, preferably users would connect to the same server. If both servers > have done changes at the same time, there can be some temporary > (possibly user-visible) confusion. Point taken - and thinking about it, the amount of traffic would be negligible, since users accessing emails remotely is much less than internally. But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. >> Timo, state above that this 'describes a design'... does this mean that >> this is doable right now? Which of the following terms would you use to >> describe it at this point in time? >> >> Potentially problematic > Not finished yet, so closest to this. :) I'm planning on making our own > mails use this within a few weeks (currently it's a dsync in crontab > every 5 minutes). Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Thanks! -- Best regards, Charles From Ralf.Hildebrandt at charite.de Tue Feb 28 23:30:21 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Tue, 28 Feb 2012 22:30:21 +0100 Subject: [Dovecot] quota warning script not working after upgrade to 2.1.1 In-Reply-To: <20120228190224.GB3250@charite.de> References: <20120228190224.GB3250@charite.de> Message-ID: <20120228213021.GG3250@charite.de> * Ralf Hildebrandt : > With 2.0.18 I was using a quota warning script like this: Must have been a PEBKAC - I fixed my installation by harmonising $PREFIX (seperate $PREFIX for 2.0 and 2.1, removig an old installation along the way) as well as cleaning up some old scripts and alas: Feb 28 22:13:15 postamt quota-warning2(lscxxxk) 90%: delivered! Feb 28 22:15:40 postamt quota-warning2(sxxxwart) 85%: delivered! Feb 28 22:17:39 postamt quota-warning2.1(sxxxwart) 90%: delivered! Feb 28 22:27:55 postamt quota-warning2.1(rxxxid) 85%: delivered! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From rago at lal.in2p3.fr Wed Feb 29 03:06:37 2012 From: rago at lal.in2p3.fr (Emiliano Rago) Date: Wed, 29 Feb 2012 02:06:37 +0100 Subject: [Dovecot] Master user In-Reply-To: <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> References: <4F4BB687.3020701@lal.in2p3.fr> <4F4CFCCB.8060903@lal.in2p3.fr> <1330443442.2081.29.camel@innu> <4F4D20BA.3060501@lal.in2p3.fr> <9C88E810-E79D-47D5-A0AB-E3303349E901@iki.fi> <4F4D2537.7070507@lal.in2p3.fr> <3CDFC11C-EBF1-45C6-824A-591BA8736DE3@iki.fi> Message-ID: <4F4D7A1D.9040002@lal.in2p3.fr> On 02/28/2012 08:09 PM, Timo Sirainen wrote: > On 28.2.2012, at 21.04, Emiliano Rago wrote: > >> USER#0113607363585#011rago#011uid=5000#011gid=5000#011home=/#011master_user=rago#011master_user=admin > > Oh, right, it doesn't work because there are two master_users and the second one overrides the first one. This might be fixed in a newer version (maybe only in v2.1), since I remember doing something related to this. Another possibility for fixing this would be to use post-login script that sets MASTER_USER environment to same as USER, that'll definitely fix it. http://wiki2.dovecot.org/PostLoginScripting > Great, with postlogin script it works! Thanks a lot, Emiliano From amk at spamfence.net Wed Feb 29 03:03:58 2012 From: amk at spamfence.net (Andreas M. Kirchwitz) Date: Wed, 29 Feb 2012 01:03:58 +0000 (UTC) Subject: [Dovecot] Dovecot 2.1 with custom OpenSSL fails to build References: <20120224012247.GA6512@krell.zikzak.de> <1330342560.11500.308.camel@innu.invalid> Message-ID: Timo Sirainen wrote: >> There seems to be a new dependency in some modules (eg, lib-storage, >> libdovecot-lda, libdovecot-ssl) on OpenSSL. In Dovecot 2.0, those >> modules didn't require OpenSSL, but 2.1 does. >> >> For the linking process the path to the OpenSSL library isn't >> specified properly (SSL_LIBS). Dovecot fails to build if OpenSSL >> is in a non-standard path. (Haven't checked if SSL_CFLAGS isn't >> properly used as well.) > > Maybe http://hg.dovecot.org/dovecot-2.1/rev/c07415305d9e fixes > everything? That's the way to go. Makes things better, but I've found three more dependencies. This patch is against the daily snapshot 20120228. (Sorry for the changes to Makefile.in which you won't need. But this way I don't have to rebuild Makefile.in from Makefile.am when compiling. ;-) diff -ur dovecot-20120228.original/src/auth/Makefile.am dovecot-20120228/src/auth/Makefile.am --- dovecot-20120228.original/src/auth/Makefile.am 2012-02-26 04:00:02.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.am 2012-02-29 01:38:41.000000000 +0100 @@ -167,7 +167,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ -I$(top_srcdir)/src/lib-imap \ diff -ur dovecot-20120228.original/src/auth/Makefile.in dovecot-20120228/src/auth/Makefile.in --- dovecot-20120228.original/src/auth/Makefile.in 2012-02-28 04:00:12.000000000 +0100 +++ dovecot-20120228/src/auth/Makefile.in 2012-02-29 01:38:44.000000000 +0100 @@ -499,7 +499,7 @@ libauthdb_imap_la_LIBADD = \ ../lib-imap-client/libimap_client.la \ ../lib-ssl-iostream/libssl_iostream.la \ - $(LIBDOVECOT) + $(LIBDOVECOT) $(SSL_LIBS) libauthdb_imap_la_CPPFLAGS = \ $(AM_CPPFLAGS) \ diff -ur dovecot-20120228.original/src/lib-storage/Makefile.am dovecot-20120228/src/lib-storage/Makefile.am --- dovecot-20120228.original/src/lib-storage/Makefile.am 2012-02-10 04:00:02.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -85,7 +85,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/lib-storage/Makefile.in dovecot-20120228/src/lib-storage/Makefile.in --- dovecot-20120228.original/src/lib-storage/Makefile.in 2012-02-28 04:00:15.000000000 +0100 +++ dovecot-20120228/src/lib-storage/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -397,7 +397,7 @@ pkglib_LTLIBRARIES = libdovecot-storage.la libdovecot_storage_la_SOURCES = -libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) +libdovecot_storage_la_LIBADD = $(shlibs) $(MODULE_LIBS) $(SSL_LIBS) libdovecot_storage_la_DEPENDENCIES = $(shlibs) libdovecot_storage_la_LDFLAGS = -export-dynamic test_programs = \ diff -ur dovecot-20120228.original/src/login-common/Makefile.am dovecot-20120228/src/login-common/Makefile.am --- dovecot-20120228.original/src/login-common/Makefile.am 2011-11-09 04:00:01.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.am 2012-02-29 01:38:21.000000000 +0100 @@ -40,6 +40,6 @@ pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic diff -ur dovecot-20120228.original/src/login-common/Makefile.in dovecot-20120228/src/login-common/Makefile.in --- dovecot-20120228.original/src/login-common/Makefile.in 2012-02-28 04:00:18.000000000 +0100 +++ dovecot-20120228/src/login-common/Makefile.in 2012-02-29 01:38:21.000000000 +0100 @@ -320,7 +320,7 @@ pkginc_lib_HEADERS = $(headers) pkglib_LTLIBRARIES = libdovecot-login.la libdovecot_login_la_SOURCES = -libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la +libdovecot_login_la_LIBADD = liblogin.la ../lib-ssl-iostream/libssl_iostream.la ../lib-dovecot/libdovecot.la $(SSL_LIBS) libdovecot_login_la_DEPENDENCIES = liblogin.la libdovecot_login_la_LDFLAGS = -export-dynamic all: all-am From robert at schetterer.org Wed Feb 29 08:44:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 29 Feb 2012 07:44:54 +0100 Subject: [Dovecot] Post-login scripting - Trash cleanup In-Reply-To: <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> References: <4F4D1C43.5000209@schetterer.org> <9c3033c0ae784ace9a2b346248418826@mail.airstreamcomm.net> Message-ID: <4F4DC966.4000609@schetterer.org> Am 28.02.2012 20:26, schrieb list at airstreamcomm.net: > On Tue, 28 Feb 2012 19:26:11 +0100, Robert Schetterer > wrote: >> Am 28.02.2012 19:11, schrieb list at airstreamcomm.net: >>> We are considering using the post-login scripting to clear trash older >>> than 90 days from user accounts. has anyone done this, and if so did >>> this >>> cause logins to slow down too much waiting for the trash to purge? One >>> idea was to execute the trash purge script once a day by tracking their >>> logins and seeing that it has already ran that day. Another idea was > to >>> call the trash purge script in the background and continue without >>> acknowledging that it has finished to keep logins speedy. >> >> look here if this match/solve your problem >> >> http://wiki2.dovecot.org/Plugins/Expire > > Expire looks to be useful, but it appears to be something that enhances > expunging of messages rather than automates the process if I am reading > correctly. We would like to make the process for expunging old Trash > messages as automated and inline as possible. wouldnt doveadm expunge -A mailbox Trash savedbefore 90d as cron job do what you want ? -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria From busseniu at in.tum.de Wed Feb 29 11:31:38 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 10:31:38 +0100 Subject: [Dovecot] doveadm fetch prints duplicate results in 2.1 Message-ID: <4F4DF07A.7020408@in.tum.de> Hi, when the private namespace has "prefix = INBOX." and you use doveadm fetch to search for "mailbox INBOX", then it prints every message twice: # doveadm fetch -u testuser 'mailbox uid guid' mailbox INBOX mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 mailbox: INBOX uid: 1 guid: 1757c403a4ec4d4fbf040000463889b5 Without "prefix = INBOX." in the configuration, the messages are only printed once. In 2.0, they were never printed twice. doveconf -n # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = mdbox:~/mdbox mail_plugins = acl mail_uid = vmail namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From michel at linuxint.com Wed Feb 29 09:53:05 2012 From: michel at linuxint.com (Michel Bisson) Date: Wed, 29 Feb 2012 07:53:05 +0000 (UTC) Subject: [Dovecot] per-user limit? References: <20120116091521.GA10944@gir.theapt.org> <1326814211.11500.11.camel@innu> Message-ID: Timo Sirainen iki.fi> writes: > On Mon, 2012-01-16 at 10:15 +0100, Peter Hessler wrote: > > I am seeing a problem where users are limited to 6 imap logins total. > > One of my users has a bunch of phones and computers, and wants them all > > on at the same time. > > I'm looking through my configuration, and I cannot see a limit on how > > many times a single user can connect. He is connecting from different > > IPs. > > Any ideas? My logs show the following error when they attempt to auth > > for a 7th time: Hi, I also get the same problem of the same user getting a password mismatch error when may devices are tryibg to login to the same IMAP account at the same time. I didn't check how manylogins triggers this nasty behavior but noticed that this unwanted feature started right after I upgraded the the server from Debian Lenny to Squeeze. If anybody can point me to a hint about how I coukd change this behavior it woul be very much appreciated. From busseniu at in.tum.de Wed Feb 29 12:03:35 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:03:35 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> Message-ID: <4F4DF7F7.8020405@in.tum.de> On 21.02.2012 01:18, Timo Sirainen wrote: > On 20.2.2012, at 19.30, Metro Domain Admin wrote: > >> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Yes, but it should still work.. As far as I can tell from my tests, squat has stopped working in 2.1. Let's take a user who does not have any mails yet, and deliver a first mail to him: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now create an IMAP session, select INBOX, and do a search: * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1330509552] UIDs valid * OK [UIDNEXT 2] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest . OK [READ-WRITE] Select completed. >>> . search text test * SEARCH . OK Search completed (0.006 secs). (Should have returned 1 message.) Deliver the same message again: echo -e 'From: \nSubject: test\n\ntest\ntest' | /usr/local/dovecot/libexec/dovecot/dovecot-lda -d testuser Now in the existing session, run the search command three more times: >>> . search text test * SEARCH * 2 EXISTS * 2 RECENT . OK Search completed (0.000 secs). >>> . search text test * SEARCH 2 . OK Search completed (0.002 secs). >>> . search text test * SEARCH . OK Search completed (0.000 secs). It found the message once, but the next time it didn't. So the squat search does not actually seem to work any more. I know it's deprecated, I just wanted to note this. Cheers, Christoph -- Christoph Bu?enius Rechnerbetriebsgruppe der Fakult?ten Informatik und Mathematik Technische Universit?t M?nchen +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From busseniu at in.tum.de Wed Feb 29 12:05:28 2012 From: busseniu at in.tum.de (=?ISO-8859-1?Q?Christoph_Bu=DFenius?=) Date: Wed, 29 Feb 2012 11:05:28 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <4F4DF868.3090107@in.tum.de> On 29.02.2012 11:03, Christoph Bu?enius wrote: > It found the message once, but the next time it didn't. So the squat > search does not actually seem to work any more. Sorry, I forgot to include the configuration: # 2.1.1: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-35-server x86_64 Ubuntu 10.04.4 LTS disable_plaintext_auth = no mail_gid = vmail mail_location = maildir:~/Maildir mail_plugins = fts fts_squat mail_uid = vmail namespace { inbox = yes location = prefix = } passdb { args = scheme=CRYPT username_format=%u /usr/local/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat fts_squat = partial=4 full=4 } protocols = imap service auth { unix_listener auth-userdb { group = vmail mode = 0660 } } ssl_cert = Raum 00.05.055 <> Boltzmannstr. 3 <> Garching From Ralf.Hildebrandt at charite.de Wed Feb 29 12:09:57 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:09:57 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <20120229100957.GX13045@charite.de> * Christoph Bu?enius : > On 21.02.2012 01:18, Timo Sirainen wrote: > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. What else do I need for fts_lucene to work? Any 3rd party packages? Libs? Compile Time Options? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mark at ecs.vuw.ac.nz Wed Feb 29 12:15:52 2012 From: mark at ecs.vuw.ac.nz (Mark Davies) Date: Wed, 29 Feb 2012 23:15:52 +1300 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> Message-ID: <4F4DFAD8.8040002@ecs.vuw.ac.nz> On 02/28/12 00:11, Timo Sirainen wrote: >>> Looks like kmail is sending some kind of garbage to Dovecot. Set >>> auth_debug_passwords=yes to make Dovecot log the auth traffic. >> >> Yeah, I did a network trace and it seems kmail is not sending the >> full authentication request before trying to carry on. > >>> 8 0.043625 130.195.5.88 130.195.5.13 IMAP 898 Request: 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBuggJJMI[...]jLyNiRZFsc9zFxpdwrZAB/WXRRS1zsM4SlDfE59CW1xfKAkqe > > It uses SASL-IR to send the first seponse. > >>> 9 0.044919 130.195.5.13 130.195.5.88 IMAP 70 Response: + > > Dovecot says "OK, give me more". I poked some more at the kmail end of this but I cant see what its doing differently from what it used to (but clearly there is something). The new kmail sends 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n and receives + \r\n from dovecot, which it doesn't like and reports clientAuthenticate: sasl_client_step failed with: -1 an older (working kmail) sends 1 AUTHENTICATE GSSAPI YIICiAYJKoZIhvcSAQICAQBugg[...]MpPurY7cZfRSEw==\r\n and receives + YIGaBgkqhkiG9xIBAgI[...]iYoSGi9/uKVGyE64TAvkf25rCbFkNqk1D12g==\r\n and carries on. So what is it that differs in the two cases to cause dovecot to respond differently? cheers mark From mstevens at imt-systems.com Wed Feb 29 12:22:06 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 11:22:06 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: On 29.02.2012 11:09, Ralf Hildebrandt wrote: > * Christoph Bu?enius : >> On 21.02.2012 01:18, Timo Sirainen wrote: >> >On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> > >> >>Squat is apparently deprecated: >> http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Build it with lucene support. --with-lucene Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 12:22:50 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 11:22:50 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229100957.GX13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> Message-ID: <20120229102250.GY13045@charite.de> * Ralf Hildebrandt : > * Christoph Bu?enius : > > On 21.02.2012 01:18, Timo Sirainen wrote: > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > What else do I need for fts_lucene to work? Any 3rd party packages? > Libs? Compile Time Options? Rebuilding with --with-lucene --with-stemmer, let's see what happens. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 12:34:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:34:44 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: <4F4DFAD8.8040002@ecs.vuw.ac.nz> References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.15, Mark Davies wrote: > 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n > > and receives > > + \r\n > > from dovecot, which it doesn't like and reports > clientAuthenticate: sasl_client_step failed with: -1 Oh, yes, this is a bug in Dovecot.. From tss at iki.fi Wed Feb 29 12:41:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:41:45 +0200 Subject: [Dovecot] GSSAPI auth failing for kmail In-Reply-To: References: <4F4AD9AC.5000300@ecs.vuw.ac.nz> <1330338730.11500.306.camel@innu> <4F4B5FFF.9090201@ecs.vuw.ac.nz> <4F4DFAD8.8040002@ecs.vuw.ac.nz> Message-ID: On 29.2.2012, at 12.34, Timo Sirainen wrote: > On 29.2.2012, at 12.15, Mark Davies wrote: > >> 1 AUTHENTICATE GSSAPI YIICWgYJKoZIhvcSAQICAQBugg[...]wfuKg4VUptzPwb\r\n >> >> and receives >> >> + \r\n >> >> from dovecot, which it doesn't like and reports >> clientAuthenticate: sasl_client_step failed with: -1 > > Oh, yes, this is a bug in Dovecot.. Hmm. Or it looked like a bug, since it replied only with "+", so I thought all auth mechanisms would have such a bug, but no.. So I'm not really sure why it's not sending more data. I don't have a Kerberos setup to test this with. v2.1's GSSAPI code is anyway identical to v2.0's. From tss at iki.fi Wed Feb 29 12:53:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 12:53:55 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <4F4DF7F7.8020405@in.tum.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> Message-ID: <8F350712-55B7-457D-BB66-DD1E2CC536C8@iki.fi> On 29.2.2012, at 12.03, Christoph Bu?enius wrote: > On 21.02.2012 01:18, Timo Sirainen wrote: >> On 20.2.2012, at 19.30, Metro Domain Admin wrote: >> >>> Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html >> >> Yes, but it should still work.. > > As far as I can tell from my tests, squat has stopped working in 2.1. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ee6f3f375dd3 From piotr.tarnowski at us.edu.pl Wed Feb 29 14:57:35 2012 From: piotr.tarnowski at us.edu.pl (Piotr Tarnowski) Date: Wed, 29 Feb 2012 13:57:35 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) Message-ID: <4F4E20BF.8040707@us.edu.pl> Hi, I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but when I want to compile Sieve for this version (dovecot-2.1-pigeonhole-0.3.0) I get an error: "sieve.c", line 578: identifier redeclared: sieve_multiscript_run and I see that declaration in sieve.h is different from definition in sieve.c, so compilation process fails with SunStudio cc compiler. I have done a quick hack to correct this, and then compiled successfully. Patch is included as attachment. Full output, OS and compiler versions below: libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I/home/soft/dovecot/dovecot-2.1.1 -I/home/soft/dovecot/dovecot-2.1.1/src/lib -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dict -I/home/soft/dovecot/dovecot-2.1.1/src/lib-dns -I/home/soft/dovecot/dovecot-2.1.1/src/lib-mail -I/home/soft/dovecot/dovecot-2.1.1/src/lib-imap -I/home/soft/dovecot/dovecot-2.1.1/src/lib-fs -I/home/soft/dovecot/dovecot-2.1.1/src/lib-charset -I/home/soft/dovecot/dovecot-2.1.1/src/lib-index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index -I/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/index/raw -I/home/soft/dovecot/dovecot-2.1.1/src/lib-lda -I/home/soft/dovecot/dovecot-2.1.1/src/lda -I/home/soft/dovecot/dovecot-2.1.1/src/lib-master -I/home/soft/dovecot/dovecot-2.1.1/src/lib-settings -DMODULEDIR=\"/usr/local/dovecot-2.1.1/lib/dovecot\" -I/usr/local/openldap/include -I/usr/local/ssl/include -I/usr/local/include -xO3 -xautopar -c sieve.c -KPIC -DPIC -o .libs/sieve.o "sieve-common.h", line 33: warning: useless declaration "/home/soft/dovecot/dovecot-2.1.1/src/lib-storage/mailbox-list.h", line 13: warning: useless declaration "sieve.c", line 578: identifier redeclared: sieve_multiscript_run current : function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, unsigned int, _Bool) returning _Bool previous: function(pointer to struct sieve_multiscript {pointer to struct sieve_instance {..} svinst, pointer to struct sieve_result {..} result, pointer to const struct sieve_message_data {..} msgdata, pointer to const struct sieve_script_env {..} scriptenv, int status, _Bool active, _Bool keep, pointer to struct ostream {..} teststream}, pointer to struct sieve_binary {}, pointer to struct sieve_error_handler {pointer to struct pool {..} pool, int refcount, pointer to struct sieve_instance {..} svinst, pointer to struct sieve_error_handler {..} parent, unsigned int max_errors, unsigned int errors, unsigned int warnings, _Bool log_info, _Bool log_debug, pointer to function(..) returning void verror, pointer to function(..) returning void vwarning, pointer to function(..) returning void vinfo, pointer to function(..) returning void vdebug, pointer to function(..) returning void free}, enum sieve_runtime_flags {SIEVE_RUNTIME_FLAG_NOGLOBAL(1)}, _Bool) returning _Bool : "sieve.h", line 168 cc: acomp failed for sieve.c *** Error code 1 dmake: Fatal error: Command failed for target `sieve.lo' Current working directory /home/soft/dovecot/dovecot-2.1-pigeonhole-0.3.0/src/lib-sieve *** Error code 1 The following command caused the error: fail= failcom='exit 1'; \ for f in x $MAKEFLAGS; do \ case $f in \ *=* | --[!k]*);; \ *k*) failcom='fail=yes';; \ esac; \ done; \ dot_seen=no; \ target=`echo all-recursive | sed s/-recursive//`; \ list='plugins'; for subdir in $list; do \ echo "Making $target in $subdir"; \ if test "$subdir" = "."; then \ dot_seen=yes; \ local_target="$target-am"; \ else \ local_target="$target"; \ fi; \ (cd $subdir && dmake $local_target) \ || eval $failcom; \ done; \ if test "$dot_seen" = "no"; then \ dmake "$target-am" || exit 1; \ fi; test -z "$fail" dmake: Fatal error: Command failed for target `all-recursive' % uname -a SunOS soft 5.10 Generic_142909-17 sun4u sparc SUNW,Sun-Fire-V890 Solaris % version Machine hardware: sun4u OS version: 5.10 Processor type: sparc Hardware: SUNW,Sun-Fire-V890 The following components are installed on your system: Sun Studio 12 update 1 Sun Studio 12 update 1 C Compiler Sun Studio 12 update 1 C++ Compiler Sun Studio 12 update 1 Tools.h++ 7.1 Sun Studio 12 update 1 C++ Standard 64-bit Class Library Sun Studio 12 update 1 Garbage Collector Sun Studio 12 update 1 Fortran 95 Sun Studio 12 update 1 Debugging Tools (including dbx) Sun Studio 12 update 1 IDE Sun Studio 12 update 1 Performance Analyzer (including collect, ...) Sun Studio 12 update 1 Performance Library Sun Studio 12 update 1 Scalapack Sun Studio 12 update 1 LockLint Sun Studio 12 update 1 Building Software (including dmake) Sun Studio 12 update 1 Documentation Set Sun Studio 12 update 1 /usr symbolic links and GNOME menu item version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/cc": Sun C 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/CC": Sun C++ 5.10 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/f90": Sun Fortran 95 8.4 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dbx": Sun DBX Debugger 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/analyzer": Sun Analyzer 7.7 SunOS_sparc 2009/06/03 version of "/opt/sunstudio12.1/bin/../prod/bin/../../bin/dmake": Sun Distributed Make 7.9 SunOS_sparc 2009/06/03 Regards -- [ Piotr Tarnowski piotr.tarnowski at us.edu.pl ] [ Unix Administrator at University of Silesia, Katowice, Poland ] -------------- next part -------------- A non-text attachment was scrubbed... Name: sieve.h-redeclared.patch Type: text/x-patch Size: 486 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5810 bytes Desc: S/MIME Cryptographic Signature URL: From Ralf.Hildebrandt at charite.de Wed Feb 29 15:27:19 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:27:19 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229102250.GY13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> Message-ID: <20120229132718.GN13045@charite.de> * Ralf Hildebrandt : > * Ralf Hildebrandt : > > * Christoph Bu?enius : > > > On 21.02.2012 01:18, Timo Sirainen wrote: > > > >On 20.2.2012, at 19.30, Metro Domain Admin wrote: > > > > > > > >>Squat is apparently deprecated: http://dovecot.org/list/dovecot/2011-December/062630.html > > > > Looking at http://wiki2.dovecot.org/Plugins/FTS/Lucene now. > > What else do I need for fts_lucene to work? Any 3rd party packages? > > Libs? Compile Time Options? > > Rebuilding with --with-lucene --with-stemmer, let's see what happens. The plugin doesn't build: postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/config/compiler.h:40:0, from /usr/include/CLucene/StdHeader.h:17, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/config/CompilerGcc.h:12:68: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. make: *** [lucene-wrapper.lo] Error 1 # dpkg -L libclucene-dev |grep -i clucene-config /usr/lib/CLucene/clucene-config.h That's retarded! Who's putting includes into /usr/lib/CLucene/ ??? ARGH! After symlinking, I'm getting: # make /bin/bash ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -g -Wl,--as-needed -g -O2 -MT lucene-wrapper.lo -MD -MP -MF .deps/lucene-wrapper.Tpo -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o lucene-wrapper.cc:24:36: fatal error: CLucene/util/CLStreams.h: No such file or directory compilation terminated. Where do I get this include from (on Debian)? # dpkg -l |grep lucene ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 15:29:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 15:29:44 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229132718.GN13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> Message-ID: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > The plugin doesn't build: > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make .. > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) It requires CLucene 2.3.2. From Ralf.Hildebrandt at charite.de Wed Feb 29 15:36:55 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:36:55 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <20120229133655.GO13045@charite.de> * Timo Sirainen : > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > The plugin doesn't build: > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > .. > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > It requires CLucene 2.3.2. ARGH. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 15:54:49 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 14:54:49 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> Message-ID: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> On 29.02.2012 14:29, Timo Sirainen wrote: > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > >> The plugin doesn't build: >> >> postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# >> make > .. >> ii libclucene-dev 0.9.21b-2+b1 library for >> full-featured text search engine (development) >> ii libclucene0ldbl 0.9.21b-2+b1 library for >> full-featured text search engine (runtime) > > It requires CLucene 2.3.2. Hi Timo, I have the same problem. It does not build with 2.3.3. [mockbuild at x86-002 result]$ cat root.log | grep clucene DEBUG util.py:257: --> clucene-core-devel-2.3.3.4-3.fc16.x86_64 DEBUG util.py:257: clucene-core-devel x86_64 2.3.3.4-3.fc16 updates-released 258 k DEBUG util.py:257: clucene-core x86_64 2.3.3.4-3.fc16 updates-released 507 k DEBUG util.py:257: clucene-core-devel.x86_64 0:2.3.3.4-3.fc16 DEBUG util.py:257: clucene-core.x86_64 0:2.3.3.4-3.fc16 build.log: Making all in fts-lucene make[4]: Entering directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o doveadm-fts-lucene.lo doveadm-fts-lucene.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-lucene-plugin.lo fts-lucene-plugin.c /bin/sh ../../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c -o fts-backend-lucene.lo fts-backend-lucene.c /bin/sh ../../../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c -o lucene-wrapper.lo lucene-wrapper.cc libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-backend-lucene.c -fPIC -DPIC -o .libs/fts-backend-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c doveadm-fts-lucene.c -fPIC -DPIC -o .libs/doveadm-fts-lucene.o libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -c fts-lucene-plugin.c -fPIC -DPIC -o .libs/fts-lucene-plugin.o libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../.. -I../../../src/lib -I../../../src/lib-mail -I../../../src/lib-index -I../../../src/lib-storage -I../../../src/plugins/fts -I../../../src/doveadm -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -c lucene-wrapper.cc -fPIC -DPIC -o .libs/lucene-wrapper.o In file included from /usr/include/CLucene/StdHeader.h:20:0, from /usr/include/CLucene.h:11, from lucene-wrapper.cc:23: /usr/include/CLucene/SharedHeader.h:18:36: fatal error: CLucene/clucene-config.h: No such file or directory compilation terminated. /bin/sh ../../../libtool --tag=CC --mode=link gcc -std=gnu99 -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -module -avoid-version -Wl,-z,relro -o lib20_doveadm_fts_lucene_plugin.la -rpath /usr/lib64/dovecot/doveadm doveadm-fts-lucene.lo -lrt make[4]: *** [lucene-wrapper.lo] Error 1 make[4]: *** Waiting for unfinished jobs.... libtool: link: gcc -shared -fPIC -DPIC .libs/doveadm-fts-lucene.o -lrt -O2 -m64 -mtune=generic -Wl,-z -Wl,relro -Wl,-soname -Wl,lib20_doveadm_fts_lucene_plugin.so -o .libs/lib20_doveadm_fts_lucene_plugin.so libtool: link: ( cd ".libs" && rm -f "lib20_doveadm_fts_lucene_plugin.la" && ln -s "../lib20_doveadm_fts_lucene_plugin.la" "lib20_doveadm_fts_lucene_plugin.la" ) make[4]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins/fts-lucene' make[3]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src/plugins' make[3]: *** [all-recursive] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1/src' make[2]: *** [all-recursive] Error 1 make[1]: Leaving directory `/builddir/build/BUILD/dovecot-2.1.1' make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 RPM build errors: error: Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Bad exit status from /var/tmp/rpm-tmp.0Z7jEb (%build) Child returncode was: 1 EXCEPTION: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/mockbuild/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mockbuild/util.py", line 352, in do raise mockbuild.exception.Error, ("Command failed. See logs for output.\n # %s" % (command,), child.returncode) Error: Command failed. See logs for output. # ['bash', '--login', '-c', 'rpmbuild -bb --target x86_64 --nodeps builddir/build/SPECS/dovecot.spec'] LEAVE do --> EXCEPTION RAISED Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 15:58:51 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 14:58:51 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> Message-ID: <20120229135851.GU13045@charite.de> * Morten Stevens : > >It requires CLucene 2.3.2. > > Hi Timo, > > I have the same problem. It does not build with 2.3.3. I'm right now trying exactly this :) > In file included from /usr/include/CLucene/StdHeader.h:20:0, > from /usr/include/CLucene.h:11, > from lucene-wrapper.cc:23: > /usr/include/CLucene/SharedHeader.h:18:36: fatal error: > CLucene/clucene-config.h: No such file or directory Did you try symlinking the file? -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:09:41 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:09:41 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229135851.GU13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> Message-ID: <1be342370509d17ae81682aede00f016@imt-systems.com> On 29.02.2012 14:58, Ralf Hildebrandt wrote: > * Morten Stevens : > >> >It requires CLucene 2.3.2. >> >> Hi Timo, >> >> I have the same problem. It does not build with 2.3.3. > > I'm right now trying exactly this :) > >> In file included from /usr/include/CLucene/StdHeader.h:20:0, >> from /usr/include/CLucene.h:11, >> from lucene-wrapper.cc:23: >> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >> CLucene/clucene-config.h: No such file or directory > > Did you try symlinking the file? I'm not sure if this is possible because we use a chroot build environment to build RPM packages for Fedora and RHEL. See: http://fedoraproject.org/wiki/Projects/Mock Best regards, Morten From Ralf.Hildebrandt at charite.de Wed Feb 29 16:27:39 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:27:39 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229133655.GO13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> Message-ID: <20120229142738.GW13045@charite.de> * Ralf Hildebrandt : > * Timo Sirainen : > > On 29.2.2012, at 15.27, Ralf Hildebrandt wrote: > > > > > The plugin doesn't build: > > > > > > postamt:/usr/src/dovecot-2.1/dovecot-2.1/src/plugins/fts-lucene# make > > .. > > > ii libclucene-dev 0.9.21b-2+b1 library for full-featured text search engine (development) > > > ii libclucene0ldbl 0.9.21b-2+b1 library for full-featured text search engine (runtime) > > > > It requires CLucene 2.3.2. > > ARGH. I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the info. Maybe the configure should check for a version >=2.3.2! -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From mstevens at imt-systems.com Wed Feb 29 16:29:29 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Wed, 29 Feb 2012 15:29:29 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <1be342370509d17ae81682aede00f016@imt-systems.com> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> Message-ID: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> On 29.02.2012 15:09, Morten Stevens wrote: > On 29.02.2012 14:58, Ralf Hildebrandt wrote: >> * Morten Stevens : >> >>> >It requires CLucene 2.3.2. >>> >>> Hi Timo, >>> >>> I have the same problem. It does not build with 2.3.3. >> >> I'm right now trying exactly this :) >> >>> In file included from /usr/include/CLucene/StdHeader.h:20:0, >>> from /usr/include/CLucene.h:11, >>> from lucene-wrapper.cc:23: >>> /usr/include/CLucene/SharedHeader.h:18:36: fatal error: >>> CLucene/clucene-config.h: No such file or directory >> >> Did you try symlinking the file? > > I'm not sure if this is possible because we use a chroot build > environment to build RPM packages for Fedora and RHEL. > See: http://fedoraproject.org/wiki/Projects/Mock Update: This is a Fedora-specific problem, because clucene (build requirement) is not correctly packaged. Best regards, Morten From tss at iki.fi Wed Feb 29 16:29:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 16:29:57 +0200 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <20120229142738.GW13045@charite.de> References: <4F3FA5D3.8050101@jkart.de> <761E2C44-272D-4F6A-8A33-7FAFB1F011E1@leuxner.net> <4F428323.8050404@fastmail.fm> <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <20120229133655.GO13045@charite.de> <20120229142738.GW13045@charite.de> Message-ID: <8DB2FDBF-02E3-4E55-A817-85D62668C756@iki.fi> On 29.2.2012, at 16.27, Ralf Hildebrandt wrote: > I installed CLucene 2.3.3 and it compiled OK for me. Thanks for the > info. Maybe the configure should check for a version >=2.3.2! I've thought about adding such checks, but autoconf seems to make it difficult to do C++ checks so I haven't bothered.. But I added that info to wiki. From Ralf.Hildebrandt at charite.de Wed Feb 29 16:30:38 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 29 Feb 2012 15:30:38 +0100 Subject: [Dovecot] squat not working in 2.1 In-Reply-To: <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> References: <6A93411B-4058-4A7D-9F94-452403AE83ED@iki.fi> <4F4DF7F7.8020405@in.tum.de> <20120229100957.GX13045@charite.de> <20120229102250.GY13045@charite.de> <20120229132718.GN13045@charite.de> <92D75C5F-46E8-4EE4-B43D-60A3261E071C@iki.fi> <46a03b5105c847df7f7491f0889ef7ec@imt-systems.com> <20120229135851.GU13045@charite.de> <1be342370509d17ae81682aede00f016@imt-systems.com> <5febb8861c0cc824b0446cb2fec98d19@imt-systems.com> Message-ID: <20120229143038.GX13045@charite.de> * Morten Stevens : > This is a Fedora-specific problem, because clucene (build > requirement) is not correctly packaged. Well, debian showed the same packaging (wrong place). -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Feb 29 17:51:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 17:51:47 +0200 Subject: [Dovecot] Dovecot clustering with dsync-based replication In-Reply-To: <4F4D34A4.6090106@Media-Brokers.com> References: <1330437834.2081.2.camel@innu> <4F4CE46C.3000507@Media-Brokers.com> <1330440218.2081.14.camel@innu> <4F4D34A4.6090106@Media-Brokers.com> Message-ID: <99E3DC46-0709-4DFD-A751-DCA9190E4874@iki.fi> On 28.2.2012, at 22.10, Charles Marcus wrote: > But a question out of curiosity: can the Director be configured to redirect userA to private IP #.#.#.# when connecting from a local subnet, but connect them to Public IP ##.##.##.##.## when accessing remotely? I don't know enough about routing to answer this myself... but if it is possible, then I could minimize traffic on the inter-office link - but there really is not enough remote traffic to worry about this too much I think. Yes, there are some ways to do this. Although you would most likely want Dovecot proxy without director feature enabled. >>> Timo, state above that this 'describes a design'... does this mean that >>> this is doable right now? Which of the following terms would you use to >>> describe it at this point in time? >>> >>> Potentially problematic > >> Not finished yet, so closest to this. :) I'm planning on making our own >> mails use this within a few weeks (currently it's a dsync in crontab >> every 5 minutes). > > Cool, it sounds like it may be safe to set this up then in say 2 or 3 months? Since I'll most likely be contracting with your company to help with this, I'll wait and see what you say once we're ready to roll this out - and maybe you'll be willing to give us a deal on the implementation and/or support costs if we're willing to serve as guinea pigs. ;) Possibly. :) From tss at iki.fi Wed Feb 29 18:10:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 18:10:11 +0200 Subject: [Dovecot] Multiple locations, 2 servers - planning questions... In-Reply-To: <4F4BB559.6050405@Media-Brokers.com> References: <4F4BB559.6050405@Media-Brokers.com> Message-ID: On 27.2.2012, at 18.54, Charles Marcus wrote: > I recall that 'dsync based replication' is actually on the map for 2.1, but, since apparently dsync can't do this now, Timo, do you have even a rough idea how much work this would be to get it working for only 2 locations (assuming it *may* be easier to get the initial support for only 2 locations, my client may be willing to pay for it if it isn't a huge amount - feel free to reply privately to this question), then you could revisit it later to make it more scalable? I'll initially build it for only 2 locations, but I think it will be pretty simple to scale to more than 2. > If that is not recommended, although I want to avoid the hassles of NFS if at all possible, maybe there is another shared filesystem that will work ok - or... since I will be forcing users to a single server always anyway, maybe NFS or some other shared filesystem is really the best option here, and just let it take care of the syncing? Synchronous drbd replication for a master/slave setup should also work, since the latency between your servers is probably quite low. I wouldn't use asynchronous replication since it can lose some of the last changes when failure happens. Then there are of course all the cluster filesystems, but I don't have much experience with them other than what I've read in this list. I think GPFS is the only one I haven't read any complains of (but it could be also that so few people have actually used it). > 3. Configure things such that each offices users are directed to the local server for that office, but connections will fail-over to the remote server in the case of one of them going down for whatever reason? With a clusterfs setup you could do this. With dsync-replicated setup you could assign a primary location for the user, and proxy the connection there if user got connected to wrong server, except when the primary server is down. > I'm fairly sure that some combination of Dovecot Proxy/Director will accomplish this, but one concern is - for internal users, my understanding is it will redirect them via the private IP, but that would result in lots of traffic across the Gb connection between the two locations, and I'd like to eliminate that if possible - so how will this work when they are accessing it from outside the office, where each office has its own public IP? I'd prefer to not rely on users using the correct hostname (currently, we just use 'mail.example.com', and I know I could set up two new ones - office1.example.com and office2.example.com - but then I'd be relying on the users to get it right, and I'd prefer to avoid that can of worms). I guess a worst case scenario (if there is no better way) would be to do it that way, then watch the logs for users who get it wrong and are using the inter-office connection, and deal with them on a case by case basis. Like other mentioned, I don't think the cross-office traffic will be that much of a problem, especially for external connections from outside office. For internal connections you should be able to mostly avoid it. From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:27:34 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:27:34 +0100 Subject: [Dovecot] What's behind the message ... Message-ID: (auto-added) ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount Hello, the above message appeared when I restarted dovecot. What has dovecot to deal with my USB-Backup-Disk? Dovecot cares about filesystem without any relation to the mail system? Why this? What's bedind? Thanks in advance, Frank Elsner From tss at iki.fi Wed Feb 29 19:31:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 29 Feb 2012 19:31:12 +0200 Subject: [Dovecot] What's behind the message ... In-Reply-To: (auto-added) References: (auto-added) Message-ID: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> On 29.2.2012, at 19.27, Frank Elsner wrote: > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > Hello, > the above message appeared when I restarted dovecot. > > What has dovecot to deal with my USB-Backup-Disk? > Dovecot cares about filesystem without any relation to the mail system? No, but Dovecot doesn't really know what the mail filesystems are. You can add: doveadm mount add '/media/*' ignore Perhaps this should be a default. > Why this? What's bedind? http://markmail.org/message/ad6f4fg3js6ljykg From frank at moltke28.B.Shuttle.DE Wed Feb 29 19:59:06 2012 From: frank at moltke28.B.Shuttle.DE (Frank Elsner) Date: Wed, 29 Feb 2012 18:59:06 +0100 Subject: [Dovecot] What's behind the message ... In-Reply-To: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> References: <0A055753-8B89-43A0-8332-616A17E19968@iki.fi> Message-ID: (auto-added) On Wed, 29 Feb 2012 19:31:12 +0200 Timo Sirainen wrote: > On 29.2.2012, at 19.27, Frank Elsner wrote: > > > ... Warning: /media/Backup28a is no longer mounted. If this is intentional, remove it with doveadm mount > > > > Hello, > > the above message appeared when I restarted dovecot. > > > > What has dovecot to deal with my USB-Backup-Disk? > > Dovecot cares about filesystem without any relation to the mail system? > > No, but Dovecot doesn't really know what the mail filesystems are. You can add: > > doveadm mount add '/media/*' ignore > > Perhaps this should be a default. > > > Why this? What's bedind? > > http://markmail.org/message/ad6f4fg3js6ljykg Thanks for the pointer! --Frank Elsner From stephan at rename-it.nl Wed Feb 29 23:54:15 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 29 Feb 2012 22:54:15 +0100 Subject: [Dovecot] dovecot-2.1-pigeonhole-0.3.0 identifier redeclared: sieve_multiscript_run (patch) In-Reply-To: <4F4E20BF.8040707@us.edu.pl> References: <4F4E20BF.8040707@us.edu.pl> Message-ID: <4F4E9E87.7020608@rename-it.nl> Hi Piotr, On 2/29/2012 1:57 PM, Piotr Tarnowski wrote: > I recently compiled dovecot-2.1.1 on Solaris 10 with Sun Studio, but > when I want to compile Sieve for this version > (dovecot-2.1-pigeonhole-0.3.0) I get an error: > > "sieve.c", line 578: identifier redeclared: sieve_multiscript_run > > and I see that declaration in sieve.h is different from definition in > sieve.c, so compilation process fails with SunStudio cc compiler. > > I have done a quick hack to correct this, and then compiled successfully. Thanks for the report. Apparently GCC doesn't see this. Fixed: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/9da3a8398ea4 Regards, Stephan.