From racuk12 at gmail.com Fri May 1 11:28:08 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Fri, 1 May 2015 12:28:08 +0100 Subject: Fatal: Unknown scheme: SHA512-CRYPT. On a Mac-mini Message-ID: Running postfix+dovecot+mysql on a mac-mini, and trying to solve this puzzle. Is this a Mac issue?, a MySQL issue? or something I haven't configured in? You can see from the output of the samples shows in the first case, that SHA512-CRYPT seems to be Unknown. The code I used is from a Linux based tutorial on setting up Postfix+Dovecot+MySQL on a site. Everything is generally going ok, apart from this bit. There are some others unknown as well. The codes used to create my database of passwords follows these output examples. MacMini:~ robert$ sudo doveadm pw -s SHA512-CRYPT -p 123456 Fatal: Unknown scheme: SHA512-CRYPT MacMini:~ robert$ sudo doveadm pw -s SHA512 -p 123456 {SHA512}ujJTh2rta8ItSm/1PYQGxq2GQZXtFEq1yHYhtsIztUi66uaVbfNG7IwX9eoQ817jy8UUeX7X3dMUVGTioLq0Ew== MacMini:~ robert$ sudo doveadm pw -s SHA256 -p 123456 {SHA256}jZae727K08KaOmKSgOaGzww/XVqGr/PKEgIMkjrcbJI= MacMini:~ robert$ sudo doveadm pw -s MD5 -p 123456 {MD5}$1$.mAQzPhO$6v/WxTUt.Y1lhopkMarEb0 MacMini:~ robert$ sudo doveadm pw -s MD5-CRYPT -p 123456 {MD5-CRYPT}$1$gNkYwx/o$3nQLmUIPOBmZdR/e/EXFV1 MacMini:~ robert$ sudo doveadm pw -s BLF-CRYPT -p 123456 Fatal: Unknown scheme: BLF-CRYPT MacMini:~ robert$ sudo doveadm pw -s SHA512 -p 1pass66U! {SHA512}EuuO6BAt+bgzXIOw7XIdiVBFwdEZ2tGqHOJ/DSTSkD+ikw7GQQo+KtHSta7p3KNv2AEVKtfOuyl5UeCEqnKogg== MacMini:~ robert$ sudo doveadm pw -s SHA256-CRYPT -p 1pass66U! Fatal: Unknown scheme: SHA256-CRYPT MacMini:~ robert$ mysql --version mysql Ver 14.14 Distrib 5.6.19, for osx10.9 (x86_64) using EditLine wrapper MacMini:~ robert$ cd My MySQL Version: mysql Ver 14.14 Distrib 5.6.19, for osx10.9 (x86_64) using EditLine wrapper The MySQL code used to create the user/password table. Which seems to work fine, and creates the table as it should - even though the resulting password doesn't appear to be quite right for SHA512-CRYPT INSERT INTO `mailserver`.`virtual_users` (`id`, `domain_id`, `password` , `email`) VALUES ('1', '1', ENCRYPT('firstpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email1 at example.com'), ('2', '1', ENCRYPT('secondpassword', CONCAT('$6$', SUBSTRING(SHA(RAND()), -16))), 'email2 at example.com'); What it actually creates is this: This is not a SHA512-CRYPT password?? mysql> SELECT * FROM mailserver.virtual_users; 3 | 1 | $6jxK9DWPBrAc | robert at zeus.example.com Just to be clear,. this is the section of the Dovecot configs that deals with handling this. BUT.. On the MAC-Mini, SHA512-CrYPT is unknown. # List of supported schemes is in # http://wiki2.dovecot.org/Authentication/PasswordSchemes # default_pass_scheme = SHA512-CRYPT So the question is, what Encryption should I be using, and is this an Apple thing, or a MySQL thing? From edgar at pettijohn-web.com Fri May 1 23:28:39 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn III) Date: Fri, 1 May 2015 18:28:39 -0500 Subject: Fatal: Unknown scheme: SHA512-CRYPT. On a Mac-mini In-Reply-To: References: Message-ID: <6D2F3161-F482-413D-AFB7-6541F6A7ECEF@pettijohn-web.com> On May 1, 2015, at 6:28 AM, Robert Chalmers wrote: > Running postfix+dovecot+mysql on a mac-mini, and trying to solve this > puzzle. Is this a Mac issue?, a MySQL issue? or something I haven't > configured in? > > You can see from the output of the samples shows in the first case, that > SHA512-CRYPT seems to be Unknown. > The code I used is from a Linux based tutorial on setting up > Postfix+Dovecot+MySQL on a site. Everything is generally going ok, apart > from this bit. There are some others unknown as well. The codes used to > create my database of passwords follows these output examples. > > > > MacMini:~ robert$ sudo doveadm pw -s SHA512-CRYPT -p 123456 > Fatal: Unknown scheme: SHA512-CRYPT > MacMini:~ robert$ sudo doveadm pw -s SHA512 -p 123456 > > {SHA512}ujJTh2rta8ItSm/1PYQGxq2GQZXtFEq1yHYhtsIztUi66uaVbfNG7IwX9eoQ817jy8UUeX7X3dMUVGTioLq0Ew== > MacMini:~ robert$ sudo doveadm pw -s SHA256 -p 123456 > {SHA256}jZae727K08KaOmKSgOaGzww/XVqGr/PKEgIMkjrcbJI= > MacMini:~ robert$ sudo doveadm pw -s MD5 -p 123456 > {MD5}$1$.mAQzPhO$6v/WxTUt.Y1lhopkMarEb0 > MacMini:~ robert$ sudo doveadm pw -s MD5-CRYPT -p 123456 > {MD5-CRYPT}$1$gNkYwx/o$3nQLmUIPOBmZdR/e/EXFV1 > MacMini:~ robert$ sudo doveadm pw -s BLF-CRYPT -p 123456 > Fatal: Unknown scheme: BLF-CRYPT > MacMini:~ robert$ sudo doveadm pw -s SHA512 -p 1pass66U! > > {SHA512}EuuO6BAt+bgzXIOw7XIdiVBFwdEZ2tGqHOJ/DSTSkD+ikw7GQQo+KtHSta7p3KNv2AEVKtfOuyl5UeCEqnKogg== > MacMini:~ robert$ sudo doveadm pw -s SHA256-CRYPT -p 1pass66U! > Fatal: Unknown scheme: SHA256-CRYPT > MacMini:~ robert$ mysql --version > mysql Ver 14.14 Distrib 5.6.19, for osx10.9 (x86_64) using EditLine > wrapper > MacMini:~ robert$ cd > > Above you show {SHA512}"lots of stuff" then the error is Unknown scheme: SHA256-CRYPT I would say something is wrong with the passwd lookup. Probably with your data in mysql. Perhaps this link will help http://dev.mysql.com/doc/mysql-security-excerpt/5.6/en/sha256-authentication-plugin.html > > My MySQL Version: > > mysql Ver 14.14 Distrib 5.6.19, for osx10.9 (x86_64) using EditLine > wrapper > > The MySQL code used to create the user/password table. Which seems to work > fine, and creates the table as it should - even though the resulting > password doesn't appear to be quite right for SHA512-CRYPT > > INSERT INTO `mailserver`.`virtual_users` > (`id`, `domain_id`, `password` , `email`) > VALUES > ('1', '1', ENCRYPT('firstpassword', CONCAT('$6$', > SUBSTRING(SHA(RAND()), -16))), 'email1 at example.com'), > ('2', '1', ENCRYPT('secondpassword', CONCAT('$6$', > SUBSTRING(SHA(RAND()), -16))), 'email2 at example.com'); > > > What it actually creates is this: This is not a SHA512-CRYPT password?? > > mysql> SELECT * FROM mailserver.virtual_users; > > 3 | 1 | $6jxK9DWPBrAc | robert at zeus.example.com > > > Just to be clear,. this is the section of the Dovecot configs that deals > with handling this. BUT.. On the MAC-Mini, SHA512-CrYPT is unknown. > > # List of supported schemes is in > # http://wiki2.dovecot.org/Authentication/PasswordSchemes > # > default_pass_scheme = SHA512-CRYPT > > > So the question is, what Encryption should I be using, and is this an Apple > thing, or a MySQL thing? From reuben-dovecot at reub.net Sun May 3 10:48:40 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Sun, 3 May 2015 20:48:40 +1000 Subject: Dovecot Replication - Architecture Endianness? Message-ID: <5545FD08.2010701@reub.net> Hi all, I've had an interesting use case come up which - to cut the story short - one way to solve the problem I am looking at may be to replicate a small number of mailboxes to a third remote server. I've currently had replication running between my main dovecot machine and another remote VM for some time and working well (so I'm not new to replication and I've got a good working config), but I've a need to add a third to the mix for a select number of mailboxes. The arch on both of those is Gentoo x86_64 and with latest 2.1.16 -hg. I have attempted this so far by rsync'ing the initial Maildirs and then once the bulk of the data has been transferred rely on dovecot's replication to keep things in sync. I figure that this should in theory mean that the subsequent updates in both directions are incremental and the bulk of the data gets moved while the device is here on my desk using rsync. I've attempted to do this using a Raspberry Pi as a remote device, but when I set it up the dovecot replication process seems to need to start the replication over from scratch even after the rsync is done. I know this is happening as the disk utilisation on the Pi skyrockets once the replication starts and I end up with thousands of double ups of all the mails ... which defeats the entire point of the process. If I do an identical configuration but on a third Gentoo x86_64 VM locally it all works as expected. No double ups of mails and the "catchup" between the two devices is practically instant. Same filesystem even. The only difference appears to be the system architecture. So main my question is this. Is there a known architecture/endian limitation on replication? I guess cross-arch replication is not something many people try but is it supposed to work anyway? Has anyone else got replication working across different arch's? Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? Thanks, Reuben From yeupou at gnu.org Sun May 3 15:17:51 2015 From: yeupou at gnu.org (Mathieu ROY) Date: Sun, 03 May 2015 17:17:51 +0200 Subject: dsync-local(klink): Error: remote: dsync-server: invalid option -- 'l' Message-ID: <7455427.jZRFZDLv34@bender> Hello, Since recently (not sure when), dsync stopped working. dsync -u klink mirror XXXX dsync-local(klink): Error: remote: dsync-server: invalid option -- 'l' dsync-local(klink): Error: remote: doveadm dsync-server [-u |-A] [-S ] dsync-local(klink): Error: read() from worker server failed: EOF I did not change anything on my setup but dovecot was upgraded (debian stable). On the box I'm starting the command I have: $ dpkg --list | grep dovecot ii dovecot-common 1:2.1.7-7+deb7u1 all Transitional package for dovecot ii dovecot-core 1:2.1.7-7+deb7u1 amd64 secure mail server that supports mbox, maildir, dbox and mdbox mailboxes ii dovecot-imapd 1:2.1.7-7+deb7u1 amd64 secure IMAP server that supports mbox, maildir, dbox and mdbox mailboxes $ doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 7.8 mail_location = maildir:~/.Maildir:LAYOUT=fs:INBOX=~/.Maildir/INBOX mail_privileged_group = mail namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } protocols = " imap" ssl = required ssl_cert = References: <1430419859.1254.YahooMailBasic@web142406.mail.bf1.yahoo.com> Message-ID: <5546A2F7.70001@rename-it.nl> On 4/30/2015 8:50 PM, E.B. wrote: >> I switched from running my extprograms execute script directly >> to running with dovecot socket. Log shows only this >> >> dovecot: lmtp(test at example.com): Debug: wdi0Tb5VPlGfPnEAM/SpMA: sieve: action execute: running program: test >> dovecot: lmtp(test at example.com): Debug: Namespace : Using permissions from /vmail/example.com/test: mode=0770 gid=default >> dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected >> >> For testing I opened up the script and socket with >> permissions 777 but the error seems to indicate >> less about permissions more about some kind of >> protocol problem i guessing. > No one can help? Is anybody using the Sieve extprograms > execute via Dovecot socket service? I think my config > is vanilla, no? All other Dovecot and Sieve things > (including quota service scripts configured very > similarly) work fine. > > Taking a look at the code, the error seems to indicate > that no input is available on the socket when Dovecot > checks. Does my script need to behave differently? > What exactly to do? > > Should I just go back to direct execute? What's the > difference anyway beside the user/permissions will > be different? > > Stephan? Anyone? Last two changes should fix this: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/05f8ce7b5c2e http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 Regards, Stephan. From tss at iki.fi Mon May 4 08:59:34 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 May 2015 11:59:34 +0300 Subject: imapc fetch optimization In-Reply-To: <553FF242.80308@fsn.hu> References: <553FF242.80308@fsn.hu> Message-ID: <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> On 28 Apr 2015, at 23:49, Nagy, Attila wrote: > > Hi, > > imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines. > > I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails. From taeuber at bbaw.de Mon May 4 11:16:33 2015 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Mon, 4 May 2015 13:16:33 +0200 Subject: backup mails from alternate storage Message-ID: <20150504131633.81678f62f07203298cb90b09@bbaw.de> Hi there! It seems that # doveadm backup -v backup -fu $MUSER mdbox:"$BKPDIR/$MUSER/mdbox" doesn't backup all messages stored in alternate storage. mail_location = mdbox:~/mdbox:ALT=/srv/mail/slow/vmail/%n/mdbox I don't know how to debug this. The sizes of the mailboxes in the filesystems of user C are strange: ~/mdbox: # du -sm 2169 ALT: # du -sm 25252 backup: # du -sm 2658 This is with dovecot 2.2.9 on Ubuntu x86_64 14.04.2 LTS. Is this a bug or am I doing something wrong? How can I test if messages stored in the alternate storage are successfully backed up? Thanks Lars From yeupou at gnu.org Mon May 4 12:05:32 2015 From: yeupou at gnu.org (Mathieu ROY) Date: Mon, 04 May 2015 14:05:32 +0200 Subject: dsync-local(klink): Error: remote: dsync-server: invalid option -- 'l' In-Reply-To: <7455427.jZRFZDLv34@bender> References: <7455427.jZRFZDLv34@bender> Message-ID: <1602634.Pa2d8EUmSx@bender> Turns out dsync from 2.1.7 is not compatible with dsync from 2.2.13. -- http://yeupou.wordpress.com/ From taeuber at bbaw.de Mon May 4 12:47:17 2015 From: taeuber at bbaw.de (Lars =?UTF-8?B?VMOkdWJlcg==?=) Date: Mon, 4 May 2015 14:47:17 +0200 Subject: backup mails from alternate storage - all-clear In-Reply-To: <20150504131633.81678f62f07203298cb90b09@bbaw.de> References: <20150504131633.81678f62f07203298cb90b09@bbaw.de> Message-ID: <20150504144717.be6dde22b40872c398ef18f1@bbaw.de> Hi again! "du" was misleading me. Most files take 10M in filesystem but are actually only some byte or kilobytes in size. This is the result: # du -h --apparent-size 2,1G . # du -h 25G . So my conclusion the backup misses some messages was wrong. Sorry for the noise Lars Mon, 4 May 2015 13:16:33 +0200 Lars T?uber ==> dovecot at dovecot.org : > Hi there! > > It seems that > # doveadm backup -v backup -fu $MUSER mdbox:"$BKPDIR/$MUSER/mdbox" > doesn't backup all messages stored in alternate storage. > > mail_location = mdbox:~/mdbox:ALT=/srv/mail/slow/vmail/%n/mdbox > > I don't know how to debug this. > The sizes of the mailboxes in the filesystems of user C are strange: > > ~/mdbox: > # du -sm > 2169 > > ALT: > # du -sm > 25252 > > backup: > # du -sm > 2658 > > This is with dovecot 2.2.9 on Ubuntu x86_64 14.04.2 LTS. > Is this a bug or am I doing something wrong? > > How can I test if messages stored in the alternate storage are successfully backed up? > > Thanks > Lars From teemu.huovila at dovecot.fi Mon May 4 13:06:25 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 04 May 2015 16:06:25 +0300 Subject: Dovecot Replication - Architecture Endianness? In-Reply-To: <5545FD08.2010701@reub.net> References: <5545FD08.2010701@reub.net> Message-ID: <55476ED1.4050705@dovecot.fi> On 05/03/2015 01:48 PM, Reuben Farrelly wrote: > Hi all, > > I've had an interesting use case come up which - to cut the story short - one way to solve the problem I am looking at may be to > replicate a small number of mailboxes to a third remote server. > > I've currently had replication running between my main dovecot machine and another remote VM for some time and working well (so > I'm not new to replication and I've got a good working config), but I've a need to add a third to the mix for a select number of > mailboxes. The arch on both of those is Gentoo x86_64 and with latest 2.1.16 -hg. > > I have attempted this so far by rsync'ing the initial Maildirs and then once the bulk of the data has been transferred rely on > dovecot's replication to keep things in sync. I figure that this should in theory mean that the subsequent updates in both > directions are incremental and the bulk of the data gets moved while the device is here on my desk using rsync. > > I've attempted to do this using a Raspberry Pi as a remote device, but when I set it up the dovecot replication process seems to > need to start the replication over from scratch even after the rsync is done. I know this is happening as the disk utilisation > on the Pi skyrockets once the replication starts and I end up with thousands of double ups of all the mails ... which defeats > the entire point of the process. > > If I do an identical configuration but on a third Gentoo x86_64 VM locally it all works as expected. No double ups of mails and > the "catchup" between the two devices is practically instant. Same filesystem even. The only difference appears to be the > system architecture. > > So main my question is this. Is there a known architecture/endian limitation on replication? I guess cross-arch replication > is not something many people try but is it supposed to work anyway? I think you are bumping against Dovecot index endianess restrictions. I dont think cross-arch dsync can currently work very efficiently. http://wiki2.dovecot.org/Design/Indexes/MainIndex?highlight=%28endian%29 > Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? You can set the userdb to return an empty mail_replica variable for users you want to exclude from replication. http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 br, Teemu Huovila From teemu.huovila at dovecot.fi Mon May 4 13:12:56 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 04 May 2015 16:12:56 +0300 Subject: Crashes in dovecot -hg (86f535375750) In-Reply-To: <553F6432.1080100@reub.net> References: <553F6432.1080100@reub.net> Message-ID: <55477058.3030500@dovecot.fi> On 04/28/2015 01:42 PM, Reuben Farrelly wrote: > Seems there is some breakage with -hg latest - 2.2.16 (86f535375750+). I've just had 4 core files created in short succession on > both servers in the replication set. Here's the first... Does it work with 1081d57b524e or later? br, Teemu Huovila > tornado reuben # gdb /usr/libexec/dovecot/imap core > GNU gdb (Gentoo 7.9 vanilla) 7.9 > Copyright (C) 2015 Free Software Foundation, Inc. > License GPLv3+: GNU GPL version 3 or later > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. Type "show copying" > and "show warranty" for details. > This GDB was configured as "x86_64-pc-linux-gnu". > Type "show configuration" for configuration details. > For bug reporting instructions, please see: > . > Find the GDB manual and other documentation resources online at: > . > For help, type "help". > Type "apropos word" to search for commands related to "word"... > Reading symbols from /usr/libexec/dovecot/imap...done. > [New LWP 20929] > > warning: Could not load shared library symbols for linux-vdso.so.1. > Do you need "set solib-search-path" or "set sysroot"? > [Thread debugging using libthread_db enabled] > Using host libthread_db library "/lib64/libthread_db.so.1". > Core was generated by `dovecot/imap'. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x00007f186087693a in fts_user_free (fuser=0x0) at fts-user.c:187 > 187 fts-user.c: No such file or directory. > (gdb) bt full > #0 0x00007f186087693a in fts_user_free (fuser=0x0) at fts-user.c:187 > user_langp = 0x300000008 > #1 0x00007f1860876ac2 in fts_mail_user_deinit (user=0x20a3eb0) > at fts-user.c:215 > fuser = 0x0 > #2 0x00007f185d7890f8 in fts_lucene_mail_user_deinit (user=0x20a3eb0) > at fts-lucene-plugin.c:99 > fuser = 0x20a5550 > #3 0x00007f185d994e0c in replication_user_deinit (user=0x20a3eb0) > at replication-plugin.c:310 > ruser = 0x20a5500 > #4 0x00007f18615b565a in mail_user_unref (_user=0x20abc28) at mail-user.c:168 > user = 0x20a3eb0 > __FUNCTION__ = "mail_user_unref" > #5 0x000000000041afef in client_default_destroy (client=0x20abbb0, reason=0x0) > at imap-client.c:284 > cmd = 0x7ffde3a18960 > __FUNCTION__ = "client_default_destroy" > #6 0x000000000041ada0 in client_destroy (client=0x20abbb0, reason=0x0) > at imap-client.c:236 > No locals. > #7 0x000000000041ccf4 in client_input (client=0x20abbb0) at imap-client.c:967 > cmd = 0x7ffde3a189a0 > output = 0x0 > bytes = 12 > __FUNCTION__ = "client_input" > #8 0x00007f18612fc992 in io_loop_call_io (io=0x20c8610) at ioloop.c:501 > ioloop = 0x2076740 > t_id = 2 > __FUNCTION__ = "io_loop_call_io" > #9 0x00007f18612fec40 in io_loop_handler_run_internal (ioloop=0x2076740) > at ioloop-epoll.c:220 > ctx = 0x2077460 > events = 0x2078290 > event = 0x2078290 > list = 0x2078e80 > io = 0x20c8610 > tv = {tv_sec = 4, tv_usec = 999387} > events_count = 5 > msecs = 5000 > ret = 1 > i = 0 > j = 0 > call = true > __FUNCTION__ = "io_loop_handler_run_internal" > #10 0x00007f18612fcb2f in io_loop_handler_run (ioloop=0x2076740) > > Reuben From teemu.huovila at dovecot.fi Mon May 4 13:27:46 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 04 May 2015 16:27:46 +0300 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: References: Message-ID: <554773D2.9060609@dovecot.fi> On 04/24/2015 10:00 PM, Michael Grimm wrote: > Hi ? > > 1) I'm trying to compile a recent hg dovecot version (008632bdfd2c) at a FBSD10-STABLE system without success: > > libtool: compile: cc -DHAVE_CONFIG_H -I. -I../.. -I../../src/lib -I../../src/lib-test -I/usr/local/include -DUDHRDIR=\"../../src/lib-fts\" -DDATADIR=\"/usr/local/share/dovecot\" -DTEST_STOPWORDS_DIR=\"../../src/lib-fts\" -I/usr/local/include -std=gnu99 -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -Wno-duplicate-decl-specifier -Wstrict-aliasing=2 -I/usr/local/include -MT fts-tokenizer-generic.lo -MD -MP -MF .deps/fts-tokenizer-generic.Tpo -c fts-tokenizer-generic.c -fPIC -DPIC -o .libs/fts-tokenizer-generic.o > fts-tokenizer-generic.c:111:18: error: use of undeclared identifier 'White_Space' > if (uint32_find(White_Space, N_ELEMENTS(White_Space), c, &idx)) > ^ > fts-tokenizer-generic.c:113:18: error: use of undeclared identifier 'Dash' > if (uint32_find(Dash, N_ELEMENTS(Dash), c, &idx)) > ^ > [?] > > fts-tokenizer-generic.c:212:18: error: use of undeclared identifier 'MidLetter' > if (uint32_find(MidLetter, N_ELEMENTS(MidLetter), c, &idx)) > ^ > fts-tokenizer-generic.c:214:18: error: use of undeclared identifier 'MidNum' > if (uint32_find(MidNum, N_ELEMENTS(MidNum), c, &idx)) > ^ > fatal error: too many errors emitted, stopping now [-ferror-limit=] > 20 errors generated. > Makefile:591: recipe for target 'fts-tokenizer-generic.lo' failed > gmake[4]: *** [fts-tokenizer-generic.lo] Error 1 > gmake[4]: Leaving directory '/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src/lib-fts' > > > 2) I don't have a python binary installed, only a python2 link to the python27 binary (FBSD, and python27 from ports). > Thus, update-version.sh will fail to evaluate hg's changeset. As a quick fix I needed to create a link: python -> python2 Both these are only run if you compile the source from hg, as you did. Official release tar-balls should not have this issue. Still, it is not optimal and Ill definitely look into solving 1) when I have time available for that. For temporarily solving 1) it is worth noticing the scripts word-break-data.sh and word-boundary-data.sh depend on /bin/bash. You could either install bash or just try if it works if you change it to /bin/sh and use whatever FreeBSD has that pointing to. br, Teemu Huovila From teemu.huovila at dovecot.fi Mon May 4 13:38:27 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Mon, 04 May 2015 16:38:27 +0300 Subject: SQLite does not depend on zlib, was: Re: [PATCH] Split sql drivers from lib-sql to plugins In-Reply-To: References: Message-ID: <55477653.9060009@dovecot.fi> On 04/22/2015 10:19 PM, Bernd Kuhls wrote: > Bernd Kuhls wrote in > news:XnsA3DF68DCAEF69berndkuhlsPkbjNfxxIA at bernd-kuhls.de: > >> Tomas Janousek wrote in news:20070413132731.GA8281 >> @redhat.com: >> >>> - SQL_LIBS="$SQL_LIBS -lsqlite3 -lz" >>> + SQLITE_LIBS="$SQLITE_LIBS -lsqlite3 -lz" >> >> Hi, >> >> this patch fixes a build error during cross compilation to a system without >> the libz target package: >> >> --- dovecot-2.2.15.org/configure.ac 2014-10-25 05:57:08.000000000 +0200 >> +++ dovecot-2.2.15/configure.ac 2014-11-08 10:06:23.015570150 +0100 >> @@ -2293,7 +2293,7 @@ >> if test $want_sqlite != no; then >> AC_CHECK_LIB(sqlite3, sqlite3_open, [ >> AC_CHECK_HEADER(sqlite3.h, [ >> - SQLITE_LIBS="$SQLITE_LIBS -lsqlite3 -lz" >> + SQLITE_LIBS="$SQLITE_LIBS -lsqlite3" >> >> AC_DEFINE(HAVE_SQLITE,, Build with SQLite3 support) >> found_sql_drivers="$found_sql_drivers sqlite" >> >> Regards, Bernd >> >> > > ping ;) > Thank you for the report http://hg.dovecot.org/dovecot-2.2/rev/e4ad83ed88c9 br, Teemu Huovila From reuben-dovecot at reub.net Mon May 4 13:45:08 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Mon, 4 May 2015 23:45:08 +1000 Subject: Crashes in dovecot -hg (86f535375750) In-Reply-To: <55477058.3030500@dovecot.fi> References: <553F6432.1080100@reub.net> <55477058.3030500@dovecot.fi> Message-ID: <554777E4.1030302@reub.net> On 4/05/2015 11:12 PM, Teemu Huovila wrote: > On 04/28/2015 01:42 PM, Reuben Farrelly wrote: >> Seems there is some breakage with -hg latest - 2.2.16 (86f535375750+). I've just had 4 core files created in short succession on >> both servers in the replication set. Here's the first... > Does it work with 1081d57b524e or later? > > br, > Teemu Huovila Yes - all good since that fix went in (I saw the commit and figured it was a fix for my crash). Haven't had a crash for a few days since so we can close that off now as "fixed". Thanks, Reuben From bra at fsn.hu Mon May 4 13:59:02 2015 From: bra at fsn.hu (Nagy, Attila) Date: Mon, 04 May 2015 15:59:02 +0200 Subject: imapc fetch optimization In-Reply-To: <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> References: <553FF242.80308@fsn.hu> <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> Message-ID: <55477B26.3090507@fsn.hu> Hi, On 05/04/15 10:59, Timo Sirainen wrote: > On 28 Apr 2015, at 23:49, Nagy, Attila wrote: >> Hi, >> >> imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines. >> >> I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? > Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails. > I'm aware of that, but it doesn't, or at least not always. For example Thunderbird issues this: 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) Dovecot does this to the IMAP backend 18 UID FETCH 333574 (BODY.PEEK[]) 19 UID FETCH 333575 (BODY.PEEK[]) 20 UID FETCH 333576 (BODY.PEEK[]) 21 UID FETCH 333577 (BODY.PEEK[]) And so on. I have mail_prefetch_count set to 100: # doveconf -n | grep prefet mail_prefetch_count = 100 From reuben-dovecot at reub.net Mon May 4 14:11:11 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Tue, 5 May 2015 00:11:11 +1000 Subject: Dovecot Replication - Architecture Endianness? In-Reply-To: <55476ED1.4050705@dovecot.fi> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> Message-ID: <55477DFF.9060000@reub.net> On 4/05/2015 11:06 PM, Teemu Huovila wrote: > On 05/03/2015 01:48 PM, Reuben Farrelly wrote: >> Hi all, >> >> I've had an interesting use case come up which - to cut the story short - one way to solve the problem I am looking at may be to >> replicate a small number of mailboxes to a third remote server. >> >> I've currently had replication running between my main dovecot machine and another remote VM for some time and working well (so >> I'm not new to replication and I've got a good working config), but I've a need to add a third to the mix for a select number of >> mailboxes. The arch on both of those is Gentoo x86_64 and with latest 2.1.16 -hg. >> >> I have attempted this so far by rsync'ing the initial Maildirs and then once the bulk of the data has been transferred rely on >> dovecot's replication to keep things in sync. I figure that this should in theory mean that the subsequent updates in both >> directions are incremental and the bulk of the data gets moved while the device is here on my desk using rsync. >> >> I've attempted to do this using a Raspberry Pi as a remote device, but when I set it up the dovecot replication process seems to >> need to start the replication over from scratch even after the rsync is done. I know this is happening as the disk utilisation >> on the Pi skyrockets once the replication starts and I end up with thousands of double ups of all the mails ... which defeats >> the entire point of the process. >> >> If I do an identical configuration but on a third Gentoo x86_64 VM locally it all works as expected. No double ups of mails and >> the "catchup" between the two devices is practically instant. Same filesystem even. The only difference appears to be the >> system architecture. >> >> So main my question is this. Is there a known architecture/endian limitation on replication? I guess cross-arch replication >> is not something many people try but is it supposed to work anyway? > I think you are bumping against Dovecot index endianess restrictions. I dont think cross-arch dsync can currently work very > efficiently. > http://wiki2.dovecot.org/Design/Indexes/MainIndex?highlight=%28endian%29 Ok. That explains why the rsync won't work. But if I kick off a dovecot to dovecot replication (without doing the rsync first) will this work any better once the system catches up? This assumes (possibly incorrectly - please correct me if I am wrong) that the index files themselves aren't dsync'd byte-by-byte but instead the metadata/content from them is sent instead, and then the indexes are written to remote disk by the remote dovecot in the right arch and format that the remote machine can read and understand. Because if that's the case then I can probably make that work - just taking a hit on the initial sync which could take longer. Even if this doesn't end up working I figure I'll get to learn a little more about the indexes themselves in the process. Thanks for any advice, Reuben From tss at iki.fi Mon May 4 15:45:42 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 May 2015 18:45:42 +0300 Subject: imapc fetch optimization In-Reply-To: <55477B26.3090507@fsn.hu> References: <553FF242.80308@fsn.hu> <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> <55477B26.3090507@fsn.hu> Message-ID: On 04 May 2015, at 16:59, Nagy, Attila wrote: > > Hi, > > On 05/04/15 10:59, Timo Sirainen wrote: >> On 28 Apr 2015, at 23:49, Nagy, Attila wrote: >>> Hi, >>> >>> imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines. >>> >>> I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? >> Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails. >> > I'm aware of that, but it doesn't, or at least not always. > For example Thunderbird issues this: > 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) > Dovecot does this to the IMAP backend > 18 UID FETCH 333574 (BODY.PEEK[]) Oh, there were several bugs related to that. Fixed: http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b From tss at iki.fi Mon May 4 15:47:44 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 4 May 2015 18:47:44 +0300 Subject: imapc fetch optimization In-Reply-To: References: <553FF242.80308@fsn.hu> <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> <55477B26.3090507@fsn.hu> Message-ID: On 04 May 2015, at 18:45, Timo Sirainen wrote: > > On 04 May 2015, at 16:59, Nagy, Attila wrote: >> >> Hi, >> >> On 05/04/15 10:59, Timo Sirainen wrote: >>> On 28 Apr 2015, at 23:49, Nagy, Attila wrote: >>>> Hi, >>>> >>>> imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines. >>>> >>>> I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? >>> Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails. >>> >> I'm aware of that, but it doesn't, or at least not always. >> For example Thunderbird issues this: >> 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) >> Dovecot does this to the IMAP backend >> 18 UID FETCH 333574 (BODY.PEEK[]) > > Oh, there were several bugs related to that. Fixed: > > http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc > http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 > http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b Also if you want imapc to use the full set of features with latest Dovecot, use: imapc_features=rfc822.size fetch-headers search After that it shouldn't be fetching BODY.PEEK[] anymore unless the body was actually wanted to be fetched. From ya.mwork at yandex.ru Mon May 4 17:30:32 2015 From: ya.mwork at yandex.ru (=?UTF-8?B?0JHQsNGB0L7QsiDQldCy0LPQtdC90LjQuQ==?=) Date: Mon, 04 May 2015 20:30:32 +0300 Subject: Dovecot 2.2.16 doesn't rotate mdbox files Message-ID: <5547ACB8.6020408@yandex.ru> Hello, there's setting in my config file: mail_location = mdbox:/var/vmail/%Ld/%Ln:INDEX=/var/indexes/%Ld/%Ln mdbox_rotate_size = 1M mdbox_preallocate_space = yes mdbox_rotate_interval = 1d As I understand, this settings means that file which contains letters with refcount=0 will be rotated after 23:59 today. I remember letter ID and execute grep whith this one in storage directory. Letter was presented. After that, I removed letter over IMAP protocol, checked and find it again. On the next day check with grep was repeated and letter was finded. After doveadm purge -u user at domain letter was expunged. What's wrong in my configuration? From bra at fsn.hu Mon May 4 17:33:00 2015 From: bra at fsn.hu (Nagy, Attila) Date: Mon, 04 May 2015 19:33:00 +0200 Subject: imapc fetch optimization In-Reply-To: References: <553FF242.80308@fsn.hu> <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> <55477B26.3090507@fsn.hu> Message-ID: <5547AD4C.5030507@fsn.hu> On 05/04/15 17:45, Timo Sirainen wrote >> I'm aware of that, but it doesn't, or at least not always. >> For example Thunderbird issues this: >> 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) >> Dovecot does this to the IMAP backend >> 18 UID FETCH 333574 (BODY.PEEK[]) > Oh, there were several bugs related to that. Fixed: > > http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc > http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 > http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b Thank you very much for the quick fix(es), I will try them. From bra at fsn.hu Mon May 4 17:40:51 2015 From: bra at fsn.hu (Nagy, Attila) Date: Mon, 04 May 2015 19:40:51 +0200 Subject: imapc fetch optimization In-Reply-To: References: <553FF242.80308@fsn.hu> <8D901FAD-3F75-456B-A30D-4385E2235C4C@iki.fi> <55477B26.3090507@fsn.hu> Message-ID: <5547AF23.7020704@fsn.hu> On 05/04/15 17:47, Timo Sirainen wrote: > Also if you want imapc to use the full set of features with latest > Dovecot, use: imapc_features=rfc822.size fetch-headers search After > that it shouldn't be fetching BODY.PEEK[] anymore unless the body was > actually wanted to be fetched. Very nice, and also the path you follow is great: allow the backend IMAP server to be dumb and yet -with these feature flags- we can exploit it more if needed. And having the ability to deliver mails through lmtpd to the IMAP backend is priceless. :) From sanvila at unex.es Mon May 4 19:04:01 2015 From: sanvila at unex.es (Santiago Vila) Date: Mon, 4 May 2015 21:04:01 +0200 (CEST) Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: References: Message-ID: Greetings. Thanks to Jelmer Vernooij, who has just uploaded version 2.2.16 for Debian unstable, I can confirm that this bug may still be reproduced in version 2.2.16, so it has *not* been fixed yet. Several months ago I was told "I can't reproduce it in dovecot hg", or something like that. Well, having this fixed in hg does not help very much if the bug is still present in the latest released version (2.2.16). Would anyone here please tell which commit exactly fixed this issue? If it's not fixed yet, I would love to provide a patch, but I'm not a programmer myself. However, I provided a very precise and detailed explanation about how to reproduce this bug, which you will still find in the Debian BTS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776094 Really, it should not be so much difficult to reproduce it. Try on a Debian 8 system. I'm sure that if you follow the steps exactly as I described, you will find the problem the same way as I did. Thanks. From rs at sys4.de Mon May 4 19:23:57 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 04 May 2015 21:23:57 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: References: Message-ID: <5547C74D.4020806@sys4.de> Am 04.05.2015 um 21:04 schrieb Santiago Vila: > Greetings. > > Thanks to Jelmer Vernooij, who has just uploaded version 2.2.16 for > Debian unstable, I can confirm that this bug may still be reproduced > in version 2.2.16, so it has *not* been fixed yet. > > Several months ago I was told "I can't reproduce it in dovecot hg", or > something like that. Well, having this fixed in hg does not help very > much if the bug is still present in the latest released version (2.2.16). > > Would anyone here please tell which commit exactly fixed this issue? > > If it's not fixed yet, I would love to provide a patch, but I'm not a > programmer myself. > > However, I provided a very precise and detailed explanation about how > to reproduce this bug, which you will still find in the Debian BTS: > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=776094 > > Really, it should not be so much difficult to reproduce it. Try on a > Debian 8 system. I'm sure that if you follow the steps exactly as I > described, you will find the problem the same way as I did. > > Thanks. > Boh fetchmail .... did you verified with getmail ? http://pyropus.ca/software/getmail/ dig you tried fetchmail with bad-header accept ? what are dovecot verbose logs tell Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From user+dovecot at localhost.localdomain.org Mon May 4 19:49:58 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 04 May 2015 19:49:58 +0000 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: References: Message-ID: <5547CD66.7040904@localhost.localdomain.org> On 04/24/2015 07:00 PM, Michael Grimm wrote: > ... > 2) I don't have a python binary installed, only a python2 link to the python27 binary (FBSD, and python27 from ports). > Thus, update-version.sh will fail to evaluate hg's changeset. As a quick fix I needed to create a link: python -> python2 > This is a less or more 'common' system configuration error. ;-) See https://forums.freebsd.org/threads/solved-python-command-not-found.46449/ Regards, Pascal -- The trapper recommends today: cafefeed.1512421 at localdomain.org From sanvila at unex.es Mon May 4 20:13:51 2015 From: sanvila at unex.es (Santiago Vila) Date: Mon, 4 May 2015 22:13:51 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5547C74D.4020806@sys4.de> References: <5547C74D.4020806@sys4.de> Message-ID: <20150504201351.GA18542@nuc> On Mon, May 04, 2015 at 09:23:57PM +0200, Robert Schetterer wrote: > Boh fetchmail .... did you verified with getmail ? No, I didn't. The client should be irrelevant. Nobody should be able to corrupt a remote mailbox only by issuing imap commands. BTW: Does getmail have an option doing the same as fetchmail --folder option at all? > dig you tried fetchmail with > > bad-header accept ? I appreciate that you are trying to help, but it seems to me that you didn't read the report, or you didn't understand it. Answer: No. I didn't. Why should I? The mbox was fine before I tried to retrieve it. Please read the report! From rs at sys4.de Mon May 4 21:43:23 2015 From: rs at sys4.de (Robert Schetterer) Date: Mon, 04 May 2015 23:43:23 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150504201351.GA18542@nuc> References: <5547C74D.4020806@sys4.de> <20150504201351.GA18542@nuc> Message-ID: <5547E7FB.6030906@sys4.de> Am 04.05.2015 um 22:13 schrieb Santiago Vila: > On Mon, May 04, 2015 at 09:23:57PM +0200, Robert Schetterer wrote: >> Boh fetchmail .... did you verified with getmail ? > > No, I didn't. The client should be irrelevant. Nobody should be able > to corrupt a remote mailbox only by issuing imap commands. > > BTW: Does getmail have an option doing the same as fetchmail --folder > option at all? http://pyropus.ca/software/getmail/configuration.html#retriever-parameters All IMAP retriever types also take the following optional parameters: mailboxes (tuple of quoted strings) ? a list of mailbox paths to retrieve mail from, expressed as a Python tuple. If not specified, the default is to retrieve mail from the mail folder named INBOX. You might want to retrieve messages from several different mail folders, using a configuration like this: mailboxes = ("INBOX", "INBOX.spam", "mailing-lists.model-railroading") > >> dig you tried fetchmail with >> >> bad-header accept ? > > I appreciate that you are trying to help, but it seems to me that you > didn't read the report, or you didn't understand it. > > Answer: No. I didn't. Why should I? The mbox was fine before I tried > to retrieve it. Please read the report! > sorry i wont invest any time in fetchmail, i quit with it years ago by tons of problems i managed to forget Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From pfaf at wisdomsoftware.net Mon May 4 22:17:46 2015 From: pfaf at wisdomsoftware.net (Panayiotis Fafakos) Date: Tue, 05 May 2015 01:17:46 +0300 Subject: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2.13 Message-ID: <5547F00A.8050001@wisdomsoftware.net> Dear all, we have succesfully configured Dovecot v2.2.13 in debian wheezy 7.8 (using backports) using Maildir structure, to use private index files for the \Seen flag on a per user basis. All users access their emails and Public Folders using IMAP protocol. The problem is that when a user moves an email from publicFolderA to publicFolderB under the same namespace the other users see this message as unread, although they have actually read it when it was in publicFolderA. Please note that this is an old message which has been moved , it was not copied, so the actual UID should be the same... Is there a way to keep the \Seen flag for the messages that are moved from folder to folder? Is there a way to keep the \Seen flag in a database, so that we can ignore the folder structure and only check the message UIDs? We could use MySQL, PgSQL or even SQLite... Below follows the Public-Folder namespace declaration: ------------------------------------------ namespace { inbox = no location = maildir:/var/vmail/Public-Folders:LAYOUT=fs:INDEXPVT=~/Maildir/public/%u prefix = Public-Folders/ separator = / subscriptions = no type = public } ------------------------------------------ With the above system configuration we have the complete folder structure under ~/Maildir/public/%u, and many log files, one for each folder a user has accessed. Could we only have one index file for each user for all the public folder structure under the same namespace? Kind regards to all, Panayiotis Fafakos From sanvila at unex.es Tue May 5 00:05:10 2015 From: sanvila at unex.es (Santiago Vila) Date: Tue, 5 May 2015 02:05:10 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5547E7FB.6030906@sys4.de> References: <5547C74D.4020806@sys4.de> <20150504201351.GA18542@nuc> <5547E7FB.6030906@sys4.de> Message-ID: <20150505000510.GA20169@cantor.unex.es> On Mon, May 04, 2015 at 11:43:23PM +0200, Robert Schetterer wrote: > > BTW: Does getmail have an option doing the same as fetchmail --folder > > option at all? > > http://pyropus.ca/software/getmail/configuration.html#retriever-parameters > > All IMAP retriever types also take the following optional parameters: > > mailboxes (tuple of quoted strings) ? a list of mailbox paths to > retrieve mail from, expressed as a Python tuple. If not specified, the > default is to retrieve mail from the mail folder named INBOX. You might > want to retrieve messages from several different mail folders, using a > configuration like this: > > mailboxes = ("INBOX", "INBOX.spam", > "mailing-lists.model-railroading") Fine, if I wanted to use getmail I would have to use this: mailboxes = ("inbox-b",) > >> dig you tried fetchmail with > >> > >> bad-header accept ? > > > > I appreciate that you are trying to help, but it seems to me that you > > didn't read the report, or you didn't understand it. > > > > Answer: No. I didn't. Why should I? The mbox was fine before I tried > > to retrieve it. Please read the report! > > > > sorry i wont invest any time in fetchmail, i quit with it years ago > by tons of problems i managed to forget But I'm not reporting a bug in fetchmail, I'm reporting a bug in dovecot. Everything fetchmail does is to issue IMAP commands to dovecot, and it's those IMAP commands what make the remote folder to be corrupted. Please tell me: Is there any set of forbidden IMAP commands that one "should not use" because of them being known to cause email corruption in the server? The corruption happens in the server running dovecot, not in the client running fetchmail. If I describe this problem with fetchmail and dovecot in the same machine is only for simpliciry about reproducing it. From emailbuilder88 at yahoo.com Tue May 5 04:59:32 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Mon, 4 May 2015 21:59:32 -0700 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <5546A2F7.70001@rename-it.nl> Message-ID: <1430801972.80312.YahooMailBasic@web142402.mail.bf1.yahoo.com> > >> I switched from running my extprograms execute script directly > >> to running with dovecot socket. Log shows only this > >> > >> dovecot: lmtp(test at example.com): Debug: wdi0Tb5VPlGfPnEAM/SpMA: sieve: action execute: running program: test > >> dovecot: lmtp(test at example.com): Debug: Namespace : Using permissions from /vmail/example.com/test: mode=0770 gid=default > >> dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > >> > >> For testing I opened up the script and socket with > >> permissions 777 but the error seems to indicate > >> less about permissions more about some kind of > >> protocol problem i guessing. > > No one can help? Is anybody using the Sieve extprograms > > execute via Dovecot socket service? I think my config > > is vanilla, no? All other Dovecot and Sieve things > > (including quota service scripts configured very > > similarly) work fine. > > > > Taking a look at the code, the error seems to indicate > > that no input is available on the socket when Dovecot > > checks. Does my script need to behave differently? > > What exactly to do? > > > > Should I just go back to direct execute? What's the > > difference anyway beside the user/permissions will > > be different? > > > > Stephan? Anyone? > > Last two changes should fix this: Not yet -- this may be unrelated(?) but here is what I have after installing the newest source package: Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: mail_deliver_ctx_get_log_var_expand_table Help? From emailbuilder88 at yahoo.com Tue May 5 06:39:09 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Mon, 4 May 2015 23:39:09 -0700 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430801972.80312.YahooMailBasic@web142402.mail.bf1.yahoo.com> Message-ID: <1430807949.39081.YahooMailBasic@web142402.mail.bf1.yahoo.com> > > Last two changes should fix this: > > > Not yet -- this may be unrelated(?) but here is what I have > after installing the newest source package: > > Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: > dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: > mail_deliver_ctx_get_log_var_expand_table Oh i guessing this requires the newest Dovecot source too (maybe you should put notice on the wiki that extprograms run-via-socket is broken until a new release?) After installed newest Dovecot and Sieve, I getting this: dovecot: lmtp(test at example.com): Debug: auth input: dovecot: lmtp(testl at example.com): Fatal: master: service(lmtp): child 7033 killed with signal 11 (core dumps disabled) With set mail_debug=yes doesn't give anything else of interesting for this. From stephan at rename-it.nl Tue May 5 07:00:14 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 05 May 2015 09:00:14 +0200 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430807949.39081.YahooMailBasic@web142402.mail.bf1.yahoo.com> References: <1430807949.39081.YahooMailBasic@web142402.mail.bf1.yahoo.com> Message-ID: <55486A7E.40502@rename-it.nl> On 5/5/2015 8:39 AM, E.B. wrote: >>> Last two changes should fix this: >> >> Not yet -- this may be unrelated(?) but here is what I have >> after installing the newest source package: >> >> Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: >> dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: >> mail_deliver_ctx_get_log_var_expand_table > Oh i guessing this requires the newest Dovecot source too (maybe you > should put notice on the wiki that extprograms run-via-socket is broken until > a new release?) > > After installed newest Dovecot and Sieve, I getting this: > > dovecot: lmtp(test at example.com): Debug: auth input: > dovecot: lmtp(testl at example.com): Fatal: master: service(lmtp): child 7033 killed with signal 11 (core dumps disabled) > > With set mail_debug=yes doesn't give anything else of interesting for this. Could you create a backtrace (from core dump; http://www.dovecot.org/bugreport.html)? Regards, Stephan. From emailbuilder88 at yahoo.com Tue May 5 07:40:55 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 5 May 2015 00:40:55 -0700 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430807949.39081.YahooMailBasic@web142402.mail.bf1.yahoo.com> Message-ID: <1430811655.90181.YahooMailBasic@web142405.mail.bf1.yahoo.com> > > > Last two changes should fix this: > > > > > > Not yet -- this may be unrelated(?) but here is what I have > > after installing the newest source package: > > > > Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: > > dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: > > mail_deliver_ctx_get_log_var_expand_table > > Oh i guessing this requires the newest Dovecot source too (maybe you > should put notice on the wiki that extprograms run-via-socket is broken until > a new release?) > > After installed newest Dovecot and Sieve, I getting this: > > dovecot: lmtp(test at example.com): Debug: auth input: > dovecot: lmtp( > testl at example.com): Fatal: master: service(lmtp): child > 7033 killed with signal 11 (core dumps disabled) > > With set mail_debug=yes doesn't give anything else of interesting for this. Sorrys. I decided to "make clean" and do new "./configure" "make" and "make install" for dovecot first, then pigeonhole to make sure nothing is off. After I do that I get same original error: dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected I went to go verifying in dovecot-pigeonhole that the two new patches are there by manual inspect the source files you patched. I found only the first one, not http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this because I downloaded a snap shot link at the top that wasn't built with the 2nd change? I put 2nd change in manually (its only one liner) rebuilded and after install..... same error with recv(MSG_PEEK) Thanks for ongoing help From stephan at rename-it.nl Tue May 5 08:12:26 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 05 May 2015 10:12:26 +0200 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430811655.90181.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1430811655.90181.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <55487B6A.5040905@rename-it.nl> E.B. schreef op 5-5-2015 om 9:40: >>>> Last two changes should fix this: >>> >>> Not yet -- this may be unrelated(?) but here is what I have >>> after installing the newest source package: >>> >>> Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: >>> dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: >>> mail_deliver_ctx_get_log_var_expand_table >> Oh i guessing this requires the newest Dovecot source too (maybe you >> should put notice on the wiki that extprograms run-via-socket is broken until >> a new release?) >> >> After installed newest Dovecot and Sieve, I getting this: >> >> dovecot: lmtp(test at example.com): Debug: auth input: >> dovecot: lmtp( >> testl at example.com): Fatal: master: service(lmtp): child >> 7033 killed with signal 11 (core dumps disabled) >> >> With set mail_debug=yes doesn't give anything else of interesting for this. > Sorrys. I decided to "make clean" and do new "./configure" "make" and "make > install" for dovecot first, then pigeonhole to make sure nothing is off. After > I do that I get same original error: > > dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > > I went to go verifying in dovecot-pigeonhole that the two new patches are there > by manual inspect the source files you patched. I found only the first one, not > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this > because I downloaded a snap shot link at the top that wasn't built with the > 2nd change? > > I put 2nd change in manually (its only one liner) rebuilded and after install..... > same error with recv(MSG_PEEK) Did you restart Dovecot? Also, can you try running the script through `sieve-test -D -t - -Tlevel=matching` (see man page). Regards, Stephan. From stephan at rename-it.nl Tue May 5 08:52:53 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 05 May 2015 10:52:53 +0200 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430811655.90181.YahooMailBasic@web142405.mail.bf1.yahoo.com> References: <1430811655.90181.YahooMailBasic@web142405.mail.bf1.yahoo.com> Message-ID: <554884E5.6070002@rename-it.nl> E.B. schreef op 5-5-2015 om 9:40: > Sorrys. I decided to "make clean" and do new "./configure" "make" and "make > install" for dovecot first, then pigeonhole to make sure nothing is off. After > I do that I get same original error: > > dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > > I went to go verifying in dovecot-pigeonhole that the two new patches are there > by manual inspect the source files you patched. I found only the first one, not > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this > because I downloaded a snap shot link at the top that wasn't built with the > 2nd change? > > I put 2nd change in manually (its only one liner) rebuilded and after install..... > same error with recv(MSG_PEEK) > > Thanks for ongoing help Argh! Ok, that fix was a bit incomplete still: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/89e0cef5b264 I wonder why my test didn't fail. The extent of this bug is limited to using execute as a command (as opposed to using it as a test in an `if' statement) and not providing it with any input. Regards, Stephan. From marcin at mejor.pl Tue May 5 09:48:46 2015 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Tue, 05 May 2015 11:48:46 +0200 Subject: [sieve] extracting address behavior changes beetwen 2.2.13 and 2.2.16 Message-ID: <554891FE.3040308@mejor.pl> Hi all! After upgrading dovecot from 2.2.13 to 2.2.16 I noticed that my sieve filters doesn't catch some emails. I used sieve-test with tracing (it's great tool!) and I can see as below: part of sieve filter: [...] elsif address :is "From" "backup at areda.pl" { [...] dovecot-2.2.13: 36: address test 36: starting `:is' match with `i;ascii-casemap' comparator: 36: extracting `From' headers from message 36: parsing address header value `=?UTF-8?B?QmFjdWxhQEFUTQ==?= ' 36: extracting `all' part from address `backup at areda.pl' 36: matching value `backup at areda.pl' 36: with key `backup at areda.pl' => 1 36: finishing match with result: matched 36: jump if result is false 36: not jumping dovecot-2.2.16: 36: address test 36: starting `:is' match with `i;ascii-casemap' comparator: 36: extracting `From' headers from message 36: parsing address header value `Bacula at ATM ' 36: extracting `all' part from address `Bacula at ATM' 36: matching value `Bacula at ATM' 36: with key `backup at areda.pl' => 0 36: finishing match with result: not matched 36: jump if result is false 36: jumping to line 43 So in newer dovecot sieve uses comment not exact email address. Did I wrongly wrote my sieve script? Thanks, Marcin. From stephan at rename-it.nl Tue May 5 11:46:14 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 05 May 2015 13:46:14 +0200 Subject: [sieve] extracting address behavior changes beetwen 2.2.13 and 2.2.16 In-Reply-To: <554891FE.3040308@mejor.pl> References: <554891FE.3040308@mejor.pl> Message-ID: <5548AD86.4050306@rename-it.nl> Marcin Miros?aw schreef op 5-5-2015 om 11:48: > Hi all! > > After upgrading dovecot from 2.2.13 to 2.2.16 I noticed that my sieve > filters doesn't catch some emails. I used sieve-test with tracing (it's > great tool!) and I can see as below: > part of sieve filter: > [...] > elsif address :is "From" "backup at areda.pl" { > [...] > > dovecot-2.2.13: > 36: address test > 36: starting `:is' match with `i;ascii-casemap' comparator: > 36: extracting `From' headers from message > 36: parsing address header value `=?UTF-8?B?QmFjdWxhQEFUTQ==?= > ' > 36: extracting `all' part from address `backup at areda.pl' > 36: matching value `backup at areda.pl' > 36: with key `backup at areda.pl' => 1 > 36: finishing match with result: matched > 36: jump if result is false > 36: not jumping > > dovecot-2.2.16: > 36: address test > 36: starting `:is' match with `i;ascii-casemap' comparator: > 36: extracting `From' headers from message > 36: parsing address header value `Bacula at ATM ' > 36: extracting `all' part from address `Bacula at ATM' > 36: matching value `Bacula at ATM' > 36: with key `backup at areda.pl' => 0 > 36: finishing match with result: not matched > 36: jump if result is false > 36: jumping to line 43 > > So in newer dovecot sieve uses comment not exact email address. Did I > wrongly wrote my sieve script? Fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/a3d26f12c2cd Regards, Stephan. From marcin at mejor.pl Tue May 5 11:49:54 2015 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Tue, 05 May 2015 13:49:54 +0200 Subject: [sieve] extracting address behavior changes beetwen 2.2.13 and 2.2.16 In-Reply-To: <5548AD86.4050306@rename-it.nl> References: <554891FE.3040308@mejor.pl> <5548AD86.4050306@rename-it.nl> Message-ID: <5548AE62.2010701@mejor.pl> W dniu 05.05.2015 o 13:46, Stephan Bosch pisze: > > > Marcin Miros?aw schreef op 5-5-2015 om 11:48: >> Hi all! >> >> After upgrading dovecot from 2.2.13 to 2.2.16 I noticed that my sieve >> filters doesn't catch some emails. I used sieve-test with tracing (it's >> great tool!) and I can see as below: >> part of sieve filter: >> [...] >> elsif address :is "From" "backup at areda.pl" { >> [...] >> >> dovecot-2.2.13: >> 36: address test >> 36: starting `:is' match with `i;ascii-casemap' comparator: >> 36: extracting `From' headers from message >> 36: parsing address header value `=?UTF-8?B?QmFjdWxhQEFUTQ==?= >> ' >> 36: extracting `all' part from address `backup at areda.pl' >> 36: matching value `backup at areda.pl' >> 36: with key `backup at areda.pl' => 1 >> 36: finishing match with result: matched >> 36: jump if result is false >> 36: not jumping >> >> dovecot-2.2.16: >> 36: address test >> 36: starting `:is' match with `i;ascii-casemap' comparator: >> 36: extracting `From' headers from message >> 36: parsing address header value `Bacula at ATM ' >> 36: extracting `all' part from address `Bacula at ATM' >> 36: matching value `Bacula at ATM' >> 36: with key `backup at areda.pl' => 0 >> 36: finishing match with result: not matched >> 36: jump if result is false >> 36: jumping to line 43 >> >> So in newer dovecot sieve uses comment not exact email address. Did I >> wrongly wrote my sieve script? > > Fixed: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/a3d26f12c2cd Thank you! Marcin From sanvila at unex.es Tue May 5 14:26:00 2015 From: sanvila at unex.es (Santiago Vila) Date: Tue, 5 May 2015 16:26:00 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5547C74D.4020806@sys4.de> References: <5547C74D.4020806@sys4.de> Message-ID: <20150505142600.GA4982@cantor.unex.es> I have just verified with IMAP commands. This is the procedure: telnet localhost 143 and then type this: A0001 CAPABILITY A0002 LOGIN "bluser" "bluser" A0003 SELECT "inbox-b" A0004 EXPUNGE A0005 FETCH 1:12 RFC822.SIZE A0006 FETCH 1 RFC822.HEADER A0007 FETCH 1 BODY.PEEK[TEXT] A0008 STORE 1 +FLAGS (\Seen \Deleted) A0009 EXPUNGE A0010 FETCH 1 RFC822.HEADER A0011 FETCH 1 BODY.PEEK[TEXT] A0012 STORE 1 +FLAGS (\Seen \Deleted) A0013 EXPUNGE A0014 FETCH 1 RFC822.HEADER A0015 FETCH 1 BODY.PEEK[TEXT] A0016 STORE 1 +FLAGS (\Seen \Deleted) A0017 EXPUNGE A0018 FETCH 1 RFC822.HEADER A0019 FETCH 1 BODY.PEEK[TEXT] A0020 STORE 1 +FLAGS (\Seen \Deleted) A0021 EXPUNGE A0022 FETCH 1 RFC822.HEADER A0023 FETCH 1 BODY.PEEK[TEXT] A0024 STORE 1 +FLAGS (\Seen \Deleted) A0025 EXPUNGE A0026 FETCH 1 RFC822.HEADER A0027 FETCH 1 BODY.PEEK[TEXT] A0028 STORE 1 +FLAGS (\Seen \Deleted) A0029 EXPUNGE A0030 FETCH 1 RFC822.HEADER A0031 FETCH 1 BODY.PEEK[TEXT] A0032 STORE 1 +FLAGS (\Seen \Deleted) A0033 EXPUNGE A0034 FETCH 1 RFC822.HEADER A0035 FETCH 1 BODY.PEEK[TEXT] A0036 STORE 1 +FLAGS (\Seen \Deleted) A0037 EXPUNGE A0038 FETCH 1 RFC822.HEADER A0039 FETCH 1 BODY.PEEK[TEXT] A0040 STORE 1 +FLAGS (\Seen \Deleted) A0041 EXPUNGE A0042 FETCH 1 RFC822.HEADER A0043 FETCH 1 BODY.PEEK[TEXT] A0044 STORE 1 +FLAGS (\Seen \Deleted) A0045 EXPUNGE A0046 FETCH 1 RFC822.HEADER A0047 FETCH 1 BODY.PEEK[TEXT] A0048 STORE 1 +FLAGS (\Seen \Deleted) A0049 EXPUNGE A0050 FETCH 1 RFC822.HEADER A0051 LOGOUT After this, mbox folder inbox-b is corrupted, as the line saying From: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com becomes rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com So: Could we please stop blaming fetchmail for this? It's just the messenger. From superinterstellar at gmail.com Tue May 5 14:52:53 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Tue, 5 May 2015 21:52:53 +0700 Subject: Mapping other Inbox method (symlink vs ACL) Message-ID: Hi, I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I would like him to see/access Inbox,Sent,Junk,Trash of help at mydomain.com . What would you reckon would be the best way to do this? ACL or Symlink? With symlink, I dont think I will be able to set permissions. Is it possible to use ACL to get the entire email account access(the inbox, junk, sent etc.)? From rs at sys4.de Tue May 5 15:15:16 2015 From: rs at sys4.de (Robert Schetterer) Date: Tue, 05 May 2015 17:15:16 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150505142600.GA4982@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> Message-ID: <5548DE84.3010405@sys4.de> Am 05.05.2015 um 16:26 schrieb Santiago Vila: > I have just verified with IMAP commands. This is the procedure: > > telnet localhost 143 > > and then type this: > > A0001 CAPABILITY > A0002 LOGIN "bluser" "bluser" > A0003 SELECT "inbox-b" > A0004 EXPUNGE > A0005 FETCH 1:12 RFC822.SIZE > A0006 FETCH 1 RFC822.HEADER > A0007 FETCH 1 BODY.PEEK[TEXT] > A0008 STORE 1 +FLAGS (\Seen \Deleted) > A0009 EXPUNGE > A0010 FETCH 1 RFC822.HEADER > A0011 FETCH 1 BODY.PEEK[TEXT] > A0012 STORE 1 +FLAGS (\Seen \Deleted) > A0013 EXPUNGE > A0014 FETCH 1 RFC822.HEADER > A0015 FETCH 1 BODY.PEEK[TEXT] > A0016 STORE 1 +FLAGS (\Seen \Deleted) > A0017 EXPUNGE > A0018 FETCH 1 RFC822.HEADER > A0019 FETCH 1 BODY.PEEK[TEXT] > A0020 STORE 1 +FLAGS (\Seen \Deleted) > A0021 EXPUNGE > A0022 FETCH 1 RFC822.HEADER > A0023 FETCH 1 BODY.PEEK[TEXT] > A0024 STORE 1 +FLAGS (\Seen \Deleted) > A0025 EXPUNGE > A0026 FETCH 1 RFC822.HEADER > A0027 FETCH 1 BODY.PEEK[TEXT] > A0028 STORE 1 +FLAGS (\Seen \Deleted) > A0029 EXPUNGE > A0030 FETCH 1 RFC822.HEADER > A0031 FETCH 1 BODY.PEEK[TEXT] > A0032 STORE 1 +FLAGS (\Seen \Deleted) > A0033 EXPUNGE > A0034 FETCH 1 RFC822.HEADER > A0035 FETCH 1 BODY.PEEK[TEXT] > A0036 STORE 1 +FLAGS (\Seen \Deleted) > A0037 EXPUNGE > A0038 FETCH 1 RFC822.HEADER > A0039 FETCH 1 BODY.PEEK[TEXT] > A0040 STORE 1 +FLAGS (\Seen \Deleted) > A0041 EXPUNGE > A0042 FETCH 1 RFC822.HEADER > A0043 FETCH 1 BODY.PEEK[TEXT] > A0044 STORE 1 +FLAGS (\Seen \Deleted) > A0045 EXPUNGE > A0046 FETCH 1 RFC822.HEADER > A0047 FETCH 1 BODY.PEEK[TEXT] > A0048 STORE 1 +FLAGS (\Seen \Deleted) > A0049 EXPUNGE > A0050 FETCH 1 RFC822.HEADER > A0051 LOGOUT > > After this, mbox folder inbox-b is corrupted, as the line saying > > From: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com > > becomes > > rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com is this related to mbox only ? i have no mbox install online to test did you try install dovecot from scratch with latest sources? i used a script to install a quick dovecot test server on ubuntu see https://sys4.de/de/blog/2013/06/06/postfix-dovecot-ceph-cluster-storage/ beyond ... Dovecot install ... then retry with telnet , sorry no time to test myself > > > So: Could we please stop blaming fetchmail for this? > It's just the messenger. > Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From sanvila at unex.es Tue May 5 17:22:20 2015 From: sanvila at unex.es (Santiago Vila) Date: Tue, 5 May 2015 19:22:20 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5548DE84.3010405@sys4.de> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5548DE84.3010405@sys4.de> Message-ID: <20150505172220.GA11670@cantor.unex.es> On Tue, May 05, 2015 at 05:15:16PM +0200, Robert Schetterer wrote: > is this related to mbox only ? I don't know. The bug happens as I explained, and current documentation says mbox is still supported (I really hope documentation is correct here). > i have no mbox install online to test A local virtual machine is more than enough to reproduce this. It does not need to be online. Also, it is not required to have client and server on different machines to reproduce this. They can be the same. > did you try install dovecot from scratch with latest sources? This is dovecot 2.2.16-1 from Debian unstable, released yesterday, which is the most recent release I can test. From emailbuilder88 at yahoo.com Tue May 5 18:39:27 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 5 May 2015 11:39:27 -0700 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <55487B6A.5040905@rename-it.nl> Message-ID: <1430851167.2657.YahooMailBasic@web142404.mail.bf1.yahoo.com> > >>>> Last two changes should fix this: > >>> > >>> Not yet -- this may be unrelated(?) but here is what I have > >>> after installing the newest source package: > >>> > >>> Error: Couldn't load required plugin /usr/local/lib/dovecot/lib90_sieve_plugin.so: > >>> dlopen() failed: /usr/local/lib/dovecot/lib90_sieve_plugin.so: undefined symbol: > >>> mail_deliver_ctx_get_log_var_expand_table > >> Oh i guessing this requires the newest Dovecot source too (maybe you > >> should put notice on the wiki that extprograms run-via-socket is broken until > >> a new release?) > >> > >> After installed newest Dovecot and Sieve, I getting this: > >> > >> dovecot: lmtp(test at example.com): Debug: auth input: > >> dovecot: lmtp( > >> testl at example.com): Fatal: master: service(lmtp): child > >> 7033 killed with signal 11 (core dumps disabled) > >> > >> With set mail_debug=yes doesn't give anything else of interesting for this. > > Sorrys. I decided to "make clean" and do new "./configure" "make" and "make > > install" for dovecot first, then pigeonhole to make sure nothing is off. After > > I do that I get same original error: > > > > dovecot: script: Fatal: recv(MSG_PEEK) failed: disconnected > > > > I went to go verifying in dovecot-pigeonhole that the two new patches are there > > by manual inspect the source files you patched. I found only the first one, not > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this > > because I downloaded a snap shot link at the top that wasn't built with the > > 2nd change? > > > > I put 2nd change in manually (its only one liner) rebuilded and after install..... > > same error with recv(MSG_PEEK) > > Did you restart Dovecot? Yes. One more restart now to make sure. > Also, can you try running the script through `sieve-test -D -t - > -Tlevel=matching` (see man page). I cannot make this working. I'm logg in as unix user "adm" and the sieve-test thinks that is the user it's delivering to?? I tried to tell the recipient with both -a and -r and I put hard-coded -l mail location but output still showing it wants to deliver to mailbox for "adm" user. Also mail file has To: header with the correct recipient but doesn't help. Users are virtual I cant logg in as the correct recipient to run sieve-test. What do I do? From rs at sys4.de Tue May 5 18:46:38 2015 From: rs at sys4.de (Robert Schetterer) Date: Tue, 05 May 2015 20:46:38 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150505172220.GA11670@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5548DE84.3010405@sys4.de> <20150505172220.GA11670@cantor.unex.es> Message-ID: <5549100E.8040102@sys4.de> Am 05.05.2015 um 19:22 schrieb Santiago Vila: > On Tue, May 05, 2015 at 05:15:16PM +0200, Robert Schetterer wrote: >> is this related to mbox only ? > > I don't know. The bug happens as I explained, and current documentation > says mbox is still supported (I really hope documentation is correct here). yeah but it may be a mbox only bug > >> i have no mbox install online to test > > A local virtual machine is more than enough to reproduce this. > It does not need to be online. i know, but i am short in time > > Also, it is not required to have client and server on different machines > to reproduce this. They can be the same. i tested your telnet sequence with maildir and could not reproduce your report which means nothing corupted > >> did you try install dovecot from scratch with latest sources? > > This is dovecot 2.2.16-1 from Debian unstable, released yesterday, > which is the most recent release I can test. > install from scratch, easy with script Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From zucca at systemschmiede.com Tue May 5 19:25:55 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Tue, 05 May 2015 21:25:55 +0200 Subject: only mark as deleted when deleted as default for some mailboxes Message-ID: <55491943.8030701@systemschmiede.com> Hi list, i wonder if this is possible with some dovecot configuration. Selected mailboxusers will only mark their mails as deleted, when a mail is being deleted through any client. No automatic expunge. This is default behaviour in Outlook, and i think in Thunderbird too, but unfortunately not in Open-Xchange for example. So i was asking myself if it would be possible to simply turn this feature on maybe globally and as a default for some users. Thank you very much for your reply. Sascha -- Systemschmiede IT-L?sungen Zucca & Teves GbR August-Thyssen-Stra?e 2-4 52511 Geilenkirchen www.systemschmiede.com Tel.: +49 (0)2451 - 9088390 Fax: +49 (0)2451 - 9124070 Mobil: +49 (0)178 - 4073254 Ust.ID: DE248030393 From emailbuilder88 at yahoo.com Tue May 5 19:28:54 2015 From: emailbuilder88 at yahoo.com (E.B.) Date: Tue, 5 May 2015 12:28:54 -0700 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <554884E5.6070002@rename-it.nl> Message-ID: <1430854134.65699.YahooMailBasic@web142402.mail.bf1.yahoo.com> > > I went to go verifying in dovecot-pigeonhole that the two new patches are there > > by manual inspect the source files you patched. I found only the first one, not > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/1eb0362461f0 maybe this > > because I downloaded a snap shot link at the top that wasn't built with the > > 2nd change? > > > > I put 2nd change in manually (its only one liner) rebuilded and after install..... > > same error with recv(MSG_PEEK) > > > > Thanks for ongoing help > > Argh! Ok, that fix was a bit incomplete still: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/89e0cef5b264 Sorry I over-looking this. I just tested with this new fix. IT WORKS. > I wonder why my test didn't fail. > > The extent of this bug is limited to using execute as a command (as > opposed to using it as a test in an `if' statement) and not providing it > with any input. That's not matching to my environment. My script is calling execute within a if block but not as a test but it IS providing input (but not get any output if that matters) if ... { execute :input "test input" "test"; } Only thing still unsolved is my other thread don't know why the exectued script has no filesystem access? (like "touch /tmp/test" ignored no error) Also is the only different between direct and socket execute that with socket I can run the script with more restricted owner and permissions? Are there other differences? From rs at sys4.de Tue May 5 20:10:28 2015 From: rs at sys4.de (Robert Schetterer) Date: Tue, 05 May 2015 22:10:28 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5549100E.8040102@sys4.de> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5548DE84.3010405@sys4.de> <20150505172220.GA11670@cantor.unex.es> <5549100E.8040102@sys4.de> Message-ID: <554923B4.60501@sys4.de> Am 05.05.2015 um 20:46 schrieb Robert Schetterer: > Am 05.05.2015 um 19:22 schrieb Santiago Vila: >> On Tue, May 05, 2015 at 05:15:16PM +0200, Robert Schetterer wrote: >>> is this related to mbox only ? >> >> I don't know. The bug happens as I explained, and current documentation >> says mbox is still supported (I really hope documentation is correct here). > > yeah but it may be a mbox only bug > >> >>> i have no mbox install online to test >> >> A local virtual machine is more than enough to reproduce this. >> It does not need to be online. > > i know, but i am short in time > >> >> Also, it is not required to have client and server on different machines >> to reproduce this. They can be the same. > > i tested your telnet sequence with maildir and could not reproduce your > report which means nothing corupted > >> >>> did you try install dovecot from scratch with latest sources? >> >> This is dovecot 2.2.16-1 from Debian unstable, released yesterday, >> which is the most recent release I can test. >> > > install from scratch, easy with script > > Best Regards > MfG Robert Schetterer > famos last words, my best bet is mbox bugs described here https://bugzilla.redhat.com/show_bug.cgi?id=1189198 arent finally fixed Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From skdovecot at smail.inf.fh-brs.de Wed May 6 06:15:31 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 May 2015 08:15:31 +0200 (CEST) Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 May 2015, Kevin Laurie wrote: > I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I > would like him to see/access Inbox,Sent,Junk,Trash of > help at mydomain.com . What would you reckon would be the best way to do > this? ACL or Symlink? to suggest a "best way" we would need to know more about your Dovecot installation. > With symlink, I dont think I will be able to set permissions. What permissions you are talking of? > Is it possible to use ACL to get the entire email account access(the > inbox, junk, sent etc.)? With ACLs help can allow bob to access selected mailboxes with selected IMAP permissions, if bob may access the mail storage of help on file system level. ACLs are more powerful, but require more setup. I use both ways: ACLs to share mailboxes in general, which appear in the users branch, and symlinks to place SPAM reporting mailboxes right into the namespace of each user. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVUmxg3z1H7kL/d9rAQLuHwf/SdVE/ZAH4Tf4/H0EacRvNEn08qUOROGq eKLpd0p/TFpbHQxfM0YLazH9QEJBIP5mpRWa5U0WVzFiLTfR1UgVPcS3xjJnq+Pl uR9/vQfUpn/B+viGCsTB8ItKCnTF892mCsiUbiFEs7BeF5jdoTOdHCsHorJL/UKa bmpTD1ORUeKcA7m075jhTVDB6jwgWRELItIx9W2sqHBu+wfWfMp5lv3qSyOXPLtH onz+aUYhzAepuAhVI36feMUEwZhzHF+3fvzJmKZJSEQ3BwEZOm0eOqeba4gk52KC rSCwLZ2aDEwAx9L95MmT+B08M6ChNOuILTiYFlv0o/3G3Qt8qhqHhA== =Q36P -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Wed May 6 06:18:50 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 May 2015 08:18:50 +0200 (CEST) Subject: only mark as deleted when deleted as default for some mailboxes In-Reply-To: <55491943.8030701@systemschmiede.com> References: <55491943.8030701@systemschmiede.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Tue, 5 May 2015, zucca at systemschmiede.com wrote: > i wonder if this is possible with some dovecot configuration. > Selected mailboxusers will only mark their mails as deleted, when a mail is > being deleted through any client. > No automatic expunge. That's a client configuration out of the reach of Dovecot. Maybe you can work around this behaviour with: http://wiki2.dovecot.org/Plugins/Lazyexpunge > This is default behaviour in Outlook, and i think in Thunderbird too, but > unfortunately not in Open-Xchange for example. Check out the settings in OX then. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVUmySnz1H7kL/d9rAQJQRwf/Z+N/c5BGqUkG0cHne0Q3WI9l8c3QYnMQ 6iI6Jr/sN6TTMI1gtxGQDD+vj/VxYtw0baNHxSrqqDV1uqK4YWaqTngdtpZGJSlj CXeJW5ifF/H5B9oQ0rn6L+71lwqvuORy9RXuIj4daBhESD0O+pcQ0e7MqDgR/89h kU3Of4g5GzNrlIQPXo1j3/PquKTCaEtCDzrUjlKQk+qD0lQHqsUmo1CkT49iGlvz MU/idJ/N08EjgRciEoElPEZoBhMSLMAm2IaGwMkWM4N+fq3gllmm/YQjv0WP339/ AM4XQPAtxtbBO4kMAQjIqNqOmqcSzGCEY1L5EuU4mO/9Yckn6/6Eew== =VKmP -----END PGP SIGNATURE----- From zucca at systemschmiede.com Wed May 6 06:25:44 2015 From: zucca at systemschmiede.com (zucca at systemschmiede.com) Date: Wed, 06 May 2015 08:25:44 +0200 Subject: only mark as deleted when deleted as default for some mailboxes In-Reply-To: References: <55491943.8030701@systemschmiede.com> Message-ID: <5549B3E8.2010209@systemschmiede.com> Well yes, i will have a look at Lazyexpunge then, as OX doesn't seem to be capable of this i'm afraid. Any ideas how to best implement this with Lazyexpunge are welcome btw. :-) cheers Sascha Am 06.05.2015 um 08:18 schrieb Steffen Kaiser: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 5 May 2015, zucca at systemschmiede.com wrote: > >> i wonder if this is possible with some dovecot configuration. >> Selected mailboxusers will only mark their mails as deleted, when a >> mail is being deleted through any client. >> No automatic expunge. > > That's a client configuration out of the reach of Dovecot. Maybe you > can work around this behaviour with: > http://wiki2.dovecot.org/Plugins/Lazyexpunge > >> This is default behaviour in Outlook, and i think in Thunderbird too, >> but unfortunately not in Open-Xchange for example. > > Check out the settings in OX then. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVUmySnz1H7kL/d9rAQJQRwf/Z+N/c5BGqUkG0cHne0Q3WI9l8c3QYnMQ > 6iI6Jr/sN6TTMI1gtxGQDD+vj/VxYtw0baNHxSrqqDV1uqK4YWaqTngdtpZGJSlj > CXeJW5ifF/H5B9oQ0rn6L+71lwqvuORy9RXuIj4daBhESD0O+pcQ0e7MqDgR/89h > kU3Of4g5GzNrlIQPXo1j3/PquKTCaEtCDzrUjlKQk+qD0lQHqsUmo1CkT49iGlvz > MU/idJ/N08EjgRciEoElPEZoBhMSLMAm2IaGwMkWM4N+fq3gllmm/YQjv0WP339/ > AM4XQPAtxtbBO4kMAQjIqNqOmqcSzGCEY1L5EuU4mO/9Yckn6/6Eew== > =VKmP > -----END PGP SIGNATURE----- From stephan at rename-it.nl Wed May 6 06:45:57 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 06 May 2015 08:45:57 +0200 Subject: Failed running extprograms execute via socket - fatal recv(MSG_PEEK) failed disconnected In-Reply-To: <1430854134.65699.YahooMailBasic@web142402.mail.bf1.yahoo.com> References: <1430854134.65699.YahooMailBasic@web142402.mail.bf1.yahoo.com> Message-ID: <5549B8A5.7000206@rename-it.nl> On 5/5/2015 9:28 PM, E.B. wrote: >> I wonder why my test didn't fail. >> >> The extent of this bug is limited to using execute as a command (as >> opposed to using it as a test in an `if' statement) and not providing it >> with any input. > That's not matching to my environment. My script is calling execute > within a if block but not as a test but it IS providing input (but not get > any output if that matters) > > if ... > { > execute :input "test input" "test"; > } Oh right, it happens when the script is not using the output of the external program. If execute is not a test and the output from the program is not used, there is no input stream to be read from the remote service. Discussing what is input and output in this context is highly confusing. :) > Only thing still unsolved is my other thread don't know why the exectued > script has no filesystem access? (like "touch /tmp/test" ignored no error) So far, I still haven't seen your full config from `dovecot -n` output. > Also is the only different between direct and socket execute that with > socket I can run the script with more restricted owner and permissions? > Are there other differences? Either side could be run in a different chroot, if any. Some of the environment variables such as SENDER and RECIPIENT are not available in the service (that is a TODO item). Regards, Stephan. From superinterstellar at gmail.com Wed May 6 06:46:35 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Wed, 6 May 2015 13:46:35 +0700 Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: Dear Steffen, Thanks for your feedback. Appreciate it. By permission I mean (read, write, look-up seen). I dont think symlink will allow these features. ACL does support such features. Only problem is that I have to setacl for individual boxes (ie Inbox,Sent,Junk etc.) On Wed, May 6, 2015 at 1:15 PM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Tue, 5 May 2015, Kevin Laurie wrote: > >> I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I >> would like him to see/access Inbox,Sent,Junk,Trash of >> help at mydomain.com . What would you reckon would be the best way to do >> this? ACL or Symlink? > > > to suggest a "best way" we would need to know more about your Dovecot > installation. > >> With symlink, I dont think I will be able to set permissions. > > > What permissions you are talking of? > >> Is it possible to use ACL to get the entire email account access(the >> inbox, junk, sent etc.)? > > > With ACLs help can allow bob to access selected mailboxes with selected IMAP > permissions, if bob may access the mail storage of help on file system > level. ACLs are more powerful, but require more setup. > > I use both ways: > > ACLs to share mailboxes in general, which appear in the users branch, and > symlinks to place SPAM reporting mailboxes right into the namespace of each > user. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVUmxg3z1H7kL/d9rAQLuHwf/SdVE/ZAH4Tf4/H0EacRvNEn08qUOROGq > eKLpd0p/TFpbHQxfM0YLazH9QEJBIP5mpRWa5U0WVzFiLTfR1UgVPcS3xjJnq+Pl > uR9/vQfUpn/B+viGCsTB8ItKCnTF892mCsiUbiFEs7BeF5jdoTOdHCsHorJL/UKa > bmpTD1ORUeKcA7m075jhTVDB6jwgWRELItIx9W2sqHBu+wfWfMp5lv3qSyOXPLtH > onz+aUYhzAepuAhVI36feMUEwZhzHF+3fvzJmKZJSEQ3BwEZOm0eOqeba4gk52KC > rSCwLZ2aDEwAx9L95MmT+B08M6ChNOuILTiYFlv0o/3G3Qt8qhqHhA== > =Q36P > -----END PGP SIGNATURE----- From teemu.huovila at dovecot.fi Wed May 6 06:57:32 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Wed, 06 May 2015 09:57:32 +0300 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150505142600.GA4982@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> Message-ID: <5549BB5C.8050502@dovecot.fi> On 05/05/2015 05:26 PM, Santiago Vila wrote: > I have just verified with IMAP commands. This is the procedure: > > telnet localhost 143 > > and then type this: > > A0001 CAPABILITY > A0002 LOGIN "bluser" "bluser" > A0003 SELECT "inbox-b" > A0004 EXPUNGE > A0005 FETCH 1:12 RFC822.SIZE > A0006 FETCH 1 RFC822.HEADER > A0007 FETCH 1 BODY.PEEK[TEXT] > A0008 STORE 1 +FLAGS (\Seen \Deleted) > A0009 EXPUNGE > A0010 FETCH 1 RFC822.HEADER > A0011 FETCH 1 BODY.PEEK[TEXT] > A0012 STORE 1 +FLAGS (\Seen \Deleted) > A0013 EXPUNGE > A0014 FETCH 1 RFC822.HEADER > A0015 FETCH 1 BODY.PEEK[TEXT] > A0016 STORE 1 +FLAGS (\Seen \Deleted) > A0017 EXPUNGE > A0018 FETCH 1 RFC822.HEADER > A0019 FETCH 1 BODY.PEEK[TEXT] > A0020 STORE 1 +FLAGS (\Seen \Deleted) > A0021 EXPUNGE > A0022 FETCH 1 RFC822.HEADER > A0023 FETCH 1 BODY.PEEK[TEXT] > A0024 STORE 1 +FLAGS (\Seen \Deleted) > A0025 EXPUNGE > A0026 FETCH 1 RFC822.HEADER > A0027 FETCH 1 BODY.PEEK[TEXT] > A0028 STORE 1 +FLAGS (\Seen \Deleted) > A0029 EXPUNGE > A0030 FETCH 1 RFC822.HEADER > A0031 FETCH 1 BODY.PEEK[TEXT] > A0032 STORE 1 +FLAGS (\Seen \Deleted) > A0033 EXPUNGE > A0034 FETCH 1 RFC822.HEADER > A0035 FETCH 1 BODY.PEEK[TEXT] > A0036 STORE 1 +FLAGS (\Seen \Deleted) > A0037 EXPUNGE > A0038 FETCH 1 RFC822.HEADER > A0039 FETCH 1 BODY.PEEK[TEXT] > A0040 STORE 1 +FLAGS (\Seen \Deleted) > A0041 EXPUNGE > A0042 FETCH 1 RFC822.HEADER > A0043 FETCH 1 BODY.PEEK[TEXT] > A0044 STORE 1 +FLAGS (\Seen \Deleted) > A0045 EXPUNGE > A0046 FETCH 1 RFC822.HEADER > A0047 FETCH 1 BODY.PEEK[TEXT] > A0048 STORE 1 +FLAGS (\Seen \Deleted) > A0049 EXPUNGE > A0050 FETCH 1 RFC822.HEADER > A0051 LOGOUT > > After this, mbox folder inbox-b is corrupted, as the line saying > > From: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com > > becomes > > rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com > > > So: Could we please stop blaming fetchmail for this? > It's just the messenger. Could you also sprovide your "dovecot -n" output and any warnings and errors in dovecot logs. br, Teemu Huovila From teemu.huovila at dovecot.fi Wed May 6 06:59:26 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Wed, 06 May 2015 09:59:26 +0300 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5549BB5C.8050502@dovecot.fi> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> Message-ID: <5549BBCE.90307@dovecot.fi> On 05/06/2015 09:57 AM, Teemu Huovila wrote: > On 05/05/2015 05:26 PM, Santiago Vila wrote: >> I have just verified with IMAP commands. This is the procedure: >> >> telnet localhost 143 >> >> and then type this: >> >> A0001 CAPABILITY >> A0002 LOGIN "bluser" "bluser" >> A0003 SELECT "inbox-b" >> A0004 EXPUNGE >> A0005 FETCH 1:12 RFC822.SIZE >> A0006 FETCH 1 RFC822.HEADER >> A0007 FETCH 1 BODY.PEEK[TEXT] >> A0008 STORE 1 +FLAGS (\Seen \Deleted) >> A0009 EXPUNGE >> A0010 FETCH 1 RFC822.HEADER >> A0011 FETCH 1 BODY.PEEK[TEXT] >> A0012 STORE 1 +FLAGS (\Seen \Deleted) >> A0013 EXPUNGE >> A0014 FETCH 1 RFC822.HEADER >> A0015 FETCH 1 BODY.PEEK[TEXT] >> A0016 STORE 1 +FLAGS (\Seen \Deleted) >> A0017 EXPUNGE >> A0018 FETCH 1 RFC822.HEADER >> A0019 FETCH 1 BODY.PEEK[TEXT] >> A0020 STORE 1 +FLAGS (\Seen \Deleted) >> A0021 EXPUNGE >> A0022 FETCH 1 RFC822.HEADER >> A0023 FETCH 1 BODY.PEEK[TEXT] >> A0024 STORE 1 +FLAGS (\Seen \Deleted) >> A0025 EXPUNGE >> A0026 FETCH 1 RFC822.HEADER >> A0027 FETCH 1 BODY.PEEK[TEXT] >> A0028 STORE 1 +FLAGS (\Seen \Deleted) >> A0029 EXPUNGE >> A0030 FETCH 1 RFC822.HEADER >> A0031 FETCH 1 BODY.PEEK[TEXT] >> A0032 STORE 1 +FLAGS (\Seen \Deleted) >> A0033 EXPUNGE >> A0034 FETCH 1 RFC822.HEADER >> A0035 FETCH 1 BODY.PEEK[TEXT] >> A0036 STORE 1 +FLAGS (\Seen \Deleted) >> A0037 EXPUNGE >> A0038 FETCH 1 RFC822.HEADER >> A0039 FETCH 1 BODY.PEEK[TEXT] >> A0040 STORE 1 +FLAGS (\Seen \Deleted) >> A0041 EXPUNGE >> A0042 FETCH 1 RFC822.HEADER >> A0043 FETCH 1 BODY.PEEK[TEXT] >> A0044 STORE 1 +FLAGS (\Seen \Deleted) >> A0045 EXPUNGE >> A0046 FETCH 1 RFC822.HEADER >> A0047 FETCH 1 BODY.PEEK[TEXT] >> A0048 STORE 1 +FLAGS (\Seen \Deleted) >> A0049 EXPUNGE >> A0050 FETCH 1 RFC822.HEADER >> A0051 LOGOUT >> >> After this, mbox folder inbox-b is corrupted, as the line saying >> >> From: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com >> >> becomes >> >> rstuvwxyzabcdefghijklmnopqrstuvwxyz at example.com >> >> >> So: Could we please stop blaming fetchmail for this? >> It's just the messenger. > Could you also sprovide your "dovecot -n" output and any warnings and errors in dovecot logs. Ah, found the dovecot -n earlier in the thread, but the logs would still be relevant. Teemu From skdovecot at smail.inf.fh-brs.de Wed May 6 07:12:21 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Wed, 6 May 2015 09:12:21 +0200 (CEST) Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 6 May 2015, Kevin Laurie wrote: > By permission I mean (read, write, look-up seen). I dont think symlink > will allow these features. Symlinks grant all IMAP permissions, the filesystem level permits. That is, with symlinks you cannot selectivly deny permissions easily. > ACL does support such features. > Only problem is that I have to setacl for individual boxes (ie > Inbox,Sent,Junk etc.) There are front ends for IMAP ACLs and more importantly for your current situation: http://wiki2.dovecot.org/Tools/Doveadm/ACL Works great. If you search the list, you will find posts how to grant permissions for several or all mailboxes of one account with a tool chain using "doveadm mailbox list ". > On Wed, May 6, 2015 at 1:15 PM, Steffen Kaiser > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Tue, 5 May 2015, Kevin Laurie wrote: >> >>> I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I >>> would like him to see/access Inbox,Sent,Junk,Trash of >>> help at mydomain.com . What would you reckon would be the best way to do >>> this? ACL or Symlink? >> >> >> to suggest a "best way" we would need to know more about your Dovecot >> installation. >> >>> With symlink, I dont think I will be able to set permissions. >> >> >> What permissions you are talking of? >> >>> Is it possible to use ACL to get the entire email account access(the >>> inbox, junk, sent etc.)? >> >> >> With ACLs help can allow bob to access selected mailboxes with selected IMAP >> permissions, if bob may access the mail storage of help on file system >> level. ACLs are more powerful, but require more setup. >> >> I use both ways: >> >> ACLs to share mailboxes in general, which appear in the users branch, and >> symlinks to place SPAM reporting mailboxes right into the namespace of each >> user. >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVUmxg3z1H7kL/d9rAQLuHwf/SdVE/ZAH4Tf4/H0EacRvNEn08qUOROGq >> eKLpd0p/TFpbHQxfM0YLazH9QEJBIP5mpRWa5U0WVzFiLTfR1UgVPcS3xjJnq+Pl >> uR9/vQfUpn/B+viGCsTB8ItKCnTF892mCsiUbiFEs7BeF5jdoTOdHCsHorJL/UKa >> bmpTD1ORUeKcA7m075jhTVDB6jwgWRELItIx9W2sqHBu+wfWfMp5lv3qSyOXPLtH >> onz+aUYhzAepuAhVI36feMUEwZhzHF+3fvzJmKZJSEQ3BwEZOm0eOqeba4gk52KC >> rSCwLZ2aDEwAx9L95MmT+B08M6ChNOuILTiYFlv0o/3G3Qt8qhqHhA== >> =Q36P >> -----END PGP SIGNATURE----- > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVUm+1Xz1H7kL/d9rAQKe6QgAnKH2zKVKZzfawIEwhpd4qY1fXP1dXNvA Ymzsf4i2MSG2hg8d1Nw91kxPQxmamHq98HLgHFjWy9of/5zW8I23iOAjxgJMpypY pXha/1T1W4rDoF7wnpHSWdkGtyFW4bQu3T1vNfU12bLw/d1ehdgcDjLHdYDncKyh ZZdFQ2BpPYyiHs3+KnZVqixdFna9+lEMOMJddVI1+8dTfRf3JlfZptEbhOp501ko w/slmqMzpZsx/+20QzI+pXh+jmQy0FFAJh8z0mWsnxdJqNbf9zSmSmvCy4lwirhL Mht3x2mudhcGk5l3Z+R86QxJiElEWpzdFv0JJRQp1oRwljAncasCGA== =cNmt -----END PGP SIGNATURE----- From amateo at um.es Wed May 6 08:53:45 2015 From: amateo at um.es (Angel L. Mateo) Date: Wed, 06 May 2015 10:53:45 +0200 Subject: lmpt director and ldap userdb Message-ID: <5549D699.9070106@um.es> Hello, I've been using dovecot for a long time, but now I'm having troubles with I change I need to make. I have a director server in front of two backend servers. All of these servers are running dovecot 2.1.16. Redirection to backend servers are based in an attribute of a ldap server. My users are defined in this same ldap server. At this moment, for user authentication we use password lookups. My current configuration at both frontend and backend servers is attached. But now I need to change it to bind authentication, so the only change I've made is changing "auth_bind=no" to "auth_bind=yes". After this change, backend servers are working fine. Director servers are also working for POP and IMAP connections, but for LMTP they are returning: May 6 10:23:35 myotis40 dovecot: lmtp(48026): Error: user myuser at um.es: Auth PASS lookup failed This error is returned without any ldap lookup, neither connection to backend servers. Any help? thanks in advance. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 -------------- next part -------------- # 2.1.16: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-24-generic x86_64 Ubuntu 12.04.5 LTS auth_cache_negative_ttl = 0 auth_cache_size = 20 M auth_cache_ttl = 1 days auth_master_user_separator = * auth_socket_path = director-userdb default_process_limit = 1024 director_mail_servers = 155.54.211.161-155.54.211.164 director_servers = 155.54.211.185 155.54.211.186 director_username_hash = %Ln disable_plaintext_auth = no doveadm_proxy_port = 24245 lmtp_proxy = yes log_timestamp = %Y-%m-%d %H:%M:%S mail_plugins = quota stats managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext default_fields = proxy=y nopassword=y proxy_timeout=600 driver = ldap } plugin { sieve = ~/.dovecot.sieve sieve_default = /etc/dovecot/default.sieve sieve_dir = ~/sieve sieve_max_redirects = 15 } protocols = " imap lmtp sieve pop3 sieve" service anvil { client_limit = 3075 } service auth { client_limit = 4096 unix_listener auth-userdb { mode = 0777 } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap-login { client_limit = 2206 executable = imap-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service imap { process_limit = 12000 process_min_avail = 6 vsz_limit = 512 M } service ipc { unix_listener ipc { user = dovecot } } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service managesieve-login { executable = managesieve-login director } service pop3-login { client_limit = 833 executable = pop3-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service pop3 { process_min_avail = 6 } service stats { fifo_listener stats-mail { mode = 0666 } } ssl = no userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol doveadm { auth_socket_path = director-userdb } local 155.54.211.160/27/27 { doveadm_password = } -------------- next part -------------- auth_bind = no base = default_pass_scheme = MD5-CRYPT dn = dnpass = iterate_attrs = uid=user iterate_filter = (&(objectClass=CourierMailAccount)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo)) pass_attrs = irisMailbox=userdb_mail,homeDirectory=userdb_home,uidNumber=userdb_uid,gidNumber=userdb_gid,=proxy=y,=proxy_timeout=600,irisMailHost=host pass_filter = (&(&(objectClass=CourierMailAccount)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo))(|(uid=%u)(mail=%u))) sasl_bind = no uris = ldap://ldap.um.es user_attrs = irisMailbox=mail,quota=quota_rule user_filter = (&(&(objectClass=CourierMailAccount)(irisUserStatus=urn:mace:rediris.es:um.es:userstatus:correo:estado:activo))(|(uid=%u)(mail=%u))) From amateo at um.es Wed May 6 09:29:19 2015 From: amateo at um.es (Angel L. Mateo) Date: Wed, 06 May 2015 11:29:19 +0200 Subject: lmpt director and ldap userdb In-Reply-To: <5549D699.9070106@um.es> References: <5549D699.9070106@um.es> Message-ID: <5549DEEF.1040307@um.es> El 06/05/15 a las 10:53, Angel L. Mateo escribi?: > Hello, > > I've been using dovecot for a long time, but now I'm having > troubles with I change I need to make. > > I have a director server in front of two backend servers. All of > these servers are running dovecot 2.1.16. Redirection to backend servers > are based in an attribute of a ldap server. My users are defined in this > same ldap server. > > At this moment, for user authentication we use password lookups. My > current configuration at both frontend and backend servers is attached. > But now I need to change it to bind authentication, so the only change > I've made is changing "auth_bind=no" to "auth_bind=yes". > > After this change, backend servers are working fine. Director > servers are also working for POP and IMAP connections, but for LMTP they > are returning: > > May 6 10:23:35 myotis40 dovecot: lmtp(48026): Error: user myuser at um.es: > Auth PASS lookup failed > > This error is returned without any ldap lookup, neither connection > to backend servers. > > Any help? thanks in advance. > I'm sorry. As soon as I ask, the sooner I find the answer :-( I have found in a reply to a similar question in this same list (http://www.dovecot.org/list/dovecot/2011-November/079687.html) that a passdb database for lmtp can't use auth_bind=yes because lmtp can't make any authentication. So I left my director server with auth_bind=no and backends with auth_bind=yes and it works. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From sanvila at unex.es Wed May 6 09:29:52 2015 From: sanvila at unex.es (Santiago Vila) Date: Wed, 6 May 2015 11:29:52 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <5549BBCE.90307@dovecot.fi> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> Message-ID: <20150506092952.GA28906@cantor.unex.es> Debug output: May 6 11:23:42 qemu-sid dovecot: master: Dovecot v2.2.16 starting up for imap (core dumps disabled) May 6 11:24:06 qemu-sid dovecot: imap-login: Login: user=, method=PLAIN, rip=192.168.122.8, lip=192.168.122.8, mpid=441, secured, session= May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: Loading modules from directory: /usr/lib/dovecot/modules May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: Module loaded: /usr/lib/dovecot/modules/lib15_notify_plugin.so May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_mail_log_plugin.so May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: Effective uid=1000, gid=1000, home=/home/bluser May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=mbox:~/mail:INBOX=/var/mail/bluser May 6 11:24:06 qemu-sid dovecot: imap(bluser): Debug: fs: root=/home/bluser/mail, index=, indexpvt=, control=, inbox=/var/mail/bluser, alt= May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15988, msgid=<20150113080229.634B31FE0F at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15988, msgid=<20150113080229.634B31FE0F at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): Error: Sync failed for mbox file /home/bluser/mail/inbox-b: seq=13 uid=15999 uid_broken=0 originally needed 0 bytes, now needs 23 bytes May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15989, msgid=<20150113080252.84E723FB47 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15989, msgid=<20150113080252.84E723FB47 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15990, msgid=<20150113080253.3E89D5F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15990, msgid=<20150113080253.3E89D5F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15991, msgid=<20150113080256.702551FD75 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15991, msgid=<20150113080256.702551FD75 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15992, msgid=<20150113081736.928595F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15992, msgid=<20150113081736.928595F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15993, msgid=<20150113090223.890703FBC5 at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15993, msgid=<20150113090223.890703FBC5 at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15994, msgid=<20150113090225.EA3BF1FDD0 at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15994, msgid=<20150113090225.EA3BF1FDD0 at example.com>, size=166 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15995, msgid=<20150113090235.947643FB11 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15995, msgid=<20150113090235.947643FB11 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15996, msgid=<20150113090252.ACE3D5F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15996, msgid=<20150113090252.ACE3D5F8B1 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15997, msgid=<20150113090252.F11061FDAE at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15997, msgid=<20150113090252.F11061FDAE at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): delete: box=inbox-b, uid=15998, msgid=<20150113091737.92CD33FC52 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): expunge: box=inbox-b, uid=15998, msgid=<20150113091737.92CD33FC52 at example.com>, size=238 May 6 11:24:06 qemu-sid dovecot: imap(bluser): Disconnected: Logged out in=1364 out=6726 From sanvila at unex.es Wed May 6 09:43:22 2015 From: sanvila at unex.es (Santiago Vila) Date: Wed, 6 May 2015 11:43:22 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150506092952.GA28906@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> <20150506092952.GA28906@cantor.unex.es> Message-ID: <20150506094322.GA29104@cantor.unex.es> For reference, this is the "dovecot -n" output matching the previous debug output: # 2.2.16: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian stretch/sid mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = mail_log notify namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = " imap" ssl = no userdb { driver = passwd } From reuben-dovecot at reub.net Wed May 6 10:52:29 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Wed, 6 May 2015 20:52:29 +1000 Subject: Dovecot Replication - Architecture Endianness? In-Reply-To: <55476ED1.4050705@dovecot.fi> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> Message-ID: <5549F26D.8010702@reub.net> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >> Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? > You can set the userdb to return an empty mail_replica variable for users you want to exclude from replication. > http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 > > br, > Teemu Huovila One last question. Is it possible to achieve this with system users and PAM or do I need to basically create a new static userdb for system users? Could this be done via a per-user LDA setting or sieve? Thanks, Reuben From memoefix at gmail.com Wed May 6 17:26:52 2015 From: memoefix at gmail.com (Alain BERNARD) Date: Wed, 6 May 2015 19:26:52 +0200 Subject: Dovecot and data migration Message-ID: Hi, Our legacy data store retains a single copy of a message regardless of the number of mailboxes in which that message resides. It does this by creating hard links to that message in the mailboxes containing that message. Thus, when we perform data migration to the server target (Dovecot), the copies of the same message are copied over with the migration process (imapsync). We use the storage format maildir. With a small message store, this means that a lot of messages are duplicated unnecessarily. How to reduce message store size due to duplicate storage of identical messages ? Does a relinking function exist and can be run in real-time mode ? how can we configure Dovecot to deduplicate for all users using a hash to determine whether the file could be already exist ? Thanks in advance. Memo From superinterstellar at gmail.com Wed May 6 20:05:13 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 7 May 2015 03:05:13 +0700 Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: Dear Steffan, Noted. Thanks for your feedback! Best Regards Kevin On Wed, May 6, 2015 at 2:12 PM, Steffen Kaiser wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 6 May 2015, Kevin Laurie wrote: > >> By permission I mean (read, write, look-up seen). I dont think symlink >> will allow these features. > > > Symlinks grant all IMAP permissions, the filesystem level permits. That is, > with symlinks you cannot selectivly deny permissions easily. > >> ACL does support such features. >> Only problem is that I have to setacl for individual boxes (ie >> Inbox,Sent,Junk etc.) > > > There are front ends for IMAP ACLs and more importantly for your current > situation: > > http://wiki2.dovecot.org/Tools/Doveadm/ACL > > Works great. If you search the list, you will find posts how to grant > permissions for several or all mailboxes of one account with a tool chain > using "doveadm mailbox list ". > > >> On Wed, May 6, 2015 at 1:15 PM, Steffen Kaiser >> wrote: >>> >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA1 >>> >>> On Tue, 5 May 2015, Kevin Laurie wrote: >>> >>>> I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I >>>> would like him to see/access Inbox,Sent,Junk,Trash of >>>> help at mydomain.com . What would you reckon would be the best way to do >>>> this? ACL or Symlink? >>> >>> >>> >>> to suggest a "best way" we would need to know more about your Dovecot >>> installation. >>> >>>> With symlink, I dont think I will be able to set permissions. >>> >>> >>> >>> What permissions you are talking of? >>> >>>> Is it possible to use ACL to get the entire email account access(the >>>> inbox, junk, sent etc.)? >>> >>> >>> >>> With ACLs help can allow bob to access selected mailboxes with selected >>> IMAP >>> permissions, if bob may access the mail storage of help on file system >>> level. ACLs are more powerful, but require more setup. >>> >>> I use both ways: >>> >>> ACLs to share mailboxes in general, which appear in the users branch, and >>> symlinks to place SPAM reporting mailboxes right into the namespace of >>> each >>> user. >>> >>> - -- Steffen Kaiser >>> -----BEGIN PGP SIGNATURE----- >>> Version: GnuPG v1 >>> >>> iQEVAwUBVUmxg3z1H7kL/d9rAQLuHwf/SdVE/ZAH4Tf4/H0EacRvNEn08qUOROGq >>> eKLpd0p/TFpbHQxfM0YLazH9QEJBIP5mpRWa5U0WVzFiLTfR1UgVPcS3xjJnq+Pl >>> uR9/vQfUpn/B+viGCsTB8ItKCnTF892mCsiUbiFEs7BeF5jdoTOdHCsHorJL/UKa >>> bmpTD1ORUeKcA7m075jhTVDB6jwgWRELItIx9W2sqHBu+wfWfMp5lv3qSyOXPLtH >>> onz+aUYhzAepuAhVI36feMUEwZhzHF+3fvzJmKZJSEQ3BwEZOm0eOqeba4gk52KC >>> rSCwLZ2aDEwAx9L95MmT+B08M6ChNOuILTiYFlv0o/3G3Qt8qhqHhA== >>> =Q36P >>> -----END PGP SIGNATURE----- >> >> > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVUm+1Xz1H7kL/d9rAQKe6QgAnKH2zKVKZzfawIEwhpd4qY1fXP1dXNvA > Ymzsf4i2MSG2hg8d1Nw91kxPQxmamHq98HLgHFjWy9of/5zW8I23iOAjxgJMpypY > pXha/1T1W4rDoF7wnpHSWdkGtyFW4bQu3T1vNfU12bLw/d1ehdgcDjLHdYDncKyh > ZZdFQ2BpPYyiHs3+KnZVqixdFna9+lEMOMJddVI1+8dTfRf3JlfZptEbhOp501ko > w/slmqMzpZsx/+20QzI+pXh+jmQy0FFAJh8z0mWsnxdJqNbf9zSmSmvCy4lwirhL > Mht3x2mudhcGk5l3Z+R86QxJiElEWpzdFv0JJRQp1oRwljAncasCGA== > =cNmt > -----END PGP SIGNATURE----- From superinterstellar at gmail.com Wed May 6 20:21:50 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Thu, 7 May 2015 03:21:50 +0700 Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: Hi Steffan, How do I specify the entire mailbox? I only use ACL via telnet. I tried to use *(wildcard) to indicate entire mailbox but I guess that does not work. Will keep looking but appreciate if you could advise briefly. Thanks Kevin On Thu, May 7, 2015 at 3:05 AM, Kevin Laurie wrote: > Dear Steffan, > Noted. Thanks for your feedback! > Best Regards > Kevin > > On Wed, May 6, 2015 at 2:12 PM, Steffen Kaiser > wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Wed, 6 May 2015, Kevin Laurie wrote: >> >>> By permission I mean (read, write, look-up seen). I dont think symlink >>> will allow these features. >> >> >> Symlinks grant all IMAP permissions, the filesystem level permits. That is, >> with symlinks you cannot selectivly deny permissions easily. >> >>> ACL does support such features. >>> Only problem is that I have to setacl for individual boxes (ie >>> Inbox,Sent,Junk etc.) >> >> >> There are front ends for IMAP ACLs and more importantly for your current >> situation: >> >> http://wiki2.dovecot.org/Tools/Doveadm/ACL >> >> Works great. If you search the list, you will find posts how to grant >> permissions for several or all mailboxes of one account with a tool chain >> using "doveadm mailbox list ". >> >> >>> On Wed, May 6, 2015 at 1:15 PM, Steffen Kaiser >>> wrote: >>>> >>>> -----BEGIN PGP SIGNED MESSAGE----- >>>> Hash: SHA1 >>>> >>>> On Tue, 5 May 2015, Kevin Laurie wrote: >>>> >>>>> I am trying to map a account(bob at mydomain.com) to help at mydomain.com. I >>>>> would like him to see/access Inbox,Sent,Junk,Trash of >>>>> help at mydomain.com . What would you reckon would be the best way to do >>>>> this? ACL or Symlink? >>>> >>>> >>>> >>>> to suggest a "best way" we would need to know more about your Dovecot >>>> installation. >>>> >>>>> With symlink, I dont think I will be able to set permissions. >>>> >>>> >>>> >>>> What permissions you are talking of? >>>> >>>>> Is it possible to use ACL to get the entire email account access(the >>>>> inbox, junk, sent etc.)? >>>> >>>> >>>> >>>> With ACLs help can allow bob to access selected mailboxes with selected >>>> IMAP >>>> permissions, if bob may access the mail storage of help on file system >>>> level. ACLs are more powerful, but require more setup. >>>> >>>> I use both ways: >>>> >>>> ACLs to share mailboxes in general, which appear in the users branch, and >>>> symlinks to place SPAM reporting mailboxes right into the namespace of >>>> each >>>> user. >>>> >>>> - -- Steffen Kaiser >>>> -----BEGIN PGP SIGNATURE----- >>>> Version: GnuPG v1 >>>> >>>> iQEVAwUBVUmxg3z1H7kL/d9rAQLuHwf/SdVE/ZAH4Tf4/H0EacRvNEn08qUOROGq >>>> eKLpd0p/TFpbHQxfM0YLazH9QEJBIP5mpRWa5U0WVzFiLTfR1UgVPcS3xjJnq+Pl >>>> uR9/vQfUpn/B+viGCsTB8ItKCnTF892mCsiUbiFEs7BeF5jdoTOdHCsHorJL/UKa >>>> bmpTD1ORUeKcA7m075jhTVDB6jwgWRELItIx9W2sqHBu+wfWfMp5lv3qSyOXPLtH >>>> onz+aUYhzAepuAhVI36feMUEwZhzHF+3fvzJmKZJSEQ3BwEZOm0eOqeba4gk52KC >>>> rSCwLZ2aDEwAx9L95MmT+B08M6ChNOuILTiYFlv0o/3G3Qt8qhqHhA== >>>> =Q36P >>>> -----END PGP SIGNATURE----- >>> >>> >> >> - -- Steffen Kaiser >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1 >> >> iQEVAwUBVUm+1Xz1H7kL/d9rAQKe6QgAnKH2zKVKZzfawIEwhpd4qY1fXP1dXNvA >> Ymzsf4i2MSG2hg8d1Nw91kxPQxmamHq98HLgHFjWy9of/5zW8I23iOAjxgJMpypY >> pXha/1T1W4rDoF7wnpHSWdkGtyFW4bQu3T1vNfU12bLw/d1ehdgcDjLHdYDncKyh >> ZZdFQ2BpPYyiHs3+KnZVqixdFna9+lEMOMJddVI1+8dTfRf3JlfZptEbhOp501ko >> w/slmqMzpZsx/+20QzI+pXh+jmQy0FFAJh8z0mWsnxdJqNbf9zSmSmvCy4lwirhL >> Mht3x2mudhcGk5l3Z+R86QxJiElEWpzdFv0JJRQp1oRwljAncasCGA== >> =cNmt >> -----END PGP SIGNATURE----- From tss at iki.fi Wed May 6 21:02:12 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 7 May 2015 00:02:12 +0300 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150506094322.GA29104@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> <20150506092952.GA28906@cantor.unex.es> <20150506094322.GA29104@cantor.unex.es> Message-ID: <6F1B94E8-A1E6-4BB6-8243-6E0F1286AEF5@iki.fi> Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/94bd895721d8 From george.vieira at netregistry.com.au Wed May 6 04:06:52 2015 From: george.vieira at netregistry.com.au (George Vieira) Date: Wed, 6 May 2015 14:06:52 +1000 Subject: dovecot-proxy with managesieve, director and backend dovecot imap Message-ID: <5549935C.4030101@netregistry.com.au> hi all, I've been tasked to add sieve/managesieve to an existing dovecot cluster running 2.1.7 on debian wheezy which is made up of 2 dovecot-proxy hosts as directors and some back end dovecot imap hosts all running the same version. My problem is that I thought to put the service on the director/proxy hosts since they wouldn't have too much load on it, but when I do I get the following error: Apr 28 11:00:28 master: Info: Dovecot v2.1.7 starting up (core dumps disabled) Apr 28 11:00:28 config: Warning: service auth { client_limit=50000 } is lower than required under max. load (60000) Apr 28 11:00:34 managesieve-login: Error: proxy: host not given: user=, method=PLAIN, rip=192.168.100.207, lip=192.168.100.119, TLS, session=<3/zPY74UOgDAqGTP> Apr 28 11:00:34 managesieve-login: Info: Aborted login (internal failure, 1 succesful auths): user=, method=PLAIN, rip=192.168.100.207, lip=192.168.100.119, TLS, session=<3/zPY74UOgDAqGTP> From searching around, only ever saw 1 result which was to add "executable = managesieve-login director" to the managesieve service, but this made no difference at all and the error is the same. So I tried to instead use the back end imap servers, but they throw errors expecting the users password to be the common proxy/director password as below: passdb { driver = static args = user=%u password=crypticpasswordagain } Apr 28 12:03:37 auth: Debug: static(mailchannel at mydomain.net,192.168.100.207,<17RTRb8UpADAqGTP>): lookup Apr 28 12:03:37 auth: Info: static(mailchannel at mydomain.net,192.168.100.207,<17RTRb8UpADAqGTP>): Password mismatch Apr 28 12:03:37 auth: Debug: static(mailchannel at mydomain.net,192.168.100.207,<17RTRb8UpADAqGTP>): PLAIN(85387v92394jks) != 'crypticpasswordagain' Apr 28 12:03:39 auth: Debug: client out: FAIL 1 user=mailchannel at mydomain.net So with configs below, how is it best to run managesieve that takes the correct login/password without directing to the cluster (or direct if it's easier but must use real user password)? -- dovecot proxy config -- # dovecot version 2.1.7 instance_name = dovecot-proxy protocols = imap pop3 lmtp sieve mail_location = maildir:~/ #listen = 192.168.101.119 listen = 0.0.0.0 # = dovecot-proxy-1 director_servers = 192.168.101.119 # = dovecot-shared-7 director_mail_servers = 192.168.100.101 base_dir = /var/run/dovecot-proxy login_greeting = Welcome to IMAP. default_internal_user = webmail lmtp_proxy = yes disable_plaintext_auth = no auth_mechanisms = plain login cram-md5 auth_verbose=yes auth_debug=yes auth_debug_passwords=yes mail_debug=yes verbose_ssl=yes auth_verbose_passwords=no #log_path = syslog log_path = /var/log/dovecot.log default_process_limit = 10000 default_client_limit = 50000 ssl = no ssl_cert = References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <55477DFF.9060000@reub.net> Message-ID: <9F1663B8-3199-4EC9-92C8-55B550474AF1@iki.fi> On 04 May 2015, at 17:11, Reuben Farrelly wrote: > > On 4/05/2015 11:06 PM, Teemu Huovila wrote: >> On 05/03/2015 01:48 PM, Reuben Farrelly wrote: >>> Hi all, >>> >>> I've had an interesting use case come up which - to cut the story short - one way to solve the problem I am looking at may be to >>> replicate a small number of mailboxes to a third remote server. >>> >>> I've currently had replication running between my main dovecot machine and another remote VM for some time and working well (so >>> I'm not new to replication and I've got a good working config), but I've a need to add a third to the mix for a select number of >>> mailboxes. The arch on both of those is Gentoo x86_64 and with latest 2.1.16 -hg. >>> >>> I have attempted this so far by rsync'ing the initial Maildirs and then once the bulk of the data has been transferred rely on >>> dovecot's replication to keep things in sync. I figure that this should in theory mean that the subsequent updates in both >>> directions are incremental and the bulk of the data gets moved while the device is here on my desk using rsync. >>> >>> I've attempted to do this using a Raspberry Pi as a remote device, but when I set it up the dovecot replication process seems to >>> need to start the replication over from scratch even after the rsync is done. I know this is happening as the disk utilisation >>> on the Pi skyrockets once the replication starts and I end up with thousands of double ups of all the mails ... which defeats >>> the entire point of the process. >>> >>> If I do an identical configuration but on a third Gentoo x86_64 VM locally it all works as expected. No double ups of mails and >>> the "catchup" between the two devices is practically instant. Same filesystem even. The only difference appears to be the >>> system architecture. >>> >>> So main my question is this. Is there a known architecture/endian limitation on replication? I guess cross-arch replication >>> is not something many people try but is it supposed to work anyway? >> I think you are bumping against Dovecot index endianess restrictions. I dont think cross-arch dsync can currently work very >> efficiently. >> http://wiki2.dovecot.org/Design/Indexes/MainIndex?highlight=%28endian%29 > > Ok. That explains why the rsync won't work. But if I kick off a dovecot to dovecot replication (without doing the rsync first) will this work any better once the system catches up? This assumes (possibly incorrectly - please correct me if I am wrong) that the index files themselves aren't dsync'd byte-by-byte but instead the metadata/content from them is sent instead, and then the indexes are written to remote disk by the remote dovecot in the right arch and format that the remote machine can read and understand. Because if that's the case then I can probably make that work - just taking a hit on the initial sync which could take longer. > > Even if this doesn't end up working I figure I'll get to learn a little more about the indexes themselves in the process. dsyncing between servers (or in general using dsync-server) transfers all data using a portable protocol. So dsync source and destination can then have different endianess and it doesn't matter. From tss at iki.fi Wed May 6 21:49:52 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 7 May 2015 00:49:52 +0300 Subject: Dovecot Replication - Architecture Endianness? In-Reply-To: <5549F26D.8010702@reub.net> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <5549F26D.8010702@reub.net> Message-ID: <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> On 06 May 2015, at 13:52, Reuben Farrelly wrote: > > On 4/05/2015 11:06 PM, Teemu Huovila wrote: >>> Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? >> You can set the userdb to return an empty mail_replica variable for users you want to exclude from replication. >> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 >> >> br, >> Teemu Huovila > > One last question. Is it possible to achieve this with system users and PAM or do I need to basically create a new static userdb for system users? You can create a new userdb passwd-file that adds extra fields. So something like: userdb { driver = passwd result_success = continue-ok } userdb { driver = passwd-file args = /etc/dovecot/passwd.extra skip = notfound } > Could this be done via a per-user LDA setting or sieve? Replication would happen also with IMAP access. From sanvila at unex.es Wed May 6 23:45:29 2015 From: sanvila at unex.es (Santiago Vila) Date: Thu, 7 May 2015 01:45:29 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <6F1B94E8-A1E6-4BB6-8243-6E0F1286AEF5@iki.fi> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> <20150506092952.GA28906@cantor.unex.es> <20150506094322.GA29104@cantor.unex.es> <6F1B94E8-A1E6-4BB6-8243-6E0F1286AEF5@iki.fi> Message-ID: <20150506234529.GA1187@cantor.unex.es> On Thu, May 07, 2015 at 12:02:12AM +0300, Timo Sirainen wrote: > Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/94bd895721d8 Unfortunately, I applied single changeset 94bd895721d8 over Debian version 2.2.16-1 to create an updated Debian package and I can still reproduce the problem when using the updated package. This is a little bit surprising considering that the patch applies cleanly over version 2.2.16. What is the minimal set of changes from hg that I would need to apply over 2.2.16 for this to be fixed? Or should we wait for 2.2.17? I'm thinking about the way to fix the package in Debian unstable first, but after that it would be desirable to fix it in stable as well. Thanks. From skdovecot at smail.inf.fh-brs.de Thu May 7 05:44:14 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 May 2015 07:44:14 +0200 (CEST) Subject: Mapping other Inbox method (symlink vs ACL) In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 7 May 2015, Kevin Laurie wrote: > How do I specify the entire mailbox? > I only use ACL via telnet. I tried to use *(wildcard) to indicate > entire mailbox but I guess that does not work. IMAP ACLs do work for one mailbox (in the meaning of one mail folder) only. The command line tool doveadm, that you can run on the mail server itself, can do it easily. If you need to do it over IMAP, use some scripting. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVUr7rnz1H7kL/d9rAQLA3wf+P1Gpus/htx3wQ3XHi7YMwKdZ+ufmqlfj pQz8IqAiBjV2N0GFO4m0Elk5bamSFkI+MjzdYgAJEAG6O7ArJTIhwORPkiTDMcCY pfMHOufegPjVQsKjCvGLrhX48q9uxk/ww1itCPb4egVwgIZovdvrEFpMbXuLnSUz Uh4nTrQ7fUA5EgDciZK7jZAmMmXZRvophPEj/zIG8bDGYA6VvevhRYiUJ45On6Dc jwJV3+o81E63yNiBhiWwsuIGhNHjAQ0JMrdznB+58pLXvkNgs4OvqFK/8TykGay8 fXnZOTtrYa8sKVA9gZFzp8m6brkvCuQGB7UEJLszA0N9EcrXcUpxWQ== =2tdP -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Thu May 7 06:12:07 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Thu, 7 May 2015 08:12:07 +0200 (CEST) Subject: Dovecot and data migration In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 6 May 2015, Alain BERNARD wrote: > Our legacy data store retains a single copy of a message regardless of the > number of mailboxes in which that message resides. It does this by creating > hard links to that message in the mailboxes containing that message. > Thus, when we perform data migration to the server target (Dovecot), the > copies of the same message are copied over with the migration process > (imapsync). We use the storage format maildir. > With a small message store, this means that a lot of messages are > duplicated unnecessarily. How to reduce message store size due to duplicate > storage of identical messages ? There is no function in Dovecot doing that. For the synchronisation you can come up with some filesystem related script doing that easily. > Does a relinking function exist and can be run in real-time mode ? how can > we configure Dovecot to deduplicate for all users using a hash to determine > whether the file could be already exist ? In Dovecot v1 I did this with an external script, that hard linked equal files in cur and new directories that resides in more than 10 or so mailboxes. But in the production phase with Dovecot v2 you will face some culprit: with LMTP all messages are different now, because of the user-related Delivered-To and final Recieved header. With deliver this does not happen, but your MTA possibly adds different headers then, because usually LDAs are called per recipient. Dovecot deliver has the "-p" option to optionally hard link to file message file to the argument of -p. But then you must use some scripting to have your MTA call that script for all final recipients. You should also check, if Sieve is compatible with -p, because I remember some bug reports. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVUsCN3z1H7kL/d9rAQJp7Af/dPVmZcYQN48P4rgThc6RLFoB4PeLTF3B X42XqLmyje0d1Hv2YJMJXdSJccYJ4vp14MWJ0h11I3jOor17lnBGBTBqPyxZI7gL bYDJI2DUSh1CoQ2Sed9vRe5uKaDDlfuPFIym5JE4EJky8m8uEYSa+RRr/jtxbzpn RyKTn0SWls818hC5rISowvYyej5tvgZcq1lQn7yglqbriudJY33PHaa4EA7aaKVC ok4kiL9R0hKLTVjmeibxe0ZfI5MALVqkr1m5UOKXVj0M8lMHxx+qOoMlmkU3fXqI vwgvgYusvp3OeJJw23CJ5T0haaltzRcHJFil9F/4CLwMrsI44NnhgA== =JbnI -----END PGP SIGNATURE----- From reuben-dovecot at reub.net Thu May 7 11:24:11 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Thu, 7 May 2015 21:24:11 +1000 Subject: Dovecot Replication - Architecture Endianness? In-Reply-To: <9F1663B8-3199-4EC9-92C8-55B550474AF1@iki.fi> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <55477DFF.9060000@reub.net> <9F1663B8-3199-4EC9-92C8-55B550474AF1@iki.fi> Message-ID: <554B4B5B.6050500@reub.net> On 7/05/2015 7:47 AM, Timo Sirainen wrote: > On 04 May 2015, at 17:11, Reuben Farrelly > wrote: >> >> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >> Even if this doesn't end up working I figure I'll get to learn a >> little more about the indexes themselves in the process. > > dsyncing between servers (or in general using dsync-server) transfers > all data using a portable protocol. So dsync source and destination > can then have different endianess and it doesn't matter. I've tested this out today and can confirm it all works well - starting from nothing and doing the entire sync using dovecot. The takeaway from this is that for cross-arch replication an initial rsync is -not- the right thing to do in this situation. Thanks! Reuben From reuben-dovecot at reub.net Thu May 7 11:32:14 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Thu, 7 May 2015 21:32:14 +1000 Subject: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?] In-Reply-To: <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <5549F26D.8010702@reub.net> <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> Message-ID: <554B4D3E.4050805@reub.net> On 7/05/2015 7:49 AM, Timo Sirainen wrote: > On 06 May 2015, at 13:52, Reuben Farrelly wrote: >> >> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >>>> Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? >>> You can set the userdb to return an empty mail_replica variable for users you want to exclude from replication. >>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 >>> >>> br, >>> Teemu Huovila >> >> One last question. Is it possible to achieve this with system users and PAM or do I need to basically create a new static userdb for system users? > > You can create a new userdb passwd-file that adds extra fields. So something like: > > userdb { > driver = passwd > result_success = continue-ok > } > > userdb { > driver = passwd-file > args = /etc/dovecot/passwd.extra > skip = notfound > } This doesn't seem to work for me and my config has that exact config. My password.extra file has just one line for the one account I am testing with at the moment: user1:::::::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814 This breaks access for other system users such as my own account which do not have entries: ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login failure (pid=22573 id=1) (internal failure, 1 successful auths): user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, local=2001:44b8:31d4:1310::20, TLS which then starts soon spitting this out 10s of times per second in the mail log: May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738): Error: Auth worker sees different passdbs/userdbs than auth server. Maybe config just changed and this goes away automatically? This is with -hg latest as of now. This system uses PAM for local users. Do I need to replicate all of the system users including those who do not need any extra settings, in the passwd.extra file too? Is my syntax above for two mail_replica servers correct? Thanks, Reuben From memoefix at gmail.com Thu May 7 14:10:54 2015 From: memoefix at gmail.com (Alain BERNARD) Date: Thu, 7 May 2015 16:10:54 +0200 Subject: Dovecot and data migration In-Reply-To: References: Message-ID: Thank you. In fact, Postfix adds an individual Delivered-To: header line with the final envelope recipient address in order to stop mail forwarding loops as early as possible. This is a real problem with a multiple recipient email and find exact duplicates by comparing the hash values of emails. To perform a test, I used the -p parameter : # /usr/libexec/dovecot/deliver -p tempfile -d fredb # /usr/libexec/dovecot/deliver -p tempfile -d gregk # ll /usr/libexec/dovecot/deliver lrwxrwxrwx 1 root mail 11 16 f?vr. 09:15 /usr/libexec/dovecot/deliver -> dovecot-lda # ll /store/vmail/gam/fredb/Maildir/cur/ 1430986604.M985408P7547.mail6.domain.org\,S\=1037\,W\=1059\:2\,a -rw------- 1 vmail vmail 1037 7 mai 10:16 /store/vmail/gam/fredb/Maildir/cur/1430986604.M985408P7547.mail6.domain.org ,S=1037,W=1059:2,a However, the file isn't hard linked. So, fredb and gregk have the same file but I see that the number of hard links isn't 2 (files with a different inode number). Regards, 2015-05-07 8:12 GMT+02:00 Steffen Kaiser : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 6 May 2015, Alain BERNARD wrote: > > Our legacy data store retains a single copy of a message regardless of the >> number of mailboxes in which that message resides. It does this by >> creating >> hard links to that message in the mailboxes containing that message. >> > > Thus, when we perform data migration to the server target (Dovecot), the >> copies of the same message are copied over with the migration process >> (imapsync). We use the storage format maildir. >> > > With a small message store, this means that a lot of messages are >> duplicated unnecessarily. How to reduce message store size due to >> duplicate >> storage of identical messages ? >> > > There is no function in Dovecot doing that. For the synchronisation you > can come up with some filesystem related script doing that easily. > > Does a relinking function exist and can be run in real-time mode ? how can >> we configure Dovecot to deduplicate for all users using a hash to >> determine >> whether the file could be already exist ? >> > > In Dovecot v1 I did this with an external script, that hard linked equal > files in cur and new directories that resides in more than 10 or so > mailboxes. > > But in the production phase with Dovecot v2 you will face some culprit: > with LMTP all messages are different now, because of the user-related > Delivered-To and final Recieved header. With deliver this does not happen, > but your MTA possibly adds different headers then, because usually LDAs are > called per recipient. Dovecot deliver has the "-p" option to optionally > hard link to file message file to the argument of -p. But then you must use > some scripting to have your MTA call that script for all final recipients. > You should also check, if Sieve is compatible with -p, because I remember > some bug reports. > > - -- Steffen Kaiser > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1 > > iQEVAwUBVUsCN3z1H7kL/d9rAQJp7Af/dPVmZcYQN48P4rgThc6RLFoB4PeLTF3B > X42XqLmyje0d1Hv2YJMJXdSJccYJ4vp14MWJ0h11I3jOor17lnBGBTBqPyxZI7gL > bYDJI2DUSh1CoQ2Sed9vRe5uKaDDlfuPFIym5JE4EJky8m8uEYSa+RRr/jtxbzpn > RyKTn0SWls818hC5rISowvYyej5tvgZcq1lQn7yglqbriudJY33PHaa4EA7aaKVC > ok4kiL9R0hKLTVjmeibxe0ZfI5MALVqkr1m5UOKXVj0M8lMHxx+qOoMlmkU3fXqI > vwgvgYusvp3OeJJw23CJ5T0haaltzRcHJFil9F/4CLwMrsI44NnhgA== > =JbnI > -----END PGP SIGNATURE----- > From rs at sys4.de Thu May 7 15:20:31 2015 From: rs at sys4.de (Robert Schetterer) Date: Thu, 07 May 2015 17:20:31 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <20150506234529.GA1187@cantor.unex.es> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> <20150506092952.GA28906@cantor.unex.es> <20150506094322.GA29104@cantor.unex.es> <6F1B94E8-A1E6-4BB6-8243-6E0F1286AEF5@iki.fi> <20150506234529.GA1187@cantor.unex.es> Message-ID: <554B82BF.4090105@sys4.de> Am 07.05.2015 um 01:45 schrieb Santiago Vila: > On Thu, May 07, 2015 at 12:02:12AM +0300, Timo Sirainen wrote: >> Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/94bd895721d8 > > Unfortunately, I applied single changeset 94bd895721d8 over Debian > version 2.2.16-1 to create an updated Debian package and I can still > reproduce the problem when using the updated package. > > This is a little bit surprising considering that the patch applies > cleanly over version 2.2.16. > > What is the minimal set of changes from hg that I would need to apply > over 2.2.16 for this to be fixed? > > Or should we wait for 2.2.17? > > I'm thinking about the way to fix the package in Debian unstable > first, but after that it would be desirable to fix it in stable as > well. > > Thanks. > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages ... deb http://xi.rename-it.nl/debian/ testing-auto/dovecot-2.2 main is up2date most of the time Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From nathan at schultheiss.fr Thu May 7 19:04:16 2015 From: nathan at schultheiss.fr (Nathan Schultheiss) Date: Thu, 7 May 2015 21:04:16 +0200 (CEST) Subject: Prebuilt Binaries xi.rename-it.nl down ? Message-ID: <795972910.14689776.1431025456941.JavaMail.zimbra@schultheiss.fr> Hello, Since several days the repo (xi.rename-it.nl (217.119.238.14)) is no longer accessible. I'd found here for Debian/Ubuntu => http://wiki2.dovecot.org/PrebuiltBinaries Is it off, or temporarily down ? Could not connect to xi.rename-it.nl:80 (217.119.238.14). - connect (110: Connection timed out) Bests Regards, Nathan From user+dovecot at localhost.localdomain.org Thu May 7 19:12:41 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Thu, 07 May 2015 19:12:41 +0000 Subject: Prebuilt Binaries xi.rename-it.nl down ? In-Reply-To: <795972910.14689776.1431025456941.JavaMail.zimbra@schultheiss.fr> References: <795972910.14689776.1431025456941.JavaMail.zimbra@schultheiss.fr> Message-ID: <554BB929.8030901@localhost.localdomain.org> On 05/07/2015 07:04 PM, Nathan Schultheiss wrote: > Hello, > > Since several days the repo (xi.rename-it.nl (217.119.238.14)) is no longer accessible. > I'd found here for Debian/Ubuntu => http://wiki2.dovecot.org/PrebuiltBinaries > > Is it off, or temporarily down ? "Xi is broken at the moment." http://dovecot.org/list/dovecot/2015-April/100682.html Regards, Pascal -- The trapper recommends today: face1e55.1512721 at localdomain.org From teemu.huovila at dovecot.fi Fri May 8 08:10:17 2015 From: teemu.huovila at dovecot.fi (Teemu Huovila) Date: Fri, 08 May 2015 11:10:17 +0300 Subject: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?] In-Reply-To: <554B4D3E.4050805@reub.net> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <5549F26D.8010702@reub.net> <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> <554B4D3E.4050805@reub.net> Message-ID: <554C6F69.6090205@dovecot.fi> On 05/07/2015 02:32 PM, Reuben Farrelly wrote: > On 7/05/2015 7:49 AM, Timo Sirainen wrote: >> On 06 May 2015, at 13:52, Reuben Farrelly wrote: >>> >>> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >>>>> Also is there a way to restrict replication users aside from a crude hack around system first and last UIDs? >>>> You can set the userdb to return an empty mail_replica variable for users you want to exclude from replication. >>>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 >>>> >>>> br, >>>> Teemu Huovila >>> >>> One last question. Is it possible to achieve this with system users and PAM or do I need to basically create a new static >>> userdb for system users? >> >> You can create a new userdb passwd-file that adds extra fields. So something like: >> >> userdb { >> driver = passwd >> result_success = continue-ok >> } >> >> userdb { >> driver = passwd-file >> args = /etc/dovecot/passwd.extra >> skip = notfound >> } > > This doesn't seem to work for me and my config has that exact config. My password.extra file has just one line for the one > account I am testing with at the moment: > > user1:::::::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814 > > This breaks access for other system users such as my own account which do not have entries: > > ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login failure (pid=22573 id=1) (internal failure, 1 successful > auths): user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, local=2001:44b8:31d4:1310::20, TLS > > which then starts soon spitting this out 10s of times per second in the mail log: > > May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738): Error: Auth worker sees different passdbs/userdbs than auth > server. Maybe config just changed and this goes away automatically? > > This is with -hg latest as of now. > > This system uses PAM for local users. Do I need to replicate all of the system users including those who do not need any extra > settings, in the passwd.extra file too? > > Is my syntax above for two mail_replica servers correct? A bit unsure about the config syntax, so I can not advice on that, but there were some bugs in auth yesterday. Maybe you could retest with f2a8e1793718 or newer. Make sure configs on both sides are in sync. Thank you for your continued testing, Teemu Huovila From paolo.cravero at csi.it Fri May 8 09:22:40 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Fri, 8 May 2015 11:22:40 +0200 (CEST) Subject: Different mdbox_rotate_size for primary and alternate storage Message-ID: <1342253338.615853.1431076961035.JavaMail.open-xchange@comunica.csi.it> Hello. In order to speed up backups of very very old messages I would like to set two different limits for mdbox_rotate_size. Like, 50M for primary storage and 100M or larger for alternate storage. There is no mention in docs or such a possibility, so I assume it is not possible. Is that correct? While I am at it, is it possible to configure primary storage as maildir (sturdy indexes) and altstorage as mdbox (more delicate indexes)? Thanks, Paolo From reuben-dovecot at reub.net Fri May 8 13:04:28 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Fri, 8 May 2015 23:04:28 +1000 Subject: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?] In-Reply-To: <554C6F69.6090205@dovecot.fi> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <5549F26D.8010702@reub.net> <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> <554B4D3E.4050805@reub.net> <554C6F69.6090205@dovecot.fi> Message-ID: <554CB45C.2080205@reub.net> On 8/05/2015 6:10 PM, Teemu Huovila wrote: > On 05/07/2015 02:32 PM, Reuben Farrelly wrote: >> On 7/05/2015 7:49 AM, Timo Sirainen wrote: >>> On 06 May 2015, at 13:52, Reuben Farrelly >>> wrote: >>>> >>>> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >>>>>> Also is there a way to restrict replication users aside >>>>>> from a crude hack around system first and last UIDs? >>>>> You can set the userdb to return an empty mail_replica >>>>> variable for users you want to exclude from replication. >>>>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 >>>>> >>>>> br, Teemu Huovila >>>> >>>> One last question. Is it possible to achieve this with system >>>> users and PAM or do I need to basically create a new static >>>> userdb for system users? >>> >>> You can create a new userdb passwd-file that adds extra fields. >>> So something like: >>> >>> userdb { driver = passwd result_success = continue-ok } >>> >>> userdb { driver = passwd-file args = /etc/dovecot/passwd.extra >>> skip = notfound } >> >> This doesn't seem to work for me and my config has that exact >> config. My password.extra file has just one line for the one >> account I am testing with at the moment: >> >> user1:::::::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814 >> >> >> This breaks access for other system users such as my own account which do not have entries: >> >> ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login >> failure (pid=22573 id=1) (internal failure, 1 successful auths): >> user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, >> local=2001:44b8:31d4:1310::20, TLS >> >> which then starts soon spitting this out 10s of times per second in >> the mail log: >> >> May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738): >> Error: Auth worker sees different passdbs/userdbs than auth server. >> Maybe config just changed and this goes away automatically? >> >> This is with -hg latest as of now. >> >> This system uses PAM for local users. Do I need to replicate all >> of the system users including those who do not need any extra >> settings, in the passwd.extra file too? >> >> Is my syntax above for two mail_replica servers correct? > A bit unsure about the config syntax, so I can not advice on that, > but there were some bugs in auth yesterday. Maybe you could retest > with f2a8e1793718 or newer. Make sure configs on both sides are in > sync. > > Thank you for your continued testing, Teemu Huovila > With -hg as of now it's still not any better: tornado log # dovecot --version 2.2.16 (f2a8e1793718+) tornado log # =================== # System users (NSS, /etc/passwd, or similiar). In many systems nowadays this # uses Name Service Switch, which is configured in /etc/nsswitch.conf. userdb { # driver = passwd # [blocking=no] #args = # Override fields from passwd #override_fields = home=/home/virtual/%u result_success = continue-ok } # Add some extra fields such as replication.. userdb { driver = passwd-file args = /etc/dovecot/passwd.extra skip = notfound } ============== May 8 22:59:11 tornado.reub.net dovecot: imap: Error: Authenticated user not found from userdb, auth lookup id=586547201 (client-pid=29035 client-id=1) May 8 22:59:11 tornado.reub.net dovecot: imap-login: Internal login failure (pid=29035 id=1) (internal failure, 1 successful auths): user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, local=2001:44b8:31d4:1310::20, TLS It logs an awful lot of those lines in short succession also, at least 15 per second... Reuben From pfaf at wisdomsoftware.net Fri May 8 15:24:11 2015 From: pfaf at wisdomsoftware.net (Panayiotis Fafakos) Date: Fri, 08 May 2015 18:24:11 +0300 Subject: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2 In-Reply-To: <5547F00A.8050001@wisdomsoftware.net> References: <5547F00A.8050001@wisdomsoftware.net> Message-ID: <554CD51B.6090709@wisdomsoftware.net> Do we have a way to keep the user \Seen flags in public folders when an email is moved in another folder? Sample test to reproduce: Step a - UserA and UserB have both read the email in PublicFolderA. Step b - UserA moves the email to PublicFolderB. UserA still sees the email as read (with the \Seen flag), this is expected behaviour and we are ok with this. Step c - UserB sees the email in PublicFolderB as an unread message!? He is puzzled since he has already read this message and asks why this is happening. Can we correct this behaviour? Thank you all, Panos. On 5/5/2015 01:17, Panayiotis Fafakos wrote: > Dear all, > > we have succesfully configured Dovecot v2.2.13 in debian wheezy 7.8 > (using backports) > using Maildir structure, to use private index files for the \Seen flag > on a per user basis. > > All users access their emails and Public Folders using IMAP protocol. > > The problem is that when a user moves an email from publicFolderA to > publicFolderB under the same namespace > the other users see this message as unread, although they have > actually read it when it was in publicFolderA. > > Please note that this is an old message which has been moved , it was > not copied, so the actual UID should be the same... > > Is there a way to keep the \Seen flag for the messages that are moved > from folder to folder? > > Is there a way to keep the \Seen flag in a database, so that we can > ignore the folder structure and only check the message UIDs? We could > use MySQL, PgSQL or even SQLite... > > Below follows the Public-Folder namespace declaration: > ------------------------------------------ > namespace { > inbox = no > location = > maildir:/var/vmail/Public-Folders:LAYOUT=fs:INDEXPVT=~/Maildir/public/%u > prefix = Public-Folders/ > separator = / > subscriptions = no > type = public > } > ------------------------------------------ > > With the above system configuration we have the complete folder > structure under ~/Maildir/public/%u, and many log files, one for each > folder a user has accessed. > Could we only have one index file for each user for all the public > folder structure under the same namespace? > > Kind regards to all, > Panayiotis Fafakos > > From tss at iki.fi Fri May 8 15:42:52 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 8 May 2015 18:42:52 +0300 Subject: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2 In-Reply-To: <554CD51B.6090709@wisdomsoftware.net> References: <5547F00A.8050001@wisdomsoftware.net> <554CD51B.6090709@wisdomsoftware.net> Message-ID: <7B0FE6A6-11F8-4D72-A17A-910595423FFA@iki.fi> On 08 May 2015, at 18:24, Panayiotis Fafakos wrote: > > Do we have a way to keep the user \Seen flags in public folders > when an email is moved in another folder? > > Sample test to reproduce: > Step a - UserA and UserB have both read the email in PublicFolderA. > Step b - UserA moves the email to PublicFolderB. UserA still sees the email as read (with the \Seen flag), this is expected behaviour and we are ok with this. > Step c - UserB sees the email in PublicFolderB as an unread message!? He is puzzled since he has already read this message and asks why this is happening. Can we correct this behaviour? No, there's no way to fix this without major changes to how Dovecot works. Per-user seen flags are stored in private per-user index files. When user A moves mail to another folder the shared index and user A's index are updated to copy the flags. User A doesn't know what other users might have the folder accessible (and especially both source and destination folder). Even if it did know, now moving a mail might involve updating a lot of indexes every time a mail is moved, which is way too slow. One possible future solution would be to move more towards GMail-like labels instead of folders. We have beginnings of such code. I'm not sure yet how that could be made to work with shared folders. From pfaf at wisdomsoftware.net Fri May 8 16:05:06 2015 From: pfaf at wisdomsoftware.net (Panayiotis Fafakos) Date: Fri, 08 May 2015 19:05:06 +0300 Subject: PublicFolders using Maildir and INDEXPVT in Dovecot v2.2 In-Reply-To: <7B0FE6A6-11F8-4D72-A17A-910595423FFA@iki.fi> References: <5547F00A.8050001@wisdomsoftware.net> <554CD51B.6090709@wisdomsoftware.net> <7B0FE6A6-11F8-4D72-A17A-910595423FFA@iki.fi> Message-ID: <554CDEB2.1050606@wisdomsoftware.net> Dear Timo, thank you very much for your answer and for the wonderful DOVECOT project. Can you please tell us where is the code that moves the index record for the specified email, so that we can perhaps provide a manually updated list of users, that we want dovecot to update also. This would be a custom code modification which we would need to keep-up-to-date, and we could also publish it if anyone would be interested. We are supporting companies up to 10 users, so this minor change would not be a problem to maintain. i.e. we could have a list of users in the main departmental public folder who's index would also be updated once an email in the underlining folder structure changes folder. We are also a software company and would be very interested in investing to get to know more for the dovecot project. Thank you in advance for your support, Panos. On 8/5/2015 18:42, Timo Sirainen wrote: > On 08 May 2015, at 18:24, Panayiotis Fafakos wrote: >> Do we have a way to keep the user \Seen flags in public folders >> when an email is moved in another folder? >> >> Sample test to reproduce: >> Step a - UserA and UserB have both read the email in PublicFolderA. >> Step b - UserA moves the email to PublicFolderB. UserA still sees the email as read (with the \Seen flag), this is expected behaviour and we are ok with this. >> Step c - UserB sees the email in PublicFolderB as an unread message!? He is puzzled since he has already read this message and asks why this is happening. Can we correct this behaviour? > No, there's no way to fix this without major changes to how Dovecot works. > > Per-user seen flags are stored in private per-user index files. When user A moves mail to another folder the shared index and user A's index are updated to copy the flags. User A doesn't know what other users might have the folder accessible (and especially both source and destination folder). Even if it did know, now moving a mail might involve updating a lot of indexes every time a mail is moved, which is way too slow. > > One possible future solution would be to move more towards GMail-like labels instead of folders. We have beginnings of such code. I'm not sure yet how that could be made to work with shared folders. > > From superinterstellar at gmail.com Fri May 8 16:36:32 2015 From: superinterstellar at gmail.com (Kevin Laurie) Date: Fri, 8 May 2015 23:36:32 +0700 Subject: Cant use doveadm to set ACL . [request for help] Message-ID: Hi, i keep getting error for using doveadm acl get commands. Below is the output for grep 'socket_path' :- grep 'socket_path' /etc/dovecot/dovecot.conf auth_socket_path = /var/run/dovecot/auth-master auth_socket_path = /var/run/dovecot/auth-master [root at mail root]# doveadm acl get -u bob at mydomain.net -S /var/run/dovecot/auth-master -m Inbox doveadm(bob at mydomain.net): Error: doveadm server sent invalid handshake: VERSION 1 1 doveadm(bob at mydomain.net): Error: /var/run/dovecot/auth-master: Internal failure for bob at mydomain.net ID Global Rights From lists at tigertech.com Fri May 8 20:34:38 2015 From: lists at tigertech.com (Robert L Mathews) Date: Fri, 08 May 2015 13:34:38 -0700 Subject: Full text search indexes not used for header/body OR queries? Message-ID: <554D1DDE.6060403@tigertech.com> I've noticed that when using Lucene full text search, most queries use the indexes and/or header cache and are fast: . SEARCH BODY test . OK Search completed (0.001 secs). . SEARCH SUBJECT test . OK Search completed (0.053 secs). . SEARCH BODY test SUBJECT test . OK Search completed (0.002 secs). . SEARCH OR SUBJECT test FROM test . OK Search completed (0.093 secs). But an OR query that mixes headers and body does not use the available FTS indexes for the BODY part and is slow: . SEARCH OR BODY test SUBJECT test * OK Searched 62% of the mailbox, ETA 0:05 * OK Searched 70% of the mailbox, ETA 0:04 . OK Search completed (15.147 secs). Is this the expected behavior? Since the FTS code can handle an AND of header and body searches, I'm surprised it doesn't do the same for an OR. I noticed this while tracking down poor performance in Thunderbird, which issues searches like this: UID SEARCH RETURN (ALL) (OR FROM "Evelyn" (OR SUBJECT "Evelyn" (OR TO "Evelyn" (OR CC "Evelyn" BODY "Evelyn")))) NOT DELETED These are slow even with FTS enabled because of this behavior. I'm using Dovecot 2.1.7 from Debian wheezy. (I know this is outdated; however, I've examined the 2.1.x and 2.2.x changelogs and found no mention of it.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From lists at tigertech.com Sat May 9 04:47:35 2015 From: lists at tigertech.com (Robert L Mathews) Date: Fri, 08 May 2015 21:47:35 -0700 Subject: Full text search indexes not used for header/body OR queries? In-Reply-To: <554D1DDE.6060403@tigertech.com> References: <554D1DDE.6060403@tigertech.com> Message-ID: <554D9167.9010606@tigertech.com> As a followup to my own message: On 5/8/15 1:34 PM, Robert L Mathews wrote: > I've noticed that when using Lucene full text search, most queries use > the indexes and/or header cache and are fast [...] But an OR query that > mixes headers and body does not use the available > FTS indexes for the BODY part and is slow: This turned out to be my own fault because of a foolish mistake I made when testing. Dovecot actually works fine on all the search queries I mentioned, even in version 2.1.7. My apologies for the noise on the list. (My mistake was that when switching from Squat to Lucene, I didn't remove a local patch that prevented FTS from being used for header searches, because I thought the patch was only affecting Squat. That patch was to workaround what I reported in . But the patch also affected Lucene.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From tss at iki.fi Sat May 9 17:41:23 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 9 May 2015 20:41:23 +0300 Subject: v2.2.17 release candidate released Message-ID: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. * Dovecot no longer checks or warns if a mountpoint is removed. This was causing more trouble than it was worth. Make sure that all the mountpoints that Dovecot accesses aren't writable by mail processes when they're unmounted. * dict server wasn't properly escaping/unescaping data. Fixing this broke backwards compatibility with data that contains line feeds. This hopefully affects only very few installations. If you're using dict to save multiline data (Sieve scripts to SQL), you may be affected. * imap: SPECIAL-USE capability is no longer advertised if there are no special_use flags specified for any mailboxes. + lmtp: Added lmtp_hdr_delivery_address setting to specify whether to include email address in Delivered-To: and Received: headers. + Added initial version of full text search library, which includes language-specific text normalization and filtering. This is still in development, but it's already possible to use for testing with fts-lucene and fts-solr. + lda, lmtp: deliver_log_format can now include %{delivery_time}, which expands to how many milliseconds it took to deliver the mail. With LMTP %{session_time} also expands to how many milliseconds the LMTP session took, not including the delivery time. + lmtp proxy: Mail delivery logging includes timing information. + imap: Most IMAP commands now include in the tagged reply how many milliseconds it took to run the command (not counting the time spent on waiting for the IMAP client to read/write data). + director: Implemented director_proxy_maybe passdb extra field to be able to run director and backend in the same Dovecot instance. (LMTP doesn't support mixed proxy/non-proxy destinations currently.) + doveadm: Added -F parameter to read a list of users from the given file and run the command for all the users. This is similar to -A parameter reading the list of users from userdb lookup. - auth: If auth_master_user_separator was set, auth process could be crashed by trying to log in with empty master username. - imap-login, pop3-login: Fixed crash on handshake failures with new OpenSSL versions (v1.0.2) when SSLv3 was disabled. - auth: If one passdb fails allow_nets check, it shouldn't have failed all the other passdb checks later on. - imap: Server METADATA couldn't be accessed - imapc: Fixed \Muted label handling in gmail-migration. - imapc: Various bugfixes and improvements. - Trash plugin fixes by Alexei Gradinari - mbox: Fixed crash/corruption in some situations when the first mail was expunged. From odhiambo at gmail.com Sat May 9 19:53:08 2015 From: odhiambo at gmail.com (Odhiambo Washington) Date: Sat, 9 May 2015 22:53:08 +0300 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: Compiled successfully on FreeBSD-{8.4|9.3|10.1}. Running also on those and watching|listening for any complains. Although on one of the 8.4, I have seen so many IMAP process started. I hope this is some uses connecting via IMAP. I have never seen that before. root at gw:/usr/home/wash/Tools/Dovecot/2.2/dovecot-2.2.17.rc1 # ps ax | grep dove 47491 ?? I 0:00.00 supervise dovecot 47498 ?? S 0:00.13 /opt/dovecot2.2/sbin/dovecot -F -c /opt/dovecot2.2/etc/dovecot/dovecot.conf 47501 ?? I 0:00.01 /usr/local/bin/multilog t s1000000 n20 /var/log/dovecot/ 47509 ?? S 0:00.02 dovecot/anvil 47510 ?? S 0:00.02 dovecot/log 47570 ?? I 0:00.02 dovecot/imap 47571 ?? I 0:00.01 dovecot/imap 47572 ?? S 0:00.01 dovecot/imap 47573 ?? I 0:00.02 dovecot/imap 47574 ?? S 0:00.02 dovecot/imap 47575 ?? S 0:00.01 dovecot/imap 47576 ?? I 0:00.02 dovecot/imap 47577 ?? I 0:00.02 dovecot/imap 47578 ?? S 0:00.01 dovecot/imap 47579 ?? S 0:00.01 dovecot/imap 47580 ?? S 0:00.02 dovecot/imap 47581 ?? S 0:00.01 dovecot/imap 47584 ?? I 0:00.01 dovecot/imap 47587 ?? I 0:00.01 dovecot/imap 47591 ?? I 0:00.02 dovecot/imap 47594 ?? I 0:00.02 dovecot/imap 47597 ?? S 0:00.02 dovecot/imap 47598 ?? S 0:00.02 dovecot/imap 47600 ?? S 0:00.01 dovecot/imap 47601 ?? S 0:00.02 dovecot/imap 47848 ?? S 0:00.04 dovecot/config 47849 ?? S 0:00.03 dovecot/auth 47879 ?? I 0:00.00 dovecot/ssl-params 47908 ?? I 0:00.01 dovecot/pop3-login 48012 ?? I 0:00.01 dovecot/pop3-login 48057 ?? S 0:00.03 dovecot/auth -w 48087 0 S+ 0:00.00 grep dove root at gw:/usr/home/wash/Tools/Dovecot/2.2/dovecot-2.2.17.rc1 # On 9 May 2015 at 20:41, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig > > Quite a lot of changes, so here's a release candidate first. If no > problems are reported I'll make v2.2.17 release early next week. Please > test especially non-Linux systems as there are some larger > configure/Makefile changes done since v2.2.16. > > * Dovecot no longer checks or warns if a mountpoint is removed. > This > was causing more trouble than it was worth. Make sure that all > the > mountpoints that Dovecot accesses aren't writable by mail > processes > when they're unmounted. > * dict server wasn't properly escaping/unescaping data. Fixing this > broke backwards compatibility with data that contains line feeds. > This hopefully affects only very few installations. If you're > using > dict to save multiline data (Sieve scripts to SQL), you may be > affected. > * imap: SPECIAL-USE capability is no longer advertised if there are > no special_use flags specified for any mailboxes. > > + lmtp: Added lmtp_hdr_delivery_address setting to specify whether > to include email address in Delivered-To: and Received: headers. > + Added initial version of full text search library, which includes > language-specific text normalization and filtering. This is still > in development, but it's already possible to use for testing with > fts-lucene and fts-solr. > + lda, lmtp: deliver_log_format can now include %{delivery_time}, > which expands to how many milliseconds it took to deliver the > mail. > With LMTP %{session_time} also expands to how many milliseconds > the > LMTP session took, not including the delivery time. > + lmtp proxy: Mail delivery logging includes timing information. > + imap: Most IMAP commands now include in the tagged reply how many > milliseconds it took to run the command (not counting the time > spent > on waiting for the IMAP client to read/write data). > + director: Implemented director_proxy_maybe passdb extra field to > be able to run director and backend in the same Dovecot instance. > (LMTP doesn't support mixed proxy/non-proxy destinations > currently.) > + doveadm: Added -F parameter to read a list of users from > the > given file and run the command for all the users. This is > similar to > -A parameter reading the list of users from userdb lookup. > - auth: If auth_master_user_separator was set, auth process could > be > crashed by trying to log in with empty master username. > - imap-login, pop3-login: Fixed crash on handshake failures with > new > OpenSSL versions (v1.0.2) when SSLv3 was disabled. > - auth: If one passdb fails allow_nets check, it shouldn't have > failed > all the other passdb checks later on. > - imap: Server METADATA couldn't be accessed > - imapc: Fixed \Muted label handling in gmail-migration. > - imapc: Various bugfixes and improvements. > - Trash plugin fixes by Alexei Gradinari > - mbox: Fixed crash/corruption in some situations when the first > mail > was expunged. > -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 "I can't hear you -- I'm using the scrambler." From trashcan at odo.in-berlin.de Sat May 9 20:17:27 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 9 May 2015 22:17:27 +0200 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> Hi ? Timo Sirainen wrote: > Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. 2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see <554773D2.9060609 at dovecot.fi> from Teemu Huovila). This is from my point of view regarding portability issues "suboptimal". Both, word-break-data.sh and word-boundary-data.sh have comments about preferable use awk, perl, or ? instead, IIRC. Teemu himself already mentioned in <554773D2.9060609 at dovecot.fi> that this is suboptimal, and that he is willing to solve that issue, thus, I am extremely far from blaming anyone involved in making this great software available. But, maybe it's worth solving this issue for those non-linux systems without bash installed per default before releasing dovecot v2.2.17, finally. Besides that: Thanks for all of you involved making dovecot available! Regards, Michael From trashcan at odo.in-berlin.de Sat May 9 20:19:22 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 9 May 2015 22:19:22 +0200 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: <5547CD66.7040904@localhost.localdomain.org> References: <5547CD66.7040904@localhost.localdomain.org> Message-ID: Hi ? Pascal Volk wrote: > On 04/24/2015 07:00 PM, Michael Grimm wrote: >> ... >> 2) I don't have a python binary installed, only a python2 link to the python27 binary (FBSD, and python27 from ports). >> Thus, update-version.sh will fail to evaluate hg's changeset. As a quick fix I needed to create a link: python -> python2 >> > > This is a less or more 'common' system configuration error. ;-) > See https://forums.freebsd.org/threads/solved-python-command-not-found.46449/ Uii. Thank you very much for that link! Fixed, now. I din't know that. Regards, Michael From trashcan at odo.in-berlin.de Sat May 9 20:25:48 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sat, 9 May 2015 22:25:48 +0200 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: <554773D2.9060609@dovecot.fi> References: <554773D2.9060609@dovecot.fi> Message-ID: Hi ? Teemu Huovila wrote: > On 04/24/2015 10:00 PM, Michael Grimm wrote: >> 1) I'm trying to compile a recent hg dovecot version (008632bdfd2c) at a FBSD10-STABLE system without success: [?] >> fts-tokenizer-generic.c:214:18: error: use of undeclared identifier 'MidNum' >> if (uint32_find(MidNum, N_ELEMENTS(MidNum), c, &idx)) >> ^ >> fatal error: too many errors emitted, stopping now [-ferror-limit=] >> 20 errors generated. >> Makefile:591: recipe for target 'fts-tokenizer-generic.lo' failed >> gmake[4]: *** [fts-tokenizer-generic.lo] Error 1 >> gmake[4]: Leaving directory '/usr/local/etc/dovecot/SOURCE/dovecot-2.2/src/lib-fts' >> >> >> 2) I don't have a python binary installed, only a python2 link to the python27 binary (FBSD, and python27 from ports). >> Thus, update-version.sh will fail to evaluate hg's changeset. As a quick fix I needed to create a link: python -> python2 > Both these are only run if you compile the source from hg, as you did. Official release tar-balls should not have this issue. > Still, it is not optimal and Ill definitely look into solving 1) when I have time available for that. > > For temporarily solving 1) it is worth noticing the scripts word-break-data.sh and word-boundary-data.sh depend on /bin/bash. Well that works, although ti's suboptimal because FBSD is more or less bash-free. One, has to install a bash port, and one has to to create a link for /usr/bin/local/bin/bash as /bin/bash. Easy to do, but no perfect solution, when it comes to OS portability, though. > You could either install bash That works well. > or just try if it works if you change it to /bin/sh and use whatever FreeBSD has that pointing to. That fails because /bin/sh equals /bin/csh at FBSD. Thanks and with kind regards, Michael From gcr+dovecot at tharned.org Sat May 9 23:00:50 2015 From: gcr+dovecot at tharned.org (Greg Rivers) Date: Sat, 09 May 2015 18:00:50 -0500 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: References: <554773D2.9060609@dovecot.fi> Message-ID: <2109150.CC7IZC85Na@flake.tharned.org> On Saturday, May 09, 2015 22:25:48 Michael Grimm wrote: > > or just try if it works if you change it to /bin/sh and use whatever > > FreeBSD has that pointing to. > That fails because /bin/sh equals /bin/csh at FBSD. > I don't know if it fails or not, but if it does this is not the reason. /bin/sh most certainly is not /bin/csh; if it were, the system would not boot given that all the rc start-up scripts are written in Bourne shell. OTOH, /bin/csh and /bin/tcsh are identical: $ freebsd-version -uk 10.1-RELEASE-p9 10.1-RELEASE-p9 $ ls -li /bin/*sh 108 -r-xr-xr-x 2 root wheel 382368 Nov 11 15:03 /bin/csh* 118 -r-xr-xr-x 1 root wheel 142184 Nov 11 15:03 /bin/sh* 108 -r-xr-xr-x 2 root wheel 382368 Nov 11 15:03 /bin/tcsh* -- Greg Rivers From jerry at seibercom.net Sat May 9 23:13:46 2015 From: jerry at seibercom.net (Jerry) Date: Sat, 9 May 2015 19:13:46 -0400 Subject: v2.2.17 release candidate released In-Reply-To: <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> Message-ID: <20150509191346.4d1567c6@seibercom.net> On Sat, 9 May 2015 22:17:27 +0200, Michael Grimm stated: >Hi ? > >Timo Sirainen wrote: > >> Please test especially non-Linux systems as there are some larger >> configure/Makefile changes done since v2.2.16. > >2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see ><554773D2.9060609 at dovecot.fi> from Teemu Huovila). This is from my point of >view regarding portability issues "suboptimal". Both, word-break-data.sh and >word-boundary-data.sh have comments about preferable use awk, perl, or ? >instead, IIRC. Teemu himself already mentioned in ><554773D2.9060609 at dovecot.fi> that this is suboptimal, and that he is >willing to solve that issue, thus, I am extremely far from blaming anyone >involved in making this great software available. But, maybe it's worth >solving this issue for those non-linux systems without bash installed per >default before releasing dovecot v2.2.17, finally. > >Besides that: Thanks for all of you involved making dovecot available! > >Regards, >Michael > For what its worth, I run FreeBSD 10.1 and I have Bash. Personally, I could not work without it. It is not a big deal to install it on FreeBSD. My question is does it have to be the default shell in dovecot v2.2.17 or just installed? -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From cross+dovecot at distal.com Sun May 10 00:59:35 2015 From: cross+dovecot at distal.com (Chris Ross) Date: Sat, 9 May 2015 20:59:35 -0400 Subject: v2.2.17 release candidate released In-Reply-To: <20150509191346.4d1567c6@seibercom.net> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> <20150509191346.4d1567c6@seibercom.net> Message-ID: <79770603-32B7-42F9-A0C8-8774AECD88F3@distal.com> On May 9, 2015, at 19:13 , Jerry wrote: >> 2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed (see >> <554773D2.9060609 at dovecot.fi> from Teemu Huovila).[...]. But, maybe it's worth >> solving this issue for those non-linux systems without bash installed per >> default before releasing dovecot v2.2.17, finally. >> >> Besides that: Thanks for all of you involved making dovecot available! > > For what its worth, I run FreeBSD 10.1 and I have Bash. Personally, I could > not work without it. It is not a big deal to install it on FreeBSD. My > question is does it have to be the default shell in dovecot v2.2.17 or just > installed? I run FreeBSD 9, 10, and 10.1. And I don't have bash installed except on the one system I use to build lots of ports. Because, there are other ports that also presume the availability of bash, and therefore require it. I'd much rather be able to build dovecot without bash, because I don't have bash installed on my FreeBSD 10.1 mail server, where I run dovecot. I'd also be willing to help by porting any of the scripts that require bash to something else that is available in the base installation of FreeBSD as well as the variety of Linux distributions. I agree that it's not hard to install the bash port. But if you don't plan to use it for anything else, it's better not to have to. - Chris From reuben-dovecot at reub.net Sun May 10 03:19:51 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Sun, 10 May 2015 13:19:51 +1000 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: <554ECE57.5040901@reub.net> I'm now seeing these crashes in the indexing code: May 10 12:22:32 tornado.reub.net dovecot: master: Dovecot v2.2.17.rc1 (3903badc4ee0+) starting up for imap, lmtp, sieve ... May 10 13:07:50 tornado.reub.net dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for reuben May 10 13:07:50 tornado.reub.net dovecot: indexer-worker(reuben): Fatal: master: service(indexer-worker): child 18532 killed with signal 11 (core not dumped) Every time a delivery happens or mail is moved the indexer crashes out. It doesn't seem to have any visible end user impact though. How do I go about debugging this further? How do I go about getting a backtrace on what is essentially a child process that doesn't produce a core? (Note that my installation normally does produce cores when it crashes, so there's something different about these indexer processes..) Reuben On 10/05/2015 3:41 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig From tss at iki.fi Sun May 10 07:27:25 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 10 May 2015 10:27:25 +0300 Subject: v2.2.17 release candidate released In-Reply-To: <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> Message-ID: <9F184DCC-29EC-43F5-9088-C1AE8D9F14DD@iki.fi> > Michael Grimm kirjoitti 9.5.2015 kello 23.17: > > Hi ? > > Timo Sirainen wrote: > >> Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. > > 2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed Dovecot hg requires bash currently, but the rc1 tarball should compile even without bash because the generated files are included in the tarball. Or should at least.. From tss at iki.fi Sun May 10 07:47:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 10 May 2015 10:47:49 +0300 Subject: v2.2.17 release candidate released In-Reply-To: <554ECE57.5040901@reub.net> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <554ECE57.5040901@reub.net> Message-ID: <8062E904-C42A-492D-B2C7-035DCCAEAD1D@iki.fi> On 10 May 2015, at 06:19, Reuben Farrelly wrote: > > I'm now seeing these crashes in the indexing code: > > May 10 12:22:32 tornado.reub.net dovecot: master: Dovecot v2.2.17.rc1 (3903badc4ee0+) starting up for imap, lmtp, sieve > ... > May 10 13:07:50 tornado.reub.net dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for reuben > May 10 13:07:50 tornado.reub.net dovecot: indexer-worker(reuben): Fatal: master: service(indexer-worker): child 18532 killed with signal 11 (core not dumped) > > Every time a delivery happens or mail is moved the indexer crashes out. It doesn't seem to have any visible end user impact though. What's your current doveconf -n output? I did a quick test with fts-lucene and it was working. > How do I go about debugging this further? How do I go about getting a backtrace on what is essentially a child process that doesn't produce a core? (Note that my installation normally does produce cores when it crashes, so there's something different about these indexer processes..) One thing that often works is: mkdir /var/core; chmod 0777 /var/core echo "/var/core/%p" > /proc/sys/kernel/core_pattern If that doesn't help, you can also try: sysctl -w fs.suid_dumpable=2 (and disable afterwards) Also if you can get indexer-worker to start without crashing (new mail delivery works?) you can attach gdb into it and wait for it to crash (gdb -p , cont). From reuben-dovecot at reub.net Sun May 10 08:47:15 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Sun, 10 May 2015 18:47:15 +1000 Subject: v2.2.17 release candidate released In-Reply-To: <8062E904-C42A-492D-B2C7-035DCCAEAD1D@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <554ECE57.5040901@reub.net> <8062E904-C42A-492D-B2C7-035DCCAEAD1D@iki.fi> Message-ID: <554F1B13.1040905@reub.net> On 10/05/2015 5:47 PM, Timo Sirainen wrote: >> How do I go about debugging this further? How do I go about >> getting a backtrace on what is essentially a child process that >> doesn't produce a core? (Note that my installation normally does >> produce cores when it crashes, so there's something different about >> these indexer processes..) > > One thing that often works is: > > mkdir /var/core; chmod 0777 /var/core echo "/var/core/%p" > > /proc/sys/kernel/core_pattern > > If that doesn't help, you can also try: sysctl -w fs.suid_dumpable=2 > (and disable afterwards) > > Also if you can get indexer-worker to start without crashing (new > mail delivery works?) you can attach gdb into it and wait for it to > crash (gdb -p , cont). Ok we have some (well, lots actually) of coredumps now: tornado core # gdb /usr/libexec/dovecot/indexer-worker 26697 GNU gdb (Gentoo 7.9 vanilla) 7.9 Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-pc-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/dovecot/indexer-worker...done. Attaching to program: /usr/libexec/dovecot/indexer-worker, process 26697 ptrace: No such process. [New LWP 26697] warning: Could not load shared library symbols for linux-vdso.so.1. Do you need "set solib-search-path" or "set sysroot"? [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/indexer-worker'. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) at fts-user.c:302 302 fts-user.c: No such file or directory. (gdb) info locals hdr = 0x1701f08 key = {uid = 8553, type = FTS_BACKEND_BUILD_KEY_HDR, part = 0x1696860, hdr_name = 0x1702120 "To", body_content_type = 0x0, body_content_disposition = 0x0} (gdb) bt full #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) at fts-user.c:302 fuser = 0x0 lang = 0x0 error = 0xf __FUNCTION__ = "fts_user_get_data_lang" #1 0x00007f4834fc1bdb in fts_build_mail_header (ctx=0x7ffd5e021ca0, block=0x7ffd5e021c80) at fts-build-mail.c:164 hdr = 0x1701f08 key = {uid = 8553, type = FTS_BACKEND_BUILD_KEY_HDR, part = 0x1696860, hdr_name = 0x1702120 "To", body_content_type = 0x0, body_content_disposition = 0x0} #2 0x00007f4834fc2b68 in fts_build_mail_real (update_ctx=0x16f5130, mail=0x17010f0) at fts-build-mail.c:540 ctx = {mail = 0x17010f0, update_ctx = 0x16f5130, content_type = 0x0, content_disposition = 0x0, body_parser = 0x0, word_buf = 0x0, pending_input = 0x0, cur_user_lang = 0x0} input = 0x1701990 parser = 0x1701b20 decoder = 0x1701ef0 raw_block = {part = 0x1696860, hdr = 0x1702050, data = 0x0, size = 0} block = {part = 0x1696860, hdr = 0x1701f08, data = 0x7ffd5e021cd8 "", size = 0} prev_part = 0x1696860 parts = 0x7f4835cc94c0 skip_body = false body_part = false body_added = false binary_body = 192 ret = 1 __FUNCTION__ = "fts_build_mail_real" #3 0x00007f4834fc2dcf in fts_build_mail (update_ctx=0x16f5130, mail=0x17010f0) at fts-build-mail.c:583 _data_stack_cur_id = 6 ret = 0 #4 0x00007f4834fca343 in fts_mail_index (_mail=0x17010f0) at fts-storage.c:484 ft = 0x16fd570 flist = 0x16cc690 #5 0x00007f4834fca440 in fts_mail_precache (_mail=0x17010f0) at fts-storage.c:501 _data_stack_cur_id = 5 mail = 0x17010f0 fmail = 0x1701580 ft = 0x16fd570 #6 0x00007f4835cf633a in mail_precache (mail=0x17010f0) at mail.c:401 _data_stack_cur_id = 4 p = 0x17010f0 #7 0x00000000004024b5 in index_mailbox_precache (conn=0x16a3040, box=0x16d0580) at master-connection.c:93 storage = 0x16cba20 username = 0x16c79e8 "reuben" box_vname = 0x16d0888 "Sent" status = {messages = 453, recent = 0, unseen = 0, uidvalidity = 1267935879, uidnext = 8558, first_unseen_seq = 0, first_recent_uid = 8557, last_cached_seq = 448, highest_modseq = 0, highest_pvt_modseq = 0, keywords = 0x0, permanent_flags = 0, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 1, have_save_guids = 1, have_only_guid128 = 0} trans = 0x16e4b80 search_args = 0x0 ctx = 0x16d0370 mail = 0x17010f0 metadata = {guid = '\000' , virtual_size = 0, cache_fields = 0x0, precache_fields = (MAIL_FETCH_STREAM_HEADER | MAIL_FETCH_STREAM_BODY | MAIL_FETCH_RECEIVED_DATE | MAIL_FETCH_SAVE_DATE | MAIL_FETCH_PHYSICAL_SIZE | MAIL_FETCH_VIRTUAL_SIZE | MAIL_FETCH_UIDL_BACKEND | MAIL_FETCH_GUID), backend_ns_prefix = 0x0, backend_ns_type = (unknown: 0)} seq = 449 percentage_str = "\000\000\000" counter = 0 max = 5 percentage = 0 percentage_sent = 0 ret = 0 #8 0x000000000040295d in index_mailbox (conn=0x16a3040, user=0x16c7900, mailbox=0x1694067 "Sent", max_recent_msgs=0, what=0x169406e "i") at master-connection.c:186 ns = 0x16cb8d0 box = 0x16d0580 status = {messages = 23709136, recent = 0, unseen = 23709136, uidvalidity = 0, uidnext = 4294966784, first_unseen_seq = 4294967295, first_recent_uid = 23683432, last_cached_seq = 0, highest_modseq = 140726180650944, highest_pvt_modseq = 4203059, keywords = 0x16a1bc0, permanent_flags = 0, permanent_keywords = 0, allow_new_keywords = 0, nonpermanent_modseqs = 0, no_modseq_tracking = 0, have_guids = 0, have_save_guids = 0, have_only_guid128 = 0} path = 0x1696180 "/home/reuben/Maildir/.Sent" errstr = 0x7f48359cafc1 "H\211E\370H\213E\370H\213" error = MAIL_ERROR_NONE sync_flags = MAILBOX_SYNC_FLAG_FULL_READ ret = 0 #9 0x0000000000402b1e in master_connection_input_line (conn=0x16a3040, line=0x16a3f42 "reuben\tSent\t0\ti") at master-connection.c:222 args = 0x1694070 input = {module = 0x403134 "mail", service = 0x403139 "indexer-worker", username = 0x1694060 "reuben", session_id = 0x0, local_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, remote_ip = {family = 0, u = {ip6 = {__in6_u = { __u6_addr8 = '\000' , __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, ip4 = { s_addr = 0}}}, local_port = 0, remote_port = 0, userdb_fields = 0x0, flags_override_add = (unknown: 0), flags_override_remove = (unknown: 0), no_userdb_lookup = 0} service_user = 0x16a6000 user = 0x16c7900 str = 0x16a3f3e "1\t0" error = 0x7f4835ff3bb0 "I\211\303f\017\032\\$0f\017\032T$ f\017\032L$\020f\017\032\004$L\213L$pL\213D$hH\213|$`H\213t$XH\213T$PH\213L$HH\213D$@H\201\304\210" max_recent_msgs = 0 ret = 0 #10 0x0000000000402c94 in master_connection_input (conn=0x16a3040) at master-connection.c:259 _data_stack_cur_id = 3 line = 0x16a3f42 "reuben\tSent\t0\ti" ret = 32584 #11 0x00007f4835a53548 in io_loop_call_io (io=0x16a3080) at ioloop.c:501 ioloop = 0x169c740 t_id = 2 __FUNCTION__ = "io_loop_call_io" #12 0x00007f4835a557f6 in io_loop_handler_run_internal (ioloop=0x169c740) at ioloop-epoll.c:220 ctx = 0x16a15f0 events = 0x16a2430 event = 0x16a2430 list = 0x16a30e0 io = 0x16a3080 tv = {tv_sec = 2147483, tv_usec = 0} events_count = 4 msecs = -1 ret = 1 i = 0 j = 0 call = true __FUNCTION__ = "io_loop_handler_run_internal" #13 0x00007f4835a536e5 in io_loop_handler_run (ioloop=0x169c740) at ioloop.c:548 No locals. #14 0x00007f4835a5363d in io_loop_run (ioloop=0x169c740) at ioloop.c:525 __FUNCTION__ = "io_loop_run" #15 0x00007f48359c8215 in master_service_run (service=0x169c5d0, callback=0x401fb6 ) at master-service.c:569 No locals. #16 0x0000000000402182 in main (argc=1, argv=0x169c390) at indexer-worker.c:78 service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN storage_service_flags = (MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT | MAIL_STORAGE_SERVICE_FLAG_USERDB_LOOKUP | MAIL_STORAGE_SERVICE_FLAG_TEMP_PRIV_DROP | MAIL_STORAGE_SERVICE_FLAG_NO_IDLE_TIMEOUT) c = -1 (gdb) Thanks, Reuben From tss at iki.fi Sun May 10 09:06:32 2015 From: tss at iki.fi (Timo Sirainen) Date: Sun, 10 May 2015 12:06:32 +0300 Subject: v2.2.17 release candidate released In-Reply-To: <554F1B13.1040905@reub.net> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <554ECE57.5040901@reub.net> <8062E904-C42A-492D-B2C7-035DCCAEAD1D@iki.fi> <554F1B13.1040905@reub.net> Message-ID: <35049DD1-FD96-4679-AAF7-3E3933B28B56@iki.fi> On 10 May 2015, at 11:47, Reuben Farrelly wrote: > Ok we have some (well, lots actually) of coredumps now: > > Program terminated with signal SIGSEGV, Segmentation fault. > #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) > at fts-user.c:302 Thanks, this should fix it: http://hg.dovecot.org/dovecot-2.2/rev/fddd3dbdf987 From trashcan at odo.in-berlin.de Sun May 10 09:40:48 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 11:40:48 +0200 Subject: v2.2.17 release candidate released In-Reply-To: <9F184DCC-29EC-43F5-9088-C1AE8D9F14DD@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> <9F184DCC-29EC-43F5-9088-C1AE8D9F14DD@iki.fi> Message-ID: <984FAAB9-1A26-4B14-9C07-829FBC48BA03@odo.in-berlin.de> Timo Sirainen wrote: > Michael Grimm kirjoitti 9.5.2015 kello 23.17: >> Timo Sirainen wrote: >>> Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. >> 2.2.17.rc1 will compile on FBSD 10.1 only, if bash has been installed > Dovecot hg requires bash currently, but the rc1 tarball should compile even without bash because the generated files are included in the tarball. Or should at least.. Yes, I know, that's what Teemu already pointed out. And yes, I can confirm, that http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz compiles without an installed bash under FBSD 10.1-STABLE I can live with having to install bash in order to be able to follow hg development (what I do in a test jail), but it's from my point of view a little bit of "overkill", if one needs bash just for creating two c-files from two bash-scripts and their corresponding txt-files once in a while. Again, please don't get me wrong, I am not complaining! I can live with the situation as is. I only wanted to report that issue because you asked for feedback from non-linux users ;-) Regards, Michael From trashcan at odo.in-berlin.de Sun May 10 09:41:37 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 11:41:37 +0200 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: <2109150.CC7IZC85Na@flake.tharned.org> References: <554773D2.9060609@dovecot.fi> <2109150.CC7IZC85Na@flake.tharned.org> Message-ID: <4BCC9FFC-F0F0-40B1-8D4A-7313F7555CC1@odo.in-berlin.de> Greg Rivers wrote: > On Saturday, May 09, 2015 22:25:48 Michael Grimm wrote: >>> or just try if it works if you change it to /bin/sh and use whatever >>> FreeBSD has that pointing to. >> That fails because /bin/sh equals /bin/csh at FBSD. > I don't know if it fails or not, but if it does this is not the reason. > /bin/sh most certainly is not /bin/csh; Ups. Yes, that is correct. Sorry, my fault. > if it were, the system would not boot > given that all the rc start-up scripts are written in Bourne shell. But here I will insist on /bin/sh != /bin/bash ;-) The FBSD OS is bourne free, if I am not mistaken again. > OTOH, /bin/csh and /bin/tcsh are identical: > > 108 -r-xr-xr-x 2 root wheel 382368 Nov 11 15:03 /bin/csh* > 118 -r-xr-xr-x 1 root wheel 142184 Nov 11 15:03 /bin/sh* > 108 -r-xr-xr-x 2 root wheel 382368 Nov 11 15:03 /bin/tcsh* ls -al /bin/sh /usr/local/bin/bash (FBSD 10.1-STABLE): | -r-xr-xr-x 1 root wheel uarch 142144 May 8 13:57 /bin/sh | -rwxr-xr-x 1 root wheel uarch 895712 May 8 13:09 /usr/local/bin/bash JFTR: Both scripts fail to run with FBSD's /bin/sh (lot of syntax errors), but run to completion when modifying the first line to "#!/usr/local/bin/bash" (needed, because ports are installed into /usr/local). Regards, Michael From reuben-dovecot at reub.net Sun May 10 09:45:15 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Sun, 10 May 2015 19:45:15 +1000 Subject: v2.2.17 release candidate released In-Reply-To: <35049DD1-FD96-4679-AAF7-3E3933B28B56@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <554ECE57.5040901@reub.net> <8062E904-C42A-492D-B2C7-035DCCAEAD1D@iki.fi> <554F1B13.1040905@reub.net> <35049DD1-FD96-4679-AAF7-3E3933B28B56@iki.fi> Message-ID: <554F28AB.1060503@reub.net> On 10/05/2015 7:06 PM, Timo Sirainen wrote: > On 10 May 2015, at 11:47, Reuben Farrelly wrote: > >> Ok we have some (well, lots actually) of coredumps now: >> >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x00007f4834fcc222 in fts_user_get_data_lang (user=0x16c7900) >> at fts-user.c:302 > > Thanks, this should fix it: http://hg.dovecot.org/dovecot-2.2/rev/fddd3dbdf987 Confirmed as fixed - all good now. Thanks! Reuben From trashcan at odo.in-berlin.de Sun May 10 11:10:04 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 13:10:04 +0200 Subject: FYI: dovecot (008632bdfd2c) compilation woes, and minor glitch regarding update-version.sh In-Reply-To: <4BCC9FFC-F0F0-40B1-8D4A-7313F7555CC1@odo.in-berlin.de> References: <554773D2.9060609@dovecot.fi> <2109150.CC7IZC85Na@flake.tharned.org> <4BCC9FFC-F0F0-40B1-8D4A-7313F7555CC1@odo.in-berlin.de> Message-ID: <62AB57C8-E838-45C4-842D-F48C34312BB1@odo.in-berlin.de> Michael Grimm wrote: > But here I will insist on /bin/sh != /bin/bash ;-) The FBSD OS is bourne free, if I am not mistaken again. Arrgh! That's wrong again, I always mixup bourne shell with bash shell :-( Thus, FBSD is bash-free, not bourne-free. Sorry, Michael From stephan at rename-it.nl Sun May 10 15:34:31 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 10 May 2015 17:34:31 +0200 Subject: Xi broken In-Reply-To: <55426BCE.1090907@rename-it.nl> References: <55426BCE.1090907@rename-it.nl> Message-ID: <554F7A87.80004@rename-it.nl> On 4/30/2015 7:52 PM, Stephan Bosch wrote: > Hi, > > Xi is broken at the moment. This XenServer version won't boot jessie > kernel. > > Can't fix this myself, so this may take some time. Since we could not upgrade XenServer any time soon, we reverted the build systems back to Debian wheezy (oldstable) by re-installing them from scratch. It should all work as normal, with the difference that wheezy is built first. Added Debian stretch (current testing) repository, while the Debian squeeze (current oldoldstable) repository was removed. I also removed repositories for dovecot-2.1 and older. The repositories can now also be used with the release names (e.g. wheezy-auto rather than oldstable-auto). Don't hesitate to notify me if there are any problems. Regards, Stephan. From tlx at leuxner.net Sun May 10 16:01:25 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Sun, 10 May 2015 18:01:25 +0200 Subject: Xi broken In-Reply-To: <554F7A87.80004@rename-it.nl> References: <55426BCE.1090907@rename-it.nl> <554F7A87.80004@rename-it.nl> Message-ID: <20150510160125.GA21919@nihlus.leuxner.net> * Stephan Bosch 2015.05.10 17:34: > Don't hesitate to notify me if there are any problems. Thanks for fixing this. As this repo stuff is mostly bleeding edge, I noticed that the systemd service file needs to be modified to allow coredumps. Not sure how it is supposed to interact with the file /etc/default/dovecot and its ALLOW_COREDUMPS=1 variable, but one needs to explicitly add LimitCORE=infinity in the 'Service' section (until someone with more systemd foo fixes it): $ cat /lib/systemd/system/dovecot.service [Unit] Description=Dovecot IMAP/POP3 email server After=local-fs.target network.target [Service] Type=simple LimitCORE=infinity ExecStart=/usr/sbin/dovecot -F NonBlocking=yes [Install] WantedBy=multi-user.target -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From user+dovecot at localhost.localdomain.org Sun May 10 17:29:49 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Sun, 10 May 2015 17:29:49 +0000 Subject: Xi broken In-Reply-To: <20150510160125.GA21919@nihlus.leuxner.net> References: <55426BCE.1090907@rename-it.nl> <554F7A87.80004@rename-it.nl> <20150510160125.GA21919@nihlus.leuxner.net> Message-ID: <554F958D.1090901@localhost.localdomain.org> On 05/10/2015 04:01 PM, Thomas Leuxner wrote: > ... Not sure how it is supposed to interact with the file /etc/default/dovecot and its ALLOW_COREDUMPS=1 variable, but one needs to explicitly add LimitCORE=infinity in the 'Service' section (until someone with more systemd foo fixes it): > The attached patch could solve the problem. Regards, Pascal -- The trapper recommends today: cafebabe.1513019 at localdomain.org -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-core.patch Type: text/x-diff Size: 1030 bytes Desc: not available URL: From trashcan at odo.in-berlin.de Sun May 10 19:15:26 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 21:15:26 +0200 Subject: v2.2.17 release candidate released In-Reply-To: <79770603-32B7-42F9-A0C8-8774AECD88F3@distal.com> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> <20150509191346.4d1567c6@seibercom.net> <79770603-32B7-42F9-A0C8-8774AECD88F3@distal.com> Message-ID: Chris Ross wrote: > I'd much rather be able to build dovecot without bash, because I don't > have bash installed on my FreeBSD 10.1 mail server, where I run dovecot. > > I'd also be willing to help by porting any of the scripts that require > bash to something else that is available in the base installation of > FreeBSD as well as the variety of Linux distributions. I started to learn awk today ;-) See an awk script attached that simulates "src/lib-fts/word-break-data.sh < src/lib-fts/PropList.txt > src/lib-fts/word-break-data.c". This script is pretty preliminary, although it does what it is supposed to do, if I am not mistaken. Please, give it a try. It lacks a proper way of referencing stolen code in function mystrtonum (never wrote open source code); I did steal that code in order to stick to awk instead of gawk, which might be a portability issue as well. This script lacks any error handling, as well. Thus, consider it pre-alpha. I do not have the time to continue for the coming days. Thus, anyone feeling to jump in: just do it ;-) Regards, Michael -------------- next part -------------- A non-text attachment was scrubbed... Name: word-break-data.awk Type: application/octet-stream Size: 1848 bytes Desc: not available URL: From stephan at rename-it.nl Sun May 10 19:24:57 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sun, 10 May 2015 21:24:57 +0200 Subject: Xi broken In-Reply-To: <554F958D.1090901@localhost.localdomain.org> References: <55426BCE.1090907@rename-it.nl> <554F7A87.80004@rename-it.nl> <20150510160125.GA21919@nihlus.leuxner.net> <554F958D.1090901@localhost.localdomain.org> Message-ID: <554FB089.2060603@rename-it.nl> On 5/10/2015 7:29 PM, Pascal Volk wrote: > On 05/10/2015 04:01 PM, Thomas Leuxner wrote: >> ... Not sure how it is supposed to interact with the file /etc/default/dovecot and its ALLOW_COREDUMPS=1 variable, but one needs to explicitly add LimitCORE=infinity in the 'Service' section (until someone with more systemd foo fixes it): >> > The attached patch could solve the problem. I base this directly on the official Debian unstable packages. So, it would be best if it is fixed there first by someone who knows how to do this properly. I'm mainly worried a bit about the fact that dovecot.service is generated by Dovecot install and /etc/default/dovecot is from Debian. Regards, Stephan. From trashcan at odo.in-berlin.de Sun May 10 19:54:08 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 21:54:08 +0200 Subject: New York: Matilda, seit Jahren mit dem immer gleichen Outfit - DIE WELT Message-ID: <1A7BEEF6-0C6D-4E8F-B1A3-09E324DA07CC@odo.in-berlin.de> Cool, und vor allem gut! > http://www.welt.de/vermischtes/article140736122/Matilda-seit-Jahren-mit-dem-immer-gleichen-Outfit.html From trashcan at odo.in-berlin.de Sun May 10 19:55:12 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Sun, 10 May 2015 21:55:12 +0200 Subject: New York: Matilda, seit Jahren mit dem immer gleichen Outfit - DIE WELT In-Reply-To: <1A7BEEF6-0C6D-4E8F-B1A3-09E324DA07CC@odo.in-berlin.de> References: <1A7BEEF6-0C6D-4E8F-B1A3-09E324DA07CC@odo.in-berlin.de> Message-ID: Sorry for the noise, damn Mail.app :-(( > On 10.05.2015, at 21:54, Michael Grimm wrote: > > Cool, und vor allem gut! >> http://www.welt.de/vermischtes/article140736122/Matilda-seit-Jahren-mit-dem-immer-gleichen-Outfit.html From jerry at seibercom.net Sun May 10 20:02:42 2015 From: jerry at seibercom.net (Jerry) Date: Sun, 10 May 2015 16:02:42 -0400 Subject: v2.2.17 release candidate released In-Reply-To: <9F184DCC-29EC-43F5-9088-C1AE8D9F14DD@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> <7EADAB26-D73C-4662-8356-AAFBCC780302@odo.in-berlin.de> <9F184DCC-29EC-43F5-9088-C1AE8D9F14DD@iki.fi> Message-ID: <20150510160242.2e7096be@seibercom.net> On Sun, 10 May 2015 10:27:25 +0300, Timo Sirainen stated: >Dovecot hg requires bash currently, but the rc1 tarball should compile even >without bash because the generated files are included in the tarball. Or >should at least.. Timo, IMHO, I would not worry too much about placating the FreeBSD crowd. They love to pretend that their way of doing things is superior to everyone else. I say this even though I use FreeBSD myself. The FreeBSD port maintain for Dovecot can make bash a dependency or he can attempt to rewrite the Dovecot scripts himself. You work too hard improving Dovecot and should not have to tweak it for every OS out there. Believe me, I know from experience that it will come back to haunt you if you try to please everyone. Again, thanks for all of your hard work on this software. -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From pixilla at macports.org Sun May 10 21:02:02 2015 From: pixilla at macports.org (Bradley Giesbrecht) Date: Sun, 10 May 2015 14:02:02 -0700 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: On May 9, 2015, at 10:41 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig > > Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. Between dovecot-2.2.17rc1 and 69630e6048fd a build issue on OS X/MacPorts appears to be gone. Regards, Bradley Giesbrecht (pixilla) -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 455 bytes Desc: Message signed with OpenPGP using GPGMail URL: From stephan at rename-it.nl Sun May 10 22:42:21 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 11 May 2015 00:42:21 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 Message-ID: <554FDECD.5080406@rename-it.nl> Hello Dovecot users, Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a few small fixes and one new feature. Changelog v0.4.8.rc1: * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to include %{delivery_time}. This prompted changes in Pigeonhole that make this release dependent on Dovecot v2.2.17. + Implemented magic to make sieve_default script visible from ManageSieve under a configurable name. This way, users can see the default rules, edit them and store a private adjusted version. This could also be achieved by copying the default script into the user's script storage, but that way updates to the global sieve_default script would be ignored. - Fixed problem in address test: erroneously decoded mime-encoded words in address headers. - extprograms plugin: Fixed failure occurring when connecting to script service without the need to read back the output from the external program. - Fixed bug in script storage path normalization occurring with relative symbolic links below root. - Fixed and updated various parts of the documentation. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From skdovecot at smail.inf.fh-brs.de Mon May 11 06:29:13 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Mon, 11 May 2015 08:29:13 +0200 (CEST) Subject: Cant use doveadm to set ACL . [request for help] In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Fri, 8 May 2015, Kevin Laurie wrote: > grep 'socket_path' /etc/dovecot/dovecot.conf first: use doveconf -n | grep 'socket_path' or in this case better: doveconf -a | grep 'socket_path' > auth_socket_path = /var/run/dovecot/auth-master > auth_socket_path = /var/run/dovecot/auth-master > > > [root at mail root]# doveadm acl get -u bob at mydomain.net -S > /var/run/dovecot/auth-master -m Inbox Why you use -S ?? doveadm should detect the correct socket itself. > doveadm(bob at mydomain.net): Error: doveadm server sent invalid > handshake: VERSION 1 1 > doveadm(bob at mydomain.net): Error: /var/run/dovecot/auth-master: > Internal failure for bob at mydomain.net > ID Global Rights > - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVVBMOXz1H7kL/d9rAQJgLAf+K3envFbMg06DtDT2BOZsUqeU9l8ibuFK I13T/MJMqWqbkHYitDRXVkYz6cOMW14QPGSKMrmRIYiKb8IipJIi7Za17u6lSsmt PozDmPLqome6RoofxJbcB9ZM2mzd8xP3DnOILuPyIkPN6K7fN2yGarTZ3UTzLjni SrP3R7AIngYa2WbM1jhMiX6uTrX6EGuGAuOrwJJNtMqHobtf4HGIvPB1P5GSOM6X ubLKbUA9T2O+aPDnAXL6kUHOAh9lxZ7rmQFS8rN2UVUNr3vFyODBBy/s4RpOvSID ebo5QS0LAmNzEyXC1fNqdHWcGZL9FaO5Bmc/pxb6GcT6rc1V2lqhAQ== =5a1D -----END PGP SIGNATURE----- From racuk12 at gmail.com Mon May 11 06:53:52 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Mon, 11 May 2015 07:53:52 +0100 Subject: What is this message in appfirewall.log telling me ? on OSX Message-ID: <81F25D07-A323-4516-A5C0-D4A408C7634C@gmail.com> I'm on OS X What is this telling me? socketfilterfw[22097] : dovecot: Deny TCP CONNECT (in:7 out:0) Thanks Robert From tolga at ozses.net Mon May 11 06:55:53 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Mon, 11 May 2015 09:55:53 +0300 Subject: What is this message in appfirewall.log telling me ? on OSX In-Reply-To: <81F25D07-A323-4516-A5C0-D4A408C7634C@gmail.com> References: <81F25D07-A323-4516-A5C0-D4A408C7634C@gmail.com> Message-ID: I'm not 100% sure, but it looks like a client was denied connection. On 11 May 2015 at 09:53, Robert Chalmers wrote: > I'm on OS X > > What is this telling me? > > socketfilterfw[22097] : dovecot: Deny TCP CONNECT (in:7 out:0) > > > Thanks > Robert > From dominik at dominikbreu.de Mon May 11 07:26:41 2015 From: dominik at dominikbreu.de (Dominik Breu) Date: Mon, 11 May 2015 09:26:41 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <554FDECD.5080406@rename-it.nl> References: <554FDECD.5080406@rename-it.nl> Message-ID: <1431329201.25926.2.camel@dominikbreu.de> Hello, did an test Install on my system get some nice SEGFAULT wich backtrace here: https://dominikbreu.de/Stikked/view/8b6b7e5b version is following: aptitude versions dovecot-sieve i 2:2.2.17~rc1-1~auto+4 best regards dominik Am Montag, den 11.05.2015, 00:42 +0200 schrieb Stephan Bosch: > Hello Dovecot users, > > Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a > few small fixes and one new feature. > > Changelog v0.4.8.rc1: > > * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to > include %{delivery_time}. This prompted changes in Pigeonhole that > make this release dependent on Dovecot v2.2.17. > + Implemented magic to make sieve_default script visible from > ManageSieve under a configurable name. This way, users can see the > default rules, edit them and store a private adjusted version. This > could also be achieved by copying the default script into the user's > script storage, but that way updates to the global sieve_default > script would be ignored. > - Fixed problem in address test: erroneously decoded mime-encoded words > in address headers. > - extprograms plugin: Fixed failure occurring when connecting to script > service without the need to read back the output from the external > program. > - Fixed bug in script storage path normalization occurring with relative > symbolic links below root. > - Fixed and updated various parts of the documentation. > > The release is available as follows: > > http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz > http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz.sig > > Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for > more information. Have fun testing this new release and don't hesitate > to notify me when there are any problems. > > Regards, > From racuk12 at gmail.com Mon May 11 07:28:02 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Mon, 11 May 2015 08:28:02 +0100 Subject: What is this message in appfirewall.log telling me ? on OSX In-Reply-To: References: <81F25D07-A323-4516-A5C0-D4A408C7634C@gmail.com> Message-ID: On 11 May 2015, at 7:55, Muzaffer Tolga Ozses wrote: > I'm not 100% sure, but it looks like a client was denied connection. > > On 11 May 2015 at 09:53, Robert Chalmers wrote: > >> I'm on OS X >> >> What is this telling me? >> >> socketfilterfw[22097] : dovecot: Deny TCP CONNECT (in:7 out:0) >> >> >> Thanks >> Robert >> Is there something I can set to make that message more verbose? Author and Writer From tolga at ozses.net Mon May 11 07:34:04 2015 From: tolga at ozses.net (Muzaffer Tolga Ozses) Date: Mon, 11 May 2015 10:34:04 +0300 Subject: What is this message in appfirewall.log telling me ? on OSX In-Reply-To: References: <81F25D07-A323-4516-A5C0-D4A408C7634C@gmail.com> Message-ID: There are multiple lines in dovecot.conf that contain *debug* to... well... debug the issue. Turn them on. On 11 May 2015 at 10:28, Robert Chalmers wrote: > On 11 May 2015, at 7:55, Muzaffer Tolga Ozses wrote: > > > I'm not 100% sure, but it looks like a client was denied connection. > > > > On 11 May 2015 at 09:53, Robert Chalmers wrote: > > > >> I'm on OS X > >> > >> What is this telling me? > >> > >> socketfilterfw[22097] : dovecot: Deny TCP CONNECT (in:7 out:0) > >> > >> > >> Thanks > >> Robert > >> > > > Is there something I can set to make that message more verbose? > > Author and Writer > From stephan at rename-it.nl Mon May 11 07:34:54 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 11 May 2015 09:34:54 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <1431329201.25926.2.camel@dominikbreu.de> References: <554FDECD.5080406@rename-it.nl> <1431329201.25926.2.camel@dominikbreu.de> Message-ID: <55505B9E.90007@rename-it.nl> On 5/11/2015 9:26 AM, Dominik Breu wrote: > Hello, > > did an test Install on my system get some nice SEGFAULT > wich backtrace here: https://dominikbreu.de/Stikked/view/8b6b7e5b > version is following: > aptitude versions dovecot-sieve i 2:2.2.17~rc1-1~auto+4 I cannot view that URL (some broken SSL error). You can just put the backtrace in the email itself. Regards, Stephan. > best regards > dominik > > Am Montag, den 11.05.2015, 00:42 +0200 schrieb Stephan Bosch: >> Hello Dovecot users, >> >> Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a >> few small fixes and one new feature. >> >> Changelog v0.4.8.rc1: >> >> * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to >> include %{delivery_time}. This prompted changes in Pigeonhole that >> make this release dependent on Dovecot v2.2.17. >> + Implemented magic to make sieve_default script visible from >> ManageSieve under a configurable name. This way, users can see the >> default rules, edit them and store a private adjusted version. This >> could also be achieved by copying the default script into the user's >> script storage, but that way updates to the global sieve_default >> script would be ignored. >> - Fixed problem in address test: erroneously decoded mime-encoded words >> in address headers. >> - extprograms plugin: Fixed failure occurring when connecting to script >> service without the need to read back the output from the external >> program. >> - Fixed bug in script storage path normalization occurring with relative >> symbolic links below root. >> - Fixed and updated various parts of the documentation. >> >> The release is available as follows: >> >> http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz >> http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc1.tar.gz.sig >> >> Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for >> more information. Have fun testing this new release and don't hesitate >> to notify me when there are any problems. >> >> Regards, >> > From stephan at rename-it.nl Mon May 11 08:05:36 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 11 May 2015 10:05:36 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <55505B9E.90007@rename-it.nl> References: <554FDECD.5080406@rename-it.nl> <1431329201.25926.2.camel@dominikbreu.de> <55505B9E.90007@rename-it.nl> Message-ID: <555062D0.9010108@rename-it.nl> On 5/11/2015 9:34 AM, Stephan Bosch wrote: > On 5/11/2015 9:26 AM, Dominik Breu wrote: >> Hello, >> >> did an test Install on my system get some nice SEGFAULT >> wich backtrace here: https://dominikbreu.de/Stikked/view/8b6b7e5b >> version is following: >> aptitude versions dovecot-sieve i 2:2.2.17~rc1-1~auto+4 > I cannot view that URL (some broken SSL error). You can just put the > backtrace in the email itself. If only Xi weren't broken all week, you'd probably have noticed sooner. Fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/c00f51f3c91a Xi will release new packages in about an hour. I will create a new RC later today. Regards, Stephan. From racuk12 at gmail.com Mon May 11 08:34:42 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Mon, 11 May 2015 09:34:42 +0100 Subject: Dovecot is not reading connections from iOS devices correctly. Any ideas? Message-ID: From the dovecot logs. dovecot-info.log May 11 09:27:45 imap-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=192.168.0.1, lip=192.168.0.15, session= dovecot-debug.log May 11 09:27:45 auth: Debug: auth client connected (pid=17388) When I try to connect using either the iOS Mail app, or Outlook mail app, nothing happens at al with ssl, and if I use plain login the iOS device tells me I have either the username or password wrong - which I haven't. Connection from Mac with MailMate works fine. iOS doesn't. so I'm stumped. I can't see why this is refusing the connection. From sanvila at unex.es Mon May 11 12:00:39 2015 From: sanvila at unex.es (Santiago Vila) Date: Mon, 11 May 2015 14:00:39 +0200 Subject: Bug#776094: dovecot-imapd: corrupts mailbox after trying to retrieve it (fwd) In-Reply-To: <554B82BF.4090105@sys4.de> References: <5547C74D.4020806@sys4.de> <20150505142600.GA4982@cantor.unex.es> <5549BB5C.8050502@dovecot.fi> <5549BBCE.90307@dovecot.fi> <20150506092952.GA28906@cantor.unex.es> <20150506094322.GA29104@cantor.unex.es> <6F1B94E8-A1E6-4BB6-8243-6E0F1286AEF5@iki.fi> <20150506234529.GA1187@cantor.unex.es> <554B82BF.4090105@sys4.de> Message-ID: <20150511120039.GA3158@cantor.unex.es> On Thu, May 07, 2015 at 05:20:31PM +0200, Robert Schetterer wrote: > http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages > > ... > deb http://xi.rename-it.nl/debian/ testing-auto/dovecot-2.2 main Thanks a lot. I installed dovecot version 2:2.2.17~rc1-1~auto+5 using that line. Changelog says: * New revision (2054:c00f51f3c91a) in pigeonhole Mercurial repository: - lib-sieve: storage: Fixed segfault problem in main storage initialization. and later: * New revision (18627:69630e6048fd) in dovecot Mercurial repository: I can still reproduce the error, which means this was not fixed by changeset 18534:94bd895721d8 after all. I wonder how you guys are testing for this bug. Please use something like this script to reproduce: #!/bin/sh rm -rf $HOME/mail/.imap cp -f $HOME/etc/inbox-b $HOME/mail fetchmail -a qemu-jessie --folder inbox-b -m "true" where $HOME/etc/inbox-b is the original test case I provided. Thanks. From sanvila at unex.es Mon May 11 12:04:34 2015 From: sanvila at unex.es (Santiago Vila) Date: Mon, 11 May 2015 14:04:34 +0200 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: <20150511120434.GB3158@cantor.unex.es> On Sat, May 09, 2015 at 08:41:23PM +0300, Timo Sirainen wrote: > - mbox: Fixed crash/corruption in some situations when the first mail > was expunged. Not really fixed. Please see my previous email. From brad at comstyle.com Mon May 11 16:24:43 2015 From: brad at comstyle.com (Brad Smith) Date: Mon, 11 May 2015 12:24:43 -0400 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <554FDECD.5080406@rename-it.nl> References: <554FDECD.5080406@rename-it.nl> Message-ID: <5550D7CB.70909@comstyle.com> On 05/10/15 18:42, Stephan Bosch wrote: > Hello Dovecot users, > > Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a > few small fixes and one new feature. > > Changelog v0.4.8.rc1: > > * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to > include %{delivery_time}. This prompted changes in Pigeonhole that > make this release dependent on Dovecot v2.2.17. > + Implemented magic to make sieve_default script visible from > ManageSieve under a configurable name. This way, users can see the > default rules, edit them and store a private adjusted version. This > could also be achieved by copying the default script into the user's > script storage, but that way updates to the global sieve_default > script would be ignored. > - Fixed problem in address test: erroneously decoded mime-encoded words > in address headers. > - extprograms plugin: Fixed failure occurring when connecting to script > service without the need to read back the output from the external > program. > - Fixed bug in script storage path normalization occurring with relative > symbolic links below root. > - Fixed and updated various parts of the documentation. Since switching to 0.4.8.rc1 all Sieve filtering has stopped working for me. No errors in the logs. Switch back to 0.4.7 and everything is working fine. The Wiki is useless for help.. "This section should contain a step-wise approach to troubleshooting.". So any hints as to how to determine what is going on behind the scenes with Pigeonhole? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From tlx at leuxner.net Mon May 11 16:55:54 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 11 May 2015 18:55:54 +0200 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: <20150511165554.GA19811@nihlus.leuxner.net> * Timo Sirainen 2015.05.09 19:41: > + lda, lmtp: deliver_log_format can now include %{delivery_time}, > which expands to how many milliseconds it took to deliver the mail. > With LMTP %{session_time} also expands to how many milliseconds the > LMTP session took, not including the delivery time. This could use some nicer defaults similar to Postfix logging e.g. deliver_log_format = msgid=%m, time=%{delivery_time}ms, status=%$ May 11 18:25:29 nihlus dovecot: lmtp(tlx at leuxner.net): CupIHNrXUFXUTAAAgUOSbA: sieve: msgid=<5550D7CB.70909 at comstyle.com>, time=133ms, status=stored mail into mailbox 'Public/Mailing-Lists/Dovecot' -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From stephan at rename-it.nl Mon May 11 19:07:30 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 11 May 2015 21:07:30 +0200 Subject: Released Pigeonhole v0.4.8.rc2 for Dovecot v2.2.17.rc1 Message-ID: <5550FDF2.8060105@rename-it.nl> Hello Dovecot users, Ok, RC1 contained one rather stupid bug that caused a segfault at delivery. It was caused by a new feature, so the list of changes is not updated. Changelog v0.4.8: * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to include %{delivery_time}. This prompted changes in Pigeonhole that make this release dependent on Dovecot v2.2.17. + Implemented magic to make sieve_default script visible from ManageSieve under a configurable name. This way, users can see the default rules, edit them and store a private adjusted version. This could also be achieved by copying the default script into the user's script storage, but that way updates to the global sieve_default script would be ignored. - Fixed problem in address test: erroneously decoded mime-encoded words in address headers. - extprograms plugin: Fixed failure occurring when connecting to script service without the need to read back the output from the external program. - Fixed bug in script storage path normalization occurring with relative symbolic links below root. - Fixed and updated various parts of the documentation. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc2.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc2.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From stephan at rename-it.nl Mon May 11 19:09:46 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 11 May 2015 21:09:46 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <5550D7CB.70909@comstyle.com> References: <554FDECD.5080406@rename-it.nl> <5550D7CB.70909@comstyle.com> Message-ID: <5550FE7A.80101@rename-it.nl> On 5/11/2015 6:24 PM, Brad Smith wrote: > On 05/10/15 18:42, Stephan Bosch wrote: >> Hello Dovecot users, >> >> Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a >> few small fixes and one new feature. >> >> Changelog v0.4.8.rc1: >> >> * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to >> include %{delivery_time}. This prompted changes in Pigeonhole that >> make this release dependent on Dovecot v2.2.17. >> + Implemented magic to make sieve_default script visible from >> ManageSieve under a configurable name. This way, users can see the >> default rules, edit them and store a private adjusted version. This >> could also be achieved by copying the default script into the user's >> script storage, but that way updates to the global sieve_default >> script would be ignored. >> - Fixed problem in address test: erroneously decoded mime-encoded words >> in address headers. >> - extprograms plugin: Fixed failure occurring when connecting to script >> service without the need to read back the output from the external >> program. >> - Fixed bug in script storage path normalization occurring with relative >> symbolic links below root. >> - Fixed and updated various parts of the documentation. > > Since switching to 0.4.8.rc1 all Sieve filtering has stopped working > for me. No errors in the logs. Switch back to 0.4.7 and everything > is working fine. You should try RC2 first. Your problem may be related, although I don't really see how. > The Wiki is useless for help.. "This section should contain a > step-wise approach to troubleshooting.". So any hints as to how to > determine what > is going on behind the scenes with Pigeonhole? You should enable mail_debug=yes. This will make Sieve spew a lot of information about what it is doing. Regards, Stephan. From jgoerzen at opencsw.org Mon May 11 20:40:54 2015 From: jgoerzen at opencsw.org (Jake Goerzen) Date: Mon, 11 May 2015 13:40:54 -0700 Subject: v2.2.17 release candidate released In-Reply-To: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> References: <21D9C652-89F8-436F-A68E-461C485BC3D6@iki.fi> Message-ID: <555113D6.2030305@opencsw.org> On 5/9/2015 10:41 AM, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz > http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc1.tar.gz.sig > > Quite a lot of changes, so here's a release candidate first. If no problems are reported I'll make v2.2.17 release early next week. Please test especially non-Linux systems as there are some larger configure/Makefile changes done since v2.2.16. > Hi Timo, I tried building on Solaris 10 and get an error: libtool: link: /opt/SUNWspro/bin/cc -G -z defs -h libfs_compress.so -o .libs/libfs_compress.so .libs/fs-compress.o -R/home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-compression/.libs -R/opt/csw/lib/dovecot -L/opt/csw/lib ../../lib-compression/.libs/libdovecot-compression.so -L/home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs -lrt -lnsl -lsocket -lsendfile -lc -m32 -xarch=pentium_pro -m32 -xarch=pentium_pro Undefined first referenced symbol in file iostream_temp_finish .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_stream .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_set_error .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) o_stream_destroy .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_stream_finish .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) fs_write_via_stream .libs/fs-compress.o (symbol belongs to implicit dependency /home/jgoerzen/opencsw/dovecot/trunk/work/solaris10-i386/build-isa-pentium_pro/dovecot-2.2.17.rc1/src/lib-dovecot/.libs/libdovecot.so.0) From jtam.home at gmail.com Mon May 11 21:21:52 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 11 May 2015 14:21:52 -0700 (PDT) Subject: Dovecot is not reading connections from iOS devices correctly. Any ideas? In-Reply-To: References: Message-ID: Robert Chalmers writes: > dovecot-info.log > May 11 09:27:45 imap-login: Info: Aborted login (no auth attempts in 0 > secs): user=<>, rip=192.168.0.1, lip=192.168.0.15, > session= > > When I try to connect using either the iOS Mail app, or Outlook mail > app, nothing happens at al with ssl, and if I use plain login the iOS > device tells me I have either the username or password wrong - which I > haven't. Some mail readers have auto-discover features that are rather stupid. If possible, manually configure the SSL style (TLS/STARTTLS) and port rather than letting it guess. To diagnose whether it is some dovecot config error, you can manually connect with both flavours of SSL openssl s_client -connect 192.168.0.15:993 ... or ... openssl s_client -starttls imap -connect 192.168.0.15:143 then if it gets past the SSL handshake, a login to test the authentication part. This will at least tell you whether it's a client side or server side issue. (However, since it is working with other client, it's probably a client side issue.) Joseph Tam From stephan at rename-it.nl Mon May 11 22:56:22 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 12 May 2015 00:56:22 +0200 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <5550D7CB.70909@comstyle.com> References: <554FDECD.5080406@rename-it.nl> <5550D7CB.70909@comstyle.com> Message-ID: <55513396.1040002@rename-it.nl> On 5/11/2015 6:24 PM, Brad Smith wrote: > On 05/10/15 18:42, Stephan Bosch wrote: >> Hello Dovecot users, >> >> Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a >> few small fixes and one new feature. >> >> Changelog v0.4.8.rc1: >> >> * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to >> include %{delivery_time}. This prompted changes in Pigeonhole that >> make this release dependent on Dovecot v2.2.17. >> + Implemented magic to make sieve_default script visible from >> ManageSieve under a configurable name. This way, users can see the >> default rules, edit them and store a private adjusted version. This >> could also be achieved by copying the default script into the user's >> script storage, but that way updates to the global sieve_default >> script would be ignored. >> - Fixed problem in address test: erroneously decoded mime-encoded words >> in address headers. >> - extprograms plugin: Fixed failure occurring when connecting to script >> service without the need to read back the output from the external >> program. >> - Fixed bug in script storage path normalization occurring with relative >> symbolic links below root. >> - Fixed and updated various parts of the documentation. > > Since switching to 0.4.8.rc1 all Sieve filtering has stopped working > for me. No errors in the logs. Switch back to 0.4.7 and everything > is working fine. I am building a crude test suite to automatically run and check dovecot-lda with Sieve in a few dozen different configurations (normal test suite only tests the interpreter, not the LDA plugin). This problem popped up too, and is now fixed: http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/59d59552247c This only happens when a sieve_default script is supposed to be executed (i.e. if the user has no active personal script) and when sieve_default_name is unset. Regards, Stephan. From brad at comstyle.com Mon May 11 23:29:06 2015 From: brad at comstyle.com (Brad Smith) Date: Mon, 11 May 2015 19:29:06 -0400 Subject: Released Pigeonhole v0.4.8.rc1 for Dovecot v2.2.17.rc1 In-Reply-To: <55513396.1040002@rename-it.nl> References: <554FDECD.5080406@rename-it.nl> <5550D7CB.70909@comstyle.com> <55513396.1040002@rename-it.nl> Message-ID: <55513B42.2010705@comstyle.com> On 05/11/15 18:56, Stephan Bosch wrote: > On 5/11/2015 6:24 PM, Brad Smith wrote: >> On 05/10/15 18:42, Stephan Bosch wrote: >>> Hello Dovecot users, >>> >>> Here's the Pigeonhole RC that goes with the Dovecot RC. It includes a >>> few small fixes and one new feature. >>> >>> Changelog v0.4.8.rc1: >>> >>> * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to >>> include %{delivery_time}. This prompted changes in Pigeonhole that >>> make this release dependent on Dovecot v2.2.17. >>> + Implemented magic to make sieve_default script visible from >>> ManageSieve under a configurable name. This way, users can see the >>> default rules, edit them and store a private adjusted version. This >>> could also be achieved by copying the default script into the user's >>> script storage, but that way updates to the global sieve_default >>> script would be ignored. >>> - Fixed problem in address test: erroneously decoded mime-encoded words >>> in address headers. >>> - extprograms plugin: Fixed failure occurring when connecting to script >>> service without the need to read back the output from the external >>> program. >>> - Fixed bug in script storage path normalization occurring with relative >>> symbolic links below root. >>> - Fixed and updated various parts of the documentation. >> >> Since switching to 0.4.8.rc1 all Sieve filtering has stopped working >> for me. No errors in the logs. Switch back to 0.4.7 and everything >> is working fine. > > I am building a crude test suite to automatically run and check > dovecot-lda with Sieve in a few dozen different configurations (normal > test suite only tests the interpreter, not the LDA plugin). This problem > popped up too, and is now fixed: > > http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/59d59552247c > > This only happens when a sieve_default script is supposed to be executed > (i.e. if the user has no active personal script) and when > sieve_default_name is unset. Thanks. Now it is back to working order. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From onno at flox.org Tue May 12 08:50:06 2015 From: onno at flox.org (Onno Molenkamp) Date: Tue, 12 May 2015 10:50:06 +0200 Subject: dovecot-lda panics when postmaster_address is given on command line Message-ID: <5551BEBE.4000303@flox.org> Hi, When passing a postmaster_address to dovecot-lda on the command line with -o, it panics: $ ./dovecot-lda -o postmaster_address=anything lda(onno,)Panic: file lda-settings.c: line 80 (lda_settings_check): assertion failed: (set->postmaster_address[0] == SETTING_STRVAR_UNEXPANDED[0]) It looks like postmaster_address has already been expanded at that point when it's given on the command line. (set->postmaster_address[0] is "1") I'm running v2.2.15, but with v2.2.17.rc1 the same thing happens. Regards, Onno -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Tue May 12 21:25:44 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 May 2015 00:25:44 +0300 Subject: v2.2.17 release candidate 2 released Message-ID: <92448172-9A8D-4898-AE59-1B2A09434E1B@iki.fi> http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc2.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.17.rc2.tar.gz.sig Changes since rc1: - Compiling fix for fs-compress in Solaris and maybe others. - Implemented Cassandra CQL support as lib-sql backend. This is mainly usable as dict-sql backend. - lib-fts fixes and improvements - Some other small random changes If no new issues are found, v2.2.17 final should be out in a day or two. (Actually looks like I have a few more bugs that probably will be fixed for it.) Changes since v2.2.16: * Dovecot no longer checks or warns if a mountpoint is removed. This was causing more trouble than it was worth. Make sure that all the mountpoints that Dovecot accesses aren't writable by mail processes when they're unmounted. * dict server wasn't properly escaping/unescaping data. Fixing this broke backwards compatibility with data that contains line feeds. This hopefully affects only very few installations. If you're using dict to save multiline data (Sieve scripts to SQL), you may be affected. * imap: SPECIAL-USE capability is no longer advertised if there are no special_use flags specified for any mailboxes. + lmtp: Added lmtp_hdr_delivery_address setting to specify whether to include email address in Delivered-To: and Received: headers. + Added initial version of full text search library, which includes language-specific text normalization and filtering. This is still in development, but it's already possible to use for testing with fts-lucene and fts-solr. + lda, lmtp: deliver_log_format can now include %{delivery_time}, which expands to how many milliseconds it took to deliver the mail. With LMTP %{session_time} also expands to how many milliseconds the LMTP session took, not including the delivery time. + lmtp proxy: Mail delivery logging includes timing information. + imap: Most IMAP commands now include in the tagged reply how many milliseconds it took to run the command (not counting the time spent on waiting for the IMAP client to read/write data). + director: Implemented director_proxy_maybe passdb extra field to be able to run director and backend in the same Dovecot instance. (LMTP doesn't support mixed proxy/non-proxy destinations currently.) + doveadm: Added -F parameter to read a list of users from the given file and run the command for all the users. This is similar to -A parameter reading the list of users from userdb lookup. + Implemented initial Cassandra CQL support as lib-sql backend. It's only usable as dict backend currently. - auth: If auth_master_user_separator was set, auth process could be crashed by trying to log in with empty master username. - imap-login, pop3-login: Fixed crash on handshake failures with new OpenSSL versions (v1.0.2) when SSLv3 was disabled. - auth: If one passdb fails allow_nets check, it shouldn't have failed all the other passdb checks later on. - imap: Server METADATA couldn't be accessed - imapc: Fixed \Muted label handling in gmail-migration. - imapc: Various bugfixes and improvements. - Trash plugin fixes by Alexei Gradinari - mbox: Fixed crash/corruption in some situations when the first mail was expunged. From sanvila at unex.es Tue May 12 22:15:14 2015 From: sanvila at unex.es (Santiago Vila) Date: Wed, 13 May 2015 00:15:14 +0200 Subject: v2.2.17 release candidate 2 released In-Reply-To: <92448172-9A8D-4898-AE59-1B2A09434E1B@iki.fi> References: <92448172-9A8D-4898-AE59-1B2A09434E1B@iki.fi> Message-ID: <20150512221514.GA14344@cantor.unex.es> On Wed, May 13, 2015 at 12:25:44AM +0300, Timo Sirainen wrote: > - mbox: Fixed crash/corruption in some situations when the first mail > was expunged. Once again: This is *not* fixed! I've just tested it on 2:2.2.17~rc1-1~auto+17 by Stephan Bosch, which is based on revision 18656:6f167f2e550d, and none of the changesets between 18656 and 18660 are related to this. Please try a script like this one: #!/bin/sh rm -rf $HOME/mail/.imap cp -f $HOME/etc/inbox-b $HOME/mail fetchmail -a qemu-jessie --folder inbox-b -m "true" to see it four yourself if you do not believe me. Thanks. From number6 at noisynotes.com Wed May 13 13:15:20 2015 From: number6 at noisynotes.com (Steve Matzura) Date: Wed, 13 May 2015 09:15:20 -0400 Subject: Trying to get mail working Message-ID: I have charge of a very mixed system--current OS (Fedora 20--OK 21's out but I just haven't upgraded yet), current (or nearly so) Mailman (2.1.18-1), Postfix 2.10 with a configuration file sfrom something a lot older which I've run through the upgrade-configuration procedure, and old Dovecot (1.0.15). Is there anything I can do to test each of these components individually, then add a second component and test the three pairs, then all three together? I am nobody's definitioin of an expert in any of these three components, so am having a lot of trouble making them work together, so I thought maybe there might be some test procedure that might straighten me out. Thanks in advance. From tss at iki.fi Wed May 13 16:30:22 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 May 2015 19:30:22 +0300 Subject: v2.2.17 release candidate 2 released In-Reply-To: <20150512221514.GA14344@cantor.unex.es> References: <92448172-9A8D-4898-AE59-1B2A09434E1B@iki.fi> <20150512221514.GA14344@cantor.unex.es> Message-ID: On 13 May 2015, at 01:15, Santiago Vila wrote: > > On Wed, May 13, 2015 at 12:25:44AM +0300, Timo Sirainen wrote: >> - mbox: Fixed crash/corruption in some situations when the first mail >> was expunged. > > Once again: This is *not* fixed! One bug was fixed, but looks like there was another one. I tried after the fix that your test case seemed to work, but I guess I tested it wrong. http://hg.dovecot.org/dovecot-2.2/rev/b6ea460e7cc4 should help. From tss at iki.fi Wed May 13 16:45:58 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 May 2015 19:45:58 +0300 Subject: dovecot-lda panics when postmaster_address is given on command line In-Reply-To: <5551BEBE.4000303@flox.org> References: <5551BEBE.4000303@flox.org> Message-ID: <838CF3F1-48EB-4279-8D2A-211E97DE1748@iki.fi> On 12 May 2015, at 11:50, Onno Molenkamp wrote: > > Hi, > > When passing a postmaster_address to dovecot-lda on the command line > with -o, it panics: > > $ ./dovecot-lda -o postmaster_address=anything > > lda(onno,)Panic: file lda-settings.c: line 80 (lda_settings_check): > assertion failed: (set->postmaster_address[0] == > SETTING_STRVAR_UNEXPANDED[0]) > > It looks like postmaster_address has already been expanded at that point > when it's given on the command line. (set->postmaster_address[0] is "1") Fixed: http://hg.dovecot.org/dovecot-2.2/rev/25dc311a5faa From sanvila at unex.es Wed May 13 18:12:30 2015 From: sanvila at unex.es (Santiago Vila) Date: Wed, 13 May 2015 20:12:30 +0200 Subject: v2.2.17 release candidate 2 released In-Reply-To: References: <92448172-9A8D-4898-AE59-1B2A09434E1B@iki.fi> <20150512221514.GA14344@cantor.unex.es> Message-ID: <20150513181230.GA12064@cantor.unex.es> On Wed, May 13, 2015 at 07:30:22PM +0300, Timo Sirainen wrote: > On 13 May 2015, at 01:15, Santiago Vila wrote: > > > > On Wed, May 13, 2015 at 12:25:44AM +0300, Timo Sirainen wrote: > >> - mbox: Fixed crash/corruption in some situations when the first mail > >> was expunged. > > > > Once again: This is *not* fixed! > > One bug was fixed, but looks like there was another one. I tried after the fix that your test case seemed to work, but I guess I tested it wrong. http://hg.dovecot.org/dovecot-2.2/rev/b6ea460e7cc4 should help. Seems fixed now. Tested on 2:2.2.17~rc2-1~auto+7 from Stephan Bosch. Thanks a lot! From tss at iki.fi Wed May 13 18:36:45 2015 From: tss at iki.fi (Timo Sirainen) Date: Wed, 13 May 2015 21:36:45 +0300 Subject: v2.2.17 released Message-ID: http://dovecot.org/releases/2.2/dovecot-2.2.17.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.17.tar.gz.sig A few director improvements since RC2, some fixes and a new plugin: http://wiki2.dovecot.org/Plugins/QuotaClone * Dovecot no longer checks or warns if a mountpoint is removed. This was causing more trouble than it was worth. Make sure that all the mountpoints that Dovecot accesses aren't writable by mail processes when they're unmounted. * dict server wasn't properly escaping/unescaping data. Fixing this broke backwards compatibility with data that contains line feeds. This hopefully affects only very few installations. If you're using dict to save multiline data (Sieve scripts to SQL), you may be affected. * imap: SPECIAL-USE capability is no longer advertised if there are no special_use flags specified for any mailboxes. + lmtp: Added lmtp_hdr_delivery_address setting to specify whether to include email address in Delivered-To: and Received: headers. + Added initial version of full text search library, which includes language-specific text normalization and filtering. This is still in development, but it's already possible to use for testing with fts-lucene and fts-solr. + lda, lmtp: deliver_log_format can now include %{delivery_time}, which expands to how many milliseconds it took to deliver the mail. With LMTP %{session_time} also expands to how many milliseconds the LMTP session took, not including the delivery time. + lmtp proxy: Mail delivery logging includes timing information. + imap: Most IMAP commands now include in the tagged reply how many milliseconds it took to run the command (not counting the time spent on waiting for the IMAP client to read/write data). + director: Implemented director_proxy_maybe passdb extra field to be able to run director and backend in the same Dovecot instance. (LMTP doesn't support mixed proxy/non-proxy destinations currently.) + doveadm: Added -F parameter to read a list of users from the given file and run the command for all the users. This is similar to -A parameter reading the list of users from userdb lookup. + Implemented initial Cassandra CQL support as lib-sql backend. It's only usable as dict backend currently. + Added quota-clone plugin to copy current quota usage to a dict. - auth: If auth_master_user_separator was set, auth process could be crashed by trying to log in with empty master username. - imap-login, pop3-login: Fixed crash on handshake failures with new OpenSSL versions (v1.0.2) when SSLv3 was disabled. - auth: If one passdb fails allow_nets check, it shouldn't have failed all the other passdb checks later on. - imap: Server METADATA couldn't be accessed - imapc: Fixed \Muted label handling in gmail-migration. - imapc: Various bugfixes and improvements. - Trash plugin fixes by Alexei Gradinari - mbox: Fixed crash/corruption in some situations From stephan at rename-it.nl Wed May 13 21:13:38 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 13 May 2015 23:13:38 +0200 Subject: Released Pigeonhole v0.4.8.rc3 for Dovecot v2.2.17 Message-ID: <5553BE82.3080809@rename-it.nl> Hello Dovecot users, RC2 had one more important bug caused by the new sieve_default_name feature. To prevent causing more of such rather stupid bugs in the future, I created an external shell-based test suite to test dovecot-lda delivery, the doveadm sieve tools and ManageSieve. These kind of tests aren't possible with the current Sieve-based test suite that is included in the Pigeonhole tarball. Now I can test most basic top-level Pigeonhole functionality in one go. Additionally, I created one new feature for ManageSieve. It allows logging various statistics at client disconnect, much like POP3 currently supports. This is e.g. useful for debugging. In the process, I fixed a few ManageSieve bugs as well. If RC3 doesn't cause any more issues, I will release 0.4.8 this Friday. Changelog v0.4.8 (updated): * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to include %{delivery_time}. This prompted changes in Pigeonhole that make this release dependent on Dovecot v2.2.17. + Implemented magic to make sieve_default script visible from ManageSieve under a configurable name. This way, users can see the default rules, edit them and store a private adjusted version. This could also be achieved by copying the default script into the user's script storage, but updates to the global sieve_default script would be ignored that way. + ManageSieve: Implemented support for reporting command statistics at disconnect. Statistics include the number of bytes and scripts uploaded/downloaded/checked and the number of scripts deleted/renamed. - Fixed problem in address test: erroneously decoded mime-encoded words in address headers. - extprograms plugin: Fixed failure occurring when connecting to script service without the need to read back the output from the external program. - Fixed bug in script storage path normalization occurring with relative symbolic links below root. - Fixed and updated various parts of the documentation - ManageSieve: Used "managesieve" rather than "sieve" as login service name, which means that all managesieve-specific settings where ignored. - Managesieve: Storage quota was not always enforced properly for scripts uploaded as quoted string. Nobody uses that, but it is allowed in the specification and we support it, so it should work properly. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc3.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.rc3.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From distler at golem.ph.utexas.edu Thu May 14 07:08:23 2015 From: distler at golem.ph.utexas.edu (Jacques Distler) Date: Thu, 14 May 2015 02:08:23 -0500 Subject: Pigeonhole v0.4.8.rc3 compilation error (bad mail-deliver.h) Message-ID: <4CA2DD2D-34C0-4306-82C7-A0DB49CA3864@golem.ph.utexas.edu> Got a /usr/local/include/dovecot/mail-deliver.h:30:17: error: field has incomplete type 'struct timeval' struct timeval delivery_time_started; ^ error. This is because mail-deliver.h in dovecot-2.2.17 is missing an #include Adding that line to mail-deliver.h fixes the problem. Oddly, I had no problem compiling dovecot itself. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From mhlavink at redhat.com Thu May 14 08:56:13 2015 From: mhlavink at redhat.com (Michal Hlavinka) Date: Thu, 14 May 2015 10:56:13 +0200 Subject: v2.2.17 released In-Reply-To: References: Message-ID: <5554632D.1080009@redhat.com> On 05/13/2015 08:36 PM, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/dovecot-2.2.17.tar.gz > http://dovecot.org/releases/2.2/dovecot-2.2.17.tar.gz.sig > > A few director improvements since RC2, some fixes and a new plugin: http://wiki2.dovecot.org/Plugins/QuotaClone Hi, there seems to be some fts problem, as 2.2.17 crashes during the test check: fts filter find ...................................................... : ok fts filter lowercase ................................................. : ok fts filter stopwords, English ........................................ : ok fts filter stopwords, Finnish ........................................ : ok fts filter stopwords, French ......................................... : ok fts filter stopwords, fail filter() (lazy init) ...................... : ok fts filter normalizer Swedish short text ............................. : ok fts filter normalizer Swedish short text using default ID ............ : ok fts filter normalizer French UDHR .................................... : ok fts filter normalizer empty tokens ................................... : ok fts filter normalizer invalid id ..................................... : ok test-fts-filter.c:512: Assert failed: fts_filter_create(fts_filter_stemmer_snowball, filter, &english_language, NULL, &stemmer, &error) == 0 ../../run-test.sh: line 11: 18401 Segmentation fault (core dumped) valgrind -q --log-file=test.out.$$ $* ==18401== Invalid read of size 8 ==18401== at 0x10DDA2: fts_filter_filter (fts-filter.c:101) ==18401== by 0x10BF0E: test_fts_filter_normalizer_stopwords_stemmer_eng (test-fts-filter.c:517) ==18401== by 0x10ECAD: test_run_funcs (test-common.c:311) ==18401== by 0x10F390: test_run (test-common.c:361) ==18401== by 0x10B931: main (test-fts-filter.c:567) ==18401== Address 0x18 is not stack'd, malloc'd or (recently) free'd ==18401== ==18401== ==18401== Process terminating with default action of signal 11 (SIGSEGV) ==18401== Access not within mapped region at address 0x18 ==18401== at 0x10DDA2: fts_filter_filter (fts-filter.c:101) ==18401== by 0x10BF0E: test_fts_filter_normalizer_stopwords_stemmer_eng (test-fts-filter.c:517) ==18401== by 0x10ECAD: test_run_funcs (test-common.c:311) ==18401== by 0x10F390: test_run (test-common.c:361) ==18401== by 0x10B931: main (test-fts-filter.c:567) ==18401== If you believe this happened as a result of a stack ==18401== overflow in your program's main thread (unlikely but ==18401== possible), you can try to increase the size of the ==18401== main thread stack using the --main-stacksize= flag. ==18401== The main thread stack size used in this run was 8388608. Makefile:840: recipe for target 'check-test' failed make[2]: *** [check-test] Error 1 From stephan at rename-it.nl Thu May 14 11:19:50 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 14 May 2015 13:19:50 +0200 Subject: Pigeonhole v0.4.8.rc3 compilation error (bad mail-deliver.h) In-Reply-To: <4CA2DD2D-34C0-4306-82C7-A0DB49CA3864@golem.ph.utexas.edu> References: <4CA2DD2D-34C0-4306-82C7-A0DB49CA3864@golem.ph.utexas.edu> Message-ID: <555484D6.6090502@rename-it.nl> On 5/14/2015 9:08 AM, Jacques Distler wrote: > Got a > > /usr/local/include/dovecot/mail-deliver.h:30:17: error: field > has incomplete type 'struct timeval' > struct timeval delivery_time_started; > ^ > error. > > This is because mail-deliver.h in dovecot-2.2.17 is missing an > > #include > > Adding that line to mail-deliver.h fixes the problem. Oddly, I had no problem compiling dovecot itself. Timo fixed it: http://hg.dovecot.org/dovecot-2.2/rev/05e14a49507f Regards, Stephan. From trashcan at odo.in-berlin.de Thu May 14 12:46:01 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Thu, 14 May 2015 14:46:01 +0200 Subject: Best practice for getting sieve scripts replicated? Message-ID: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> Hi ? I do have replication between two servers running, and mail is replicated without any issues. I do have sieve running at both servers without any issues. But replication of sieve scripts doesn't work as "expected": Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? Thanks and regards, Michael From stephan at rename-it.nl Thu May 14 17:21:21 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 14 May 2015 19:21:21 +0200 Subject: Best practice for getting sieve scripts replicated? In-Reply-To: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> References: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> Message-ID: <5554D991.9050505@rename-it.nl> On 5/14/2015 2:46 PM, Michael Grimm wrote: > Hi ? > > I do have replication between two servers running, and mail is replicated without any issues. > I do have sieve running at both servers without any issues. > > But replication of sieve scripts doesn't work as "expected": > > Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. > > Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. > > Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? That smells like a bug. What is your configuration (dovecot -n)? This should also show the versions of the packages you have installed. Regards, Stephan. From adam at os.inf.tu-dresden.de Thu May 14 18:14:06 2015 From: adam at os.inf.tu-dresden.de (Adam Lackorzynski) Date: Thu, 14 May 2015 20:14:06 +0200 Subject: v2.2.17 Segfault Message-ID: <20150514181405.GA13172@os.inf.tu-dresden.de> Hi, I'm using libexec/dovecot/imap via a ssh tunnel. 2.2.17 segfaults while accessing a NULL pointer. The following change fixes it for me: --- dovecot-2.2.17/src/lib-imap/imap-keepalive.c 2015-05-03 11:12:11.000000000 +0200 +++ dovecot-2.2.17/src/lib-imap/imap-keepalive.c 2015-05-14 20:06:29.490617405 +0200 @@ -11,6 +11,8 @@ { unsigned int addr; + if (!ip) + return FALSE; if (ip->family == 0) return FALSE; if (ip->family == AF_INET) { Thanks, Adam -- Adam adam at os.inf.tu-dresden.de Lackorzynski http://os.inf.tu-dresden.de/~adam/ From tss at iki.fi Thu May 14 18:45:20 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 14 May 2015 21:45:20 +0300 Subject: v2.2.17 Segfault In-Reply-To: <20150514181405.GA13172@os.inf.tu-dresden.de> References: <20150514181405.GA13172@os.inf.tu-dresden.de> Message-ID: On 14 May 2015, at 21:14, Adam Lackorzynski wrote: > > Hi, > > I'm using libexec/dovecot/imap via a ssh tunnel. 2.2.17 segfaults while > accessing a NULL pointer. The following change fixes it for me: > > > --- dovecot-2.2.17/src/lib-imap/imap-keepalive.c 2015-05-03 11:12:11.000000000 +0200 > +++ dovecot-2.2.17/src/lib-imap/imap-keepalive.c 2015-05-14 20:06:29.490617405 +0200 > @@ -11,6 +11,8 @@ > { > unsigned int addr; > > + if (!ip) > + return FALSE; > if (ip->family == 0) > return FALSE; > if (ip->family == AF_INET) { Slightly different fix: http://hg.dovecot.org/dovecot-2.2/rev/4c4eff229426 From trashcan at odo.in-berlin.de Thu May 14 19:32:09 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Thu, 14 May 2015 21:32:09 +0200 Subject: Best practice for getting sieve scripts replicated? In-Reply-To: <5554D991.9050505@rename-it.nl> References: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> <5554D991.9050505@rename-it.nl> Message-ID: <40BCD372-C0E2-4B31-BD9F-4518CE7EB2B4@odo.in-berlin.de> > On 14.05.2015, at 19:21, Stephan Bosch wrote: > > On 5/14/2015 2:46 PM, Michael Grimm wrote: >> Hi ? >> >> I do have replication between two servers running, and mail is replicated without any issues. >> I do have sieve running at both servers without any issues. >> >> But replication of sieve scripts doesn't work as "expected": >> >> Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. >> >> Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. >> >> Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? > > > That smells like a bug. What is your configuration (dovecot -n)? This > should also show the versions of the packages you have installed. -------------------------------------------------------------------------------------- mail> doveconf -n # 2.2.17 (05e14a49507f): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8.rc3 (b491aa4139bc) # OS: FreeBSD 10.1-STABLE amd64 zfs auth_username_format = %Ln auth_verbose = yes deliver_log_format = msgid=%m, time=%{delivery_time}ms, status=%$ doveadm_password = Hi, I'd like to have a log per user, and protocol (imap+pop3), where this log would tell me/save the message sender+subject+operation (deleted, marked to expunge/other/was read/other). Is there something similar to this that already exists? If not is there something like this on the TODO/whishlist? Thanks in advanced, Jorge Bastos From daniel.colchete at gmail.com Thu May 14 23:50:35 2015 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Thu, 14 May 2015 20:50:35 -0300 Subject: Alternate Storage altmove not working bug Message-ID: Hello everyone! I would like to report a bug here. We are using dovecot's sdbox with alternate storage here. From time to time my SSD partition gets full and I always find users with lots of e-mails stuck there, and altmove just won't move them. It moves most of the e-mails correctly, but a few was getting stuck everyday. So, I think I have found the problem! When a user moves the message from one mailbox to another, the move will keep all the flags! Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this message was already moved and doesn't do anything. I did a doveadm dump on the destination mailbox's index and what I see is a lot of this: RECORD: seq=104, uid=104, flags=0x48 (Seen) - ext 0 keywords : (0000) - ext 1 modseq : 12 (0c00000000000000) - ext 3 cache : 14004 (b4360000) - cache offset=14004 size=76, prev_offset = 0 - mime.parts: (48000000b404000000000000cc04000000000000e542000000000000fd4300000000000018010000) (pos=0 hdr.p=1204 hdr.v=1228 body.p=17125 body.v=17405 flags=48) - size.physical: 18329 (9947000000000000) - date.received: 1418235366 (e68d8854) Although the 0x40 flag is set, the file still lives on the main/ssd storage area. My config: http://pastebin.com/8Wk4bfRK Dovecot version: 2.2.5-0~auto+22 Host is Xeon E3 64bit, running Debian 7. Filesystem is plain local ext4 on both storage areas I know my version is outdated but I didn't find anything related to this there. Just one last question: how can I remove that flag from the message? I have taken a look at doveadm-flags(1) but it doesn't seem to allow anything on that flag. Thank you very much! Best, Daniel Colchete From skdovecot at smail.inf.fh-brs.de Fri May 15 06:20:51 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 15 May 2015 08:20:51 +0200 (CEST) Subject: Trying to get mail working In-Reply-To: References: Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Wed, 13 May 2015, Steve Matzura wrote: > I have charge of a very mixed system--current OS (Fedora 20--OK 21's > out but I just haven't upgraded yet), current (or nearly so) Mailman > (2.1.18-1), Postfix 2.10 with a configuration file sfrom something a > lot older which I've run through the upgrade-configuration procedure, > and old Dovecot (1.0.15). Is there anything I can do to test each of > these components individually, then add a second component and test > the three pairs, then all three together? I am nobody's definitioin of > an expert in any of these three components, so am having a lot of > trouble making them work together, so I thought maybe there might be > some test procedure that might straighten me out. Hmm, if you are building a new system, I would go in this order: 1) Dovecot 2) Postfix 3) Mailman Because Postfix might use Dovecot SASL and will eventually call Dovecot LDA or LMTP. Mailman requires a MTA. Also, I would create the conf for Dovecot from scratch, because there are changes from v1 to v2 one should know about. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVVWQQ3z1H7kL/d9rAQKEDAf/V/RRcslYSYlatlQq3Ue3iIZMgqxNPw9V FDEHTIsDKEcvKp9/p/1M0FcOD3z8dqqLUylOueOLontvOgdhGEVMommftiwno0kw 0iJZ6SeJil5gVQJqqWRG61jP9B+m0grFtVu+qg9AinBhF2dFKxhqLimHc3/NloN9 R0gQdRXe7FZHWjOjCIGNvHYpFw/cMAK1iaZFSkP3UhBGBQH270zo8WPUstaH8XWG Xut9mIpvTHZ3DddPPJPg+926eP6jxPBm9Ssb1JOxP+Of9pCAhPbhwftUiVInlZZ2 JozVZl3RJ0Zk1JPw1txUJCJKaDaKOBtl6P6CjIIsFPUZ+WOBhIVRWA== =y8Gv -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Fri May 15 06:28:40 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 15 May 2015 08:28:40 +0200 (CEST) Subject: [Dovecot] IMAP and POP3 operations log per user In-Reply-To: <011201d08e8f$e2f096f0$a8d1c4d0$@jorge@decimal.pt> References: <011201d08e8f$e2f096f0$a8d1c4d0$@jorge@decimal.pt> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 14 May 2015, Jorge Bastos wrote: > I'd like to have a log per user, and protocol (imap+pop3), where this log > would tell me/save the message sender+subject+operation (deleted, marked to > expunge/other/was read/other). > Is there something similar to this that already exists? Check out the mail_log plugin. You'll have to break the logfiles by users via script. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVVWSGHz1H7kL/d9rAQLIzAf/ZtNOLa1n12LnxwhMaS1B+6LEHDOCZWtw ldTarGE0uovu/OH84PEq4MkE5Y1Oww+VWmA+b2csBx1C0bZmcQeF2g7O65fESYsI Qu/xSuJLhOyUm4sppE62Zqyxba+rG2tyj9XpKMrHI6nU3ZTCOhVu/RJ2TB2WkhkB 21pgYEpkBYvRZK2PPqRwl+Lwc3oGQvWoYHcyniXL0v2P5tNNQj1gTuGVm53dPliM cFm9p4EjAOKEZyFgQokBljDpp03OMpEu16GT8XjCg8Fxg2BkF2XHBx/Wvv8DxQCC vyLA1SFMLNNRoizwF+xVlUJu4AW8fWBLhCgvOAFw4mFXkbMoi+lUwQ== =vRwJ -----END PGP SIGNATURE----- From tss at iki.fi Fri May 15 08:17:22 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 May 2015 11:17:22 +0300 Subject: Alternate Storage altmove not working bug In-Reply-To: References: Message-ID: <10B48A8A-F268-4C84-95A4-C078318ED430@iki.fi> On 15 May 2015, at 02:50, Daniel van Ham Colchete wrote: > > Hello everyone! > > I would like to report a bug here. We are using dovecot's sdbox with > alternate storage here. From time to time my SSD partition gets full and I > always find users with lots of e-mails stuck there, and altmove just won't > move them. It moves most of the e-mails correctly, but a few was getting > stuck everyday. > > So, I think I have found the problem! When a user moves the message from > one mailbox to another, the move will keep all the flags! > Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this message > was already moved and doesn't do anything. Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/9ad3f11617f1 > Just one last question: how can I remove that flag from the message? I have > taken a look at doveadm-flags(1) but it doesn't seem to allow anything on > that flag. You can use doveadm altmove -r parameter to "move" the mail back to the primary storage. This fixes the flag. Then you can move it to alt storage. From swehack at gmail.com Fri May 15 09:43:34 2015 From: swehack at gmail.com (Stefan Midjich) Date: Fri, 15 May 2015 11:43:34 +0200 Subject: Return code 139 when enabling auth debugging Message-ID: Hi I'm running dovecot 2.2.10 installed from base repos on CentOS 7. I'm trying to use sqlite as an auth backend but it's not working so I enabled some auth debugging settings in conf.d/10-logging.conf and then dovecot refused to start. sudo dovecot -F produced 139. Here is doveconf -n: http://dpaste.com/2BNB2F7 Unfortunately when I have these settings enabled doveconf does not work and only produces the first comment line. So here is the output of sudo grep -v '^#' /etc/dovecot/conf.d/10-logging.conf | grep -v '^$': http://dpaste.com/0G0VVK8 And when these settings are uncommented dovecot -F produces return code 139 and no output. So I'm having a hard time troubleshooting my authentication when I can't even troubleshoot my debug options not working. ;) -- H?lsningar / Greetings http://Stefan.Midjich.name From stephan at rename-it.nl Fri May 15 11:03:00 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 15 May 2015 13:03:00 +0200 Subject: Best practice for getting sieve scripts replicated? In-Reply-To: <40BCD372-C0E2-4B31-BD9F-4518CE7EB2B4@odo.in-berlin.de> References: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> <5554D991.9050505@rename-it.nl> <40BCD372-C0E2-4B31-BD9F-4518CE7EB2B4@odo.in-berlin.de> Message-ID: <5555D264.8010707@rename-it.nl> On 5/14/2015 9:32 PM, Michael Grimm wrote: > On 14.05.2015, at 19:21, Stephan Bosch wrote: > On 5/14/2015 2:46 PM, Michael Grimm wrote: >>> I do have replication between two servers running, and mail is replicated without any issues. >>> I do have sieve running at both servers without any issues. >>> >>> But replication of sieve scripts doesn't work as "expected": >>> >>> Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. >>> >>> Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. >>> >>> Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? >> >> That smells like a bug. What is your configuration (dovecot -n)? This >> should also show the versions of the packages you have installed. > Sorry, I should thought about providing more before :-( Sorry, again. > > Feel free to ask for potential tests from my site. I had to setup my own replication (I usually only test with manual dsync invocation). Still, I cannot reproduce your problem: replication occurs instantly. Our setups aren't very different, with the exception that I installed two Dovecot instances on one server with replication configured between them. Do your logs show anything strange? What do your logs show when you enable mail_debug=yes? What do the doveadm commands show when the -D option is added? Is your configuration completely equal at both servers (except of course for the configured replication peer)? Regards, Stephan. From tss at iki.fi Fri May 15 11:11:49 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 May 2015 14:11:49 +0300 Subject: v2.2.18 released Message-ID: <4A5CAA76-DAA9-47E8-942C-E210E67F3C47@iki.fi> http://dovecot.org/releases/2.2/dovecot-2.2.18.tar.gz http://dovecot.org/releases/2.2/dovecot-2.2.18.tar.gz.sig Oops, director was somewhat broken in 2.2.17. I thought I tested the last changes in it well enough, but looks like not. To avoid all existing Dovecot director installations from breaking I decided to make 2.2.18 release quickly afterwards.. BTW. We're planning on making some changes soon to how version control is used in Dovecot development, which should help avoid these kind of problems. The main problem now is that everything gets directly committed to the master branch (although we have a separate more stable dovecot-ee repo also). The new plan is to do something like this: http://nvie.com/posts/a-successful-git-branching-model/ Besides some minor compiling issues and other small changes, the important ones are: - director: Login UNIX sockets were normally detected as doveadm or director ring sockets, causing it to break in existing installations. - sdbox: When copying a mail in alt storage, place the destination to alt storage as well. From stephan at rename-it.nl Fri May 15 11:25:40 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 15 May 2015 13:25:40 +0200 Subject: Best practice for getting sieve scripts replicated? In-Reply-To: <5555D264.8010707@rename-it.nl> References: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> <5554D991.9050505@rename-it.nl> <40BCD372-C0E2-4B31-BD9F-4518CE7EB2B4@odo.in-berlin.de> <5555D264.8010707@rename-it.nl> Message-ID: <5555D7B4.6090000@rename-it.nl> On 5/15/2015 1:03 PM, Stephan Bosch wrote: > On 5/14/2015 9:32 PM, Michael Grimm wrote: >> On 14.05.2015, at 19:21, Stephan Bosch wrote: >> On 5/14/2015 2:46 PM, Michael Grimm wrote: >>>> I do have replication between two servers running, and mail is replicated without any issues. >>>> I do have sieve running at both servers without any issues. >>>> >>>> But replication of sieve scripts doesn't work as "expected": >>>> >>>> Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. >>>> >>>> Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. >>>> >>>> Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? >>> That smells like a bug. What is your configuration (dovecot -n)? This >>> should also show the versions of the packages you have installed. >> Sorry, I should thought about providing more before :-( Sorry, again. >> >> Feel free to ask for potential tests from my site. > I had to setup my own replication (I usually only test with manual dsync > invocation). Still, I cannot reproduce your problem: replication occurs > instantly. Our setups aren't very different, with the exception that I > installed two Dovecot instances on one server with replication > configured between them. > > Do your logs show anything strange? What do your logs show when you > enable mail_debug=yes? > > What do the doveadm commands show when the -D option is added? > > Is your configuration completely equal at both servers (except of course > for the configured replication peer)? This may be helpful: http://hg.dovecot.org/dovecot-2.2/rev/7cf62f28fd62 (now included in 2.2.18). Regards, Stephan. From bluewind at xinu.at Fri May 15 11:30:14 2015 From: bluewind at xinu.at (Florian Pritz) Date: Fri, 15 May 2015 13:30:14 +0200 Subject: v2.2.18 released In-Reply-To: <4A5CAA76-DAA9-47E8-942C-E210E67F3C47@iki.fi> References: <4A5CAA76-DAA9-47E8-942C-E210E67F3C47@iki.fi> Message-ID: <5555D8C6.802@xinu.at> On 15.05.2015 13:11, Timo Sirainen wrote: > http://dovecot.org/releases/2.2/dovecot-2.2.18.tar.gz > http://dovecot.org/releases/2.2/dovecot-2.2.18.tar.gz.sig > > Oops, director was somewhat broken in 2.2.17. I thought I tested the > last changes in it well enough, but looks like not. Thanks for being better than qemu and actually releasing a new version rather than just a patch. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 801 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Fri May 15 11:59:48 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 15 May 2015 14:59:48 +0300 Subject: imap_logout_format improvements / ideas Message-ID: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> http://hg.dovecot.org/dovecot-2.2/rev/93bba97afb2a http://hg.dovecot.org/dovecot-2.2/rev/e0a17714f0c9 So all in all, you can now add to imap_logout_format: # %i - total number of bytes read from client # %o - total number of bytes sent to client # %{fetch_hdr_count} - Number of mails with mail header data sent to client # %{fetch_hdr_bytes} - Number of bytes with mail header data sent to client # %{fetch_body_count} - Number of mails with mail body data sent to client # %{fetch_body_bytes} - Number of bytes with mail body data sent to client # %{deleted} - Number of mails where client added \Deleted flag # %{expunged} - Number of mails that client expunged # %{trashed} - Number of mails that client copied/moved to the # special_use=\Trash mailbox. Any thoughts on what else would be useful? I was also thinking about separating (deletes and) expunges from Trash vs non-Trash, but I'm not sure how useful that would be (and what would it be called?) It's also a bit annoying that a copy+expunge to a Trash will return expunged=1 trashed=1, but a move to Trash (via IMAP MOVE) will return expunged=0 trashed=1. But trying to fix that would be too much trouble. From daniel.colchete at gmail.com Fri May 15 12:27:52 2015 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Fri, 15 May 2015 09:27:52 -0300 Subject: Alternate Storage altmove not working bug In-Reply-To: <10B48A8A-F268-4C84-95A4-C078318ED430@iki.fi> References: <10B48A8A-F268-4C84-95A4-C078318ED430@iki.fi> Message-ID: 9 hours for a fix and a release! Wow! That's really fast! Thank you very much! On Fri, May 15, 2015 at 5:17 AM, Timo Sirainen wrote: > On 15 May 2015, at 02:50, Daniel van Ham Colchete < > daniel.colchete at gmail.com> wrote: > > > > Hello everyone! > > > > I would like to report a bug here. We are using dovecot's sdbox with > > alternate storage here. From time to time my SSD partition gets full and > I > > always find users with lots of e-mails stuck there, and altmove just > won't > > move them. It moves most of the e-mails correctly, but a few was getting > > stuck everyday. > > > > So, I think I have found the problem! When a user moves the message from > > one mailbox to another, the move will keep all the flags! > > Including MAIL_INDEX_MAIL_FLAG_BACKEND 0x40. So, altmove think this > message > > was already moved and doesn't do anything. > > Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/9ad3f11617f1 > > > Just one last question: how can I remove that flag from the message? I > have > > taken a look at doveadm-flags(1) but it doesn't seem to allow anything on > > that flag. > > You can use doveadm altmove -r parameter to "move" the mail back to the > primary storage. This fixes the flag. Then you can move it to alt storage. > From trashcan at odo.in-berlin.de Fri May 15 12:55:17 2015 From: trashcan at odo.in-berlin.de (Michael Grimm) Date: Fri, 15 May 2015 14:55:17 +0200 Subject: Best practice for getting sieve scripts replicated? In-Reply-To: <5555D7B4.6090000@rename-it.nl> References: <0DE883A6-C7B6-4A77-A9E0-BFEE3F4396C4@odo.in-berlin.de> <5554D991.9050505@rename-it.nl> <40BCD372-C0E2-4B31-BD9F-4518CE7EB2B4@odo.in-berlin.de> <5555D264.8010707@rename-it.nl> <5555D7B4.6090000@rename-it.nl> Message-ID: <51B99945-58D3-4955-9006-325135D7DBDA@odo.in-berlin.de> Stephan Bosch wrote: > On 5/15/2015 1:03 PM, Stephan Bosch wrote: >> On 5/14/2015 9:32 PM, Michael Grimm wrote: >>> On 14.05.2015, at 19:21, Stephan Bosch wrote: >>> On 5/14/2015 2:46 PM, Michael Grimm wrote: >>>>> I do have replication between two servers running, and mail is replicated without any issues. >>>>> I do have sieve running at both servers without any issues. >>>>> >>>>> But replication of sieve scripts doesn't work as "expected": >>>>> >>>>> Whenever I do modify my default script und upload it via "doveadm sieve put -u abc -a default < default" it becomes uploaded to the home directory of user abc as expected, but it will not be replicated to the partner server. >>>>> >>>>> Whenever I do deactivate the default script by "doveadm sieve deactivate -u abc", followed by a deletion "doveadm sieve delete -u abc default" plus subsequent upload of the script "doveadm sieve put -u abc -a default < default", replication takes place instantly. >>>>> >>>>> Questions: Is a modification of a sieve script supposed to be triggered the latter way? Or am I doing it the wrong way? >>>> That smells like a bug. What is your configuration (dovecot -n)? This >>>> should also show the versions of the packages you have installed. >>> Sorry, I should thought about providing more before :-( Sorry, again. >>> >>> Feel free to ask for potential tests from my site. >> I had to setup my own replication (I usually only test with manual dsync >> invocation). Still, I cannot reproduce your problem: replication occurs >> instantly. Our setups aren't very different, with the exception that I >> installed two Dovecot instances on one server with replication >> configured between them. >> >> Do your logs show anything strange? No, I never found something strange. That why I began thinking about my expectations were wrong regarding how sieve replication is supposed to work. >> What do your logs show when you >> enable mail_debug=yes? Nothing strange as well. >> What do the doveadm commands show when the -D option is added? This I didn't try :-( >> Is your configuration completely equal at both servers (except of course >> for the configured replication peer)? Yes, absolutely identical. Same config, same hg revisions, same OS, etc. > This may be helpful: > > http://hg.dovecot.org/dovecot-2.2/rev/7cf62f28fd62 (now included in 2.2.18). Well, I do feel like an idiot! I swear, sieve replication never ran as expected, I always had to do it the way mentioned above. Today, after compiling brand new dovecot, sieve replication works as expected and as you confirmed it should work. I really tried hard to reproduce my issue, but: to no avail ;-) No idea whether those last minute revisions made the trick or something else ? So I feel very sorry having bothered you without real cause :-( Thanks and with kind regards, Michael @Timo: There's a typo in src/plugins/replication/replication-plugin.c: s/transction commit/transaction commit/ From mail at tomsommer.dk Fri May 15 13:17:24 2015 From: mail at tomsommer.dk (Tom Sommer) Date: Fri, 15 May 2015 15:17:24 +0200 Subject: v2.2.18 released In-Reply-To: <4A5CAA76-DAA9-47E8-942C-E210E67F3C47@iki.fi> References: <4A5CAA76-DAA9-47E8-942C-E210E67F3C47@iki.fi> Message-ID: On 2015-05-15 13:11, Timo Sirainen wrote: > BTW. We're planning on making some changes soon to how version control > is used in Dovecot development, which should help avoid these kind of > problems. The main problem now is that everything gets directly > committed to the master branch (although we have a separate more > stable dovecot-ee repo also). The new plan is to do something like > this: http://nvie.com/posts/a-successful-git-branching-model/ GitHub please? :) // Tom From listas at adminlinux.com.br Fri May 15 13:39:10 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Fri, 15 May 2015 10:39:10 -0300 Subject: Many Duplicated messages after migrate from 2.0.19 to 2.2.15 Message-ID: <5555F6FE.7060308@adminlinux.com.br> Hi, In my company, we have several failover clusters (~50k accounts per cluster) with Ubuntu12.04 and Dovecot-2.0.19 (without dsync, replication at block level with DRBD). We are migrating to active/active clusters with Ubuntu14.04 and Dovecot-2.2.15 with replication via dovecot-dsync. Each new cluster consists of two machines and each machine runs two instances of Dovecot (one for backend and the other for proxy Dovecot-director). We did the migration of message files, indexes and sieve scripts using rsync to one of the backends. After we turn on the second backend and let the dsync work in replication for him. Now, many accounts are with duplicated messages and some new emails delivered are also being duplicated. Doveadm deduplicate doesn't work for me. I ran: doveadm deduplicate -u -m ALL Does do the migration of files with rsync was a mistake ? We are using mdbox on both clusters. My confs with doveconf -n for both clusters: http://www.adminlinux.com.br/newcluster_doveconf.txt http://www.adminlinux.com.br/oldcluster_doveconf.txt Thanks and regards, Thiago Henrique From paolo.cravero at csi.it Fri May 15 14:37:14 2015 From: paolo.cravero at csi.it (Paolo Cravero) Date: Fri, 15 May 2015 16:37:14 +0200 (CEST) Subject: imap_logout_format improvements / ideas In-Reply-To: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> References: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> Message-ID: <1141929509.786195.1431700634782.JavaMail.open-xchange@comunica.csi.it> > # %i - total number of bytes read from client > # %o - total number of bytes sent to client > # %{fetch_hdr_count} - Number of mails with mail header data sent to > client > # %{fetch_hdr_bytes} - Number of bytes with mail header data sent to > client > # %{fetch_body_count} - Number of mails with mail body data sent to client > # %{fetch_body_bytes} - Number of bytes with mail body data sent to client > # %{deleted} - Number of mails where client added \Deleted flag > # %{expunged} - Number of mails that client expunged > # %{trashed} - Number of mails that client copied/moved to the > # special_use=\Trash mailbox. > > Any thoughts on what else would be useful? Number of mails that have been \Seen ? I have been running Dovecot with full debug logging so I am not aware if the same information is already available elsewhere in the log at production loglevel. Paolo From larryrtx at gmail.com Fri May 15 14:39:01 2015 From: larryrtx at gmail.com (Larry Rosenman) Date: Fri, 15 May 2015 09:39:01 -0500 Subject: imap_logout_format improvements / ideas In-Reply-To: <1141929509.786195.1431700634782.JavaMail.open-xchange@comunica.csi.it> References: <8CF3F5F9-0E00-401A-920D-74B2FF338338@iki.fi> <1141929509.786195.1431700634782.JavaMail.open-xchange@comunica.csi.it> Message-ID: are the above captioned patches n 2.2.18? On 5/15/15, Paolo Cravero wrote: > >> # %i - total number of bytes read from client >> # %o - total number of bytes sent to client >> # %{fetch_hdr_count} - Number of mails with mail header data sent to >> client >> # %{fetch_hdr_bytes} - Number of bytes with mail header data sent to >> client >> # %{fetch_body_count} - Number of mails with mail body data sent to >> client >> # %{fetch_body_bytes} - Number of bytes with mail body data sent to >> client >> # %{deleted} - Number of mails where client added \Deleted flag >> # %{expunged} - Number of mails that client expunged >> # %{trashed} - Number of mails that client copied/moved to the >> # special_use=\Trash mailbox. >> >> Any thoughts on what else would be useful? > > Number of mails that have been \Seen ? > > I have been running Dovecot with full debug logging so I am not aware if > the > same information is already available elsewhere in the log at production > loglevel. > > Paolo > -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com US Mail: 108 Turvey Cove, Hutto, TX 78634-5688 From cma at cmadams.net Fri May 15 14:58:17 2015 From: cma at cmadams.net (Chris Adams) Date: Fri, 15 May 2015 09:58:17 -0500 Subject: Problem with sieve not triggering randomly? Message-ID: <20150515145817.GA20600@cmadams.net> I'm running CentOS 7.1 with Dovecot 2.2.10 (CentOS packaged version). I have Postfix feeding messages to Dovecot for local delivery via LMTP, and I have Amavisd for spam filtering. For messages above a certain threshold, Amavisd adds the "X-Spam-Flag: YES" header, and I file that into the Spam folder using a default (system-wide) sieve script (there aren't any per-user sieve scripts set up): require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } I can see it work on my test messages and from lmtp log entries. However, it doesn't seem to always work. I had a user complain about a spam increase, and I looked in their mailbox. I see a message with the "X-Spam-Flag: YES" header, but it is in the INBOX (and the corresponding lmtp log entry shows that's where it went). Why would the sieve script work on some messages and not others? I don't see any errors in the log. I see a bunch of messages from the same source to different users, and they were all filed into INBOX, not Spam. Looking at the message in question, it is a pretty bogus message. There are multiple From: and Subject: headers. Would that cause sieve to "punt" on the message (but not log anything about it)? -- Chris Adams From stephan at rename-it.nl Fri May 15 15:08:54 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 15 May 2015 17:08:54 +0200 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150515145817.GA20600@cmadams.net> References: <20150515145817.GA20600@cmadams.net> Message-ID: <55560C06.3080904@rename-it.nl> On 5/15/2015 4:58 PM, Chris Adams wrote: > I'm running CentOS 7.1 with Dovecot 2.2.10 (CentOS packaged version). I > have Postfix feeding messages to Dovecot for local delivery via LMTP, > and I have Amavisd for spam filtering. For messages above a certain > threshold, Amavisd adds the "X-Spam-Flag: YES" header, and I file that > into the Spam folder using a default (system-wide) sieve script (there > aren't any per-user sieve scripts set up): > > require "fileinto"; > if header :contains "X-Spam-Flag" "YES" { > fileinto "Spam"; > } > > I can see it work on my test messages and from lmtp log entries. > > However, it doesn't seem to always work. I had a user complain about a > spam increase, and I looked in their mailbox. I see a message with the > "X-Spam-Flag: YES" header, but it is in the INBOX (and the corresponding > lmtp log entry shows that's where it went). > > Why would the sieve script work on some messages and not others? I > don't see any errors in the log. I see a bunch of messages from the > same source to different users, and they were all filed into INBOX, not > Spam. > > Looking at the message in question, it is a pretty bogus message. There > are multiple From: and Subject: headers. Would that cause sieve to > "punt" on the message (but not log anything about it)? You can check the handling of a particular message yourself using the sieve-test tool (there is a man page for it). By specifying the `-t - -Tlevel=matching` options, you'll get detailed information on why a particular decision is made. Without insight in your configuration (output from `dovecot -n`) and an example of a message that is mishandled, I cannot tell you more. Regards, Stephan. From cma at cmadams.net Fri May 15 15:56:04 2015 From: cma at cmadams.net (Chris Adams) Date: Fri, 15 May 2015 10:56:04 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <55560C06.3080904@rename-it.nl> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> Message-ID: <20150515155604.GB20600@cmadams.net> Once upon a time, Stephan Bosch said: > You can check the handling of a particular message yourself using the > sieve-test tool (there is a man page for it). By specifying the `-t - > -Tlevel=matching` options, you'll get detailed information on why a > particular decision is made. Hmm, that's weird. sieve-test says it would store the message into the Spam folder (as expected), but it was definitely delivered to INBOX. > Without insight in your configuration (output from `dovecot -n`) and an > example of a message that is mishandled, I cannot tell you more. Here you go. I changed the local part of the user's email address in the spam message, and the doveadm password in the doveconf output, but that's all. http://www.cmadams.net/misc/dove/doveconf-n.txt http://www.cmadams.net/misc/dove/spam-message.txt I have three Dovecot VMs behind a pair of Dovecot director VMs; this is from the actual host that handled the delivery (all three backend hosts are showing the same issue). Users are in MySQL and mail is on NFS. There are some things configured, like managesieve, that aren't currently used. This is a new setup, migrated from Courier; I made the switch a couple of weeks ago (not my first time using Dovecot, just finally got a chance to migrate this setup). Thanks. -- Chris Adams From delphij at delphij.net Fri May 15 17:58:23 2015 From: delphij at delphij.net (Xin Li) Date: Fri, 15 May 2015 10:58:23 -0700 Subject: charset_to_utf8 assertion of remaining src_size Message-ID: <555633BF.6010109@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, I have seen the following assertion fails on my own mail server (indexer-worker): Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) However, when charset_to_utf8_try returns FALSE (e.g. iconv() got EINVAL or EILSEQ), the for loop in charset_to_utf8 may end earlier, and there may be additional pending characters, and the assertion doesn't seem always hold in the error case, perhaps we should assert that pos is non-zero in the error case? Cheers, - -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.1.2 (FreeBSD) iQIcBAEBCgAGBQJVVjO/AAoJEJW2GBstM+nsbKUP/29fYdZArCiUSStK0hCtRvC3 o4L35zq4iP4WUry7VqNtdMg2Uiythei9l0JokNOJAy2ZDktJ3ns9CArXHi/pJE7/ V8374Iu9ygb4tzDvcNY/353SFkN+Sw4pJzSGJTelVT8nHxahQMtN7E0u26l54zNx p1H8rgt2nVA8eRvKL9MrhudN3eSI2+Jq4HX+dR4/Tu4qLkSU1nqi8sopQxNXzW4C 4XG4haSbTtXLu0H1MGa8y5pmwrFmZEluNOBm2SOww+UqeteD/MwE/pgA7g+cQx8l /t3FWzSe4C9HROV1VGW5V5HDM7cDvNJZ+nxQGI6L6qInkCBqQx/muIb9nIe7dhro tiHIuX4h8X2zaga+iD+B9fkf4QpDUc11WYE+0DUArb2cNTTAgeTXQ68+g0Q+UC8s OxUVDA4KwOSzkpEifDgvVCdgPG7U87SGe9NlMGjSZvAIEPsww16A3f2IaM4fhFa2 lVuTh1wq8phdWXLbCbZ0F36hAyxVbbwC5Xyj4Ygy3QL4RiBC7CJJ8I4f1lRLnoFz 1Wg0/M7psXBLVN7vx2MKtK5Rj1OvMOKmL4VpkhKcnp2z/Y9I5Qq2CP4C5uOv3DLJ JjoCsnVDpWJKQo6EPleb9OxVJF+NmjHd63zimXbuUyDJ7AJCsx/0m9mJZ5QYf+jU U3OqESCU5e5q++QLd2Kl =ksoA -----END PGP SIGNATURE----- From stephan at rename-it.nl Fri May 15 18:15:37 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 15 May 2015 20:15:37 +0200 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150515155604.GB20600@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> Message-ID: <555637C9.1060901@rename-it.nl> On 5/15/2015 5:56 PM, Chris Adams wrote: > Once upon a time, Stephan Bosch said: >> You can check the handling of a particular message yourself using the >> sieve-test tool (there is a man page for it). By specifying the `-t - >> -Tlevel=matching` options, you'll get detailed information on why a >> particular decision is made. > Hmm, that's weird. sieve-test says it would store the message into the > Spam folder (as expected), but it was definitely delivered to INBOX. BTW, did you consult your logs for any errors? If an error occurs during Sieve processing, the default behavior is to file the message into INBOX (e.g. when the Spam folder doesn't exist). Regards, Stephan. From cma at cmadams.net Fri May 15 18:50:24 2015 From: cma at cmadams.net (Chris Adams) Date: Fri, 15 May 2015 13:50:24 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <555637C9.1060901@rename-it.nl> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> Message-ID: <20150515185024.GE20600@cmadams.net> Once upon a time, Stephan Bosch said: > On 5/15/2015 5:56 PM, Chris Adams wrote: > > Once upon a time, Stephan Bosch said: > >> You can check the handling of a particular message yourself using the > >> sieve-test tool (there is a man page for it). By specifying the `-t - > >> -Tlevel=matching` options, you'll get detailed information on why a > >> particular decision is made. > > Hmm, that's weird. sieve-test says it would store the message into the > > Spam folder (as expected), but it was definitely delivered to INBOX. > > BTW, did you consult your logs for any errors? If an error occurs during > Sieve processing, the default behavior is to file the message into INBOX > (e.g. when the Spam folder doesn't exist). Yes, I didn't find any errors, just the lmtp log entry for storing into INBOX. In the several cases I looked at, the Spam folder exists, is getting most spam-flagged messages filed into it, it just seems to be something about some messages (for example, got a bunch of copies of this particular spam to different users and they all went to INBOX instead of Spam). May 15 09:44:04 dovecot2 dovecot: lmtp(10157): Connect from 10.0.9.71 May 15 09:44:05 dovecot2 dovecot: lmtp(10157, localuser9 at foothills.net): MikHD8/YVVWtJwAAIYJ+iw: sieve: msgid=<438088200.476329351.0075518260474.JavaMail.root at sjmas02.lococandles.co>: stored mail into mailbox 'INBOX' May 15 09:44:05 dovecot2 dovecot: lmtp(10157): Disconnect from 10.0.9.57: Successful quit Also, if the Spam folder didn't exist, I have it set to be autocreated in the Dovecot config. I also know that Dovecot can write to the Spam folder okay, because I did a "doveadm move" to move several "X-Spam-Flag: YES" messages to the Spam folder, and that worked. -- Chris Adams From stephan at rename-it.nl Fri May 15 19:07:13 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 15 May 2015 21:07:13 +0200 Subject: Released Pigeonhole v0.4.8 for Dovecot v2.2.18 Message-ID: <555643E1.8070001@rename-it.nl> Hello Dovecot users, Here is the final 0.4.8 release. No significant changes were committed since the last release candidate. Changelog v0.4.8: * LDA Sieve plugin: Dovecot changed the deliver_log_format setting to include %{delivery_time}. This prompted changes in Pigeonhole that make this release dependent on Dovecot v2.2.17. + Implemented magic to make sieve_default script visible from ManageSieve under a configurable name. This way, users can see the default rules, edit them and store a private adjusted version. This could also be achieved by copying the default script into the user's script storage, but updates to the global sieve_default script would be ignored that way. + ManageSieve: Implemented support for reporting command statistics at disconnect. Statistics include the number of bytes and scripts uploaded/downloaded/checked and the number of scripts deleted/renamed. - Fixed problem in address test: erroneously decoded mime-encoded words in address headers. - extprograms plugin: Fixed failure occurring when connecting to script service without the need to read back the output from the external program. - Fixed bug in script storage path normalization occurring with relative symbolic links below root. - Fixed and updated various parts of the documentation - ManageSieve: Used "managesieve" rather than "sieve" as login service name, which means that all managesieve-specific settings where ignored. - Managesieve: Storage quota was not always enforced properly for scripts uploaded as quoted string. Nobody uses that, but it is allowed in the specification and we support it, so it should work properly. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.tar.gz http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.8.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan at rename-it.nl From jtam.home at gmail.com Fri May 15 21:18:54 2015 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 15 May 2015 14:18:54 -0700 (PDT) Subject: imap_logout_format improvements In-Reply-To: References: Message-ID: Timo Sirainen wrote: > So all in all, you can now add to imap_logout_format: >... > Any thoughts on what else would be useful? Maybe quota stats like number of quota errors, or quota usage? Bytes used by Email marked \Deleted? Joseph Tam From ghostlovescorebg at gmail.com Fri May 15 22:01:07 2015 From: ghostlovescorebg at gmail.com (=?UTF-8?B?TWlsb8WhIER1a2nEhw==?=) Date: Sat, 16 May 2015 00:01:07 +0200 Subject: [Dovecot] IMAP and POP3 operations log per user In-Reply-To: <55551899.c35bc20a.6fd6.ffffbf39SMTPIN_ADDED_BROKEN@mx.google.com> References: <55551899.c35bc20a.6fd6.ffffbf39SMTPIN_ADDED_BROKEN@mx.google.com> Message-ID: <55566CA3.9090208@gmail.com> On 05/14/15 23:49, Jorge Bastos wrote: > Hi, > > > > I'd like to have a log per user, and protocol (imap+pop3), where this log > would tell me/save the message sender+subject+operation (deleted, marked to > expunge/other/was read/other). > > Is there something similar to this that already exists? > > > > If not is there something like this on the TODO/whishlist? > > > > Thanks in advanced, > > Jorge Bastos > . > Jorge, Per user loggin is not available. You can enable Dovecot events in 10-logging.conf. Just uncomment mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename in plugin section. Depending on your setup, it will be logged to /var/log/maillog, or whatever you defined in your syslog configuration. From tss at iki.fi Sat May 16 00:57:23 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 May 2015 03:57:23 +0300 Subject: imap_logout_format improvements In-Reply-To: References: Message-ID: <35D0856E-4712-4B3C-A492-AECB465C6BBD@iki.fi> On 16 May 2015, at 00:18, Joseph Tam wrote: > > Timo Sirainen wrote: > >> So all in all, you can now add to imap_logout_format: >> ... >> Any thoughts on what else would be useful? > > Maybe quota stats like number of quota errors, or quota usage? Quota is an optional plugin. Kludgy to implement that.. > Bytes used by Email marked \Deleted? Either unreliable or potentially very slow.. I think better not to give bad numbers. From reuben-dovecot at reub.net Sat May 16 02:46:35 2015 From: reuben-dovecot at reub.net (Reuben Farrelly) Date: Sat, 16 May 2015 12:46:35 +1000 Subject: Additional userdb variables in passwd [was Re: Dovecot Replication - Architecture Endianness?] In-Reply-To: <554CB45C.2080205@reub.net> References: <5545FD08.2010701@reub.net> <55476ED1.4050705@dovecot.fi> <5549F26D.8010702@reub.net> <9A41FF8B-4DD4-43C8-BB98-D7BBBF77FDAE@iki.fi> <554B4D3E.4050805@reub.net> <554C6F69.6090205@dovecot.fi> <554CB45C.2080205@reub.net> Message-ID: <5556AF8B.5060903@reub.net> On 8/05/2015 11:04 PM, Reuben Farrelly wrote: > On 8/05/2015 6:10 PM, Teemu Huovila wrote: >> On 05/07/2015 02:32 PM, Reuben Farrelly wrote: >>> On 7/05/2015 7:49 AM, Timo Sirainen wrote: >>>> On 06 May 2015, at 13:52, Reuben Farrelly >>>> wrote: >>>>> >>>>> On 4/05/2015 11:06 PM, Teemu Huovila wrote: >>>>>>> Also is there a way to restrict replication users aside >>>>>>> from a crude hack around system first and last UIDs? >>>>>> You can set the userdb to return an empty mail_replica >>>>>> variable for users you want to exclude from replication. >>>>>> http://hg.dovecot.org/dovecot-2.2/rev/c1c67bdc8752 >>>>>> >>>>>> br, Teemu Huovila >>>>> >>>>> One last question. Is it possible to achieve this with system >>>>> users and PAM or do I need to basically create a new static >>>>> userdb for system users? >>>> >>>> You can create a new userdb passwd-file that adds extra fields. >>>> So something like: >>>> >>>> userdb { driver = passwd result_success = continue-ok } >>>> >>>> userdb { driver = passwd-file args = /etc/dovecot/passwd.extra >>>> skip = notfound } >>> >>> This doesn't seem to work for me and my config has that exact >>> config. My password.extra file has just one line for the one >>> account I am testing with at the moment: >>> >>> user1:::::::userdb_mail_replica=tcps:lightning.reub.net:4813,userdb_mail_replica=tcp:pi.x.y:4814 >>> >>> >>> >>> > This breaks access for other system users such as my own account which > do not have entries: >>> >>> ay 7 21:19:06 tornado.reub.net dovecot: imap-login: Internal login >>> failure (pid=22573 id=1) (internal failure, 1 successful auths): >>> user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, >>> local=2001:44b8:31d4:1310::20, TLS >>> >>> which then starts soon spitting this out 10s of times per second in >>> the mail log: >>> >>> May 7 21:19:32 tornado.reub.net dovecot: auth-worker(23738): >>> Error: Auth worker sees different passdbs/userdbs than auth server. >>> Maybe config just changed and this goes away automatically? >>> >>> This is with -hg latest as of now. >>> >>> This system uses PAM for local users. Do I need to replicate all >>> of the system users including those who do not need any extra >>> settings, in the passwd.extra file too? >>> >>> Is my syntax above for two mail_replica servers correct? >> A bit unsure about the config syntax, so I can not advice on that, >> but there were some bugs in auth yesterday. Maybe you could retest >> with f2a8e1793718 or newer. Make sure configs on both sides are in >> sync. >> >> Thank you for your continued testing, Teemu Huovila >> > > With -hg as of now it's still not any better: > > tornado log # dovecot --version > 2.2.16 (f2a8e1793718+) > tornado log # > > =================== > > # System users (NSS, /etc/passwd, or similiar). In many systems nowadays > this > # uses Name Service Switch, which is configured in /etc/nsswitch.conf. > userdb { > # > driver = passwd > # [blocking=no] > #args = > > # Override fields from passwd > #override_fields = home=/home/virtual/%u > > result_success = continue-ok > } > > # Add some extra fields such as replication.. > > userdb { > driver = passwd-file > args = /etc/dovecot/passwd.extra > skip = notfound > } > > ============== > > May 8 22:59:11 tornado.reub.net dovecot: imap: Error: Authenticated > user not found from userdb, auth lookup id=586547201 (client-pid=29035 > client-id=1) > May 8 22:59:11 tornado.reub.net dovecot: imap-login: Internal login > failure (pid=29035 id=1) (internal failure, 1 successful auths): > user=, auth-method=PLAIN, remote=2001:44b8:31d4:1311::50, > local=2001:44b8:31d4:1310::20, TLS > > It logs an awful lot of those lines in short succession also, at least > 15 per second... > > Reuben Following on from this I've managed to get it to work - but there is one outstanding problem which I suspect may be a bug. Running -hg build as of today. In case anyone else tries this, I had to separate each userdb_mail_replica entry with a space. This is however, documented in the wiki. The outstanding issue is that even though I've had 'skip = notfound' in the second userdb as above, if I don't add all of the users to that file (even with no extra variables set) those users who are not added cannot log in. They fail with the error above about an 'internal failure'. It seems that the second passdb is not actually being skipped at all if the user is not listed in it...Timo? Thanks, Reuben From ebroch at whitehorsetc.com Sat May 16 03:51:21 2015 From: ebroch at whitehorsetc.com (Eric Broch) Date: Fri, 15 May 2015 21:51:21 -0600 Subject: fs_api_module_register Message-ID: <5556BEB9.9010709@whitehorsetc.com> Hello list, I'm trying to build a binary rpm from dovecot sources 2.2.18 and pigeonhole.0.4.8. During the make portion I'm getting the following error: make[3]: Leaving directory `/root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage' for bin in test-mail-search-args-imap test-mail-search-args-simplify test-mailbox-get; do \ if ! /bin/sh ../../run-test.sh ../.. ./$bin; then exit 1; fi; \ done /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: symbol lookup error: /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: undefined symbol: fs_api_module_register I created a binary rpm from dovecot sources 2.2.16 and pigeonhole sources 0.4.7 and all went well. Can anyone give me any hints as to the problem? Eric From delphij at delphij.net Sat May 16 06:39:39 2015 From: delphij at delphij.net (Xin Li) Date: Fri, 15 May 2015 23:39:39 -0700 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: <555633BF.6010109@delphij.net> References: <555633BF.6010109@delphij.net> Message-ID: <5556E62B.20306@delphij.net> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Hi, On 5/15/15 10:58, Xin Li wrote: > Hi, > > I have seen the following assertion fails on my own mail server > (indexer-worker): > > Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion > failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) I've managed to narrow down the issue to one mail, and can reliably trigger the assertion with 'doveadm index -u user INBOX' with only that email. Maybe we should reset the iconv descriptor after each full conversion? i.e.: diff -r 91bbd6753689 src/lib-mail/message-decoder.c - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 @@ -347,6 +347,8 @@ sizeof(ctx->translation_buf)); memcpy(ctx->translation_buf, data + pos, ctx->translation_size); + } else { + charset_to_utf8_reset(ctx->charset_trans); } output->data = ctx->buf2->data; output->size = ctx->buf2->used; Cheers, -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJVVuYpAAoJEJW2GBstM+nsrr8P+wZRjznaTJuAE7eaKlmO5Zba m9bUHsHh/rZdjfTCfUiwYwU66+EI8O2WEgr0sXynMrRhRWp5BVWMKGTl2loJ/oNL O3qXrGoVmAJoWmnd+ZQ49s+zSJUgXk9rJneN1i4erId6vjN1pcWXfrLbmeKhreN8 EmOMDO8WtV/XBq20UerKu+kGnatC8tNzpTQ5PCTnMc9z1nvNytE/NxM3Gli3wTRG 8nhZY8fS+y6guJfi68RZK6Th1W6jKQQW8+sNPlNbCRPxDb5delXLGq58DnQVk4mM Y6VxzQF4ItTU5h+ikNOaY7v4LoC3uydUdhPkMVVb5JVH+wx9en7PefhV4dH2MNjJ a8dJ4gvsYNBLZ1r3QLFF0NqJMhPXtidKLy7iQkBW0Z1LqmQrEpwZpLVGrdDtWLpA E4HkOmTMEyIbTuK5hG5zNyoF2+XLHSCAELZBHxJAxoHhwakm3n4MhJJao+aJho8+ DbgbGMFk4dDgvOAD0gqANSrIfpZkWGAzUiNiMP015Trg4bQm7CSztuvM6+xhr8EE zzvYjbeCAl3A1+u2q2UFaw8Ze60kD345FyRmkkxLxgJY3r32fSaU+1VvazlbIFKa zq4R2aospgIIxwVQOH3drqGgXpxEe54QG++3IQxmZyNCh3qocb+FnatVupHIJzX/ vvqgdswgAsXP04duqKKn =QaAX -----END PGP SIGNATURE----- From dominik at dominikbreu.de Sat May 16 08:28:42 2015 From: dominik at dominikbreu.de (Dominik Breu) Date: Sat, 16 May 2015 10:28:42 +0200 Subject: Core dump at imap process Message-ID: <1431764922.26227.1.camel@dominikbreu.de> hello list, testing the 2.18 release i got following core dump. Maybe Timo you can get a look at it. greetings dominik Core Dump: root at hbs-buko:/var/vmail/hbs-buko.info/dominik.breu# gdb /usr/lib/dovecot/imap core GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/lib/dovecot/imap...Reading symbols from /usr/lib/debug//usr/lib/dovecot/imap...done. done. [New LWP 4069] Core was generated by `dovecot/imap'. Program terminated with signal SIGSEGV, Segmentation fault. #0 imap_msgpart_contains_body (msgpart=0x0) at imap-msgpart.c:311 311 imap-msgpart.c: Datei oder Verzeichnis nicht gefunden. (gdb) bt full #0 imap_msgpart_contains_body (msgpart=0x0) at imap-msgpart.c:311 No locals. #1 0x00007f4eba177054 in fetch_state_update_stats (msgpart=, ctx=0x7f4ebacfa910, ctx=0x7f4ebacfa910) at imap-fetch-body.c:169 No locals. #2 0x00007f4eba17747b in fetch_rfc822_header (ctx=0x7f4ebacfa910, mail=0x7f4ebad08a10, context=) at imap-fetch-body.c:529 msgpart = 0x0 str = #3 0x00007f4eba1754cc in imap_fetch_more_int (ctx=ctx at entry=0x7f4ebacfa910, cancel=false) at imap-fetch.c:506 h = _data_stack_cur_id = 4 state = 0x7f4ebacfa960 client = 0x7f4ebacf9bb0 count = ret = __FUNCTION__ = "imap_fetch_more_int" #4 0x00007f4eba1765ca in imap_fetch_more (ctx=0x7f4ebacfa910, cmd=cmd at entry=0x7f4ebacfa7b0) at imap-fetch.c:558 ret = __FUNCTION__ = "imap_fetch_more" #5 0x00007f4eba169864 in cmd_fetch (cmd=0x7f4ebacfa7b0) at cmd-fetch.c:286 client = ctx = 0x7f4ebacfa910 args = 0x7f4ebace2b08 next_arg = list_arg = 0x7f4ebad03b50 search_args = 0x0 qresync_args = {qresync_sample_seqset = 0x7f4ebacd71f8, qresync_sample_uidset = 0x1} messageset = 0x7f4ebace2c70 "353" send_vanished = ret = #6 0x00007f4eba173cbc in command_exec (cmd=cmd at entry=0x7f4ebacfa7b0) at imap-commands.c:167 hook = 0x7f4ebace00f0 diff = finished = #7 0x00007f4eba172b32 in client_command_input (cmd=0x7f4ebacfa7b0) at imap-client.c:828 client = 0x7f4ebacf9bb0 command = __FUNCTION__ = "client_command_input" #8 0x00007f4eba172f2d in client_handle_next_command (remove_io_r=, client=0x7f4ebacf9bb0) at imap-client.c:926 No locals. #9 client_handle_input (client=0x7f4ebacf9bb0) at imap-client.c:938 _data_stack_cur_id = 3 remove_io = false handled_commands = false client = 0x7f4ebacf9bb0 #10 0x00007f4eba1732f5 in client_input (client=0x7f4ebacf9bb0) at imap-client.c:980 cmd = 0xea880 output = 0x7f4ebacfa5d0 bytes = 2 __FUNCTION__ = "client_input" #11 0x00007f4eb99a2b6c in io_loop_call_io (io=0x7f4ebacfa6c0) at ioloop.c:501 ioloop = 0x7f4ebacdf740 t_id = 2 __FUNCTION__ = "io_loop_call_io" ---Type to continue, or q to quit--- #12 0x00007f4eb99a3ba5 in io_loop_handler_run_internal (ioloop=ioloop at entry=0x7f4ebacdf740) at ioloop-epoll.c:220 ctx = 0x7f4ebace0460 io = tv = {tv_sec = 1799, tv_usec = 999300} events_count = msecs = ret = 1 i = 0 j = call = __FUNCTION__ = "io_loop_handler_run_internal" #13 0x00007f4eb99a2bd9 in io_loop_handler_run (ioloop=ioloop at entry=0x7f4ebacdf740) at ioloop.c:548 No locals. #14 0x00007f4eb99a2c58 in io_loop_run (ioloop=0x7f4ebacdf740) at ioloop.c:525 __FUNCTION__ = "io_loop_run" #15 0x00007f4eb9941243 in master_service_run (service=0x7f4ebacdf5d0, callback=) at master-service.c:581 No locals. #16 0x00007f4eba16696c in main (argc=1, argv=0x7f4ebacdf390) at main.c:431 set_roots = {0x7f4eba38b940 , 0x0} login_set = {auth_socket_path = 0x7f4ebacd7048 "\001", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x7f4eba17ccc0 , failure_callback = 0x7f4eba17c930 , request_auth_token = 1} service_flags = storage_service_flags = username = 0x0 c = From tss at iki.fi Sat May 16 08:48:09 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 May 2015 11:48:09 +0300 Subject: Core dump at imap process In-Reply-To: <1431764922.26227.1.camel@dominikbreu.de> References: <1431764922.26227.1.camel@dominikbreu.de> Message-ID: <730AF8C5-8465-4787-8F37-B4EB8A738472@iki.fi> On 16 May 2015, at 11:28, Dominik Breu wrote: > > hello list, > > testing the 2.18 release i got following core dump. Maybe Timo you can > get a look at it. That's not v2.2.18 release though, it's the hg version after v2.2.18 release. > Program terminated with signal SIGSEGV, Segmentation fault. > #0 imap_msgpart_contains_body (msgpart=0x0) at imap-msgpart.c:311 > No locals. > #1 0x00007f4eba177054 in fetch_state_update_stats (msgpart= out>, ctx=0x7f4ebacfa910, ctx=0x7f4ebacfa910) at imap-fetch-body.c:169 Fixed: http://hg.dovecot.org/dovecot-2.2/rev/1044c55fb4ef From racuk12 at gmail.com Sat May 16 09:10:08 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Sat, 16 May 2015 10:10:08 +0100 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required Message-ID: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit thanks Robert From tss at iki.fi Sat May 16 09:32:29 2015 From: tss at iki.fi (Timo Sirainen) Date: Sat, 16 May 2015 12:32:29 +0300 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: <5556E62B.20306@delphij.net> References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> Message-ID: <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> On 16 May 2015, at 09:39, Xin Li wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Hi, > > On 5/15/15 10:58, Xin Li wrote: >> Hi, >> >> I have seen the following assertion fails on my own mail server >> (indexer-worker): >> >> Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion >> failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) > > I've managed to narrow down the issue to one mail, and can reliably > trigger the assertion with 'doveadm index -u user INBOX' with only that > email. Can you send me some mail that triggers the crash? I'm having trouble reproducing. > Maybe we should reset the iconv descriptor after each full conversion? > i.e.: > > diff -r 91bbd6753689 src/lib-mail/message-decoder.c > - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 > +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 > @@ -347,6 +347,8 @@ > sizeof(ctx->translation_buf)); > memcpy(ctx->translation_buf, data + pos, > ctx->translation_size); > + } else { > + charset_to_utf8_reset(ctx->charset_trans); > } I think the fix probably belongs to charset-iconv.c From dominik at dominikbreu.de Sat May 16 09:37:10 2015 From: dominik at dominikbreu.de (Dominik Breu) Date: Sat, 16 May 2015 11:37:10 +0200 Subject: Core dump at imap process In-Reply-To: <730AF8C5-8465-4787-8F37-B4EB8A738472@iki.fi> References: <1431764922.26227.1.camel@dominikbreu.de> <730AF8C5-8465-4787-8F37-B4EB8A738472@iki.fi> Message-ID: Hey Timo, You are correct I meant the 2.2.18 version. Thx for the fix. Greetings Dominik Am 16. Mai 2015 10:48:09 MESZ, schrieb Timo Sirainen : >On 16 May 2015, at 11:28, Dominik Breu wrote: >> >> hello list, >> >> testing the 2.18 release i got following core dump. Maybe Timo you >can >> get a look at it. > >That's not v2.2.18 release though, it's the hg version after v2.2.18 >release. > >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 imap_msgpart_contains_body (msgpart=0x0) at imap-msgpart.c:311 >> No locals. >> #1 0x00007f4eba177054 in fetch_state_update_stats >(msgpart=> out>, ctx=0x7f4ebacfa910, ctx=0x7f4ebacfa910) at >imap-fetch-body.c:169 > >Fixed: http://hg.dovecot.org/dovecot-2.2/rev/1044c55fb4ef -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. From darix at opensu.se Sat May 16 11:31:41 2015 From: darix at opensu.se (Marcus Rueckert) Date: Sat, 16 May 2015 13:31:41 +0200 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required In-Reply-To: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> References: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> Message-ID: <20150516113141.GC3466@nordisch.org> On 2015-05-16 10:10:08 +0100, Robert Chalmers wrote: > I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. > > > master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit Increase the ulimit before starting dovecot. the how depends on your distro. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From darix at opensu.se Sat May 16 11:33:04 2015 From: darix at opensu.se (Marcus Rueckert) Date: Sat, 16 May 2015 13:33:04 +0200 Subject: fs_api_module_register In-Reply-To: <5556BEB9.9010709@whitehorsetc.com> References: <5556BEB9.9010709@whitehorsetc.com> Message-ID: <20150516113303.GD3466@nordisch.org> On 2015-05-15 21:51:21 -0600, Eric Broch wrote: > I'm trying to build a binary rpm from dovecot sources 2.2.18 and > pigeonhole.0.4.8. During the make portion I'm getting the following error: > > make[3]: Leaving directory > `/root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage' > for bin in test-mail-search-args-imap test-mail-search-args-simplify > test-mailbox-get; do \ > if ! /bin/sh ../../run-test.sh ../.. ./$bin; then exit 1; fi; \ > done > /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: > symbol lookup error: > /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: > undefined symbol: fs_api_module_register > > I created a binary rpm from dovecot sources 2.2.16 and pigeonhole > sources 0.4.7 and all went well. > > Can anyone give me any hints as to the problem? I built rpms for this just fine. you would need to share more details. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From racuk12 at gmail.com Sat May 16 11:55:31 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Sat, 16 May 2015 12:55:31 +0100 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required In-Reply-To: <20150516113141.GC3466@nordisch.org> References: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> <20150516113141.GC3466@nordisch.org> Message-ID: That's my next question. 'The how' I'm on OSX 10.10.3, latest Dovecot. I need to either increase limit -n globally, or for Dovect at boot time. ulimit -n 4096 for example. I can set it in /etc/profile, but that seems like it my be a brute force method? Robert Sent from my iPad > On 16 May 2015, at 12:31 pm, Marcus Rueckert wrote: > >> On 2015-05-16 10:10:08 +0100, Robert Chalmers wrote: >> I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. >> >> >> master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit > > Increase the ulimit before starting dovecot. the how depends on your > distro. > > darix > > -- > openSUSE - SUSE Linux is my linux > openSUSE is good for you > www.opensuse.org From edgar at pettijohn-web.com Sat May 16 12:54:51 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn III) Date: Sat, 16 May 2015 07:54:51 -0500 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required In-Reply-To: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> References: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> Message-ID: On May 16, 2015, at 4:10 AM, Robert Chalmers wrote: > I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. > > > master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit > > thanks > Robert It may be easier for you to just adjust it in: dovecot/conf.d/10-master.conf default_client_limit = 500 (to start with, you may need to adjust further) From listas at adminlinux.com.br Sat May 16 13:06:36 2015 From: listas at adminlinux.com.br (listas at adminlinux.com.br) Date: Sat, 16 May 2015 10:06:36 -0300 Subject: Couldn't create lock .dovecot-sync.lock Message-ID: <555740DC.7090409@adminlinux.com.br> Hi, In a cluster with two servers and replication via dovecot-dsync, this error is logged: server1 dovecot: dsync-server(): Error: Couldn't create lock /var/lib/imap/user/6a//.dovecot-sync.lock: No such file or directory This is because "/var/lib/imap/user/6a//" doesn't exist in server1. In another cluster node, the directory exists and everything seems to work fine. In my view, this indicates that the first sync is failing. In the first user access Dovecot creates home_dir for him. But the sync is not happening at this time to create the home_dir on the other node. My confs with doveconf -n: http://www.adminlinux.com.br/newcluster_doveconf.txt Thanks in advance! -- Thiago Henrique From racuk12 at gmail.com Sat May 16 13:22:56 2015 From: racuk12 at gmail.com (Robert Chalmers) Date: Sat, 16 May 2015 14:22:56 +0100 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required In-Reply-To: References: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> Message-ID: Yes,tried a range from 100 to 1000, but the warning still happens, with varying outputs saying its either over or under. I'm looung for a way to set the limit when dovecot starts, or as part of the startup process actually. ----- From my iPhone. > On 16 May 2015, at 1:54 pm, Edgar Pettijohn III wrote: > > >> On May 16, 2015, at 4:10 AM, Robert Chalmers wrote: >> >> I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. >> >> >> master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit >> >> thanks >> Robert > > It may be easier for you to just adjust it in: > > dovecot/conf.d/10-master.conf > default_client_limit = 500 (to start with, you may need to adjust further) From ebroch at whitehorsetc.com Sat May 16 17:58:09 2015 From: ebroch at whitehorsetc.com (Eric Broch) Date: Sat, 16 May 2015 11:58:09 -0600 Subject: fs_api_module_register In-Reply-To: <20150516113303.GD3466@nordisch.org> References: <5556BEB9.9010709@whitehorsetc.com> <20150516113303.GD3466@nordisch.org> Message-ID: <55578531.5080107@whitehorsetc.com> I'm building on CentOS 5 (x86) and CentOS 6 (x86_64). The rpmbuild failed on both with the same error. Is there any other information you would need from me? I'm not sure if I'm missing some dependency, or not. May I trouble you for the dovecot.spec file you used to build your RPM? On 5/16/2015 5:33 AM, Marcus Rueckert wrote: > On 2015-05-15 21:51:21 -0600, Eric Broch wrote: >> I'm trying to build a binary rpm from dovecot sources 2.2.18 and >> pigeonhole.0.4.8. During the make portion I'm getting the following error: >> >> make[3]: Leaving directory >> `/root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage' >> for bin in test-mail-search-args-imap test-mail-search-args-simplify >> test-mailbox-get; do \ >> if ! /bin/sh ../../run-test.sh ../.. ./$bin; then exit 1; fi; \ >> done >> /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: >> symbol lookup error: >> /root/rpmbuild/BUILD/dovecot-2.2.18/src/lib-storage/.libs/lt-test-mail-search-args-imap: >> undefined symbol: fs_api_module_register >> >> I created a binary rpm from dovecot sources 2.2.16 and pigeonhole >> sources 0.4.7 and all went well. >> >> Can anyone give me any hints as to the problem? > I built rpms for this just fine. you would need to share more details. > > darix > From edgar at pettijohn-web.com Sat May 16 18:04:17 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn III) Date: Sat, 16 May 2015 13:04:17 -0500 Subject: How do I fix this: master: Warning: fd limit (ulimit -n) is lower than required In-Reply-To: References: <343CE251-E4C2-4D11-9208-D9ABE731274A@gmail.com> Message-ID: <9729EC8C-67DD-4291-96DD-8A639C7B2EC6@pettijohn-web.com> On May 16, 2015, at 8:22 AM, Robert Chalmers wrote: > Yes,tried a range from 100 to 1000, but the warning still happens, with varying outputs saying its either over or under. I'm looung for a way to set the limit when dovecot starts, or as part of the startup process actually. > > > ----- > From my iPhone. > > >> On 16 May 2015, at 1:54 pm, Edgar Pettijohn III wrote: >> >> >>> On May 16, 2015, at 4:10 AM, Robert Chalmers wrote: >>> >>> I get this when I reload dovecot after any changes. How can I fix it? I can?t find any reference to it anywhere. >>> >>> >>> master: Warning: fd limit (ulimit -n) is lower than required under max. load (256 < 1000), because of default_client_limit >>> >>> thanks >>> Robert >> >> It may be easier for you to just adjust it in: >> >> dovecot/conf.d/10-master.conf >> default_client_limit = 500 (to start with, you may need to adjust further) look at man pages: launchctl(1) launchd(8) launchd.conf(5) From hanns at hannsmattes.de Sun May 17 15:29:54 2015 From: hanns at hannsmattes.de (Hanns Mattes) Date: Sun, 17 May 2015 17:29:54 +0200 Subject: fts_lucene error Message-ID: <87382vmdcc.fsf@hannsmattes.de> Hi, after upgrading to 2.2.18 I get the following error: lmtp(root at bruecko.de): Error: Couldn't load required plugin /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: dlopen() failed: /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: undefined symbol: _ZN6lucene8analysis8snowball16SnowballAnalyzerC1EPFiPKvmP6bufferEPKc Running Opensuse 13.2 I'm using the RPMs from the server-Mail Repo. So I'm not sure, if the error is related to this builds, my system or something else. Any hints? Regards Hanns From darix at opensu.se Sun May 17 15:45:12 2015 From: darix at opensu.se (Marcus Rueckert) Date: Sun, 17 May 2015 17:45:12 +0200 Subject: fts_lucene error In-Reply-To: <87382vmdcc.fsf@hannsmattes.de> References: <87382vmdcc.fsf@hannsmattes.de> Message-ID: <20150517174512.3b05bf4a@pixie.lan> On Sun, 17 May 2015 17:29:54 +0200 Hanns Mattes wrote: > lmtp(root at bruecko.de): Error: Couldn't load required > plugin /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: > dlopen() > failed: /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: > undefined symbol: > _ZN6lucene8analysis8snowball16SnowballAnalyzerC1EPFiPKvmP6bufferEPKc I asked Timo about it already on irc but no answer yet. could be related to enabling textcat support, which might have indirectly turn on some libstemmer code. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From ebroch at whitehorsetc.com Sun May 17 15:55:04 2015 From: ebroch at whitehorsetc.com (Eric Broch) Date: Sun, 17 May 2015 09:55:04 -0600 Subject: fs_api_module_register In-Reply-To: <5558AC6F.7040206@whitehorsetc.com> References: <5558AC6F.7040206@whitehorsetc.com> Message-ID: <5558B9D8.7010702@whitehorsetc.com> Marcus, Thanks for the spec file. I'm fairly new to creating RPM's and I just found out that the issue is taking place during the 'make check' portion of the spec file. I'm not sure what 'make check' does after having googled it, although, I think it checks for run-time errors. Any hints? Eric On 5/16/2015 12:52 PM, Marcus Rueckert wrote: > On 2015-05-16 11:52:55 -0600, Eric Broch wrote: >> I'm building on CentOS 5 (x86) and CentOS 6 (x86_64). The rpmbuild >> failed on both with the same error. Is there any other information you >> would need from me? I'm not sure if I'm missing some dependency, or not. >> May I trouble you for the dovecot.spec file you used to build your RPM? > https://build.opensuse.org/package/show/server:mail/dovecot22 > > sles 11 should be close to centos 6. > > darix > From hanns at hannsmattes.de Sun May 17 16:46:33 2015 From: hanns at hannsmattes.de (Hanns Mattes) Date: Sun, 17 May 2015 18:46:33 +0200 Subject: fts_lucene error In-Reply-To: <20150517174512.3b05bf4a@pixie.lan> (Marcus Rueckert's message of "Sun, 17 May 2015 17:45:12 +0200") References: <87382vmdcc.fsf@hannsmattes.de> <20150517174512.3b05bf4a@pixie.lan> Message-ID: <87twvbkv86.fsf@hannsmattes.de> Hi, Darix, Marcus Rueckert schrieb: >On Sun, 17 May 2015 17:29:54 +0200 >Hanns Mattes wrote: >> [lucene-fts error] >I asked Timo about it already on irc but no answer yet. could be >related to enabling textcat support, which might have indirectly turn >on some libstemmer code. Ah, good to know. I've switched to fts_squat and will wait for news. Thanx Hanns From david at davidwbrown.name Sun May 17 22:35:34 2015 From: david at davidwbrown.name (david at davidwbrown.name) Date: Sun, 17 May 2015 22:35:34 +0000 (UTC) Subject: Googling: dovecot: auth-worker(default): pam_start() failed: Critical error - immediate abort Message-ID: <1431902134.8371@davidwbrown.name> Howdy, Googling: dovecot: auth-worker(default): pam_start() failed: Critical error - immediate abort dobbeltganger dovecot: auth-worker(default): pam(user,1.2.3.4): lookup service=dovecot dobbeltganger out of memory [4543] returns just 3 references. Examining memory server side reveals use of virtual memory but not 100% OS: Ubuntu 14.04 client 10.04 server-side. Email client: Thunderbird (Thunderbird prompts for the password which has worked for years). dovecot -n # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.34.1-rscloud x86_64 Ubuntu 10.04.4 LTS log_timestamp: %Y-%m-%d %H:%M:%S ssl: required verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-logina login_processes_count: 5 login_max_processes_count: 256 mail_max_userip_connections: 40 mail_privileged_group: mail mail_location: maildir:~/Maildir mail_debug: yes mbox_write_locks: fcntl dotlock auth default: realms: davidwbrown.name, karlbrown.name default_realm: dobbeltganger.com username_format: %n verbose: yes debug: yes debug_passwords: yes passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix From nick+dovecot at bunbun.be Mon May 18 13:11:42 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Mon, 18 May 2015 15:11:42 +0200 Subject: fts_lucene error In-Reply-To: <20150517174512.3b05bf4a@pixie.lan> References: <87382vmdcc.fsf@hannsmattes.de> <20150517174512.3b05bf4a@pixie.lan> Message-ID: <3e2ae1b370634ac3a86e2a667251538b@bunbun.be> On 2015-05-17 17:45, Marcus Rueckert wrote: > On Sun, 17 May 2015 17:29:54 +0200 > Hanns Mattes wrote: >> lmtp(root at bruecko.de): Error: Couldn't load required >> plugin /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: >> dlopen() >> failed: /usr/lib64/dovecot/modules/lib21_fts_lucene_plugin.so: >> undefined symbol: >> _ZN6lucene8analysis8snowball16SnowballAnalyzerC1EPFiPKvmP6bufferEPKc > > I asked Timo about it already on irc but no answer yet. could be > related to enabling textcat support, which might have indirectly turn > on some libstemmer code. I can confirm: got the same error when enabling textcat support on the latest FreeBSD-port of dovecot. After disabling it, FTS lucene worked again. Rgds, N. From dovecot at outputservices.com Mon May 18 15:11:20 2015 From: dovecot at outputservices.com (dovecot at outputservices.com) Date: Mon, 18 May 2015 09:11:20 -0600 (MDT) Subject: v2.2.18 released Message-ID: <201505181511.t4IFBKk05050@outputservices.com> I am just starting out and trying to get dovecot and postfix working in Solaris 10 environment. Went from 2.2.15 to 2.2.17, then to 2.2.18 Both version 2.2.17 & 2.2.18 gave this same error. 2.2.15 does not. ----------------------------------- May 15 09:27:37 master: Info: Dovecot v2.2.18 starting up for imap May 15 09:27:37 ssl-params: Info: Generating SSL parameters May 15 09:27:37 ssl-params: Fatal: Couldn't load required plugin /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: dlopen() failed: ld.so.1: ssl-params: fatal: relocation error: file /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: symbol SSL_clear_options: referenced symbol not found May 15 09:27:37 ssl-params: Error: child process failed with status 22784 May 15 09:27:46 master: Warning: Killed with signal 15 (by pid=1848 uid=0 code=kill) ------------------------------------------------ doveconf -n # 2.2.18: /tmp/tocevod/etc/dovecot/dovecot.conf # OS: SunOS 5.10 i86pc log_path = /tmp/tocevod/log/log login_trusted_networks = 137.106.0.0/16 mail_location = mbox:~/posta:INBOX=/var/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /dovecot/etc/dovecot/conf.d/ldap.settings driver = ldap } protocols = imap service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } } ssl_cert = References: <87382vmdcc.fsf@hannsmattes.de> <20150517174512.3b05bf4a@pixie.lan> <87twvbkv86.fsf@hannsmattes.de> Message-ID: <20150518154610.GC16619@nordisch.org> On 2015-05-17 18:46:33 +0200, Hanns Mattes wrote: > >On Sun, 17 May 2015 17:29:54 +0200 > >Hanns Mattes wrote: > >> [lucene-fts error] > >I asked Timo about it already on irc but no answer yet. could be > >related to enabling textcat support, which might have indirectly turn > >on some libstemmer code. > > Ah, good to know. I've switched to fts_squat and will wait for news. disabled textcat support for now. package submitted. might take a few minutes to be build. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From listas at adminlinux.com.br Mon May 18 20:32:08 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Mon, 18 May 2015 17:32:08 -0300 Subject: Failure at run doveadm deduplicate Message-ID: <555A4C48.1000102@adminlinux.com.br> Hi, In my server, accounts are receiving many doubled emails in its mailbox. In some accounts the "doveadm deduplicate" is doing nothing, even if there are duplicate messages. But in some cases, this error has left me particularly concerned: # doveadm deduplicate -u -m ALL doveadm(): Panic: file mail-transaction-log.c: line 271 (mail_transaction_log_rotate): assertion failed: (file->locked) doveadm(): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x67f30) [0x7f77b9f85f30] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f77b9f85f9a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f77b9f39515] -> /usr/lib/dovecot/libdovecot-storage.so.0(+0xc296f) [0x7f77ba2b996f] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_write+0x1be) [0x7f77ba2b8c3e] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x262) [0x7f77ba2b1012] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_map_atomic_finish+0x26) [0x7f77ba225a36] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_sync+0x61) [0x7f77ba229d91] -> /usr/lib/dovecot/libdovecot-storage.so.0(mdbox_storage_sync_init+0x66) [0x7f77ba229e26] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x2b) [0x7f77ba26954b] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f77ba269657] -> doveadm(+0x1b5f4) [0x7f77bab995f4] -> doveadm(+0x1970b) [0x7f77bab9770b] -> doveadm(doveadm_mail_try_run+0x26c) [0x7f77bab9845c] -> doveadm(main+0x414) [0x7f77bab946a4] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f77b9b7aec5] -> doveadm(+0x168f7) [0x7f77bab948f7] I'm using dovecot-2.2.15. Is this known? What other kind of information could be useful to help find this issue? Full log on the link: http://pastebin.com/CMfjxydy My confs with doveconf -n : http://www.adminlinux.com.br/newcluster_doveconf.txt Thanks Thiago Henrique From listas at adminlinux.com.br Mon May 18 20:59:05 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Mon, 18 May 2015 17:59:05 -0300 Subject: Should I use "dsync_remote_cmd" with dsync via tcp? Message-ID: <555A5299.9020403@adminlinux.com.br> Hi, By the Documentation the directive "dsync_remote_cmd" seems to me required only to dsync via ssh. Is this right? I have set dsync via tcp. Should I have set "dsync_remote_cmd" ? Thanks! Thiago Henrique From gjn at gjn.priv.at Mon May 18 21:06:07 2015 From: gjn at gjn.priv.at (=?ISO-8859-1?Q?G=FCnther_J=2E?= Niederwimmer) Date: Mon, 18 May 2015 23:06:07 +0200 Subject: sssd and dovecot Message-ID: <2980387.RkkmM20aQS@techz> Hello, I like to install dovecot on a centos 7.1 with freeIPA all information for dovecot is from sssd. but I found no way to configure this correct. Have any a link, or can help me please. Thanks for a answer. -- mit freundlichen Gr?ssen / best regards, G?nther J. Niederwimmer From darix at opensu.se Mon May 18 21:38:24 2015 From: darix at opensu.se (Marcus Rueckert) Date: Mon, 18 May 2015 23:38:24 +0200 Subject: sssd and dovecot In-Reply-To: <2980387.RkkmM20aQS@techz> References: <2980387.RkkmM20aQS@techz> Message-ID: <20150518213824.GD16619@nordisch.org> On 2015-05-18 23:06:07 +0200, G?nther J. Niederwimmer wrote: > I like to install dovecot on a centos 7.1 with freeIPA > > all information for dovecot is from sssd. > > but I found no way to configure this correct. > > Have any a link, or can help me please. just an assumption: If you want to use SSSD you would use "system" accounts via nss/pam. The other option is talk directly do the ldap server, that is behind sssd. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From fgmcoelho at yahoo.com.br Mon May 18 23:40:47 2015 From: fgmcoelho at yahoo.com.br (Fernando Coelho) Date: Mon, 18 May 2015 23:40:47 +0000 (UTC) Subject: Imap process timeout to quota proxy on logout after async commit Message-ID: <770463928.1805287.1431992447920.JavaMail.yahoo@mail.yahoo.com> Hi there, I am having some trouble implementing a quota proxy server for a memcached based dict (I am using the version 2.2.15). I created a multi-threaded daemon that is using non-blocking sockets to receive the data and reply to dovecot. Right now everything but the atomic increase message is working great. Here is an example of such a message: B1 A1??? priv/quota/storage??? 86583 A1??? priv/quota/messages??? 1 D1 The program process it and I replies:OA1 Everything keeps working as expects until the user sends the imap a logout command.? The only problem is that for some reason dovecot tries to read data from server without sending anything, and after 30 seconds it exits with an timeout error. The following message is logged: May 18 22:46:00 imap[11654]: user 29908633#perm!terra (29908633#perm!terra) from 127.0.0.1: Error: read(/tmp/quotatest) failed: Timeout after 30 seconds May 18 22:46:00 imap[11654]: user 29908633#perm!terra (29908633#perm!terra) from 127.0.0.1: Info: Disconnected: Logged out in=40 out=827 May 18 22:46:00 imap[11654]: user 29908633#perm!terra (29908633#perm!terra) from 127.0.0.1: Error: dict quota: Quota update failed, it's now desynced The really strange part is that it only fails if and atomic increase was sent and only when the user logs out. I use the same pattern ("OA\n") to reply for other commits (such as when there is an unset followed by a set) and there is no issue. Is there anything that I am missing? Thanks in advance. From ebroch at whitehorsetc.com Tue May 19 02:16:20 2015 From: ebroch at whitehorsetc.com (Eric Broch) Date: Mon, 18 May 2015 20:16:20 -0600 Subject: fs_api_module_register In-Reply-To: <5558B9D8.7010702@whitehorsetc.com> References: <5558AC6F.7040206@whitehorsetc.com> <5558B9D8.7010702@whitehorsetc.com> Message-ID: <555A9CF4.8010107@whitehorsetc.com> I found the reason for this build error. It has to do with the rpath setting. Under Fedora packaging guidelines the following settings are changed in the spec file under %configure for libtool: %configure sed -i 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' libtool sed -i 's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' libtool commenting them out solved the problem, but I'm not sure that this is the best, or the appropriate, solution...yet. Anyway, thanks for providing the SUSE spec file, Marcus. EricB On 5/17/2015 9:55 AM, Eric Broch wrote: > > > > > Marcus, > > Thanks for the spec file. > > I'm fairly new to creating RPM's and I just found out that the issue is > taking place during the 'make check' portion of the spec file. I'm not > sure what 'make check' does after having googled it, although, I think > it checks for run-time errors. Any hints? > > Eric > > On 5/16/2015 12:52 PM, Marcus Rueckert wrote: >> On 2015-05-16 11:52:55 -0600, Eric Broch wrote: >>> I'm building on CentOS 5 (x86) and CentOS 6 (x86_64). The rpmbuild >>> failed on both with the same error. Is there any other information you >>> would need from me? I'm not sure if I'm missing some dependency, or not. >>> May I trouble you for the dovecot.spec file you used to build your RPM? >> https://build.opensuse.org/package/show/server:mail/dovecot22 >> >> sles 11 should be close to centos 6. >> >> darix >> From nick+dovecot at bunbun.be Tue May 19 09:44:04 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Tue, 19 May 2015 11:44:04 +0200 Subject: doveadm index panic Message-ID: <6cc537e45deb46e6e1a2977e244e1216@bunbun.be> Hi, running dovecot 2.2.18. when I try to re-index my INBOX I get following error: # doveadm index -u user at domain.com INBOX doveadm(user at domain.com): Warning: I/O leak: 0x80101ada0 (line 120, fd 15) doveadm(user at domain.com): Warning: Timeout leak: 0x801019fe0 (line 325) doveadm(user at domain.com): Panic: file ioloop.c: line 39 (io_add_file): assertion failed: (callback != NULL) zsh: abort doveadm index -u user at domain.com INBOX Any ideas? Rgds, N. From nick+dovecot at bunbun.be Tue May 19 11:12:21 2015 From: nick+dovecot at bunbun.be (Nick Rosier) Date: Tue, 19 May 2015 13:12:21 +0200 Subject: doveadm index panic In-Reply-To: <6cc537e45deb46e6e1a2977e244e1216@bunbun.be> References: <6cc537e45deb46e6e1a2977e244e1216@bunbun.be> Message-ID: <7650810267c32fc7e723220aa5bda2fd@bunbun.be> On 2015-05-19 11:44, Nick Rosier wrote: > Hi, > > running dovecot 2.2.18. when I try to re-index my INBOX I get following > error: > > # doveadm index -u user at domain.com INBOX > doveadm(user at domain.com): Warning: I/O leak: 0x80101ada0 (line 120, fd > 15) > doveadm(user at domain.com): Warning: Timeout leak: 0x801019fe0 (line 325) > doveadm(user at domain.com): Panic: file ioloop.c: line 39 (io_add_file): > assertion failed: (callback != NULL) > zsh: abort doveadm index -u user at domain.com INBOX Doesn't seem to be limited to running the indexer manually: 2015-05-19T13:08:21.825235+02:00 mail dovecot: indexer-worker(user at domain.com): Warning: I/O leak: 0x10b80da0 (line 120, fd 20) 2015-05-19T13:08:21.825253+02:00 mail dovecot: indexer-worker(user at domain.com): Warning: Timeout leak: 0x10b7ffe0 (line 325) 2015-05-19T13:08:22.853198+02:00 mail dovecot: indexer-worker(user at domain.com): Panic: file ioloop.c: line 39 (io_add_file): assertion failed: (callback != NULL) 2015-05-19T13:08:22.853325+02:00 mail dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for user at domain.com 2015-05-19T13:08:22.853367+02:00 mail dovecot: imap(user at domain.com): Error: indexer failed to index mailbox INBOX 2015-05-19T13:08:22.854205+02:00 mail dovecot: indexer-worker(user at domain.com): Fatal: master: service(indexer-worker): child 42599 killed with signal 6 (core not dumped - set service indexer-worker { drop_priv_before_exec=yes }) Never seen this before. Don't know if it's related but I just switched from fts_lucene to fts_solr. Rgds, N. From compte-dovecot at ingescom.com Tue May 19 11:23:33 2015 From: compte-dovecot at ingescom.com (Samuel) Date: Tue, 19 May 2015 13:23:33 +0200 Subject: Can't achieve alias working with SQL Message-ID: <555B1D35.4050801@ingescom.com> Hello, On a fresh install of Jessie, I achieve to make dovecot/postfix working with SQL, but I still can't send email to alias for wich one I get : "Unknown user" After debug enable in dovecot, I see that the user_query SQL , just query for the "users" table and not the "forward" table so the final alias user is not known. But when I check postfix for alias, it seems ok : root at cluster-dovecot:/etc/dovecot# postmap -q contact2 at XXXXXXXX.com mysql:alias_maps contact at XXXXXXXXXX.com So what do I missed for alias working with dovecot ? Do I need a special conf in master.conf for sending alias AND final recipient or do I have to change the dovecot user_query for taking in acount the "forward" table in the SQL query ? .... or other thing I could miss ... Thanks a lot ... Samuel. DOVECOT : root at cluster-dovecot:/etc/dovecot# dovecot --version 2.2.13 root at cluster-dovecot:/etc/dovecot# dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.0 auth_debug_passwords = yes auth_mechanisms = plain login dict { quota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf quota_domain = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf } lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = quota fts fts_solr acl zlib mail_log notify managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { quota = dict:user::proxy::quota quota2 = dict:domain:%d:proxy::quota_domain quota_grace = 1M quota_warning = storage=100%% quota-warning 100 %u quota_warning2 = storage=95%% quota-warning 95 %u quota_warning3 = storage=80%% quota-warning 80 %u quota_warning4 = -storage=100%% quota-warning below %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-master { mode = 0660 user = vmail } unix_listener auth-userdb { mode = 0660 user = vmail } } service dict { unix_listener dict { mode = 0666 user = dovecot } } service imap-login { inet_listener imap { address = * port = 143 } inet_listener imaps { address = * port = 993 } process_limit = 256 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 1024 } service quota-status { executable = quota-status -p postfix inet_listener { port = 12340 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = vmail } user = vmail } ssl_cert = References: <555B1D35.4050801@ingescom.com> <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> Message-ID: <555B33F9.4080008@ingescom.com> Hi, Le 19/05/2015 14:54, Jorge Bastos a ?crit : > Hi Samuel, > >> On a fresh install of Jessie, I achieve to make dovecot/postfix working >> with SQL, but I still can't send email to alias for wich one I get : >> "Unknown user" >> >> After debug enable in dovecot, I see that the user_query SQL , just >> query for the "users" table and not the "forward" table so the final >> alias user is not known. >> >> But when I check postfix for alias, it seems ok : >> >> root at cluster-dovecot:/etc/dovecot# postmap -q contact2 at XXXXXXXX.com >> mysql:alias_maps contact at XXXXXXXXXX.com >> >> So what do I missed for alias working with dovecot ? >> > I had similar question on how to make it working, and it ended up being: > > virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf Finally I achieved making alias to work by changing the user_query: #FROM users WHERE username = '%n' AND domain = '%d' TO : FROM users WHERE username = '%n' AND domain = '%d' or email=(select destination from forwards where source='%u'); But I don't know if that's the right way. Wanting for any suggestion .... Samuel. From compte-dovecot at ingescom.com Tue May 19 13:05:20 2015 From: compte-dovecot at ingescom.com (Samuel) Date: Tue, 19 May 2015 15:05:20 +0200 Subject: Can't achieve alias working with SQL In-Reply-To: <555B33F9.4080008@ingescom.com> References: <555B1D35.4050801@ingescom.com> <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> <555B33F9.4080008@ingescom.com> Message-ID: <555B3510.2040302@ingescom.com> > Wanting for any suggestion .... Should read : Waiting .... From jcea at jcea.es Tue May 19 13:19:01 2015 From: jcea at jcea.es (Jesus Cea) Date: Tue, 19 May 2015 15:19:01 +0200 Subject: "doveadm backup" doesn't work anymore after upgrading to 2.2.18 Message-ID: <555B3845.2080304@jcea.es> Until today I could do this to backup my primary IMAP4 server: """ doveadm backup ssh csi doveadm dsync-server """ It doesn't work anymore after upgrading to Dovecot 2.2.18: """ jcea at ubuntu:~$ doveadm backup ssh csi doveadm dsync-server Enter passphrase for key '/home/jcea/.ssh/id_rsa': dsync-remote(root): Error: Mailbox INBOX: Failed to get attribute vendor/vendor.dovecot/pvt/server/sieve/default: Mailbox attributes not enabled dsync-local(jcea): Error: Remote command returned error 65: ssh csi doveadm dsync-server dsync-server """ The first error is regular business: I don't have SIEVE in the backup, since it is a backup. I have seen this error forever, no consequences. Now in 2.2.18 I get the second error and the backup is aborted. It doesn't work anymore. I don't find any reference to error 65 anywhere. -- Jes?s Cea Avi?n _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From cma at cmadams.net Tue May 19 14:56:34 2015 From: cma at cmadams.net (Chris Adams) Date: Tue, 19 May 2015 09:56:34 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150515185024.GE20600@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> Message-ID: <20150519145634.GA23038@cmadams.net> Once upon a time, Chris Adams said: > In the several cases I looked at, the Spam folder exists, is > getting most spam-flagged messages filed into it, it just seems to be > something about some messages (for example, got a bunch of copies of > this particular spam to different users and they all went to INBOX > instead of Spam). I can confirm that a message with multiple Subject: and multiple From: headers does not get filed correctly into the Spam folder. The sieve-test tools shows the correct action, but when the message comes in via LMTP, it goes into INBOX. -- Chris Adams From listas at adminlinux.com.br Tue May 19 16:51:48 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Tue, 19 May 2015 13:51:48 -0300 Subject: Understanding the "doveadm replicator status" output Message-ID: <555B6A24.2090202@adminlinux.com.br> Where can I get more detailed information about the output of the 'doveadm replicator status' ? man doveadm-replicator isn't enough. For example, Total number of known users = 29409, but the server has only 17667 users. This is confusing. Look: # doveadm user '*' |wc -l 17667 # doveadm replicator status Queued 'sync' requests 0 Queued 'high' requests 0 Queued 'low' requests 0 Queued 'failed' requests 0 Queued 'full resync' requests 12962 Waiting 'failed' requests 16447 Total number of known users 29409 What it means to "Queued full resync requests" ? thanks! Thiago Henrique From cma at cmadams.net Tue May 19 19:38:20 2015 From: cma at cmadams.net (Chris Adams) Date: Tue, 19 May 2015 14:38:20 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150519145634.GA23038@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> Message-ID: <20150519193820.GA26451@cmadams.net> Once upon a time, Chris Adams said: > I can confirm that a message with multiple Subject: and multiple From: > headers does not get filed correctly into the Spam folder. The > sieve-test tools shows the correct action, but when the message comes in > via LMTP, it goes into INBOX. Okay, digging some more, it looks like something in sieve is overwriting the wrong thing when it gets messages with some headers (at least From: and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin, and used this sieve script: require [ "fileinto", "variables", "vnd.dovecot.debug" ]; if header :matches "from" "*" { set "from" "${1}"; } if header :matches "subject" "*" { set "subject" "${1}"; } if header :matches "x-spam-flag" "*" { set "spam" "${1}"; } debug_log "From='${from}' Subject='${subject}' Spam='${spam}'"; if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } When I feed a message to dovecot-lda with single From: and Subject: headers and X-Spam-Flag: YES set, I get this logged: May 19 14:25:25 hosting dovecot: lda(local at example.com): sieve: default: line 6: DEBUG: From='test at cmadams.net' Subject='Test' Spam='YES' If I duplicate the From: and Subject: headers, I get: May 19 14:25:29 hosting dovecot: lda(local at example.com): sieve: default: line 6: DEBUG: From='test at cmadams.net' Subject='Test' Spam='local at example.com' So, X-Spam-Flag: is somehow getting the To: address in it? Oddly, with this debugging enabled, even though the message with the duplicate headers appears to have the wrong value for X-Spam-Flag:, it then procedes to actually match the YES somehow and file the message in the Spam folder. -- Chris Adams From cma at cmadams.net Tue May 19 20:00:50 2015 From: cma at cmadams.net (Chris Adams) Date: Tue, 19 May 2015 15:00:50 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150519193820.GA26451@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> <20150519193820.GA26451@cmadams.net> Message-ID: <20150519200050.GB26451@cmadams.net> Once upon a time, Chris Adams said: > Okay, digging some more, it looks like something in sieve is overwriting > the wrong thing when it gets messages with some headers (at least From: > and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin, > and used this sieve script: And I guess something is re-parsing them at some point? The following sieve script has the problem: # compile with "sievec /etc/dovecot/default.sieve" require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } Doubling up the test makes it work however: # compile with "sievec /etc/dovecot/default.sieve" require "fileinto"; if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } if header :contains "X-Spam-Flag" "YES" { fileinto "Spam"; } ??? -- Chris Adams From compte-dovecot at ingescom.com Tue May 19 20:08:49 2015 From: compte-dovecot at ingescom.com (Samuel) Date: Tue, 19 May 2015 22:08:49 +0200 Subject: Can't achieve alias working with SQL In-Reply-To: <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> References: <555B1D35.4050801@ingescom.com> <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> Message-ID: <555B9851.8010109@ingescom.com> Le 19/05/2015 14:54, Jorge Bastos a ?crit : > Hi Samuel, > >> On a fresh install of Jessie, I achieve to make dovecot/postfix working >> with SQL, but I still can't send email to alias for wich one I get : >> "Unknown user" >> >> After debug enable in dovecot, I see that the user_query SQL , just >> query for the "users" table and not the "forward" table so the final >> alias user is not known. >> >> But when I check postfix for alias, it seems ok : >> >> root at cluster-dovecot:/etc/dovecot# postmap -q contact2 at XXXXXXXX.com >> mysql:alias_maps contact at XXXXXXXXXX.com >> >> So what do I missed for alias working with dovecot ? >> > I had similar question on how to make it working, and it ended up being: > > virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf In fact, this part is yet in my main.conf and is working (see my postmap query above). And I don't understand why dovecot doesn't use it. So I revert back my dirty user_query and I'm still searching why I can't send email to alias .... Thanks. Samuel. From andy at thecsillags.com Tue May 19 20:27:40 2015 From: andy at thecsillags.com (andy at thecsillags.com) Date: Tue, 19 May 2015 20:27:40 +0000 Subject: Controlling shared folder rename? In-Reply-To: <8ad9f3fc7db1316397c486fba8c79961@www.thecsillags.com> References: <8ad9f3fc7db1316397c486fba8c79961@www.thecsillags.com> Message-ID: <01e02d805746d66e4d37633be8632109@www.thecsillags.com> Eventually I figured out a solution that looks like it should work; postlogin scripting described here http://wiki2.dovecot.org/PostLoginScripting .? Specifically, the sample at the bottom that adds a shared folder, obviously will need some work to make it work the way I want, but it should work. May 19 2015 3:18 PM, andy at thecsillags.com (mailto:andy at thecsillags.com) wrote: Dovecot version 2.2.16 I'm using shared folders and what I'd like to do is either have the folder that gets shared be able to be renamed by the user, or failing that, to be able to have an administrator define how the shared folder shows up to the account it's shared to. For example, right now, the INBOX owned by testuser at mydomain.com (mailto:testuser at mydomain.com) is shared to otheruser at mydomain as shows up as shared/testuser at mydomain.com (mailto:testuser at mydomain.com) because of the prefix directive in the shared namespace in 10-mail.conf.? It would be nice if, either via a dict (like the way the acls work), or via normal IMAP folder commands, that I control how it shows up to otheruser at mydomain.? So if I could do something like: ?? prefix = proxy::sharedname or similar.? Currently trying to rename from IMAP fails -- given the way my understanding of how things works is, I'm not surprised, but hey, it was worth a shot right? First, does the question make sense? Two, is there a way to do this? Thanks, :Drew ? From compte-dovecot at ingescom.com Tue May 19 21:05:04 2015 From: compte-dovecot at ingescom.com (Samuel) Date: Tue, 19 May 2015 23:05:04 +0200 Subject: Can't achieve alias working with SQL In-Reply-To: <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> References: <555B1D35.4050801@ingescom.com> <00f801d09232$f58a0e30$e09e2a90$@jorge@decimal.pt> Message-ID: <555BA580.3060809@ingescom.com> Le 19/05/2015 14:54, Jorge Bastos a ?crit : > Hi Samuel, > >> On a fresh install of Jessie, I achieve to make dovecot/postfix working >> with SQL, but I still can't send email to alias for wich one I get : >> "Unknown user" >> >> After debug enable in dovecot, I see that the user_query SQL , just >> query for the "users" table and not the "forward" table so the final >> alias user is not known. >> >> But when I check postfix for alias, it seems ok : >> >> root at cluster-dovecot:/etc/dovecot# postmap -q contact2 at XXXXXXXX.com >> mysql:alias_maps contact at XXXXXXXXXX.com >> >> So what do I missed for alias working with dovecot ? >> > I had similar question on how to make it working, and it ended up being: > > virtual_alias_maps = mysql:/etc/postfix/mysql-virtual-alias-maps.cf I have found : In fact, it was not dovecot but postfix policy fault because of check_policy_service making alias down. I'm digging on it now. Thanks. Samuel. From liuyingying5496 at 126.com Wed May 20 03:46:47 2015 From: liuyingying5496 at 126.com (=?GBK?B?wfXTqNOo?=) Date: Wed, 20 May 2015 11:46:47 +0800 (CST) Subject: what's the different between the processes dovecot-auth and dovecot -w Message-ID: <492ac164.1a03a.14d6f6e4513.Coremail.liuyingying5496@126.com> hi? what's the different between the processes dovecot-auth and dovecot -w ? What's the function on each of them ? thanks From gessel at blackrosetech.com Wed May 20 08:44:28 2015 From: gessel at blackrosetech.com (David Gessel) Date: Wed, 20 May 2015 12:44:28 +0400 Subject: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads In-Reply-To: <55390406.9070701@blackrosetech.com> References: <553247D0.1020400@blackrosetech.com> <5532557D.10000@blackrosetech.com> <3C3A4690-1FB3-4972-92E3-B2E1B554ECB6@iki.fi> <55390406.9070701@blackrosetech.com> Message-ID: <555C496C.8030809@blackrosetech.com> The update to dovecot2-2.2.18/dovecot-pigeonhole-0.4.8 seems to have made problems worse. Whole directories don't load at all - or they may load a weird segment - like mail from a few months ago back and mail from the last two days, but nothing in between. I did a # find /mail -name "dovecot.index*" -type f -delete and then on the client side (K9) things clear up, on TB a Right click on folder -> properties -> repair folder cleans things up. Of course all messages have to be redownloaded so it isn't exactly a performant option. Is there a way to completely disable caching on dovecot? I know it is sort of the raison d'?tre, but if I can't solve the problems with dovecot, I'll have to try a different, non-caching/indexing server anyway and at least it would help narrow down the issues. -------- Original Message -------- Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads From: David Gessel To: Timo Sirainen Date: Thu Apr 23 2015 18:39:02 GMT+0400 (Arabian Standard Time) > very_dirty_syncs helped a lot - the problems still happen but only about every 30 minutes. It seems this flag prevents re-indexing on whatever was constantly inducing it (with every client-check, if there was a change written to the directory, is my best guess watching the logs go). > > I still get all the same symptoms, but far less frequently. > > I'm inclined to believe, as trivial as it may be to enumerate, that: > > Something is triggering dovecot to believe the indexes need to be rebuilt. > > When checking mail during the rebuild, clients get confused by UIDs in transition. > > Today I had to quit TB and restart because duplicates filled my preview pane. Mulberry has been better, when it gets confused now, it is usually sufficient to initiate a manual check or expunge to get the list to update. > > I would think that sdbox would alleviate these issues, no? > > -------- Original Message -------- > Subject: Re: Dovecot 2.2.16: disappearing messages, mismatched summaries, duplicated messages, excessive full re-downloads > From: Timo Sirainen > To: David Gessel > Date: Sat Apr 18 2015 17:38:02 GMT+0300 (Arabic Standard Time) > >> On 18 Apr 2015, at 16:00, David Gessel wrote: >>> >>>> No. My best guess is that (your) ZFS+FreeBSD is simply not behaving the way Dovecot wants filesystems to behave (readdir() sometimes skipping over files). I don't have any other suggestion than trying a different filesystem or OS. I haven't heard ZFS having these problems before though. >>> >>> >>> Or migrating to sdbox. Absent any further suggestions, that will be my next step. My commitment to Maildir is far lower than my commitment to FreeBSD/ZFS. It is just a historical preference coming from a decade+ of courier IMAP. >> >> Actually as a workaround you could try maildir_very_dirty_syncs=yes which gets rid of most of the readdir()s. >> From bruell at simiatech.com Wed May 20 10:11:38 2015 From: bruell at simiatech.com (=?UTF-8?B?UGhpbGlwcCBCcsO8bGw=?=) Date: Wed, 20 May 2015 12:11:38 +0200 Subject: Log mail access Message-ID: <555C5DDA.5040304@simiatech.com> Hello, is it somehow possible to log the access to a mail? The log entry should be written, if a user fetches a mail via imap or pop3. It would be convenient for debugging. I've checked the Logging-Wiki and the docs for the mail_log plugin, but they seem to log any event but "mail access". Best regards, Philipp From listas at adminlinux.com.br Wed May 20 13:10:56 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Wed, 20 May 2015 10:10:56 -0300 Subject: Couldn't create lock .dovecot-sync.lock In-Reply-To: <555740DC.7090409@adminlinux.com.br> References: <555740DC.7090409@adminlinux.com.br> Message-ID: <555C87E0.7040804@adminlinux.com.br> My users are like this "username=domain_name". Could the '=' character be the reason? Despite my settings always worked: # doveconf auth_username_chars auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@= All opinions are welcome. Thanks! -- Thiago Henrique Em 16-05-2015 10:06, listas at adminlinux.com.br escreveu: > Hi, > > In a cluster with two servers and replication via dovecot-dsync, this > error is logged: > server1 dovecot: dsync-server(): Error: Couldn't create lock > /var/lib/imap/user/6a//.dovecot-sync.lock: No such file or directory > > This is because "/var/lib/imap/user/6a//" doesn't exist in > server1. In another cluster node, the directory exists and everything > seems to work fine. > > In my view, this indicates that the first sync is failing. In the first > user access Dovecot creates home_dir for him. But the sync is not > happening at this time to create the home_dir on the other node. > > My confs with doveconf -n: > http://www.adminlinux.com.br/newcluster_doveconf.txt > > Thanks in advance! > -- > Thiago Henrique From lists at tigertech.com Wed May 20 19:53:15 2015 From: lists at tigertech.com (Robert L Mathews) Date: Wed, 20 May 2015 12:53:15 -0700 Subject: "doveadm backup" doesn't work anymore after upgrading to 2.2.18 In-Reply-To: <555B3845.2080304@jcea.es> References: <555B3845.2080304@jcea.es> Message-ID: <555CE62B.3000203@tigertech.com> On 5/19/15 6:19 AM, Jesus Cea wrote: > dsync-local(jcea): Error: Remote command returned error 65: ssh csi > doveadm dsync-server dsync-server > """ > > Now in 2.2.18 I get the second error and the backup is aborted. It > doesn't work anymore. > > I don't find any reference to error 65 anywhere. Error 65 is likely the SSH process exit code. Pages like this suggest it's "Host Not Allowed To Connect": http://support.attachmate.com/techdocs/2285.html If you haven't already done so, check that running the same SSH command manually ("ssh csi" and variations) works. -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/ From stephan at rename-it.nl Wed May 20 22:35:22 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 21 May 2015 00:35:22 +0200 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150519200050.GB26451@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> <20150519193820.GA26451@cmadams.net> <20150519200050.GB26451@cmadams.net> Message-ID: <555D0C2A.2000605@rename-it.nl> On 5/19/2015 10:00 PM, Chris Adams wrote: > Once upon a time, Chris Adams said: >> Okay, digging some more, it looks like something in sieve is overwriting >> the wrong thing when it gets messages with some headers (at least From: >> and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin, >> and used this sieve script: > And I guess something is re-parsing them at some point? The following > sieve script has the problem: > > # compile with "sievec /etc/dovecot/default.sieve" > require "fileinto"; > if header :contains "X-Spam-Flag" "YES" { > fileinto "Spam"; > } > > Doubling up the test makes it work however: > > # compile with "sievec /etc/dovecot/default.sieve" > require "fileinto"; > if header :contains "X-Spam-Flag" "YES" { > fileinto "Spam"; > } > if header :contains "X-Spam-Flag" "YES" { > fileinto "Spam"; > } I will probably have time to investigate this more this weekend. BTW, you're using a pretty old version of Dovecot and Pigeonhole. Do you have the possibility to try a newer version? Regards, Stephan. From BHueck at kevag-telekom.de Thu May 21 07:31:45 2015 From: BHueck at kevag-telekom.de (=?utf-8?B?QmVuamFtaW4gSMO8Y2s=?=) Date: Thu, 21 May 2015 07:31:45 +0000 Subject: Dovecot-Director, (Manage)-Sieve und Remote-IP Message-ID: <555D89EE.2000502@kevag-telekom.de> Hello, I`ve got a question about Dovecot-Director and (Manage)-Sieve: is there any possibility to pass-through the original IP-address to the backend server? In this case I try to pass-through the IP-address of a webmail server. I`ve looked for a solution in different forums and tried it with the parameter "login_trusted_networks", however no success. The above solution is working with IMAP/POP3/LMTP, but not with Sieve. For any solutions or any hint I would be grateful. Best regards Benjamin H?ck -- ________________________________ ________________________________ KEVAG Telekom GmbH Cusanusstr. 7 D-56073 Koblenz Fon: +49 261 20162-0 Fax: +49 261 20162-25100 http://www.kevag-telekom.de/ Gesch?ftsf?hrer: Bernd Gowitzke, Gerd Thewalt Sitz der Gesellschaft: Koblenz, Amtsgericht Koblenz, HRB Nr. 5343 USt.IdNr. DE 18 77 67 843 St-Nr. 22/650/0182/7 From stephan at rename-it.nl Thu May 21 10:38:17 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 21 May 2015 12:38:17 +0200 Subject: Dovecot-Director, (Manage)-Sieve und Remote-IP In-Reply-To: <555D89EE.2000502@kevag-telekom.de> References: <555D89EE.2000502@kevag-telekom.de> Message-ID: <555DB599.6020302@rename-it.nl> Benjamin H?ck schreef op 21-5-2015 om 9:31: > Hello, > > I`ve got a question about Dovecot-Director and (Manage)-Sieve: is there > any possibility to pass-through the original IP-address to the backend > server? > > In this case I try to pass-through the IP-address of a webmail server. > > I`ve looked for a solution in different forums and tried it with the > parameter "login_trusted_networks", however no success. > > The above solution is working with IMAP/POP3/LMTP, but not with Sieve. > > For any solutions or any hint I would be grateful. Well... ManageSieve doesn't support an XCLIENT or equivalent command at the moment, because... ehmm... probably nobody ever asked for it. Given the example in the pop3 service, it should be rather trivial to implement in ManageSieve. I will look at that later this week. Regards, Stephan. From jcea at jcea.es Thu May 21 12:52:35 2015 From: jcea at jcea.es (Jesus Cea) Date: Thu, 21 May 2015 14:52:35 +0200 Subject: "doveadm backup" doesn't work anymore after upgrading to 2.2.18 In-Reply-To: <555CE62B.3000203@tigertech.com> References: <555B3845.2080304@jcea.es> <555CE62B.3000203@tigertech.com> Message-ID: <555DD513.7080502@jcea.es> On 20/05/15 21:53, Robert L Mathews wrote: > On 5/19/15 6:19 AM, Jesus Cea wrote: > >> dsync-local(jcea): Error: Remote command returned error 65: ssh csi >> doveadm dsync-server dsync-server >> """ >> >> Now in 2.2.18 I get the second error and the backup is aborted. It >> doesn't work anymore. >> >> I don't find any reference to error 65 anywhere. > > Error 65 is likely the SSH process exit code. Pages like this suggest > it's "Host Not Allowed To Connect": It is not a SSH error because to get the first error (the sieve related one) "doveadm" already connected to the remote backup server via SSH. I have installed & configured sieve in the backup server, with an empty filter set. I can now do the backup correctly. Interestingly if I disable the sieve plugin in the backup server, the backup keeps working. WTF! My guess is that sieve has set the "vendor/vendor.dovecot/pvt/server/sieve/default" attribute in the mailbox and the backup process is happy now, even if I disable sieve after the fact. I consider this issue closed but it would be nice to be documented somewhere. Thanks. -- Jes?s Cea Avi?n _/_/ _/_/_/ _/_/_/ jcea at jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From number6 at noisynotes.com Thu May 21 14:25:30 2015 From: number6 at noisynotes.com (Steve Matzura) Date: Thu, 21 May 2015 10:25:30 -0400 Subject: Can email be shared between POP3 and shell account access? Message-ID: <5iqrla9kqmskifpi8mprfmh2sq0vea17ql@4ax.com> I needed to receive a message to admin at mydomain.com. I waited and waited for it, but it never showed up in the admin mailbox on the mydomain.com machine. So I looked for the admin account in /etc/dovecot.passwd, and there it was, plain-text password and all! I set up a retrieval system for the admin POP3 account and got all the messages I was waiting for, not to mention about 20 that I wasn't, which were obviously old and discardable. This got me to thinking, is it possible to have an account accessible not just via POP3 from the ouside, but through mail/mailx on the inside, and if so, how is that set up? From chayes at afo.net Thu May 21 14:29:50 2015 From: chayes at afo.net (Cliff Hayes) Date: Thu, 21 May 2015 09:29:50 -0500 Subject: sieve permission problem with dovecot.svbin Message-ID: <555DEBDE.9010006@afo.net> Hello list, I am starting to see these in maillog: dovecot: lmtp(31949, dwright): Error: kXK3KnTpXFXNfAAAU+Cu/Q: sieve: binary open: failed to open: open(/var/spool/mail/dwright/.dovecot.svbin) failed: Permission denied (euid=520(dwright) egid=12(mail) missing +r perm: /var/spool/mail/dwright/.dovecot.svbin, euid is not dir owner) But the directory /var/spool/mail/dwright is owned by dwright:mail with permission set to 700. How do I fix? Here is my config: # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-431.29.2.el6.x86_64 x86_64 Scientific Linux release 6.5 (Carbon) auth_failure_delay = 5 secs auth_username_format = %Ln auth_verbose_passwords = plain mail_access_groups = mail mail_location = maildir:~/Maildir mail_plugins = " quota" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { driver = pam } plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+100M sieve = ~/.dovecot.sieve sieve_after = /dovecotSieveAfter/ sieve_dir = ~/sieve } protocols = imap lmtp sieve service managesieve-login { inet_listener sieve { port = 4190 } } ssl_ca = References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> Message-ID: I've been seeing this same error as well now: Fatal: master: service(indexer-worker): child 20979 killed with signal 6 (core not dumped - set service indexer-worker { drop_priv_before_exec=yes }) Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) It's only shown up on one particular user, and after tracking down the message(s) that caused it I found out that there were two identical messages with the same exact MD5/SHA hashes from our storage server in 2012 or so. Removing one of the duplicates fixed it for me. Not sure how that happened, but wonder if it's worth checking for? Thanks, -c On Sat, May 16, 2015 at 3:32 AM, Timo Sirainen wrote: > On 16 May 2015, at 09:39, Xin Li wrote: > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > Hi, > > > > On 5/15/15 10:58, Xin Li wrote: > >> Hi, > >> > >> I have seen the following assertion fails on my own mail server > >> (indexer-worker): > >> > >> Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion > >> failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) > > > > I've managed to narrow down the issue to one mail, and can reliably > > trigger the assertion with 'doveadm index -u user INBOX' with only that > > email. > > Can you send me some mail that triggers the crash? I'm having trouble > reproducing. > > > Maybe we should reset the iconv descriptor after each full conversion? > > i.e.: > > > > diff -r 91bbd6753689 src/lib-mail/message-decoder.c > > - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 > > +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 > > @@ -347,6 +347,8 @@ > > sizeof(ctx->translation_buf)); > > memcpy(ctx->translation_buf, data + pos, > > ctx->translation_size); > > + } else { > > + charset_to_utf8_reset(ctx->charset_trans); > > } > > I think the fix probably belongs to charset-iconv.c > From tss at iki.fi Thu May 21 22:59:09 2015 From: tss at iki.fi (Timo Sirainen) Date: Thu, 21 May 2015 18:59:09 -0400 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> Message-ID: <92869D20-E239-461E-A57C-E589A89230F2@iki.fi> On 21 May 2015, at 18:47, Cassidy Larson wrote: > > I've been seeing this same error as well now: > > Fatal: master: service(indexer-worker): child 20979 killed with signal 6 (core not dumped - set service indexer-worker { drop_priv_before_exec=yes }) > Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) > > It's only shown up on one particular user, and after tracking down the message(s) that caused it I found out that there were two identical messages with the same exact MD5/SHA hashes from our storage server in 2012 or so. Removing one of the duplicates fixed it for me. Not sure how that happened, but wonder if it's worth checking for? What OS or Linux distro are you using? > > Thanks, > > -c > > On Sat, May 16, 2015 at 3:32 AM, Timo Sirainen > wrote: > On 16 May 2015, at 09:39, Xin Li > wrote: > > > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA512 > > > > Hi, > > > > On 5/15/15 10:58, Xin Li wrote: > >> Hi, > >> > >> I have seen the following assertion fails on my own mail server > >> (indexer-worker): > >> > >> Panic: file charset-iconv.c: line 132 (charset_to_utf8): assertion > >> failed: (*src_size - pos <= CHARSET_MAX_PENDING_BUF_SIZE) > > > > I've managed to narrow down the issue to one mail, and can reliably > > trigger the assertion with 'doveadm index -u user INBOX' with only that > > email. > > Can you send me some mail that triggers the crash? I'm having trouble reproducing. > > > Maybe we should reset the iconv descriptor after each full conversion? > > i.e.: > > > > diff -r 91bbd6753689 src/lib-mail/message-decoder.c > > - --- a/src/lib-mail/message-decoder.c Thu May 14 23:49:12 2015 +0300 > > +++ b/src/lib-mail/message-decoder.c Fri May 15 22:52:41 2015 -0700 > > @@ -347,6 +347,8 @@ > > sizeof(ctx->translation_buf)); > > memcpy(ctx->translation_buf, data + pos, > > ctx->translation_size); > > + } else { > > + charset_to_utf8_reset(ctx->charset_trans); > > } > > I think the fix probably belongs to charset-iconv.c > From edgar at pettijohn-web.com Fri May 22 00:21:41 2015 From: edgar at pettijohn-web.com (Edgar Pettijohn III) Date: Thu, 21 May 2015 19:21:41 -0500 Subject: Can email be shared between POP3 and shell account access? In-Reply-To: <5iqrla9kqmskifpi8mprfmh2sq0vea17ql@4ax.com> References: <5iqrla9kqmskifpi8mprfmh2sq0vea17ql@4ax.com> Message-ID: <6F79A41C-CD29-4D67-9676-39F22B45A85B@pettijohn-web.com> On May 21, 2015, at 9:25 AM, Steve Matzura wrote: > I needed to receive a message to admin at mydomain.com. I waited and > waited for it, but it never showed up in the admin mailbox on the > mydomain.com machine. So I looked for the admin account in > /etc/dovecot.passwd, and there it was, plain-text password and all! I > set up a retrieval system for the admin POP3 account and got all the > messages I was waiting for, not to mention about 20 that I wasn't, > which were obviously old and discardable. This got me to thinking, is > it possible to have an account accessible not just via POP3 from the > ouside, but through mail/mailx on the inside, and if so, how is that > set up? If you have root or sudo then all you need is mutt. # mutt -f /path/to/Maildir From django at nausch.org Fri May 22 07:08:28 2015 From: django at nausch.org (django at nausch.org) Date: Fri, 22 May 2015 09:08:28 +0200 Subject: dovecot 2.2.18 and ssl_parameters_regenerate Message-ID: <20150522090828.Horde.TwjySX091W3No3LSfO7TkoU@xn--bro-hoa.nausch.org> HI Timo, On http://wiki2.dovecot.org/SSL/DovecotConfiguration I read in chapter SSL security settings: When Dovecot starts up for the first time, it generates new 512bit and 1024bit Diffie Hellman parameters and saves them into /var/lib/dovecot/ssl-parameters.dat. After the initial creation they're by default regenerated every week. With newer computers the generation shouldn't take more than a few seconds, but with older computers it can take as long as half an hour. The extra security gained by the regeneration is quite small, so with slower computers, for Dovecot versions prior to v2.2, you might want to disable it If I discover the default-value of ssl_parameters_regenerate I receive: # doveconf -d ssl_parameters_regenerate ssl_parameters_regenerate = 0 In your doku you wrote, that dovecot will regenerate every week. :/ ? I set it to "1 hours" and watch if /var/lib/dovecot/ssl-parameters.dat is build every hour, but nothing happens. ssl-parameters.dat is only rebuild, if I change ssl_dh_parameters_length What's the problem? What I've made false? ttyl Django -- http://dokuwiki.nausch.org http://wetterstation-pliening.info http://ebersberger-liedersammlung.de From skdovecot at smail.inf.fh-brs.de Fri May 22 07:11:54 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 22 May 2015 09:11:54 +0200 (CEST) Subject: sieve permission problem with dovecot.svbin In-Reply-To: <555DEBDE.9010006@afo.net> References: <555DEBDE.9010006@afo.net> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Thu, 21 May 2015, Cliff Hayes wrote: > I am starting to see these in maillog: > > dovecot: lmtp(31949, dwright): Error: kXK3KnTpXFXNfAAAU+Cu/Q: sieve: binary > open: failed to open: open(/var/spool/mail/dwright/.dovecot.svbin) failed: > Permission denied (euid=520(dwright) egid=12(mail) missing +r perm: > /var/spool/mail/dwright/.dovecot.svbin, euid is not dir owner) > > But the directory /var/spool/mail/dwright is owned by dwright:mail with > permission set to 700. Post the output of: ls -ald /var/spool/mail/dwright /var/spool/mail /var/spool \ /var/spool/mail/dwright/.dovecot.svbin Do you have SELinux enabled? http://wiki2.dovecot.org/WhyDoesItNotWork?highlight=%28selinux%29 - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVV7Wunz1H7kL/d9rAQLVVAgAi9Jml2IPT6TuSpqzmY3sviQx4cwPdjXS qEmNCyAUX90FGJ1AITTYJFwDJaQ2qaRchejAxaVW8jjlY0JvM5Ge6WbxMUN36JQx hE4/342NNaLi5ZWPW0KeP7FV+xBI8w5wLSWaa8gcoyTA1W03MAB+729UDhPbPFxl 4cFCGepTu8ROMhnwVHxstj/VNPjjsu+i2N7jG9zOEqdXwThI8dEdbbZKSwf/efI+ 9owGbQsGWHVZikloRQ/isZUiQ73nNTHxY1FHabIrhkJTc+Yd+lKCjabQaAN6nTfp nyTN+sPt+qNvOtzmlXYYFeg4m8evQrz7f6Akyk+RCx00zckFh044Dw== =+e+C -----END PGP SIGNATURE----- From skdovecot at smail.inf.fh-brs.de Fri May 22 09:04:40 2015 From: skdovecot at smail.inf.fh-brs.de (Steffen Kaiser) Date: Fri, 22 May 2015 11:04:40 +0200 (CEST) Subject: race condition? -> Error: dict quota: Quota update failed, it's now desynced Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I get this error now and then, but only for users, that share or use shared mailboxes. See this snippet: 2015-05-22 02:27:09 IMAP() [28776]: Info: Disconnected: Logged out in=21450 out=76933 2015-05-22 02:27:09 IMAP() [28774]: Info: Disconnected: Logged out in=3769 out=16379 2015-05-22 02:27:09 IMAP() [28770]: Info: Disconnected: Logged out in=641 out=394985 2015-05-22 02:27:09 IMAP() [28614]: Info: Disconnected: Logged out in=510 out=2178 2015-05-22 02:27:09 IMAP() [28768]: Info: expunge: box=users..INBOX, uid=8711, msgid=<008801d093ef$f46c00d0$dd440270$@example.com>, size=17433, vsize=17849, from=[cut] 2015-05-22 02:27:09 IMAP() [28830]: Info: Disconnected: Logged out in=555 out=2763 2015-05-22 02:27:09 IMAP() [28768]: Info: Disconnected: Logged out in=628 out=2851717 2015-05-22 02:27:09 IMAP() [28768]: Error: dict quota: Quota update failed, it's now desynced 2015-05-22 02:27:09 IMAP() [28827]: Info: Disconnected: Logged out in=267 out=1256 2015-05-22 02:27:09 IMAP() [28824]: Info: Disconnected: Logged out in=175 out=1073 2015-05-22 02:27:09 IMAP() [28831]: Info: Connection closed in=550 out=3197 2015-05-22 02:27:09 IMAP() [28663]: Info: expunge: box=INBOX, uid=8711, msgid=<008801d093ef$f46c00d0$dd440270$@example.com>, size=17433, vsize=17849, from=[cut] 2015-05-22 02:27:09 IMAP() [28663]: Info: Disconnected: Logged out in=800 out=25925 OTHER-UID shares the INBOX to uid1. Now, the _same_ message is about to delete once via the "sharing" uid and once from the consumer at the same time. My quota backend is Postgres, via quota = pgsql:/usr/local/dovecot-2.2.15/etc/dovecot/dovecot-dict-sql.conf.ext on Dovecot v2.2 hg 18272:466596200825 . I tried to replicate the error, by setting up two accounts, A shares INBOX to B, setup A and B in the same Thunderbird, configured that both A and B expunge the INBOX on termination of MUA, but I cannot reproduce that the same message is deleted twice in parallel. I also tried current hg's tip 8737:8906101589f9, but I cannot reproduce the error, no matter what I try. Also, I think, that in this very case the quota is OK still, because it had been updated for the sharing user OTHER-UID. - -- Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEVAwUBVV7xKHz1H7kL/d9rAQIlWAf+M0KHX2IjJufhamNlnlWJRhbGO+qmwnlh A6644yzn8vEtx3zHvOfvULAEZyWPsHlmMnu2YNKa3JrUYXte2zl1M7AzLR0Wsl9k xcBGxsbeRhqkDOrDoNRXMAZaTzWdqHDvXDGBujsh7+whjDh+Uv9uD0/LYxuRo+lb r54QF9qgSv4NLB4DlWhPMcwWjBWmTRu5kJGpU05jwlCgroZxkvB59FXaccMQXe5r ay9YWhBLz1qVVRg7ZsnY6ogwmlM/MyreI0o8A6iPfgDmrOcHjYbQ7jLGb7771vHJ SUjyC5NsU/A9HWr3jI2gmi2kSMIvPKOxHtUKpschLqBprbUkj0tNyg== =0bWX -----END PGP SIGNATURE----- From pug at felsing.net Fri May 22 09:49:48 2015 From: pug at felsing.net (Christian Felsing) Date: Fri, 22 May 2015 11:49:48 +0200 Subject: Problems To Get Dovecot Running On FreeBSD: auth process crashes Message-ID: <555EFBBC.7030702@felsing.net> Hello, I ran into problems to get Dovecot running on FreeBSD. Steps for reproduction. Start Docot with dovecot -F (same result if started by rc script) telnet 192.168.116.38 110 Trying 192.168.116.38... Connected to freebsd. Escape character is '^]'. -ERR Disconnected: Auth process broken Connection closed by foreign host. logfile shows following: May 22 11:41:24 freebsd dovecot: master: Dovecot v2.2.18 starting up for imap, pop3, lmtp, sieve May 22 11:41:27 freebsd dovecot: auth: Error: auth: environment corrupt; missing value for DOVECOT_ May 22 11:41:27 freebsd dovecot: auth: Fatal: unsetenv(RESTRICT_SETUID) failed: Bad address May 22 11:41:27 freebsd dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs May 22 11:41:27 freebsd dovecot: pop3-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 1 secs): user=<>, rip=192.168.116.1, lip=192.168.116.38, session= May 22 11:41:32 freebsd dovecot: master: Warning: Killed with signal 2 (by pid=0 uid=0 code=kernel) Error message is quite meaningless regarding reason for crash. I am running same config on a Linux (Debian 8) host w/o problems. Is it a problem specific to FreeBSD? Christian ---cut here--- [root at freebsd /usr/local/etc/dovecot]# doveconf -n # 2.2.18: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.8 (0c4ae064f307+) # OS: FreeBSD 10.1-RELEASE-p10 amd64 auth_debug = yes auth_debug_passwords = yes auth_master_user_separator = * auth_mechanisms = plain login auth_ssl_username_from_cert = yes auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@#" auth_username_translation = "@#" auth_verbose = yes base_dir = /var/run/dovecot/ disable_plaintext_auth = no first_valid_uid = 124 hostname = mail.taunusstein.net last_valid_uid = 124 lda_mailbox_autocreate = yes listen = 192.168.116.38 log_timestamp = %Y-%m-%d %H:%M:%S login_greeting = mail.taunusstein.net imap4/pop3 ready. mail_gid = 124 mail_location = maildir:~/Maildir mail_privileged_group = vmail mail_uid = 124 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate imapflags notify namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes } passdb { driver = pam } plugin { acl = vfile:/etc/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes autocreate = Trash autocreate2 = Drafts autosubscribe = Trash autosubscribe2 = Drafts fts = solr fts_solr = break-imap-search url=http://solr.lan:8983/solr/ quota = maildir:User quota quota_rule = *:storage=500M quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } postmaster_address = postmaster at taunusstein.net protocols = imap pop3 lmtp sieve sendmail_path = /usr/lib/sendmail service anvil { client_limit = 4000 } service auth-worker { group = vmail } service auth { client_limit = 8000 unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = postfix mode = 0666 user = postfix } user = root } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 1024 } service imap-postlogin { executable = script-login /opt/dovecot-cf/bin/lastlogin.py } service imap { executable = imap imap-postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 1024 } service pop3-postlogin { executable = script-login /opt/dovecot-cf/bin/lastlogin.py } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { user = vmail } ssl_cert = References: <555EFBBC.7030702@felsing.net> Message-ID: <20150522062641.61386ce3@seibercom.net> On Fri, 22 May 2015 11:49:48 +0200, Christian Felsing stated: >I am running same config on a Linux (Debian 8) host w/o problems. Is it >a problem specific to FreeBSD? I am running Dovecot on a FreeBSD 10.1 amd 64 system also, sans any problems. How did you install dovecot? -- Jerry -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 473 bytes Desc: OpenPGP digital signature URL: From listas at adminlinux.com.br Fri May 22 14:41:33 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Fri, 22 May 2015 11:41:33 -0300 Subject: Couldn't create lock .dovecot-sync.lock In-Reply-To: <555C87E0.7040804@adminlinux.com.br> References: <555740DC.7090409@adminlinux.com.br> <555C87E0.7040804@adminlinux.com.br> Message-ID: <555F401D.2040709@adminlinux.com.br> Hi, The point here is: doveadm replicate does not create mail location in both backends. I have: # cd /var/run/dovecot/ # ls -l replicat* srw-rw-rw- 1 dovemail dovemail 0 May 14 19:44 replication-notify prw-rw-rw- 1 dovemail dovemail 0 May 22 14:28 replication-notify-fifo srw------- 1 dovecot root 0 May 14 19:44 replicator srw-rw-rw- 1 dovemail dovemail 0 May 14 19:44 replicator-doveadm # ls -ld /var/lib/imap/user/ drwxr-xr-x 251 dovemail mail 4096 May 20 22:01 /var/lib/imap/user/ # ls -ld /var/lib/imap/user/6a drwx------ 14 dovemail dovemail 4096 May 22 00:02 /var/lib/imap/user/6a These permissions are sufficient. Right? if I run manually: doveadm sync -u tcp:: the directory /var/lib/imap/user/6a/ is created correctly Em 20-05-2015 10:10, Listas at Adminlinux escreveu: > dsync-server(): Error: Couldn't create lock > /var/lib/imap/user/6a//.dovecot-sync.lock: No such file or directory From dovecot at outputservices.com Fri May 22 15:27:35 2015 From: dovecot at outputservices.com (dovecot at outputservices.com) Date: Fri, 22 May 2015 09:27:35 -0600 (MDT) Subject: v2.2.18 released Message-ID: <201505221527.t4MFRZT00965@outputservices.com> I am just starting out and trying to get dovecot and postfix working in Solaris 10 environment. Went from 2.2.15 to 2.2.17, then to 2.2.18 Both version 2.2.17 & 2.2.18 gave this same error. 2.2.15 does not. ----------------------------------- May 15 09:27:37 master: Info: Dovecot v2.2.18 starting up for imap May 15 09:27:37 ssl-params: Info: Generating SSL parameters May 15 09:27:37 ssl-params: Fatal: Couldn't load required plugin /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: dlopen() failed: ld.so.1: ssl-params: fatal: relocation error: file /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: symbol SSL_clear_options: referenced symbol not found May 15 09:27:37 ssl-params: Error: child process failed with status 22784 May 15 09:27:46 master: Warning: Killed with signal 15 (by pid=1848 uid=0 code=kill) ------------------------------------------------ doveconf -n # 2.2.18: /tmp/tocevod/etc/dovecot/dovecot.conf # OS: SunOS 5.10 i86pc log_path = /tmp/tocevod/log/log login_trusted_networks = 137.106.0.0/16 mail_location = mbox:~/posta:INBOX=/var/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /dovecot/etc/dovecot/conf.d/ldap.settings driver = ldap } protocols = imap service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } } ssl_cert = References: <201505221527.t4MFRZT00965@outputservices.com> Message-ID: <555F7448.3020407@xdrv.co.uk> On 22/05/2015 16:27, dovecot at outputservices.com wrote: > I am just starting out and trying to get dovecot and postfix working in Solaris 10 environment. > > Went from 2.2.15 to 2.2.17, then to 2.2.18 > > Both version 2.2.17 & 2.2.18 gave this same error. 2.2.15 does not. > > ----------------------------------- > May 15 09:27:37 master: Info: Dovecot v2.2.18 starting up for imap > May 15 09:27:37 ssl-params: Info: Generating SSL parameters > May 15 09:27:37 ssl-params: Fatal: Couldn't load required plugin /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: dlopen() failed: ld.so.1: ssl-params: fatal: relocation error: file /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: symbol SSL_clear_options: referenced symbol not found I suspect you have compiled with one SSL library and are running with another. Hints: 1. Check RPATH in libs: $ dump -Lv /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so 2. unset LD_LIBRARY_PATH 3. Check which ssl libs are being used: $ ldd /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so 4. and symbol check: $ ldd -r /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so I assume your use of /tmp is temporary. Set prefix to, eg, /opt/tocevod. James. From dovecot at outputservices.com Fri May 22 18:47:09 2015 From: dovecot at outputservices.com (dovecot at outputservices.com) Date: Fri, 22 May 2015 12:47:09 -0600 (MDT) Subject: v2.2.18 released Message-ID: <201505221847.t4MIl9v19598@outputservices.com> On 05/22/15 12:24, James wrote: > On 22/05/2015 16:27, dovecot at outputservices.com wrote: >> I am just starting out and trying to get dovecot and postfix working in Solaris 10 environment. >> >> Went from 2.2.15 to 2.2.17, then to 2.2.18 >> >> Both version 2.2.17 & 2.2.18 gave this same error. 2.2.15 does not. >> >> ----------------------------------- >> May 15 09:27:37 master: Info: Dovecot v2.2.18 starting up for imap >> May 15 09:27:37 ssl-params: Info: Generating SSL parameters >> May 15 09:27:37 ssl-params: Fatal: Couldn't load required plugin /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: dlopen() failed: ld.so.1: ssl-params: fatal: relocation error: file /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so: symbol SSL_clear_options: referenced symbol not found > > > I suspect you have compiled with one SSL library and are running with another. > > Hints: > > 1. Check RPATH in libs: > $ dump -Lv /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so > > 2. unset LD_LIBRARY_PATH > > 3. Check which ssl libs are being used: > $ ldd /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so Yep, that is it. It found some old ssl libraries. I will go back and look at the compile command better. > > 4. and symbol check: > $ ldd -r /tmp/tocevod/lib/dovecot/libssl_iostream_openssl.so > > > > I assume your use of /tmp is temporary. Set prefix to, eg, /opt/tocevod. > > > > James. Thanks From alandaluz at gmail.com Fri May 22 20:18:38 2015 From: alandaluz at gmail.com (Cassidy Larson) Date: Fri, 22 May 2015 14:18:38 -0600 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: <555E629F.6040401@delphij.net> References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> <555E629F.6040401@delphij.net> Message-ID: We're using FreeBSD 10.1 Thanks On Thu, May 21, 2015 at 4:56 PM, Xin Li wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > On 05/21/15 15:47, Cassidy Larson wrote: > > I've been seeing this same error as well now: > > > > Fatal: master: service(indexer-worker): child 20979 killed with > > signal 6 (core not dumped - set service indexer-worker { > > drop_priv_before_exec=yes }) Panic: file charset-iconv.c: line 132 > > (charset_to_utf8): assertion failed: (*src_size - pos <= > > CHARSET_MAX_PENDING_BUF_SIZE) > > Quick question -- Are you using FreeBSD (10.0+)? > > Cheers, > - -- > Xin LI https://www.delphij.net/ > FreeBSD - The Power to Serve! Live free or die > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v2.1.4 (FreeBSD) > > iQIcBAEBCgAGBQJVXmKfAAoJEJW2GBstM+nspmoP/0CKS0g8ka/4uPF+jaq7+DD6 > 1PxCufdz/p528rclbXvEK3XE++sULihNVtO24TgXMOIJZB6jCfbfo+O17N9ZS7g9 > Vaz0r0URU28joAeDX/NR1yzYpHdo277RhXiPlKuyep+fEgi6Szm/mNaR+aCtYrge > 8Qm2U2eaHFQk/2qpyvC4ZlFzXSVIR+AIMSh87YqJSWSVd3PjNDLxZLDeUaQyygG5 > 1lW3lsy0yDOTISI58rXavOi7qa4qKo0ySDOMLct3GrdXzL5S8N/Pc3RnRTP5p9dY > 22cGoH3XloijWAef93qHSuPFuC9Yiko4g/FZAtfuQg6mCETvhPDfhXBgRf3IFYAr > vuMOyBX+3TrQQI+zz3aolou10WH+E+PPbJFFZcNq4sRGQYtSXBA4RmDDKU4NgwoB > gIzS43GjPidJTxdzEC+eedmS6Rt5wvyFaDK1h5+x/mgnfEie6E9Hfk8IBqRHZMUW > HixCmBnC3p1GN+R8rbPxjuycP06bB7YXLJ28o06gM2agrGv5vu83j+WqO2g6wJIP > hRKnyCVVPUeAnkLpS+7nXZwW5R/j8M5y7V8rbiYCX4q+zWCJZl/ac1G12zwva5k1 > MwyjciDP951soMM2RYsQQSE5RxRKMC03kqGt+qnBx3jBR7afzhOmcO9gcKQJEY5R > qjwnZ0voZtV7u2Ptcv1Q > =hWt9 > -----END PGP SIGNATURE----- > From delphij at delphij.net Fri May 22 21:44:49 2015 From: delphij at delphij.net (Xin Li) Date: Fri, 22 May 2015 14:44:49 -0700 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> <555E629F.6040401@delphij.net> Message-ID: <555FA351.3090700@delphij.net> On 05/22/15 13:18, Cassidy Larson wrote: > We're using FreeBSD 10.1 I see. Yes that's the same problem I have seen. There is a behavioral difference (I think it's a FreeBSD bug) between FreeBSD's iconv(3) and GNU implementation, and there is arguably a bug with Dovecot that iconv(3) state should be reset for each multipart part. The two together would trigger the problem when the previous part was ended with UTF-7 shift sequence. Timo have applied a fix (hg: 18721:9809f68aaa36) for Dovecot which should mitigates this issue at Dovecot side a few days ago. I have created a reproduction program and discussed the issue with our iconv(3) developer, Tijl Coosemans , who have created a patch to correct this FreeBSD iconv(3) bug (not yet committed against -HEAD but I'd expect it happening soon). Cheers, -- Xin LI https://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -------------- next part -------------- A non-text attachment was scrubbed... Name: iconv-utf7.patch Type: text/x-patch Size: 2887 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: OpenPGP digital signature URL: From number6 at noisynotes.com Fri May 22 22:24:25 2015 From: number6 at noisynotes.com (Steve Matzura) Date: Fri, 22 May 2015 18:24:25 -0400 Subject: Can email be shared between POP3 and shell account access? In-Reply-To: <6F79A41C-CD29-4D67-9676-39F22B45A85B@pettijohn-web.com> References: <5iqrla9kqmskifpi8mprfmh2sq0vea17ql@4ax.com> <6F79A41C-CD29-4D67-9676-39F22B45A85B@pettijohn-web.com> Message-ID: On Thu, 21 May 2015 19:21:41 -0500, you wrote: > >On May 21, 2015, at 9:25 AM, Steve Matzura wrote: > >> I needed to receive a message to admin at mydomain.com. I waited and >> waited for it, but it never showed up in the admin mailbox on the >> mydomain.com machine. So I looked for the admin account in >> /etc/dovecot.passwd, and there it was, plain-text password and all! I >> set up a retrieval system for the admin POP3 account and got all the >> messages I was waiting for, not to mention about 20 that I wasn't, >> which were obviously old and discardable. This got me to thinking, is >> it possible to have an account accessible not just via POP3 from the >> ouside, but through mail/mailx on the inside, and if so, how is that >> set up? > >If you have root or sudo then all you need is mutt. > ># mutt -f /path/to/Maildir H'mm. All I need is mutt. There's a song in there somewhere. Just looked it up and will install forthwith. All of this going to further prove my depth of un-knowledge about this stuff. THANKS! From tss at iki.fi Fri May 22 22:55:38 2015 From: tss at iki.fi (Timo Sirainen) Date: Fri, 22 May 2015 18:55:38 -0400 Subject: charset_to_utf8 assertion of remaining src_size In-Reply-To: <555FA351.3090700@delphij.net> References: <555633BF.6010109@delphij.net> <5556E62B.20306@delphij.net> <2B678317-C6AA-4D7F-98D9-DA2D40D1EFAE@iki.fi> <555E629F.6040401@delphij.net> <555FA351.3090700@delphij.net> Message-ID: <8E32D455-FBE0-4C9B-AE0A-FB0C9AE18437@iki.fi> On 22 May 2015, at 17:44, Xin Li wrote: > > On 05/22/15 13:18, Cassidy Larson wrote: >> We're using FreeBSD 10.1 > > I see. Yes that's the same problem I have seen. > > There is a behavioral difference (I think it's a FreeBSD bug) between > FreeBSD's iconv(3) and GNU implementation, and there is arguably a bug > with Dovecot that iconv(3) state should be reset for each multipart > part. The two together would trigger the problem when the previous part > was ended with UTF-7 shift sequence. Timo have applied a fix (hg: > 18721:9809f68aaa36) for Dovecot which should mitigates this issue at > Dovecot side a few days ago. Could you or somebody help with getting this included as a unit test to Dovecot? I've attached a patch that I guess might cause the crash. -------------- next part -------------- A non-text attachment was scrubbed... Name: charset-test.diff Type: application/octet-stream Size: 1215 bytes Desc: not available URL: -------------- next part -------------- From mysqlstudent at gmail.com Sat May 23 02:02:49 2015 From: mysqlstudent at gmail.com (Alex) Date: Fri, 22 May 2015 22:02:49 -0400 Subject: Controlling IP addresses for services Message-ID: Hi, I'm using dovecot-2.2.15 on fedora21 and would like to configure pop3s and imaps to only listen on localhost and internal addresses. It seems the listen= parameter only supports ports, not IP addresses. Is it possible to control which IPs the services listen on? I'm also interested in knowing the differences between pop3-login and just the pop3 service? Same with imap-login and imap? I've searched the wiki and still a bit confused. Thanks, Alex From darix at opensu.se Sat May 23 02:44:13 2015 From: darix at opensu.se (Marcus Rueckert) Date: Sat, 23 May 2015 04:44:13 +0200 Subject: Controlling IP addresses for services In-Reply-To: References: Message-ID: <20150523024413.GR2934@nordisch.org> On 2015-05-22 22:02:49 -0400, Alex wrote: > I'm using dovecot-2.2.15 on fedora21 and would like to configure pop3s > and imaps to only listen on localhost and internal addresses. > > It seems the listen= parameter only supports ports, not IP addresses. > Is it possible to control which IPs the services listen on? > > I'm also interested in knowing the differences between pop3-login and > just the pop3 service? Same with imap-login and imap? > > I've searched the wiki and still a bit confused. listen=127.0.0.1,::1 -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From mysqlstudent at gmail.com Sat May 23 03:40:21 2015 From: mysqlstudent at gmail.com (Alex Regan) Date: Fri, 22 May 2015 23:40:21 -0400 Subject: Controlling IP addresses for services In-Reply-To: <20150523024413.GR2934@nordisch.org> References: <20150523024413.GR2934@nordisch.org> Message-ID: <555FF6A5.7080108@gmail.com> Hi, On 05/22/2015 10:44 PM, Marcus Rueckert wrote: > On 2015-05-22 22:02:49 -0400, Alex wrote: >> I'm using dovecot-2.2.15 on fedora21 and would like to configure pop3s >> and imaps to only listen on localhost and internal addresses. >> >> It seems the listen= parameter only supports ports, not IP addresses. >> Is it possible to control which IPs the services listen on? >> >> I'm also interested in knowing the differences between pop3-login and >> just the pop3 service? Same with imap-login and imap? >> >> I've searched the wiki and still a bit confused. > > listen=127.0.0.1,::1 Ah, looks like I should have been more clear. I meant on a per-service basis. Apparently adding listen= in a specific service section isn't supported? service imap-login { inet_listener imaps { listen=192.168.1.100 port = 993 } } # dovecot -n # 2.2.15: /etc/dovecot/dovecot.conf doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf line 54: Unknown setting: listen If not, I'll have to use iptables to block it on the external ports, but I'd rather just not have it listen there to begin with, and in addition to my firewall policy. Thanks, Alex From gedalya at gedalya.net Sat May 23 04:44:17 2015 From: gedalya at gedalya.net (Gedalya) Date: Sat, 23 May 2015 00:44:17 -0400 Subject: Controlling IP addresses for services In-Reply-To: <555FF6A5.7080108@gmail.com> References: <20150523024413.GR2934@nordisch.org> <555FF6A5.7080108@gmail.com> Message-ID: <556005A1.8010906@gedalya.net> On 05/22/2015 11:40 PM, Alex Regan wrote: > service imap-login { > inet_listener imaps { > listen=192.168.1.100 > port = 993 > } > } > > > # dovecot -n > # 2.2.15: /etc/dovecot/dovecot.conf > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf > line 54: Unknown setting: listen http://wiki2.dovecot.org/Services#inet_listeners Try address instead of listen From stephan at rename-it.nl Sat May 23 07:15:32 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 23 May 2015 09:15:32 +0200 Subject: Dovecot-Director, (Manage)-Sieve und Remote-IP In-Reply-To: <555DB599.6020302@rename-it.nl> References: <555D89EE.2000502@kevag-telekom.de> <555DB599.6020302@rename-it.nl> Message-ID: <55602914.7020806@rename-it.nl> On 5/21/2015 12:38 PM, Stephan Bosch wrote: > > > Benjamin H?ck schreef op 21-5-2015 om 9:31: >> Hello, >> >> I`ve got a question about Dovecot-Director and (Manage)-Sieve: is there >> any possibility to pass-through the original IP-address to the backend >> server? >> >> In this case I try to pass-through the IP-address of a webmail server. >> >> I`ve looked for a solution in different forums and tried it with the >> parameter "login_trusted_networks", however no success. >> >> The above solution is working with IMAP/POP3/LMTP, but not with Sieve. >> >> For any solutions or any hint I would be grateful. > > Well... ManageSieve doesn't support an XCLIENT or equivalent command > at the moment, because... ehmm... probably nobody ever asked for it. > > Given the example in the pop3 service, it should be rather trivial to > implement in ManageSieve. I will look at that later this week. This is now implemented and will be part of the next release. Regards, Stephan. From stephan at rename-it.nl Sat May 23 07:48:12 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Sat, 23 May 2015 09:48:12 +0200 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150519200050.GB26451@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> <20150519193820.GA26451@cmadams.net> <20150519200050.GB26451@cmadams.net> Message-ID: <556030BC.6020906@rename-it.nl> On 5/19/2015 10:00 PM, Chris Adams wrote: > Once upon a time, Chris Adams said: >> Okay, digging some more, it looks like something in sieve is overwriting >> the wrong thing when it gets messages with some headers (at least From: >> and Subject:) repeated. I enabled the vnd.dovecot.debug sieve plugin, >> and used this sieve script: > And I guess something is re-parsing them at some point? The following > sieve script has the problem: You're using Dovecot 2.2.10, which is quite old. I remembered a bug like this, but I had to look it up. This is the original bug report: http://www.dovecot.org/list/dovecot/2014-August/097375.html Here's the fix: http://hg.dovecot.org/dovecot-2.2/rev/0e1a3c909a13 You'll need to upgrade to a version >= 2.2.14 to fix your problem. Regards, Stephan. From me at junc.eu Sat May 23 08:54:04 2015 From: me at junc.eu (Benny Pedersen) Date: Sat, 23 May 2015 10:54:04 +0200 Subject: Controlling IP addresses for services In-Reply-To: <555FF6A5.7080108@gmail.com> References: <20150523024413.GR2934@nordisch.org> <555FF6A5.7080108@gmail.com> Message-ID: <0ef89c5138f3fc4afaa0ef4ef9dde05d@junc.eu> Alex Regan skrev den 2015-05-23 05:40: > # dovecot -n > # 2.2.15: /etc/dovecot/dovecot.conf > doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf > line 54: Unknown setting: listen http://wiki2.dovecot.org/LMTP hope it works for imap aswell :=) From cma at cmadams.net Sat May 23 17:41:54 2015 From: cma at cmadams.net (Chris Adams) Date: Sat, 23 May 2015 12:41:54 -0500 Subject: Problem with sieve not triggering randomly? In-Reply-To: <556030BC.6020906@rename-it.nl> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> <20150519193820.GA26451@cmadams.net> <20150519200050.GB26451@cmadams.net> <556030BC.6020906@rename-it.nl> Message-ID: <20150523174154.GA31791@cmadams.net> Once upon a time, Stephan Bosch said: > You're using Dovecot 2.2.10, which is quite old. I remembered a bug like > this, but I had to look it up. This is the original bug report: Ah, sorry for taking your time on an old bug. For most of my servers, I try to use the RHEL/CentOS-provided packages where practical (because with over 100 servers running a wide variety of services, building everything from source takes more hours than I have in a day). The hazzard of course is chasing already-fixed bugs from time to time. I also understand Red Hat's "stability" approach, where they tend to stick with a given upstream version and just patch it; few Open Source software developers have the time to maintain bugfix-only branches for the lifetime of a RHEL major version, and new features tend to mean new bugs as well. Thanks for taking the time to point out the fix. I was hoping to have time this weekend to try the latest Dovecot release, but hadn't yet done that. I have opened a Red Hat bug to try to get this patch applied. https://bugzilla.redhat.com/show_bug.cgi?id=1224496 -- Chris Adams From darix at opensu.se Sat May 23 19:43:28 2015 From: darix at opensu.se (Marcus Rueckert) Date: Sat, 23 May 2015 21:43:28 +0200 Subject: Problem with sieve not triggering randomly? In-Reply-To: <20150523174154.GA31791@cmadams.net> References: <20150515145817.GA20600@cmadams.net> <55560C06.3080904@rename-it.nl> <20150515155604.GB20600@cmadams.net> <555637C9.1060901@rename-it.nl> <20150515185024.GE20600@cmadams.net> <20150519145634.GA23038@cmadams.net> <20150519193820.GA26451@cmadams.net> <20150519200050.GB26451@cmadams.net> <556030BC.6020906@rename-it.nl> <20150523174154.GA31791@cmadams.net> Message-ID: <20150523194327.GS2934@nordisch.org> On 2015-05-23 12:41:54 -0500, Chris Adams wrote: > Once upon a time, Stephan Bosch said: > > You're using Dovecot 2.2.10, which is quite old. I remembered a bug like > > this, but I had to look it up. This is the original bug report: > > Ah, sorry for taking your time on an old bug. For most of my servers, I > try to use the RHEL/CentOS-provided packages where practical (because > with over 100 servers running a wide variety of services, building > everything from source takes more hours than I have in a day). The > hazzard of course is chasing already-fixed bugs from time to time. > > I also understand Red Hat's "stability" approach, where they tend to > stick with a given upstream version and just patch it; few Open Source > software developers have the time to maintain bugfix-only branches for > the lifetime of a RHEL major version, and new features tend to mean new > bugs as well. > > Thanks for taking the time to point out the fix. I was hoping to have > time this weekend to try the latest Dovecot release, but hadn't yet done > that. I have opened a Red Hat bug to try to get this patch applied. > > https://bugzilla.redhat.com/show_bug.cgi?id=1224496 Dovecot Enterprise might be an option. darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From gedalya at gedalya.net Sun May 24 07:08:58 2015 From: gedalya at gedalya.net (Gedalya) Date: Sun, 24 May 2015 03:08:58 -0400 Subject: How about an option to disbale headers? (was Re: Patch for "doveadm -f table" nit) In-Reply-To: <43DCC0A1-BD9B-4A1E-88EE-51517B3EBBB0@iki.fi> References: <78D069B1-3D22-456B-B13E-1FD9AC871157@iki.fi> <550A891D.8040108@gedalya.net> <43DCC0A1-BD9B-4A1E-88EE-51517B3EBBB0@iki.fi> Message-ID: <5561790A.1020607@gedalya.net> On 03/20/2015 02:47 PM, Timo Sirainen wrote: > Added -h parameter now to hg. Using 2.2.18. With -f table this behaves as expected, however with -t tab the output seems to include the separating tabs of the header line prepended to the first line of output. In other words, the header line is printed partially - only the tabs, no actual headers and no newline. From dovecot at zp1.net Sun May 24 14:34:15 2015 From: dovecot at zp1.net (Christian Eichert) Date: Sun, 24 May 2015 16:34:15 +0200 Subject: timeout for POP3 query. Message-ID: <5561E167.2020708@zp1.net> Is there a way to timeout POP3 query if they come to often? More precise I want to know if there is a setting in dovecot that makes it ignore pop3 requests from a certain user if they come more often then a certain amount of time. I am looking for a setting in dovecot, not a firewall solution. I would like dovecot to answer something like "requests come to often, please come back later." regards christian From mysqlstudent at gmail.com Sun May 24 16:32:16 2015 From: mysqlstudent at gmail.com (Alex Regan) Date: Sun, 24 May 2015 12:32:16 -0400 Subject: Controlling IP addresses for services In-Reply-To: <0ef89c5138f3fc4afaa0ef4ef9dde05d@junc.eu> References: <20150523024413.GR2934@nordisch.org> <555FF6A5.7080108@gmail.com> <0ef89c5138f3fc4afaa0ef4ef9dde05d@junc.eu> Message-ID: <5561FD10.2010000@gmail.com> > >> # dovecot -n >> # 2.2.15: /etc/dovecot/dovecot.conf >> doveconf: Fatal: Error in configuration file /etc/dovecot/dovecot.conf >> line 54: Unknown setting: listen > > http://wiki2.dovecot.org/LMTP > > hope it works for imap aswell :=) This worked well, thanks everyone! Best regards, Alex From tlx at leuxner.net Mon May 25 06:14:41 2015 From: tlx at leuxner.net (Thomas Leuxner) Date: Mon, 25 May 2015 08:14:41 +0200 Subject: Typo in systemd commit HG 8dc79a437858 Message-ID: <20150525061441.GA12212@nihlus.leuxner.net> The apostrophes need to be removed from the LimitCORE example, otherwise the parameter will not be recognized: $ cat /etc/systemd/system/dovecot.service.d/service.conf [Service] #Environment='OPTIONS=-p' #LimitCORE=8192 LimitCORE=infinity -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 181 bytes Desc: Digital signature URL: From user+dovecot at localhost.localdomain.org Mon May 25 14:53:15 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 25 May 2015 14:53:15 +0000 Subject: Typo in systemd commit HG 8dc79a437858 In-Reply-To: <20150525061441.GA12212@nihlus.leuxner.net> References: <20150525061441.GA12212@nihlus.leuxner.net> Message-ID: <5563375B.7000108@localhost.localdomain.org> On 05/25/2015 06:14 AM, Thomas Leuxner wrote: > The apostrophes need to be removed from the LimitCORE example, otherwise the parameter will not be recognized: > > $ cat /etc/systemd/system/dovecot.service.d/service.conf > [Service] > #Environment='OPTIONS=-p' > #LimitCORE=8192 > LimitCORE=infinity Oops, my bad. On my system i have (w/o drop-in files): ,--[ systemctl show --property=LimitCORE dovecot ]-- | LimitCORE=18446744073709551615 `-- But your message showed me just another typo. The setting name, for configuring the file descriptor limit, is LimitNOFILE - instead of LimitCORE. Stephan's next auto-build should contain the corrected version. Regards, Pascal -- The trapper recommends today: defaced.1514516 at localdomain.org From micah at riseup.net Mon May 25 15:13:46 2015 From: micah at riseup.net (micah) Date: Mon, 25 May 2015 11:13:46 -0400 Subject: Couldn't set transaction log view (seq 9..13) In-Reply-To: <527818E0-2CB2-4B51-BA1A-14A823B21BD3@iki.fi> References: <87mw2rvfn5.fsf@muck.riseup.net> <527818E0-2CB2-4B51-BA1A-14A823B21BD3@iki.fi> Message-ID: <87pp5o666t.fsf@muck.riseup.net> Hi Timo, Thanks for your quick response, apologies for my slow one. Timo Sirainen writes: >> I'm using 2.2.13 on Debian stable, and I perform regular dsync >> backups. Typically if there is an error, dsync can recover from it the >> next time it is run, but I've been encountering a new error that doesn't >> seem to self-heal: >> >> # /usr/bin/dsync -u micah backup ssh -i /root/.ssh/id_rsa backmaildir at internal.example /usr/bin/dsync -u micah >> dsync-remote(micah): Error: /srv/maildirbackups/m/micah/daily.1/mailboxes/INBOX/dbox-Mails/dovecot.index: Couldn't set transaction log view (seq 9..13) >> dsync-remote(micah): Error: Failed to read transaction log for mailbox INBOX >> >> Re-running the dsync never fixes the issue. >> >> I'm happy to provide any additional information if so required to help >> track down this problem! > > Can you create a tarball of such mailbox directories that reproduces the problem? Probably this would work: > > - copy source and destination dboxes to /tmp/test/1 and /tmp/test/2 > - delete all the mailboxes except INBOX I noticed an interesting bit of information doing this: the users that are having a problem have no dbox-Mails directory in their source mailboxes/INBOX directory - but on the destination where I am sending the backups, there is one. If I remove the backup's INBOX/ and then re-do the dsync, things work again. > - delete all the u.* files from the INBOXes. This probably doesn't > affect the dsync run - if it does run the mails instead through > http://dovecot.org/tools/dbox-anonymize.pl I have no u.* files on either side. There are m.* files though? > - delete dovecot.index.cache files > - make sure you can still reproduce this error by running: doveadm -o > mail=sdbox:/tmp/test/1 sync sdbox:/tmp/test/2 did you mean s/sdbox/mdbox/ here? also, I was using 'backup' and not sync, I dont know if that matters? I tried this with some that had issues, but I couldn't reproduce the error this way :p > - tar czf test.tar.gz /tmp/test > > Then send the tarball to me and I can fix the bug. Oh, also it would > be a good idea to test with a newer Dovecot version since there are > some indexing fixes since v2.2.13.. I'm working on getting a newer version of dovecot installed to test this. > Also I added some code that should give a better-debuggable error message if this happens: http://hg.dovecot.org/dovecot-2.2/rev/8146fdc0de34 great, this will certainly be helpful! micah From slitt at troubleshooters.com Mon May 25 15:44:27 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Mon, 25 May 2015 11:44:27 -0400 Subject: Typo in systemd commit HG 8dc79a437858 In-Reply-To: <20150525061441.GA12212@nihlus.leuxner.net> References: <20150525061441.GA12212@nihlus.leuxner.net> Message-ID: <20150525114427.7eb07b24@mydesq2.domain.cxm> On Mon, 25 May 2015 08:14:41 +0200 Thomas Leuxner wrote: > The apostrophes need to be removed from the LimitCORE example, > otherwise the parameter will not be recognized: > > $ cat /etc/systemd/system/dovecot.service.d/service.conf > [Service] > #Environment='OPTIONS=-p' > #LimitCORE=8192 > LimitCORE=infinity Hey guys, please tell me that Dovecot does not depend on systemd! SteveT Steve Litt May 2015 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz From user+dovecot at localhost.localdomain.org Mon May 25 15:50:08 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 25 May 2015 15:50:08 +0000 Subject: Typo in systemd commit HG 8dc79a437858 In-Reply-To: <20150525114427.7eb07b24@mydesq2.domain.cxm> References: <20150525061441.GA12212@nihlus.leuxner.net> <20150525114427.7eb07b24@mydesq2.domain.cxm> Message-ID: <556344B0.4020101@localhost.localdomain.org> On 05/25/2015 03:44 PM, Steve Litt wrote: > Hey guys, please tell me that Dovecot does not depend on systemd! > Nope, Dovecot doesn't depend on systemd. Why should it depend on systemd? But: Dovecot supports systemd. If you want to enable systemd, use: ./configure --with-systemdsystemunitdir Regards, Pascal -- The trapper recommends today: beeffeed.1514517 at localdomain.org From tss at iki.fi Mon May 25 15:53:48 2015 From: tss at iki.fi (Timo Sirainen) Date: Mon, 25 May 2015 11:53:48 -0400 Subject: timeout for POP3 query. In-Reply-To: <5561E167.2020708@zp1.net> References: <5561E167.2020708@zp1.net> Message-ID: <43F505F5-7212-4ECA-A7CB-CAE02B633C7C@iki.fi> On 24 May 2015, at 10:34, Christian Eichert wrote: > > Is there a way to timeout POP3 query if they come to often? > > More precise I want to know if there is a setting in dovecot that makes it ignore pop3 requests from a certain user if they come more often then a certain amount of time. I am looking for a setting in dovecot, not a firewall solution. > I would like dovecot to answer something like "requests come to often, please come back later." http://dovecot.org/patches/2.0/pop3-throttle-plugin.c may be useful for you. Although that was written for Dovecot v2.0 and it probably requires some changes to compile. From cross+dovecot at distal.com Mon May 25 16:59:45 2015 From: cross+dovecot at distal.com (Chris Ross) Date: Mon, 25 May 2015 12:59:45 -0400 Subject: Shared inbox? Message-ID: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> I'm running dovecot 2.2.16 on my FreeBSD mail server. I've read information on the wiki about setting up shared mailboxes, but I want to do something that isn't really coved by the instructions I was reading there. My son (now 7 years old) has an account on the system, but doesn't use it directly. But, for things he's interested in like Minecraft, and/or the local zoo, we have given out his email address in a small number of places. What I would like to set up, both for now while he's not actually ever reading his email himself, and perhaps even for the future when we teach and/or allow for that, is for my wife and I to be able to "view" his inbox from our accounts. The instructions for setting up shared folders all are written so that they're secondary folders to all accounts. Is there a way to either (a) configure sharing someone elses Inbox by other accounts, or (b) setting up a separately configured shared folder to _act_ as the inbox for a single account? Thanks. Any suggestions to achieve the above described end goal would be appreciated. - Chris From slitt at troubleshooters.com Mon May 25 17:22:48 2015 From: slitt at troubleshooters.com (Steve Litt) Date: Mon, 25 May 2015 13:22:48 -0400 Subject: Typo in systemd commit HG 8dc79a437858 In-Reply-To: <556344B0.4020101@localhost.localdomain.org> References: <20150525061441.GA12212@nihlus.leuxner.net> <20150525114427.7eb07b24@mydesq2.domain.cxm> <556344B0.4020101@localhost.localdomain.org> Message-ID: <20150525132248.71c2afb5@mydesq2.domain.cxm> On Mon, 25 May 2015 15:50:08 +0000 Pascal Volk wrote: > On 05/25/2015 03:44 PM, Steve Litt wrote: > > Hey guys, please tell me that Dovecot does not depend on systemd! > > > > Nope, Dovecot doesn't depend on systemd. Why should it depend on > systemd? I could ask the question "why should it depend on systemd" about a whole raft of programs that do depend on it. It's a mystery to me. Anyway, I'm glad that it doesn't. > But: Dovecot supports systemd. If you want to enable systemd, use: > > ./configure --with-systemdsystemunitdir It sounds like the preceding is some sort of dohickey to enable Dovecot to more easily be started/managed by systemd. I hope that's all it is, and I hope it always remains voluntary, because installing systemd on my machine, the way I have it set up, would have all sorts of side effects. Anyway, in the near future I'll probably be managing Dovecot through Daemontools. Thanks for the explanation, SteveT Steve Litt May 2015 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz From andy at thecsillags.com Mon May 25 19:55:47 2015 From: andy at thecsillags.com (andy at thecsillags.com) Date: Mon, 25 May 2015 19:55:47 +0000 Subject: Shared inbox? In-Reply-To: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> References: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> Message-ID: <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> May 25 2015 1:01 PM, "Chris Ross" wrote: > I'm running dovecot 2.2.16 on my FreeBSD mail server. I've read information on the wiki about > setting up shared mailboxes, but I want to do something that isn't really coved by the instructions > I was reading there. My son (now 7 years old) has an account on the system, but doesn't use it > directly. But, for things he's interested in like Minecraft, and/or the local zoo, we have given > out his email address in a small number of places. What I would like to set up, both for now while > he's not actually ever reading his email himself, and perhaps even for the future when we teach > and/or allow for that, is for my wife and I to be able to "view" his inbox from our accounts. > > The instructions for setting up shared folders all are written so that they're secondary folders to > all accounts. Is there a way to either (a) configure sharing someone elses Inbox by other accounts, > or (b) setting up a separately configured shared folder to _act_ as the inbox for a single account? > > Thanks. Any suggestions to achieve the above described end goal would be appreciated. > > - Chris I'll assume you've seen these: http://wiki2.dovecot.org/SharedMailboxes/Shared and http://wiki2.dovecot.org/ACL When I set them up that way, I shared the target inbox (we'll call it foo at example.com) to be accessible by user bar at example.com. When I go into my email client, I'll see something like: shared/foo at example.com/INBOX as a folder. To give bar@ access to foo@'s INBOX, I ran something like this: doveadm acl add \ -u foo at example.com \ INBOX user=bar at example.com \ lookup read write write-seen write-deleted insert\ post expunge create delete admin http://wiki2.dovecot.org/Tools/Doveadm/ACL covers the details of what all of the "lookup read..." bits and more do. The boxes don't have to be shared publicly if you use the "private" namespace. Hopefully that helps! Cheers, Drew Csillag From listas at adminlinux.com.br Mon May 25 21:16:20 2015 From: listas at adminlinux.com.br (Listas@Adminlinux) Date: Mon, 25 May 2015 18:16:20 -0300 Subject: Segfault on "doveadm sync -u tcp::port" Message-ID: <55639124.4020509@adminlinux.com.br> Hi! When I run "doveadm sync -u tcp:192.168.0.2:12345", I get this error: http://pastebin.com/NxyyNGJk Only to one specific user... I've tried to repair with "doveadm force-resync", unsuccessfully. Is this known? Ubuntu14.04 + Dovecot-2.2.15. My doveconf -n: http://pastebin.com/25sURwJL Thanks! -- Thiago Henrique From mysqlstudent at gmail.com Tue May 26 01:38:43 2015 From: mysqlstudent at gmail.com (Alex Regan) Date: Mon, 25 May 2015 21:38:43 -0400 Subject: Creating IMAP folders/subfolders with Thunderbird Message-ID: <5563CEA3.2050503@gmail.com> Hi, I have dovecot-2.2.15 on fedora21 with Thunderbird and having some difficulty creating subfolders and deleting folders. Creating subfolders results in a folder at the root with a caret instead of a slash called folder^subfolder with an entry in .subscriptions with that name, instead of a new folder under the directory. I'm also unable to delete any folders. The command from within Thunderbird is just completely ignored. Any idea what's happening here? Thanks, Alex From i.thierack at iwt-promotion.com Tue May 26 14:27:32 2015 From: i.thierack at iwt-promotion.com (Ingo Thierack) Date: Tue, 26 May 2015 16:27:32 +0200 Subject: sieve: extract first chars of body in sieve script Message-ID: <556482D4.20407@iwt-promotion.com> Hello, is it possible to extract the first x chars of the body of an message to forward it, or the complete body inside an sieve script. I want to forward specific mails via telegram-messenger. No Problem to extract the "from" and the "subject". if header :matches "Subject" "*" { set "subject" "${1}"; } if header :matches "from" "*" { set "from" "${1}"; } pipe :copy :try "notify.sh" [ "(${from})# ${subject}" ]; But is it also possible for the body? If I can only pass the complete body, I can cut out the first x chars inside the notify-script. Can't find any hint for that. Regards Ingo From ronleach at tesco.net Tue May 26 14:37:39 2015 From: ronleach at tesco.net (Ron Leach) Date: Tue, 26 May 2015 15:37:39 +0100 Subject: FREAK/Logjam, and SSL protocols to use Message-ID: <55648533.8070408@tesco.net> List, good afternoon, I was reading up on a TLS Diffie Hellman protocol weakness described here https://weakdh.org/sysadmin.html which is similar to the earlier FREAK attack, and can result in downgrade of cipher suites. Part of the solution workaround that the researchers describe for Dovecot here https://weakdh.org/sysadmin.html includes altering DH parameters length to 2048, and re-specifying the allowable cipher suites - they give their suggestion. But the researchers make no comment on the allowable protocols that, in Dovecot, are controlled in 10-ssl.conf in the stanza headed # SSL protocols to use #ssl_protocols = !SSLv2 I couldn't find any advice on up-to-date SSL/TLS protocol settings in the Dovecot wiki, either (I looked in http://wiki2.dovecot.org/SSL/DovecotConfiguration ). At the moment our installation is using only the default settings for allowed protocols; I'm not sure what those defaults might be, but our 10-ssl.conf may only be disallowing SSLv2, if the 'example' entry is the default. We're running 2.2.15, and are due to upgrade to 2.2.18 shortly; quite possibly the defaults differ in a more recent release. What SSL protocols do folk on the list recommend should be allowed in Dovecot these days? (Actually, I mean which protocols really 'must' be disallowed?) regards, Ron From stephan at rename-it.nl Tue May 26 15:02:33 2015 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 26 May 2015 17:02:33 +0200 Subject: sieve: extract first chars of body in sieve script In-Reply-To: <556482D4.20407@iwt-promotion.com> References: <556482D4.20407@iwt-promotion.com> Message-ID: <55648B09.5030401@rename-it.nl> On 5/26/2015 4:27 PM, Ingo Thierack wrote: > Hello, > > is it possible to extract the first x chars of the body of an message to > forward it, > or the complete body inside an sieve script. > > I want to forward specific mails via telegram-messenger. No Problem to > extract > the "from" and the "subject". > > if header :matches "Subject" "*" { > set "subject" "${1}"; > } > if header :matches "from" "*" { > set "from" "${1}"; > } > > pipe :copy :try "notify.sh" [ "(${from})# ${subject}" ]; > > But is it also possible for the body? That would require this feature: https://tools.ietf.org/html/rfc5703#section-7 Unfortunately, it is currently not implemented. I am currently doing some groundwork for that, but I have no idea when that will be done. > If I can only pass the complete body, I > can cut out the first x chars inside the notify-script. That would be an option. Regards, Stephan. From ronleach at tesco.net Tue May 26 15:04:22 2015 From: ronleach at tesco.net (Ron Leach) Date: Tue, 26 May 2015 16:04:22 +0100 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <55648533.8070408@tesco.net> References: <55648533.8070408@tesco.net> Message-ID: <55648B76.6050807@tesco.net> On 26/05/2015 15:37, Ron Leach wrote: > > I was reading up on a TLS Diffie Hellman protocol weakness described here > > https://weakdh.org/sysadmin.html > Sorry, wrong link. The weakness is described at the front page here https://weakdh.org/ The page with the Dovecot advice was correct: > > https://weakdh.org/sysadmin.html > regards, Ron From rs at sys4.de Tue May 26 15:08:49 2015 From: rs at sys4.de (Robert Schetterer) Date: Tue, 26 May 2015 17:08:49 +0200 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <55648533.8070408@tesco.net> References: <55648533.8070408@tesco.net> Message-ID: <55648C81.3000006@sys4.de> Am 26.05.2015 um 16:37 schrieb Ron Leach: > # SSL protocols to use > #ssl_protocols = !SSLv2 you should use ssl_protocols = !SSLv2 you may use ssl_protocols = !SSLv2 !SSLv3 if you havent any clients that can only work on SSLv3 i.e outdated Netscape etc Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstra?e 15, 81669 M?nchen Sitz der Gesellschaft: M?nchen, Amtsgericht M?nchen: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein From manu at netbsd.org Tue May 26 15:18:58 2015 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 26 May 2015 15:18:58 +0000 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <55648533.8070408@tesco.net> References: <55648533.8070408@tesco.net> Message-ID: <20150526151858.GF19722@homeworld.netbsd.org> On Tue, May 26, 2015 at 03:37:39PM +0100, Ron Leach wrote: > What SSL protocols do folk on the list recommend should be allowed in > Dovecot these days? (Actually, I mean which protocols really 'must' be > disallowed?) I use this: ssl_protocols = !SSLv2 !SSLv3 ssl_cipher_list = ECDH at STRENGTH:DH at STRENGTH:HIGH:!RC4:!MD5:!DES:!aNULL:!eNULL ssl_dh_parameters_length = 4096 Kissing SSLv3 good bye did not cause harm to clients. Next to be phased out is 3DES which accounts for 0.25% o the connexions according to the logs. I suspect the offending clients could do better. -- Emmanuel Dreyfus manu at netbsd.org From symbiat at gmail.com Tue May 26 17:38:04 2015 From: symbiat at gmail.com (Ajai Khattri) Date: Tue, 26 May 2015 13:38:04 -0400 Subject: Can email be shared between POP3 and shell account access? In-Reply-To: References: <5iqrla9kqmskifpi8mprfmh2sq0vea17ql@4ax.com> <6F79A41C-CD29-4D67-9676-39F22B45A85B@pettijohn-web.com> Message-ID: I have mutt configured to login via IMAP... On Fri, May 22, 2015 at 6:24 PM, Steve Matzura wrote: > On Thu, 21 May 2015 19:21:41 -0500, you wrote: > > > > >On May 21, 2015, at 9:25 AM, Steve Matzura wrote: > > > >> I needed to receive a message to admin at mydomain.com. I waited and > >> waited for it, but it never showed up in the admin mailbox on the > >> mydomain.com machine. So I looked for the admin account in > >> /etc/dovecot.passwd, and there it was, plain-text password and all! I > >> set up a retrieval system for the admin POP3 account and got all the > >> messages I was waiting for, not to mention about 20 that I wasn't, > >> which were obviously old and discardable. This got me to thinking, is > >> it possible to have an account accessible not just via POP3 from the > >> ouside, but through mail/mailx on the inside, and if so, how is that > >> set up? > > > >If you have root or sudo then all you need is mutt. > > > ># mutt -f /path/to/Maildir > > H'mm. All I need is mutt. There's a song in there somewhere. Just > looked it up and will install forthwith. All of this going to further > prove my depth of un-knowledge about this stuff. THANKS! > -- Aj. Facebook Enoland From sergei.franco at gmail.com Tue May 26 23:23:12 2015 From: sergei.franco at gmail.com (Sergei Franco) Date: Wed, 27 May 2015 11:23:12 +1200 Subject: maildir, how to always add size tags to filename (,S=). Message-ID: Hi There, I have thoroughly read through documentation and wiki and I simply cannot find how to force dovecot to always add the size tag to the filename when using maildir format. We are using dovecot 2.2.9 (whatever comes from ubuntu 14.04 LTS). We are using dovecot in conjunction with exim with maidirsize file enabled and maildir quotas. For example exim has following configuration: maildir_tag = ,S=$message_size Which adds the size to the filename of the message. This has enormous (orders of magnitude) performance benefit as to calculate quota one does not need to stat every single file (only getents for each directory). I have wrote a script that adds the size tag retrospectively to files it didn't had it, and rebuild the dovecot index/cache if changes where made. This worked fine. Behaviour of dovecot is very peculiar: it generally adds the ,S= tag (along with ,W= tag) but occasionally it does not (I wander if it depends on the IMAP client). Is there anyway to force dovecot to add size tag to the files (like with exim example above)? Thank you very much. Sergei. From gedalya at gedalya.net Wed May 27 04:22:59 2015 From: gedalya at gedalya.net (Gedalya) Date: Wed, 27 May 2015 00:22:59 -0400 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <55648533.8070408@tesco.net> References: <55648533.8070408@tesco.net> Message-ID: <556546A3.5010208@gedalya.net> On 05/26/2015 10:37 AM, Ron Leach wrote: > > https://weakdh.org/sysadmin.html > > includes altering DH parameters length to 2048, and re-specifying the > allowable cipher suites - they give their suggestion. It looks like there is an error on this page regarding regeneration. In current dovecots ssl_parameters_regenerate defaults to zero, and this means regeneration is disabled. The old default was 168 hours (1 week). The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is confusing and could be understood to mean that the current default is one week. To enable regeneration you can manually set: ssl_parameters_regenerate = 60 days or: ssl_parameters_regenerate = 1 weeks From rick at havokmon.com Wed May 27 13:55:13 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 27 May 2015 08:55:13 -0500 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <556546A3.5010208@gedalya.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> Message-ID: <20150527085513.Horde.NWnmclIg7By6rN6jjBfHvg2@www.vfemail.net> Quoting Gedalya : > On 05/26/2015 10:37 AM, Ron Leach wrote: >> https://weakdh.org/sysadmin.html >> >> includes altering DH parameters length to 2048, and re-specifying the >> allowable cipher suites - they give their suggestion. > > It looks like there is an error on this page regarding regeneration. In > current dovecots ssl_parameters_regenerate defaults to zero, and this > means regeneration is disabled. The old default was 168 hours (1 week). > The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is > confusing and could be understood to mean that the current default is > one week. > To enable regeneration you can manually set: > ssl_parameters_regenerate = 60 days > or:ssl_parameters_regenerate = 1 weeks This is really cool and all, but for a low power proxy, it takes a good 5 minutes to regenerate the dh params, and Dovecot listens the entire time. If the socket were closed during regeneration, then a (basic) front-end load balancer wouldn't still push connections to that proxy during regen. Rick From darix at opensu.se Wed May 27 14:14:00 2015 From: darix at opensu.se (Marcus =?UTF-8?B?UsO8Y2tlcnQ=?=) Date: Wed, 27 May 2015 16:14:00 +0200 Subject: /etc/ssl/certs/dovecot.pem erased by OpenSuse's update mechanism In-Reply-To: <54E1B3BC.29818.34369A@WGross.uni-hd.de> References: <54E1B3BC.29818.34369A@WGross.uni-hd.de> Message-ID: <20150527161400.77ee304f@tengu.suse.de> On Mon, 16 Feb 2015 10:09:16 +0100 "Wolfgang Gross" wrote: > Hi, > > this is not a genuine Dovecot bug, more a nuisance. > It applies to OpenSuse 13.2 but maybe also to other Linux's. > > The standard installation of Dovecot (especially 10-ssl.conf) places > the certificate dovecot.pem in /etc/ssl/certs. > Sometimes during updates does OpenSuse renew all certificates > in /etc/ssl/certs and erases dovecot.pem. This blocks further access > to the mailbox. > > I found a similar report here: > https://bbs.archlinux.de/viewtopic.php?id=27288 > > Workaround: Move dovecot.pem to another directory and change > 10-ssl.conf accordingly. This is *not* our update mechanism. This is update-ca-certificates, which will wipe /etc/ssl/certs/ when it is called. This can happen to you on any distro using it. My recommendation is to use /etc/ssl/private/ for all service related files. Certs and keys. HTH darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org From gedalya at gedalya.net Wed May 27 15:35:09 2015 From: gedalya at gedalya.net (Gedalya) Date: Wed, 27 May 2015 11:35:09 -0400 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <20150527085513.Horde.NWnmclIg7By6rN6jjBfHvg2@www.vfemail.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> <20150527085513.Horde.NWnmclIg7By6rN6jjBfHvg2@www.vfemail.net> Message-ID: <5565E42D.5010704@gedalya.net> On 05/27/2015 09:55 AM, Rick Romero wrote: > Quoting Gedalya : > >> On 05/26/2015 10:37 AM, Ron Leach wrote: >>> https://weakdh.org/sysadmin.html >>> >>> includes altering DH parameters length to 2048, and re-specifying the >>> allowable cipher suites - they give their suggestion. >> >> It looks like there is an error on this page regarding regeneration. In >> current dovecots ssl_parameters_regenerate defaults to zero, and this >> means regeneration is disabled. The old default was 168 hours (1 week). >> The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is >> confusing and could be understood to mean that the current default is >> one week. >> To enable regeneration you can manually set: >> ssl_parameters_regenerate = 60 days >> or:ssl_parameters_regenerate = 1 weeks > > This is really cool and all, but for a low power proxy, it takes a good 5 > minutes to regenerate the dh params, and Dovecot listens the entire time. > > If the socket were closed during regeneration, then a (basic) front-end > load balancer wouldn't still push connections to that proxy during regen. > > Rick I wonder if what is taking 5 minutes is CPU usage or entropy starvation. Might be worth looking into. However the entire reason why I wrote this comment was to correct the mistaken line saying "#regenerates every week". It is not at this point emphasized anywhere, including on weakdh.org, that it is actually of high importance to regenerate your DH parameters frequently. This has been discussed extensively e.g. within the exim project and other places, and on dovecot too the default was changed to not regenerate. It seems that people are mostly just saying you should have locally generated parameters unique to your site. But to address your point, if this feature is deemed worth maintaining, it seems it would be best to spawn a thread working on the new parameters in the background and replacing them when ready. Otherwise dovecot can just implement a dh_parameters config option like all other daemons and you can maintain that externally as you please. But we're supposed to be focusing on EC anyway :-) From rick at havokmon.com Wed May 27 15:56:41 2015 From: rick at havokmon.com (Rick Romero) Date: Wed, 27 May 2015 10:56:41 -0500 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <5565E42D.5010704@gedalya.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> <20150527085513.Horde.NWnmclIg7By6rN6jjBfHvg2@www.vfemail.net> <5565E42D.5010704@gedalya.net> Message-ID: <20150527105641.Horde.apyLkpRoB9DzwoigBTSwxw3@www.vfemail.net> Quoting Gedalya : > On 05/27/2015 09:55 AM, Rick Romero wrote: >> Quoting Gedalya : >> >>> On 05/26/2015 10:37 AM, Ron Leach wrote: >>>> https://weakdh.org/sysadmin.html >>>> >>>> includes altering DH parameters length to 2048, and re-specifying the >>>> allowable cipher suites - they give their suggestion. >>> >>> It looks like there is an error on this page regarding regeneration. In >>> current dovecots ssl_parameters_regenerate defaults to zero, and this >>> means regeneration is disabled. The old default was 168 hours (1 week). >>> The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is >>> confusing and could be understood to mean that the current default is >>> one week. >>> To enable regeneration you can manually set: >>> ssl_parameters_regenerate = 60 days >>> or:ssl_parameters_regenerate = 1 weeks >> >> This is really cool and all, but for a low power proxy, it takes a good 5 >> minutes to regenerate the dh params, and Dovecot listens the entire time. >> >> If the socket were closed during regeneration, then a (basic) front-end >> load balancer wouldn't still push connections to that proxy during regen. >> >> Rick > > I wonder if what is taking 5 minutes is CPU usage or entropy starvation. > Might be worth looking into. I'd say CPU usage - I have two identical VMs for dovecot proxies, one is hosted on a dual Xeon 5450, the other a dual Opteron 2347HE.? Both hosts are under similar load, but the Xeon host was done within 30 seconds. I assume the Xeon, besides having a faster base CPU frequency, is just better for that sort of workload. I noticed a similar difference when generating params for the web servers, but I did that externally. I assume it'd probably be easier to do the dh_parameters config than to fully disable the socket during regen.. Rick From gedalya at gedalya.net Wed May 27 16:10:37 2015 From: gedalya at gedalya.net (Gedalya) Date: Wed, 27 May 2015 12:10:37 -0400 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <20150527105641.Horde.apyLkpRoB9DzwoigBTSwxw3@www.vfemail.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> <20150527085513.Horde.NWnmclIg7By6rN6jjBfHvg2@www.vfemail.net> <5565E42D.5010704@gedalya.net> <20150527105641.Horde.apyLkpRoB9DzwoigBTSwxw3@www.vfemail.net> Message-ID: <5565EC7D.5000303@gedalya.net> On 05/27/2015 11:56 AM, Rick Romero wrote: > Quoting Gedalya : > >> On 05/27/2015 09:55 AM, Rick Romero wrote: >>> Quoting Gedalya : >>> >>>> On 05/26/2015 10:37 AM, Ron Leach wrote: >>>>> https://weakdh.org/sysadmin.html >>>>> >>>>> includes altering DH parameters length to 2048, and re-specifying the >>>>> allowable cipher suites - they give their suggestion. >>>> >>>> It looks like there is an error on this page regarding >>>> regeneration. In >>>> current dovecots ssl_parameters_regenerate defaults to zero, and this >>>> means regeneration is disabled. The old default was 168 hours (1 >>>> week). >>>> The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is >>>> confusing and could be understood to mean that the current default is >>>> one week. >>>> To enable regeneration you can manually set: >>>> ssl_parameters_regenerate = 60 days >>>> or:ssl_parameters_regenerate = 1 weeks >>> >>> This is really cool and all, but for a low power proxy, it takes a good > 5 >>> minutes to regenerate the dh params, and Dovecot listens the entire > time. >>> >>> If the socket were closed during regeneration, then a (basic) front-end >>> load balancer wouldn't still push connections to that proxy during > regen. >>> >>> Rick >> >> I wonder if what is taking 5 minutes is CPU usage or entropy starvation. >> Might be worth looking into. > > I'd say CPU usage - I have two identical VMs for dovecot proxies, one is > hosted on a dual Xeon 5450, the other a dual Opteron 2347HE. Both hosts > are under similar load, but the Xeon host was done within 30 seconds. I > assume the Xeon, besides having a faster base CPU frequency, is just > better > for that sort of workload. > > I noticed a similar difference when generating params for the web > servers, > but I did that externally. > > I assume it'd probably be easier to do the dh_parameters config than to > fully disable the socket during regen.. > > Rick Are the results repeatable? The time it takes openssl to generate new params is, well, literally random. I wish someone could tell me why, but 'certtool --generate-dh-params --bits 2048' (from gnutls) takes just a few seconds, and openssl can take several minutes. And BTW on second thought I think openssl doesn't actually read much from /dev/random but just uses its own PRNG so entropy starvation might not actually apply here. So yea it's just CPU and sheer luck in terms of how quickly it stumbles upon the right primes. As for gnutls - I have no idea how that works, but it's very fast. Anyway I've certainly seen newer Xeons than 5450 take well over 30 seconds to generate 2048-bit dh params. If yours consistently does it in 30 seconds then I'd love to understand how come. From ronleach at tesco.net Wed May 27 16:15:27 2015 From: ronleach at tesco.net (Ron Leach) Date: Wed, 27 May 2015 17:15:27 +0100 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <556546A3.5010208@gedalya.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> Message-ID: <5565ED9F.6060608@tesco.net> On 27/05/2015 05:22, Gedalya wrote: > It looks like there is an error on this page regarding regeneration. > In current dovecots ssl_parameters_regenerate defaults to zero, and > this means regeneration is disabled. The old default was 168 hours (1 > week). > The language on http://wiki2.dovecot.org/SSL/DovecotConfiguration is > confusing and could be understood to mean that the current default is > one week. I'd read that dovecot wiki page, and the weakdh page, and - indeed - formed the impression that the defaults on our server were ok. > To enable regeneration you can manually set: > ssl_parameters_regenerate = 60 days > or: > ssl_parameters_regenerate = 1 weeks I couldn't find an entry in 10-ssl.config that covered regeneration (though our version is 2.2.15 and the current release, 2.2.18, may differ). I created an entry from scratch, with the example you posted but set to 7 days, and placed that in 10-ssl.config . Thank you very much for the advice. regards, Ron From gedalya at gedalya.net Wed May 27 16:20:35 2015 From: gedalya at gedalya.net (Gedalya) Date: Wed, 27 May 2015 12:20:35 -0400 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <5565ED9F.6060608@tesco.net> References: <55648533.8070408@tesco.net> <556546A3.5010208@gedalya.net> <5565ED9F.6060608@tesco.net> Message-ID: <5565EED3.1020200@gedalya.net> On 05/27/2015 12:15 PM, Ron Leach wrote: > > I couldn't find an entry in 10-ssl.config that covered regeneration > (though our version is 2.2.15 and the current release, 2.2.18, may > differ). Yea it's just not there. You can 'discover' these 'hidden' options using doveconf -a, scattered docs, and the source code ;-) From distler at golem.ph.utexas.edu Wed May 27 16:29:18 2015 From: distler at golem.ph.utexas.edu (Jacques Distler) Date: Wed, 27 May 2015 11:29:18 -0500 Subject: FREAK/Logjam, and SSL protocols to use Message-ID: <045B4DA1-54C0-446D-9765-FD40DB70C93A@golem.ph.utexas.edu> >It is not at this point emphasized anywhere, including on weakdh.org, that it is actually of high importance to regenerate your DH parameters frequently. That's not really correct. If you're using a prime of length at least 2048 bits, then the corresponding discrete-log problem is well-beyond the pre-computation ability of the NSA (or anyone else). It is computationally intensive to generate such large primes, p (and corresponding base parameter, g). You need to ensure that p is actually prime (the costly step [1]) and that g is primitive. Which is why most implementations have used shorter (<= 1024 bit) primes. Using shorter primes, and regenerating DH parameters at regular intervals, is only a linear-time improvement. By contrast, generating longer DH parameters (without bothering to regenerate) is an EXPONENTIAL improvement in security. So the best setting is to set ssl_dh_parameters_length as large as feasible ([2] recommends 2048 bits), and NOT to regenerate. [1] http://en.wikipedia.org/wiki/Miller%E2%80%93Rabin_primality_test [2] https://weakdh.org/sysadmin.html -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From gedalya at gedalya.net Wed May 27 16:40:01 2015 From: gedalya at gedalya.net (Gedalya) Date: Wed, 27 May 2015 12:40:01 -0400 Subject: FREAK/Logjam, and SSL protocols to use In-Reply-To: <045B4DA1-54C0-446D-9765-FD40DB70C93A@golem.ph.utexas.edu> References: <045B4DA1-54C0-446D-9765-FD40DB70C93A@golem.ph.utexas.edu> Message-ID: <5565F361.90704@gedalya.net> On 05/27/2015 12:29 PM, Jacques Distler wrote: >> It is not at this point emphasized anywhere, including on weakdh.org, that it is actually of high importance to regenerate your DH parameters frequently. > That's not really correct. > > If you're using a prime of length at least 2048 bits, then the corresponding discrete-log problem is well-beyond the pre-computation ability of the NSA (or anyone else). > > It is computationally intensive to generate such large primes, p (and corresponding base parameter, g). You need to ensure that p is actually prime (the costly step [1]) and that g is primitive. > > Which is why most implementations have used shorter (<= 1024 bit) primes. > > Using shorter primes, and regenerating DH parameters at regular intervals, is only a linear-time improvement. By contrast, generating longer DH parameters (without bothering to regenerate) is an EXPONENTIAL improvement in security. > > So the best setting is to set ssl_dh_parameters_length as large as feasible ([2] recommends 2048 bits), and NOT to regenerate. > > Well that's certainly what I meant to say. By referring to weakdh.org (and placing my message in the context of this entire thread) I was at the very least subtly alluding to the recommendation loudly stated there to use at least 2048 bits, which has been the recommendation for a very long time, anyway. The implementation in the various TLS libraries was never a very good reference point, to put it mildly. Some bad choices have been made presumably for "pragmatic" (= lazy) reasons and the harm is that these things are not transparent to most people. But when you write NOT to regenerate, are you saying that using larger primes makes regenerating unnecessary, or are you telling us that it's somehow harmful? From distler at golem.ph.utexas.edu Wed May 27 18:58:21 2015 From: distler at golem.ph.utexas.edu (Jacques Distler) Date: Wed, 27 May 2015 13:58:21 -0500 Subject: FREAK/Logjam, and SSL protocols to use Message-ID: <0AF53DCB-5407-4B0B-BC36-9D2E64201CED@golem.ph.utexas.edu> >But when you write NOT to regenerate, are you saying that using larger primes makes regenerating unnecessary, or are you telling us that it's somehow harmful? For a given computational effort, you get the most bang-for-the-buck by choosing large parameters (and checking very carefully that they are "safe") rather than smaller parameters (and/or checking them less carefully) which you then regenerate. Every time you regenerate, there's a small (but finite) probability that the new parameters are actually unsafe. You'd do better using those CPU cycles to improve the proof that your original set of parameters was safe (admittedly, no one actually does this), rather than generating a new set. Remember, the DH parameters (p,g) are NOT secret; they are transmitted in the clear everytime. As long as you're using Ephemeral Diffie-Hellman (choosing new exponents, a and b, for each session) with large safe DH parameters, it's hard to think of a threat model where you improve the security AT ALL by regenerating the DH parameters. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From distler at golem.ph.utexas.edu Wed May 27 19:06:15 2015 From: distler at golem.ph.utexas.edu (Jacques Distler) Date: Wed, 27 May 2015 14:06:15 -0500 Subject: FREAK/Logjam, and SSL protocols to use Message-ID: <8AD8546B-517D-4A66-98A9-9F56AED75D72@golem.ph.utexas.edu> > For a given computational effort, you get the most bang-for-the-buck by > choosing large parameters (and checking very carefully that they are > "safe") rather than smaller parameters (and/or checking them less > carefully) which you then regenerate. This discussion (on the OpenSSH mailing list) http://marc.info/?t=143221614200001 may be helpful to those thinking about this. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From cross+dovecot at distal.com Thu May 28 02:57:52 2015 From: cross+dovecot at distal.com (Chris Ross) Date: Wed, 27 May 2015 22:57:52 -0400 Subject: Shared inbox? In-Reply-To: <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> References: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> Message-ID: On May 25, 2015, at 15:55 , andy at thecsillags.com wrote: > I'll assume you've seen these: > http://wiki2.dovecot.org/SharedMailboxes/Shared and http://wiki2.dovecot.org/ACL Yup, I'd looked at those, but still left not understanding all of the options. And you gave a doveadm command, which I appreciate very much, but surprises me since the materials in the ACL wiki page are all about file contents. But I'll try your command. > When I set them up that way, I shared the target inbox (we'll call it foo at example.com) to be accessible by user bar at example.com. When I go into my email client, I'll see something like: shared/foo at example.com/INBOX as a folder. When you say "set them up that way", do you mean following the example config at SharedMailboxes/Shared ? In that example, a mail_location is set at the outer level, which I think I don't want to do, and overridden in the shared namespace. I'm assuming I don't want either of those, or at least that's what I'm going to try first. A question I have given your example above, is, did you set mail_shared_explicit_inbox ? It's only briefly described what that effects, so I'm not sure which setting (on or off) will cause the shared/foo at example.com/INBOX you describe seeing. I assume on, so I've turned it on. > To give bar@ access to foo@'s INBOX, I ran something like this: > > doveadm acl add \ > -u foo at example.com \ > INBOX user=bar at example.com \ > lookup read write write-seen write-deleted insert\ > post expunge create delete admin > > http://wiki2.dovecot.org/Tools/Doveadm/ACL covers the details of what all of the "lookup read..." bits and more do. > > The boxes don't have to be shared publicly if you use the "private" namespace. By "use the "private" namespace", do you mean specifying a separate namespace block in the config, like the one is declared in the example at http://wiki2.dovecot.org/SharedMailboxes/Shared ? If so, that's what I've done. If there is a way to have a single namespace declared that's both private and shared, I don't see that, so I have two as presented on that wiki page. And did/do you have "subscriptions=no" and "list=children" defined? Again, as their only briefly described, I'm not 100% sure I understand the difference in behavior I'll see with different settings. And I'm pretty sure I read that there are ways to have mailboxes shared such that who has or hasn't read contents of the mailboxes is stored within the mailbox, and also ways to have it stored per reading user. I'm not sure which I'll be seeing/using here. Thanks. I'll ask more questions if I have more questions after a little trial and error on my own server. :-) Thanks for your help! - Chris From cross+dovecot at distal.com Thu May 28 04:48:09 2015 From: cross+dovecot at distal.com (Chris Ross) Date: Thu, 28 May 2015 00:48:09 -0400 Subject: Shared inbox? In-Reply-To: References: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> Message-ID: <01A37FE9-E8E9-4A7D-9B28-61700701C8A9@distal.com> On May 27, 2015, at 22:57 , Chris Ross wrote: > On May 25, 2015, at 15:55 , andy at thecsillags.com wrote: >> When I set them up that way, I shared the target inbox (we'll call it foo at example.com) to be accessible by user bar at example.com. When I go into my email client, I'll see something like: shared/foo at example.com/INBOX as a folder. > > When you say "set them up that way", do you mean following the example config at SharedMailboxes/Shared ? In that example, a mail_location is set at the outer level, which I think I don't want to do, and overridden in the shared namespace. I'm assuming I don't want either of those, or at least that's what I'm going to try first. > > A question I have given your example above, is, did you set mail_shared_explicit_inbox ? It's only briefly described what that effects, so I'm not sure which setting (on or off) will cause the shared/foo at example.com/INBOX you describe seeing. I assume on, so I've turned it on. > >> The boxes don't have to be shared publicly if you use the "private" namespace. > > By "use the "private" namespace", do you mean specifying a separate namespace block in the config, like the one is declared in the example at http://wiki2.dovecot.org/SharedMailboxes/Shared ? If so, that's what I've done. If there is a way to have a single namespace declared that's both private and shared, I don't see that, so I have two as presented on that wiki page. Okay. Lots of trial and error and error, and I at least have some configuration that I think includes pieces about shared mailboxes and ACLs, and it's actually up and running. But, I'm not seeing the shared folder. I was getting a lot of errors about separators (namespace configuration error: All list=yes namespaces must use the same separator) and prefixes (namespace configuration error: list=yes requires prefix=/ not to start with separator), but have it working now. The namespace private block from the example at http://wiki2.dovecot.org/SharedMailboxes/Public caused many of those problems, and I didn't have a separator configured at all (ie, was still commented out) in my main namespace, so that apparently conflicted with the "separator = /" in the shared namespace. But, all that resolved. The config file I've added into conf.d/15-shared.conf is: namespace { type = shared separator = / prefix = shared/%%u/ location = maildir:/home/%%n/Maildir:INDEX=~/Maildir/shared/%%u:INDEXPVT=~/Maildir/shared/%%u subscriptions = no list = children } mail_shared_explicit_inbox = yes protocol imap { mail_plugins = $mail_plugins acl imap_acl } plugin { acl = vfile } This comes from the aforementioned wiki page. I ran the doveadm acl command you suggested (though my usernames have no domain part, like your example did have domain parts) Now, while dovecot is back to working, I don't see a shared folder anywhere. "doveadm mailbox list" lists the mailboxes for users, including the INBOX I've tried to configure an ACL to let me share. Clearly I'm missing a piece. Andy, or anyone else, if you see what I've missed, please let me know. I'm not seeing anything back from the server with my mail client under "Subscription List", so I can't choose to subscribe. Maybe I've turned off subscriptions somehow, and it's not auto-subscribing me? Or, maybe having "list=children", and I haven't set up the ACL correctly? I ran: sudo doveadm acl add -u childuser INBOX user=cross lookup read write write-seen write-deleted insert post expunge create delete admin After removing the domains from your example, that's about what you had. I can see the INBOX for childuser with "doveadm mailbox list -u childuser". Ahh. I think I may be on to something. I tried the "doveadm acl add" again, which seems to succeed, but "doveadm acl debug" then doesn't mention anything about me, or my access to childuser's INBOX. Running the "acl add" with "doveadm -D" shows: doveadm(childuser): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled and later: doveadm(childuser): Debug: acl vfile: Global ACLs disabled The first of those looks like it might be a problem. Do you have an "acl_shared_dict" set up in your config anywhere, Andy? Thanks. Any help appreciated. - Chris From mkawada at redhat.com Thu May 28 08:50:58 2015 From: mkawada at redhat.com (mkawada at redhat.com) Date: Thu, 28 May 2015 17:50:58 +0900 Subject: X-UID in mail header is removed? Message-ID: <5566D6F2.2010700@redhat.com> Dear Dovecot Mailing List, I have 2 questions regarding dovecot and would very much appreciate if you would kindly give me helps. Is 'X-UID' no longer shown in mail header via pop3? I have checked with the following 3 version of dovecot packages in order to make sure if a message sent by dovecot have 'X-UID' in its mail header, and the result is that only "dovecot-0.99.11-10.EL4" shows 'X-UID' and the other ones do not(Please see the result written in the second half of this mail). Is 'X-UID' used for checking if a message has already been sent to a user so that he never gets any duplicated mails that are already in his mbox? The reason why I am asking this here is that I am working on a problem that a user sometimes gets a duplicated mail from dovecot if the user keeps mails on the server as its settings. I actually thought this kind of thing would happen if the 'UIDL' gets corrupted or changed for some reasons. However, it is said that this issue never happened in using "dovecot-0.99.11-10.EL4" which shows 'X-UID' in mail header and in using "dovecot-2.0.9-8.el6_6.4" which does not show 'X-UID', the issue happens sometimes. Thus, I would like information indicating what is exactly used to avoid getting any duplicated messages while downloading a message via pop3 from dovecot. Here is the test result I checked: (dovecot-0.99.11-10.EL4) ]# telnet localhost 110 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. +OK dovecot ready. user test +OK pass password +OK Logged in. list +OK 1 messages: 1 786 . RETR 1 +OK 786 octets Return-Path: Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by localhost.localdomain (8.13.1/8.13.1) with ESMTP id t4S4dBcu006057 for ; Thu, 28 May 2015 13:39:11 +0900 Received: (from root at localhost) by localhost.localdomain (8.13.1/8.13.1/Submit) id t4S4dB6u006056 for test; Thu, 28 May 2015 13:39:11 +0900 Date: Thu, 28 May 2015 13:39:11 +0900 From: root Message-Id: <201505280439.t4S4dB6u006056 at localhost.localdomain> To: test at localhost.localdomain Subject: test mail X-IMAPbase: 1432787986 1 Status: O X-UID: 1 <--X-UID shown here Content-Length: 5 X-Keywords: test . UIDL +OK 1 1432787986.1 (dovecot-1.0.7-9.el5_11.4) ]# telnet localhost 110 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. +OK Dovecot ready. user test +OK pass password +OK Logged in. list +OK 1 messages: 1 606 . RETR 1 +OK 606 octets Return-Path: Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by localhost.localdomain (8.13.8/8.13.8) with ESMTP id t4S27gBo007199 for ; Thu, 28 May 2015 11:07:42 +0900 Received: (from root at localhost) by localhost.localdomain (8.13.8/8.13.8/Submit) id t4S27fXd007197 for test; Thu, 28 May 2015 11:07:41 +0900 Date: Thu, 28 May 2015 11:07:41 +0900 From: root Message-Id: <201505280207.t4S27fXd007197 at localhost.localdomain> To: test at localhost.localdomain Subject: testmail test . UIDL +OK 1 0000000155667888 . (dovecot-2.0.9-8.el6_6.4) ]# telnet localhost 110 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user test +OK pass password +OK Logged in. list +OK 1 messages: 1 605 . RETR 1 +OK 605 octets Return-Path: X-Original-To: test Delivered-To: test at localhost.localdomain Received: by localhost.localdomain (Postfix, from userid 0) id 3629F4336A; Wed, 27 May 2015 11:34:39 +0900 (JST) Date: Wed, 27 May 2015 11:34:39 +0900 To: test at localhost.localdomain Subject: test mail User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-Id: <20150527023439.3629F4336A at localhost.localdomain> From: root at localhost.localdomain (root) test . UIDL +OK 1 0000000155652c77 Thanks, Masaharu Kawada From andy at thecsillags.com Thu May 28 13:08:26 2015 From: andy at thecsillags.com (andy at thecsillags.com) Date: Thu, 28 May 2015 13:08:26 +0000 Subject: Shared inbox? In-Reply-To: <01A37FE9-E8E9-4A7D-9B28-61700701C8A9@distal.com> References: <01A37FE9-E8E9-4A7D-9B28-61700701C8A9@distal.com> <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> Message-ID: <35c6455ae08d7cb600ec90426eb802e9@www.thecsillags.com> Chris, I do indeed have an acl_shared_dict set up. That may be the ticket. That makes it so that the IMAP server knows that you have acls on the other mailbox, so it can know to then look in that mailbox to find out precisely what the ACLs are. Cheers, Andy May 28 2015 12:49 AM, "Chris Ross" wrote: > On May 27, 2015, at 22:57 , Chris Ross wrote: > >> On May 25, 2015, at 15:55 , andy at thecsillags.com wrote: >>> When I set them up that way, I shared the target inbox (we'll call it foo at example.com) to be >>> accessible by user bar at example.com. When I go into my email client, I'll see something like: >>> shared/foo at example.com/INBOX as a folder. >> >> When you say "set them up that way", do you mean following the example config at >> SharedMailboxes/Shared ? In that example, a mail_location is set at the outer level, which I think >> I don't want to do, and overridden in the shared namespace. I'm assuming I don't want either of >> those, or at least that's what I'm going to try first. >> >> A question I have given your example above, is, did you set mail_shared_explicit_inbox ? It's only >> briefly described what that effects, so I'm not sure which setting (on or off) will cause the >> shared/foo at example.com/INBOX you describe seeing. I assume on, so I've turned it on. >> >>> The boxes don't have to be shared publicly if you use the "private" namespace. >> >> By "use the "private" namespace", do you mean specifying a separate namespace block in the config, >> like the one is declared in the example at http://wiki2.dovecot.org/SharedMailboxes/Shared ? If so, >> that's what I've done. If there is a way to have a single namespace declared that's both private >> and shared, I don't see that, so I have two as presented on that wiki page. > > Okay. Lots of trial and error and error, and I at least have some configuration that I think > includes pieces about shared mailboxes and ACLs, and it's actually up and running. But, I'm not > seeing the shared folder. > > I was getting a lot of errors about separators (namespace configuration error: All list=yes > namespaces must use the same separator) and prefixes (namespace configuration error: list=yes > requires prefix=/ not to start with separator), but have it working now. The namespace private > block from the example at http://wiki2.dovecot.org/SharedMailboxes/Public caused many of those > problems, and I didn't have a separator configured at all (ie, was still commented out) in my main > namespace, so that apparently conflicted with the "separator = /" in the shared namespace. But, all > that resolved. > > The config file I've added into conf.d/15-shared.conf is: > > namespace { > type = shared > separator = / > prefix = shared/%%u/ > location = maildir:/home/%%n/Maildir:INDEX=~/Maildir/shared/%%u:INDEXPVT=~/Maildir/shared/%%u > subscriptions = no > list = children > } > mail_shared_explicit_inbox = yes > protocol imap { > mail_plugins = $mail_plugins acl imap_acl > } > plugin { > acl = vfile > } > > This comes from the aforementioned wiki page. I ran the doveadm acl command you suggested (though > my usernames have no domain part, like your example did have domain parts) > > Now, while dovecot is back to working, I don't see a shared folder anywhere. "doveadm mailbox list" > lists the mailboxes for users, including the INBOX I've tried to configure an ACL to let me share. > > Clearly I'm missing a piece. Andy, or anyone else, if you see what I've missed, please let me know. > I'm not seeing anything back from the server with my mail client under "Subscription List", so I > can't choose to subscribe. Maybe I've turned off subscriptions somehow, and it's not > auto-subscribing me? Or, maybe having "list=children", and I haven't set up the ACL correctly? > > I ran: > > sudo doveadm acl add -u childuser INBOX user=cross lookup read write write-seen write-deleted > insert post expunge create delete admin > > After removing the domains from your example, that's about what you had. I can see the INBOX for > childuser with "doveadm mailbox list -u childuser". > > Ahh. I think I may be on to something. I tried the "doveadm acl add" again, which seems to succeed, > but "doveadm acl debug" then doesn't mention anything about me, or my access to childuser's INBOX. > Running the "acl add" with "doveadm -D" shows: > > doveadm(childuser): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled > > and later: > > doveadm(childuser): Debug: acl vfile: Global ACLs disabled > > The first of those looks like it might be a problem. Do you have an "acl_shared_dict" set up in > your config anywhere, Andy? > > Thanks. Any help appreciated. > > - Chris From list at airstreamcomm.net Thu May 28 20:24:16 2015 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 28 May 2015 15:24:16 -0500 Subject: OT: Central sieve management Message-ID: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> A bit off topic, but I was wondering if anyone here has a solution for centrally managing sieve for multiple users from a custom web application? We would like to implement pigeonhole sieve on our dovecot cluster, however we need to be able to access user?s sieve configurations from a central location for troubleshooting and support purposes. From uothrawn at yahoo.com Fri May 29 03:57:21 2015 From: uothrawn at yahoo.com (G H) Date: Thu, 28 May 2015 23:57:21 -0400 Subject: Shared inbox? In-Reply-To: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> References: <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> Message-ID: <98C5A687-AF3F-455A-9CF6-F2CF1B59574D@yahoo.com> On May 25, 2015 12:59:45 PM EDT, Chris Ross wrote: > >I'm running dovecot 2.2.16 on my FreeBSD mail server. I've read >information on the wiki about setting up shared mailboxes, but I want >to do something that isn't really coved by the instructions I was >reading there. My son (now 7 years old) has an account on the system, >but doesn't use it directly. But, for things he's interested in like >Minecraft, and/or the local zoo, we have given out his email address in >a small number of places. What I would like to set up, both for now >while he's not actually ever reading his email himself, and perhaps >even for the future when we teach and/or allow for that, is for my wife >and I to be able to "view" his inbox from our accounts. > >The instructions for setting up shared folders all are written so that >they're secondary folders to all accounts. Is there a way to either >(a) configure sharing someone elses Inbox by other accounts, or (b) >setting up a separately configured shared folder to _act_ as the inbox >for a single account? > >Thanks. Any suggestions to achieve the above described end goal would >be appreciated. > > - Chris Look in to dovecot's master user feature as well. From jost+lists at dimejo.at Fri May 29 10:17:13 2015 From: jost+lists at dimejo.at (Alex JOST) Date: Fri, 29 May 2015 12:17:13 +0200 Subject: Creating IMAP folders/subfolders with Thunderbird In-Reply-To: <5563CEA3.2050503@gmail.com> References: <5563CEA3.2050503@gmail.com> Message-ID: <55683CA9.1020704@dimejo.at> Am 26.05.2015 um 03:38 schrieb Alex Regan: > Hi, > > I have dovecot-2.2.15 on fedora21 with Thunderbird and having some > difficulty creating subfolders and deleting folders. > > Creating subfolders results in a folder at the root with a caret instead > of a slash called folder^subfolder with an entry in .subscriptions with > that name, instead of a new folder under the directory. > > I'm also unable to delete any folders. The command from within > Thunderbird is just completely ignored. > > Any idea what's happening here? > > Thanks, > Alex This seems to be a bug in Thunderbird: https://bugzilla.mozilla.org/show_bug.cgi?id=773579 -- Alex JOST From rblayzor.bulk at inoc.net Fri May 29 12:46:26 2015 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Fri, 29 May 2015 08:46:26 -0400 Subject: OT: Central sieve management In-Reply-To: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> References: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> Message-ID: <929F4843-131A-4CED-BD40-AA11FDA0A127@inoc.net> On May 28, 2015, at 4:24 PM, list at airstreamcomm.net wrote: > > A bit off topic, but I was wondering if anyone here has a solution for centrally managing sieve for multiple users from a custom web application? We would like to implement pigeonhole sieve on our dovecot cluster, however we need to be able to access user?s sieve configurations from a central location for troubleshooting and support purposes. Couldn?t this be done with ManageSieve and a master login? -- Robert inoc.net!rblayzor Jabber: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu From CMarcus at Media-Brokers.com Fri May 29 13:03:58 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 29 May 2015 09:03:58 -0400 Subject: Enabling Master User for migration Message-ID: <556863BE.90402@Media-Brokers.com> Hi all, Ok, just need a quick-n-dirty way to enable a master user for a migration... Reading the docs, I'm not sure what the bare minimum is I need to do this... At: http://wiki2.dovecot.org/Authentication/MasterUsers The Example config shows: auth_master_user_separator = * passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } passdb { driver = shadow } userdb { driver = passwd } But, if I'm using a non-real user (just need a master user that can log in as all other users for the migration), do I need all three of these? Or just the first two? Or, just the first one? I think I need the first two and not the last one, but am unsure. This is on a production system so don't want to break things, and I'm unfamiliar with using multiple passdb's... Thanks, Charles From mysqlstudent at gmail.com Fri May 29 13:16:04 2015 From: mysqlstudent at gmail.com (Alex Regan) Date: Fri, 29 May 2015 09:16:04 -0400 Subject: Creating IMAP folders/subfolders with Thunderbird In-Reply-To: <55683CA9.1020704@dimejo.at> References: <5563CEA3.2050503@gmail.com> <55683CA9.1020704@dimejo.at> Message-ID: <55686694.40505@gmail.com> Hi, On 05/29/2015 06:17 AM, Alex JOST wrote: > Am 26.05.2015 um 03:38 schrieb Alex Regan: >> Hi, >> >> I have dovecot-2.2.15 on fedora21 with Thunderbird and having some >> difficulty creating subfolders and deleting folders. >> >> Creating subfolders results in a folder at the root with a caret instead >> of a slash called folder^subfolder with an entry in .subscriptions with >> that name, instead of a new folder under the directory. >> >> I'm also unable to delete any folders. The command from within >> Thunderbird is just completely ignored. >> >> Any idea what's happening here? >> >> Thanks, >> Alex > > This seems to be a bug in Thunderbird: > https://bugzilla.mozilla.org/show_bug.cgi?id=773579 Wow, that sucks. Open since 2012. Thanks for the info. Thanks, Alex From dominik at dominikbreu.de Fri May 29 13:25:35 2015 From: dominik at dominikbreu.de (Dominik Breu) Date: Fri, 29 May 2015 15:25:35 +0200 Subject: Enabling Master User for migration In-Reply-To: <556863BE.90402@Media-Brokers.com> References: <556863BE.90402@Media-Brokers.com> Message-ID: <1432905935.4021.6.camel@dominikbreu.de> Hello Charles, the bare minimum ist just the first passdb entry auth_master_user_separator = * passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } this will do the trick. greetings dominik Am Freitag, den 29.05.2015, 09:03 -0400 schrieb Charles Marcus: > Hi all, > > Ok, just need a quick-n-dirty way to enable a master user for a migration... > > Reading the docs, I'm not sure what the bare minimum is I need to do this... > > At: http://wiki2.dovecot.org/Authentication/MasterUsers > > The Example config shows: > > auth_master_user_separator = * > passdb { > driver = passwd-file > args = /etc/dovecot/passwd.masterusers > master = yes > pass = yes > } > passdb { > driver = shadow > } > userdb { > driver = passwd > } > > But, if I'm using a non-real user (just need a master user that can log > in as all other users for the migration), do I need all three of these? > Or just the first two? Or, just the first one? > > I think I need the first two and not the last one, but am unsure. This > is on a production system so don't want to break things, and I'm > unfamiliar with using multiple passdb's... > > Thanks, > > Charles From CMarcus at Media-Brokers.com Fri May 29 13:36:39 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 29 May 2015 09:36:39 -0400 Subject: Enabling Master User for migration In-Reply-To: <1432905935.4021.6.camel@dominikbreu.de> References: <556863BE.90402@Media-Brokers.com> <1432905935.4021.6.camel@dominikbreu.de> Message-ID: <55686B67.5090809@Media-Brokers.com> On 5/29/2015 9:25 AM, Dominik Breu wrote: > Hello Charles, > > the bare minimum ist just the first passdb entry > auth_master_user_separator = * > passdb { > driver = passwd-file > args = /etc/dovecot/passwd.masterusers > master = yes > pass = yes > } > > this will do the trick. Thanks... so, just checking, was it normal to see a whole bunch of these errors for a few seconds after reloading the dovecot config: Error: Auth worker sees different passdbs/userdbs than auth server. Maybe config just changed and this goes away automatically? ? As the error shows, they did go away by themselves after about 10+ seconds... Thanks again! Charles From CMarcus at Media-Brokers.com Fri May 29 13:55:32 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 29 May 2015 09:55:32 -0400 Subject: Enabling Master User for migration In-Reply-To: <1432905935.4021.6.camel@dominikbreu.de> References: <556863BE.90402@Media-Brokers.com> <1432905935.4021.6.camel@dominikbreu.de> Message-ID: <55686FD4.2070008@Media-Brokers.com> On 5/29/2015 9:25 AM, Dominik Breu wrote: > Hello Charles, > > the bare minimum ist just the first passdb entry > auth_master_user_separator = * > passdb { > driver = passwd-file > args = /etc/dovecot/passwd.masterusers > master = yes > pass = yes > } > > this will do the trick. Ok, this isn't working, I'm getting auth failed when testing either with telnet or doing it in Thunderbird. I tried both using the htpasswd method for generating the password in the passwd file, and when that failed, I also tried just putting the password in the file in plain text... So... what am I missing/doing wrong? Thanks, Charles From list at airstreamcomm.net Fri May 29 14:01:57 2015 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Fri, 29 May 2015 09:01:57 -0500 Subject: OT: Central sieve management In-Reply-To: <929F4843-131A-4CED-BD40-AA11FDA0A127@inoc.net> References: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> <929F4843-131A-4CED-BD40-AA11FDA0A127@inoc.net> Message-ID: > On May 29, 2015, at 7:46 AM, Robert Blayzor wrote: > > On May 28, 2015, at 4:24 PM, list at airstreamcomm.net wrote: >> >> A bit off topic, but I was wondering if anyone here has a solution for centrally managing sieve for multiple users from a custom web application? We would like to implement pigeonhole sieve on our dovecot cluster, however we need to be able to access user?s sieve configurations from a central location for troubleshooting and support purposes. > > > Couldn?t this be done with ManageSieve and a master login? > > -- > Robert > inoc.net!rblayzor > Jabber: rblayzor.AT.inoc.net > PGP Key: 78BEDCE1 @ pgp.mit.edu > Correct me if I am wrong, isn?t ManageSieve just a protocol? We were looking for a library or prebuilt tool that would talk to ManageSieve and could then hack into our in-house management application. From CMarcus at Media-Brokers.com Fri May 29 14:14:17 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 29 May 2015 10:14:17 -0400 Subject: Enabling Master User for migration In-Reply-To: <55686FD4.2070008@Media-Brokers.com> References: <556863BE.90402@Media-Brokers.com> <1432905935.4021.6.camel@dominikbreu.de> <55686FD4.2070008@Media-Brokers.com> Message-ID: <55687439.6050404@Media-Brokers.com> Apologies - maybe doveconf -n shows a hint at the problem? Maybe it has to do with I'm using the default_realm? But I've tried adding the user as both the local part, and with the fqdn... > # doveconf -n > # 2.2.15: /etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.6 (3e924b1b6c5c+) > # OS: Linux 3.10.41-gentoo-r1 x86_64 Gentoo Base System release 2.2 > auth_default_realm = media-brokers.com > auth_master_user_separator = * > auth_mechanisms = plain login > deliver_log_format = msgid=%m: subject=%s: %$ > first_valid_uid = 200 > last_valid_uid = 200 > listen = * > login_log_format_elements = user=<%u> method=%m %c rip=%r lport=%{lport} > mail_attachment_dir = /var/vmail/%d/attachments > mail_gid = vmail > mail_location = maildir:~/Maildir > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date ihave > namespace inbox { > inbox = yes > location = > mailbox "Deleted Items" { > special_use = \Trash > } > mailbox Drafts { > special_use = \Drafts > } > mailbox Junk { > special_use = \Junk > } > mailbox "Junk E-mail" { > special_use = \Junk > } > mailbox Sent { > special_use = \Sent > } > mailbox "Sent Messages" { > special_use = \Sent > } > mailbox Trash { > special_use = \Trash > } > prefix = > separator = . > } > passdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > passdb { > args = /etc/dovecot/passwd.masterusers > driver = passwd-file > master = yes > pass = yes > } > plugin { > sieve = ~/.dovecot.sieve > sieve_dir = ~/sieve > } > protocols = imap lmtp > service auth-worker { > user = $default_internal_user > } > service auth { > unix_listener /var/spool/postfix/private/auth { > group = postfix > mode = 0660 > user = postfix > } > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > } > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > service_count = 0 > } > service imap { > process_limit = 2048 > } > ssl = required > ssl_cert = ssl_key = userdb { > args = /etc/dovecot/dovecot-sql.conf.ext > driver = sql > } > protocol smtp { > auth_verbose = yes > } > protocol lda { > auth_verbose = yes > mail_plugins = sieve > } > protocol imap { > mail_max_userip_connections = 20 > } Charles On 5/29/2015 9:55 AM, Charles Marcus wrote: > On 5/29/2015 9:25 AM, Dominik Breu wrote: >> Hello Charles, >> >> the bare minimum ist just the first passdb entry >> auth_master_user_separator = * >> passdb { >> driver = passwd-file >> args = /etc/dovecot/passwd.masterusers >> master = yes >> pass = yes >> } >> >> this will do the trick. > Ok, this isn't working, I'm getting auth failed when testing either with > telnet or doing it in Thunderbird. > > I tried both using the htpasswd method for generating the password in > the passwd file, and when that failed, I also tried just putting the > password in the file in plain text... > > So... what am I missing/doing wrong? > > Thanks, > > Charles > From cross+dovecot at distal.com Fri May 29 14:18:16 2015 From: cross+dovecot at distal.com (Chris Ross) Date: Fri, 29 May 2015 10:18:16 -0400 Subject: Shared inbox? In-Reply-To: <35c6455ae08d7cb600ec90426eb802e9@www.thecsillags.com> References: <01A37FE9-E8E9-4A7D-9B28-61700701C8A9@distal.com> <0F07971F-44FB-4F8E-8319-C96664C89633@distal.com> <5eee7a3fefd3995589630f9c14f59f01@www.thecsillags.com> <35c6455ae08d7cb600ec90426eb802e9@www.thecsillags.com> Message-ID: > On May 28, 2015, at 09:08, andy at thecsillags.com wrote: > > Chris, > > I do indeed have an acl_shared_dict set up. That may be the ticket. That makes it so that the IMAP server knows that you have acls on the other mailbox, so it can know to then look in that mailbox to find out precisely what the ACLs are. Okay. I did set up an acl_shared_dict, but I?m not sure the file ownership/permissions aren?t somewhat in the way. It appears dovecot, and doveadm, operate as the user in question, and I think the permission problems may be getting in my way. I eventually made ~childuser/Maildir group readable (to a group I?m in), and then was seeing permission denied issues when I tried: % sudo doveadm acl debug -u cross shared/childuser/INBOX [?] doveadm(cross): Error: open(/home/childuser/Maildir/dovecot-acl-list) failed: Permission denied So, I made that file readable to a group I?m in, and that went away, but cal debug still shows no access. I think UNIX permissions are what is impeding my progress at the moment. Is that normal? Any idea how you?re avoiding this problem, Andy? > On May 28, 2015, at 23:57, G H wrote: > > Look in to dovecot's master user feature as well. Okay. This looks like it may help with my above problem. I haven?t looked into it yet, but will check on that as it sounds valuable. Thanks! - Chris From dominik at dominikbreu.de Fri May 29 14:58:13 2015 From: dominik at dominikbreu.de (Dominik Breu) Date: Fri, 29 May 2015 16:58:13 +0200 Subject: Enabling Master User for migration In-Reply-To: <55687439.6050404@Media-Brokers.com> References: <556863BE.90402@Media-Brokers.com> <1432905935.4021.6.camel@dominikbreu.de> <55686FD4.2070008@Media-Brokers.com> <55687439.6050404@Media-Brokers.com> Message-ID: <1432911493.4021.9.camel@dominikbreu.de> hello, in which way do you login ? loginuser*masteruser or just masteruser? You have to do the login with loginuser*masteruser masterpass greetings dominik Am Freitag, den 29.05.2015, 10:14 -0400 schrieb Charles Marcus: > Apologies - maybe doveconf -n shows a hint at the problem? > > Maybe it has to do with I'm using the default_realm? But I've tried > adding the user as both the local part, and with the fqdn... > > > # doveconf -n > > # 2.2.15: /etc/dovecot/dovecot.conf > > # Pigeonhole version 0.4.6 (3e924b1b6c5c+) > > # OS: Linux 3.10.41-gentoo-r1 x86_64 Gentoo Base System release 2.2 > > auth_default_realm = media-brokers.com > > auth_master_user_separator = * > > auth_mechanisms = plain login > > deliver_log_format = msgid=%m: subject=%s: %$ > > first_valid_uid = 200 > > last_valid_uid = 200 > > listen = * > > login_log_format_elements = user=<%u> method=%m %c rip=%r lport=%{lport} > > mail_attachment_dir = /var/vmail/%d/attachments > > mail_gid = vmail > > mail_location = maildir:~/Maildir > > mail_uid = vmail > > managesieve_notify_capability = mailto > > managesieve_sieve_capability = fileinto reject envelope > > encoded-character vacation subaddress comparator-i;ascii-numeric > > relational regex imap4flags copy include variables body enotify > > environment mailbox date ihave > > namespace inbox { > > inbox = yes > > location = > > mailbox "Deleted Items" { > > special_use = \Trash > > } > > mailbox Drafts { > > special_use = \Drafts > > } > > mailbox Junk { > > special_use = \Junk > > } > > mailbox "Junk E-mail" { > > special_use = \Junk > > } > > mailbox Sent { > > special_use = \Sent > > } > > mailbox "Sent Messages" { > > special_use = \Sent > > } > > mailbox Trash { > > special_use = \Trash > > } > > prefix = > > separator = . > > } > > passdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > } > > passdb { > > args = /etc/dovecot/passwd.masterusers > > driver = passwd-file > > master = yes > > pass = yes > > } > > plugin { > > sieve = ~/.dovecot.sieve > > sieve_dir = ~/sieve > > } > > protocols = imap lmtp > > service auth-worker { > > user = $default_internal_user > > } > > service auth { > > unix_listener /var/spool/postfix/private/auth { > > group = postfix > > mode = 0660 > > user = postfix > > } > > unix_listener auth-userdb { > > group = vmail > > mode = 0600 > > user = vmail > > } > > } > > service imap-login { > > inet_listener imap { > > port = 143 > > } > > inet_listener imaps { > > port = 993 > > ssl = yes > > } > > service_count = 0 > > } > > service imap { > > process_limit = 2048 > > } > > ssl = required > > ssl_cert = > ssl_key = > userdb { > > args = /etc/dovecot/dovecot-sql.conf.ext > > driver = sql > > } > > protocol smtp { > > auth_verbose = yes > > } > > protocol lda { > > auth_verbose = yes > > mail_plugins = sieve > > } > > protocol imap { > > mail_max_userip_connections = 20 > > } > > > > Charles > > > > On 5/29/2015 9:55 AM, Charles Marcus wrote: > > On 5/29/2015 9:25 AM, Dominik Breu wrote: > >> Hello Charles, > >> > >> the bare minimum ist just the first passdb entry > >> auth_master_user_separator = * > >> passdb { > >> driver = passwd-file > >> args = /etc/dovecot/passwd.masterusers > >> master = yes > >> pass = yes > >> } > >> > >> this will do the trick. > > Ok, this isn't working, I'm getting auth failed when testing either with > > telnet or doing it in Thunderbird. > > > > I tried both using the htpasswd method for generating the password in > > the passwd file, and when that failed, I also tried just putting the > > password in the file in plain text... > > > > So... what am I missing/doing wrong? > > > > Thanks, > > > > Charles > > From CMarcus at Media-Brokers.com Fri May 29 15:08:24 2015 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 29 May 2015 11:08:24 -0400 Subject: Enabling Master User for migration In-Reply-To: <1432911493.4021.9.camel@dominikbreu.de> References: <556863BE.90402@Media-Brokers.com> <1432905935.4021.6.camel@dominikbreu.de> <55686FD4.2070008@Media-Brokers.com> <55687439.6050404@Media-Brokers.com> <1432911493.4021.9.camel@dominikbreu.de> Message-ID: <556880E8.8060601@Media-Brokers.com> On 5/29/2015 10:58 AM, Dominik Breu wrote: > hello, > > in which way do you login ? > > loginuser*masteruser or just masteruser? > > You have to do the login with loginuser*masteruser masterpass Thanks dominik, but I figured out what I did wrong... I changed the password using htpasswd again, but neglected to use the -s switch to tell it to use SHA1 encryption method (finally noticed the $apr1$ at the beginning of the encrypted passwd in the file)... Changing the passwd again using the -s switch now it works... :) Thanks again! Charles From b-dovecot.org at grmbl.net Fri May 29 19:25:39 2015 From: b-dovecot.org at grmbl.net (b-dovecot.org at grmbl.net) Date: Fri, 29 May 2015 21:25:39 +0200 Subject: multi sync (>2 servers) + selective sync + trigger Message-ID: <20150529192539.GC14288@mx.grmbl.net> Hi all, I've been researching ways to replicate mail across multiple mailstores and have a few questions. Synching 2 mailstores (M1 & M2) via dsync works fine. I want to add a 3rd and 4th (M3 & M4) server to also be synced. Multi-sync (>2 servers): 1) How do I tell M1 and/or M2 to also sync to M3 and/or M4? Selective sync: 2) How do I sync specific domains to specific servers and not to others? Trigger: 3) Does a normal sync check only check the mailbox being changed or does it get triggered on mailbox changes to check ALL indexes and sync all changed mailboxes accordingly? 4) Can I use an email to trigger a sync for any other mailboxes which might be out of sync? TIA B From user+dovecot at localhost.localdomain.org Fri May 29 20:10:31 2015 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Fri, 29 May 2015 20:10:31 +0000 Subject: OT: Central sieve management In-Reply-To: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> References: <9CA0345D-0393-4780-99A7-95AAA1071E4D@airstreamcomm.net> Message-ID: <5568C7B7.8090103@localhost.localdomain.org> On 05/28/2015 08:24 PM, list at airstreamcomm.net wrote: > A bit off topic, but I was wondering if anyone here has a solution for centrally managing sieve for multiple users from a custom web application? We would like to implement pigeonhole sieve on our dovecot cluster, however we need to be able to access user?s sieve configurations from a central location for troubleshooting and support purposes. > You may store the scripts on a LDAP server or in a sql database using dict. See http://hg.rename-it.nl/dovecot-2.2-pigeonhole/file/0.4.8/doc/locations Regards, Pascal -- The trapper recommends today: c01dcafe.1514922 at localdomain.org From daniel.colchete at gmail.com Sat May 30 12:26:32 2015 From: daniel.colchete at gmail.com (Daniel van Ham Colchete) Date: Sat, 30 May 2015 09:26:32 -0300 Subject: multi sync (>2 servers) + selective sync + trigger In-Reply-To: <20150529192539.GC14288@mx.grmbl.net> References: <20150529192539.GC14288@mx.grmbl.net> Message-ID: B, I really like the idea of N-way replication. Pairs are ugly, they cost double! Even if you have 20 servers, when one goes down all that IO traffic goes to just one. So, what I did here was a (kind of) DHT-based n-way replication, where the node for the second copy is independent of where the first copy is. For that you will have to use the mail_replica inside your userdb. Here I'm using MySQL. The catch is that the mail_replica is now always the same for the same user. The first server need a mail_replica pointing to the second while the second need a mail_replica pointing to the first. In order to do that I have a table with two fields: hostA and hostB. Each dovecot will have to figure out witch one to use. So, my dovecot-sql.conf-ext looks like: user_query = select email, uid, gid, home, concat('tcp:', IF(hostA = '10.0.3.11', hostB, hostA), ':12345') as mail_replica from users where email='%Lu' and (hostA='10.0.3.11' or hostB='10.0.3.11') password_query = SELECT email as user, password FROM users WHERE email = '%Lu' iterate_query = SELECT email AS user FROM users WHERE hostA = '10.0.3.11' or hostB='10.0.3.11' On each host you have to put the right IP on each dovecot-sql.conf.ext. It works perfectly! And it is nice too! Imagine you have a 10 node cluster and the first 2 nodes fail. Instead of having 10% of your users down, only 2.2% (2 x 1/10 x 1/9) of your users will be offline! Also if each node can handle 10k users and if you want to have N+1 redundancy you can have 90k users in the same servers, instead of 50k if you had pairs. So, availability is UP and costs are DOWN! Is there really a choice here? Hehe. Caveats: - Dovecot Proxy don't understand the concept of having 2 (or more) active places for the same account. It seems like you would need an outside monitor that would check the network every x seconds and change the proxy field on the database when a node went down to hostA or hostB. I think this is madness so I had to write my own POP3/IMAP/LMTP Proxy (it's not that hard) to try the primary first and if that socket don't connect in 1 second we go straight to the second without the user noticing it. It is really bad that Dovecot Proxy can't talk replication. - Your availability goes really UP (lots of nines) when you add a third copy while still having lower costs than with pairs, but you can't use Dovecot's notificator/aggregator/replication plugins for that to make everything run smoothly and I don't think that an outside/cron-based dsync is an option. - If you really use DHT-like algorithms, rebalancing your cluster will require mininum data movement and you add or remove nodes. It will require a small change on the SQL queries above so that the new node also recognizes the new user, but it is easy to do. Let me know what you think! Best, Daniel Colchete On Fri, May 29, 2015 at 4:25 PM, wrote: > Hi all, > > > I've been researching ways to replicate mail across multiple mailstores > and have a few questions. > > Synching 2 mailstores (M1 & M2) via dsync works fine. > I want to add a 3rd and 4th (M3 & M4) server to also be synced. > > > Multi-sync (>2 servers): > > 1) How do I tell M1 and/or M2 to also sync to M3 and/or M4? > > Selective sync: > > 2) How do I sync specific domains to specific servers and not to others? > > Trigger: > > 3) Does a normal sync check only check the mailbox being changed or does > it get triggered on mailbox changes to check ALL indexes and sync all > changed mailboxes accordingly? > 4) Can I use an email to trigger a sync for any other mailboxes which > might be out of sync? > > > TIA > > B > From b-dovecot.org at grmbl.net Sun May 31 09:23:30 2015 From: b-dovecot.org at grmbl.net (b-dovecot.org at grmbl.net) Date: Sun, 31 May 2015 11:23:30 +0200 Subject: multi sync (>2 servers) + selective sync + trigger In-Reply-To: References: <20150529192539.GC14288@mx.grmbl.net> Message-ID: <20150531092330.GB29029@mx.grmbl.net> Daniel, On Sat, May 30, 2015 at 09:26:32AM -0300, Daniel van Ham Colchete wrote: > B, > I really like the idea of N-way replication. Pairs are ugly, they cost double! Even if you have 20 servers, when one goes down all that IO traffic goes to just one. > So, what I did here was a (kind of) DHT-based n-way replication, where the node for the second copy is independent of where the first copy is. > For that you will have to use the mail_replica inside your userdb. Here I'm using MySQL. The catch is that the mail_replica is now always the same for the same user. The first server need a mail_replica pointing to the second while the second need a mail_replica pointing to the first. In order to do that I have a table with two fields: hostA and hostB. Each dovecot will have to figure out witch one to use. So, my dovecot-sql.conf-ext ? looks like: > user_query = select email, uid, gid, home, concat('tcp:', IF(hostA = '10.0.3.11', hostB, hostA), ':12345') as mail_replica from users where email='%Lu' and (hostA='10.0.3.11' or hostB='10.0.3.11') > password_query = SELECT email as user, password FROM users WHERE email = '%Lu' > iterate_query = SELECT email AS user FROM users WHERE hostA = '10.0.3.11' or hostB='10.0.3.11' > On each host you have to put the right IP on each dovecot-sql.conf.ext.? > It works perfectly! And it is nice too! Imagine you have a 10 node cluster and the first 2 nodes fail. Instead of having 10% of your users down, only 2.2% (2 x 1/10 x 1/9) of ? your users will be offline! Also if each node can handle 10k users and if you want to have N+1 redundancy you can have 90k users in the same servers, instead of 50k if you had pairs. So, availability is UP and costs are DOWN! Is there really a choice here? Hehe. > Caveats: > - Dovecot Proxy don't understand the concept of having 2 (or more) active places for the same account. It seems like you would need an outside monitor that would check the network every x seconds and change the proxy field on the database when a node went down to hostA or hostB. I think this is madness so I had to write my own POP3/IMAP/LMTP Proxy (it's not that hard) to try the primary first and if that socket don't connect in 1 second we go straight to the second without the user noticing it. It is really bad that Dovecot Proxy can't talk replication. This is very similar to how I would work around it: RR-DNS or service discovery for the other nodes in mail_replica, which would randomly sync data to another node. The big problem with this is that propagation is slow. Therefor my question if any mail triggers all mailboxes with changes to be synced or just the mailbox for the one being delivered which would speed that up. For POP3/IMAP, what do you use? perdition? What for LMTP? > - Your availability goes really UP (lots of nines) when you add a third copy while still having lower costs than with pairs, but you can't use Dovecot's notificator/aggregator/replication plugins for that to make everything run smoothly and I don't think that an outside/cron-based dsync is an option. I guess for full syncs it would but that removes the "real-time" aspect of things. > ? - If you really use DHT-like algorithms, rebalancing your cluster will require mininum data movement and you add or remove nodes. It will require a small change on the SQL queries above so that the new node also recognizes the new user, but it is easy to do.? > Let me know what you think! I'm wonder why Timo didn't expand mail_replica to be a list of servers rather than just accepting one. That would sort out a lot of this already. mail_replica = 'server1,server2, .. serverN' Am I missing something? Cheers! B